JSON Parser

Overview

When using AI providers that stream JSON responses, the individual chunks often contain incomplete JSON that cannot be parsed directly. This plugin automatically detects and fixes partial JSON chunks by adding the necessary closing braces, brackets, and quotes to make them valid JSON.

Features

Automatic JSON Completion: Detects partial JSON and adds missing closing characters
Streaming Only: Processes only streaming responses (non-streaming responses are ignored)
Flexible Usage Modes: Supports two usage types for different deployment scenarios
Safe Fallback: Returns original content if JSON cannot be fixed
Memory Leak Prevention: Automatic cleanup of stale accumulated content with configurable intervals
Zero Dependencies: Only depends on Go’s standard library

Usage

Usage Types

The plugin supports two usage types:

AllRequests: Processes all streaming responses automatically
PerRequest: Processes only when explicitly enabled via request context

package main

import (
    "time"
    "vikat.local/core"
    "vikat.local/core/schemas"
    "vikat.local/plugins/jsonparser"
)

func main() {
    // Create the JSON parser plugin for all requests
    jsonPlugin := jsonparser.NewJsonParserPlugin(jsonparser.PluginConfig{
        Usage:           jsonparser.AllRequests,
        CleanupInterval: 2 * time.Minute,  // Cleanup every 2 minutes
        MaxAge:          10 * time.Minute,  // Remove entries older than 10 minutes
    })
    
    // Initialize Vikat with the plugin
    client, err := vikat.Init(context.Background(), schemas.VikatConfig{
        Account: &MyAccount{},
        LLMPlugins: []schemas.LLMPlugin{
            jsonPlugin,
        },
    })
    
    if err != nil {
        panic(err)
    }
    
    // Use the client normally - JSON parsing happens automatically
    // in the PostLLMHook for all streaming responses
}

PerRequest Mode

package main

import (
    "context"
    "time"
    "vikat.local/core"
    "vikat.local/core/schemas"
    "vikat.local/plugins/jsonparser"
)

func main() {
    // Create the JSON parser plugin for per-request control
    jsonPlugin := jsonparser.NewJsonParserPlugin(jsonparser.PluginConfig{
        Usage:           jsonparser.PerRequest,
        CleanupInterval: 2 * time.Minute,  // Cleanup every 2 minutes
        MaxAge:          10 * time.Minute,  // Remove entries older than 10 minutes
    })
    
    // Initialize Vikat with the plugin
    client, err := vikat.Init(context.Background(), schemas.VikatConfig{
        Account: &MyAccount{},
        LLMPlugins: []schemas.LLMPlugin{
            jsonPlugin,
        },
    })
    
    if err != nil {
        panic(err)
    }

    ctx := context.WithValue(context.Background(), jsonparser.EnableStreamingJSONParser, true)
    
    // Enable JSON parsing for specific requests
    stream, vikatErr := client.ChatCompletionStreamRequest(schemas.NewVikatContext(ctx, schemas.NoDeadline), request)
    if vikatErr != nil {
            // handle error
    }
    for chunk := range stream {
        _ = chunk // handle each streaming chunk
    }
}

Configuration

// Custom cleanup configuration
plugin := jsonparser.NewJsonParserPlugin(jsonparser.PluginConfig{
    Usage:           jsonparser.AllRequests,
    CleanupInterval: 2 * time.Minute,  // Cleanup every 2 minutes
    MaxAge:          10 * time.Minute,  // Remove entries older than 10 minutes
})

Default Values

CleanupInterval: 5 minutes (how often to run cleanup)
MaxAge: 30 minutes (how old entries can be before cleanup)
Usage: Must be specified (AllRequests or PerRequest)

Context Key for PerRequest Mode

When using PerRequest mode, the plugin checks for the context key jsonparser.EnableStreamingJSONParser with a boolean value:

true: Enable JSON parsing for this request
false: Disable JSON parsing for this request
Key not present: Disable JSON parsing for this request

Example:

import (
    "context"

    "vikat.local/plugins/jsonparser"
)

// Enable JSON parsing for this request
ctx := context.WithValue(context.Background(), jsonparser.EnableStreamingJSONParser, true)

// Disable JSON parsing for this request
ctx := context.WithValue(context.Background(), jsonparser.EnableStreamingJSONParser, false)

// No context key - JSON parsing disabled (default behavior)
ctx := context.Background()

How It Works

The plugin implements an optimized parsePartialJSON function with the following steps:

Usage Check: Determines if processing should occur based on usage type and context
Validates Input: First tries to parse the string as valid JSON
Character Analysis: If invalid, processes the string character-by-character to track:
- String boundaries (inside/outside quotes)
- Escape sequences
- Opening/closing braces and brackets
Auto-Completion: Adds missing closing characters in the correct order
Validation: Verifies the completed JSON is valid
Fallback: Returns original content if completion fails

Memory Management

The plugin automatically manages memory by:

Accumulating Content: Stores partial JSON chunks with timestamps for each request
Periodic Cleanup: Runs a background goroutine that removes stale entries based on MaxAge
Request Completion: Automatically clears accumulated content when requests complete successfully
Configurable Intervals: Allows customization of cleanup frequency and retention periods

Real-Life Streaming Example

Here’s a practical example showing how the JSON parser plugin fixes broken JSON chunks in streaming responses:

package main

import (
    "context"
    "encoding/json"
    "fmt"
    "time"
    "vikat.local/core"
    "vikat.local/core/schemas"
    "vikat.local/plugins/jsonparser"
)

func main() {
    // Create JSON parser plugin
    jsonPlugin := jsonparser.NewJsonParserPlugin(jsonparser.PluginConfig{
        Usage:           jsonparser.AllRequests,
        CleanupInterval: 2 * time.Minute,
        MaxAge:          10 * time.Minute,
    })
    
    // Initialize Vikat with the plugin
    client, err := vikat.Init(context.Background(), schemas.VikatConfig{
        Account: &MyAccount{},
        LLMPlugins: []schemas.LLMPlugin{jsonPlugin},
    })
    if err != nil {
        panic(err)
    }
    defer client.Shutdown()

    // Request structured JSON response  
    request := &schemas.VikatChatRequest{
        Provider: schemas.OpenAI,
        Model:    "gpt-4o-mini",
        Input: []schemas.ChatMessage{
            {
                Role: schemas.ChatMessageRoleUser,
                Content: schemas.ChatMessageContent{
                    ContentStr: vikat.Ptr("Return user profile as JSON: {\"name\": \"John Doe\", \"email\": \"john@example.com\"}"),
                },
            },
        },
    }

    // Stream the response
    stream, vikatErr := client.ChatCompletionStreamRequest(schemas.NewVikatContext(context.Background(), schemas.NoDeadline), request)
    if vikatErr != nil {
        panic(vikatErr)
    }

    fmt.Println("Streaming JSON response:")
    for chunk := range stream {
        if chunk.VikatChatResponse != nil && len(chunk.VikatChatResponse.Choices) > 0 {
            choice := chunk.VikatChatResponse.Choices[0]
            if choice.ChatStreamResponseChoice != nil && choice.ChatStreamResponseChoice.Delta != nil {
                content := *choice.ChatStreamResponseChoice.Delta.Content
                fmt.Printf("Chunk: %s\n", content)

                // With JSON parser, you can parse each chunk immediately
                var jsonData map[string]interface{}
                if err := json.Unmarshal([]byte(content), &jsonData); err == nil {
                    fmt.Printf("✅ Valid JSON parsed successfully\n")
                } else {
                    fmt.Printf("❌ Invalid JSON: %v\n", err)
                }
            }
        }
    }
}

Without JSON Parser (raw streaming chunks):

Chunk 1: `{`                    ❌ Invalid JSON
Chunk 2: `{"name"`              ❌ Invalid JSON  
Chunk 3: `{"name": "John"`      ❌ Invalid JSON
Chunk 4: `{"name": "John Doe"`  ❌ Invalid JSON

With JSON Parser (processed chunks):

Chunk 1: `{}`                               ✅ Valid JSON
Chunk 2: `{"name": ""}`                     ✅ Valid JSON
Chunk 3: `{"name": "John"}`                 ✅ Valid JSON  
Chunk 4: `{"name": "John Doe"}`             ✅ Valid JSON

Use Cases

Function Calling: Stream tool call arguments as valid JSON throughout the response
Structured Data: Stream complex JSON objects (user profiles, product catalogs) progressively
Real-time Parsing: Enable client-side JSON parsing at each streaming step without waiting for completion
API Integration: Forward streaming JSON to downstream services that expect valid JSON
Live Updates: Update UI components with valid JSON data as it streams in

Example Transformations

Input	Output
`{"name": "John"`	`{"name": "John"}`
`["apple", "banana"`	`["apple", "banana"]`
`{"user": {"name": "John"`	`{"user": {"name": "John"}}`
`{"message": "Hello\nWorld"`	`{"message": "Hello\nWorld"}`
`""` (empty string)	`{}`
`" "` (whitespace only)	`{}`

Testing

Run the test suite:

cd plugins/jsonparser
go test -v

The tests cover:

Plugin interface compliance
Both usage types (AllRequests and PerRequest)
Context-based enabling/disabling
Streaming responses only (non-streaming responses are ignored)
Various JSON completion scenarios
Edge cases and error conditions
Memory cleanup functionality with real and simulated requests
Configuration options and default values

Overview

Quick Start

Release Cadence

Migration Guides

SDK Integrations

Providers & Guides

MCP Gateway

Custom plugins

Open Source Features

Overview

Features

Usage

Usage Types

PerRequest Mode

Configuration

Default Values

Context Key for PerRequest Mode

How It Works

Memory Management

Real-Life Streaming Example

Use Cases

Example Transformations

Testing

​Overview

​Features

​Usage

​Usage Types

​PerRequest Mode

​Configuration

​Default Values

​Context Key for PerRequest Mode

​How It Works

​Memory Management

​Real-Life Streaming Example

​Use Cases

​Example Transformations

​Testing

Overview

Features

Usage

Usage Types

PerRequest Mode

Configuration

Default Values

Context Key for PerRequest Mode

How It Works

Memory Management

Real-Life Streaming Example

Use Cases

Example Transformations

Testing