Working with JSON in Go: Marshaling and Unmarshaling 2025

Table Of Contents

• Introduction
• JSON Fundamentals in Go
• Advanced JSON Techniques
• Performance Optimization
• FAQ
• Conclusion

Introduction

JSON (JavaScript Object Notation) is the de facto standard for data interchange in modern web applications, APIs, and microservices. Go's built-in encoding/json package provides powerful tools for converting between Go data structures and JSON, but mastering these capabilities requires understanding the nuances of marshaling, unmarshaling, and the various customization options available.

Many developers struggle with JSON handling in Go, especially when dealing with complex data structures, custom types, or performance-critical applications. Common challenges include handling nested objects, working with dynamic JSON structures, customizing field names and formats, and optimizing JSON operations for high-throughput scenarios.

In this comprehensive guide, we'll explore Go's JSON capabilities from basic operations to advanced techniques. You'll learn how to handle complex data structures, implement custom marshaling logic, optimize performance, and solve real-world JSON processing challenges that you'll encounter in production applications.

JSON Fundamentals in Go

Basic Marshaling and Unmarshaling

Let's start with the essential JSON operations in Go:

package main

import (
    "encoding/json"
    "fmt"
    "log"
)

// Basic struct for JSON operations
type User struct {
    ID       int    `json:"id"`
    Name     string `json:"name"`
    Email    string `json:"email"`
    Age      int    `json:"age"`
    IsActive bool   `json:"is_active"`
}

func demonstrateBasicJSON() {
    // Create a user instance
    user := User{
        ID:       1,
        Name:     "John Doe",
        Email:    "john@example.com",
        Age:      30,
        IsActive: true,
    }
    
    // Marshal struct to JSON
    jsonData, err := json.Marshal(user)
    if err != nil {
        log.Fatal("Error marshaling:", err)
    }
    
    fmt.Printf("Marshaled JSON: %s\n", jsonData)
    
    // Marshal with indentation (pretty print)
    prettyJSON, err := json.MarshalIndent(user, "", "  ")
    if err != nil {
        log.Fatal("Error marshaling with indent:", err)
    }
    
    fmt.Printf("Pretty JSON:\n%s\n", prettyJSON)
    
    // Unmarshal JSON back to struct
    jsonString := `{"id":2,"name":"Jane Smith","email":"jane@example.com","age":28,"is_active":false}`
    
    var newUser User
    err = json.Unmarshal([]byte(jsonString), &newUser)
    if err != nil {
        log.Fatal("Error unmarshaling:", err)
    }
    
    fmt.Printf("Unmarshaled user: %+v\n", newUser)
}

func demonstrateBasicTypes() {
    // Working with basic Go types
    
    // Slice to JSON array
    numbers := []int{1, 2, 3, 4, 5}
    numbersJSON, _ := json.Marshal(numbers)
    fmt.Printf("Numbers JSON: %s\n", numbersJSON)
    
    // Map to JSON object
    person := map[string]interface{}{
        "name":    "Alice",
        "age":     25,
        "city":    "New York",
        "hobbies": []string{"reading", "hiking", "coding"},
    }
    personJSON, _ := json.Marshal(person)
    fmt.Printf("Person JSON: %s\n", personJSON)
    
    // Unmarshal JSON array
    arrayJSON := `[10, 20, 30, 40, 50]`
    var values []int
    json.Unmarshal([]byte(arrayJSON), &values)
    fmt.Printf("Unmarshaled array: %v\n", values)
    
    // Unmarshal to map[string]interface{}
    objectJSON := `{"title":"Go Programming","pages":300,"available":true}`
    var book map[string]interface{}
    json.Unmarshal([]byte(objectJSON), &book)
    fmt.Printf("Unmarshaled object: %v\n", book)
}

func main() {
    fmt.Println("Basic JSON operations:")
    demonstrateBasicJSON()
    
    fmt.Println("\nBasic types with JSON:")
    demonstrateBasicTypes()
}

JSON Struct Tags

Understanding and using struct tags for JSON customization:

package main

import (
    "encoding/json"
    "fmt"
    "time"
)

// Struct with various JSON tag options
type Product struct {
    ID          int       `json:"id"`
    Name        string    `json:"name"`
    Description string    `json:"description,omitempty"`
    Price       float64   `json:"price"`
    Currency    string    `json:"currency"`
    InStock     bool      `json:"in_stock"`
    Tags        []string  `json:"tags,omitempty"`
    CreatedAt   time.Time `json:"created_at"`
    UpdatedAt   time.Time `json:"updated_at,omitempty"`
    
    // Ignored field
    InternalID string `json:"-"`
    
    // Field with different external name
    CategoryID int `json:"category_id"`
    
    // String representation of number
    Quantity int `json:"quantity,string"`
}

// Example with embedded structs
type Address struct {
    Street   string `json:"street"`
    City     string `json:"city"`
    Country  string `json:"country"`
    PostCode string `json:"post_code,omitempty"`
}

type Customer struct {
    ID      int     `json:"id"`
    Name    string  `json:"name"`
    Email   string  `json:"email"`
    Address Address `json:"address"` // Embedded struct
    
    // Pointer to struct (can be nil)
    BillingAddress *Address `json:"billing_address,omitempty"`
}

func demonstrateStructTags() {
    product := Product{
        ID:          1,
        Name:        "Laptop",
        Description: "High-performance laptop",
        Price:       999.99,
        Currency:    "USD",
        InStock:     true,
        Tags:        []string{"electronics", "computers"},
        CreatedAt:   time.Now(),
        InternalID:  "INTERNAL-123", // This will be ignored
        CategoryID:  10,
        Quantity:    5,
    }
    
    jsonData, _ := json.MarshalIndent(product, "", "  ")
    fmt.Printf("Product JSON:\n%s\n", jsonData)
    
    // Product with empty optional fields
    emptyProduct := Product{
        ID:       2,
        Name:     "Mouse",
        Price:    29.99,
        Currency: "USD",
        InStock:  false,
        CreatedAt: time.Now(),
        Quantity: 0,
    }
    
    emptyJSON, _ := json.MarshalIndent(emptyProduct, "", "  ")
    fmt.Printf("\nProduct with omitempty:\n%s\n", emptyJSON)
}

func demonstrateEmbeddedStructs() {
    customer := Customer{
        ID:    1,
        Name:  "John Doe",
        Email: "john@example.com",
        Address: Address{
            Street:   "123 Main St",
            City:     "New York",
            Country:  "USA",
            PostCode: "10001",
        },
        BillingAddress: &Address{
            Street:  "456 Oak Ave",
            City:    "Boston",
            Country: "USA",
        },
    }
    
    jsonData, _ := json.MarshalIndent(customer, "", "  ")
    fmt.Printf("Customer JSON:\n%s\n", jsonData)
    
    // Customer without billing address
    simpleCustomer := Customer{
        ID:    2,
        Name:  "Jane Smith",
        Email: "jane@example.com",
        Address: Address{
            Street:  "789 Pine St",
            City:    "Seattle",
            Country: "USA",
        },
        // BillingAddress is nil, so it will be omitted
    }
    
    simpleJSON, _ := json.MarshalIndent(simpleCustomer, "", "  ")
    fmt.Printf("\nSimple customer JSON:\n%s\n", simpleJSON)
}

func demonstrateStringTags() {
    // JSON with string representation of numbers
    jsonWithStringNumbers := `{
        "id": 1,
        "name": "Test Product",
        "price": 99.99,
        "currency": "USD",
        "in_stock": true,
        "created_at": "2023-07-30T10:30:00Z",
        "category_id": 5,
        "quantity": "10"
    }`
    
    var product Product
    err := json.Unmarshal([]byte(jsonWithStringNumbers), &product)
    if err != nil {
        fmt.Printf("Error unmarshaling: %v\n", err)
        return
    }
    
    fmt.Printf("Unmarshaled product with string quantity: %+v\n", product)
}

func main() {
    fmt.Println("Struct tags demonstration:")
    demonstrateStructTags()
    
    fmt.Println("\nEmbedded structs:")
    demonstrateEmbeddedStructs()
    
    fmt.Println("\nString tags:")
    demonstrateStringTags()
}

Advanced JSON Techniques

Custom Marshaling and Unmarshaling

Implementing custom JSON behavior for complex types:

package main

import (
    "encoding/json"
    "fmt"
    "strconv"
    "strings"
    "time"
)

// Custom time format
type CustomTime time.Time

const customTimeFormat = "2006-01-02 15:04:05"

func (ct CustomTime) MarshalJSON() ([]byte, error) {
    t := time.Time(ct)
    if t.IsZero() {
        return []byte("null"), nil
    }
    return []byte(`"` + t.Format(customTimeFormat) + `"`), nil
}

func (ct *CustomTime) UnmarshalJSON(data []byte) error {
    str := strings.Trim(string(data), `"`)
    if str == "null" || str == "" {
        *ct = CustomTime(time.Time{})
        return nil
    }
    
    t, err := time.Parse(customTimeFormat, str)
    if err != nil {
        return err
    }
    
    *ct = CustomTime(t)
    return nil
}

// Currency type with custom formatting
type Currency struct {
    Amount   float64
    Code     string
}

func (c Currency) MarshalJSON() ([]byte, error) {
    formatted := fmt.Sprintf("%.2f %s", c.Amount, c.Code)
    return json.Marshal(formatted)
}

func (c *Currency) UnmarshalJSON(data []byte) error {
    var str string
    if err := json.Unmarshal(data, &str); err != nil {
        return err
    }
    
    parts := strings.Split(str, " ")
    if len(parts) != 2 {
        return fmt.Errorf("invalid currency format: %s", str)
    }
    
    amount, err := strconv.ParseFloat(parts[0], 64)
    if err != nil {
        return err
    }
    
    c.Amount = amount
    c.Code = parts[1]
    return nil
}

// Status enum with custom JSON representation
type OrderStatus int

const (
    StatusPending OrderStatus = iota
    StatusProcessing
    StatusShipped
    StatusDelivered
    StatusCancelled
)

var statusNames = map[OrderStatus]string{
    StatusPending:    "pending",
    StatusProcessing: "processing",
    StatusShipped:    "shipped",
    StatusDelivered:  "delivered",
    StatusCancelled:  "cancelled",
}

var statusValues = map[string]OrderStatus{
    "pending":    StatusPending,
    "processing": StatusProcessing,
    "shipped":    StatusShipped,
    "delivered":  StatusDelivered,
    "cancelled":  StatusCancelled,
}

func (os OrderStatus) MarshalJSON() ([]byte, error) {
    if name, exists := statusNames[os]; exists {
        return json.Marshal(name)
    }
    return json.Marshal("unknown")
}

func (os *OrderStatus) UnmarshalJSON(data []byte) error {
    var str string
    if err := json.Unmarshal(data, &str); err != nil {
        return err
    }
    
    if status, exists := statusValues[strings.ToLower(str)]; exists {
        *os = status
        return nil
    }
    
    return fmt.Errorf("invalid order status: %s", str)
}

// Order struct using custom types
type Order struct {
    ID        int         `json:"id"`
    CustomerID int        `json:"customer_id"`
    Total     Currency    `json:"total"`
    Status    OrderStatus `json:"status"`
    CreatedAt CustomTime  `json:"created_at"`
    UpdatedAt CustomTime  `json:"updated_at,omitempty"`
}

func demonstrateCustomMarshaling() {
    order := Order{
        ID:         1,
        CustomerID: 123,
        Total:      Currency{Amount: 99.99, Code: "USD"},
        Status:     StatusProcessing,
        CreatedAt:  CustomTime(time.Now()),
        UpdatedAt:  CustomTime(time.Now().Add(time.Hour)),
    }
    
    jsonData, err := json.MarshalIndent(order, "", "  ")
    if err != nil {
        fmt.Printf("Error marshaling: %v\n", err)
        return
    }
    
    fmt.Printf("Custom marshaled order:\n%s\n", jsonData)
    
    // Unmarshal back
    var unmarshaledOrder Order
    err = json.Unmarshal(jsonData, &unmarshaledOrder)
    if err != nil {
        fmt.Printf("Error unmarshaling: %v\n", err)
        return
    }
    
    fmt.Printf("Unmarshaled order: %+v\n", unmarshaledOrder)
}

func demonstrateCustomUnmarshaling() {
    // JSON with different format
    orderJSON := `{
        "id": 2,
        "customer_id": 456,
        "total": "149.99 EUR",
        "status": "SHIPPED",
        "created_at": "2023-07-30 14:30:45",
        "updated_at": "2023-07-30 16:15:30"
    }`
    
    var order Order
    err := json.Unmarshal([]byte(orderJSON), &order)
    if err != nil {
        fmt.Printf("Error unmarshaling: %v\n", err)
        return
    }
    
    fmt.Printf("Unmarshaled order from custom format: %+v\n", order)
    fmt.Printf("Total amount: %.2f %s\n", order.Total.Amount, order.Total.Code)
    fmt.Printf("Status: %s\n", statusNames[order.Status])
}

func main() {
    fmt.Println("Custom marshaling:")
    demonstrateCustomMarshaling()
    
    fmt.Println("\nCustom unmarshaling:")
    demonstrateCustomUnmarshaling()
}

Handling Dynamic JSON

Working with JSON structures that vary at runtime:

package main

import (
    "encoding/json"
    "fmt"
)

// Generic response wrapper
type APIResponse struct {
    Success bool        `json:"success"`
    Message string      `json:"message"`
    Data    interface{} `json:"data,omitempty"`
    Error   string      `json:"error,omitempty"`
}

// Using json.RawMessage for delayed parsing
type FlexibleEvent struct {
    Type      string          `json:"type"`
    Timestamp int64           `json:"timestamp"`
    UserID    int             `json:"user_id"`
    Data      json.RawMessage `json:"data"` // Raw JSON, parsed later
}

// Specific event types
type UserLoginEvent struct {
    IP        string `json:"ip"`
    UserAgent string `json:"user_agent"`
    Success   bool   `json:"success"`
}

type OrderCreatedEvent struct {
    OrderID int     `json:"order_id"`
    Total   float64 `json:"total"`
    Items   int     `json:"items"`
}

type ProductViewEvent struct {
    ProductID int    `json:"product_id"`
    Category  string `json:"category"`
    Source    string `json:"source"`
}

func demonstrateDynamicJSON() {
    // Mixed responses with different data types
    responses := []string{
        `{"success": true, "message": "User created", "data": {"id": 123, "name": "John"}}`,
        `{"success": false, "message": "Validation failed", "error": "Email already exists"}`,
        `{"success": true, "message": "Orders retrieved", "data": [{"id": 1, "total": 99.99}, {"id": 2, "total": 149.99}]}`,
    }
    
    for i, respJSON := range responses {
        var response APIResponse
        err := json.Unmarshal([]byte(respJSON), &response)
        if err != nil {
            fmt.Printf("Error unmarshaling response %d: %v\n", i+1, err)
            continue
        }
        
        fmt.Printf("Response %d:\n", i+1)
        fmt.Printf("  Success: %t\n", response.Success)
        fmt.Printf("  Message: %s\n", response.Message)
        
        if response.Data != nil {
            fmt.Printf("  Data type: %T\n", response.Data)
            fmt.Printf("  Data: %v\n", response.Data)
        }
        
        if response.Error != "" {
            fmt.Printf("  Error: %s\n", response.Error)
        }
        fmt.Println()
    }
}

func demonstrateRawMessage() {
    events := []string{
        `{"type": "user_login", "timestamp": 1690712345, "user_id": 123, "data": {"ip": "192.168.1.1", "user_agent": "Mozilla/5.0", "success": true}}`,
        `{"type": "order_created", "timestamp": 1690712400, "user_id": 456, "data": {"order_id": 789, "total": 99.99, "items": 3}}`,
        `{"type": "product_view", "timestamp": 1690712500, "user_id": 789, "data": {"product_id": 101, "category": "electronics", "source": "search"}}`,
    }
    
    for i, eventJSON := range events {
        var event FlexibleEvent
        err := json.Unmarshal([]byte(eventJSON), &event)
        if err != nil {
            fmt.Printf("Error unmarshaling event %d: %v\n", i+1, err)
            continue
        }
        
        fmt.Printf("Event %d - Type: %s, User: %d\n", i+1, event.Type, event.UserID)
        
        // Parse specific event data based on type
        switch event.Type {
        case "user_login":
            var loginData UserLoginEvent
            if err := json.Unmarshal(event.Data, &loginData); err == nil {
                fmt.Printf("  Login from %s, Success: %t\n", loginData.IP, loginData.Success)
            }
            
        case "order_created":
            var orderData OrderCreatedEvent
            if err := json.Unmarshal(event.Data, &orderData); err == nil {
                fmt.Printf("  Order %d created, Total: $%.2f, Items: %d\n", 
                    orderData.OrderID, orderData.Total, orderData.Items)
            }
            
        case "product_view":
            var viewData ProductViewEvent
            if err := json.Unmarshal(event.Data, &viewData); err == nil {
                fmt.Printf("  Product %d viewed, Category: %s, Source: %s\n", 
                    viewData.ProductID, viewData.Category, viewData.Source)
            }
        }
        fmt.Println()
    }
}

// Type assertion helpers for interface{} data
func demonstrateTypeAssertions() {
    jsonData := `{
        "string_value": "hello",
        "number_value": 42,
        "float_value": 3.14,
        "bool_value": true,
        "array_value": [1, 2, 3],
        "object_value": {"key": "value"}
    }`
    
    var data map[string]interface{}
    json.Unmarshal([]byte(jsonData), &data)
    
    fmt.Println("Type assertions on interface{} values:")
    
    for key, value := range data {
        fmt.Printf("%s: ", key)
        
        switch v := value.(type) {
        case string:
            fmt.Printf("string(%s)\n", v)
        case float64: // JSON numbers are always float64
            fmt.Printf("number(%.2f)\n", v)
        case bool:
            fmt.Printf("bool(%t)\n", v)
        case []interface{}:
            fmt.Printf("array with %d elements\n", len(v))
        case map[string]interface{}:
            fmt.Printf("object with %d keys\n", len(v))
        default:
            fmt.Printf("unknown type: %T\n", v)
        }
    }
}

// Helper function to safely extract values
func getStringValue(data map[string]interface{}, key string) (string, bool) {
    if value, exists := data[key]; exists {
        if str, ok := value.(string); ok {
            return str, true
        }
    }
    return "", false
}

func getFloatValue(data map[string]interface{}, key string) (float64, bool) {
    if value, exists := data[key]; exists {
        if num, ok := value.(float64); ok {
            return num, true
        }
    }
    return 0, false
}

func demonstrateSafeExtraction() {
    jsonData := `{"name": "John", "age": 30, "salary": 75000.50, "active": true}`
    
    var data map[string]interface{}
    json.Unmarshal([]byte(jsonData), &data)
    
    fmt.Println("Safe value extraction:")
    
    if name, ok := getStringValue(data, "name"); ok {
        fmt.Printf("Name: %s\n", name)
    }
    
    if age, ok := getFloatValue(data, "age"); ok {
        fmt.Printf("Age: %.0f years\n", age)
    }
    
    if salary, ok := getFloatValue(data, "salary"); ok {
        fmt.Printf("Salary: $%.2f\n", salary)
    }
    
    // Direct type assertion with safety check
    if active, ok := data["active"].(bool); ok {
        fmt.Printf("Active: %t\n", active)
    }
}

func main() {
    fmt.Println("Dynamic JSON handling:")
    demonstrateDynamicJSON()
    
    fmt.Println("Raw message processing:")
    demonstrateRawMessage()
    
    fmt.Println("Type assertions:")
    demonstrateTypeAssertions()
    
    fmt.Println("\nSafe value extraction:")
    demonstrateSafeExtraction()
}

Performance Optimization

Efficient JSON Processing

Techniques for optimizing JSON operations:

package main

import (
    "encoding/json"
    "fmt"
    "strings"
    "time"
)

// Streaming JSON processing for large datasets
func demonstrateStreamingJSON() {
    // Simulate large JSON array
    largeJSON := `[
        {"id": 1, "name": "Item 1", "value": 100},
        {"id": 2, "name": "Item 2", "value": 200},
        {"id": 3, "name": "Item 3", "value": 300}
    ]`
    
    // Method 1: Load everything into memory (inefficient for large data)
    start := time.Now()
    var items []map[string]interface{}
    json.Unmarshal([]byte(largeJSON), &items)
    loadAllTime := time.Since(start)
    
    fmt.Printf("Load all method: %v, Items: %d\n", loadAllTime, len(items))
    
    // Method 2: Streaming decode (efficient for large data)
    start = time.Now()
    decoder := json.NewDecoder(strings.NewReader(largeJSON))
    
    // Read opening bracket
    token, _ := decoder.Token()
    fmt.Printf("Opening token: %v\n", token)
    
    count := 0
    for decoder.More() {
        var item map[string]interface{}
        decoder.Decode(&item)
        count++
        // Process item without keeping it in memory
    }
    
    streamTime := time.Since(start)
    fmt.Printf("Streaming method: %v, Items processed: %d\n", streamTime, count)
}

// Custom JSON encoder for better performance
type FastUser struct {
    ID    int    `json:"id"`
    Name  string `json:"name"`
    Email string `json:"email"`
}

func (u FastUser) MarshalJSON() ([]byte, error) {
    // Pre-allocate buffer with estimated size
    var buf strings.Builder
    buf.Grow(100) // Estimate based on expected JSON size
    
    buf.WriteString(`{"id":`)
    buf.WriteString(fmt.Sprintf("%d", u.ID))
    buf.WriteString(`,"name":"`)
    buf.WriteString(u.Name)
    buf.WriteString(`","email":"`)
    buf.WriteString(u.Email)
    buf.WriteString(`"}`)
    
    return []byte(buf.String()), nil
}

func benchmarkJSONMethods() {
    users := make([]FastUser, 1000)
    for i := 0; i < 1000; i++ {
        users[i] = FastUser{
            ID:    i + 1,
            Name:  fmt.Sprintf("User %d", i+1),
            Email: fmt.Sprintf("user%d@example.com", i+1),
        }
    }
    
    // Standard JSON marshaling
    start := time.Now()
    for i := 0; i < 100; i++ {
        for _, user := range users {
            json.Marshal(user)
        }
    }
    standardTime := time.Since(start)
    
    // Custom marshaling (without it actually - this is just for demo)
    start = time.Now()
    for i := 0; i < 100; i++ {
        for _, user := range users {
            user.MarshalJSON()
        }
    }
    customTime := time.Since(start)
    
    fmt.Printf("Standard JSON: %v\n", standardTime)
    fmt.Printf("Custom JSON: %v\n", customTime)
}

// Pool-based JSON processing
type JSONProcessor struct {
    encoder *json.Encoder
    decoder *json.Decoder
    buffer  *strings.Builder
}

func NewJSONProcessor() *JSONProcessor {
    buffer := &strings.Builder{}
    return &JSONProcessor{
        encoder: json.NewEncoder(buffer),
        buffer:  buffer,
    }
}

func (jp *JSONProcessor) EncodeToString(v interface{}) (string, error) {
    jp.buffer.Reset()
    err := jp.encoder.Encode(v)
    if err != nil {
        return "", err
    }
    
    result := jp.buffer.String()
    // Remove trailing newline added by Encode
    if len(result) > 0 && result[len(result)-1] == '\n' {
        result = result[:len(result)-1]
    }
    
    return result, nil
}

func demonstratePooledProcessing() {
    processor := NewJSONProcessor()
    
    data := map[string]interface{}{
        "id":    123,
        "name":  "Test User",
        "email": "test@example.com",
        "metadata": map[string]string{
            "source": "api",
            "version": "1.0",
        },
    }
    
    // Reuse the same processor for multiple operations
    for i := 0; i < 5; i++ {
        data["id"] = 123 + i
        result, err := processor.EncodeToString(data)
        if err != nil {
            fmt.Printf("Error encoding: %v\n", err)
            continue
        }
        fmt.Printf("Encoded %d: %s\n", i+1, result)
    }
}

// Optimized struct for JSON operations
type OptimizedProduct struct {
    ID          int     `json:"id"`
    Name        string  `json:"name"`
    Price       float64 `json:"price"`
    InStock     bool    `json:"in_stock"`
    
    // Use pointers for optional fields to save memory
    Description *string  `json:"description,omitempty"`
    Tags        []string `json:"tags,omitempty"`
}

func demonstrateMemoryOptimization() {
    // Create products with and without optional fields
    products := []OptimizedProduct{
        {
            ID:      1,
            Name:    "Basic Item",
            Price:   29.99,
            InStock: true,
            // No description or tags
        },
        {
            ID:      2,
            Name:    "Detailed Item",
            Price:   99.99,
            InStock: true,
            Description: stringPtr("High-quality product with many features"),
            Tags:        []string{"premium", "featured"},
        },
    }
    
    for i, product := range products {
        jsonData, _ := json.MarshalIndent(product, "", "  ")
        fmt.Printf("Product %d JSON:\n%s\n\n", i+1, jsonData)
    }
}

func stringPtr(s string) *string {
    return &s
}

// Batch JSON processing
func demonstrateBatchProcessing() {
    // Process multiple items efficiently
    items := []map[string]interface{}{
        {"id": 1, "type": "user", "data": "user data 1"},
        {"id": 2, "type": "order", "data": "order data 2"},
        {"id": 3, "type": "product", "data": "product data 3"},
    }
    
    // Build JSON array manually for better control
    var result strings.Builder
    result.WriteString("[")
    
    for i, item := range items {
        if i > 0 {
            result.WriteString(",")
        }
        
        itemJSON, _ := json.Marshal(item)
        result.Write(itemJSON)
    }
    
    result.WriteString("]")
    
    fmt.Printf("Batch processed JSON:\n%s\n", result.String())
    
    // Alternative: Use json.Marshal on the slice (simpler but less control)
    standardJSON, _ := json.Marshal(items)
    fmt.Printf("Standard batch JSON:\n%s\n", standardJSON)
}

func main() {
    fmt.Println("Streaming JSON processing:")
    demonstrateStreamingJSON()
    
    fmt.Println("\nJSON performance benchmarking:")
    benchmarkJSONMethods()
    
    fmt.Println("\nPooled JSON processing:")
    demonstratePooledProcessing()
    
    fmt.Println("\nMemory optimization:")
    demonstrateMemoryOptimization()
    
    fmt.Println("Batch processing:")
    demonstrateBatchProcessing()
}

Error Handling and Validation

Robust JSON processing with proper error handling:

package main

import (
    "encoding/json"
    "fmt"
    "reflect"
    "strings"
)

// Custom error types for better error handling
type JSONValidationError struct {
    Field   string
    Message string
}

func (e JSONValidationError) Error() string {
    return fmt.Sprintf("validation error on field '%s': %s", e.Field, e.Message)
}

type JSONParseError struct {
    RawJSON string
    Err     error
}

func (e JSONParseError) Error() string {
    return fmt.Sprintf("JSON parse error: %v", e.Err)
}

// Validatable interface for structs that can validate themselves
type Validatable interface {
    Validate() error
}

// User struct with validation
type ValidatedUser struct {
    ID       int    `json:"id"`
    Name     string `json:"name"`
    Email    string `json:"email"`
    Age      int    `json:"age"`
    Password string `json:"password,omitempty"`
}

func (u ValidatedUser) Validate() error {
    if u.ID <= 0 {
        return JSONValidationError{Field: "id", Message: "must be positive"}
    }
    
    if strings.TrimSpace(u.Name) == "" {
        return JSONValidationError{Field: "name", Message: "cannot be empty"}
    }
    
    if !strings.Contains(u.Email, "@") {
        return JSONValidationError{Field: "email", Message: "invalid format"}
    }
    
    if u.Age < 0 || u.Age > 150 {
        return JSONValidationError{Field: "age", Message: "must be between 0 and 150"}
    }
    
    return nil
}

// Safe JSON unmarshaling with validation
func SafeUnmarshalAndValidate(data []byte, v interface{}) error {
    // First, try to unmarshal
    if err := json.Unmarshal(data, v); err != nil {
        return JSONParseError{RawJSON: string(data), Err: err}
    }
    
    // Then validate if the type supports it
    if validator, ok := v.(Validatable); ok {
        if err := validator.Validate(); err != nil {
            return err
        }
    }
    
    return nil
}

func demonstrateErrorHandling() {
    testCases := []string{
        `{"id": 1, "name": "John Doe", "email": "john@example.com", "age": 30}`, // Valid
        `{"id": -1, "name": "Jane", "email": "jane@example.com", "age": 25}`,    // Invalid ID
        `{"id": 2, "name": "", "email": "empty@example.com", "age": 28}`,        // Empty name
        `{"id": 3, "name": "Bob", "email": "invalid-email", "age": 35}`,         // Invalid email
        `{"id": 4, "name": "Alice", "email": "alice@example.com", "age": 200}`,  // Invalid age
        `{"id": "not-a-number", "name": "Error", "email": "error@example.com"}`, // Parse error
        `invalid json`,                                                           // Invalid JSON
    }
    
    for i, testJSON := range testCases {
        var user ValidatedUser
        err := SafeUnmarshalAndValidate([]byte(testJSON), &user)
        
        fmt.Printf("Test case %d:\n", i+1)
        fmt.Printf("  JSON: %s\n", testJSON)
        
        if err != nil {
            switch e := err.(type) {
            case JSONValidationError:
                fmt.Printf("  ✗ Validation Error: %s\n", e.Error())
            case JSONParseError:
                fmt.Printf("  ✗ Parse Error: %s\n", e.Error())
            default:
                fmt.Printf("  ✗ Unknown Error: %s\n", e.Error())
            }
        } else {
            fmt.Printf("  ✓ Valid: %+v\n", user)
        }
        fmt.Println()
    }
}

// Partial JSON validation
func demonstratePartialValidation() {
    // Sometimes we only want to validate specific fields
    partialJSON := `{
        "id": 123,
        "name": "Partial User",
        "extra_field": "this will be ignored"
    }`
    
    // Define a struct with only the fields we care about
    type PartialUser struct {
        ID   int    `json:"id"`
        Name string `json:"name"`
    }
    
    var partial PartialUser
    err := json.Unmarshal([]byte(partialJSON), &partial)
    if err != nil {
        fmt.Printf("Error unmarshaling partial: %v\n", err)
        return
    }
    
    fmt.Printf("Partial user: %+v\n", partial)
    
    // We can also unmarshal to map first for more control
    var rawData map[string]interface{}
    json.Unmarshal([]byte(partialJSON), &rawData)
    
    fmt.Println("Raw data fields:")
    for key, value := range rawData {
        fmt.Printf("  %s: %v (%T)\n", key, value, value)
    }
}

// Strict JSON parsing (reject unknown fields)
func demonstrateStrictParsing() {
    strictJSON := `{
        "id": 1,
        "name": "Strict User",
        "email": "strict@example.com",
        "unknown_field": "this should cause an error"
    }`
    
    var user ValidatedUser
    decoder := json.NewDecoder(strings.NewReader(strictJSON))
    decoder.DisallowUnknownFields() // This makes parsing strict
    
    err := decoder.Decode(&user)
    if err != nil {
        fmt.Printf("Strict parsing error: %v\n", err)
    } else {
        fmt.Printf("Strict parsing success: %+v\n", user)
    }
    
    // Regular parsing (allows unknown fields)
    err = json.Unmarshal([]byte(strictJSON), &user)
    if err != nil {
        fmt.Printf("Regular parsing error: %v\n", err)
    } else {
        fmt.Printf("Regular parsing success: %+v\n", user)
    }
}

// JSON schema-like validation
func validateJSONStructure(data map[string]interface{}, schema map[string]string) []error {
    var errors []error
    
    // Check required fields
    for field, fieldType := range schema {
        value, exists := data[field]
        if !exists {
            errors = append(errors, fmt.Errorf("missing required field: %s", field))
            continue
        }
        
        // Check type
        actualType := reflect.TypeOf(value).String()
        expectedType := fieldType
        
        // JSON numbers are always float64
        if expectedType == "int" && actualType == "float64" {
            if val, ok := value.(float64); ok && val == float64(int(val)) {
                continue // It's a whole number, acceptable as int
            }
        }
        
        if expectedType == "string" && actualType != "string" {
            errors = append(errors, fmt.Errorf("field %s: expected %s, got %s", field, expectedType, actualType))
        }
    }
    
    return errors
}

func demonstrateSchemaValidation() {
    schema := map[string]string{
        "id":    "int",
        "name":  "string",
        "email": "string",
    }
    
    testJSON := `{"id": 123, "name": "Test User", "email": "test@example.com", "age": 30}`
    
    var data map[string]interface{}
    json.Unmarshal([]byte(testJSON), &data)
    
    errors := validateJSONStructure(data, schema)
    if len(errors) > 0 {
        fmt.Println("Schema validation errors:")
        for _, err := range errors {
            fmt.Printf("  - %s\n", err)
        }
    } else {
        fmt.Println("Schema validation passed")
    }
}

func main() {
    fmt.Println("Error handling and validation:")
    demonstrateErrorHandling()
    
    fmt.Println("Partial validation:")
    demonstratePartialValidation()
    
    fmt.Println("\nStrict vs regular parsing:")
    demonstrateStrictParsing()
    
    fmt.Println("\nSchema validation:")
    demonstrateSchemaValidation()
}

FAQ

Q: When should I use json.RawMessage vs interface{}? A: Use json.RawMessage when you want to delay parsing of JSON data until later, especially when the structure depends on other fields. Use interface{} when you need to handle completely dynamic data structures immediately.

Q: How do I handle JSON with inconsistent field types? A: Implement custom UnmarshalJSON methods that can handle multiple types for the same field, or use interface{} with type assertions to check for different possible types.

Q: What's the performance difference between json.Marshal and custom MarshalJSON? A: Custom MarshalJSON can be faster for simple structures as it avoids reflection, but it requires more maintenance. For complex objects, the standard library is usually sufficient and more robust.

Q: How do I handle very large JSON files efficiently? A: Use json.Decoder with streaming to process JSON incrementally instead of loading everything into memory. This is especially important for JSON arrays with many elements.

Q: Can I marshal private fields in Go structs? A: No, json.Marshal only works with exported (capitalized) fields. You can use custom MarshalJSON methods if you need to include unexported data in JSON output.

Q: How do I handle JSON null values in Go? A: Use pointers for optional fields, or implement custom UnmarshalJSON methods. Pointers will be nil for JSON null values, while zero values are used for missing fields.

Conclusion

Mastering JSON handling in Go is crucial for building modern web applications and APIs. The key insights from this comprehensive guide include:

• Understanding the fundamentals of marshaling and unmarshaling with proper struct tags
• Implementing custom JSON behavior for complex types and business logic
• Handling dynamic JSON structures with json.RawMessage and interface{} types
• Optimizing JSON performance through streaming, pooling, and efficient memory usage
• Implementing robust error handling and validation for production applications
• Using appropriate techniques for different JSON processing scenarios

Go's encoding/json package provides powerful, flexible tools for JSON processing. By combining these with proper error handling, validation, and performance optimization techniques, you can build applications that handle JSON efficiently and reliably at scale.

Ready to improve your JSON handling? Start by reviewing your current JSON processing code for opportunities to add validation, optimize performance, or handle edge cases better. Consider implementing custom marshaling for complex business types and streaming for large datasets. Share your experiences and questions in the comments below!