add new tab `Docs/Schema` right after `Indexes` tab

[Fujio-Turner]

add a new Docs/Schema tab after indexes tab

and a new third upload or paste scheme in the Data & Filters Div.

[Fujio-Turner]

Feature ID: Issue #212
Version: 3.27.0-post
Status: Development (Behind ?dev=true flag)
Last Updated: 2025-11-01

---

Table of Contents

1. Overview
2. How to Enable
3. How to Use
4. Feature Capabilities
5. Architecture
6. Problems and Fixes
7. Code Examples
8. Future Enhancements

---

Overview

The Schema Inference feature analyzes Couchbase INFER query results to provide comprehensive schema analysis and visualization. It parses the JSON schema output from Couchbase INFER queries and presents the data in an interactive hierarchical tree view with detailed analysis at every level (Bucket → Scope → Collection → Schema).

Key Benefits

• Visual Schema Exploration: Navigate schemas through an intuitive folder tree structure
• Type Analysis: Pie charts showing data type distribution at all levels
• Consistency Checking: Automatically flags fields with mixed or inconsistent types
• Mock Data Generation: Creates realistic sample documents from schema definitions
• Multi-Collection Analysis: Supports analyzing multiple collections in one view
• Aggregate Views: View combined analysis across buckets, scopes, and collections

---

How to Enable

The Schema Inference feature is currently behind a feature flag for testing and development.

Enabling the Feature

Add ?dev=true to the URL:

file:///path/to/en/index.html?dev=true

or

http://localhost:8000/en/index.html?dev=true

What Gets Enabled

When ?dev=true is set:

• Schema Inference tab appears in the main tab navigation
• Third upload input box appears at the top (Schema Inference JSON input)
• Console log shows: 🔬 Schema Inference feature enabled (dev mode)

Implementation Code

// Show Schema Inference feature if dev mode is enabled
if (isDevMode()) {
    const schemaTab = document.getElementById('schema-inference-tab');
    const schemaInput = document.getElementById('schema-input-section');
    if (schemaTab) schemaTab.style.display = '';
    if (schemaInput) schemaInput.style.display = '';
    Logger.info('🔬 Schema Inference feature enabled (dev mode)');
}

---

How to Use

Step 1: Run INFER Queries in Couchbase

Run INFER queries in Couchbase Query Workbench:

Without Sample Values (faster):

INFER `travel-sample`._default._default WITH {"num_sample_values":0};

With Sample Values (recommended for full analysis):

INFER `travel-sample`._default._default;

Multiple Collections:

INFER `travel-sample`._default._default;
INFER yama.users.profiles;

Step 2: Upload or Paste Results

Option A: Upload JSON File

1. Click "Upload .json" button in the third input box (Schema Inference JSON)
2. Select your INFER results JSON file
3. Automatically switches to Schema Inference tab and parses data

Option B: Paste JSON

1. Copy INFER results from Couchbase Query Workbench
2. Paste into the third textarea at the top
3. Auto-parses and displays results

Step 3: Explore the Schema Tree

Tree Navigation:

• 📦 Bucket (e.g., travel-sample)
• 📁 Scope (e.g., _default)
• 📄 Collection (e.g., _default)
• 🔷 Schema (e.g., Schema 1, Schema 2 - different document types)

Interaction:

• Arrow (▶/▼): Expand/collapse folders
• Name/Icon Click: View aggregate analysis for that level
• Expand All / Collapse All: Buttons at top of tree

Step 4: View Analysis

Click any level to see:

• Type Distribution Pie Chart: Visual breakdown of data types
• Type Consistency Panel: Warnings for fields with mixed types
• Mock Data from Schema: Sample JSON document (Collection/Schema levels only)
• Property Table: Detailed field-by-field analysis

---

Feature Capabilities

1. Hierarchical Tree View

Left Panel Features:

• 4-Level Hierarchy: Bucket → Scope → Collection → Schema
• Badge Counts: Shows number of children at each level
• Document Counts: Displays total documents per schema
• Expand/Collapse Controls: Individual toggles plus global buttons
• Visual Icons: Emoji icons for each hierarchy level

Tree Structure Example:

📦 travel-sample (1 scope)
  └─ 📁 _default (1 coll)
      └─ 📄 _default (5 schemas)
          ├─ 🔷 Schema 1 (718 docs)
          ├─ 🔷 Schema 2 (160 docs)
          ├─ 🔷 Schema 3 (38 docs)
          ├─ 🔷 Schema 4 (8 docs)
          └─ 🔷 Schema 5 (76 docs)

2. Type Distribution Analysis

Pie Chart Features:

• Shows counts for: String, Number, Boolean, Array, Object, Null, Mixed
• Numbers in Legend: e.g., "String (152)", "Number (89)"
• Numbers on Chart: Uses leader lines for small slices
• Interactive Tooltips: Shows count and percentage
• Color Coded: Each data type has a distinct color

Type Categories:

• String (#36a2eb - Blue)
• Number (#ff6384 - Pink)
• Boolean (#4bc0c0 - Teal)
• Array (#ff9f40 - Orange)
• Object (#9966ff - Purple)
• Null (#c9cbcf - Gray)
• Mixed (#ffcd56 - Yellow) - Fields with inconsistent types

3. Type Consistency Checking

Detects Inconsistent Fields:

• Fields where type is not 100% consistent across all documents
• Example: equipment field that is null in 0.13% of docs, string in 99.86%

Visual Warnings:

• Yellow Warning Box: Shows count of fields with mixed types
• Red Background Rows: Inconsistent fields highlighted in property table
• ⚠️ Warning Icons: Prepended to field names with type issues
• Red Text: Field names and types shown in red (#dc3545)
• Bold Coverage %: Shows exact type distribution

Example Warning:

⚠️ 1 field(s) with mixed types

⚠️ equipment
Types: null | string
Coverage: null: 0.13%, string: 99.86%

4. Special Field Detection

Type Discriminator Fields:
Automatically detects fields that identify document types:

• type, docType, class, _class, _type, kind, category
• Shows unique values found in samples
• Example: type = "route", "landmark", "hotel", "airline", "airport"

Unique Identifier Fields:
Detects fields that likely contain unique IDs:

• id, _id, ID, uuid, guid, key, record, recordId, recordKey
• Verifies uniqueness by checking sample values
• Shows field type (number, string, etc.)

Date/Time Fields:
Automatically detects temporal fields:

By Name Pattern:

• date, time, timestamp, created, updated, modified, expires, born, anniversary

By Value Format:

• ISO-8601: 2025-05-01T04:10:30.876930+00:00
• Date (YYYY-MM-DD): 2025-11-01
• Unix Time (seconds): 1746072630
• Unix Time (milliseconds): 1746072630000
• Month/Year: 2025-11
• Day of Week: Monday, Tue, etc.

Visual Indicators:

• 📅 icon shown in property table
• Blue text for date field indicators
• Format detection shown in analysis

5. Mock Data Generation

Intelligent Sample Document Creation:
When sample values are available in INFER output, generates realistic mock documents:

Sample Generation Logic:

• Strings: Uses first sample value from INFER
• Numbers: Uses first sample number
• Booleans: Uses sample boolean or defaults to true
• Arrays: Uses sample array with items
• Objects: Recursively generates nested objects
• Null: Shows null

Display Features:

• Pretty-printed JSON with 2-space indentation
• Limited Height: Default 200px max-height with scroll
• 📋 Copy JSON Button: One-click copy to clipboard
• ▼ Show Less / ▲ Show More Button: Toggle between collapsed and expanded view
• Syntax Highlighting: Monospace font for readability

Example Generated Document:

{
  "airline": "AA",
  "airlineid": "airline_2009",
  "destinationairport": "ATL",
  "distance": 370.3106930431154,
  "equipment": "753 738 757",
  "id": 13137,
  "schedule": [
    {
      "day": 0,
      "flight": "AZ289",
      "utc": "22:44:00"
    }
  ],
  "sourceairport": "CDG",
  "stops": 0,
  "type": "route"
}

6. Multi-Level Aggregate Analysis

Bucket Level Analysis:

• Aggregates ALL schemas across all scopes and collections in the bucket
• Shows combined type distribution
• Lists all unique properties found across all schemas
• Identifies type inconsistencies across the entire bucket

Scope Level Analysis:

• Aggregates schemas within the scope
• Shows properties common across collections
• Detects fields that appear in multiple collections (relationship indicators)

Collection Level Analysis:

• Aggregates all schema types within the collection
• Useful for collections with polymorphic documents
• Shows which fields are consistent across all document types

Schema Level Analysis:

• Detailed view of a single document type/schema
• Full property listing with doc counts and percentages
• Mock data generation using sample values

7. Property Table Details

Columns:

1. Property: Field name with warning/date icons
2. Type: Data type(s) found
3. Doc Count: Number of documents with this field
4. % Coverage: Percentage of documents with this field
5. Details: Additional info (array items, nested properties, date format)

Visual Highlights:

• Red Background: Rows with inconsistent types
• ⚠️ Icon: Warning for mixed types
• 📅 Icon: Indicates potential date fields
• Bold Red %: Coverage less than 100%

---

Architecture

File Structure

en/index.html
├── HTML Structure
│   ├── Tab Navigation (line 964-973)
│   ├── Top Input Section (line 762-777)
│   └── Tab Content (line 2314-2340)
│
├── CSS Styling (line 671-723)
│   ├── .schema-tree-node
│   ├── .schema-tree-item
│   ├── .schema-tree-children
│   └── .schema-badge
│
└── JavaScript Functions (line 25610-26453)
    ├── parseSchemaInference()
    ├── renderSchemaTree()
    ├── toggleSchemaNode()
    ├── selectBucket/Scope/Collection/Schema()
    ├── displayAggregateAnalysis()
    ├── analyzeSchemaTypes()
    ├── analyzeSampleValues()
    ├── generateSampleDocument()
    ├── renderSchemaTypeChart()
    ├── expandAllSchemaTree()
    ├── collapseAllSchemaTree()
    ├── copySampleDocument()
    └── toggleSampleDocument()

Data Flow

graph TD
    A[User Uploads/Pastes INFER JSON] --> B[parseSchemaInference]
    B --> C[Extract bucket.scope.collection from query]
    C --> D[Build Hierarchical Tree Structure]
    D --> E[Store Schemas in schemaDataStore]
    E --> F[renderSchemaTree]
    F --> G[Auto-select First Bucket]
    G --> H[displayAggregateAnalysis]
    H --> I[analyzeSchemaTypes]
    I --> J[Render Pie Chart + Warnings]
    J --> K[Display Property Table]
    
    L[User Clicks Tree Node] --> M{Which Level?}
    M -->|Bucket| N[selectBucket]
    M -->|Scope| O[selectScope]
    M -->|Collection| P[selectCollection]
    M -->|Schema| Q[selectSchema]
    N --> H
    O --> H
    P --> H
    Q --> R[Display Single Schema + Mock Data]

Loading

Global Variables

let schemaDataStore = {};  // Stores full schema data by path ID
// Format: { "bucket.scope.collection.schema0": { schema, bucket, scope, collection, query, status } }

---

Problems and Fixes

Problem 1: Redundant Input Areas

Issue:
Initially, the Schema Inference tab had its own textarea and Parse button, duplicating the top input functionality.

Fix:

• Removed the redundant input section from the tab content
• Updated parseSchemaInference() to read from top input box (schemaJsonInput)
• Increased output area to use full available space

Code Before:

<div id="schema-inference">
    <div class="input-area">
        <textarea id="schema-infer-input"></textarea>
        <button onclick="parseSchemaInference()">Parse Schema</button>
    </div>
    <div id="schema-inference-output"></div>
</div>

Code After:

<div id="schema-inference">
    <div id="schema-tree-container"></div>
    <div id="schema-detail-panel"></div>
</div>

Function Update:

// Before
const input = document.getElementById('schema-infer-input').value.trim();

// After
const inputTextarea = document.getElementById('schemaJsonInput');
const input = (inputTextarea ? inputTextarea.value.trim() : '') || window._uploadedSchemaJsonRaw || '';

---

Problem 2: Event Propagation in Tree Nodes

Issue:
Clicking on node names would trigger both the toggle (arrow) and the selection handler, causing unexpected behavior.

Fix:
Added event.stopPropagation() to separate toggle and selection actions.

Code Fix:

// Toggle arrow - only expands/collapses
html += `<div class="schema-tree-item" onclick="event.stopPropagation(); toggleSchemaNode('${bucketId}')">`;
html += `<span class="schema-tree-toggle" id="${bucketId}-toggle">▶</span>`;

// Name/Icon - selects and shows analysis
html += `<span onclick="event.stopPropagation(); selectBucket('${bucket}', '${bucketId}')" style="cursor: pointer;">
    ${bucket}
</span>`;

Result:

• Arrow click: Only expands/collapses the node
• Name/Icon click: Shows analysis without toggling expansion

---

Problem 3: Parsing Bucket.Scope.Collection from Queries

Issue:
INFER queries have various formats that needed proper extraction:

• With backticks: INFER travel-sample._default._default
• Without backticks: INFER yama.users.profiles
• With options: INFER bucket.scope.coll WITH {"num_sample_values":0};

Fix:
Created robust regex pattern to handle all variations.

Code:

// Parse bucket.scope.collection from query
// Example: "INFER `travel-sample`._default._default" or "INFER bucket.scope.collection"
const inferMatch = query.match(/INFER\s+`?([^`\s]+)`?\.([^.\s]+)\.([^.\s;]+)/i);
if (!inferMatch) {
    Logger.warn('Could not parse bucket.scope.collection from query:', query);
    return;
}

const bucket = inferMatch[1].replace(/`/g, '');
const scope = inferMatch[2];
const collection = inferMatch[3];

Regex Breakdown:

• INFER\s+ - Match "INFER" followed by whitespace
• `?([^`\s]+)`? - Optional backticks around bucket name
• \.([^.\s]+) - Dot + scope name
• \.([^.\s;]+) - Dot + collection name (stops at semicolon or space)

---

Problem 4: Chart Numbers Not Showing in Pie Chart

Issue:
Initial pie chart implementation didn't show numbers on slices or in the legend.

Fix:
Implemented the same pattern used by Dashboard's state-chart with custom legend labels and the drawPieLabelsWithLeaders plugin.

Code Implementation:

const total = data.reduce((a, b) => a + b, 0);

new Chart(canvas, {
    type: 'pie',
    data: { /* ... */ },
    options: {
        plugins: {
            legend: {
                position: 'bottom',
                labels: {
                    generateLabels: function(chart) {
                        const data = chart.data;
                        return data.labels.map((label, index) => ({
                            text: label + ' (' + data.datasets[0].data[index].toLocaleString() + ')',
                            fillStyle: data.datasets[0].backgroundColor[index],
                            strokeStyle: data.datasets[0].backgroundColor[index],
                            lineWidth: 1,
                            hidden: false,
                            index: index
                        }));
                    }
                }
            }
        }
    },
    plugins: [{
        afterDatasetsDraw: function(chart) {
            drawPieLabelsWithLeaders(chart, total, { insideThreshold: 10 });
        }
    }]
});

Result:

• Legend shows: "String (152)", "Number (89)", etc.
• Chart slices show actual counts with leader lines
• Tooltips show: "String: 152 (60.3%)"

---

Problem 5: Mock Document Generation for Complex Nested Structures

Issue:
INFER results can have deeply nested objects and arrays with their own sample data. Needed to recursively handle these structures.

Fix:
Created recursive generateSampleDocument() function that intelligently handles all data types.

Code:

function generateSampleDocument(properties) {
    const doc = {};
    
    Object.entries(properties).forEach(([propName, propData]) => {
        const samples = propData.samples || [];
        const types = Array.isArray(propData.type) ? propData.type : [propData.type];
        const primaryType = types[0];
        
        const flatSamples = samples.flat().filter(s => s !== null && s !== undefined);
        
        if (flatSamples.length > 0) {
            // Use first sample value
            doc[propName] = flatSamples[0];
        } else if (primaryType === 'object') {
            if (propData.properties) {
                // Recursively generate nested object
                doc[propName] = generateSampleDocument(propData.properties);
            } else {
                doc[propName] = {};
            }
        } else if (primaryType === 'array') {
            if (propData.items && propData.items.samples) {
                const itemSamples = propData.items.samples.flat();
                doc[propName] = itemSamples.length > 0 ? [itemSamples[0]] : [];
            } else {
                doc[propName] = [];
            }
        }
        // ... other types
    });
    
    return doc;
}

Handles:

• Simple types (string, number, boolean, null)
• Nested objects with recursive generation
• Arrays with sample items
• Complex nested arrays of objects

---

Problem 6: Copy Button Not Finding Pre Element

Issue:
After adding Show More/Less button, the Copy button couldn't find the <pre> element using simple nextElementSibling.

Fix:
Updated to use closest() and querySelector() for more reliable element selection.

Code Before:

const pre = button.nextElementSibling;

Code After:

const container = button.closest('div[style*="background"]');
const pre = container ? container.querySelector('pre') : null;

---

Problem 7: Full Height Layout Not Utilizing Space

Issue:
Initial layout used fixed calc(100vh - 300px) which didn't adapt well to different screen sizes.

Fix:
Changed to flex-based layout for better responsiveness.

Code:

<!-- Before -->
<div id="schema-inference" style="padding: 15px;">
    <div style="height: calc(100vh - 300px);">
        <!-- content -->
    </div>
</div>

<!-- After -->
<div id="schema-inference" style="padding: 15px; height: 100%; display: flex; flex-direction: column;">
    <div style="display: flex; gap: 15px; flex: 1; min-height: 0;">
        <!-- content -->
    </div>
</div>

Benefits:

• Adapts to available space
• Prevents overflow issues
• Better use of screen real estate

---

Code Examples

Example 1: Complete INFER Data Upload Flow

// 1. User uploads file
document.getElementById('upload-schema-btn').addEventListener('click', () => {
    document.getElementById('schema-file').click();
});

// 2. File is read and stored
document.getElementById('schema-file').addEventListener('change', function() {
    const file = this.files[0];
    readJsonFile(file, (text) => {
        window._uploadedSchemaJsonRaw = text;
        parseSchemaInference();
        
        // Auto-switch to Schema Inference tab (tab index 6)
        $('#tabs').tabs('option', 'active', 6);
    });
});

// 3. Parse and build tree
function parseSchemaInference() {
    const data = JSON.parse(input);
    const tree = {};
    
    // Build hierarchy
    data.forEach(sequence => {
        const [bucket, scope, collection] = parseQuery(sequence._sequence_query);
        tree[bucket][scope][collection].push(schemas);
    });
    
    renderSchemaTree(tree);
    selectFirstBucket(); // Auto-select
}

Example 2: Tree Node HTML Generation

function renderSchemaTree(tree, container) {
    let html = '';
    
    Object.keys(tree).sort().forEach(bucket => {
        const bucketId = `bucket-${bucket.replace(/[^a-zA-Z0-9]/g, '-')}`;
        
        html += `<div class="schema-tree-node">`;
        
        // Clickable bucket row with toggle
        html += `<div class="schema-tree-item" onclick="event.stopPropagation(); toggleSchemaNode('${bucketId}')">`;
        html += `<span class="schema-tree-toggle" id="${bucketId}-toggle">▶</span>`;
        html += `<span class="schema-tree-icon" onclick="event.stopPropagation(); selectBucket('${bucket}', '${bucketId}')" style="cursor: pointer;">📦</span>`;
        html += `<span onclick="event.stopPropagation(); selectBucket('${bucket}', '${bucketId}')">${bucket}</span>`;
        html += `<span class="schema-badge">${scopeCount} scope(s)</span>`;
        html += `</div>`;
        
        // Collapsible children
        html += `<div class="schema-tree-children collapsed" id="${bucketId}-children">`;
        // ... render scopes, collections, schemas
        html += `</div>`;
        
        html += `</div>`;
    });
    
    container.innerHTML = html;
}

Example 3: Type Analysis with Sample Detection

function analyzeSchemaTypes(properties) {
    const typeDiscriminators = [];
    const uniqueIdentifiers = [];
    const dateTimeFields = [];
    
    Object.entries(properties).forEach(([propName, propData]) => {
        const samples = propData.samples || [];
        const sampleAnalysis = analyzeSampleValues(samples);
        
        // Detect type discriminator
        if (/^(type|docType|class|_class|_type)$/i.test(propName)) {
            const sampleValues = samples.flat().filter(s => s !== null);
            typeDiscriminators.push({
                name: propName,
                values: [...new Set(sampleValues)]
            });
        }
        
        // Detect unique identifiers
        if (/^(id|_id|ID|uuid|guid|key)$/i.test(propName) && sampleAnalysis.isUnique) {
            uniqueIdentifiers.push({ name: propName, type: propData.type });
        }
        
        // Detect date/time fields
        if (sampleAnalysis.dateFormat) {
            dateTimeFields.push({
                name: propName,
                format: sampleAnalysis.dateFormat,
                samples: samples.flat().slice(0, 3)
            });
        }
    });
    
    return { typeDiscriminators, uniqueIdentifiers, dateTimeFields };
}

Example 4: Date Format Detection

function analyzeSampleValues(samples) {
    const flatSamples = samples.flat().filter(s => s !== null && s !== undefined);
    if (flatSamples.length === 0) return { dateFormat: null, isUnique: false };
    
    const sample = flatSamples[0];
    let dateFormat = null;
    
    if (typeof sample === 'string') {
        // ISO-8601: "2025-05-01T04:10:30.876930+00:00"
        if (/^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}/.test(sample)) {
            dateFormat = 'ISO-8601';
        }
        // Date: "2025-11-01"
        else if (/^\d{4}-\d{2}-\d{2}$/.test(sample)) {
            dateFormat = 'Date (YYYY-MM-DD)';
        }
        // Day of week: "Monday", "Tue"
        else if (/^(Mon|Tue|Wed|Thu|Fri|Sat|Sun)/i.test(sample)) {
            dateFormat = 'Day of Week';
        }
    } else if (typeof sample === 'number') {
        // Unix timestamp seconds: 1746072630
        if (sample > 1000000000 && sample < 10000000000) {
            dateFormat = 'Unix Time (seconds)';
        }
        // Unix timestamp milliseconds: 1746072630000
        else if (sample > 1000000000000 && sample < 10000000000000) {
            dateFormat = 'Unix Time (milliseconds)';
        }
    }
    
    // Check uniqueness (for ID detection)
    const isUnique = flatSamples.length === new Set(flatSamples).size;
    
    return { dateFormat, isUnique };
}

---

Problem 8: Large Mock Documents Overwhelming Display

Issue:
Generated mock documents could be very large (especially with nested arrays and objects), making the page difficult to scroll.

Fix:
Implemented collapsible container with Show More/Less toggle.

Code:

// HTML generation
const sampleDocId = `sample-doc-${escapedId}`;
html += '<h4>Mock Data from Schema</h4>';
html += '<div style="position: relative; padding-top: 45px;">';

// Buttons at top-right
html += '<div style="position: absolute; top: 10px; right: 10px; display: flex; gap: 5px;">';
html += '<button onclick="copySampleDocument(this)">📋 Copy JSON</button>';
html += `<button onclick="toggleSampleDocument('${sampleDocId}', this)">▼ Show Less</button>`;
html += '</div>';

// Collapsible JSON container
html += `<div id="${sampleDocId}" style="max-height: 200px; overflow-y: auto; transition: max-height 0.3s ease;">`;
html += '<pre>' + JSON.stringify(sampleDoc, null, 2) + '</pre>';
html += '</div></div>';

// Toggle function
function toggleSampleDocument(divId, button) {
    const div = document.getElementById(divId);
    if (div.style.maxHeight === 'none') {
        div.style.maxHeight = '200px';
        button.textContent = '▼ Show Less';
    } else {
        div.style.maxHeight = 'none';
        button.textContent = '▲ Show More';
    }
}

User Experience:

• Starts collapsed at 200px (showing ~10-15 lines)
• Click "Show Less" → Expands fully
• Click "Show More" → Collapses back
• Smooth 0.3s transition animation

---

Problem 9: Aggregate Analysis for Bucket/Scope/Collection

Issue:
Initially only individual schemas had analysis. Users wanted to see aggregate views at higher levels.

Fix:
Created separate selection functions for each level that aggregate data from all child schemas.

Code:

function selectBucket(bucket, bucketId) {
    // Collect all schemas in this bucket
    const schemas = [];
    Object.keys(schemaDataStore).forEach(schemaId => {
        const data = schemaDataStore[schemaId];
        if (data.bucket === bucket) {
            schemas.push(data);
        }
    });
    
    displayAggregateAnalysis('Bucket', bucket, schemas, bucketId);
}

function displayAggregateAnalysis(level, name, schemas, escapedId) {
    // Aggregate all properties from all schemas
    const aggregatedProperties = {};
    let totalDocs = 0;
    
    schemas.forEach(schemaData => {
        const properties = schemaData.schema.properties || {};
        totalDocs += schemaData.schema['#docs'] || 0;
        
        Object.entries(properties).forEach(([propName, propData]) => {
            if (!aggregatedProperties[propName]) {
                aggregatedProperties[propName] = {
                    types: new Set(),
                    allData: []
                };
            }
            const types = Array.isArray(propData.type) ? propData.type : [propData.type];
            types.forEach(t => aggregatedProperties[propName].types.add(t));
            aggregatedProperties[propName].allData.push(propData);
        });
    });
    
    // Display aggregate analysis
    // ... pie chart, consistency warnings, property table
}

Shows at Each Level:

• Bucket: All schemas across all scopes/collections
• Scope: All schemas within that scope
• Collection: All schemas (document types) in that collection
• Detects relationships (same field appearing in multiple schemas)

---

Code Examples

Example Usage Scenario 1: Analyzing Multiple Collections

Input JSON:

[
  {
    "_sequence_num": 1,
    "_sequence_query": "INFER `travel-sample`._default._default;",
    "_sequence_query_status": "success",
    "_sequence_result": [[{...}]]
  },
  {
    "_sequence_num": 2,
    "_sequence_query": "INFER yama.users.profiles;",
    "_sequence_query_status": "success",
    "_sequence_result": [[{...}]]
  }
]

Result:

📦 travel-sample (1 scope)
    📁 _default (1 coll)
        📄 _default (4 schemas)
            🔷 Schema 1 (718 docs) - route
            🔷 Schema 2 (160 docs) - landmark
            🔷 Schema 3 (38 docs) - hotel
            🔷 Schema 4 (8 docs) - airline

📦 yama (1 scope)
    📁 users (1 coll)
        📄 profiles (1 schema)
            🔷 Schema 1 (4 docs)

Example Usage Scenario 2: Finding Type Inconsistencies

Scenario:
Field equipment in travel-sample has mixed types:

• null in 1 document (0.13%)
• string in 717 documents (99.86%)

Detection:

// analyzeSchemaTypes detects this
const isMixed = types.length > 1; // ['null', 'string']
const docPercents = [0.13, 99.86];

inconsistentFields.push({
    name: 'equipment',
    types: ['null', 'string'],
    coverage: 'null: 0.13%, string: 99.86%'
});

Display:

• Property table row has red background
• Field name shows ⚠️ equipment in red
• Type column shows null | string in bold red
• Coverage shows 0.13%, 99.86% in red
• Warning panel lists the field with details

Example Usage Scenario 3: Detecting Date Fields

Scenario:
Field createdAt contains ISO-8601 timestamps.

Detection:

const samples = ["2025-05-01T04:10:30.876930+00:00"];
const sampleAnalysis = analyzeSampleValues(samples);
// Returns: { dateFormat: 'ISO-8601', isUnique: false }

dateTimeFields.push({
    name: 'createdAt',
    type: 'string',
    format: 'ISO-8601',
    samples: ["2025-05-01T04:10:30.876930+00:00", "2025-05-05T13:35:08.263867+00:00"]
});

Display:

• Shows 📅 icon next to field name
• Details column shows: "📅 Possible date field"
• Blue text indicator for date detection

---

Future Enhancements

1. Cross-Collection Relationship Detection

Currently analyzed but not yet displayed:

• Fields with same name appearing across multiple collections
• Potential foreign key relationships
• Common naming patterns across schemas

Planned Display:

🔗 Potential Relationships Detected:

Common Field: "updatedDate" (datetime)
  ├─ travel-sample._default.hotels (38 docs) - ISO-8601
  ├─ travel-sample._default.landmarks (160 docs) - ISO-8601
  └─ yama.users.profiles (4 docs) - ISO-8601
  
Suggestion: These collections may have related update timestamps

2. Schema Evolution Tracking

• Compare multiple INFER results over time
• Detect schema changes
• Flag new/removed/modified fields

3. Index Recommendations

• Suggest indexes based on detected unique identifiers
• Recommend covering indexes for frequently queried fields
• Detect missing indexes on date/time fields for time-range queries

4. Data Quality Scoring

• Assign quality score based on type consistency
• Penalize high null percentages
• Reward consistent typing and good field naming

5. Export Functionality

• Export schema documentation as Markdown
• Generate DDL/CREATE statements
• Export mock data samples for testing

---

Testing

Test Data Location

Sample INFER data is available at:

• sample/test_infer_schema.json

Test Checklist

• Feature hidden without ?dev=true
• Feature visible with ?dev=true
• Upload .json file works
• Paste JSON works
• Tree renders correctly
• First bucket auto-selected
• Expand/Collapse All buttons work
• Click bucket shows aggregate analysis
• Click scope shows aggregate analysis
• Click collection shows aggregate analysis
• Click schema shows detailed analysis
• Pie chart displays with numbers
• Type inconsistencies flagged in red
• Mock data generates correctly
• Show More/Less toggle works
• Copy JSON button works
• Date fields detected and marked with 📅
• Type discriminators detected

---

Performance Considerations

Optimization Strategies

1. Lazy Rendering: Tree rendered once, analysis on-demand
2. Chart Reuse: Charts destroyed and recreated to prevent memory leaks
3. Sample Limiting: Only first 3-5 sample values used for display
4. DOM Minimal Updates: Build HTML strings, single innerHTML update

Memory Management

// Destroy chart before creating new one
if (window[`schemaChart_${escapedId}`]) {
    window[`schemaChart_${escapedId}`].destroy();
}
window[`schemaChart_${escapedId}`] = new Chart(canvas, config);

---

Related Documentation

• Feature Flag Documentation: FLOW_DIAGRAM_FEATURE_FLAG.md
• Debug Logging: DEBUG_LOGGING_IMPLEMENTATION.md
• Main README: README.md

---

Summary

The Schema Inference feature provides a powerful, visual way to understand Couchbase database schemas through:

• Hierarchical tree navigation (Bucket → Scope → Collection → Schema)
• Type distribution visualization with interactive pie charts
• Automatic inconsistency detection with visual warnings
• Smart field detection (IDs, dates, type discriminators)
• Mock data generation for testing and documentation
• Multi-level aggregate analysis for relationship discovery

Currently in development mode behind ?dev=true flag for testing and refinement before general release.