Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/docker-build-push.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,8 @@ jobs:
type=ref,event=pr
type=sha,prefix=dev-,enable={{is_not_default_branch}}
type=raw,value=latest,enable={{is_default_branch}}
type=raw,value=3.29.3,enable={{is_default_branch}}
type=raw,value=v3.29.3,enable={{is_default_branch}}
type=raw,value=3.29.4,enable={{is_default_branch}}
type=raw,value=v3.29.4,enable={{is_default_branch}}

- name: Build and push Docker image
uses: docker/build-push-action@v6
Expand Down
4 changes: 2 additions & 2 deletions AGENT.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Couchbase Slow Query Analysis Tool v3.29.3
# Couchbase Slow Query Analysis Tool v3.29.4

## Setup and Installation

Expand Down Expand Up @@ -33,7 +33,7 @@ npm test
See [PLAYWRIGHT_TESTING.md](./PLAYWRIGHT_TESTING.md) and [settings/TESTING_WORKFLOW.md](./settings/TESTING_WORKFLOW.md) for details.

## Version Management
- **Current Version**: 3.29.3 (Last Updated: 2025-12-02)
- **Current Version**: 3.29.4 (Last Updated: 2026-06-30)

### Workflow Order for Updates
When making changes, follow this order:
Expand Down
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
FROM nginx:alpine

# Add version and metadata labels
LABEL version="3.29.3"
LABEL version="3.29.4"
LABEL description="Couchbase Slow Query Analysis Tool"
LABEL maintainer="Fujio Turner"

Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Couchbase Slow Query Analysis Tool v3.29.3
# Couchbase Slow Query Analysis Tool v3.29.4



Expand Down
153 changes: 129 additions & 24 deletions en/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@
<html lang="en">
<!--
Couchbase Query Analyzer
Version: 3.29.3
Last Updated: 2025-12-02
Version: 3.29.4
Last Updated: 2026-06-30

🤖 AI AGENT NOTE: When updating versions, follow the detailed guide in settings/VERSION_UPDATE_GUIDE.md
This guide covers all HTML elements, JavaScript variables, and documentation files that need updating.
Expand All @@ -18,9 +18,9 @@

<head>
<meta charset="UTF-8" />
<meta name="version" content="3.29.3" />
<meta name="last-updated" content="2025-11-15" />
<title>Query Analyzer v3.29.3</title>
<meta name="version" content="3.29.4" />
<meta name="last-updated" content="2026-06-30" />
<title>Query Analyzer v3.29.4</title>
<!-- Include jQuery UI CSS -->
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/jqueryui/1.14.1/themes/base/jquery-ui.min.css" integrity="sha512-TFee0335YRJoyiqz8hA8KV3P0tXa5CpRBSoM0Wnkn7JoJx1kaq1yXL/rb8YFpWXkMOjRcv5txv+C6UluttluCQ==" crossorigin="anonymous" referrerpolicy="no-referrer" />

Expand Down Expand Up @@ -617,7 +617,7 @@
<body>
<!-- Version info -->
<div class="version-info" title="Couchbase Query Analyzer Version">
v3.29.3
v3.29.4
</div>

<!-- Bug report link (bottom-right) -->
Expand Down Expand Up @@ -6183,9 +6183,10 @@ <h4>${operatorName}</h4>

// Group requests by normalized_statement
const groups = {};
let _grpSkippedNoStmt = 0;
requests.forEach((request) => {
const stmt = request.statement || request.preparedText;
if (!stmt) return;
if (!stmt) { _grpSkippedNoStmt++; return; }
// Note: System query filtering is now handled globally in parseJSON()
const normalized = normalizeStatement(stmt);
if (!groups[normalized]) {
Expand All @@ -6194,6 +6195,26 @@ <h4>${operatorName}</h4>
groups[normalized].push(request);
});

if (isDebugMode()) {
const groupKeys = Object.keys(groups);
Logger.debug('[Query Groups] Grouping diagnostics:', {
inputRequests: requests.length,
skippedNoStatement: _grpSkippedNoStmt,
uniqueNormalizedGroups: groupKeys.length
});
// Show the largest groups so we can spot over-normalization collapsing distinct queries
const topGroups = groupKeys
.map(k => ({ count: groups[k].length, normalized: k.slice(0, 200) }))
.sort((a, b) => b.count - a.count)
.slice(0, 5);
Logger.debug('[Query Groups] Top normalized groups (count + normalized statement preview):', topGroups);
if (groupKeys.length <= 1 && requests.length > 1) {
Logger.warn('[Query Groups] All requests collapsed into a single normalized group — check raw statement variety below:');
Logger.debug('[Query Groups] First 5 raw statements:',
requests.slice(0, 5).map(r => String(r.statement || r.preparedText || '').slice(0, 200)));
}
}

// Calculate stats for each group
const groupData = Object.keys(groups)
.map((key) => calculateGroupStats(key, groups[key]))
Expand Down Expand Up @@ -16053,7 +16074,7 @@ <h4>${operatorName}</h4>
const data = JSON.parse(jsonInput);
if (Array.isArray(data)) {
requests = data.map((item) => ({
...item.completed_requests,
...unwrapCompletedRequest(item),
plan: item.plan
? typeof item.plan === "string"
? JSON.parse(item.plan)
Expand Down Expand Up @@ -20358,8 +20379,43 @@ <h4>${operatorName}</h4>
}

// Centralized request data processor with single-pass optimization
// Unwrap a single completed_requests row regardless of the SELECT * wrapper key.
// `SELECT *, meta().plan FROM system:completed_requests` wraps the row under
// `completed_requests`, but a keyspace alias (e.g. `... FROM system:completed_requests request`)
// wraps it under that alias instead (commonly `request`). This handles both, plus any
// single-alias wrapper, and falls back to the item itself when already unwrapped.
function unwrapCompletedRequest(item) {
if (!item || typeof item !== "object") return item;

// Known wrapper keys first (fast path)
if (item.completed_requests && typeof item.completed_requests === "object") {
return item.completed_requests;
}
if (item.request && typeof item.request === "object") {
return item.request;
}

// Generic fallback: a single non-plan wrapper whose value looks like a
// completed_requests row (has statement/requestId/elapsedTime).
const keys = Object.keys(item).filter((k) => k !== "plan");
if (keys.length === 1) {
const inner = item[keys[0]];
if (
inner &&
typeof inner === "object" &&
(inner.statement !== undefined ||
inner.requestId !== undefined ||
inner.elapsedTime !== undefined)
) {
return inner;
}
}

return item;
}

function processRequestData(item) {
const request = item.completed_requests || item;
const request = unwrapCompletedRequest(item);

// Parse and cache plan data immediately
let plan = null;
Expand Down Expand Up @@ -21098,7 +21154,7 @@ <h4>${operatorName}</h4>
for (let i = startIndex; i < endIndex; i++) {
try {
const item = processData[i];
const request = item.completed_requests || item;
const request = unwrapCompletedRequest(item);

// Combined filter check (Step 5 optimization)
if (!shouldProcessRequest(request)) {
Expand Down Expand Up @@ -21543,12 +21599,65 @@ <h4>${operatorName}</h4>

// Collect all unique collections from requests
const collectionsSet = new Set();

// --- Diagnostics (only computed in debug mode) ---
const diag = isDebugMode() ? {
total: 0,
hasStatement: 0,
hasPreparedText: 0,
hasNeither: 0,
emptyAfterTrim: 0,
zeroCollections: 0,
noStatementKeySamples: [],
zeroCollectionSamples: []
} : null;

requests.forEach(request => {
const sql = request.statement || request.preparedText || "";
const collections = extractCollectionsFromSQL(sql);
collections.forEach(collection => collectionsSet.add(collection));

if (diag) {
diag.total++;
if (request.statement) diag.hasStatement++;
if (request.preparedText) diag.hasPreparedText++;
if (!request.statement && !request.preparedText) {
diag.hasNeither++;
if (diag.noStatementKeySamples.length < 3) {
// Print keys inline (stringified) so the console shows them without expansion
diag.noStatementKeySamples.push(JSON.stringify(Object.keys(request).slice(0, 40)));
}
}
if (!String(sql).trim()) diag.emptyAfterTrim++;
if (String(sql).trim() && collections.length === 0) {
diag.zeroCollections++;
if (diag.zeroCollectionSamples.length < 5) {
diag.zeroCollectionSamples.push(String(sql).slice(0, 160));
}
}
}
});

if (diag) {
Logger.debug('[populateCollectionFilter] Statement-field diagnostics:', {
totalRequests: diag.total,
hasStatement: diag.hasStatement,
hasPreparedTextOnly: diag.hasPreparedText,
hasNeitherStatementNorPreparedText: diag.hasNeither,
emptySqlAfterTrim: diag.emptyAfterTrim,
sqlPresentButZeroCollectionsExtracted: diag.zeroCollections,
uniqueCollectionsFound: collectionsSet.size
});
if (diag.noStatementKeySamples.length) {
diag.noStatementKeySamples.forEach((keys, i) => {
Logger.debug(`[populateCollectionFilter] Missing-statement record #${i} top-level keys:`, keys);
});
}
if (diag.zeroCollectionSamples.length) {
Logger.debug('[populateCollectionFilter] Sample SQL where extractCollectionsFromSQL() returned 0 collections (regex may not match FROM clause):', diag.zeroCollectionSamples);
}
}

// Sort collections alphabetically
const sortedCollections = Array.from(collectionsSet).sort();

Expand Down Expand Up @@ -24772,8 +24881,8 @@ <h4>${operatorName}</h4>
// Hook removed - buildIndexQueryFlow will be called directly after data processing

// Version management
const APP_VERSION = "3.29.3";
const LAST_UPDATED = "2025-11-15";
const APP_VERSION = "3.29.4";
const LAST_UPDATED = "2026-06-30";

// Timezone management - initialize early to avoid undefined errors
let detectedTimezone = "UTC"; // Timezone detected from data
Expand All @@ -24790,7 +24899,7 @@ <h4>${operatorName}</h4>
// Get first requestTime that exists
for (let i = 0; i < Math.min(processData.length, 10); i++) {
const item = processData[i];
const request = item.completed_requests || item;
const request = unwrapCompletedRequest(item);
if (request.requestTime) {
const requestTime = request.requestTime;
// Check for timezone offset (e.g., "2025-01-15T10:30:00-05:00" or "2025-01-15T10:30:00+00:00")
Expand Down Expand Up @@ -24920,16 +25029,12 @@ <h4>${operatorName}</h4>
let totalIndexReferences = 0;

requests.forEach((request, index) => {
// Handle different JSON structures
let actualRequest = request;
if (request.completed_requests) {
// User's format: { completed_requests: {...}, plan: "..." }
actualRequest = {
...request.completed_requests,
plan: request.plan,
};
} else {
}
// Handle different JSON structures (raw wrapped rows vs already-processed rows).
// Wrapper key may be `completed_requests` or a keyspace alias like `request`.
const inner = unwrapCompletedRequest(request);
let actualRequest = inner === request
? request
: { ...inner, plan: request.plan };

if (actualRequest.plan) {
}
Expand Down Expand Up @@ -25977,7 +26082,7 @@ <h4>${index.name} ${usedBadge}</h4>
const info = getVersionInfo();
console.log(`
🔍 Couchbase Query Analyzer v${info.version}
📅 Last Updated: 2025-10-20${info.lastUpdated}
📅 Last Updated: ${info.lastUpdated}
🎯 Purpose: Analyze Couchbase N1QL query performance from system:completed_requests

🚀 Features:
Expand Down
10 changes: 5 additions & 5 deletions index.html
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,13 @@
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="version" content="3.29.3" />
<meta name="last-updated" content="2025-11-15" />
<title>Couchbase Slow Query Analyzer & N1QL Query Optimizer v3.29.3 (Execution Plan Analyzer & Index Advisor)</title>
<meta name="version" content="3.29.4" />
<meta name="last-updated" content="2026-06-30" />
<title>Couchbase Slow Query Analyzer & N1QL Query Optimizer v3.29.4 (Execution Plan Analyzer & Index Advisor)</title>
<meta name="description" content="Open-source Couchbase slow query analyzer and N1QL query optimizer. Execution plan analyzer, index advisor, profiler, timeline charts, and tuning guide.">
<meta name="keywords" content="Couchbase query optimizer tool, How to debug Couchbase N1QL queries, Couchbase slow query analyzer, N1QL query performance tuning guide, Couchbase execution plan analyzer, Optimize Couchbase SQL++ queries, Couchbase index advisor tool, Debugging slow Couchbase queries, Couchbase query monitoring dashboard, Tune Couchbase query performance, Couchbase N1QL profiler, Analyze Couchbase completed requests, Couchbase query optimization tips, How to fix slow N1QL queries, Couchbase cost-based optimizer guide, N1QL query execution timeline, Couchbase performance insights tool, Debug Couchbase query errors, Couchbase query tuning best practices, Offline Couchbase query analyzer">
<link rel="canonical" href="https://cb.fuj.io/en/">
<meta property="og:title" content="Couchbase Slow Query Analyzer & N1QL Query Optimizer v3.29.3">
<meta property="og:title" content="Couchbase Slow Query Analyzer & N1QL Query Optimizer v3.29.4">
<meta property="og:description" content="Open-source Couchbase slow query analyzer and N1QL query optimizer. Execution plan analyzer, index advisor, profiler, timeline charts, and tuning guide.">
<meta property="og:url" content="https://cb.fuj.io/en/">
<meta property="og:type" content="website">
Expand Down Expand Up @@ -394,7 +394,7 @@ <h2>FAQs</h2>
</section>
</main>

<div class="version-info">Couchbase Slow Query Analysis Tool v3.29.3</div>
<div class="version-info">Couchbase Slow Query Analysis Tool v3.29.4</div>

<footer>
<p>&copy; 2025 <a href="https://fuj.io" target="_blank" rel="noopener noreferrer">fuj.io</a>.</p>
Expand Down
5 changes: 5 additions & 0 deletions release_notes.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
### Version 3.29.4 (June 30, 2026)
- **Fix: Statement Field Not Parsed When Keyspace Is Aliased** - The analyzer only unwrapped rows wrapped under `completed_requests`. Exports produced by `SELECT *, meta().plan FROM system:completed_requests AS request` (or any keyspace alias) wrap each row under the alias instead, so `statement` was unreachable and nearly all queries collapsed into a single group with empty insights. Added a robust `unwrapCompletedRequest()` helper that unwraps `completed_requests`, `request`, or any single-alias wrapper (closes #245)
- **Fix: Duplicate Date in `QueryAnalyzer.about()`** - Removed a stray hardcoded date that was concatenated before the real "Last Updated" value in the console about output
- **Dev: Statement-Field Diagnostics** - Added `?debug=true`-gated diagnostics in query grouping and collection-filter population to surface statement-field/wrapper issues

### Version 3.29.3 (December 2, 2025)
- **Fix: Parse JSON Visibility on Load** - Keep Parse JSON input section visible on initial page load to avoid confusing new users (closes #242)

Expand Down
49 changes: 49 additions & 0 deletions settings/logs/release_20260630_075354.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# Release Log - Version 3.29.4
**Release Date:** 2026-06-30
**Previous Version:** 3.29.3
**New Version:** 3.29.4
**Version Type:** PATCH - bug fix (statement-field unwrapping for aliased keyspaces)

## Branch & Issue Information
- Branch: issue/245 (based on main @ d5fe26e "release 3.29.3")
- Issues Resolved: #245

## Step 0 - Pre-release E2E (MANDATORY, run first)
- npx playwright test --project=chromium -> 36 passed (Static Edition, en/index.html)

## Version Updates (Static Edition only)
- en/index.html: header comment Version+Last Updated, <meta version>, <meta last-updated>,
<title>, footer version-info, APP_VERSION, LAST_UPDATED -> 3.29.4 / 2026-06-30
- index.html (root): <meta version>, <meta last-updated>, <title>, og:title, version-info -> 3.29.4 / 2026-06-30
- AGENT.md: H1 + Current Version line -> 3.29.4 / 2026-06-30
- README.md: H1 -> 3.29.4
- release_notes.md: new 3.29.4 section at top
- Dockerfile: LABEL version -> 3.29.4
- .github/workflows/docker-build-push.yml: raw tags 3.29.4 / v3.29.4

## Code Changes
- en/index.html: added unwrapCompletedRequest() helper; wired into processRequestData,
processBatch, detectTimezoneFromData, time-grouping fallback parser, extractUsedIndexes.
Root cause: rows wrapped under `request` alias (SELECT *, meta().plan FROM
system:completed_requests AS request) were not unwrapped; only `completed_requests` was.
- en/index.html: removed stray hardcoded date in QueryAnalyzer.about() output.
- en/index.html: added ?debug=true-gated statement-field diagnostics in query grouping
and populateCollectionFilter.

## Verification
- python3 python/validate_js_syntax.py -> PASS (index.html, en/index.html)
- python3 python/RELEASE_WORK_CHECK.py -> ALL CHECKS PASSED (0 issues)
- npx playwright test --project=chromium (post-bump) -> 36 passed

## Files Modified
- en/index.html
- index.html
- AGENT.md
- README.md
- release_notes.md
- Dockerfile
- .github/workflows/docker-build-push.yml

## Notes
- Static Edition (v3.29.x) release only; Server Edition (4.0.0-beta) untouched.
- Normalized inconsistent dates (header said 2025-12-02, meta said 2025-11-15) to 2026-06-30.
Loading