Implement structured JSON logging with Pino for enhanced observability

[Copilot]

Overview

This PR implements structured JSON logging throughout the application using Pino, replacing all console.* calls with a production-ready logging system that supports user/session tracing and log aggregation.

Problem

The application was logging in plaintext without any structured context, making it difficult to:

• Identify which user's request a log message relates to
• Query and filter logs in production environments
• Trace user sessions across multiple requests
• Integrate with log aggregation tools

Before:

console.error('Error creating OpenAI response:', error)
console.warn('Failed to fetch servers:', err)

Solution

Implemented a comprehensive structured logging system using Pino that:

1. Provides structured JSON logging - All logs include structured metadata (service, environment, module, operation, etc.)
2. Enables user/request tracing - API routes include userId, model, and contextual information for tracking user sessions
3. Supports dual environments - Automatically detects and adapts for server (Node.js with Pino) vs browser (structured console logging)
4. Production-ready - JSON output compatible with Elasticsearch, Datadog, CloudWatch, and other log aggregation tools

After:

contextLog.error({
  err: error,
  userId: '[email protected]',
  model: 'gpt-4o',
  operation: 'openai-create-response',
}, 'Error creating OpenAI response')

Key Changes

Logger Implementation (src/lib/logger.ts)

• Created dual-mode logger that detects browser vs server environment
• Server-side uses Pino with JSON output (pretty-printed in development)
• Browser-side uses console with structured JSON format
• Logs disabled in tests by default (configurable via ENABLE_TEST_LOGGING)

API Routes Enhanced with Context

• src/routes/api/chat.ts - Added userId, model, and operation context
• src/routes/api/models.ts - Structured error logging
• src/routes/api/container-file.ts - File operation logging with IDs

Core Libraries Updated

• src/lib/streaming.ts - Stream processing with messageId tracking
• All hooks and components updated with appropriate module loggers

Test Updates

• Updated tests to not depend on console.* assertions
• Maintained existing test coverage with no regressions

Example Output

Development (pretty-printed):

[06:30:45 GMT] INFO (mcp-app-demo/api-chat): Creating OpenAI response
    userId: "[email protected]"
    model: "gpt-4o"
    messageCount: 3
    toolCount: 2
    operation: "chat-request"

Production (JSON for aggregation):

{"level":"info","time":1704789045000,"service":"mcp-app-demo","module":"api-chat","userId":"[email protected]","model":"gpt-4o","messageCount":3,"toolCount":2,"operation":"chat-request","msg":"Creating OpenAI response"}

Benefits

• 🔍 Searchable - Query logs by userId, requestId, operation, or any structured field
• 📊 Analyzable - JSON format integrates with log aggregation and monitoring tools
• 🐛 Debuggable - Rich context in every log entry
• 🎯 Traceable - Follow user sessions and requests across the application
• 🚀 Production-ready - Configurable log levels, async logging, minimal overhead

Configuration

Set log level via environment variable:

LOG_LEVEL=debug  # Options: trace, debug, info, warn, error, fatal

Testing

• ✅ All existing tests pass (same baseline)
• ✅ Linting passes
• ✅ Build successful
• ✅ No regressions introduced

Closes #[issue-number]

Original prompt

---

This section details on the original issue you should resolve

<issue_title>structured logging</issue_title>
<issue_description>Is your feature request related to a problem? Please describe.

currently the application is logging in plaintext without any identification of which user's request the message is related to.

Describe the solution you'd like

1. use structured JSON logging
2. we need a way to identify a user current chat with whatever is happening in the logs

Describe alternatives you've considered

Explain any additional use-cases

If there are any use-cases that would help us understand the use/need/value please share them as they can help us decide on acceptance and prioritization.

Additional context

Add any other context or screenshots about the feature request here.
</issue_description>

Comments on the Issue (you are @copilot in this section)

@nickytonline replace all console.* with structured logging as defined in the PRD

PRD: Structured Logging in TypeScript Applications

Overview

This document provides guidance for implementing structured logging in TypeScript applications using Pino, a high-performance JSON logger optimized for Node.js applications.

Why Structured Logging?

Benefits:

• Searchable: Query logs by any field (userId, requestId, status, etc.)
• Machine-readable: JSON format for log aggregation tools
• Rich context: Include structured metadata with every log entry
• Performance: Async logging with minimal overhead
• Production-ready: Integrates with Elasticsearch, Datadog, CloudWatch, etc.

Use Cases:

• Web APIs and REST services
• Microservices architectures
• Applications requiring observability
• Production systems with centralized logging
• Any project needing log analysis and monitoring

---

Implementation Guide

Step 1: Install Dependencies

npm install pino
npm install --save-dev pino-pretty

Package versions:

• pino: ^8.16.0 or later (production logger)
• pino-pretty: ^10.2.0 (development formatting, optional)

Step 2: Create Logger Configuration

// src/utils/logger.ts
import pino from 'pino';

const isProduction = process.env.NODE_ENV === 'production';
const isTest = process.env.NODE_ENV === 'test';

export const logger = pino({
  // Log level configuration
  level: process.env.LOG_LEVEL || (isProduction ? 'info' : 'debug'),

  // Disable logging during tests unless explicitly enabled
  enabled: !isTest || process.env.ENABLE_TEST_LOGGING === 'true',

  // Pretty printing in development (optional)
  transport: !isProduction ? {
    target: 'pino-pretty',
    options: {
      colorize: true,
      translateTime: 'HH:MM:ss Z',
      ignore: 'pid,hostname'
    }
  } : undefined,

  // Base fields included in every log entry
  base: {
    service: 'your-service-name',
    environment: process.env.NODE_ENV || 'development'
  },

  // Custom formatters
  formatters: {
    level: (label) => {
      return { level: label };
    }
  }
});

// Create child loggers for different modules
export const createLogger = (module: string) => {
  return logger.child({ module });
};

Step 3: Basic Usage

import { createLogger } from './utils/logger.js';

const log = createLogger('user-service');

// Simple logging
log.info('Application started');
log.error('Database connection failed');
log.warn('Configuration missing, using defaults');

// Structured logging with context
log.info({
  userId: 'user-123',
  action: 'login',
  ip: '192.168.1.1'
}, 'User logged in');

// Error logging with error objects
try {
  await performOperation();
} catch (error) {
  log.error({
    err: error,
    operation: 'performOperation',
    userId: 'user-123'
  }, 'Operation failed');
}

Step 4: Log Levels

Use appropriate log levels for different scenarios:

// FATAL (60) - System is unusable, immediate attention required
log.fatal({ err, dbHost }, 'Database connection permanently lost');

// ERROR (50) - Error conditions that need attention
log.error({ err, userId, endpoint }, 'Request processing failed');

// WARN (40) - Warning messages, potentially harmful situations
log.warn({ retryCount: 3, endpoint }, 'Approaching rate limit');

// INFO (30) - Informational messages (default in production)
log.info({ operation: 'cache-refresh', duration: 150 }, 'Cache refreshed');

// DEBUG (20) - Debug-level messages (development/troubleshooting)
log.debug({ cacheSize: 1024, hitRate: 0.85 }, 'Cache statistics');

// TRACE (10) - Very detailed debugging information
log.trace({ headers, body }, 'HTTP request received');

Step 5: Contextual Logging

Create loggers with persistent context for related operations:

// Define context interface
export interface LogContext {
  userId?: string;
  requestId?: string;
  sessionId?: string;
  correlationId?: string;
  operation?: string;
  [key: ...

</details>
Fixes pomerium/mcp-app-demo#145

<!-- START COPILOT CODING AGENT TIPS -->
---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more [Copilot coding agent tips](https://gh.io/copilot-coding-agent-tips) in the docs.

[netlify]

✅ Deploy Preview for mcp-storybook ready!

Name	Link
🔨 Latest commit	1f5eaa6
🔍 Latest deploy log	https://app.netlify.com/projects/mcp-storybook/deploys/68e08c0365a55b0008f1f034
😎 Deploy Preview	https://deploy-preview-164--mcp-storybook.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[Copilot]

Pull Request Overview

This PR implements a comprehensive structured JSON logging system using Pino, replacing all console.* calls with production-ready logging that supports user/session tracing and log aggregation.

• Replaces plaintext console logging with structured JSON logging for enhanced observability
• Enables user and request tracing with contextual information (userId, model, operation)
• Provides dual-mode logger supporting both server-side (Pino) and browser environments

Reviewed Changes

Copilot reviewed 13 out of 14 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
src/lib/logger.ts	New dual-mode logger implementation with Pino for server-side and structured console for browser
src/routes/api/chat.ts	Enhanced API route with contextual logging including userId, model, and operation tracking
src/routes/api/models.ts	Updated error logging with structured context for model fetching operations
src/routes/api/container-file.ts	Added structured logging for file operations with fileId and containerId context
src/lib/streaming.ts	Updated streaming functionality to use structured logging with messageId tracking
src/hooks/useStreamingChat.ts	Replaced console calls with structured logging throughout chat functionality
src/hooks/useLocalStorage.ts	Added structured error logging for localStorage operations
src/hooks/useDisconnectServer.ts	Updated server disconnection error logging
src/components/ServerSelector.tsx	Enhanced server management logging with structured context
src/components/CodeInterpreterMessage.tsx	Added structured logging for image loading errors
package.json	Added Pino dependencies for production logging
src/lib/streaming.test.ts	Updated test to mock logger instead of console
src/hooks/useDisconnectServer.tsx.test.tsx	Removed console error assertions to match new logging approach