Skip to content

Add HTTP Streamable Transport support #5

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
MAVRICK-1 wants to merge 1 commit into from
Closed

Add HTTP Streamable Transport support #5

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 55 additions & 4 deletions CLAUDE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,16 +14,38 @@ This is a Model Context Protocol (MCP) server implementation for Supadata web sc
- `npm run lint` - Run ESLint on TypeScript files
- `npm run lint:fix` - Run ESLint with auto-fix
- `npm run format` - Format code with Prettier
- `npm run start` - Start the compiled server
- `npm run start` - Start the server (defaults to STDIO transport)
- `npm run start:stdio` - Start with STDIO transport (legacy mode)
- `npm run start:http` - Start with HTTP Streamable transport
- `npm run dev:http` - Start HTTP server on port 3000 for development

### Single Test Execution
- `npm run test -- --testNamePattern="should handle scrape request"` - Run specific test
- `npm run test -- src/index.test.ts` - Run specific test file

## Architecture

### Transport Support
The server supports both transport modes as defined in the MCP specification:

#### STDIO Transport (Legacy Mode)
- **Protocol Version**: 2024-11-05 and earlier
- **Usage**: Suitable for local integrations and command-line tools
- **Start Command**: `npm run start:stdio` or `MCP_TRANSPORT_MODE=stdio npm start`
- **Communication**: JSON-RPC over standard input/output streams

#### HTTP Streamable Transport (Current Standard)
- **Protocol Version**: 2025-03-26
- **Usage**: Suitable for web-based integrations and remote server deployments
- **Start Command**: `npm run start:http` or `MCP_TRANSPORT_MODE=http npm start`
- **Endpoints**:
- `POST/GET/DELETE /mcp` - Main MCP endpoint
- `GET /health` - Health check endpoint
- **Features**: Session management, resumability support, CORS enabled
- **Port**: Defaults to 3000 (configurable via `PORT` environment variable)

### MCP Server Structure
The server is built using the `@modelcontextprotocol/sdk` and runs on stdio transport. The main server logic is in `src/index.ts` with the following key components:
The server is built using the `@modelcontextprotocol/sdk` and supports both STDIO and HTTP Streamable transports. The main server logic is in `src/index.ts` with the following key components:

- **Server Creation**: `createServer()` function creates an McpServer instance
- **Tool Registration**: Six tools are registered with input validation using Zod schemas
Expand Down Expand Up @@ -81,6 +103,10 @@ The server integrates with Supadata's JavaScript SDK (`@supadata/js`) and provid
### Required Environment Variables
- `SUPADATA_API_KEY` - Supadata API key for authentication

### Transport Configuration
- `MCP_TRANSPORT_MODE` - Transport mode: "stdio" (default) or "http"
- `PORT` - HTTP server port (default: 3000, only used in HTTP mode)

### Optional Environment Variables
- `SUPADATA_RETRY_MAX_ATTEMPTS` - Max retry attempts (default: 3)
- `SUPADATA_RETRY_INITIAL_DELAY` - Initial retry delay in ms (default: 1000)
Expand Down Expand Up @@ -112,12 +138,37 @@ Key test files:

## Deployment

The server can be deployed via:
### STDIO Transport (Legacy)
- **NPX**: `npx -y @supadata/mcp`
- **Global Install**: `npm install -g @supadata/mcp`
- **Docker**: Using provided Dockerfile
- **MCP Integrations**: Cursor, VS Code, Claude Desktop, Windsurf

### HTTP Streamable Transport (Recommended)
- **Local Development**: `MCP_TRANSPORT_MODE=http npm start`
- **Production**: Deploy as HTTP service on port 3000 (or custom port)
- **Docker**: Using provided Dockerfile with HTTP transport mode
- **Cloud Deployment**: Suitable for AWS, Google Cloud, Azure, Heroku, etc.
- **Load Balancing**: Supports multiple instances with session management

### Environment Setup Examples

```bash
# STDIO mode (legacy)
export SUPADATA_API_KEY=your_api_key_here
npm run start:stdio

# HTTP mode (recommended)
export SUPADATA_API_KEY=your_api_key_here
export PORT=3000
npm run start:http

# Production HTTP deployment
export MCP_TRANSPORT_MODE=http
export PORT=8080
export SUPADATA_API_KEY=your_api_key_here
npm start
```

## Development Notes

- Uses TypeScript with strict mode enabled
Expand Down
15 changes: 13 additions & 2 deletions package-lock.json
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 4 additions & 0 deletions package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,9 @@
"build": "tsc && node -e \"require('fs').chmodSync('dist/index.js', '755')\"",
"test": "node --experimental-vm-modules node_modules/jest/bin/jest.js",
"start": "node dist/index.js",
"start:stdio": "MCP_TRANSPORT_MODE=stdio node dist/index.js",
"start:http": "MCP_TRANSPORT_MODE=http node dist/index.js",
"dev:http": "MCP_TRANSPORT_MODE=http PORT=3000 node dist/index.js",
"lint": "eslint src/**/*.ts",
"lint:fix": "eslint src/**/*.ts --fix",
"format": "prettier --write .",
Expand All @@ -35,6 +38,7 @@
},
"devDependencies": {
"@jest/globals": "^29.7.0",
"@types/cors": "^2.8.17",
"@types/express": "^5.0.1",
"@types/jest": "^29.5.14",
"@types/node": "^20.10.5",
Expand Down
166 changes: 166 additions & 0 deletions src/httpServer.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,166 @@
#!/usr/bin/env node

import express from 'express';
import cors from 'cors';
import { randomUUID } from 'node:crypto';
import { StreamableHTTPServerTransport } from '@modelcontextprotocol/sdk/server/streamableHttp.js';
import { isInitializeRequest } from '@modelcontextprotocol/sdk/types.js';
import createServer from './index.js';

/**
* HTTP server implementation using Streamable HTTP transport
* Based on MCP specification version 2025-03-26
*/

const app = express();
app.use(express.json());

// Configure CORS to expose Mcp-Session-Id header for browser-based clients
app.use(cors({
origin: '*', // Allow all origins - adjust as needed for production
exposedHeaders: ['Mcp-Session-Id']
}));

// Store transports by session ID for session management
const transports: Record<string, StreamableHTTPServerTransport> = {};

// Handle all MCP Streamable HTTP requests (GET, POST, DELETE) on a single endpoint
app.all('/mcp', async (req, res) => {
console.error(`Received ${req.method} request to /mcp`);

try {
// Check for existing session ID
const sessionId = req.headers['mcp-session-id'] as string;
let transport: StreamableHTTPServerTransport;

if (sessionId && transports[sessionId]) {
// Reuse existing transport
transport = transports[sessionId];
} else if (!sessionId && req.method === 'POST' && isInitializeRequest(req.body)) {
// Create new transport for initialization request
transport = new StreamableHTTPServerTransport({
sessionIdGenerator: () => randomUUID(),
onsessioninitialized: (sessionId: string) => {
console.error(`StreamableHTTP session initialized with ID: ${sessionId}`);
transports[sessionId] = transport;
}
});

// Set up onclose handler to clean up transport when closed
transport.onclose = () => {
const sid = transport.sessionId;
if (sid && transports[sid]) {
console.error(`Transport closed for session ${sid}, removing from transports map`);
delete transports[sid];
}
};

// Connect the transport to the MCP server
const server = createServer();
await server.connect(transport);
} else {
// Invalid request - no session ID or not initialization request
res.status(400).json({
jsonrpc: '2.0',
error: {
code: -32000,
message: 'Bad Request: No valid session ID provided or not an initialization request',
},
id: null,
});
return;
}

// Handle the request with the transport
await transport.handleRequest(req, res, req.body);
} catch (error) {
console.error('Error handling MCP request:', error);
if (!res.headersSent) {
res.status(500).json({
jsonrpc: '2.0',
error: {
code: -32603,
message: 'Internal server error',
},
id: null,
});
}
}
});

// Health check endpoint
app.get('/health', (_req, res) => {
res.json({ status: 'ok', transport: 'streamable-http' });
});

// Start the HTTP server
export async function runHttpServer(port: number = 3000) {
return new Promise<void>((resolve, reject) => {
try {
console.error('Initializing Supadata MCP Server with HTTP transport...');

const server = app.listen(port, () => {
console.error(`Supadata MCP Server running on HTTP transport at port ${port}`);
console.error('MCP endpoint: POST/GET/DELETE /mcp');
console.error('Health check: GET /health');
resolve();
});

server.on('error', reject);

// Handle server shutdown
process.on('SIGINT', async () => {
console.error('Shutting down HTTP server...');

// Close all active transports
for (const sessionId in transports) {
try {
console.error(`Closing transport for session ${sessionId}`);
await transports[sessionId].close();
delete transports[sessionId];
} catch (error) {
console.error(`Error closing transport for session ${sessionId}:`, error);
}
}

server.close(() => {
console.error('HTTP server shutdown complete');
process.exit(0);
});
});

process.on('SIGTERM', async () => {
console.error('Shutting down HTTP server...');

// Close all active transports
for (const sessionId in transports) {
try {
console.error(`Closing transport for session ${sessionId}`);
await transports[sessionId].close();
delete transports[sessionId];
} catch (error) {
console.error(`Error closing transport for session ${sessionId}:`, error);
}
}

server.close(() => {
console.error('HTTP server shutdown complete');
process.exit(0);
});
});

} catch (error) {
console.error('Fatal error starting HTTP server:', error);
reject(error);
}
});
}

// Only run the server if this file is executed directly
if (import.meta.url === `file://${process.argv[1]}`) {
const port = process.env.PORT ? parseInt(process.env.PORT) : 3000;
runHttpServer(port).catch((error: any) => {
console.error('Fatal error running HTTP server:', error);
process.exit(1);
});
}
38 changes: 29 additions & 9 deletions src/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,7 @@ async function withRetry<T>(
}
}

export default function createServer() {
export function createServer() {
const server = new McpServer({
name: '@supadata/mcp',
version: '1.0.0',
Expand Down Expand Up @@ -525,10 +525,10 @@ export default function createServer() {
return server.server;
}

// Server startup
async function runServer() {
// Server startup for STDIO transport
async function runStdioServer() {
try {
console.error('Initializing Supadata MCP Server...');
console.error('Initializing Supadata MCP Server with STDIO transport...');

const server = createServer();
const transport = new StdioServerTransport();
Expand All @@ -540,13 +540,33 @@ async function runServer() {
console.error('Supadata MCP Server initialized successfully');
console.error('Supadata MCP Server running on stdio');
} catch (error) {
console.error('Fatal error running server:', error);
console.error('Fatal error running STDIO server:', error);
process.exit(1);
}
}

// Server startup logic - support both STDIO and HTTP modes
async function runServer() {
const transportMode = process.env.MCP_TRANSPORT_MODE || 'stdio';

if (transportMode === 'http') {
// Import and run HTTP server
const { runHttpServer } = await import('./httpServer.js');
const port = process.env.PORT ? parseInt(process.env.PORT) : 3000;
await runHttpServer(port);
} else {
// Default to STDIO transport
await runStdioServer();
}
}

// Only run the server if this file is executed directly
runServer().catch((error: any) => {
console.error('Fatal error running server:', error);
process.exit(1);
});
if (import.meta.url === `file://${process.argv[1]}`) {
runServer().catch((error: any) => {
console.error('Fatal error running server:', error);
process.exit(1);
});
}

// Default export for compatibility
export default createServer;