feat: add enterprise proxy support to pulse-fetch with automatic detection

[tadasant]

Summary

• Added opt-in enterprise proxy support to pulse-fetch MCP server using proxy-agent
• Proxy support is disabled by default and must be enabled with ENABLE_PROXY_SETTINGS=true
• When enabled, automatically detects proxy settings from environment variables, system settings, and PAC files
• Enables pulse-fetch to work seamlessly in corporate environments with restricted internet access

Implementation Details

• Uses proxy-agent library for automatic proxy detection and configuration
• Opt-in by default: Requires ENABLE_PROXY_SETTINGS=true to activate proxy support
• Zero-configuration experience when enabled: if your browser works with your proxy, pulse-fetch works too
• Supports multiple proxy sources when enabled:
  • Environment variables (HTTP_PROXY, HTTPS_PROXY, NO_PROXY)
  • npm configuration
  • System proxy settings (macOS and Windows)
  • PAC (Proxy Auto-Config) files
• Supports multiple proxy types: HTTP, HTTPS, SOCKS, authenticated proxies
• All HTTP/HTTPS requests (native fetch, Firecrawl API, BrightData API, LLM APIs) respect proxy settings when enabled

Testing

• Added integration tests for proxy functionality
• Updated manual tests for proxy-agent behavior
• Manual test results: 94% pass rate (15/16 tests passed)

Testing the Proxy Support

Claude Desktop Testing

To test the proxy support in Claude Desktop, you MUST enable it first:

{
  "mcpServers": {
    "pulse-fetch": {
      "command": "npx",
      "args": ["-y", "@pulsemcp/pulse-fetch"],
      "env": {
        "ENABLE_PROXY_SETTINGS": "true",
        "FIRECRAWL_API_KEY": "your-api-key",
        "BRIGHTDATA_API_KEY": "your-bearer-token"
      }
    }
  }
}

Note: Without ENABLE_PROXY_SETTINGS=true, proxy support remains disabled regardless of other proxy environment variables.

Version Bump

• Bumped @pulsemcp/pulse-fetch from 0.2.13 to 0.2.14

Context

This implementation was updated based on user feedback about Kerberos proxy support. The proxy-agent library provides better enterprise support than undici, including automatic detection of system proxy settings and support for various proxy authentication methods. Proxy support is opt-in to avoid unexpected behavior in non-enterprise environments.

🤖 Generated with Claude Code

[tadasant]

✅ CI Status: All checks passing!

Summary of Implementation

This PR adds opt-in enterprise proxy support to pulse-fetch using the proxy-agent library.

Key features:

• 🔒 Opt-in by default via ENABLE_PROXY_SETTINGS=true
• 🔍 Automatic detection from multiple sources when enabled
• 🏢 Enterprise-ready with support for Kerberos/NTLM via system tools
• ⚡ Zero configuration for users with system proxy settings
• 📚 Comprehensive documentation and testing instructions

Next steps:

• Review the implementation
• Test in your environment using the instructions above
• Let me know if any adjustments are needed

The implementation is ready for review and testing! 🚀

[tadasant]

🚀 Step-by-Step Testing Guide

1️⃣ Clone and Build

# Clone the repository
git clone https://github.com/pulsemcp/mcp-servers.git
cd mcp-servers

# Checkout this PR branch
git checkout tadasant/add-pulse-fetch-proxy

# Install dependencies from root
npm install

# Navigate to pulse-fetch
cd productionized/pulse-fetch

# Build the project
npm run build

2️⃣ Configure Claude Desktop

Find your Claude Desktop config file:

• macOS: ~/Library/Application Support/Claude/claude_desktop_config.json
• Windows: %APPDATA%\Claude\claude_desktop_config.json

Add this configuration (update the path to match where you cloned):

{
  "mcpServers": {
    "pulse-fetch": {
      "command": "node",
      "args": ["/Users/YOUR_USERNAME/mcp-servers/productionized/pulse-fetch/local/build/index.js"],
      "env": {
        "ENABLE_PROXY_SETTINGS": "true",
        "FIRECRAWL_API_KEY": "fc-YOUR_API_KEY",
        "BRIGHTDATA_API_KEY": "Bearer YOUR_BRIGHTDATA_KEY"
      }
    }
  }
}

Important: Replace:

• /Users/YOUR_USERNAME/mcp-servers with your actual path
• API keys with your actual keys

3️⃣ Restart Claude Desktop

1. Quit Claude Desktop completely (not just close the window)
2. Start Claude Desktop again

4️⃣ Test Your Setup

In Claude, try these commands in order:

Test 1 - Basic connectivity:

Scrape https://example.com and tell me the page title

✅ Success: You should see the Example Domain page title

Test 2 - HTTPS through proxy:

Scrape https://httpbin.org/ip and show me the origin IP

✅ Success: The IP shown should be your proxy's IP, not your direct IP

Test 3 - API-enhanced scraping (if you have API keys):

Scrape https://news.ycombinator.com using Firecrawl

✅ Success: You should get clean, structured content

❌ Troubleshooting

"Proxy support disabled" message:

• Make sure ENABLE_PROXY_SETTINGS is set to "true" (with quotes)

Connection errors:

• Verify proxy address is correct
• Check if proxy requires authentication: http://user:pass@proxy:8080

"Cannot find module" errors:

• Make sure you ran npm run build in the pulse-fetch directory
• Check the path in your config points to .../local/build/index.js