bug(browser): Browser Service HTTP 404 — hardcoded port in web_crawler.py fallback + deployment config investigation needed

[mrveiss]

Symptom

Service health widget shows "Browser Service HTTP 404" — _check_http_health receives a 404 from the browser service health endpoint.

Root Causes Found

1. Hardcoded :3000 in web_crawler.py fallback (fixed in this issue)

autobot-backend/knowledge/connectors/web_crawler.py:163

# BEFORE (broken)
return f"http://{os.environ.get('AUTOBOT_BROWSER_SERVICE_HOST', '')}:3000"
# AFTER (fixed)
return f"http://{os.environ.get('AUTOBOT_BROWSER_SERVICE_HOST', '')}:{NetworkConstants.BROWSER_SERVICE_PORT}"

2. Deployment config investigation needed

The 404 means the browser VM IP is reachable (not empty/connection refused) but the endpoint returns 404. Possible causes:

• .env.example ships with AUTOBOT_BROWSER_SERVICE_HOST=172.16.168.25 — if the production env copied this but the actual browser VM is at a different IP, the request hits the wrong host
• Browser worker (playwright-server.js) has /health at line 71 and should return 200 if running
• NetworkConstants.BROWSER_VM_IP defaults to ConfigRegistry.get("vm.browser", "") — empty string if not configured via SLM settings

Fix Applied

Changed web_crawler.py:163 to use NetworkConstants.BROWSER_SERVICE_PORT instead of hardcoded 3000.

Investigation Needed in Deployment

1. Check vm.browser in SLM ConfigRegistry / Redis: redis-cli -h $AUTOBOT_REDIS_HOST get config:vm.browser
2. Check backend env: grep BROWSER /etc/autobot/autobot.env
3. Verify what's running at the configured IP:port: curl -v http://$BROWSER_VM_IP:3000/health
4. Check backend logs for the exact URL being called when the 404 occurs

Related Code

• autobot-backend/knowledge/connectors/web_crawler.py:163 — hardcoded port (fixed)
• autobot-backend/services/playwright_service.py:465 — uses AUTOBOT_BROWSER_SERVICE_HOST env var
• autobot-backend/api/service_monitor.py:76 — health check uses config.get_service_url("browser", "health")
• autobot-backend/api/playwright.py:77 — BROWSER_VM_URL from NetworkConstants.BROWSER_VM_IP

[mrveiss]

✅ Fixed in PR #3731 (merged).