feat: add capabilities endpoint and enhance AGUI event handling#613
feat: add capabilities endpoint and enhance AGUI event handling#613Gkrumbach07 wants to merge 6 commits intoambient-code:mainfrom
Conversation
- Introduced a new endpoint for retrieving runner capabilities at `/agentic-sessions/:sessionName/agui/capabilities`. - Implemented the `HandleCapabilities` function to authenticate users, verify permissions, and proxy requests to the runner. - Enhanced AGUI event handling by adding support for custom events and persisting message snapshots for faster reconnections. - Updated the frontend to utilize the new capabilities endpoint and replaced the existing chat component with `CopilotChatPanel` for improved user experience. This update improves the overall functionality and performance of the AG-UI system, allowing for better integration with the runner's capabilities and enhancing user interactions.
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
This comment has been minimized.
This comment has been minimized.
- Fixed a typo in the event type constant from `EventTypStateDelta` to `EventTypeStateDelta`. - Added a new event type constant `EventTypeCustom` for platform extensions. - Refactored message extraction logic from snapshots to improve handling of messages from persisted snapshots. - Removed the deprecated `loadCompactedMessages` function and updated the event streaming logic to utilize persisted message snapshots for better performance and reliability. These changes enhance the overall stability and functionality of the AG-UI event handling system.
Claude Code ReviewSummaryThis PR introduces a new capabilities endpoint and significantly refactors the AGUI event handling system. The changes replace custom event compaction logic with runner-emitted snapshots and integrate CopilotKit for the frontend chat UI. Overall, the implementation demonstrates strong security practices and architectural clarity, with a few areas requiring attention before merge. Key Changes:
Issues by Severity🚫 Blocker IssuesNone - No critical security or correctness issues that block merge. 🔴 Critical Issues1. Frontend Type Definitions Violate StandardsLocation: The codebase standard is to always use Problem: // Added in this PR - violates guidelines
interface Capabilities { ... }Fix Required: // Should be:
type Capabilities = { ... }Reference: CLAUDE.md lines 1141-1145, frontend-development.md lines 73-76 2. Missing Type Safety in Capabilities ResponseLocation: var result map[string]interface{}
if err := json.NewDecoder(resp.Body).Decode(&result); err != nil {
log.Printf("Capabilities: Failed to decode response: %v", err)
c.JSON(http.StatusInternalServerError, gin.H{"error": "Failed to parse runner response"})
return
}
c.JSON(http.StatusOK, result)Issues:
Recommendation:
Pattern: See error-handling.md lines 199-220 for proper error exposure patterns. 3. Large Dependency Additions Without JustificationLocation: Added Dependencies:
Impact:
Missing:
Recommendation:
🟡 Major Issues4. Fallback Capabilities Response May Hide ErrorsLocation: if err != nil {
log.Printf("Capabilities: Request failed: %v", err)
// Runner not ready — return minimal default
c.JSON(http.StatusOK, gin.H{
"framework": "unknown",
"agent_features": []interface{}{},
"platform_features": []interface{}{},
"file_system": false,
"mcp": false,
})
return
}Issue:
Recommendation: c.JSON(http.StatusServiceUnavailable, gin.H{
"error": "Runner not available",
"message": "Session is starting or runner is unavailable",
})Frontend can then show appropriate loading/error state. 5. Missing Error Context in LogsLocation: if eventType == types.EventTypeMessagesSnapshot {
go persistMessagesSnapshot(sessionID, event)
}Issue:
Recommendation: 6. Deleted Compaction Logic Without Migration PathLocation: Issue:
Questions:
Recommendation: 🔵 Minor Issues7. Frontend Component Missing Loading StatesLocation: Issue:
Recommendation: export function CopilotChatPanel({ projectName, sessionName }: Props) {
const { data: capabilities, isLoading, error } = useCapabilities(projectName, sessionName);
if (isLoading) return <div>Initializing chat...</div>;
if (error) return <div>Failed to connect: {error.message}</div>;
return <CopilotKit runtimeUrl={...}>...</CopilotKit>;
}Reference: frontend-development.md line 156 (all buttons/components need loading states) 8. Typo Fixed But Inconsistent NamingLocation: -EventTypStateDelta = "STATE_DELTA" // Typo fixed
+EventTypeStateDelta = "STATE_DELTA"Good: Typo fixed ✅ Issue: Existing code may reference grep -r "EventTypStateDelta" components/backend components/operator9. Missing Test Coverage for New EndpointLocation: Issue:
Recommendation: func TestHandleCapabilities_Unauthorized(t *testing.T) { ... }
func TestHandleCapabilities_RunnerUnavailable(t *testing.T) { ... }
func TestHandleCapabilities_Success(t *testing.T) { ... }10. Runner Endpoint Uses Global StateLocation: has_langfuse = state._obs is not None and state._obs.langfuse_client is not NoneIssue:
Recommendation: def has_observability() -> bool:
return state._obs is not None and state._obs.langfuse_client is not NonePositive Highlights✅ Security Done Right
Reference Compliance: Follows k8s-client-usage.md patterns exactly. ✅ ✅ Excellent Code Organization
✅ React Query UsageThe new
Reference Compliance: Follows react-query-usage.md patterns exactly. ✅ ✅ Backend Proxy PatternThe
✅ DocumentationAdded comprehensive ADR (0006-ambient-runner-sdk-architecture.md, +423 lines) explaining architectural decisions. Strong documentation culture. 📚 RecommendationsPriority 1 (Before Merge)
Priority 2 (Nice to Have)
Priority 3 (Future Work)
Overall AssessmentCode Quality: 8.5/10 — Strong security practices, good architectural clarity, minor issues with frontend standards compliance. Readiness: APPROVE with changes — Fix the frontend The removal of compaction logic is a significant architectural improvement (MESSAGES_SNAPSHOT from runner is cleaner). The new capabilities endpoint follows all security patterns correctly. Great work on the refactor! 🚀 References:
🔍 View AI decision process (logs available for 90 days) 📋 View memory system files loaded (click to expand)What Amber Loaded for Code ReviewAmber automatically loaded these repository standards from the memory system:
Impact: This review used your repository's specific code quality standards, security patterns, and best practices from the memory system (PRs #359, #360) - not just generic code review guidelines. |
…ent variable handling - Removed outdated content service initialization and related handlers from the backend. - Updated GitHub Actions workflows to eliminate backend environment variable updates, streamlining deployment processes. - Adjusted operator environment variable settings to reflect changes in image tagging and deployment strategies. These changes enhance the clarity and maintainability of the codebase while improving deployment efficiency.
Claude Code ReviewSummaryThis PR introduces AG-UI capabilities endpoint and enhances event handling with significant architectural improvements. The changes replace the legacy content service with a CopilotKit-based chat interface and implement proper event persistence/replay. Issues by Severity🚫 Blocker IssuesNone - no blocking issues found. 🔴 Critical Issues1. Token Handling Priority Mismatch (Security)Location: The token extraction logic prefers Issue: This suggests the Authorization header may contain invalid tokens from untrusted sources (browser OAuth session cookies forwarded by CopilotKit). While the current implementation is secure (it validates whichever token it uses), the root cause should be addressed. Recommendation:
Risk: Medium - current code is secure, but relies on header priority rather than fixing the source. 2. Missing Error Context in Proxy HandlersLocation:
resp, err := (&http.Client{Timeout: 10 * time.Second}).Do(req)
if err != nil {
c.JSON(http.StatusOK, gin.H{"framework": "unknown"}) // ❌ No error logged
return
}Issue: Silent failures make debugging runner connectivity issues impossible. Required Fix: if err != nil {
log.Printf("AGUI Capabilities: runner unavailable for %s: %v", sessionName, err)
c.JSON(http.StatusOK, gin.H{"framework": "unknown"})
return
}Pattern: Follows established pattern from 3. Orphaned Tool Result Repair Missing ValidationLocation:
Missing validation:
Recommendation: // Validate args are parseable JSON before adding
var argsTest interface{}
if err := json.Unmarshal([]byte(td.args), &argsTest); err != nil {
log.Printf("AGUI Store: skipping tool %s with invalid args: %v", td.name, err)
continue
}
// Limit repair count
if len(repairedToolCalls) > 100 {
log.Printf("AGUI Store: too many orphaned results (%d), truncating", len(orphanedIDs))
break
}🟡 Major Issues4. Frontend Type Safety ViolationsLocation: Multiple frontend files Issues found:
Required Fix: // ❌ BAD
agents: { session: agent as any },
// ✅ GOOD
type CompatibleAgent = Agent & { compatVersion?: string }
agents: { session: agent as CompatibleAgent },Pattern Violation: Frontend Development Standards require ZERO 5. Event Timestamp Handling InconsistencyLocation: Issue: The proxy deliberately does NOT inject timestamps (line 236 comment), but Concern:
Recommendation:
6. React Query Polling LogicLocation: refetchInterval: (query) => {
if (query.state.data?.framework && query.state.data.framework !== "unknown") {
return false;
}
const updatedCount = (query.state as { dataUpdatedCount?: number }).dataUpdatedCount ?? 0;
if (updatedCount >= 6) return false;
return 10 * 1000;
}Issue: Accessing Recommended Fix: let pollAttempts = 0;
refetchInterval: (query) => {
if (query.state.data?.framework && query.state.data.framework !== "unknown") {
return false;
}
if (++pollAttempts >= 6) return false;
return 10 * 1000;
}🔵 Minor Issues7. Inconsistent Error Response FormatLocation:
Recommendation: Standardize error response shape across all AG-UI endpoints. 8. Missing RBAC Check ContextLocation:
res, err := reqK8s.AuthorizationV1().SelfSubjectAccessReviews().Create(
context.Background(), ssar, metav1.CreateOptions{}, // ❌ Should use request context
)Recommendation: Pass request context for proper timeout/cancellation handling. 9. Frontend Component SizeLocation:
Guideline Violation: Frontend standards recommend components under 200 lines. Recommendation: Extract sub-components:
10. Logging InconsistencyLocation: Various files Some logs use structured prefixes ( log.Printf("AGUI Proxy: run=%s session=%s/%s msgs=%d", ...) // ✅ Good
log.Printf("Failed to create job: %v", err) // ❌ Missing prefixRecommendation: Standardize all AGUI-related logs with Positive Highlights✅ Excellent Architecture Decisions
Testing Coverage✅ Tests Found
|
- Updated the AGUI proxy to replay compacted events individually on reconnect, improving the handling of conversation history. - Refactored event persistence logic to support efficient event compaction and replay, aligning with the InMemoryAgentRunner pattern. - Enhanced the frontend session event display by adding an expandable view for older events, improving user experience. - Normalized argument comparison in tool call rendering to ensure accurate matching. These changes enhance the performance and usability of the AG-UI system, providing a more responsive and reliable user experience.
Claude Code ReviewSummaryPR #613 is a significant refactoring that migrates the AG-UI (Agentic UI) system to a new architecture with several key improvements:
Overall Assessment: The architectural direction is sound, but there are critical security and code quality issues that must be addressed before merging. Issues by Severity🚫 Blocker IssuesNone identified - no issues that completely prevent functionality. 🔴 Critical Issues1. Missing User Token Authentication in Capabilities EndpointLocation: Issue: The capabilities endpoint does not validate user authentication using Evidence: Based on the PR description mentioning "authenticate users, verify permissions", but the standard pattern from reqK8s, reqDyn := GetK8sClientsForRequest(c)
if reqK8s == nil {
c.JSON(http.StatusUnauthorized, gin.H{"error": "Invalid or missing token"})
c.Abort()
return
}Why Critical: Violates Critical Rule #1 from CLAUDE.md - "User Token Authentication Required". Could allow unauthorized access to runner capabilities. Fix Required: Add user token authentication check at the beginning of 2. Potential Token Logging in Event PersistenceLocation: Issue: Event persistence writes entire events to JSONL logs, but there's no evidence of token redaction in the event data. Risk: If events contain request metadata with tokens, they could be written to disk unredacted. Why Critical: Violates Critical Rule #3 from CLAUDE.md - "Token Security and Redaction". Tokens must never be logged. Fix Required:
3. Type Safety Issues in Event HandlingLocation: Issue: Multiple uses of var evt map[string]interface{}
// Direct access without checking
evt["type"] // Could panic if key doesn't existWhy Critical: Violates Critical Rule #4 from CLAUDE.md - "Type-Safe Unstructured Access". Can cause panics in production. Fix Required: Use type assertions with checks: eventType, ok := evt["type"].(string)
if !ok {
log.Printf("Invalid event type")
return
}🟡 Major Issues4. Missing Error Context in Backend HandlersLocation: Issue: Some error returns don't include wrapped errors with context: return fmt.Errorf("failed to X: %w", err) // Good
return err // Bad - loses contextPattern from error-handling.md:
Fix Required: Review all error handling in session handlers and ensure proper wrapping. 5. Frontend: Possible
|
- Updated the AGUI proxy to persist events synchronously, ensuring correct JSONL ordering and preventing race conditions in event writing. - Enhanced comments in the code to clarify the handling of various event types, including the treatment of MESSAGES_SNAPSHOT and other events during streaming. - Adjusted the compactStreamingEvents function documentation to reflect the inclusion of specific event types in the unchanged flow. These changes enhance the reliability and clarity of the AG-UI event handling system.
Claude Code ReviewSummaryPR #613 introduces a new capabilities endpoint and enhances AGUI event handling. The implementation follows all critical security patterns and architectural standards. APPROVED with one optional minor improvement. Issues by Severity🚫 Blocker IssuesNone - All critical security patterns are correctly implemented. 🔴 Critical IssuesNone - No critical issues found. 🟡 Major IssuesNone - No major issues found. 🔵 Minor Issues1. Missing Log Sanitization in HandleCapabilities
Positive Highlights✅ Security - Exemplary Implementation
✅ Error Handling - Graceful Degradation
✅ Type Safety - Zero Issues
✅ Architecture - Well-Designed
✅ Testing - Comprehensive
RecommendationsPriority 1 (Optional)Add log sanitization to HandleCapabilities for consistency with other handlers: // In agui_proxy.go:390-391
projectName := handlers.SanitizeForLog(c.Param("projectName"))
sessionName := handlers.SanitizeForLog(c.Param("sessionName"))Priority 2 (Future Enhancement)Consider adding Go unit tests for HandleCapabilities, though integration tests likely cover this. Files Reviewed
Final Verdict✅ APPROVE - This PR is production-ready. The single minor issue is not a blocker and can be addressed in a follow-up if desired. 🤖 Generated with Claude Code 🔍 View AI decision process (logs available for 90 days) 📋 View memory system files loaded (click to expand)What Amber Loaded for Code ReviewAmber automatically loaded these repository standards from the memory system:
Impact: This review used your repository's specific code quality standards, security patterns, and best practices from the memory system (PRs #359, #360) - not just generic code review guidelines. |
- Updated the AGUI proxy to clarify the handling of empty messages and reconnections, ensuring that the frontend manages reconnects while the backend focuses on event persistence. - Removed deprecated event replay logic and streamlined the event persistence process to enhance performance and reliability. - Enhanced comments and documentation throughout the code to provide clearer guidance on event processing and the role of the InMemoryAgentRunner. These changes improve the overall clarity and efficiency of the AG-UI event handling system.
Claude Code ReviewSummaryThis PR introduces a significant architectural improvement by migrating the AG-UI (Agentic UI) system from a custom WebSocket-based implementation to the industry-standard CopilotKit framework. The changes include:
Overall code quality is excellent with strong adherence to project standards. The refactoring significantly reduces complexity while improving maintainability. Issues by Severity🚫 Blocker IssuesNone 🔴 Critical IssuesNone 🟡 Major Issues1. Missing Test Coverage for New Capabilities Endpoint // components/backend/websocket/agui_proxy.go:315
func HandleCapabilities(c *gin.Context) {
// ... authentication and RBAC checks
// ... proxies to runner /capabilities endpoint
}Issue: No tests found for the new Impact: Cannot verify RBAC enforcement, error handling, or fallback behavior when runner is unavailable. Recommendation: Add tests similar to existing handler tests:
Reference: See 2. Potential Race Condition in Frontend Capabilities Polling // components/frontend/src/services/queries/use-capabilities.ts:29-38
refetchInterval: (query) => {
if (query.state.data?.framework && query.state.data.framework !== "unknown") {
return false;
}
const updatedCount = (query.state as { dataUpdatedCount?: number }).dataUpdatedCount ?? 0;
if (updatedCount >= 6) return false;
return 10 * 1000;
}Issue: Type assertion Impact: Silent failure if React Query API changes. Polling may not stop as expected. Recommendation:
🔵 Minor Issues1. Inconsistent Route Parameter Format // components/backend/routes.go:65-73
projectGroup.POST("/agentic-sessions:sessionName/agui/run", ...) // uses colon
projectGroup.GET("/agentic-sessions/:sessionName/agui/capabilities", ...) // uses slashIssue: Route parameter syntax inconsistency ( Impact: Route Recommendation: Verify all routes use consistent parameter syntax: projectGroup.POST("/agentic-sessions/:sessionName/agui/run", ...)
projectGroup.POST("/agentic-sessions/:sessionName/agui/interrupt", ...)
projectGroup.GET("/agentic-sessions/:sessionName/agui/capabilities", ...)2. Hardcoded Timeout in HTTP Client // components/backend/websocket/agui_proxy.go:339
resp, err := (&http.Client{Timeout: 10 * time.Second}).Do(req)Issue: 10-second timeout is hardcoded for capabilities endpoint. Impact: Not configurable for different deployment scenarios (slow networks, resource-constrained environments). Recommendation: Extract to a constant or environment variable: const runnerRequestTimeout = 10 * time.Second // or from env3. Silent Error Handling in Capabilities Endpoint // components/backend/websocket/agui_proxy.go:340-349
if err != nil {
c.JSON(http.StatusOK, gin.H{
"framework": "unknown",
// ... default values
})
return
}Issue: Returns 200 OK with default values when runner is unavailable, making it hard to distinguish between "runner not ready" and "capabilities are actually unknown". Impact: Frontend polling may not behave correctly. Observability reduced (cannot tell if runner is down vs. uninitialized). Recommendation: Consider one of:
4. Missing JSDoc Comments in Frontend Components // components/frontend/src/components/session/CopilotChatPanel.tsx:47-55
export function CopilotSessionProvider({
projectName,
sessionName,
children,
}: {
projectName: string;
sessionName: string;
children: React.ReactNode;
}) {Issue: No JSDoc explaining the purpose of Impact: Developers may misuse the component or create duplicate instances. Recommendation: Add JSDoc: /**
* Provides CopilotKit context with AG-UI agent connection.
*
* Mount ONCE per session (at page level) to ensure chat state persists
* across desktop/mobile layout switches.
*
* @param projectName - K8s namespace
* @param sessionName - AgenticSession name (also used as threadId)
*/
export function CopilotSessionProvider({ ... }) {Positive Highlights✅ Excellent Security Practices1. User Token Authentication Enforced // components/backend/websocket/agui_proxy.go:46-56
reqK8s, _ := handlers.GetK8sClientsForRequest(c)
if reqK8s == nil {
c.JSON(http.StatusUnauthorized, gin.H{"error": "Invalid or missing token"})
c.Abort()
return
}
if !checkAccess(reqK8s, projectName, sessionName, "update") {
c.JSON(http.StatusForbidden, gin.H{"error": "Unauthorized"})
c.Abort()
return
}✅ Follows 2. No Token Leaks ✅ Strong TypeScript Type Safety1. Zero // components/frontend/src/app/api/copilotkit/[project]/[session]/route.ts:49-50
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- AbstractAgent version mismatch
agents: { session: agent as any },✅ Only ONE 2. Proper React Query Patterns // components/frontend/src/services/queries/use-capabilities.ts:22-26
return useQuery({
queryKey: capabilitiesKeys.session(projectName, sessionName),
queryFn: () => sessionsApi.getCapabilities(projectName, sessionName),
enabled: enabled && !!projectName && !!sessionName,
staleTime: 60 * 1000,✅ Query keys include all parameters (no cache collisions) ✅ Clean Error Handling1. Non-Fatal Errors Logged, Operation Continues // components/backend/websocket/agui_proxy.go:129-133
if statusCode != http.StatusOK {
log.Printf("AGUI Proxy: runner returned %d for run %s", statusCode, truncID(runID))
writeSSEError(c.Writer, fmt.Sprintf("Runner returned HTTP %d", statusCode))
return
}✅ Errors logged with context (run ID, status code) 2. IsNotFound Handled Gracefully // components/operator/internal/handlers/sessions.go:54-60
if errors.IsNotFound(err) {
log.Printf("AgenticSession %s no longer exists, skipping processing", name)
return nil // Not an error - resource deleted
}✅ Correctly treats ✅ Excellent Code Simplification1. Massive Reduction in Complexity
2. Improved Separation of Concerns ✅ Each file has a single, clear responsibility ✅ Strong Documentation1. Inline Comments Explain Architecture // agui_proxy.go:7-12
// Two jobs:
// 1. Passthrough: POST to runner, pipe SSE back to client.
// 2. Persist: append every event to agui-events.jsonl as it flows through.
//
// Reconnection is handled by InMemoryAgentRunner on the frontend.
// The backend only persists events for cross-restart recovery.✅ Clarifies division of responsibility between frontend and backend 2. README Updates
RecommendationsPriority 1: Address Before Merge
Priority 2: Consider for Follow-Up PR
Priority 3: Future Improvements
Final AssessmentOverall Grade: A (Excellent) This is a well-architected refactoring that significantly improves code quality while maintaining backward compatibility. The migration to CopilotKit is executed cleanly with strong adherence to security standards, type safety, and error handling patterns. Key strengths:
Minor issues are mostly documentation/observability improvements that don't affect correctness. Recommendation: Approve after addressing Priority 1 items (route syntax fix, tests, type assertion fix). Appendix: Standards ComplianceVerified compliance with:
🔍 View AI decision process (logs available for 90 days) 📋 View memory system files loaded (click to expand)What Amber Loaded for Code ReviewAmber automatically loaded these repository standards from the memory system:
Impact: This review used your repository's specific code quality standards, security patterns, and best practices from the memory system (PRs #359, #360) - not just generic code review guidelines. |
1 participant
/agentic-sessions/:sessionName/agui/capabilities.HandleCapabilitiesfunction to authenticate users, verify permissions, and proxy requests to the runner.CopilotChatPanelfor improved user experience.This update improves the overall functionality and performance of the AG-UI system, allowing for better integration with the runner's capabilities and enhancing user interactions.