Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
102 changes: 5 additions & 97 deletions multimodal/websites/tarko/docs/en/guide/advanced/agent-hooks.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,9 @@ Agent Hooks are callback methods that execute at specific points during agent op
- **Termination Hooks**: Custom completion criteria enforcement
- **Request Preparation**: Dynamic system prompt and tool modification

## Core Hook Categories
## Core Hooks

### LLM Communication Hooks
### Hooks interacting with LLM

Intercept and monitor LLM requests and responses:

Expand Down Expand Up @@ -401,106 +401,14 @@ Understanding the hook execution sequence is crucial for proper implementation:

## Testing Hooks

### Unit Testing Individual Hooks

```typescript
import { Agent, AgentOptions } from '@tarko/agent';

class TestableAgent extends Agent {
public hookCallLog: string[] = [];

override async onLLMRequest(id: string, payload: any) {
this.hookCallLog.push(`onLLMRequest:${id}`);
}

override async onBeforeToolCall(id: string, toolCall: any, args: any) {
this.hookCallLog.push(`onBeforeToolCall:${toolCall.name}`);
return args;
}
}

describe('Agent Hooks', () => {
let agent: TestableAgent;

beforeEach(() => {
agent = new TestableAgent({});
});

it('should call hooks in correct order', async () => {
// Test hook execution order
await agent.onLLMRequest('test-session', { model: 'gpt-4', messages: [] });
await agent.onBeforeToolCall('test-session', { name: 'test-tool' }, {});

expect(agent.hookCallLog).toEqual([
'onLLMRequest:test-session',
'onBeforeToolCall:test-tool'
]);
});
});
```

### Integration Testing

```typescript
describe('Hook Integration', () => {
it('should prevent termination when required', async () => {
const agent = new ValidatingAgent({
tools: [/* required tools */]
});

// Mock a scenario where not all required tools are called
const result = await agent.onBeforeLoopTermination(
'test-session',
{ content: 'Final answer' } as any
);

expect(result.finished).toBe(false);
expect(result.message).toContain('Must call required tools');
});
});
```

## Best Practices

### 1. Hook Design
- Keep hooks focused and lightweight
- Always call `super.hookMethod()` when overriding
- Handle errors gracefully without breaking agent execution
- Use async/await for asynchronous operations

### 2. State Management
- Store hook-specific state in instance variables
- Clean up state in `onAgentLoopEnd`
- Use session IDs to track per-conversation state

### 3. Performance
- Avoid blocking operations in critical hooks
- Use caching for expensive computations
- Implement timeouts for external calls

### 4. Error Handling
- Always handle hook errors gracefully
- Provide fallback behavior when possible
- Log errors for debugging without exposing sensitive data

### 5. Testing
- Unit test hooks in isolation
- Test hook composition and ordering
- Mock external dependencies
- Test error scenarios
WIP

## Real-World Examples

See our [Examples](/examples/custom-hooks) section for complete implementations of:

- Analytics and metrics collection
- User permission enforcement
- Rate limiting and quota management
- Multi-step workflow validation
- Error recovery and retry mechanisms
WIP

## Next Steps

- [Agent Protocol](/guide/advanced/agent-protocol) - Understand event handling in hooks
- [Tool Management](/guide/basic/tools) - Learn about tool registration and execution
- [Context Engineering](/guide/advanced/context-engineering) - Advanced context management
- [Context Engineering](/guide/advanced/context-engineering) - Advanced context management
153 changes: 29 additions & 124 deletions multimodal/websites/tarko/docs/zh/guide/advanced/agent-hooks.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,9 @@ Agent Hooks 是在 Agent 运行特定节点执行的回调方法。所有 Hook
- **Termination Hooks**:自定义完成条件强制执行
- **Request Preparation**:动态系统提示和工具修改

## 核心 Hook 类别
## 核心 Hook

### LLM 通信 Hooks
### LLM 交互 Hook

拦截和监控 LLM 请求和响应:

Expand All @@ -30,32 +30,32 @@ class MonitoringAgent extends Agent {
// 在每个 LLM 请求之前调用
override async onLLMRequest(id: string, payload: LLMRequestHookPayload) {
console.log(`[${id}] 发送请求到 ${payload.model}`);
console.log('消息数量:', payload.messages.length);
console.log('Messages count:', payload.messages.length);

// 记录 token 使用估算
const tokenEstimate = this.estimateTokens(payload.messages);
console.log('预估 tokens:', tokenEstimate);
console.log('Estimated tokens:', tokenEstimate);
}

// 在每个 LLM 响应之后调用
override async onLLMResponse(id: string, payload: LLMResponseHookPayload) {
const response = payload.response;
console.log(`[${id}] 收到响应:`);
console.log('使用情况:', response.usage);
console.log('完成原因:', response.choices[0]?.finish_reason);
console.log('Usage:', response.usage);
console.log('Finish reason:', response.choices[0]?.finish_reason);

// 如果存在工具调用则记录
const toolCalls = response.choices[0]?.message?.tool_calls;
if (toolCalls?.length) {
console.log('工具调用:', toolCalls.map(tc => tc.function.name));
console.log('Tool calls:', toolCalls.map(tc => tc.function.name));
}
}

// 在流式响应期间调用
override onLLMStreamingResponse(id: string, payload: LLMStreamingResponseHookPayload) {
// 实时监控流式块
// 实时监控 streaming chunks
const chunks = payload.chunks;
console.log(`[${id}] 收到 ${chunks.length} 个流式块`);
console.log(`[${id}] 收到 ${chunks.length} 个 streaming chunks`);
}
}
```
Expand All @@ -74,16 +74,16 @@ class ToolMonitoringAgent extends Agent {
toolCall: { toolCallId: string; name: string },
args: any
) {
console.log(`[${id}] 执行工具: ${toolCall.name}`);
console.log('参数:', JSON.stringify(args, null, 2));
console.log(`[${id}] Executing tool: ${toolCall.name}`);
console.log('Arguments:', JSON.stringify(args, null, 2));

// 跟踪工具使用情况
const currentCount = this.toolUsageStats.get(toolCall.name) || 0;
this.toolUsageStats.set(toolCall.name, currentCount + 1);

// 验证参数或应用速率限制
if (toolCall.name === 'expensive_api' && currentCount >= 5) {
throw new Error('expensive_api 超出速率限制');
throw new Error('Rate limit exceeded for expensive_api');
}

// 返回可能修改的参数
Expand All @@ -96,14 +96,14 @@ class ToolMonitoringAgent extends Agent {
toolCall: { toolCallId: string; name: string },
result: any
) {
console.log(`[${id}] 工具 ${toolCall.name} 完成`);
console.log('结果类型:', typeof result);
console.log(`[${id}] Tool ${toolCall.name} completed`);
console.log('Result type:', typeof result);

// 记录错误或成功结果
if (result?.error) {
console.error('工具执行失败:', result.error);
console.error('Tool execution failed:', result.error);
} else {
console.log('工具执行成功');
console.log('Tool execution successful');
}

// 返回可能修改的结果
Expand All @@ -116,14 +116,14 @@ class ToolMonitoringAgent extends Agent {
toolCall: { toolCallId: string; name: string },
error: any
) {
console.error(`[${id}] 工具 ${toolCall.name} 失败:`, error);
console.error(`[${id}] Tool ${toolCall.name} failed:`, error);

// 实现重试逻辑或错误转换
if (error.message?.includes('timeout')) {
return '工具执行超时。请稍后重试。';
return 'Tool execution timed out. Please try again later.';
}

return `错误: ${error.message || error}`;
return `Error: ${error.message || error}`;
}

// 完全覆盖工具调用处理
Expand All @@ -138,7 +138,7 @@ class ToolMonitoringAgent extends Agent {
if (process.env.NODE_ENV === 'test') {
return toolCalls.map(tc => ({
toolCallId: tc.id,
result: `${tc.function.name} 的模拟结果`,
result: `Mocked result for ${tc.function.name}`,
success: true
}));
}
Expand All @@ -148,7 +148,7 @@ class ToolMonitoringAgent extends Agent {
}
```

### 循环生命周期 Hooks
### 循环生命周期 Hook

控制 Agent 循环迭代和终止:

Expand All @@ -159,30 +159,30 @@ class LoopControlAgent extends Agent {
// 在每个循环迭代开始时调用
override async onEachAgentLoopStart(sessionId: string) {
this.iterationStartTimes.set(sessionId, Date.now());
console.log(`[${sessionId}] 开始迭代 ${this.getCurrentLoopIteration()}`);
console.log(`[${sessionId}] Starting iteration ${this.getCurrentLoopIteration()}`);

// 注入额外上下文或执行设置
const currentTime = new Date().toISOString();
console.log(`当前时间: ${currentTime}`);
console.log(`Current time: ${currentTime}`);
}

// 在每个循环迭代结束时调用
override async onEachAgentLoopEnd(context: EachAgentLoopEndContext) {
const startTime = this.iterationStartTimes.get(context.sessionId);
if (startTime) {
const duration = Date.now() - startTime;
console.log(`[${context.sessionId}] 迭代在 ${duration}ms 内完成`);
console.log(`[${context.sessionId}] Iteration completed in ${duration}ms`);
}

// 记录迭代结果
console.log('此迭代中的事件:', context.events?.length || 0);
console.log('进行的工具调用:', context.toolCallResults?.length || 0);
console.log('Events in this iteration:', context.events?.length || 0);
console.log('Tool calls made:', context.toolCallResults?.length || 0);
}

// 当整个 Agent 循环结束时调用
override async onAgentLoopEnd(id: string) {
console.log(`[${id}] Agent 循环完成`);
console.log('总迭代次数:', this.getCurrentLoopIteration());
console.log(`[${id}] Agent loop completed`);
console.log('Total iterations:', this.getCurrentLoopIteration());

// 清理迭代跟踪
this.iterationStartTimes.delete(id);
Expand Down Expand Up @@ -399,105 +399,10 @@ class ResilientAgent extends Agent {
9. [否则从步骤 2 重复]
```

## 测试 Hooks

### 单元测试单个 Hooks

```typescript
import { Agent, AgentOptions } from '@tarko/agent';

class TestableAgent extends Agent {
public hookCallLog: string[] = [];

override async onLLMRequest(id: string, payload: any) {
this.hookCallLog.push(`onLLMRequest:${id}`);
}

override async onBeforeToolCall(id: string, toolCall: any, args: any) {
this.hookCallLog.push(`onBeforeToolCall:${toolCall.name}`);
return args;
}
}

describe('Agent Hooks', () => {
let agent: TestableAgent;

beforeEach(() => {
agent = new TestableAgent({});
});

it('should call Hook in correct order', async () => {
// 测试 hook 执行顺序
await agent.onLLMRequest('test-session', { model: 'gpt-4', messages: [] });
await agent.onBeforeToolCall('test-session', { name: 'test-tool' }, {});

expect(agent.hookCallLog).toEqual([
'onLLMRequest:test-session',
'onBeforeToolCall:test-tool'
]);
});
});
```

### 集成测试

```typescript
describe('Hook Integration', () => {
it('should prevent termination when required', async () => {
const agent = new ValidatingAgent({
tools: [/* 必需工具 */]
});

// 模拟未调用所有必需工具的场景
const result = await agent.onBeforeLoopTermination(
'test-session',
{ content: '最终答案' } as any
);

expect(result.finished).toBe(false);
expect(result.message).toContain('必须调用必需工具');
});
});
```

## 最佳实践

### 1. Hook 设计
- 保持 Hook 专注和轻量级
- 重写时始终调用 `super.hookMethod()`
- 优雅处理错误而不破坏 Agent 执行
- 对异步操作使用 async/await

### 2. 状态管理
- 在实例变量中存储 hook 特定状态
- 在 `onAgentLoopEnd` 中清理状态
- 使用会话 ID 跟踪每个对话的状态

### 3. 性能
- 避免在关键 Hook 中阻塞操作
- 对昂贵的计算使用缓存
- 为外部调用实现超时

### 4. 错误处理
- 始终优雅处理 hook 错误
- 在可能的情况下提供回退行为
- 记录错误以便调试,但不暴露敏感数据

### 5. 测试
- 单独单元测试 Hook
- 测试 hook 组合和顺序
- 模拟外部依赖
- 测试错误场景

## 实际示例

查看我们的[示例](/examples/custom-hooks)部分以获取完整实现:

- 分析和指标收集
- 用户权限强制执行
- 速率限制和配额管理
- 多步骤工作流验证
- 错误恢复和重试机制
WIP

## 下一步

Expand Down