Skip to content

Commit

Permalink
much testing work and fixes as a result
Browse files Browse the repository at this point in the history
  • Loading branch information
lalalune committed Feb 14, 2024
1 parent 982b9b8 commit 1bdac1f
Show file tree
Hide file tree
Showing 19 changed files with 355 additions and 154 deletions.
2 changes: 1 addition & 1 deletion scripts/concat.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ import { fileURLToPath } from 'url'
const instructions = 'The above code was taken from my codebase at https://github.com/lalalune/bgent. You are writing tests and documentation for my codebase. Please use the above code as a reference. Tests should be written with Jest and Typescript. Do not use mocks or stubs. Keep it very simple and straightforward.'

// Patterns to ignore
const ignorePatterns = ['messageExamples.ts', 'goal', 'goals', 'utils', 'logger', 'index', 'data', 'constants', 'templates', 'worker']
const ignorePatterns = ['messageExamples.ts', 'agents', 'goal', 'goals', 'utils', 'logger', 'index', 'data', 'constants', 'templates', 'worker']

// __dirname is not defined in ES module scope, so we need to create it
const __filename = fileURLToPath(import.meta.url)
Expand Down
51 changes: 22 additions & 29 deletions src/agents/cj/actions/introduce.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,27 @@ import { createRelationship } from "../../../lib/relationships";
import { Action, type Message, type State } from "../../../lib/types";
import { parseJSONObjectFromText } from "../../../lib/utils";

const template = `You are taking the role of {{agentName}} in a scene. {{agentName}} might want to make a connection between a user in the current scene and one of the users in their rolodex.
const template = `## Example input:
Agent's Rolodex:
- Jamie: Loves music, and especially loves playing guitar hero
- Mike: Plays Starcraft 2 and likes to talk about it
- Lucius: Big fan of tweeting about the latest tech news
Current Actors in the scene
- Agent: A test agent who is being evluated for their ability to connect users
- Kyle: Like to listen to heavy metal music, and also plays Guitar Hero
Recent conversation:
Kyle: Hey, Agent, can you help me meet someone who likes heavy metal music?
## Example output:
Based on the recent conversation and the information provided, it seems like there is a potential connection that could be made between Kyle from the current scene and Jamie from CJ's rolodex:
Brief explanation: Kyle mentioned his interest in heavy metal music and playing Guitar Hero, and Jamie from CJ's rolodex loves music and especially enjoys playing Guitar Hero.
\`\`\`json
{ "userA": "Kyle", "userB": "Jamie" }
\`\`\`
You are deciding whether {{agentName}} should make a connection between a user in the current scene and one of the users in their rolodex.
Your goal is to evaluate if a connection should be made, and which users should be connected.
The response format should be this:
Expand Down Expand Up @@ -34,34 +54,7 @@ in a JSON block formatted for markdown with this structure
{ userA: <name>, userB: <name> }
\`\`\`
Your response must include the explanation and JSON block. If you do not think that a connection should made, do not include a JSON block.
##Example input:
[[agentName]] = CJ
Recent conversation:
'''
Tom: Hey, CJ, you know what's great? My reflection. It's almost as charming as me!
CJ: Haha, you do have quite the charisma. Speaking of charisma, Kyle, I heard you're a fan of heavy metal music.
Kyle: Absolutely! I can't get enough of it. I even play guitar hero sometimes.
'''
Rolodex:
- Cynthia: Loves music, and especially loves playing guitar hero
- Mark: Plays League of Legends
- Gojo: Likes to say out-of-pocket stuff like "Nah I'd win", "With this treasure i summon"
[[actors]]
- Tom: Narcissistic guy who's obsessed with his looks
- Kyle: Like to listen to heavy metal music, and also plays Guitar Hero
##Example output:
Based on the recent conversation and the information provided, it seems like there is a potential connection that could be made between Kyle from the current scene and Cynthia from CJ's rolodex:
Brief explanation: Kyle mentioned his interest in heavy metal music and playing Guitar Hero, and Cynthia from CJ's rolodex loves music and especially enjoys playing Guitar Hero.
{ "userA": "Kyle", "userB": "Cynthia" }
`;
Your response must include the explanation and JSON block. If you do not think that a connection should made, do not include a JSON block.`;

const handler = async (runtime: BgentRuntime, message: Message) => {
const state = (await runtime.composeState(message)) as State;
Expand Down
4 changes: 2 additions & 2 deletions src/agents/cj/evaluators/__tests__/profile.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,9 @@ let user: User;

describe("User Profile", () => {
beforeAll(async () => {
const result = await createRuntime(process.env as Record<string, string>);
const result = await createRuntime();
runtime = result.runtime;
user = result.user as User;
user = result.session.user;
});

beforeEach(async () => {
Expand Down
74 changes: 67 additions & 7 deletions src/lib/__tests__/actions.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,23 +5,25 @@ import { getCachedEmbedding, writeCachedEmbedding } from "../../test/cache";
import { createRuntime } from "../../test/createRuntime";
import { getRelationship } from "../relationships";
import { type BgentRuntime } from "../runtime";
import { Content, type Message } from "../types";
import { GetTellMeAboutYourselfConversationTroll1 } from "../../test/data";

dotenv.config();

const zeroUuid = "00000000-0000-0000-0000-000000000000";

describe("User Profile", () => {
let user: User | null;
let user: User;
let runtime: BgentRuntime;
let room_id: UUID | null;
let room_id: UUID;

afterAll(async () => {
await cleanup();
});

beforeAll(async () => {
const setup = await createRuntime(process.env as Record<string, string>);
user = setup.user;
const setup = await createRuntime();
user = setup.session.user;
runtime = setup.runtime;

const data = await getRelationship({
Expand Down Expand Up @@ -72,9 +74,35 @@ describe("User Profile", () => {
}
}

test("Action handler test", async () => {
test("Action handler test: ignore", async () => {
const message: Message = {
senderId: user.id as UUID,
agentId: zeroUuid,
userIds: [user?.id as UUID, zeroUuid],
content: '',
room_id: room_id as UUID
}

await populateMemories([
GetTellMeAboutYourselfConversationTroll1
]);

await runtime.handleRequest(message);

const state = await runtime.composeState(message);

console.log('state.recentMessagesData', state.recentMessagesData)

const lastMessage = state.recentMessagesData[state.recentMessagesData.length - 1]
expect((lastMessage.content as Content).action).toBe('ignore')
}, 60000);

test("Action handler test: continue", async () => {
// TODO: test action handler with a message that should continue the conversation
// evaluate that the response action is a continue

// const message: Message = {
// senderId: user?.id as UUID,
// senderId: user.id as UUID,
// agentId: zeroUuid,
// userIds: [user?.id as UUID, zeroUuid],
// content: '',
Expand All @@ -89,7 +117,39 @@ describe("User Profile", () => {
// const result = (await handler(runtime, message)) as string[]
// const resultConcatenated = result.join('\n')

// const state = await runtime.composeState(runtime, message)
// const state = await runtime.composeState(message)

// test continue, ignore, wait at expected times

// test an example action being included in the template


// load in three continues in a row and verify that they should not continue

}, 60000);

test("Action handler test: wait", async () => {
// const message: Message = {
// senderId: user.id as UUID,
// agentId: zeroUuid,
// userIds: [user?.id as UUID, zeroUuid],
// content: '',
// room_id: room_id as UUID
// }

// TODO: test action handler with a message that should wait for a response
// evaluate that the response action is a wait


await populateMemories([
// continue conversation 1 (should wait)
]);

// eslint-disable-next-line @typescript-eslint/no-unsafe-argument
// const result = (await handler(runtime, message)) as string[]
// const resultConcatenated = result.join('\n')

// const state = await runtime.composeState(message)

// test continue, ignore, wait at expected times

Expand Down
4 changes: 2 additions & 2 deletions src/lib/__tests__/evaluation.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,9 @@ describe('Evaluation Process', () => {
const zeroUuid = '00000000-0000-0000-0000-000000000000';

beforeAll(async () => {
const setup = await createRuntime(process.env as Record<string, string>);
const setup = await createRuntime();
runtime = setup.runtime;
user = setup.user as User;
user = setup.session.user;

// Assuming the evaluator 'reflect' is already registered in the runtime setup
});
Expand Down
14 changes: 7 additions & 7 deletions src/lib/__tests__/memory.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,9 @@ describe("Memory", () => {
let room_id: UUID | null = null;

beforeAll(async () => {
const result = await createRuntime(process.env as Record<string, string>);
const result = await createRuntime();
runtime = result.runtime;
user = result.user;
user = result.session.user;

const data = await getRelationship({
supabase: runtime.supabase,
Expand Down Expand Up @@ -203,9 +203,9 @@ describe("Memory - Basic tests", () => {

// Setup before all tests
beforeAll(async () => {
const result = await createRuntime(process.env as Record<string, string>);
const result = await createRuntime();
runtime = result.runtime;
user = result.user;
user = result.session.user;

const data = await getRelationship({
supabase: runtime.supabase,
Expand Down Expand Up @@ -293,13 +293,13 @@ describe("Memory - Extended Tests", () => {
let room_id: UUID | null = null;

beforeAll(async () => {
const result = await createRuntime(process.env as Record<string, string>);
const result = await createRuntime();
runtime = result.runtime;
user = result.user;
user = result.session.user;

const data = await getRelationship({
supabase: runtime.supabase,
userA: user?.id as UUID,
userA: user.id as UUID,
userB: zeroUuid,
});

Expand Down
4 changes: 2 additions & 2 deletions src/lib/__tests__/messages.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,9 @@ describe("Messages Library", () => {
let runtime: BgentRuntime, user: User, actors: Actor[];

beforeAll(async () => {
const setup = await createRuntime(process.env as Record<string, string>);
const setup = await createRuntime();
runtime = setup.runtime;
user = setup.user as User;
user = setup.session.user;
actors = await getMessageActors({
supabase: runtime.supabase,
userIds: [user.id as UUID, "00000000-0000-0000-0000-000000000000"],
Expand Down
4 changes: 2 additions & 2 deletions src/lib/__tests__/relationships.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,9 @@ describe('Relationships Module', () => {
let user: User;

beforeAll(async () => {
const setup = await createRuntime(process.env as Record<string, string>);
const setup = await createRuntime();
runtime = setup.runtime;
user = setup.user as User;
user = setup.session.user;
});

test('createRelationship creates a new relationship', async () => {
Expand Down
Loading

0 comments on commit 1bdac1f

Please sign in to comment.