You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
feat(js): Add current repetition property to jestlike test wrapper (#1856)
### Problem
When running tests with multiple repetitions using LangSmith's
Jest/Vitest test wrapper, there's no way to get the current repetition
number.
### Solution
This PR adds an `repetition` property that gets passed to the test
function, providing the current repetition index
### Use Case
For multi-turn conversations, each repetition needs its own unique
threadId that remains consistent throughout all tests in that
conversation.
```javascripts
const ConversationTestDataset = [{
inputs: {
userId: "test-user-123",
message: "What's the weather like today?",
conversationId: "weather-chat",
},
referenceOutputs: {
responseType: "weather_info",
requiresLocation: true,
},
},
{
inputs: {
userId: "test-user-123",
message: "How about tomorrow?",
conversationId: "weather-chat",
},
referenceOutputs: {
responseType: "forecast",
},
},
];
ls.describe("Conversation Flow Tests", () => {
ls.test.each(ConversationTestDataset, {
repetitions: 5, // Run each test case 5 times
})(
"Multi-turn conversation",
async ({ inputs, referenceOutputs, repetition }) => {
// Now we can create unique thread IDs for each repetition
const threadId = `${inputs.conversationId}-run-${repetition}`;
const response = await conversationAgent({
...inputs,
threadId,
});
// Validate response matches expected outputs
expect(response.type).toBe(referenceOutputs.responseType);
// Log outputs with repetition context
ls.logOutputs({
response: response.content,
threadId,
repetitionIndex: repetition
});
}
);
});
```
---------
Co-authored-by: jacoblee93 <[email protected]>
0 commit comments