Counting Input and Output Tokens from an OpenAI Chat Engine

[evanwechsler]

I am building a Next.js application largely based on the default project generated by create-llama. I need to count the input and output tokens for each chat for tracking and cost estimation. I managed to do this in the past by using vercel's streamText function directly with the onFinish callback but the usage output is now giving me undefined values.

Ideally i would like to just use the llamaindex context chat engine directly and save the tokens once the stream has completed with some callback. I believe there is a token counting handler in the python version of this package but I haven't been able to find anything similar in the ts version (maybe its yet to be implemented).

I can probably hack together as solution to count the tokens but it would be nice to use the usage data directly provided by openai for more accuracy.

Any advice would be greatly appreciated.

[evanwechsler]

Nevermind... Found the answer here.