displaying context and token stats in real time

[NoahOksuz]

Add parsing progress display to webui

Fixes #17079

Adds parsing progress during prompt tokenization, similar to token generation progress.

Changes:

• Display Input: processed / total (percent%) during prompt parsing
• Display parsing tokens/second (X.X t/s) during parsing
• Calculate parsing speed from prompt_progress timing data

Implementation:

• Added parsingTokensPerSecond, inputTokensProcessed, and inputTokensTotal to ApiProcessingState
• Updated parseCompletionTimingData() to calculate parsing speed from prompt_progress.time_ms and prompt_progress.processed
• Updated getProcessingDetails() to show input parsing progress when status is 'preparing'

Users now see real-time parsing progress instead of just "Processing...".

[catap]

@NoahOksuz I can't figure out how to enable it. I see:

[NoahOksuz]

@NoahOksuz I can't figure out how to enable it. I see:

works locally. im just working on another PR atm ill take a second look in 5 minutes at this

[catap]

aha, I just patched sources of used build. Which seems not enough :(

If you share a picture how it looks on your side, I can genuely how I think it looks.

[gSUz92nc]

I think you need to enable progress updates from the backend. Setting return_progress to true in the request body should enable it.

// tools/server/webui/src/lib/services/chat.ts   chat.ts  ~ line 117
const requestBody: ApiChatCompletionRequest = {
	messages: processedMessages.map((msg: ApiChatMessageData) => ({
		role: msg.role,
		content: msg.content
	})),
	stream
};

should be

const requestBody: ApiChatCompletionRequest = {
	messages: processedMessages.map((msg: ApiChatMessageData) => ({
		role: msg.role,
		content: msg.content
	})),
	stream,
	return_progress: true // Add this
};

Then it works for me

[allozaur]

Hey @NoahOksuz, thanks a lot for this contribution. Found one minor improvement to add, also please make sure that npm run lint && npm run check succeeds locally on your end and passes the checks in GH Actions.

Lastly, please provide the short testing instruction on how this improvement can be reproduced consistently, it's a bit unclear how it can be tested reliably at the moment.

[allozaur]

Suggested change

	const parsingTokensPerSecond = promptProgress && promptProgress.time_ms > 0
	? (promptProgress.processed / promptProgress.time_ms) * 1000
	: undefined;
	const parsingTokensPerSecond = promptProgress && promptProgress.time_ms > 0 && promptProgress.processed > 0
	? (promptProgress.processed / promptProgress.time_ms) * 1000
	: undefined;