Add support for Muse Glimmer and its tool format

[neilmehta24]

• Update mlx-vlm pin to add support for Muse Glimmer
• Add support for glimmer's tool calling grammar, in order to constrain invalid logits during tool call generation
• Update the batch generator to eval the prefill arrays before moving on to decode, in order to fix the prefill/decode latency attribution. Previously, the final prefill work was being tracked as the first decode token latency, which misattributes the latency profiles. E2E request completion time is unaffected.

Performance on M2 Mac Studio:

• Decode: 31.5 tok/s at low context; 29 tok/s at 8k context length
• Prefill: 175 tok/s for 8k context

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d3cdc582c1

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 691175209d

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".