We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Linux
CUDA 12.x
3.10
2.5
No response
When running lmformatenformatenforcer integration for JSON output I get:
Error during execution: 'ExLlamaV2TokenEnforcerFilter' object has no attribute 'background_drop'
and if I comment that call in the generator code it fails with another missing stuff related with logit mask.
Try to enforce json output on v0.2.6
Integration works.
The text was updated successfully, but these errors were encountered:
No branches or pull requests
OS
Linux
GPU Library
CUDA 12.x
Python version
3.10
Pytorch version
2.5
Model
No response
Describe the bug
When running lmformatenformatenforcer integration for JSON output I get:
Error during execution: 'ExLlamaV2TokenEnforcerFilter' object has no attribute 'background_drop'
and if I comment that call in the generator code it fails with another missing stuff related with logit mask.
Reproduction steps
Try to enforce json output on v0.2.6
Expected behavior
Integration works.
Logs
No response
Additional context
No response
Acknowledgements
The text was updated successfully, but these errors were encountered: