[BUG] lmformatenforcer integration seems to be broken on new versions

[hvico]

OS

Linux

GPU Library

CUDA 12.x

Python version

3.10

Pytorch version

2.5

Model

No response

Describe the bug

When running lmformatenformatenforcer integration for JSON output I get:

Error during execution: 'ExLlamaV2TokenEnforcerFilter' object has no attribute 'background_drop'

and if I comment that call in the generator code it fails with another missing stuff related with logit mask.

Reproduction steps

Try to enforce json output on v0.2.6

Expected behavior

Integration works.

Logs

No response

Additional context

No response

Acknowledgements

• I have looked for similar issues before submitting this one.
• I understand that the developers have lives and my issue will be answered when possible.
• I understand the developers of this program are human, and I will ask my questions politely.

[turboderp]

Problem with LMFE is that it implements the filter class that ExLlamaV2 needs, but it doesn't derive from the base class in ExLlamaV2. You can use the filter class from here instead until it is fixed in LMFE.

[hvico]

Thanks for the workaround!