Trying to make BF16 / GGUF
#1
by
Phr00t
- opened
Any trick to getting this into a GGUF?
Would you be able to provide a BF16 GGUF (then I can make quants myself)?
Getting this error when I try it myself: https://github.com/ggml-org/llama.cpp/issues/18065
Looks like an error in your files:
https://github.com/ggml-org/llama.cpp/issues/18065#issuecomment-3657509286
Yeah, I will need some file from original model here, it seems pushing directly from heretic doesn’t adds them.
I used transformers 5.0 and that seems to be causing this, I will update the model with transformer 4.x compatibility.