Load model as torch.bfloat16
#2
by
martin-q-ma
- opened
Hi Authors,
Thank you very much for releasing the code.
In Line 14 of the example inference code, should it be:
model = AutoModel.from_pretrained(model_path, trust_remote_code=True).half().cuda().to(torch.bfloat16)
instead of
model = AutoModel.from_pretrained(model_path, trust_remote_code=True).half().cuda()
According to https://huggingface.co/OpenGVLab/InternVideo2_5_Chat_8B?
Thanks!