Load model as torch.bfloat16

#2
by martin-q-ma - opened

Hi Authors,

Thank you very much for releasing the code.

In Line 14 of the example inference code, should it be:

model = AutoModel.from_pretrained(model_path, trust_remote_code=True).half().cuda().to(torch.bfloat16)

instead of

model = AutoModel.from_pretrained(model_path, trust_remote_code=True).half().cuda()

According to https://huggingface.co/OpenGVLab/InternVideo2_5_Chat_8B?

Thanks!

Sign up or log in to comment