One question about running inference

[Li-private]

Thanks for your great work， but due to my poor knowledge， what's the 'hf_token' in your code of inference? How can I get it?

[yunbinmo]

You can get a token here https://huggingface.co/settings/tokens and put it in a hf_token.txt and then replace .hf_token in the code with that.

[Li-private]

You can get a token here https://huggingface.co/settings/tokens and put it in a hf_token.txt and then replace .hf_token in the code with that.

Thank you very much，I had tried to run inference with llama2+7b and I have 1 A100 with 80GB Memory，however when I used the inference code on GitHub I've noticed that loading the pre-trained weights for SigLIP, DINOv2, llama2-7b, and this model is quite time-consuming, especially the weights for llama2-7b. Do you have any good solutions?

[yunbinmo]

You can get a token here https://huggingface.co/settings/tokens and put it in a hf_token.txt and then replace .hf_token in the code with that.

Thank you very much，I had tried to run inference with llama2+7b and I have 1 A100 with 80GB Memory，however when I used the inference code on GitHub I've noticed that loading the pre-trained weights for SigLIP, DINOv2, llama2-7b, and this model is quite time-consuming, especially the weights for llama2-7b. Do you have any good solutions?

I noticed that too, I am using RTX3090 with 24GB memory, pretty slow for me too.

[Li-private]

You can get a token here https://huggingface.co/settings/tokens and put it in a hf_token.txt and then replace .hf_token in the code with that.

Thank you very much，I had tried to run inference with llama2+7b and I have 1 A100 with 80GB Memory，however when I used the inference code on GitHub I've noticed that loading the pre-trained weights for SigLIP, DINOv2, llama2-7b, and this model is quite time-consuming, especially the weights for llama2-7b. Do you have any good solutions?

I noticed that too, I am using RTX3090 with 24GB memory, pretty slow for me too.

It's so sad.