how to use few-shot?

[xingchen0120]

Hello,I want to use the few-shot method to assist LLAMA in inference. May I ask how the input format should be set?

[adi-ydv-1]

from transformers import LLaMAForCausalLM, LLaMATokenizer

Initialize model and tokenizer

model = LLaMAForCausalLM.from_pretrained("llama-7b")
tokenizer = LLaMATokenizer.from_pretrained("llama-7b")

Define prompt, few-shot examples, and query

prompt = "Translate English to Spanish."
examples = [
"English: Hello, how are you? Spanish: Hola, ¿cómo estás?",
"English: What's your name? Spanish: ¿Cuál es tu nombre?",
]
query = "Hello, I'm John."

Format input and tokenize

input_text = f"{prompt}\n{'\n'.join(examples)}\n{query}"
inputs = tokenizer(input_text, return_tensors="pt")

Generate output

output = model.generate(**inputs)

Print output

print(tokenizer.decode(output[0], skip_special_tokens=True))