đŸȘŽ Anil's Garden

Llama 3 Model Cards and Prompt formats

02 Apr 20252 min read

Llama 3 | Model Cards and Prompt formats

Excerpt

Special Tokens used with Llama 3. A prompt should contain a single system message, can contain multiple alternating user and assistant messages, and always ends with the last user message followed by the assistant header.

Code to generate this prompt format can be found here.

Notes:

  • Newlines (0x0A) are part of the prompt format, for clarity in the examples, they have been represented as actual new lines.
  • The model expects the assistant header at the end of the prompt to start completing it.

Decomposing an example instruct prompt with a system message:

<|begin_of_text|><|start_header_id|>system<|end_header_id|>

You are a helpful AI assistant for travel tips and recommendations<|eot_id|><|start_header_id|>user<|end_header_id|>

What can you help me with?<|eot_id|><|start_header_id|>assistant<|end_header_id|>

<|begin_of_text|>: Specifies the start of the prompt

<|start_header_id|>system<|end_header_id|>: Specifies the role for the following message, i.e. “system”

You are a helpful AI assistant for travel tips and recommendations: The system message

<|eot_id|>: Specifies the end of the input message

<|start_header_id|>user<|end_header_id|>: Specifies the role for the following message i.e. “user”

What can you help me with?: The user message

<|start_header_id|>assistant<|end_header_id|>: Ends with the assistant header, to prompt the model to start generation.

Following this prompt, Llama 3 completes it by generating the {{assistant_message}}. It signals the end of the {{assistant_message}} by generating the <|eot_id|>.

Example prompt with a single user message

&lt;|begin_of_text|&gt;&lt;|start_header_id|&gt;user&lt;|end_header_id|&gt;

What is France's capital?&lt;|eot_id|&gt;&lt;|start_header_id|&gt;assistant&lt;|end_header_id|&gt;

System prompt and multiple turn conversation between the user and assistant

&lt;|begin_of_text|&gt;&lt;|start_header_id|&gt;system&lt;|end_header_id|&gt;

You are a helpful AI assistant for travel tips and recommendations&lt;|eot_id|&gt;&lt;|start_header_id|&gt;user&lt;|end_header_id|&gt;

What is France's capital?&lt;|eot_id|&gt;&lt;|start_header_id|&gt;assistant&lt;|end_header_id|&gt;

Bonjour! The capital of France is Paris!&lt;|eot_id|&gt;&lt;|start_header_id|&gt;user&lt;|end_header_id|&gt;

What can I do there?&lt;|eot_id|&gt;&lt;|start_header_id|&gt;assistant&lt;|end_header_id|&gt;

Paris, the City of Light, offers a romantic getaway with must-see attractions like the Eiffel Tower and Louvre Museum, romantic experiences like river cruises and charming neighborhoods, and delicious food and drink options, with helpful tips for making the most of your trip.&lt;|eot_id|&gt;&lt;|start_header_id|&gt;user&lt;|end_header_id|&gt;

Give me a detailed list of the attractions I should visit, and time it takes in each one, to plan my trip accordingly.&lt;|eot_id|&gt;&lt;|start_header_id|&gt;assistant&lt;|end_header_id|&gt;

Graph View

Backlinks