Practice
Placing additional information at prompt beginning
Putting context at the start of prompts for caching benefits and to prevent task confusion
Placing additional information at prompt beginning
1 visible of 1 connections
Context
Related Frameworks2
Transcript mentions
So subsequent calls to the LM with that same context at the top of the prompt are cheaper because the model provider stores that initial context for you as well as the embeddings for it. So it saves a ton of computation from being done.
Practice
Placing additional information at prompt beginning
Putting context at the start of prompts for caching benefits and to prevent task confusion
Context
Related Frameworks2
Transcript mentions
So subsequent calls to the LM with that same context at the top of the prompt are cheaper because the model provider stores that initial context for you as well as the embeddings for it. So it saves a ton of computation from being done.