Concept
Model Evaluation
A curated set of test cases and golden answers used to measure whether a model reliably succeeds at a specific behavior, used to define, track, and validate improvements.
Model Evaluation
8 visible of 9 connections
Related Concepts5
Related Practices15
Ask about this concept
Transcript mentions
And then that actually is your eval set. And you could have essentially uh a prompt and a response.
… essentially teach AI models what's good and what's bad. So we train them using human data, and there's a lot of different products that we have, like SFT, RHF, rubrics, verifiers, RL environments, and so on and so on, and then we also…
… back to the types of post-training, just because I think this might be helpful, at least for me the mental model of there's pre-training, there's post-training, within post-training there's reinforcement learning, human feedback, there's…
Concept
Model Evaluation
A curated set of test cases and golden answers used to measure whether a model reliably succeeds at a specific behavior, used to define, track, and validate improvements.
Related Concepts5
Related Practices15
Ask about this concept
Transcript mentions
And then that actually is your eval set. And you could have essentially uh a prompt and a response.
… essentially teach AI models what's good and what's bad. So we train them using human data, and there's a lot of different products that we have, like SFT, RHF, rubrics, verifiers, RL environments, and so on and so on, and then we also…
… back to the types of post-training, just because I think this might be helpful, at least for me the mental model of there's pre-training, there's post-training, within post-training there's reinforcement learning, human feedback, there's…