New
You can now define a "Conversation History" column in your test datasets to include prior multi-turn interactions between the user and LLM alongside your "Input" while running prompt tests.
Passing chat history to the LLM provides critical context, enabling it to understand the ongoing dialogue rather than treating each input as an isolated query. This allows you to simulate real-world interactions more accurately. Here’s how it works:
- The Conversation History is sent to the LLM in sequence with the Prompt version and the Input column data.
- The history must be formatted as a JSON array containing messages with roles (user, assistant, or tool) and the corresponding content.
- The content can be either a simple string or a structured array supporting text and image attachments.
Read more about the Conversation History column.
