A RAG prompt fails in two quiet ways. The model answers from memory because the “use only this context” rule was mixed into the user message and then edited away. Or the call fails because the retrieved chunk, not the question, blew the window.
You can assemble and check that prompt locally: rules in one place, context in another, tokens visible before any request. Prompt Workbox does not retrieve documents and does not call your model. It is the workbench around the prompt you are about to send.
Put rules and context in different messages
Start from the built-in RAG system template on Templates. It is already ChatML.
System:
You are a retrieval-augmented assistant for {{domain}}.
Rules:
- Answer ONLY using the provided CONTEXT
- If CONTEXT is insufficient, say what is missing
- Cite chunk ids like [chunk:N] when you use a fact
- Prefer concise, actionable answers in {{language}}
User:
CONTEXT:
{{context}}
QUESTION:
{{question}}
Keep those roles. If the rules slide into the user message, the next person to “just append another chunk” will edit the policy by accident. The ChatML editor is where you adjust the cards after the template fill-in.
{{domain}} and {{language}} change rarely. {{context}} and {{question}} change every call. That split is what makes a template reusable instead of a one-off paste.
Prepare the chunk before it enters the prompt
Retrieved text is often Markdown from a wiki, a README, or a PDF extract. Clean it before it becomes {{context}}.
- MDWorkbox is the place to tidy source Markdown when the raw extract is noisy
- Clean strips leftover markup, HTML, URLs, emoji, and signature lines, and can escape the result for a JSON body
- Do not compress the chunk and the system rules in one pass. Compress is for token savings. Diff the result afterward so a heading did not fuse with a rule
When you need chunk boundaries and a finer token count than the editor’s estimate, send the context to TokenWorkbox. Bring the pieces back as the {{context}} value. Prompt Workbox does not chunk for you.
See the budget with the chunk included
Open the filled prompt in the editor, ChatML mode, and select the model you will actually call.
- The system card should stay small and stable
- The user card should jump when you paste a long chunk
- Amber at about 70% of the window means the next chunk will hurt
- Red at about 90% or over the limit means trim the context, not the citation rule
OpenAI models are counted exactly. Other vendors show an estimate until you Send to TokenWorkbox. Details are in Check prompt tokens.
If you are comparing “cite chunk ids” against “quote a short span”, put the two system cards on A/B Compare with the same model and the same user context. Otherwise you are comparing tokenizers, not instructions.
Batch the questions, freeze the rules
On Variable batch test, keep one template and several rows:
| domain | language | context | question |
|---|---|---|---|
| billing | English | [chunk:1] Refunds take 5 business days. |
How long is a refund? |
| billing | English | [chunk:1] Refunds take 5 business days. |
What is the enterprise SLA? |
| billing | English | (empty) | How long is a refund? |
Row 1 should answer from the chunk and cite [chunk:1]. Row 2 should say the context is insufficient, not invent an SLA. Row 3 should refuse rather than answer from the empty string. If row 2 answers anyway, the system rule is too soft — change the template once, not the row.
Lint after you fill a row. It reports unresolved {{context}} placeholders, a missing system message, empty cards, and JSON fences that will not parse. Ship the row that lint accepts.
Snapshot the release prompt
Save three snapshots on Versions if this prompt will live for a while:
- Draft — template still full of placeholders
- Slim — after the chunk was cleaned, rules untouched
- Release — one real question filled, under the context bar, lint clean
Diff Draft and Slim before you trust the cleaner. Diff Slim and Release only to confirm the question and context were the only insertions.
Export ChatML JSON from the editor when Release looks right. That JSON is what you paste into the API. The template with {{context}} stays in the library for the next question.
The workbench will not embed your corpus or log the calls. Retrieval, the model request, and production traces stay in your stack. The part you can finish offline is the prompt shape, the variable batch, and the token ceiling.