← All posts

Compare prompt versions locally with snapshots, diffs, and A/B

Prompt edits are small and easy to lose. You tighten a system rule, compress whitespace, then cannot remember which sentences the compress step deleted. A version list with a diff fixes that without a team workspace or a cloud history.

Version snapshots and A/B Compare both stay in this browser. They answer different questions.

Snapshots: what changed since last time

Snapshots are taken from the editor draft. Edit on the editor first — the draft autosaves — then open Versions.

Each save stores:

  • A title you choose (for example System v2 after trim)
  • A label: Draft, Slim, Release, or Custom
  • The time and the token count for the model selected in the editor
  • The full text, so you can open that snapshot in the editor again

Labels are a convention, not a workflow engine:

Label Use it for
Draft First working copy, placeholders allowed
Slim After a clean or a deliberate trim
Release The wording you are about to paste into an API
Custom Anything else (a client name, a ticket id)

Pick two snapshots to see a line diff. Unchanged lines are counted; added and removed lines are marked. If a compress pass merged a heading into the previous paragraph, the diff shows it before you ship.

Open in Editor writes that snapshot back into the live draft. It does not upload anything. Deleting a snapshot only removes it from this browser’s localStorage.

Token counts on the card are from save time and the model in use then. If you change the target model later, save again before you compare budgets. How those counts relate to the 70% and 90% warnings is covered in Check prompt tokens.

A/B: which wording is shorter, side by side

A diff is for two moments of the same draft. A/B is for two candidates you want to judge together.

A/B Compare gives you two panes, A and B:

  • One shared target model, so the token bars use the same encoding and the same context window
  • Independent text or ChatML editing on each side
  • Swap A ↔ B when you want the other wording in the left pane
  • Editor → A or Editor → B to pull the current editor draft in
  • A → Editor or B → Editor to send the winner back
  • Copy ChatML or download per side

Compare keeps its own local draft. It does not silently share keystrokes with the editor. Transfer with those buttons on purpose.

Fair token comparison needs the shared model. If A is counted as GPT-4o and B as an estimate for another vendor, the bars are not a decision.

Text mode exports as one user message. ChatML mode exports the message array. Match the mode to the API you will call — the ChatML editor note on framing overhead applies to both panes.

A sequence that stays reviewable

  1. Save the first complete prompt as Draft.
  2. Run Clean if you are stripping markup or compressing, paste the result back, and save Slim.
  3. Diff Draft against Slim. Restore any sentence the cleaner should not have dropped.
  4. If you are choosing between two system prompts, load them into A and B. Decide using the token bars and a read-through, then send the winner to the editor.
  5. Fill any {{variables}} — or batch them on Variables — and save Release.
  6. Run Lint on the release text. Unresolved placeholders and a missing system message are cheaper to catch here than in production.

For RAG, snapshot the system rules separately from experiments on the chunk. Changing both in one save makes the diff useless. The RAG walkthrough keeps those pieces in different messages for that reason.

None of this is a collaboration product. There is no shared timeline and no login. If you need the release wording on another machine, export the text or ChatML JSON and keep that file yourself.