A prompt edit shipped on a Friday renamed one key in a JSON output contract. Nothing threw. The pipeline ran, produced structured output, and the consumer wrote nulls into a database for eight hours before a scheduled report failed. Reverting the three-word edit meant a file change, a pull request, and a deployment. The fix for that is not a better review process, it is a commit model: an immutable snapshot for every prompt change and a mutable pointer that decides which snapshot production runs. This post covers how that model works, the code that actually exists to drive it, and why the question of where the registry lives is genuinely unsettled in 2026.
What changed since this was first written. The rollback and CI code no longer match the shipped SDK, and the framing has moved: OpenAI is retiring its hosted prompt objects while Anthropic shipped server-side immutable versions. Everything below is rebuilt against current documentation.
Why a hardcoded prompt has no rollback path
Teams hold a production prompt three ways, in increasing order of false confidence. A hardcoded string makes every edit a deployment, and nothing signals a broken output contract because the pipeline still runs. Reviewers read logic; almost nobody reads a multi-paragraph system prompt for semantic drift in a diff.
An environment variable only moves the problem. The version history becomes the environment’s history: no diff, no author, and a rollback that means sequencing an env change against a deployment.
A database row is real progress, because you can change the prompt without deploying. What is missing is a commit model. A row can be overwritten, so you know the current value but not what it was an hour ago or whether anyone approved it.
What all three lack is the thing version control gave code: an immutable snapshot per change, and a mutable pointer that moves between snapshots. Promoting means moving a pointer, not overwriting a value, and that is what every serious provider has now converged on.
The commit model, and the two identifiers it gives you
In LangSmith a prompt is not a string but a named artifact holding a full ChatPromptTemplate, with message roles, input variables and format markers. Each push creates a commit addressed by a hash, and old commits are never overwritten.
Tags are mutable pointers to commits. document-analyzer:staging and document-analyzer:prod can point at the same commit or different ones. LangChain’s documentation puts the payoff plainly: “Instead of using commit hashes directly, you can reference tags that can be updated without changing your code” (LangChain, “Manage prompts”).
That gives two identifiers with opposite properties, and the mistake is using the wrong one:
- A tag in application code, so a promotion reaches running processes without a release.
- A commit hash in evaluation pipelines, where the prompt must not change underneath an hour-long run.
Pin to a tag in the application, pin to a hash in the eval. A tag that moves mid-evaluation quietly invalidates the results, and a hash in application code reintroduces the deployment you were trying to remove.
Pushing and pulling a versioned prompt
Pushing, tagging the commit as it is created:
from langsmith import Client
from langchain_core.prompts import ChatPromptTemplate
client = Client()
prompt = ChatPromptTemplate.from_messages([
("system", "You are a senior legal analyst. Extract "
"obligations, deadlines and risk factors. "
"Return keys: obligations, deadlines, risks."),
("human", "{document_text}"),
])
client.push_prompt(
"document-analyzer",
object=prompt,
commit_tags=["staging"],
)commit_tags tags the commit this call creates. It is distinct from tags, which sets metadata on the prompt itself, and confusing the two is the most common way a promotion silently does nothing.
Pulling at runtime is one call, and the tag is the only identifier the application knows.
from langsmith import Client
client = Client()
prompt = client.pull_prompt("your-org/document-analyzer:prod")
chain = prompt | model
result = chain.invoke({"document_text": contract_text})If part of your stack skips LangChain, convert_prompt_to_anthropic_format and convert_prompt_to_openai_format deserialize a Hub prompt into the provider’s native shape. Note the argument: both take an already-formatted prompt value, not the template plus a dict of inputs.
import os, anthropic
from langsmith.client import convert_prompt_to_anthropic_format
prompt_value = prompt.invoke({"document_text": contract_text})
payload = convert_prompt_to_anthropic_format(prompt_value)
response = anthropic.Anthropic().messages.create(
model=os.environ["ANTHROPIC_MODEL"],
max_tokens=1024,
**payload,
)The model id lives in configuration, not in the prompt and not in an article. Provider model tables change faster than any blog post.
Rollback is a forward commit, not a pointer rewind
Here the theory and the shipped SDK diverge, and the wrong version of this code fails at runtime.
There is no public SDK method that moves an existing tag to an existing commit. LangChain documents tag movement as a console operation: select the destination commit, click Tag, choose the tag, and “this automatically updates the tag to point to the new commit”. The Python client’s update_prompt only edits prompt metadata, and its own docstring says so: “To update the content of a prompt, use push_prompt or create_commit instead.” Its tags argument sets prompt-level metadata tags, not commit tags.
So a programmatic rollback is not a rewind. List the commits, pull the last good manifest, and push it again as a new commit carrying the prod tag:
from langsmith import Client
client = Client()
REPO = "your-org/document-analyzer"
for c in client.list_prompt_commits(REPO, limit=5):
print(c.commit_hash, c.created_at)
good = client.pull_prompt(f"{REPO}:<stable-hash>")
client.push_prompt(
"document-analyzer",
object=good,
commit_tags=["prod"],
)The method is list_prompt_commits, not list_commits, and it paginates in pages of 100. Re-pushing is closer to git revert than git reset, which is better behaviour anyway: the rollback becomes an auditable commit with an author and a timestamp, rather than a pointer that quietly moved.
The cache is why runtime updates are affordable, and why they are not instant
Without a cache every inference call would need a network round trip. The client ships one, and its defaults set your rollback window, so they are worth knowing rather than guessing.
From the SDK’s prompt_cache module: 100 entries with LRU eviction, staleness after 300 seconds, and a background thread sweeping for stale entries every 60 seconds. Stale entries are still served while the refresh runs, the stale-while-revalidate pattern CDNs use. The calling thread never blocks after the first fetch.
So the window is five to six minutes, not five exactly, because staleness and the sweep are different clocks. Design an SLA around that rather than discover it during an incident. Three levers retune it, none needing a redeploy:
from langsmith import Client, configure_global_prompt_cache
configure_global_prompt_cache(max_size=200, ttl_seconds=60)
client = Client()
client.pull_prompt(
"your-org/document-analyzer:prod", skip_cache=True
)
configure_global_prompt_cache retunes size and TTL, skip_cache=True forces one fresh read, and the cache exposes invalidate(key) and clear() for an explicit cutover. Advice to build your own invalidation endpoint or restart the process is out of date: those primitives ship in the client.
One caveat is genuinely cross-process: pods refresh at different offsets inside the window, so a promotion rolls out staggered rather than simultaneously. When drift across pods is unacceptable, pull by commit hash during the transition, then switch back to the tag.
A pulled prompt is executable configuration, not text
This part has the real blast radius. A manifest carries serialized LangChain objects and model configuration, and the SDK’s warning is direct: it can “intentionally configure a model with a custom base URL, headers, model name, or other constructor arguments”, and “the prompt contents should be treated as executable configuration rather than plain text.” That is why pulling a public prompt by owner and name now requires an explicit dangerously_pull_public_prompt=True, and why the docs add that a prompt from your own organisation “can still be unsafe if that account or prompt was compromised.”
Treat the registry as a code path. Anyone who can push a commit can, in principle, redirect your inference traffic. Restrict write access, review manifests the way you review a dependency bump, and point the instincts you reserve for prompt injection at your own supply chain. This is LLMOps with the security posture of package management, because structurally that is what it is.
Gating a promotion on a webhook
LangSmith fires an outbound webhook on commit: the hook starts an evaluation pinned to the commit hash, and a pass rate over a threshold promotes.
The field names matter, because the obvious guesses are wrong. The payload carries exactly six fields: prompt_id, prompt_name, commit_hash, created_at, created_by and manifest. There is no tags field and no repo_name, so a handler that gates on payload["tags"] never runs, and one that reads payload["repo_name"] raises a KeyError on the first commit.
from fastapi import FastAPI, Request
app = FastAPI()
@app.post("/hooks/langsmith-prompt")
async def on_commit(request: Request) -> dict:
p = await request.json()
name, sha = p["prompt_name"], p["commit_hash"]
# Pinned to the hash, so the eval is reproducible
rate = await run_eval_suite(name, commit_hash=sha)
if rate < 0.90:
return {"status": "blocked", "pass_rate": rate}
await request_promotion(name, sha, rate)
return {"status": "queued", "pass_rate": rate}With no tag information in the payload, staging and production have to be separate prompt names for the hook to tell them apart, and the promotion itself is still a console action or a re-push.
Most teams need none of this on day one. Manual tag promotion alone removes the deployment from the rollback path, which is most of the pain. Add the webhook gate once you have a stable evaluation set. Ten representative cases at a 90% threshold turn the release check into prompt regression testing that catches output-contract regressions far more reliably than a human reading a diff. Same argument as deterministic confidence gates: gate on a signal you can measure.
Where the registry should live is the unsettled question
What changed between this post being written and now is not a detail.
OpenAI is retiring hosted prompt objects. Reusable prompt objects and the v1/prompts endpoint are de-emphasised from 3 June 2026 and shut down on 30 November 2026. Their migration guidance is to “move the prompt content out of the managed prompt object and into your application code”, because “This gives you more control over review, testing, deployment, and versioning” (OpenAI, “Migrate from prompt objects”).
Anthropic moved the other way. With Managed Agents the prompt stays server-side, and “Every agents.update produces a new immutable version, and sessions choose which version to use by ID.” Rollback is exactly the pointer move described above: “Rolling back isn’t a deploy; callers just go back to passing version: 1” (Anthropic, “Prompt versioning and rollback”).
LangSmith kept its hosted registry and hardened it, adding cache controls, the skip_cache escape hatch and a trust boundary on public pulls.
The honest read is not that one of these is right. It is that the commit model is settled and the registry is not. Every option gives you immutable versions and a pointer; they disagree about whether the artifact belongs in your repository or on someone’s server, and one major vendor changed its mind inside a single year.
The engineering conclusion follows: depend on the commit model, never on a particular registry. Keep the prompt’s source of truth somewhere you can export, keep the pointer indirection behind one function in your code, and make sure you could switch the backing store in an afternoon. A team that wrapped OpenAI’s prompt objects behind a get_prompt() helper has a migration. A team that scattered prompt_id across forty call sites has a project.
The takeaway
- Immutable snapshot, mutable pointer. The whole model, and the part that will not change.
- Tag in the application, hash in the eval. Reverse them and you either reintroduce the deployment or invalidate the evaluation.
- Rollback is a re-push, not a rewind. No public SDK call moves a tag, so commit the good manifest forward.
- The cache sets your rollback SLA. 300-second staleness plus a 60-second sweep, with
invalidateandskip_cachewhen that is too slow. - A manifest is executable configuration. Guard write access like a dependency.
- The registry is swappable, so keep it that way. One vendor is retiring theirs in 2026.
At Soba Labs that is how we run LangSmith in production: one accessor function, tags in the application, hashes in the evaluation harness, and a rollback rehearsed before it is needed. The related economics are in prompt caching across Claude, GPT and Gemini, and reading model limits at runtime instead of hardcoding them is in LangChain model profiles for context management.
Sources
- LangChain, “Manage prompts”: commit tags as mutable pointers, moving a tag in the console, and the six webhook payload fields
- LangChain, “Manage prompts programmatically”: push and pull by tag or commit hash, and the Anthropic and OpenAI format converters
- LangSmith Python SDK: the shipped signatures for
push_prompt,update_promptandlist_prompt_commits, the prompt-cache defaults, and the executable-configuration warning on public pulls - OpenAI, “Migrate from prompt objects”: the de-emphasis and shutdown dates for hosted prompt objects and the move back to code-managed prompts
- Anthropic, “Prompt versioning and rollback”: immutable versions from
agents.updateand rollback by pointing callers at an earlier version
Last updated: