LangChain model profiles for context management

We had a summarization middleware that triggered at 8,000 tokens, and it worked fine for months until it did not. The bug was never the number. It was that a number was there at all: a hardcoded threshold is a model-specific assumption wearing the costume of a constant. LangChain’s .profile attribute fixes this by exposing the active model’s real capabilities at runtime, so context engineering adapts to the model in use instead of the model you happened to be using the day you wrote the code. This post covers what .profile exposes, how to drive dynamic summarization from it, how to gate capabilities on it, and what to do when the registry data is wrong.

The logic was simple. Once the conversation grew past that threshold, compress the older messages before sending the next request. Then we switched one workflow to Claude Sonnet 4.5, which carries a 200k context window, and it started summarizing conversations that were barely getting started. We would hit 8,000 tokens on turn four of a ten-turn research session, collapse half the context, and wonder why the agent kept forgetting what the user had said two minutes earlier.

What we had actually done was encode one model’s comfort zone as a universal constant. Every time we swapped models after that, the assumption broke silently.

Why hardcoded context thresholds break when you swap models

The hardcoded-threshold pattern is everywhere. You will see it in production LangGraph graphs, in LangChain middleware, in RAG pipelines:

MAX_CONTEXT_MESSAGES = 20
SUMMARIZE_AFTER_TOKENS = 8_000

if len(messages) > MAX_CONTEXT_MESSAGES:
    messages = summarize_and_trim(messages)

The intent is reasonable: prevent context overflow, keep latency predictable. But those constants are secretly tied to whatever model you were using when you wrote them. When you move across the model landscape, they become a liability.

Consider what one 8,000-token trigger looks like across three common models:

• GPT-4o mini caps at 128,000 input tokens per OpenAI’s model documentation. An 8k trigger uses about 6% of capacity.
• Claude Sonnet 4.5 has a 200,000-token window per Anthropic’s context windows documentation, so an 8k trigger uses about 4% of capacity.
• Gemini 2.5 Flash supports 1,048,576 input tokens per Google’s model documentation. That same 8k trigger uses about 0.8% of what is available.

The obvious fix is to define per-model constants and branch on the model name. That scales poorly. Add five models, add five branches. Add a new provider, then remember to update every piece of middleware that makes a model assumption. Miss one and you get silent degradation, wrong context limits, disabled capabilities, wasted tokens, with no error to trace.

What you actually want is to ask the model what it can do, then write logic against the answer.

What model.profile exposes

LangChain’s .profile attribute is a dict of capability metadata that chat models can expose, and it requires langchain>=1.1. The data comes from models.dev, an open-source registry covering hundreds of models across the major providers.

from langchain.chat_models import init_chat_model

model = init_chat_model("claude-sonnet-4-5-20250929")
print(model.profile)
# {
#   "max_input_tokens": 200000,
#   "tool_calling": True,
#   "structured_output": True,
#   "image_inputs": True,
#   "reasoning_output": False,
#   ...
# }

That number is the reason to read it rather than remember it. Anthropic’s newer models ship a 1M-token window as the default, with no beta header and standard pricing, while Sonnet 4.5 stays at 200k. A constant written against one of them is wrong for the other, and nothing in your code will say so.

The fields that matter most at runtime, and what each one unlocks:

• max_input_tokens (int): drive dynamic summarization thresholds.
• tool_calling (bool): gate tool calling registration and invocation.
• structured_output (bool): gate .with_structured_output() calls, covered under structured output.
• image_inputs (bool): gate multimodal message construction.
• reasoning_output (bool): detect models that can return reasoning content.

LangChain merges the models.dev data with per-provider augmentations it maintains in each provider package, and you reach it through the same interface regardless of provider.

How to drive summarization from the model’s real window

The core use case is triggering summarization on model.profile["max_input_tokens"] rather than a constant. Pick a threshold fraction, where 80% is a reasonable default, and let the model’s actual window decide when to compress.

from typing import TypedDict

from langchain_core.messages import (
    BaseMessage,
    HumanMessage,
    SystemMessage,
)
from langchain_core.messages.utils import (
    count_tokens_approximately,
)


class AgentState(TypedDict):
    messages: list[BaseMessage]


def should_summarize(
    messages: list[BaseMessage],
    model,
    threshold: float = 0.8,
) -> bool:
    """Trigger at a fraction of the active model's window."""
    profile = getattr(model, "profile", None) or {}
    max_tokens = profile.get("max_input_tokens")

    if not max_tokens:
        # Fallback when profile data is unavailable.
        return len(messages) > 20

    estimated = count_tokens_approximately(messages)
    return estimated > int(max_tokens * threshold)


async def summarize_messages(
    messages: list[BaseMessage],
    model,
) -> str:
    """Compress older messages into a summary."""
    summary_prompt = [
        SystemMessage(content=(
            "Summarize the following conversation "
            "concisely, preserving key facts and decisions."
        )),
        HumanMessage(content="\n".join(
            f"{m.type}: {m.content}" for m in messages
        )),
    ]
    response = await model.ainvoke(summary_prompt)
    content = response.content
    return content if isinstance(content, str) else str(content)


async def context_management_node(
    state: AgentState,
    model,
) -> dict:
    messages = state["messages"]
    if not should_summarize(messages, model):
        return {}

    # Keep the first system message and the last 4 turns.
    head = (
        [messages[0]]
        if messages and isinstance(messages[0], SystemMessage)
        else []
    )
    keep_last_n = 8
    split_idx = max(len(head), len(messages) - keep_last_n)
    to_summarize = messages[len(head):split_idx]
    tail = messages[split_idx:]

    if not to_summarize:
        return {}

    summary_text = await summarize_messages(to_summarize, model)
    summary_msg = SystemMessage(
        content=f"[Conversation summary: {summary_text}]"
    )
    return {"messages": head + [summary_msg] + tail}

The critical property is that one node now handles GPT-4o mini (a 128k window, triggering at roughly 102,000 tokens) and Gemini 2.5 Flash (a 1M window, triggering at roughly 839,000 tokens) with no model-specific branching. Swap the model at the top of your graph and the middleware recalibrates itself.

The 80% threshold is deliberate. You want headroom for the model’s response tokens, tool call outputs, and any system prompt expansion. Running at 100% invites truncation errors, and running too conservatively wastes capacity. In practice 75% to 85% is the useful band.

A threshold fraction is a ratio applied to the model’s own reported window, not a token count. max_input_tokens * 0.8 is the same instruction on a 128k model and a 1M one, which is the entire point.

That band is not an outlier, and you no longer have to hand-roll it. LangChain’s built-in SummarizationMiddleware accepts a fraction directly, as trigger=("fraction", 0.8) with keep=("fraction", 0.3), which is this entire pattern as a first-class parameter. Note that trigger is optional and carries no default, so a middleware constructed without one never fires on its own. The 85% of max_input_tokens figure people quote, keeping roughly 10% of tokens as recent context, comes from the deepagents default middleware stack rather than from the bare middleware. We wrote up how that plays out over a long agent run in context engineering in deep agents, including what happens when the profile is missing and the fallback trigger is a flat 170,000 tokens.

One thing worth keeping in mind while you tune this: summarization is lossy by construction. A compacted history drops reversals, negative findings, and the things that did not happen, which is exactly the failure covered in your agent’s summary is not a source. Reading the real window is what buys you the right to summarize later rather than sooner.

How to gate tools, structured output, and vision on the profile

The same read-then-branch shape extends to any capability check. Two cases come up constantly in multi-model deployments.

Structured output fallback

Not every model supports .with_structured_output(). For those that do not, you need a prompt-based extraction path:

from pydantic import BaseModel


class ExtractionResult(BaseModel):
    entities: list[str]
    sentiment: str


def extract_structured(text: str, model) -> ExtractionResult:
    profile = getattr(model, "profile", None) or {}

    if profile.get("structured_output"):
        structured = model.with_structured_output(
            ExtractionResult
        )
        return structured.invoke(text)

    # Prompt-based fallback for models without
    # native structured output.
    response = model.invoke(
        f"Extract entities and sentiment from: {text}\n"
        'Reply in JSON: {"entities": [...], '
        '"sentiment": "..."}'
    )
    import json
    data = json.loads(response.content)
    return ExtractionResult(**data)

Modality gating

Passing image URLs to a model without vision support causes a runtime error, and usually one with an opaque message. Gate it at construction time instead:

from typing import Any


def build_message_content(
    text: str,
    image_url: str | None,
    model,
) -> list[dict[str, Any]]:
    content = [{"type": "text", "text": text}]
    profile = getattr(model, "profile", None) or {}

    if image_url and profile.get("image_inputs"):
        # Cross-provider standard content block.
        content.append({
            "type": "image",
            "source_type": "url",
            "url": image_url,
        })
    elif image_url:
        # Degrade gracefully: mention the image in text.
        content[0]["text"] += (
            f"\n[Image provided but not supported by "
            f"this model: {image_url}]"
        )
    return content

Both patterns follow the same structure. Read the capability from the profile, branch on the result, and handle the unsupported case explicitly. No try/except wrapped around model calls, no provider-specific conditionals, no magic strings.

What to do when the profile data is wrong

models.dev is community-maintained. For new model releases, niche fine-tunes, or models from smaller providers, the profile may be missing, stale, or simply wrong. This is a real limitation and worth knowing before you ship.

There are two override strategies, depending on how much control you need.

Quick fix at instantiation. Pass profile= directly to init_chat_model, which overrides the registry lookup entirely:

model = init_chat_model("some-new-model", profile={
    "max_input_tokens": 100_000,
    "tool_calling": True,
    "structured_output": True,
    "image_inputs": False,
})

Non-mutating update for a single invocation. When you need different profile values in a scoped context without mutating shared model state, use model_copy:

# Override max_input_tokens without touching
# the original model object.
conservative_profile = (
    (model.profile or {}) | {"max_input_tokens": 50_000}
)
conservative_model = model.model_copy(
    update={"profile": conservative_profile}
)

This is particularly useful in multi-tenant applications where different users or workflows need different effective limits on the same underlying model.

One honest caveat: model profiles are in beta. Field names and structure may change as LangChain finalizes the API. Pin your LangChain version in production and check the changelog when upgrading. The direction is clear enough, though. This surface area is expanding, not contracting.

Where model assumptions should live

Model profiles let you write context management that adapts to the model in use rather than encoding assumptions about a specific one. The principle is short: query capabilities at runtime, branch on the results, and handle unsupported cases deliberately.

Four things worth internalizing:

• Use fractions, not constants. max_input_tokens * 0.8 ages better than 8_000 when models change.
• Gate before you invoke. Checking profile["image_inputs"] before constructing a multimodal message is cheaper than catching a runtime error.
• Degrade explicitly. When a capability is absent, do something intentional (fall back, skip, or inform) rather than silently passing wrong input.
• Override when needed, contribute when possible. The registry is only useful if it is accurate, and model-specific workarounds you keep local are debt.

The broader shift is about where model assumptions live. In most codebases today they are scattered across middleware constants, provider-specific branches, and environment variable configs. model.profile gives you one place to centralise them, and one place to remove them when the model changes.

That is how we build agentic systems at Soba Labs: the runtime reads what the model can actually do, and the humans spend their attention on the decisions that carry risk rather than on chasing a constant somebody set eighteen months ago.

Sources

• LangChain, “Models” (model profiles): what .profile exposes and how providers populate it
• LangChain, “Built-in middleware”: SummarizationMiddleware, including its fractional trigger
• LangChain, “deepagents customization”: the default middleware stack and its 85% summarization trigger
• models.dev, the open-source model capabilities registry: the data behind the profiles
• Anthropic, “Context windows”: the 200k standard window and the 1M beta header
• Google, “Gemini 2.5 Flash”: the 1,048,576-token input limit
• OpenAI, “GPT-4o mini”: the 128k input limit