Skip to main content

LangChain Integration

BriefcaseLangChainHandler is a LangChain callback handler that captures AI decisions for every LLM call, chain execution, tool invocation, and retriever query in your application.

Installation​

pip install briefcase-ai
pip install langchain-core langchain-openai  # or your LLM provider

Constructor​

from briefcase.integrations.frameworks import BriefcaseLangChainHandler

handler = BriefcaseLangChainHandler(
    engagement_id="",          # project identifier
    workstream_id="",          # workflow identifier
    capture_llm=True,          # capture LLM/chat model calls
    capture_chains=True,       # capture chain start/end
    capture_tools=True,        # capture tool invocations
    capture_retrievers=True,   # capture retriever queries
    max_input_chars=10000,     # truncation limit for inputs
    max_output_chars=10000,    # truncation limit for outputs
    context_version=None,      # version tag added to all records
    async_capture=True,        # export runs in background thread
    client=None,               # reserved for future use
)

Basic Usage​

Pass the handler as a callback to any LangChain component:

from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate

llm = ChatOpenAI(model="gpt-4o", callbacks=[handler])
prompt = ChatPromptTemplate.from_template("Explain {topic} in one sentence.")
chain = prompt | llm

result = chain.invoke(
    {"topic": "quantum computing"},
    config={"callbacks": [handler]},
)

Retrieving Decisions​

# Get CapturedDecision objects
decisions = handler.get_decisions()

# Get serializable dicts (for storage or logging)
dicts = handler.get_decisions_as_dicts()

print(f"Captured {handler.decision_count} decisions")

# Reset for next request
handler.clear()

CapturedDecision Fields​

@dataclass
class CapturedDecision:
    decision_id: str          # UUID string
    decision_type: str        # "llm", "chain", "tool", "retriever"
    function_name: str        # model name or chain/tool class name
    inputs: Dict[str, Any]    # truncated input data
    outputs: Dict[str, Any]   # truncated output data
    model_parameters: Dict[str, Any]   # temperature, max_tokens, etc.
    error: Optional[str]      # set if the call raised an exception
    started_at: Optional[datetime]
    ended_at: Optional[datetime]
    execution_time_ms: Optional[float]
    parent_run_id: Optional[str]  # parent chain run_id for nesting
    engagement_id: str
    workstream_id: str
    tags: Dict[str, str]      # merged from LangChain tags + metadata
    token_usage: Optional[Dict[str, int]]   # prompt/completion/total
    context_version: Optional[str]

Capturing What Each Event Type Records​

LLM calls (decision_type = "llm"):

  • inputs.prompts or inputs.messages — the prompt strings or chat messages
  • outputs.text — the generated completion text
  • token_usage — {"prompt_tokens": N, "completion_tokens": N, "total_tokens": N}
  • model_parameters — temperature, max_tokens, top_p extracted from config

Chain events (decision_type = "chain"):

  • inputs — full chain input dict (keys depend on chain type)
  • outputs — full chain output dict
  • function_name — the chain class name (e.g., "LLMChain", "RetrievalQA")

Tool invocations (decision_type = "tool"):

  • inputs.input — the tool input string
  • outputs.output — the tool output string
  • function_name — the tool name from serialized["name"]

Retriever queries (decision_type = "retriever"):

  • inputs.query — the retrieval query string
  • outputs.document_count — number of documents returned
  • outputs.documents — list of {content_preview, metadata} dicts (200 char preview)

Advanced: Async Chains​

The handler works with both sync and async LangChain invocations. For async, pass the same handler object:

result = await chain.ainvoke(
    {"topic": "quantum computing"},
    config={"callbacks": [handler]},
)

Advanced: Per-Request Context Versions​

Set context_version to tag decisions with the knowledge version active at request time:

handler = BriefcaseLangChainHandler(
    engagement_id="my-project",
    workstream_id="rag-pipeline",
    context_version="knowledge-base-v2024-01",
)

All decisions from this handler instance will include context_version = "knowledge-base-v2024-01".

Advanced: Export on Chain Completion​

When a top-level chain ends (no parent run), the handler calls _trigger_export which passes the full decision record (including child spans) to the configured BriefcaseConfig.exporter.

To configure an exporter:

from briefcase.config import setup
from briefcase.exporters import SplunkHECExporter

setup(
    exporter=SplunkHECExporter(
        url="https://splunk.example.com:8088",
        token="your-hec-token",
    )
)

The export runs in a background daemon thread when async_capture=True (the default), so it never blocks the calling thread.

Troubleshooting​

No decisions captured: Confirm you passed the handler to both the LLM and the chain's config={"callbacks": [...]}. LangChain propagates callbacks downward, but not all chain types propagate automatically.

Missing token usage: Token usage is only available when the LLM response includes llm_output.token_usage. This depends on your provider and model.

Handler not called: The handler uses duck-typing to avoid requiring langchain-core at import time. If your LangChain version is older than 0.1.x, verify the callback interface is compatible.

See Also​