Most LLM failures in production aren’t model failures — they’re task design failures. You hand a single prompt a problem that requires research, synthesis, conditional logic, and a final decision, then wonder why the output is vague or hallucinates details. Prompt chaining agents solve this by decomposing the problem into discrete, verifiable steps where each prompt does one job well and passes structured output to the next stage.
This isn’t just a cleaner architecture pattern. It measurably reduces hallucinations, makes debugging tractable, and lets you swap individual steps without rebuilding the whole pipeline. If you’ve hit the ceiling of what a single-shot prompt can reliably do, this is what you build next.
Why Single Prompts Break on Complex Tasks
The core problem is working memory and task interference. When you ask a model to “research competitor pricing, identify gaps in our positioning, draft three campaign angles, and recommend the best one,” you’re asking it to context-switch between four fundamentally different cognitive modes in a single forward pass. It will do all four passably and none of them well.
Research tasks need retrieval and citation. Analysis tasks need systematic comparison. Generation tasks need creativity. Decision tasks need evaluation criteria applied consistently. These aren’t just different prompts — they’re different system states, and conflating them in one prompt is why you get confidently wrong answers.
The second failure mode is error propagation without checkpoints. If your single mega-prompt gets the research wrong, the analysis, generation, and decision are all built on bad data. With chaining, you can validate outputs at each step before proceeding.
The Anatomy of a Prompt Chain
A prompt chain is a directed sequence of LLM calls where the output of step N becomes structured input for step N+1. The key word is structured. Passing raw text between steps is asking for trouble — you want typed, validated data moving through your pipeline.
Basic Chain Structure
Here’s the pattern I use for most production chains. The state object is the backbone — every step reads from it and writes back to it:
import anthropic
import json
from dataclasses import dataclass, field
from typing import Any
client = anthropic.Anthropic()
@dataclass
class ChainState:
task: str
steps_completed: list[str] = field(default_factory=list)
data: dict[str, Any] = field(default_factory=dict)
errors: list[str] = field(default_factory=list)
def run_chain_step(
state: ChainState,
step_name: str,
system_prompt: str,
user_prompt: str,
output_schema: dict
) -> ChainState:
"""Run a single step, validate output, update state."""
# Inject current state context into the prompt
context = json.dumps(state.data, indent=2)
full_prompt = f"Current context:\n{context}\n\n{user_prompt}"
response = client.messages.create(
model="claude-haiku-4-5", # use Haiku for cheap intermediate steps
max_tokens=1024,
system=system_prompt,
messages=[{"role": "user", "content": full_prompt}]
)
raw_output = response.content[0].text
# Attempt to parse structured output
try:
parsed = json.loads(raw_output)
# Basic schema validation — use jsonschema in production
for required_key in output_schema.get("required", []):
if required_key not in parsed:
raise ValueError(f"Missing required key: {required_key}")
state.data[step_name] = parsed
state.steps_completed.append(step_name)
except (json.JSONDecodeError, ValueError) as e:
state.errors.append(f"Step '{step_name}' output error: {e}")
# Don't update steps_completed — caller decides whether to retry or abort
return state
At current Haiku pricing (~$0.00025 per 1K input tokens, ~$0.00125 per 1K output tokens), a five-step chain with 500-token prompts and 300-token outputs runs about $0.003 per full execution. That’s cheap enough to run hundreds of daily automations without meaningful cost pressure.
Designing Step Boundaries
The hardest design decision is where to cut the chain. My rule: a step boundary goes wherever you’d want a human checkpoint in a manual process. If you’d want to review “what did we find in research before writing the analysis?”, that’s a step boundary.
Practical criteria for splitting a step:
- The output is a different data type than the input (text → structured data, data → decision)
- You’d want to log, validate, or route differently based on what this step produces
- The step requires a different persona or expertise (researcher vs. copywriter vs. critic)
- Failure at this step should trigger different recovery logic than failure at adjacent steps
Building a Real Multi-Step Agent Workflow
Let’s make this concrete with a workflow I actually run: competitive intelligence analysis. The task is “given a competitor URL, produce a positioning gap analysis with three actionable recommendations.” Doing this in one prompt produces generic output. Chained, it’s reliable enough to automate.
Step 1: Extract and Structure Raw Information
STEP1_SYSTEM = """You are a data extraction specialist. Your only job is to extract
factual information from the provided text and return it as clean JSON.
Do not interpret, analyze, or editorialize. If something is not stated explicitly,
omit it rather than infer it."""
STEP1_SCHEMA = {
"required": ["product_name", "pricing_tiers", "key_features", "target_segments"]
}
def step_extract(state: ChainState, raw_page_content: str) -> ChainState:
prompt = f"""Extract the following from this competitor page content.
Return ONLY valid JSON matching this structure:
{{
"product_name": "string",
"pricing_tiers": [{{"name": "string", "price": "string", "features": ["string"]}}],
"key_features": ["string"],
"target_segments": ["string"]
}}
Page content:
{raw_page_content[:3000]}""" # truncate to avoid context bloat
return run_chain_step(state, "extraction", STEP1_SYSTEM, prompt, STEP1_SCHEMA)
Step 2: Comparative Analysis Against Your Own Data
STEP2_SYSTEM = """You are a strategic analyst. You receive structured data about
a competitor and your own product. Identify concrete gaps and overlaps.
Be specific — cite actual features and price points, not vague themes."""
def step_analyze(state: ChainState, our_product_data: dict) -> ChainState:
prompt = f"""Compare the competitor data in context with our product data below.
Return JSON with keys: "gaps" (things they have we don't),
"advantages" (things we have they don't), "pricing_position" (string summary).
Our product data:
{json.dumps(our_product_data, indent=2)}"""
return run_chain_step(
state, "analysis", STEP2_SYSTEM, prompt,
{"required": ["gaps", "advantages", "pricing_position"]}
)
Step 3: Conditional Routing Based on Analysis Output
This is where chains get genuinely powerful. After analysis, you can branch based on what was found:
def step_route_and_recommend(state: ChainState) -> ChainState:
analysis = state.data.get("analysis", {})
gaps = analysis.get("gaps", [])
# Conditional logic in Python, not buried in a prompt
if len(gaps) > 3:
persona = "You are a product strategist focused on rapid feature development."
angle = "prioritize closing critical feature gaps"
else:
persona = "You are a brand strategist focused on positioning and messaging."
angle = "emphasize differentiation on existing strengths"
system = f"""{persona} Generate exactly 3 specific, actionable recommendations.
Each recommendation must reference specific data from the analysis context."""
prompt = f"""Based on the analysis, {angle}.
Return JSON: {{"recommendations": [{{"title": "string", "rationale": "string",
"effort": "low|medium|high", "impact": "low|medium|high"}}]}}"""
return run_chain_step(
state, "recommendations", system, prompt,
{"required": ["recommendations"]}
)
Notice that the routing decision happens in Python, not in a prompt. Never ask an LLM to make a routing decision if you can make it deterministically from structured output. This is where most agent frameworks go wrong — they trust the model to decide what to do next when structured data from the previous step already tells you.
Error Recovery Without Crashing the Pipeline
Production chains break in predictable ways: the model returns malformed JSON, a step times out, or the output of step 2 is technically valid but semantically wrong (e.g., empty lists where data was expected). You need recovery logic at two levels.
Step-Level Retry with Prompt Repair
def run_step_with_retry(state, step_name, system, prompt, schema, max_retries=2):
for attempt in range(max_retries + 1):
state = run_chain_step(state, step_name, system, prompt, schema)
if step_name in state.steps_completed:
return state # success
if attempt < max_retries:
# Repair prompt — tell the model what went wrong
last_error = state.errors[-1] if state.errors else "Invalid output format"
prompt = f"""Previous attempt failed: {last_error}
IMPORTANT: Return ONLY valid JSON. No explanation text before or after.
Original task:
{prompt}"""
state.errors = [] # clear for retry
return state # caller checks if step completed
Chain-Level Fallbacks
If a non-critical step fails after retries, you have options: skip it and proceed with degraded output, substitute a default value, or escalate to a more capable model for that step only. Escalating to Sonnet for a single failed step costs roughly 5-10x more for that call, but saves the entire chain run. For a three-minute automation that generates revenue, that’s almost always worth it.
def run_chain(task: str, raw_content: str, our_data: dict) -> ChainState:
state = ChainState(task=task)
state = step_extract(state, raw_content)
if "extraction" not in state.steps_completed:
# Critical step — abort
raise RuntimeError(f"Chain aborted at extraction: {state.errors}")
state = step_analyze(state, our_data)
if "analysis" not in state.steps_completed:
# Degrade gracefully — use empty analysis
state.data["analysis"] = {"gaps": [], "advantages": [], "pricing_position": "unknown"}
state.steps_completed.append("analysis")
state = step_route_and_recommend(state)
return state
State Management Across Longer Chains
For chains with more than five steps, or chains that run asynchronously, the in-memory state object won’t cut it. You need persistence. I store chain state in Redis for short-lived workflows (TTL of a few hours) and Postgres for anything that needs an audit trail.
The minimal schema I use for persisted chains: chain_id, task, status (pending/running/completed/failed), steps_completed (JSON array), data (JSONB), created_at, updated_at. This gives you resumability — if a chain fails at step 4 of 6, you can restart from step 4 rather than the beginning. At scale, this matters both for cost and for user experience.
When to Use Chain-of-Thought Instead
Prompt chaining and chain-of-thought (CoT) solve different problems. CoT keeps reasoning inside a single prompt — you add “think step by step” or use extended thinking modes to let the model work through a problem before committing to an answer. It’s better for: math and logic problems, cases where the reasoning path isn’t predictable in advance, and when you’re under latency constraints that make multiple API calls impractical.
Use prompt chaining when: steps require different system personas, intermediate outputs need validation or routing, steps have different cost profiles (expensive research + cheap synthesis), or you need to inject external data between steps (tool calls, database lookups, API results).
Use CoT when: the task is fundamentally a single reasoning problem, you need a latency under ~2 seconds, or the steps aren’t cleanly separable with typed interfaces.
Many production systems use both: CoT within individual chain steps for the hard reasoning parts, chaining between steps for orchestration and state management.
Integrating Prompt Chaining Agents with n8n and Make
If you’re building automation workflows rather than application code, you can implement this pattern in n8n without writing the orchestration layer yourself. Each chain step becomes an LLM node with a system prompt scoped to that step’s role. State passes between nodes via the workflow’s data object.
The key n8n-specific tip: use the “Set” node between LLM calls to extract and validate the JSON fields you actually need. Don’t pass the full raw LLM response to the next node — parse it first. This makes your workflow resilient to model output variations and much easier to debug when something breaks.
In Make (formerly Integromat), the same pattern applies via HTTP modules hitting the API directly, with JSON parsers between steps. It’s more manual but gives you finer control over error handling paths.
When to Reach for This Pattern
Solo founders and small teams: Start with three-step chains maximum. Extract → Analyze → Recommend covers 80% of automation use cases and is maintainable without dedicated infrastructure. Use Claude Haiku for extraction and analysis, escalate to Sonnet only for the final decision step if quality matters.
Teams building production agents: Invest in the state persistence layer early. A chain that can resume from failure is worth 3x more in production than one that can’t. Add structured logging at each step boundary — it’s the only way to debug a chain that fails intermittently on specific input types.
Enterprise workflows: Prompt chaining agents compose naturally with tool use and retrieval-augmented generation. Each step can call external tools, and the chain orchestrates the sequence. This is effectively what most “agent frameworks” are doing under the hood — building this yourself gives you control over the parts that frameworks abstract away badly.
The bottom line: if your single-prompt outputs are inconsistent, hard to debug, or routinely miss parts of the task, you don’t need a better prompt. You need a better architecture. Decompose the problem, validate between steps, and route conditionally on structured output. That’s what makes prompt chaining agents reliable enough to actually deploy.
Editorial note: API pricing, model capabilities, and tool features change frequently — always verify current details on the vendor’s website before building in production. Code examples are tested at time of writing; pin your dependency versions to avoid breaking changes. Some links in this article may be affiliate links — we may earn a commission if you sign up, at no extra cost to you.
