In my previous post, we looked at Inter-Agent Communication (A2A) — how independent agents talk to each other across process and organizational boundaries. That pattern is about making your system wider. This one is about making it cheaper and faster without making it dumber.
Here’s a habit that quietly bankrupts agentic systems: sending every single request to your most capable, most expensive model. “What’s the capital of Australia?” and “Compare the data-consistency trade-offs of microservices versus monoliths” hit the exact same frontier model, at the exact same per-token price, with the exact same latency budget. One of those questions deserves it. The other is burning money.
That’s where the Resource-Aware Optimization pattern comes in. The agent doesn’t just answer — it first decides what it should answer with.
Pattern #16: Resource-Aware Optimization#
The Problem#
LLM calls are not free, and they are not uniform. A frontier reasoning model can cost 10-30x more per token than a fast, lightweight one, and it’s often several times slower. At a handful of requests a day, nobody notices. At production scale — thousands or millions of requests — the difference between “always use the big model” and “use the right model” is the difference between a viable product and a line item your CFO circles in red.
The naive fix is to just downgrade to the cheap model everywhere. But then your genuinely hard queries get shallow, wrong answers, and you lose users for a different reason.
The real problem is that request difficulty is a distribution, not a constant. Most traffic is easy. A minority is hard. A slice needs fresh information your model doesn’t have. Treating all of it identically — whether you over-provision or under-provision — is leaving either money or quality on the table. Resource-Aware Optimization is the pattern for matching each request to the cheapest resource that can still answer it well.
The Solution#
The core move is to insert a classification / triage step before the work happens, then route to a model, tool, or configuration sized for the job. There are two ways to do it, and they sit at different points on the cost-vs-accuracy curve.
1. LLM-based classification + model selection (LangChain). Use a cheap, fast model as a router. It reads the prompt and returns a single label — simple, reasoning, or internet_search — and that label decides everything downstream:
def classify_prompt(llm, prompt: str) -> str:
"""
Classifies the prompt to choose the most cost-effective/appropriate model/tool.
"""
system_prompt = (
"You are a classifier that analyzes user prompts and returns one of three categories ONLY:\n\n"
"- simple\n"
"- reasoning\n"
"- internet_search\n\n"
"Respond ONLY with the category name."
)
response = llm.invoke([
("system", system_prompt),
("user", prompt)
])
classification = response.content.strip().lower()
if classification not in ["simple", "reasoning", "internet_search"]:
classification = "simple"
return classificationNotice the defensive default: if the classifier returns garbage, it falls back to simple. You never want your router to be a single point of failure that crashes the whole request.
The classification then drives resource selection. Each category gets a different model and a different prompt envelope:
def generate_optimized_response(prompt: str, classification: str, search_results=None):
if classification == "simple":
model = "gemini-2.5-flash"
full_content = prompt
elif classification == "reasoning":
model = "gemini-1.5-pro"
full_content = f"Think deeply and provide a detailed response: {prompt}"
elif classification == "internet_search":
model = "gemini-2.5-flash"
context = ""
if search_results:
context = "\n".join([f"Title: {r.get('title')}\nSnippet: {r.get('snippet')}" for r in search_results])
full_content = f"Use these search results to answer precisely:\n{context}\n\nQuery: {prompt}"
else:
model = "gemini-2.5-flash"
full_content = prompt
print(f"Using model: {model} for classification: {classification}")
llm = get_llm(temperature=0)
response = llm.invoke(full_content)
return response.contentThe internet_search branch is the interesting one: it’s not just picking a model, it’s deciding to spend a different resource entirely — a Google Custom Search call — before the LLM ever runs. Resource-awareness isn’t only about model size; it’s about which tools you pay for at all.
2. Heuristic routing as a graph (LangGraph). The LLM classifier above is accurate but adds a whole extra LLM call to every request — which is itself a cost. Sometimes a dumb heuristic is the right trade. Here the router is plain word-count, wired as a node in a state graph:
class ResourceState(TypedDict):
query: str
complexity: str
word_count: int
response: str
model_used: str
def analyze_complexity(state: ResourceState) -> dict:
"""Analyzes query complexity using a simple heuristic (word count)."""
word_count = len(state["query"].split())
complexity = "complex" if word_count > 15 else "simple"
return {"complexity": complexity, "word_count": word_count}
def route_by_complexity(state: ResourceState) -> Literal["fast_model_handler", "powerful_model_handler"]:
if state["complexity"] == "simple":
return "fast_model_handler"
return "powerful_model_handler"The two handlers don’t just swap models — they tune the whole generation envelope. The fast path runs at temperature=0 with a tight max_output_tokens=256 and a “be concise” system prompt. The powerful path runs at temperature=0.3, max_output_tokens=1024, and asks for structured analysis:
def fast_model_handler(state: ResourceState) -> dict:
llm = get_llm(temperature=0, max_output_tokens=256)
prompt = ChatPromptTemplate.from_messages([
("system", "You are a concise assistant. Answer briefly and directly."),
("user", "{query}")
])
chain = prompt | llm | StrOutputParser()
response = chain.invoke({"query": state["query"]})
return {"response": response, "model_used": "gemini-2.5-flash (fast/concise)"}Wiring it together is a textbook conditional edge — analyze, branch, terminate:
builder.add_edge(START, "analyze_complexity")
builder.add_conditional_edges(
"analyze_complexity", route_by_complexity,
{"fast_model_handler": "fast_model_handler", "powerful_model_handler": "powerful_model_handler"}
)
builder.add_edge("fast_model_handler", END)
builder.add_edge("powerful_model_handler", END)max_output_tokens is the under-appreciated lever here. Output tokens are usually the priciest part of a call, and capping them on the fast path saves money on every simple request — independent of which model you picked.
Why This Matters#
Resource-Aware Optimization is what turns a working prototype into something you can afford to run at scale:
- Cost control at volume: route the 80% of easy traffic to a cheap model and reserve the frontier model for the 20% that needs it. The blended cost drops dramatically without the median user noticing.
- Latency budgets: a fast model on the simple path means snappy answers for the common case, while complex queries are allowed to take their time.
- Tool gating: don’t pay for a search API, a code sandbox, or a retrieval pass unless the request actually needs it.
- Tiered SLAs: free-tier users get the heuristic router; paying customers get the LLM classifier and the powerful model.
The trade-offs are real and worth stating plainly:
- The router is overhead. The LLM-based classifier adds a full extra call to every request. If your classifier costs nearly as much as the model it’s protecting, you’ve gained nothing. The word-count heuristic has near-zero cost but misclassifies — a short, profound question gets the cheap model and a bad answer.
- Misrouting has asymmetric cost. Sending a hard query to the weak model produces a wrong answer (expensive in trust). Sending an easy query to the strong model just wastes a little money (cheap). Tune your thresholds knowing the failure modes aren’t symmetric.
- It’s another thing to monitor. You now need to track routing distribution, per-tier accuracy, and cost-per-request, or your “optimization” quietly drifts into “everything routes to simple and quality tanks.”
Rule of thumb from the book: reach for this pattern once request volume or model cost stops being negligible, and your traffic has a genuine spread of difficulty. If every request is hard, or you’re at toy scale, the routing machinery is just complexity you don’t need yet.
The Bigger Picture#
This is post #16 in my series documenting Antonio Gulli’s Agentic Design Patterns. All credit for the conceptual work goes to him — I’m focused on producing clean, runnable Python implementations that you can clone, modify, and ship.
What makes Resource-Aware Optimization sit so well with the rest of the series is that it’s really Routing (Pattern #2) pointed at a different target. Earlier we routed by intent — which specialist handles this. Here we route by cost and difficulty — which resource can handle this cheaply enough. Same machinery, different objective function. It also composes naturally with the upcoming Prioritization and Evaluation chapters, where deciding what to spend resources on and measuring whether it paid off become first-class concerns.
All the code from this post is in my repository: carlosprados/Agentic_Design_Patterns, specifically under 16_Resource_Aware_Optimization/. Both the LangChain classifier-router and the LangGraph heuristic graph run with uv run, support Gemini and Ollama via the shared get_llm() abstraction, and are ready to fork.
What’s Next#
In the next post we’ll tackle Reasoning Techniques — the patterns that get an agent to think before it answers: chain-of-thought, self-consistency, and the structured deliberation strategies that turn a one-shot guess into a worked-through solution. It’s the other side of the cost coin: sometimes the right answer to “how do I spend less?” is “spend more, but deliberately.”
Stay tuned.