We stopped treating agent latency as one number.

Hey folks,

In the last post, we wrote about the Tactix Co-Pilot as a routing agent. It does not hold the data. It decides which tools to call, gets the right signals, and turns them into an answer an operator can use.

That architecture gave us a safe path to the answer.

It did not give us a fast one.

To the operator, latency looked like one number: the time between asking a question and seeing a useful answer. Inside the system, that wait was spread across entity extraction, model turns, Model Context Protocol (MCP) calls, BigQuery work, network hops, and sometimes another trip through the whole loop.

We initially looked for the slow part. That was the wrong model.

There was no single slow part. The request path had accumulated several kinds of waiting:

• the data layer rebuilt analytical shapes during a live request,

• the agent asked for one tool at a time,

• the MCP client serialized calls that could have run together,

• the model could take up to ten turns,

• and our timing data could tell us that a request was slow, but not which layer owned the delay.

So we treated latency as a systems problem. We moved repeated data work out of the request path, widened the tool turn, tightened the reasoning loop, and added clocks at the boundaries between them.

Moving from hourly to daily data was one important improvement. It was not the whole story.

Agent latency is not one wait. It is every wait we allow to accumulate between the question and the answer.

The optimization had four layers

One question such as “How did my stores perform last week?” crosses three services. The agent interprets the question. The MCP service translates that intent into warehouse queries. BigQuery assembles the evidence. The model then turns that evidence into an answer.

We changed that path in four places.

1. Data path: give the agent smaller, purpose-built surfaces and precompute repeated transformations.

2. Tool path: let independent MCP calls execute together instead of waiting in a queue.

3. Reasoning path: let one model turn request several tools, then reduce the maximum turn count.

4. Measurement path: time the model, tools, warehouse, and surrounding infrastructure separately.

These changes solve different waits. A faster query does not remove an unnecessary model turn. Parallel tools do not help when every call sits behind the same lock. A lower turn ceiling does not help if the first query rebuilds hourly weather and labor data.

The agent only feels fast when the whole path does less work.

Improvement one: move repeated data work off the request path

The original agent surfaces were built on hour-grain models such as agents.generic and agents.forecast. That detail was useful when a question genuinely depended on intraday shape.

Most Co-Pilot questions did not.

“What was revenue last week?” “How is labor tracking?” “Which menu items are moving?” These are usually daily questions. Starting from hourly rows meant scanning roughly 24 times more records per store-day, joining more aggressively, and assembling the business grain during a live conversation.

So we built a day-level agent layer in dbt:

• generic_day for revenue, orders, average order value, labor, weather, and anomalies,

• forecast_day for daily forecast versus actual,

• channel_day for channel mix,

• labor_day for daily labor deployment and efficiency,

• menu_item_day for item performance.

The hour-level models stayed in place for backward compatibility and for the smaller set of questions that need them. The change was not “daily is always better.” It was “the default surface should match the default question.”

That distinction matters for agents. A tool schema is not just a data-access contract. It is a performance contract. Every extra row and field can lengthen the warehouse query, the network payload, and the next model turn.

The best agent table is not the most detailed table. It is the narrowest truthful surface for the question.

The day-grain migration helped, but not as much as we expected.

agents.generic_day returned daily rows, but it was still materialized as a view. BigQuery had to execute the logic behind that view for each request. The plan expanded hourly weather visibility through CROSS JOIN UNNEST and grouped tactix_labor_by_hour back to a day.

We had changed the grain of the result, not the grain of the work.

The structural fix was to turn generic_day into a table:

materialized='table',
partition_by={'field': 'order_date', 'data_type': 'date'},
cluster_by=['store_id', 'order_date']

Now the expensive weather and labor transformations run during the scheduled dbt build. A Co-Pilot request reads the precomputed daily result instead of replaying the transformation.

We also clustered the upstream daily order table and daily anomaly staging table on (store_id, order_date). Those are the columns the MCP tools use for store-scoped date windows, so the physical layout now matches the dominant filter pattern.

This is a trade, not a free lunch. We moved work from query time to build time. The chat path gets shorter, but freshness now depends on the scheduled agents job rebuilding the table successfully.

That is still the right trade for this product. Operators ask the same classes of questions far more often than we rebuild the analytical model.

Precomputation is not avoiding the work. It is choosing when the user has to wait for it.

Improvement two: let independent tools move together

Once the data path was smaller, the tool path became the next visible constraint.

The original Co-Pilot could request one tool per model turn. Every MCP call also sat behind a global session lock. A question that needed performance, channel mix, labor, and weather became a very courteous queue.

We changed the loop so one model turn can request up to four tools:

{
  "action": "tool_call",
  "calls": [
    {"tool": "fetch_store_performance_by_day"},
    {"tool": "fetch_channel_breakdown_by_day"},
    {"tool": "fetch_labor_deployment_by_day"},
    {"tool": "fetch_weather_actuals_by_day"}
  ]
}

Independent calls now execute together with asyncio.gather. The tool portion of the critical path becomes the duration of the slowest independent call, not the sum of all four.

The MCP client now locks connection setup and teardown, not every call_tool. Requests can share one session without being serialized behind _session_lock.

Parallelism only applies when the inputs are already known. If one result determines the next question, that dependency stays sequential. The point was not to make the agent do four things at once. It was to stop making independent work wait in line.

Parallelism removes waiting between independent calls. It does not remove the reasoning between dependent ones.

Improvement three: give the reasoning loop a tighter boundary

A wider tool turn changed how many model turns the Co-Pilot needed.

The original ceiling was ten turns. That gave the agent room to recover, but it also gave weak routing more room to wander. Once a turn could gather a coherent evidence set, we lowered MAX_TURNS from ten to six.

This was not simply a timeout reduction. It depended on the changes underneath it:

• the tool surface now matched the common business question,

• one turn could request several independent tools,

• those calls could actually run concurrently,

• and each result returned a smaller daily payload.

A lower turn ceiling without those changes would only make the agent fail sooner. With them, six turns became a useful constraint on the reasoning path.

The principle is simple: give the agent enough room to resolve the question, but do not make extra turns the default form of resilience.

Improvement four: keep the fast path truthful

The optimization surfaced a correctness problem that is easy to miss in a latency project.

The entity extractor can resolve “last week” into an exact calendar window. That is normally what we want. But some stores had generic_day data that lagged the underlying order table. A strict date filter could return zero rows even though recent data existed.

The agent would then answer quickly and incorrectly: data unavailable.

We added a bounded fallback for weekly performance:

1. Run the requested date window.

2. If it returns no rows, retry with a four-week lookback.

3. Keep the most recent week that actually has data.

4. Attach a data_period_note so the model tells the operator which period it used.

This is not permission to silently substitute dates. The date shift is part of the answer.

There is a separate freshness boundary around forecast actuals. The daily performance table can be current while the forecast-actualized pipeline is still catching up. We keep those signals distinct instead of pretending every model advances at the same pace.

Once you precompute the hot path, freshness becomes part of your interface. A table can be fast, well-partitioned, and wrong for the user’s requested window.

Fallbacks should preserve truth, not merely avoid an empty state.

Measure the phase that owns the wait

We did not have a trustworthy percentage to put in the title. The instrumentation landed with the optimization work, so a neat before-and-after number would have been more marketing than measurement.

Instead, we added clocks at the boundaries that matter.

The agent now records:

• entity-extraction time,

• each model turn’s duration,

• total model time and tool time,

• model calls, tool calls, and turns used.

The MCP and BigQuery path records:

• MCP tool-call and tool-result timing,

• BigQuery wall time and job time,

• bytes processed and billed,

• total slot milliseconds,

• whether BigQuery returned a query-cache hit.

Those measurements let us separate a slow model turn from a slow tool, a slow tool from a slow query, and a slow query from a fast query hidden behind connection or cold-start time.

They also keep us honest about the next optimization. If model time dominates, another table change will not fix it. If the query is fast but returns a large payload, the next model turn may still pay the price. If the materialized table is stale, lower latency is not a win at all.

We will publish performance numbers when we have enough production traces to compare like with like. A single-fetch revenue question and a four-tool causal investigation should not be averaged into one flattering metric.

One stopwatch tells you that the user waited. Phase timing tells you which system made them wait.

What still goes wrong

Precomputed tables can drift behind their sources. The scheduled dbt job is now part of the product path. We need freshness checks around generic_day, not just successful builds.

Parallel tools can create wider waste. Four available lanes do not mean every question needs four tools. Tool descriptions and prompts still have to keep the evidence set tight.

A lower turn ceiling exposes weak routing. Six turns are enough when the tool surface is clear. When dates, stores, or intents are unresolved, the agent simply fails sooner.

Cloud Run still contributes baseline latency. Data and loop changes do not remove service startup or the HTTP hop between the agent and MCP service.

Forecast actuals can lag performance actuals. Those pipelines have different freshness characteristics, and the answer needs to say so when the windows do not line up.

Takeaways

A few things I would carry into the next tool-using agent.

• Map the full request path. The user experiences one wait, but each layer needs a different fix.

• Match the data grain to the common question. Keep detailed models available, but do not make every request start there.

• Move stable computation out of the live turn. Precompute analytical shapes that are read far more often than they are rebuilt.

• Batch only independent tools. Widen the turn without erasing causal order.

• Lock the connection, not the work. A shared client should not quietly serialize calls the rest of the system thinks are concurrent.

• Tighten the loop after improving the evidence path. A lower ceiling is useful only when each turn can do coherent work.

• Treat freshness as part of correctness. A fallback can use the latest complete period, but it must tell the user what changed.

• Instrument each boundary. Agent, MCP, warehouse, and infrastructure delays need different fixes.

The main lesson was not that BigQuery needed tuning, that the agent needed fewer turns, or that the hourly models needed daily counterparts.

It was that agent latency belongs to the whole request path.

The data layer was repeating work. The tool layer was forcing independent calls to wait. The reasoning loop had more room than it needed. And our original timing made all of those delays look like one number.

Once we treated each wait separately, the system became easier to improve. The MCP tools read smaller surfaces. Independent calls stopped queuing behind one another. The model had fewer chances to wander. And when a turn was still slow, we finally had enough timing data to know where to look.

If you are tuning an agent with a real warehouse and a real tool surface, map the full turn before reaching for a faster model. I would love to compare notes.

Further reading

• Google Cloud, Introduction to partitioned tables. How partition pruning reduces the data a date-scoped query needs to read.

• Google Cloud, Introduction to clustered tables. How clustered storage colocates values and supports block pruning for common filters.

• Google Cloud, Optimize query computation. The warehouse-side principles behind precomputing and narrowing repeated analytical work.

• Python, Running tasks concurrently. The asyncio.gather pattern used for independent MCP calls.

• dbt Labs, Materializations. The practical difference between keeping a model as a view and building it as a table.

Until next time,

Amit