At VB Transform 2026, three enterprise infrastructure leaders compared notes on what actually broke when they moved AI agents from pilot to production. Their shared conclusion: the models aren't the problem. The infrastructure around them is.
The panel featured Animesh Singh, senior director of AI platform and infrastructure at LinkedIn; Desiree Gosby, SVP of corporate technology services at Walmart; and Sami Ghoche, VP of applied AI at Zendesk. Each arrived at the same diagnosis from a different direction.
As Gosby framed it, the core challenge is ensuring that "engineering doesn't once again become the bottleneck for what it is we're trying to do."
Where the Walls Were
Each company ran into a distinct version of the same structural problem: enterprise infrastructure was designed around human work cadences, not the millisecond-speed demands of autonomous agents.
LinkedIn's first bottleneck was Kubernetes. The platform assumes containers spin up on demand — a process measured in seconds, which Singh said is simply too slow for agentic workloads. The fix: pre-provisioned pools of containers that swap workloads in and out in real time.
A harder problem surfaced when LinkedIn gave agents control over their own orchestration. A five-point evaluation system appeared solid, but hallucinations kept appearing. The diagnosis was structural: an LLM evaluating another LLM's output shares the same failure modes as the model being evaluated.
"We built our own harness, our own control flow, and pushed the LLMs to the leaf instead of them orchestrating the loop," Singh said.
Today, roughly 80% of LinkedIn's agentic workflows are scripted, deterministic code. LLMs are invoked only where genuine reasoning is required, and each step's evidence is committed to disk before execution continues.
Walmart's bottleneck came from unexpected success. When an agent harness was put directly into employees' hands, it went viral internally. What Gosby called "citizen developers" began building their own agents without formal engineering involvement — creating real innovation, but also dozens of overlapping, uncoordinated tools. The solution wasn't to restrict access; it was to build governance infrastructure to identify duplication, surface the best version of a given agent, and fast-track it to production.
Zendesk's wall was on the data side. Ghoche — who joined through Zendesk's acquisition of Forethought, which closed in March 2026 — described sitting on a repository of 20 billion customer conversations. The intuitive move is to feed that history into a large context window and let a model generate the agents a business needs.
"You can't really do that, so instead you have to really invest in the underlying data pipelines and all the data infrastructure that comes with that," Ghoche said.
The Open Source Posture
All three companies landed on a similar instinct around model ownership: control what you can, and rely on frontier labs only where they retain a clear edge.
Ghoche said most enterprises prefer to own their models and infrastructure where possible, and that reasoning directly shapes Zendesk's approach. The exception is frontier reasoning tasks — though he noted that category is shrinking relative to everything else enterprises now do with AI.
LinkedIn built two independence-focused subsystems:
- An AI gateway — a single interface through which every outbound model call is routed, regardless of provider
- A memory subsystem that holds context independent of any model vendor
"Every single outbound call going to an LLM, whether it's on a public cloud or on-prem in our own data centers, follows the same semantics, the same API calls. We can quickly switch between different providers," Singh said.
Walmart built its own internal gateway to stay vendor-agnostic across three workload types: fully deterministic workflows, planner-and-reasoner workflows for open-ended tasks, and hybrids of the two. The choice between a frontier model and an open-weight model is decided per workload, not by blanket policy.
Three Pieces of Actionable Advice
Each panelist distilled their hard-won experience into direct guidance:
- Invest in evals before anything else. Ghoche called robust evaluations the common thread across every use case, internal or customer-facing. "Once you have a robust set of evals, you can move a lot faster."
- Own your agent harness from day one. Gosby recommended putting the harness in employees' hands early — paired with monitoring infrastructure. "It will unlock a huge amount of innovation."
- Build for model and context independence. Singh's advice: architect for flexibility now. "Build for independence, whether it's a frontier model of today versus an open source model of tomorrow. Keep that context within your enterprise so that you can reuse it when you ship the model or the harness tomorrow."
What This Means for Builders
For startup founders and product teams building on top of AI, the lesson from three of the largest enterprise deployments in the industry is sobering: the hard engineering work isn't in the model layer — it's in the scaffolding around it. Evals, gateways, memory management, and governance infrastructure are where production reliability is actually won or lost. Teams that treat those as afterthoughts will hit the same walls LinkedIn, Walmart, and Zendesk already climbed.
