What do enterprises want from agents?
I recently spent the day at the RDI Agentic AI Summit at UC Berkeley, and a popular topic was how enterprises are adopting agents, and everything that still needs to be figured out before these systems can be trusted at scale. There were a handful of themes I pulled out of the talks I attended, but four stood out to me in ways that I felt I could articulate to a decent degree.
Theme 1: Standards and interoperability
Enterprises want agents to speak the same language, connect across systems, and plug into existing tools without cobbling things together. Right now the ecosystem is fragmented, and without shared standards large organizations are going to struggle to adopt agents at scale. For instance:
- Efforts like Natural Language Interaction Protocol (NLIP) and the better-known Model Context Protocol (MCP) are hoping to get agents to talk to each other similar to HTTP, which made the web interoperable.
- Cloud vendors are pushing agent APIs, but enterprises are (and will likely remain) wary of lock-in. For example, Anthropic positions MCP as an interoperability layer, but today it is generally shipped “out of the box” with Claude.
- Enterprise buyers will demand turn-key agents that can be swapped out as better models emerge. For example, a customer service bot should work the same if you swap the model from GPT-4 to Claude.
Theme 2: Human-readable logic
Most enterprise workflows are not owned by engineers. They are owned by ops, compliance, or service teams. If those folks cannot see and edit what the agent is doing, adoption will falter. Agents need logic that non-engineers can comprehend, not just hard-to-read prompts or Python.
- UCSB demoed agent files that look more like SQL or SOPs than code (you can see some examples around this spot in the talk).
- Similarly, Oracle Health described digital care managers whose rules had to be auditable by non-technical staff.
Theme 3: Context and continuity
One-off conversations do not cut it in an enterprise setting. Whether it is a patient record, a financial transaction, or an incident response situation, continuity is critical. Agents need memory, state, and grounding to be useful in real-world workflows.
- Oracle Health emphasized longitudinal state, which they defined as “an agent cannot forget the last six months of a patient’s treatment.”
- In customer service, context must carry across multiple channels such as chat, email, and phone.
- Incident management agents need to maintain playbooks across long-running outages.
Theme 4: Safety and trust
This feels like table stakes everywhere, but the stakes are particularly high in healthcare, finance, and enterprise IT. Agents need to be safe, auditable, and compliant. Enterprises will not deploy agents that cannot explain themselves or that create regulatory risk.
- Audit trails: who made the decision, why, and what the result was.
- Compliance baked in (HIPAA, SOC2, GDPR).
- Guardrails to prevent runaway autonomy and define safe failure modes.
- Hybrid decision-making: humans in the loop for sensitive steps.
For all of these themes, the best I could come up with for the question “so what?” is this: AI hype can sometimes make it feel like machines are about to do everything, but the reality is they are creating plenty of new issues. As enterprises adopt these tools, the list of problems grows. It gives me a small degree of calm that there is still plenty of work to do.
A lot of the talks I reference here were from these morning sessions, but you should explore the other videos on the YouTube channel if you’re interested.