
68 Minutes with a Java Champion Changed How I Think About Production Agents.
James Ward (Java Champion, Principal Developer Advocate) discusses Spring AI 2.0, MCP Model Context Protocol, Java AI agents in production, LangChain4j vs Spring AI vs Embabel, agent memory strategies, tool calling patterns, AgentCore deployment, and why Java is the production runtime for agentic AI workloads.
In April, right after our talk at GIDS Bangalore, I sat down in a studio with James Ward. Java Champion. Building software since 1997. Contributor to the 12-Factor App. Currently shaping how Java developers think about AI at AWS.
We hit record. Didn’t stop for 3 hours.

What started as a podcast became something else — a 68-minute map of the entire Java AI landscape as it exists today. Frameworks. Protocols. Memory architectures. The “too many tools” problem (and three solutions that actually work). Production deployment. The question of RAG vs Skills vs Tools — and when each one wins.
And here’s the thing between the day we recorded and the day I’m writing this, half of what we discussed as “coming soon” shipped. Spring AI 2.0. Embabel 1.0. MCP’s biggest protocol revision ever. The Spring AI AgentCore SDK. All GA. All in 90 days.
The stack crystallized while we were still editing the episode.
So I wrote this blog as the companion piece. The architectural decisions, the framework tradeoffs, and the production patterns that emerged from that conversation and 100+ developer conversations at conferences since. If you’re a Java developer building with AI in 2026, this is the map I wish existed three months ago.
The Foundational Claim: Python Trains AI. Java Runs AI.
Let me ground this with numbers.
- 90% of Fortune 500 companies run Java in production. (Oracle, 2025)
- 62% of enterprises now use Java to power AI applications. (Azul survey, January 2026)
- A Spring AI survey indicates 65% of Spring users prefer building agents in Spring AI/Embabel over Python/TypeScript alternatives (17%) or other JVM frameworks (18%).
The distinction matters because people conflate building with AI (integrating models into applications — the Java play) with building AI (training models — the Python play).
Already I talked about the same in my previous articles — https://medium.com/javarevisited/are-java-developers-still-relevant-in-the-age-of-ai-9d1c0d480a1a
Simply put:
When an AI agent needs to call your payment gateway at 3am with proper retry logic, circuit breakers, type-safe validation, and an audit trail — that’s a distributed systems problem. It always was. The model is a component in that system, not the system itself.
The agents of 2026 are calling the same enterprise APIs, hitting the same backends, running on the same infrastructure that Java has owned for two decades. The language that runs those backends is the language best positioned to serve them to autonomous consumers.
The Framework Decision: A Concrete Comparison
The question I get most — at meetups, in YouTube comments, at the MCP Dev Summit booth (100+ conversations across Bangalore and Mumbai) — is: “Spring AI or LangChain4j?”
James’s answer was blunt: pick one, go deep. The architectural conceptstool calling, RAG, structured outputs, memory, observability — are transferable.
Press enter or click to view image in full size
The framework matters less than understanding what’s underneath. If you understand how tool calling works at the protocol level (MCP), how memory strategies differ (semantic vs episodic), and how retry responsibility splits between infra and reasoning — switching between these is a weekend of refactoring, not a rewrite.
The anti-pattern I see repeatedly: Teams spending 4 weeks evaluating frameworks, building proof-of-concepts in each, writing comparison docs for leadership — and never shipping anything. The framework you ship with is better than the three you evaluated.
Embabel 1.0: Why Rod Johnson Built a Framework on Top of His Own Framework
The InfoQ coverage frames it precisely: “An analogy: Spring AI exists at the level of the Servlet API, while Embabel is more like Spring MVC.”
Rod Johnson — who created Spring in 2003 — is making the same layering bet he made 23 years ago.
What makes Embabel architecturally different:
Embabel uses Goal-Oriented Action Planning (GOAP), a technique borrowed from video game AI. Instead of wiring a static graph of nodes and edges (the LangGraph approach), you define:
- Typed goals — what you want to achieve
- Typed actions — each with preconditions and effects
- A planner — that searches for a sequence of actions reaching the goal at runtime
If the world changes mid-task (a tool fails, new data arrives), the planner reassesses and finds a new path. You don’t need to have anticipated every branch in advance.
This matters for production because agents fail mid-execution constantly. An API returns a 503. The user changes their mind. A guardrail blocks an intermediate step. With a static graph, you either crash or you need explicit error edges for every possible failure. With GOAP, the planner routes around the failure — the same way a GPS recalculates when you miss a turn.
The distinction from LangGraph4j: LangGraph requires you to define the graph upfront — which nodes exist, which edges connect them, under what conditions. Embabel’s planner discovers viable paths through typed actions at runtime. Both approaches have merit. LangGraph gives you predictability and auditability. Embabel gives you adaptability and less upfront wiring. Embabel even supports mixing both: GOAP planning alongside explicit state machines in the same agent.
Memory Is the Silent Killer of Agent Reliability
The most technically interesting segment of our conversation and the one that’s generated the most YouTube and LinkedIn comments was about agent memory architecture.
The failure mode nobody warns you about: Your agent keeps the last 20 messages in a sliding window. On message 21, the original user intent rolls off. The agent continues executing — but toward a goal it’s forgotten. No crash. No error. Just quiet drift. Your trace shows every tool call succeeded. The output is confidently wrong.
The 4-strategy architecture:
Press enter or click to view image in full size
The production pattern: Short-term memory (sliding window) handles the current conversation. Long-term memory (the 4 strategies above) handles everything that should survive across sessions. The consolidation happens asynchronously — the system extracts relevant facts without explicit developer intervention.
The anti-pattern: Using only short-term memory and increasing the window size to compensate. This burns tokens (every message is re-sent to the model) and eventually hits context limits.
The Principle I Now Repeat in Every Talk
“Don’t let the non-deterministic system make the final move.”
LLMs reason. Deterministic tools execute. The boundary between them must be type-safe, validated, and auditable.
In practice, this means your
@McpTool methods (or @Tool in LangChain4j) have:- Strict input types (not
Map<String, Object>)
- Validation annotations (
@NotNull,@Min,@Pattern)
- Structured error responses the agent can reason about
- No direct writes without validation passing first
@McpTool(description = "Transfer funds between accounts. Validates balance before executing.")public TransferResult transfer( @NotNull @Pattern(regexp = "ACC-\\d{10}") String fromAccount, @NotNull @Pattern(regexp = "ACC-\\d{10}") String toAccount, @Min(1) @Max(100000) BigDecimal amount, @NotBlank String currency) { // The agent reasoned about WHAT to transfer. // This method validates HOW — deterministically. // If validation fails, the structured error tells the agent why.}This is just dependency injection and interface contracts — the things Java developers have been doing for 20 years. The reasoning is new. The execution boundary isn’t.
What This Means for Your Career in 2026
The Java AI stack crystallized in 90 days. Before May 2026, it was emerging. Now it’s production-ready.
The decision tree:
- Already on Spring Boot? → Spring AI 2.0. No contest. Annotation-driven, advisor-chain composable, MCP built in.
- Want high-level goal-driven agents? → Embabel 1.0 (on top of Spring AI). GOAP replanning, typed domain objects.
- Not on Spring? → LangChain4j 1.18. BDI pattern, broadest provider support, crash-resilient HITL.
- Kotlin-first? → Koog 1.0. Multiplatform, type-safe DSL, A2A support.
- Need managed deployment? → Spring AI AgentCore SDK. One annotation → managed runtime, memory, observability, scaling.
But the framework is 20% of the decision.
The 80% is: Do you understand how MCP tool discovery works? Do you understand the 4 memory strategies? Do you understand where retry responsibility lives (infra vs reasoning)?
If yes — you can build production agents in any of these frameworks by next week.
If no — learning a second framework teaches you nothing new.
The full conversation goes deeper into each of these areas. Timestamps in the video description — jump to what matters most to you.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article