AI Experience

There’s a philosophical debate, particularly in the community of people building MCP servers, about tool calling abstraction. A simplification of the argument would be one side says tool calls should be high level, abstracting away the internals of an application, while the other side says that tool calls can and should look like application-level function calls. The high abstraction adherents have several reasons behind their view: We already have APIs, why have MCP or any other tool calling framework if tool calls are just API calls? Code-like calls expose internals of systems to LLMs. LLMs aren’t great at structured output, so let’s keep it simple. Context and token management suggest that simpler is better. The tool calls as API calls side has a bit of a practical bent - to get the LLM to do what we want it to do, it requires the LLM to utilize all this stuff we’ve already built. Models are getting better at structured output and tokens are getting cheaper. So the concerns of the abstraction side are time-bound. 

But I think the two sides belie the point really. I think a better way to think about it is, “who is your LLM talking to when it makes a tool call?” That flips the question on its head. If it’s another LLM or ultimately a person, then the high abstraction tool call makes sense. Canva does this beautifully. When your client talks to the Canva MCP server, it’s prompting a specialized (probably fine-tuned) agent to make Canva designs. Rather than expose the complexity of pushing pixels and aligning objects, Canva MCP says, you specify and our agent will work with our systems to generate designs. If you’re talking to a more traditional system, then speak the language of the system. I suspect Canva’s internal LLM is making much more sophisticated API-type tool calls. If you’re talking to a software system that has an existing API, then tool calls should make an LLM a very sophisticated user of that API. Ultimately, the decision isn’t about tokens, context management, or application internals. It’s about experience. Human user experience first and foremost for the humans working with the LLMs, but also user experience from an LLM’s perspective or ‘AI experience’. How can you design your tool calls so that LLMs/agents can most effectively get stuff done?

While this is a welcome, great feature, relying on it will create platform lock-in. You can’t switch to fit for purpose models. Although everyone will eventually catchup, the better approach is to implement a quasi-linter for your complex tool calls. Linters are well understood, easy to create, and most models respond well to the feedback. You then control the quality of structured output downstream regardless of the model.