One gateway for 83 tools
I re-architected WYNI’s agent tool integration around an MCP gateway. The agent sees three meta-tools and loads a tool’s schema only when it needs it.
The platform runs AI agents that work inside a company’s own systems. They read the CRM, search internal documents and file tickets. Each system is connected over the Model Context Protocol (MCP), an open standard that lets a system publish a set of tools an agent can call. A user picks which connected systems, or MCP servers, an agent may use in a conversation.
To call a tool, the model needs that tool’s schema: its name, a description of what it does, and a JSON definition of the inputs it accepts. Schemas are sent to the model as part of the prompt, so every schema costs input tokens.
1The cost of 83 tools
A typical agent setup had 83 tools across its selected MCP servers. Before the change, the platform loaded all 83 schemas into the prompt at the start of every turn, meaning each user message and the agent’s work to answer it, whether or not the task needed them.
The user’s actual request was the smallest part of the prompt. Most of the input was the manual for 83 tools, and a typical task used one or two of them. Every task paid for every tool.
2Before and after
- Tokens. All 83 schemas were in the prompt on every model call.
- A hard ceiling. A turn could have at most 100 tools. One more server could push a working setup over the limit.
- Fragility. Discovery was all-or-nothing. If one server was slow or down, the whole turn failed, even if the task never needed it.
list_mcp_toolsget_mcp_toolcall_mcp_tool- Tokens. The prompt holds three schemas, plus one line per selected server: its ID, name and a one-sentence description.
- No ceiling. Three schemas, however many tools the servers publish.
- No up-front failure. A server is contacted the first time the agent uses it, so a server that is down only affects tasks that need it.
| Meta-tool | What it does | What it returns |
|---|---|---|
list_mcp_tools | Lists the tools on one server | Tool names and a short description of each (up to 250 characters). No schemas. |
get_mcp_tool | Loads the full schema for the tools the agent picked | Full descriptions and input definitions, for up to 10 tools at a time |
call_mcp_tool | Runs one tool | The tool’s result |
Server descriptions are now required when a server is added, because the agent relies on them to choose a server before it lists any tools.
3How one task runs
A single task against a CRM server, lightly sanitised. Each meta-tool call also takes a purpose: a few words on why the agent is taking that step. The user sees it in the interface, and it is stored with the call.
- user
What plan is customer 123 on, and when does it renew?
- agent
list_mcp_tools- purpose
- "Find a customer lookup tool"
- server_id
- "acme_crm"
lookup_customer Look up one customer by ID… search_customers Search customers by name or email… list_invoices List invoices for a customer… … names and short descriptions only
- agent
get_mcp_tool- purpose
- "Check lookup inputs"
- server_id
- "acme_crm"
- tool
- ["lookup_customer"]
lookup_customer customer_id: string, required
- agent
call_mcp_tool- purpose
- "Look up customer 123"
- server_id
- "acme_crm"
- tool
- "lookup_customer"
- arguments_json
- "{\"customer_id\": \"123\"}"
{ "plan": "Business", "renews_on": "2027-03-01" } - agent
Customer 123 is on the Business plan. It renews on 1 March 2027.
The sequence is find tools, load schema, call tool, result. Only one full schema, lookup_customer, entered the conversation.
- Arguments are checked before the call. The gateway validates the agent’s arguments against the tool’s schema. If they don’t match, the agent gets an error naming the problem and can try again.
- Some servers need a person’s approval. For those,
call_mcp_toolpauses until someone approves the action. Listing tools and loading schemas never need approval, because they don’t change anything. - Discovery is remembered within a turn. A second call to the same server reuses what the gateway already found, and so does a turn that resumes after an approval.
4The eval before shipping
The obvious risk was accuracy. With all 83 schemas in view, the agent could compare tools side by side. Now it chooses from short descriptions and loads the details afterwards. If two tools sound alike, it might load the wrong one and never see the right one.
So before shipping, we built an eval of 60 tasks with known correct answers. 40 of them were written to be confusable: each has a wrong tool that looks like a reasonable choice. A task passed only if the agent called the right tool on the right server.
Two searches over different data
search_customers on the CRM finds customer records. search_documents on the document store finds files that mention a customer. Both “search for Acme”.
Read versus write on the same object
get_ticket and update_ticket take the same ticket ID. “Check the status of ticket 512” must not change it.
confusable case (40) standard case (20)
5Result
- Input tokens
- −92%+
- Cost
- −70%+
- Wrong tool calls
- 0 of 60
Cost falls less than input tokens for two reasons. The agent’s output, its reasoning and answers, doesn’t shrink. And the extra listing and loading steps each add a small model call of their own.
6Trade-offs
Extra round trips
The first time the agent uses a server in a turn, it makes up to two extra model calls before the real one: one to list the tools and one to load a schema. Each adds the time of one short model response.
The cost doesn’t grow with every call. A server’s tools are listed once per turn and each schema is loaded once. When the agent already knows the tool and its inputs, it can go straight to call_mcp_tool, and its arguments are still checked. For multi-step jobs against company systems, that was a fair price for prompts about a tenth of the size.
Why not split into specialised agents?
A CRM agent, a documents agent and so on, with a router in front, moves the problem rather than solving it. The router has to choose between agents, so the confusable cases become routing errors. Each handoff passes the task in summary and loses context, and approvals would have to work across several agents. One agent kept the task easier to follow and the approval rules simpler to enforce.
Why not retrieve tools by embedding?
Fetching the closest few tool descriptions for each request fails where it matters most. Two search tools over different data have nearly identical descriptions, so they land next to each other. Retrieval also runs before the model has thought about the task, and when it misses the right tool nothing signals the miss. And it needs every tool indexed in advance, which brings back the up-front connection to every server.