From Rule-Based Routing to Tool Chains

In our previous post (Accuracy Experiment #1 of Function Calling–Based AI Agents), we introduced our first experiment applying Function Calling to AI agents. At that stage, the system relied on loading the entire enterprise asset list and letting the AI search internally. While straightforward, this approach quickly hit limitations: excessive token usage, scalability bottlenecks, and inconsistent user experience.

For our second experiment, we shifted gears. We initially introduced rule-based routing to improve accuracy, but ultimately evolved the design toward a Tool Chain architecture. This shift significantly enhanced accuracy, scalability, and user experience. In this article, we walk through that journey.

1. Limitations of the First Experiment

The initial structure was simple:

• User asks a question → Load full asset list → AI searches internally → Return result

However, in real-world enterprise environments, several problems surfaced:

• Excessive token usage: Input length grew exponentially as asset volume increased, driving up costs
• Data size constraints: Worked only up to ~500 assets; beyond that, the model broke down
• Lack of scalability: Enterprises managing thousands of assets could not use the approach effectively

In short, the answers were technically correct but structurally inefficient—unsuitable for scaling.

2. The Rule-Based Approach

2-1. What is Rule-Based Routing?

Rule-based routing relies on manually defined rules to determine which API the AI should call.

• “Whose asset is this?” → Call Asset Search API
• “Show me last quarter’s cost savings” → Call Depreciation API + Usage API

2-2. Benefits

• Clear and predictable behavior
• Fast and reliable for simple queries

2-3. Limitations

• Management overhead increases as query patterns grow
• Rules must be updated whenever new features are added
• The AI simply executes rules, without real reasoning or flexible composition

Rule-based routing solved part of the problem but proved restrictive in terms of flexibility and scalability.

3. The Tool Chain Approach

3-1. What is a Tool Chain?

A Tool Chain lets the AI decide which Tool (Function) to call, invoke it, and then synthesize a structured response. Instead of hard-coded rules, the AI orchestrates tools dynamically to resolve user queries.

3-2. How it Works

1. User asks: “Which laptops are unused at the Seoul branch?”
2. AI interprets the natural language and maps it to Tool parameters
3. AssetSearch Tool is invoked → Returns results
4. AI processes the results and generates a clean, structured answer

3-3. Advantages

• Flexibility: Add a new Tool and the AI can leverage it without rewriting rules
• Scalability: Handles thousands or tens of thousands of assets reliably
• Consistency: Tool schema definitions enforce standardized terminology and output
• Enhanced UX: Supports streaming output and well-organized responses

4. Rule-Based vs. Tool Chain

‍

‍

👉 In short: Rule-based routing is stable but rigid; Tool Chains are more intelligent and adaptive.

5. User Experience Improvements

Transitioning to a Tool Chain also brought major UX enhancements:

• Streaming responses: Instead of waiting for the AI to finish computing, users see answers flow in real time
• Structured formatting: Answers are presented with bullet points, sections, and dividers—clear and professional
• Context awareness: If an answer is reused from previous dialogue, the AI now adds a disclaimer (e.g., “This response is based on prior context. For the latest data, please re-ask using the word ‘current’ or ‘again’.”)

The result is a user experience that feels faster, clearer, and more trustworthy.

6. Key Outcomes

By moving to a Tool Chain architecture, we achieved:

1. Higher accuracy – Terminology and expressions remain consistent
2. True scalability – Reliable performance with large enterprise datasets
3. Cost efficiency – Significant reduction in token consumption
4. Improved UX – Real-time responses with structured, easy-to-read formatting

Conclusion

Accuracy Experiment #2 of Function Calling–Based AI Agents went far beyond solving the “token overload” challenge. Starting from a rule-based approach and evolving into a full Tool Chain architecture, we achieved simultaneous gains in accuracy, scalability, and user experience.

At SELLEASE, we will continue advancing our AI agent architecture to help enterprises leverage their vast asset data more intelligently, more efficiently, and more intuitively. 🚀

‍

‍