Episode 4 — Generative AI Engineering / 4.7 — Function Calling Tool Calling
4.7.e --- Building an AI Tool Router
In one sentence: A production-grade AI tool router defines multiple tools, lets the AI pick which to call based on user intent, executes the function with full error handling and validation, returns the result through a logging pipeline, and handles edge cases like unknown functions, malformed arguments, multi-turn conversations, and chained tool calls.
Navigation: <- 4.7.d --- Hybrid Logic | 4.7 Overview
1. What We Are Building
In this section we build a complete, production-ready AI tool router for a dating app assistant. The system:
- Defines 5 tools (3 core + 2 utility)
- Routes user messages to the right tool using AI
- Validates all inputs and outputs
- Handles errors gracefully
- Logs every interaction for debugging and analytics
- Supports multi-turn conversations
- Handles parallel tool calls
- Includes retry logic
+------------------------------------------------------------------------+
| COMPLETE AI TOOL ROUTER |
| |
| User Message |
| | |
| v |
| +------------------+ |
| | Input Validator | -- rejects empty/too-long messages |
| +--------+---------+ |
| | |
| v |
| +------------------+ |
| | LLM Router | -- decides which tool(s) to call |
| +--+---+---+---+---+ |
| | | | | | |
| v v v v v |
| +----+----+----+----+----+ |
| | Bio|Open|Mod |Prof|Help| <-- Tool handlers |
| +----+----+----+----+----+ |
| | | | | | |
| v v v v v |
| +------------------+ |
| | Result Logger | -- logs tool call + result for analytics |
| +--------+---------+ |
| | |
| v |
| +------------------+ |
| | LLM Formatter | -- turns result into natural response |
| +--------+---------+ |
| | |
| v |
| Final response to user |
+------------------------------------------------------------------------+
2. Project Structure
dating-assistant/
index.js -- Entry point and router
tools.js -- Tool definitions (JSON schemas)
handlers.js -- Function implementations
logger.js -- Logging utility
validator.js -- Input/output validation
config.js -- Configuration
3. Configuration
// config.js
export const config = {
model: 'gpt-4o',
routerTemperature: 0, // Deterministic routing decisions
generationTemperature: 0.8, // Creative for content generation
maxRetries: 2,
maxConversationTokens: 8000,
maxToolResultLength: 4000, // Characters, not tokens
logLevel: 'info', // 'debug' | 'info' | 'warn' | 'error'
};
4. Tool Definitions
// tools.js
export const tools = [
// ---- Core tools ----
{
type: 'function',
function: {
name: 'improveBio',
description:
'Improve a user\'s dating profile bio to be more engaging and authentic. ' +
'Call when the user wants to rewrite, enhance, or fix their profile bio.',
parameters: {
type: 'object',
properties: {
currentBio: {
type: 'string',
description: 'The user\'s current bio text',
},
tone: {
type: 'string',
enum: ['witty', 'sincere', 'adventurous', 'intellectual'],
description: 'Desired tone for the improved bio',
},
},
required: ['currentBio'],
additionalProperties: false,
},
},
},
{
type: 'function',
function: {
name: 'generateOpeners',
description:
'Generate conversation opener messages for starting a chat with a dating match. ' +
'Call when the user wants icebreakers, opening lines, or conversation starters.',
parameters: {
type: 'object',
properties: {
profileDescription: {
type: 'string',
description: 'Info about the match\'s profile or interests',
},
count: {
type: 'number',
description: 'Number of openers to generate (1-5, default 3)',
},
style: {
type: 'string',
enum: ['funny', 'thoughtful', 'flirty', 'casual'],
description: 'Style of the openers',
},
},
required: ['profileDescription'],
additionalProperties: false,
},
},
},
{
type: 'function',
function: {
name: 'moderateText',
description:
'Check if a message is appropriate for a dating platform. ' +
'Call when the user asks to review, check, or verify a message before sending.',
parameters: {
type: 'object',
properties: {
text: {
type: 'string',
description: 'The message text to check',
},
},
required: ['text'],
additionalProperties: false,
},
},
},
// ---- Utility tools ----
{
type: 'function',
function: {
name: 'getProfileTips',
description:
'Get specific tips for improving a dating profile based on a category. ' +
'Call when the user asks for general advice about profiles, photos, or messaging.',
parameters: {
type: 'object',
properties: {
category: {
type: 'string',
enum: ['bio', 'photos', 'messaging', 'general'],
description: 'Category of tips to retrieve',
},
},
required: ['category'],
additionalProperties: false,
},
},
},
{
type: 'function',
function: {
name: 'analyzeProfile',
description:
'Analyze a dating profile and provide a score with improvement suggestions. ' +
'Call when the user wants feedback on their overall profile or a profile audit.',
parameters: {
type: 'object',
properties: {
bio: {
type: 'string',
description: 'The profile bio text',
},
photoCount: {
type: 'number',
description: 'Number of photos on the profile',
},
hasPrompts: {
type: 'boolean',
description: 'Whether the profile has conversation prompt answers',
},
},
required: ['bio'],
additionalProperties: false,
},
},
},
];
5. Input/Output Validation
// validator.js
/**
* Validates user message before sending to the router.
*/
export function validateUserInput(message) {
if (!message || typeof message !== 'string') {
return { valid: false, error: 'Message must be a non-empty string' };
}
if (message.trim().length === 0) {
return { valid: false, error: 'Message cannot be empty' };
}
if (message.length > 5000) {
return { valid: false, error: 'Message too long. Maximum 5000 characters.' };
}
return { valid: true };
}
/**
* Safely parses tool call arguments.
* Returns a structured result instead of throwing.
*/
export function safeParseArguments(argumentsString) {
try {
const parsed = JSON.parse(argumentsString);
if (typeof parsed !== 'object' || parsed === null) {
return { success: false, error: 'Arguments must be a JSON object' };
}
return { success: true, data: parsed };
} catch (error) {
return {
success: false,
error: `Invalid JSON in tool arguments: ${error.message}`,
raw: argumentsString,
};
}
}
/**
* Validates that a tool result is safe to send back to the model.
* Prevents excessively large results from consuming too many tokens.
*/
export function validateToolResult(result, maxLength = 4000) {
const serialized = typeof result === 'string' ? result : JSON.stringify(result);
if (serialized.length > maxLength) {
return {
valid: false,
truncated: serialized.slice(0, maxLength),
warning: `Result truncated from ${serialized.length} to ${maxLength} characters`,
};
}
return { valid: true, serialized };
}
6. Logging
// logger.js
const LOG_LEVELS = { debug: 0, info: 1, warn: 2, error: 3 };
class ToolLogger {
constructor(level = 'info') {
this.level = LOG_LEVELS[level] || 1;
this.logs = []; // In-memory store; replace with your logging service
}
/**
* Log a tool router event.
*/
log(level, event, data = {}) {
if (LOG_LEVELS[level] < this.level) return;
const entry = {
timestamp: new Date().toISOString(),
level,
event,
...data,
};
this.logs.push(entry);
console.log(`[${entry.timestamp}] [${level.toUpperCase()}] ${event}`, data);
}
/**
* Log a complete tool call cycle.
*/
logToolCall({ userMessage, toolName, toolArgs, result, durationMs, error }) {
this.log('info', 'tool_call', {
userMessage: userMessage.slice(0, 200),
toolName,
toolArgs,
resultPreview: typeof result === 'string'
? result.slice(0, 200)
: JSON.stringify(result).slice(0, 200),
durationMs,
success: !error,
error: error ? error.message || error : undefined,
});
}
/**
* Log routing decision (which tool was chosen).
*/
logRoutingDecision({ userMessage, chosenTool, allToolCalls, durationMs }) {
this.log('info', 'routing_decision', {
userMessage: userMessage.slice(0, 200),
chosenTool,
toolCallCount: allToolCalls?.length || 0,
durationMs,
});
}
/**
* Log when no tool was called (direct text response).
*/
logDirectResponse({ userMessage, durationMs }) {
this.log('debug', 'direct_response', {
userMessage: userMessage.slice(0, 200),
durationMs,
});
}
/**
* Get all logs (for debugging).
*/
getLogs() {
return [...this.logs];
}
}
export const logger = new ToolLogger('info');
7. Function Handlers
// handlers.js
import OpenAI from 'openai';
import { config } from './config.js';
const openai = new OpenAI();
// ======================================================================
// improveBio
// ======================================================================
export async function improveBio({ currentBio, tone = 'witty' }) {
// --- Validation ---
if (!currentBio || currentBio.trim().length === 0) {
return { success: false, error: 'Bio text cannot be empty.' };
}
if (currentBio.length > 2000) {
return { success: false, error: 'Bio too long. Maximum 2000 characters.' };
}
// --- Banned word filter ---
const bannedPatterns = [
/\b(snapchat|instagram|tiktok)\b/i,
/\b\d{3}[-.]?\d{3}[-.]?\d{4}\b/,
/\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b/,
];
for (const pattern of bannedPatterns) {
if (pattern.test(currentBio)) {
return {
success: false,
error: 'Bio contains disallowed content (social handles or contact info). Remove it first.',
};
}
}
// --- AI-assisted bio generation ---
const toneGuide = {
witty: 'Use humor and clever wordplay. Be playful but not cheesy.',
sincere: 'Be warm, genuine, and emotionally honest.',
adventurous: 'Emphasize excitement, exploration, and spontaneity.',
intellectual: 'Highlight curiosity, depth, and thoughtfulness.',
};
const response = await openai.chat.completions.create({
model: config.model,
temperature: config.generationTemperature,
messages: [
{
role: 'system',
content:
`You are an expert dating profile writer. Rewrite the bio below.\n\n` +
`Tone: ${tone} - ${toneGuide[tone] || toneGuide.witty}\n\n` +
`Rules:\n` +
`- Maximum 500 characters\n` +
`- Be specific and authentic\n` +
`- Avoid cliches ("love to laugh", "partner in crime")\n` +
`- No contact information\n` +
`- Return ONLY the new bio text`,
},
{ role: 'user', content: currentBio },
],
});
let improvedBio = response.choices[0].message.content.trim();
// --- Deterministic post-processing ---
// Remove quotes if the AI wrapped the bio in them
if (improvedBio.startsWith('"') && improvedBio.endsWith('"')) {
improvedBio = improvedBio.slice(1, -1);
}
// Enforce character limit
improvedBio = improvedBio.slice(0, 500);
return {
success: true,
original: currentBio,
improved: improvedBio,
tone,
characterCount: improvedBio.length,
maxCharacters: 500,
};
}
// ======================================================================
// generateOpeners
// ======================================================================
export async function generateOpeners({
profileDescription,
count = 3,
style = 'casual',
}) {
// --- Validation ---
if (!profileDescription || profileDescription.trim().length === 0) {
return { success: false, error: 'Profile description cannot be empty.' };
}
const clampedCount = Math.min(Math.max(Math.round(count), 1), 5);
// --- AI-assisted opener generation ---
const response = await openai.chat.completions.create({
model: config.model,
temperature: config.generationTemperature,
messages: [
{
role: 'system',
content:
`Generate exactly ${clampedCount} ${style} conversation openers ` +
`for a dating app based on the profile below.\n\n` +
`Rules:\n` +
`- Each opener under 200 characters\n` +
`- Ask a genuine question showing interest\n` +
`- No comments on physical appearance\n` +
`- No generic greetings ("Hey", "Hi there")\n` +
`- Return a JSON object: { "openers": ["...", "..."] }`,
},
{ role: 'user', content: profileDescription },
],
response_format: { type: 'json_object' },
});
// --- Parse and validate ---
let openers;
try {
const parsed = JSON.parse(response.choices[0].message.content);
openers = Array.isArray(parsed.openers) ? parsed.openers : [];
} catch {
return { success: false, error: 'Failed to generate openers. Please try again.' };
}
// --- Deterministic post-processing ---
const cleanOpeners = openers
.slice(0, clampedCount)
.map((o) => String(o).trim().slice(0, 200))
.filter((o) => o.length > 10); // Remove very short/empty entries
if (cleanOpeners.length === 0) {
return { success: false, error: 'Could not generate valid openers. Please try again.' };
}
return {
success: true,
openers: cleanOpeners,
count: cleanOpeners.length,
style,
};
}
// ======================================================================
// moderateText
// ======================================================================
export function moderateText({ text }) {
// --- Validation ---
if (!text || text.trim().length === 0) {
return { success: false, error: 'Text cannot be empty.' };
}
// --- Purely deterministic checks ---
const issues = [];
// Phone numbers
if (/\b\d{3}[-.\s]?\d{3}[-.\s]?\d{4}\b/.test(text)) {
issues.push({ type: 'personal_info', detail: 'Phone number detected', severity: 'high' });
}
// Email addresses
if (/\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b/.test(text)) {
issues.push({ type: 'personal_info', detail: 'Email address detected', severity: 'high' });
}
// Social media handles
if (/(?:^|\s)@[A-Za-z0-9_]{2,}\b/.test(text)) {
issues.push({ type: 'social_media', detail: 'Social media handle detected', severity: 'medium' });
}
// Payment platforms
if (/\b(venmo|cashapp|paypal|zelle|crypto)\b/i.test(text)) {
issues.push({ type: 'financial', detail: 'Payment platform reference', severity: 'high' });
}
// URLs
if (/https?:\/\/[^\s]+/.test(text) || /www\.[^\s]+/.test(text)) {
issues.push({ type: 'external_link', detail: 'External URL detected', severity: 'medium' });
}
// Excessive caps
const capsRatio = text.length > 10
? (text.match(/[A-Z]/g) || []).length / text.replace(/\s/g, '').length
: 0;
if (capsRatio > 0.7) {
issues.push({ type: 'tone', detail: 'Excessive capitalization', severity: 'low' });
}
// Very short messages (potential low-effort)
if (text.trim().length < 5) {
issues.push({ type: 'quality', detail: 'Message is very short', severity: 'low' });
}
const highSeverity = issues.filter((i) => i.severity === 'high');
return {
success: true,
text,
safe: highSeverity.length === 0,
issueCount: issues.length,
issues,
summary:
issues.length === 0
? 'Message looks good to send!'
: highSeverity.length > 0
? `Found ${highSeverity.length} serious issue(s) that should be fixed before sending.`
: `Found ${issues.length} minor issue(s). Consider revising.`,
};
}
// ======================================================================
// getProfileTips
// ======================================================================
export function getProfileTips({ category }) {
const tipDatabase = {
bio: {
tips: [
'Lead with your most unique trait -- what makes you different from everyone else?',
'Show, don\'t tell. Instead of "I\'m funny," write something that makes them laugh.',
'Include a conversation hook -- a question or statement that invites a response.',
'Keep it under 500 characters. People skim on dating apps.',
'Mention specific interests, not generic ones. "Sunday morning crossword puzzles" beats "I like puzzles."',
],
category: 'bio',
},
photos: {
tips: [
'Your first photo should be a clear, solo headshot with a genuine smile.',
'Include at least one full-body photo and one activity photo.',
'Avoid group photos where it\'s hard to tell which person you are.',
'Natural lighting looks better than harsh flash or heavy filters.',
'Variety matters: show different sides of yourself (casual, dressed up, doing a hobby).',
],
category: 'photos',
},
messaging: {
tips: [
'Reference something specific from their profile -- it shows you read it.',
'Ask open-ended questions, not yes/no questions.',
'Keep first messages short (1-2 sentences). Save the long messages for later.',
'Avoid generic openers like "Hey" or "What\'s up?" -- they get lost in the crowd.',
'Be genuine rather than trying to be impressive.',
],
category: 'messaging',
},
general: {
tips: [
'Profiles with 4-6 photos get the most matches.',
'Bio, photos, and prompt answers all matter -- don\'t skip any.',
'Update your profile seasonally to keep it fresh.',
'Be honest about what you\'re looking for -- it saves everyone time.',
'A complete profile signals effort, which signals genuine interest.',
],
category: 'general',
},
};
const tips = tipDatabase[category] || tipDatabase.general;
return { success: true, ...tips };
}
// ======================================================================
// analyzeProfile
// ======================================================================
export function analyzeProfile({ bio, photoCount = 0, hasPrompts = false }) {
// --- Scoring algorithm ---
let score = 0;
const suggestions = [];
// Bio scoring (max 40 points)
if (bio && bio.trim().length > 0) {
score += 10; // Has a bio at all
if (bio.length >= 50) score += 10; // Reasonable length
if (bio.length >= 100) score += 10; // Good length
if (bio.length <= 500) score += 5; // Not too long
if (/\?/.test(bio)) score += 5; // Contains a question (conversation hook)
} else {
suggestions.push('Add a bio! An empty profile gets far fewer matches.');
}
if (bio && bio.length < 50) {
suggestions.push('Your bio is very short. Aim for at least 100 characters.');
}
if (bio && bio.length > 500) {
suggestions.push('Your bio is quite long. Consider trimming to under 500 characters.');
}
if (bio && !/\?/.test(bio)) {
suggestions.push('Add a question or conversation hook to your bio to invite responses.');
}
// Photo scoring (max 30 points)
if (photoCount >= 1) score += 10;
if (photoCount >= 3) score += 10;
if (photoCount >= 5) score += 10;
if (photoCount === 0) {
suggestions.push('Add photos! Profiles without photos are almost never swiped right.');
} else if (photoCount < 3) {
suggestions.push(`You have ${photoCount} photo(s). Aim for at least 4-6 for best results.`);
} else if (photoCount > 8) {
suggestions.push('Consider trimming to your best 5-6 photos. Quality over quantity.');
}
// Prompts scoring (max 20 points)
if (hasPrompts) {
score += 20;
} else {
suggestions.push('Fill in conversation prompts/answers -- they help people find things to talk about.');
}
// Completeness bonus (max 10 points)
if (bio && bio.length >= 50 && photoCount >= 3 && hasPrompts) {
score += 10;
}
const grade =
score >= 90 ? 'A' :
score >= 75 ? 'B' :
score >= 60 ? 'C' :
score >= 40 ? 'D' : 'F';
return {
success: true,
score,
maxScore: 100,
grade,
suggestions: suggestions.length > 0 ? suggestions : ['Great profile! Keep it updated.'],
breakdown: {
bio: Math.min(40, score), // Simplified breakdown
photos: Math.min(30, photoCount >= 5 ? 30 : photoCount >= 3 ? 20 : photoCount >= 1 ? 10 : 0),
prompts: hasPrompts ? 20 : 0,
completeness: bio && bio.length >= 50 && photoCount >= 3 && hasPrompts ? 10 : 0,
},
};
}
// ======================================================================
// Handler registry
// ======================================================================
export const handlerMap = {
improveBio,
generateOpeners,
moderateText,
getProfileTips,
analyzeProfile,
};
8. The Router (Main Entry Point)
// index.js
import OpenAI from 'openai';
import { tools } from './tools.js';
import { handlerMap } from './handlers.js';
import { validateUserInput, safeParseArguments, validateToolResult } from './validator.js';
import { logger } from './logger.js';
import { config } from './config.js';
const openai = new OpenAI();
const SYSTEM_PROMPT =
'You are a friendly, knowledgeable dating app assistant. ' +
'Help users improve their profiles, craft messages, and navigate online dating. ' +
'Use the available tools when the user\'s request matches a tool\'s purpose. ' +
'For general conversation or questions, respond directly without tools. ' +
'Be encouraging but honest.';
/**
* Execute a single tool call with error handling.
*/
async function executeToolCall(toolCall) {
const startTime = Date.now();
const fnName = toolCall.function.name;
// Validate function exists
if (!handlerMap[fnName]) {
logger.log('error', 'unknown_function', { functionName: fnName });
return {
role: 'tool',
tool_call_id: toolCall.id,
content: JSON.stringify({
success: false,
error: `Unknown function: ${fnName}. Available functions: ${Object.keys(handlerMap).join(', ')}`,
}),
};
}
// Parse arguments
const parsed = safeParseArguments(toolCall.function.arguments);
if (!parsed.success) {
logger.log('error', 'argument_parse_error', {
functionName: fnName,
error: parsed.error,
raw: parsed.raw,
});
return {
role: 'tool',
tool_call_id: toolCall.id,
content: JSON.stringify({
success: false,
error: `Failed to parse arguments: ${parsed.error}`,
}),
};
}
// Execute the function
try {
const result = await handlerMap[fnName](parsed.data);
const durationMs = Date.now() - startTime;
// Validate result size
const validated = validateToolResult(result, config.maxToolResultLength);
const content = validated.valid ? validated.serialized : validated.truncated;
if (!validated.valid) {
logger.log('warn', 'result_truncated', {
functionName: fnName,
warning: validated.warning,
});
}
logger.logToolCall({
userMessage: '', // Filled by caller
toolName: fnName,
toolArgs: parsed.data,
result,
durationMs,
});
return {
role: 'tool',
tool_call_id: toolCall.id,
content,
};
} catch (error) {
const durationMs = Date.now() - startTime;
logger.logToolCall({
userMessage: '',
toolName: fnName,
toolArgs: parsed.data,
result: null,
durationMs,
error,
});
return {
role: 'tool',
tool_call_id: toolCall.id,
content: JSON.stringify({
success: false,
error: `Function execution failed: ${error.message}`,
}),
};
}
}
/**
* Process a single round of tool calls (handles parallel calls).
*/
async function processToolCalls(assistantMessage) {
const toolCalls = assistantMessage.tool_calls || [];
if (toolCalls.length === 0) return [];
// Execute all tool calls in parallel
const results = await Promise.all(
toolCalls.map((tc) => executeToolCall(tc))
);
return results;
}
/**
* Main router: takes a user message, routes to the right tool,
* executes, and returns a formatted response.
*
* Supports multi-turn via conversationHistory parameter.
*/
export async function router(userMessage, conversationHistory = []) {
// --- Input validation ---
const inputCheck = validateUserInput(userMessage);
if (!inputCheck.valid) {
return { response: inputCheck.error, toolsCalled: [], error: true };
}
// --- Build messages ---
const messages = [
{ role: 'system', content: SYSTEM_PROMPT },
...conversationHistory,
{ role: 'user', content: userMessage },
];
// --- Routing: Ask the model what to do ---
const routeStart = Date.now();
let routingResponse;
try {
routingResponse = await openai.chat.completions.create({
model: config.model,
messages,
tools,
tool_choice: 'auto',
temperature: config.routerTemperature,
});
} catch (error) {
logger.log('error', 'routing_api_error', { error: error.message });
return {
response: 'Sorry, I encountered an error. Please try again.',
toolsCalled: [],
error: true,
};
}
const assistantMessage = routingResponse.choices[0].message;
const finishReason = routingResponse.choices[0].finish_reason;
const routeDurationMs = Date.now() - routeStart;
// --- Direct text response (no tools needed) ---
if (finishReason !== 'tool_calls') {
logger.logDirectResponse({ userMessage, durationMs: routeDurationMs });
return {
response: assistantMessage.content,
toolsCalled: [],
conversationHistory: [
...conversationHistory,
{ role: 'user', content: userMessage },
assistantMessage,
],
};
}
// --- Tool calls detected ---
const toolNames = assistantMessage.tool_calls.map((tc) => tc.function.name);
logger.logRoutingDecision({
userMessage,
chosenTool: toolNames.join(', '),
allToolCalls: assistantMessage.tool_calls,
durationMs: routeDurationMs,
});
// --- Execute tools ---
const toolResults = await processToolCalls(assistantMessage);
// --- Get final formatted response ---
let finalResponse;
try {
finalResponse = await openai.chat.completions.create({
model: config.model,
messages: [...messages, assistantMessage, ...toolResults],
tools,
temperature: 0.7,
});
} catch (error) {
logger.log('error', 'final_response_api_error', { error: error.message });
// Fallback: return raw tool results
const fallback = toolResults
.map((r) => {
const content = JSON.parse(r.content);
return content.error || JSON.stringify(content, null, 2);
})
.join('\n');
return {
response: `Here are the results:\n${fallback}`,
toolsCalled: toolNames,
error: true,
};
}
// --- Check if the model wants to call MORE tools (chained calls) ---
const followUpMessage = finalResponse.choices[0].message;
if (finalResponse.choices[0].finish_reason === 'tool_calls') {
// Recursive: handle chained tool calls
const chainedResults = await processToolCalls(followUpMessage);
const chainedFinal = await openai.chat.completions.create({
model: config.model,
messages: [
...messages,
assistantMessage,
...toolResults,
followUpMessage,
...chainedResults,
],
tools,
});
return {
response: chainedFinal.choices[0].message.content,
toolsCalled: [
...toolNames,
...followUpMessage.tool_calls.map((tc) => tc.function.name),
],
conversationHistory: [
...conversationHistory,
{ role: 'user', content: userMessage },
assistantMessage,
...toolResults,
followUpMessage,
...chainedResults,
chainedFinal.choices[0].message,
],
};
}
return {
response: followUpMessage.content,
toolsCalled: toolNames,
conversationHistory: [
...conversationHistory,
{ role: 'user', content: userMessage },
assistantMessage,
...toolResults,
followUpMessage,
],
};
}
9. Using the Router
// main.js
import { router } from './index.js';
// ---- Single message ----
async function singleMessage() {
const result = await router('My bio says "I like dogs." Make it better!');
console.log('Response:', result.response);
console.log('Tools called:', result.toolsCalled);
}
// ---- Multi-turn conversation ----
async function conversation() {
let history = [];
// Turn 1
const turn1 = await router('Improve my bio: "I like hiking and coffee"', history);
console.log('Turn 1:', turn1.response);
history = turn1.conversationHistory || [];
// Turn 2
const turn2 = await router('Now make it more sincere instead of witty', history);
console.log('Turn 2:', turn2.response);
history = turn2.conversationHistory || [];
// Turn 3: Different tool
const turn3 = await router(
'Great! Now help me message someone who likes photography and hiking',
history
);
console.log('Turn 3:', turn3.response);
history = turn3.conversationHistory || [];
// Turn 4: No tool needed
const turn4 = await router('Thanks for all the help!', history);
console.log('Turn 4:', turn4.response);
}
// ---- Parallel tool calls ----
async function parallelCalls() {
const result = await router(
'Two things: improve my bio "I enjoy reading" and also check ' +
'if this message is safe: "Hey, text me at 555-1234"'
);
console.log('Response:', result.response);
console.log('Tools called:', result.toolsCalled);
// Expected: ['improveBio', 'moderateText']
}
// ---- Profile analysis ----
async function profileAudit() {
const result = await router(
'Can you analyze my profile? My bio is "Love traveling and trying new foods. ' +
'Looking for someone who doesn\'t take life too seriously." ' +
'I have 3 photos and no prompt answers filled in.'
);
console.log('Response:', result.response);
console.log('Tools called:', result.toolsCalled);
}
await singleMessage();
await conversation();
await parallelCalls();
await profileAudit();
10. Retry Logic
When a tool call fails or the model's response is unsatisfactory, implement retry logic:
/**
* Router with retry support.
*/
export async function routerWithRetry(
userMessage,
conversationHistory = [],
retriesLeft = config.maxRetries
) {
const result = await router(userMessage, conversationHistory);
// Retry on error
if (result.error && retriesLeft > 0) {
logger.log('warn', 'retrying', {
userMessage: userMessage.slice(0, 100),
retriesLeft,
});
// Exponential backoff
await new Promise((resolve) =>
setTimeout(resolve, 1000 * (config.maxRetries - retriesLeft + 1))
);
return routerWithRetry(userMessage, conversationHistory, retriesLeft - 1);
}
return result;
}
11. Production Patterns
Pattern: Context-aware tool selection
In production, you may want to dynamically include or exclude tools based on the user's context:
function getToolsForUser(user) {
const baseTools = [tools[2]]; // moderateText always available
if (user.isPremium) {
baseTools.push(tools[0]); // improveBio (premium feature)
baseTools.push(tools[1]); // generateOpeners (premium feature)
baseTools.push(tools[4]); // analyzeProfile (premium feature)
}
baseTools.push(tools[3]); // getProfileTips always available
return baseTools;
}
async function routerForUser(userMessage, user, history = []) {
const userTools = getToolsForUser(user);
// Use userTools instead of all tools in the API call
}
Pattern: Rate limiting tool calls
const toolCallCounts = new Map(); // In production, use Redis
function checkRateLimit(userId, toolName, limit = 10, windowMs = 60000) {
const key = `${userId}:${toolName}`;
const now = Date.now();
const calls = toolCallCounts.get(key) || [];
// Remove expired entries
const recent = calls.filter((ts) => now - ts < windowMs);
if (recent.length >= limit) {
return {
allowed: false,
error: `Rate limit: max ${limit} ${toolName} calls per minute. Try again in a moment.`,
};
}
recent.push(now);
toolCallCounts.set(key, recent);
return { allowed: true };
}
Pattern: Tool call analytics dashboard
function getToolAnalytics() {
const logs = logger.getLogs().filter((l) => l.event === 'tool_call');
const byTool = {};
for (const log of logs) {
const name = log.toolName;
if (!byTool[name]) {
byTool[name] = { calls: 0, errors: 0, totalDurationMs: 0 };
}
byTool[name].calls++;
if (!log.success) byTool[name].errors++;
byTool[name].totalDurationMs += log.durationMs || 0;
}
return Object.entries(byTool).map(([name, stats]) => ({
tool: name,
totalCalls: stats.calls,
errorRate: ((stats.errors / stats.calls) * 100).toFixed(1) + '%',
avgDurationMs: Math.round(stats.totalDurationMs / stats.calls),
}));
}
// Usage:
// [
// { tool: 'improveBio', totalCalls: 150, errorRate: '2.0%', avgDurationMs: 1200 },
// { tool: 'moderateText', totalCalls: 300, errorRate: '0.0%', avgDurationMs: 5 },
// { tool: 'generateOpeners', totalCalls: 80, errorRate: '3.8%', avgDurationMs: 1500 },
// ]
12. Error Handling Strategy
+------------------------------------------------------------------------+
| ERROR HANDLING LAYERS |
| |
| Layer 1: Input Validation |
| +------------------------------------------------------------------+ |
| | validateUserInput() -- catches empty, too-long messages | |
| | Happens BEFORE any API call | |
| +------------------------------------------------------------------+ |
| |
| Layer 2: API Errors |
| +------------------------------------------------------------------+ |
| | try/catch around openai.chat.completions.create() | |
| | Handles: rate limits, auth errors, network failures | |
| | Strategy: retry with backoff, then fallback message | |
| +------------------------------------------------------------------+ |
| |
| Layer 3: Argument Parsing |
| +------------------------------------------------------------------+ |
| | safeParseArguments() -- catches malformed JSON | |
| | Returns error as tool result so model can self-correct | |
| +------------------------------------------------------------------+ |
| |
| Layer 4: Unknown Functions |
| +------------------------------------------------------------------+ |
| | handlerMap[fnName] check -- catches hallucinated function names | |
| | Returns error listing available functions | |
| +------------------------------------------------------------------+ |
| |
| Layer 5: Function Execution |
| +------------------------------------------------------------------+ |
| | try/catch around handlerMap[fnName](args) | |
| | Catches: DB errors, API failures, validation errors | |
| | Returns error as tool result for graceful degradation | |
| +------------------------------------------------------------------+ |
| |
| Layer 6: Result Validation |
| +------------------------------------------------------------------+ |
| | validateToolResult() -- catches oversized results | |
| | Truncates to prevent token budget overflow | |
| +------------------------------------------------------------------+ |
| |
| Layer 7: Final Response Fallback |
| +------------------------------------------------------------------+ |
| | If the final LLM call fails, return raw tool results | |
| | User still gets useful information | |
| +------------------------------------------------------------------+ |
+------------------------------------------------------------------------+
13. Testing the Router End-to-End
// test-router.js
import { router } from './index.js';
const testCases = [
// Routing accuracy tests
{
name: 'Bio improvement routes correctly',
input: 'My bio is "I like coffee." Help!',
expectTools: ['improveBio'],
},
{
name: 'Opener generation routes correctly',
input: 'Help me message a dog lover who hikes',
expectTools: ['generateOpeners'],
},
{
name: 'Moderation routes correctly',
input: 'Is "Hey text me at 555-1234" safe to send?',
expectTools: ['moderateText'],
},
{
name: 'Profile analysis routes correctly',
input: 'Analyze my profile: bio is "Love food" with 2 photos',
expectTools: ['analyzeProfile'],
},
{
name: 'Tips route correctly',
input: 'Any tips for better photos?',
expectTools: ['getProfileTips'],
},
{
name: 'No tool for conversation',
input: 'Thanks for the help!',
expectTools: [],
},
{
name: 'Parallel tool calls',
input: 'Improve bio "I like dogs" AND check if "call 555-1234" is safe',
expectTools: ['improveBio', 'moderateText'],
},
// Error handling tests
{
name: 'Empty message handled',
input: '',
expectError: true,
},
{
name: 'Very long message handled',
input: 'a'.repeat(6000),
expectError: true,
},
];
async function runTests() {
let passed = 0;
let failed = 0;
for (const tc of testCases) {
try {
const result = await router(tc.input);
if (tc.expectError) {
if (result.error) {
console.log(`PASS: ${tc.name}`);
passed++;
} else {
console.log(`FAIL: ${tc.name} -- expected error, got success`);
failed++;
}
continue;
}
const toolsMatch =
tc.expectTools.length === result.toolsCalled.length &&
tc.expectTools.every((t) => result.toolsCalled.includes(t));
if (toolsMatch) {
console.log(`PASS: ${tc.name}`);
passed++;
} else {
console.log(
`FAIL: ${tc.name} -- ` +
`expected [${tc.expectTools}], got [${result.toolsCalled}]`
);
failed++;
}
} catch (error) {
console.log(`ERROR: ${tc.name} -- ${error.message}`);
failed++;
}
}
console.log(`\nResults: ${passed} passed, ${failed} failed out of ${testCases.length}`);
}
await runTests();
14. Key Takeaways
- A production tool router needs validation (input, arguments, results), error handling (5+ layers), logging (every tool call), and retry logic.
- Separate concerns: tool definitions (schema), handlers (business logic), router (orchestration), validation, and logging into distinct modules.
- Use
Promise.allfor parallel tool calls and handle chained tool calls (model calls a second tool after seeing the first result). - Rate limiting and context-aware tool selection (premium vs free users) are essential for production.
- Test routing accuracy independently from handler correctness --- the AI's routing decision and your function's execution are separate concerns.
- Log everything --- tool name, arguments, result, duration, errors. This is your debugging lifeline for a non-deterministic system.
- Return errors as tool results (not thrown exceptions) so the model can explain the problem naturally to the user.
Explain-It Challenge
- A production user reports: "The assistant said it improved my bio, but my bio didn't actually change." Trace through the router to identify all the places this could have gone wrong. What logging would help you debug it?
- Your router has 5 tools. A new requirement adds 10 more. What problems will this cause, and how would you redesign the system?
- The model sometimes calls
improveBiowhen the user asks for general profile advice (which should route togetProfileTips). How do you fix this without changing the model?
Navigation: <- 4.7.d --- Hybrid Logic | 4.7 Overview