Episode 4 — Generative AI Engineering / 4.7 — Function Calling Tool Calling

4.7.c --- Deterministic Tool Invocation

In one sentence: The tool calling flow is a structured conversation: you define tools as JSON schemas, send them with the user message, the model returns a tool_calls object, you execute the function in your code, return the result to the model, and the model generates the final response --- all through clearly defined API message roles.

Navigation: <- 4.7.b --- When to Use Tool Calling | 4.7.d --- Hybrid Logic ->

1. The Six-Step Flow

Every tool calling interaction follows the same six steps:

+------------------------------------------------------------------------+
|                    THE TOOL CALLING FLOW                                 |
|                                                                         |
|  Step 1: DEFINE tools with JSON Schema                                  |
|          (name, description, parameters)                                |
|                        |                                                |
|                        v                                                |
|  Step 2: SEND messages + tools to the LLM API                          |
|          (system prompt, user message, tool definitions)                |
|                        |                                                |
|                        v                                                |
|  Step 3: MODEL DECIDES                                                  |
|          - No tool needed? Returns text (finish_reason: "stop")         |
|          - Tool needed? Returns tool_calls (finish_reason: "tool_calls")|
|                        |                                                |
|                        v                                                |
|  Step 4: EXECUTE the function in YOUR code                              |
|          - Parse function name and arguments                            |
|          - Validate arguments                                           |
|          - Call the actual function                                     |
|                        |                                                |
|                        v                                                |
|  Step 5: RETURN the result to the model                                 |
|          - Add assistant message (with tool_calls)                      |
|          - Add tool message (with result)                               |
|                        |                                                |
|                        v                                                |
|  Step 6: MODEL GENERATES final response                                 |
|          - Uses the tool result to craft a natural response             |
|          - This is a normal text completion                             |
+------------------------------------------------------------------------+

Let us walk through each step with working code.

2. Step 1: Define Tools with JSON Schema

Tools are defined as an array of objects. Each tool has a type (always "function" currently), a name, a description, and parameters defined using JSON Schema.

const tools = [
  {
    type: 'function',
    function: {
      name: 'improveBio',
      description:
        'Improve a user dating profile bio to make it more engaging, ' +
        'authentic, and likely to attract matches. ' +
        'Call this when the user wants to improve, rewrite, or enhance their bio.',
      parameters: {
        type: 'object',
        properties: {
          currentBio: {
            type: 'string',
            description: 'The user\'s current bio text to improve',
          },
          tone: {
            type: 'string',
            enum: ['witty', 'sincere', 'adventurous', 'intellectual'],
            description: 'Desired tone for the improved bio (default: witty)',
          },
          maxLength: {
            type: 'number',
            description: 'Maximum character length for the new bio (default: 500)',
          },
        },
        required: ['currentBio'],
        additionalProperties: false,
      },
    },
  },
  {
    type: 'function',
    function: {
      name: 'generateOpeners',
      description:
        'Generate conversation opener messages based on someone\'s profile. ' +
        'Call this when the user wants help starting a conversation or ' +
        'needs icebreaker messages.',
      parameters: {
        type: 'object',
        properties: {
          profileDescription: {
            type: 'string',
            description: 'Description of the person\'s profile or interests',
          },
          count: {
            type: 'number',
            description: 'Number of openers to generate (1-5, default: 3)',
          },
          style: {
            type: 'string',
            enum: ['funny', 'thoughtful', 'flirty', 'casual'],
            description: 'Style of the opener messages',
          },
        },
        required: ['profileDescription'],
        additionalProperties: false,
      },
    },
  },
  {
    type: 'function',
    function: {
      name: 'moderateText',
      description:
        'Check if a message is appropriate for a dating platform. ' +
        'Call this when the user asks you to review, check, or moderate a message.',
      parameters: {
        type: 'object',
        properties: {
          text: {
            type: 'string',
            description: 'The message text to check for appropriateness',
          },
        },
        required: ['text'],
        additionalProperties: false,
      },
    },
  },
];

JSON Schema best practices for tool definitions

PracticeWhyExample
Detailed descriptionsThe model uses descriptions to decide which tool to call"Call this when the user wants to improve their bio"
Use enum for constrained valuesPrevents the model from inventing invalid valuesenum: ['witty', 'sincere', 'adventurous']
Mark required fieldsModel knows which arguments it must providerequired: ['currentBio']
Set additionalProperties: falsePrevents the model from adding unexpected fieldsCleaner argument objects
Use description on every propertyHelps the model understand what to extract from user input"The user's current bio text to improve"
Keep parameter count lowMore parameters = more chances for errors3-5 parameters per tool is ideal

3. Step 2: Send Messages + Tools to the API

import OpenAI from 'openai';

const openai = new OpenAI();

const response = await openai.chat.completions.create({
  model: 'gpt-4o',
  messages: [
    {
      role: 'system',
      content:
        'You are a helpful dating app assistant. ' +
        'Use the available tools to help users with their profiles and messages. ' +
        'If the user\'s request doesn\'t match any tool, respond conversationally.',
    },
    {
      role: 'user',
      content:
        'Can you help me improve my bio? ' +
        'Right now it just says "I like hiking and coffee." ' +
        'Make it sound more fun!',
    },
  ],
  tools,                  // Array of tool definitions from Step 1
  tool_choice: 'auto',    // Let the model decide (default)
  temperature: 0,         // Deterministic for consistent routing
});

The tool_choice parameter

ValueBehaviorWhen to Use
'auto'Model decides whether to call a tool or respond with textDefault --- most common
'none'Model will NOT call any tool, even if one matchesWhen you want text-only response
'required'Model MUST call at least one toolWhen you know a tool is needed
{ type: 'function', function: { name: 'improveBio' } }Model MUST call this specific toolWhen you know exactly which tool

4. Step 3: Model Returns a Decision

The model's response tells you what it decided:

const message = response.choices[0].message;
const finishReason = response.choices[0].finish_reason;

console.log(finishReason);
// "tool_calls" --> model wants to call a function
// "stop"       --> model responded with text (no tool needed)

if (finishReason === 'tool_calls') {
  console.log(message.tool_calls);
  // [
  //   {
  //     id: 'call_abc123def456',       <-- Unique ID for this call
  //     type: 'function',
  //     function: {
  //       name: 'improveBio',          <-- Which function to call
  //       arguments: '{"currentBio":"I like hiking and coffee.","tone":"witty"}'
  //                                     ^-- Arguments as a JSON STRING
  //     }
  //   }
  // ]
} else {
  // Model responded with text (no tool call)
  console.log(message.content);
}

Critical detail: The arguments field is a JSON string, not an object. You must JSON.parse() it.

const toolCall = message.tool_calls[0];
const functionName = toolCall.function.name;       // 'improveBio'
const args = JSON.parse(toolCall.function.arguments); // { currentBio: '...', tone: 'witty' }

5. Step 4: Execute the Function

Now your code runs the actual function. The model did NOT execute it --- you do.

// Your actual function implementations
function improveBio({ currentBio, tone = 'witty', maxLength = 500 }) {
  // In a real app, this might:
  //   - Call another LLM with a specialized prompt
  //   - Apply business rules (banned words, length limits)
  //   - Query a database for similar successful bios
  //   - Log the transformation for analytics

  // For this example, a simple implementation:
  const toneInstructions = {
    witty: 'Add humor and clever wordplay',
    sincere: 'Make it warm and genuine',
    adventurous: 'Emphasize excitement and exploration',
    intellectual: 'Highlight curiosity and depth',
  };

  // This could be another LLM call with a focused prompt
  return {
    originalBio: currentBio,
    improvedBio:
      `Weekend adventurer fueled by espresso. ` +
      `If you can keep up on a trail, you can definitely keep up in conversation. ` +
      `Currently accepting applications for a hiking buddy who appreciates a good pour-over.`,
    tone,
    characterCount: 198,
  };
}

function generateOpeners({ profileDescription, count = 3, style = 'casual' }) {
  // In production: call an LLM with a specialized opener prompt,
  // filter through moderation, rank by quality
  return {
    openers: [
      'I noticed you\'re into rock climbing -- what\'s the scariest wall you\'ve conquered?',
      'Fellow coffee snob here. What\'s your go-to order when you need to impress?',
      'Your travel photos are amazing! Where\'s next on your list?',
    ],
    style,
    count: 3,
  };
}

function moderateText({ text }) {
  // In production: run through content filters, check against rules,
  // flag for human review if borderline
  const bannedPatterns = [
    /\b\d{3}[-.]?\d{3}[-.]?\d{4}\b/,  // Phone numbers
    /\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b/, // Emails
    /\b(venmo|cashapp|paypal)\b/i,      // Payment references
  ];

  const violations = bannedPatterns
    .filter((pattern) => pattern.test(text))
    .map((pattern) => pattern.source);

  return {
    text,
    safe: violations.length === 0,
    violations: violations.length > 0 ? violations : [],
    suggestion: violations.length > 0
      ? 'Remove personal contact information before sending.'
      : 'Message looks good to send!',
  };
}

// Dispatch: map function names to implementations
const functionMap = {
  improveBio,
  generateOpeners,
  moderateText,
};

// Execute the function the model chose
const toolCall = message.tool_calls[0];
const functionName = toolCall.function.name;
const args = JSON.parse(toolCall.function.arguments);

// Safety check: verify the function exists
if (!functionMap[functionName]) {
  throw new Error(`Unknown function: ${functionName}`);
}

const result = functionMap[functionName](args);
console.log('Function result:', result);

6. Step 5: Return the Result to the Model

This is where the conversation structure is critical. You must send back:

  1. The assistant's original message (including its tool_calls).
  2. A tool message with the result, referencing the tool_call_id.
// Build the messages array for the follow-up call
const followUpMessages = [
  // Original system prompt
  {
    role: 'system',
    content:
      'You are a helpful dating app assistant. ' +
      'Use the available tools to help users with their profiles and messages.',
  },
  // Original user message
  {
    role: 'user',
    content:
      'Can you help me improve my bio? ' +
      'Right now it just says "I like hiking and coffee." ' +
      'Make it sound more fun!',
  },
  // The assistant's response (with tool_calls)
  message, // <-- This is response.choices[0].message from Step 3
  // The tool result
  {
    role: 'tool',                           // <-- Special role for tool results
    tool_call_id: toolCall.id,              // <-- Must match the tool_call's id
    content: JSON.stringify(result),         // <-- Result as a string
  },
];

Important rules for tool result messages:

  • role must be 'tool' (not 'user' or 'assistant').
  • tool_call_id must match the id from the tool call in the assistant's message.
  • content must be a string. If your result is an object, JSON.stringify() it.
  • The assistant message with tool_calls must appear before the tool result message.

7. Step 6: Model Generates the Final Response

// Send the follow-up request
const finalResponse = await openai.chat.completions.create({
  model: 'gpt-4o',
  messages: followUpMessages,
  tools,  // Still include tools in case the model needs to call another
});

const finalMessage = finalResponse.choices[0].message.content;
console.log(finalMessage);
// "Here's your improved bio! I gave it a witty spin:
//
//  'Weekend adventurer fueled by espresso. If you can keep up on a trail,
//   you can definitely keep up in conversation. Currently accepting
//   applications for a hiking buddy who appreciates a good pour-over.'
//
//  It's 198 characters -- nice and concise. Want me to adjust the tone
//  or try a different style?"

The model takes the raw result from your function and crafts it into a natural, friendly response for the user. This is the magic of tool calling: deterministic execution + natural language presentation.

8. The Complete Flow in One Script

Here is the entire flow as a single, runnable script:

import OpenAI from 'openai';

const openai = new OpenAI();

// ---- Tool definitions ----
const tools = [
  {
    type: 'function',
    function: {
      name: 'improveBio',
      description: 'Improve a dating profile bio to be more engaging',
      parameters: {
        type: 'object',
        properties: {
          currentBio: { type: 'string', description: 'Current bio text' },
          tone: {
            type: 'string',
            enum: ['witty', 'sincere', 'adventurous', 'intellectual'],
          },
        },
        required: ['currentBio'],
        additionalProperties: false,
      },
    },
  },
  {
    type: 'function',
    function: {
      name: 'generateOpeners',
      description: 'Generate conversation opener messages for a dating match',
      parameters: {
        type: 'object',
        properties: {
          profileDescription: { type: 'string', description: 'The match\'s profile info' },
          count: { type: 'number', description: 'Number of openers (default 3)' },
        },
        required: ['profileDescription'],
        additionalProperties: false,
      },
    },
  },
  {
    type: 'function',
    function: {
      name: 'moderateText',
      description: 'Check if a message is appropriate for the dating platform',
      parameters: {
        type: 'object',
        properties: {
          text: { type: 'string', description: 'Message to check' },
        },
        required: ['text'],
        additionalProperties: false,
      },
    },
  },
];

// ---- Function implementations ----
function improveBio({ currentBio, tone = 'witty' }) {
  return {
    improvedBio:
      'Weekend adventurer fueled by espresso. ' +
      'If you can keep up on a trail, you can definitely keep up in conversation.',
    tone,
    characterCount: 113,
  };
}

function generateOpeners({ profileDescription, count = 3 }) {
  return {
    openers: [
      'I noticed you love hiking -- what\'s the best trail you\'ve found?',
      'Fellow coffee enthusiast here. Pour-over or French press?',
      'Your profile made me smile. What\'s the story behind your last photo?',
    ].slice(0, count),
  };
}

function moderateText({ text }) {
  const hasPhone = /\b\d{3}[-.]?\d{3}[-.]?\d{4}\b/.test(text);
  const hasEmail = /\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b/.test(text);
  return {
    safe: !hasPhone && !hasEmail,
    issues: [
      ...(hasPhone ? ['Contains phone number'] : []),
      ...(hasEmail ? ['Contains email address'] : []),
    ],
  };
}

const functionMap = { improveBio, generateOpeners, moderateText };

// ---- Main flow ----
async function handleUserMessage(userMessage) {
  const messages = [
    {
      role: 'system',
      content: 'You are a dating app assistant. Use tools when appropriate.',
    },
    { role: 'user', content: userMessage },
  ];

  // Step 2 + 3: Send to API and get model's decision
  const response = await openai.chat.completions.create({
    model: 'gpt-4o',
    messages,
    tools,
    tool_choice: 'auto',
  });

  const assistantMessage = response.choices[0].message;

  // Check if the model wants to call a tool
  if (response.choices[0].finish_reason !== 'tool_calls') {
    // No tool call -- just return the text response
    return assistantMessage.content;
  }

  // Step 4: Execute each tool call
  const toolResults = [];
  for (const toolCall of assistantMessage.tool_calls) {
    const fnName = toolCall.function.name;
    const fnArgs = JSON.parse(toolCall.function.arguments);

    console.log(`Calling ${fnName} with:`, fnArgs);

    if (!functionMap[fnName]) {
      toolResults.push({
        role: 'tool',
        tool_call_id: toolCall.id,
        content: JSON.stringify({ error: `Unknown function: ${fnName}` }),
      });
      continue;
    }

    const result = functionMap[fnName](fnArgs);
    toolResults.push({
      role: 'tool',
      tool_call_id: toolCall.id,
      content: JSON.stringify(result),
    });
  }

  // Step 5 + 6: Return results and get final response
  const finalResponse = await openai.chat.completions.create({
    model: 'gpt-4o',
    messages: [
      ...messages,
      assistantMessage,   // Assistant's message with tool_calls
      ...toolResults,     // Tool result messages
    ],
    tools,
  });

  return finalResponse.choices[0].message.content;
}

// ---- Test it ----
const result = await handleUserMessage(
  'My bio just says "I like hiking and coffee." Can you make it better?'
);
console.log(result);

9. Parallel Tool Calls

The model can request multiple tool calls in a single response. This happens when the user's message requires more than one action.

// User: "Improve my bio AND check if this message is okay: 'Hey call me at 555-1234'"
// Model returns TWO tool_calls in a single response:

const response = await openai.chat.completions.create({
  model: 'gpt-4o',
  messages: [
    { role: 'system', content: 'You are a dating app assistant.' },
    {
      role: 'user',
      content:
        'Two things: improve my bio "I like dogs" and also check if this ' +
        'message is appropriate: "Hey call me at 555-0123"',
    },
  ],
  tools,
  tool_choice: 'auto',
});

const assistantMessage = response.choices[0].message;
console.log(assistantMessage.tool_calls.length); // 2

// tool_calls[0]: { name: 'improveBio', args: { currentBio: 'I like dogs' } }
// tool_calls[1]: { name: 'moderateText', args: { text: 'Hey call me at 555-0123' } }

Handling parallel tool calls

You must return a result for every tool call. Each result references its specific tool_call_id:

// Execute all tool calls (can be done in parallel!)
const toolResults = await Promise.all(
  assistantMessage.tool_calls.map(async (toolCall) => {
    const fnName = toolCall.function.name;
    const fnArgs = JSON.parse(toolCall.function.arguments);
    const result = functionMap[fnName](fnArgs);

    return {
      role: 'tool',
      tool_call_id: toolCall.id,  // Each result matches its call
      content: JSON.stringify(result),
    };
  })
);

// Send ALL results back at once
const finalResponse = await openai.chat.completions.create({
  model: 'gpt-4o',
  messages: [
    ...originalMessages,
    assistantMessage,
    ...toolResults,  // Both tool results included
  ],
  tools,
});

Controlling parallel tool calls

// Allow parallel tool calls (default)
const response = await openai.chat.completions.create({
  model: 'gpt-4o',
  messages,
  tools,
  parallel_tool_calls: true,  // Default
});

// Force sequential tool calls (one at a time)
const response = await openai.chat.completions.create({
  model: 'gpt-4o',
  messages,
  tools,
  parallel_tool_calls: false,
});

10. The Message Structure Explained

Understanding the message array is critical. Here is exactly what each API call contains:

First API call (user message + tools)

messages: [
  { role: 'system', content: '...' },
  { role: 'user', content: 'Improve my bio: "I like hiking"' },
]
// + tools array
// + tool_choice: 'auto'

Model's response (tool call)

// response.choices[0].message:
{
  role: 'assistant',
  content: null,          // null when making tool calls
  tool_calls: [
    {
      id: 'call_abc123',
      type: 'function',
      function: {
        name: 'improveBio',
        arguments: '{"currentBio":"I like hiking","tone":"witty"}',
      },
    },
  ],
}

Second API call (with tool results)

messages: [
  { role: 'system', content: '...' },
  { role: 'user', content: 'Improve my bio: "I like hiking"' },
  {
    role: 'assistant',
    content: null,
    tool_calls: [{ id: 'call_abc123', type: 'function', function: { name: 'improveBio', arguments: '...' } }],
  },
  {
    role: 'tool',
    tool_call_id: 'call_abc123',   // Must match!
    content: '{"improvedBio":"Weekend adventurer...","tone":"witty"}',
  },
]

Model's final response (text for user)

// response.choices[0].message:
{
  role: 'assistant',
  content: 'Here\'s your improved bio with a witty twist: ...',
  tool_calls: undefined,  // No tool calls this time
}

11. Argument Parsing Safety

The arguments field is a JSON string. In rare cases, it can be malformed. Always handle parsing safely:

function safeParseArguments(argumentsString) {
  try {
    return { success: true, data: JSON.parse(argumentsString) };
  } catch (error) {
    return {
      success: false,
      error: `Failed to parse tool arguments: ${error.message}`,
      raw: argumentsString,
    };
  }
}

// Usage
const toolCall = assistantMessage.tool_calls[0];
const parsed = safeParseArguments(toolCall.function.arguments);

if (!parsed.success) {
  // Return error to model so it can try again or explain
  const toolResult = {
    role: 'tool',
    tool_call_id: toolCall.id,
    content: JSON.stringify({ error: parsed.error }),
  };
} else {
  // Execute the function
  const result = functionMap[toolCall.function.name](parsed.data);
}

12. Multi-Turn Tool Calling

Tool calling works across multi-turn conversations. The full conversation history is preserved:

async function chatWithTools(conversationHistory) {
  const response = await openai.chat.completions.create({
    model: 'gpt-4o',
    messages: conversationHistory,
    tools,
    tool_choice: 'auto',
  });

  const assistantMessage = response.choices[0].message;
  conversationHistory.push(assistantMessage);

  if (response.choices[0].finish_reason === 'tool_calls') {
    // Execute tools and add results
    for (const toolCall of assistantMessage.tool_calls) {
      const fnName = toolCall.function.name;
      const fnArgs = JSON.parse(toolCall.function.arguments);
      const result = functionMap[fnName](fnArgs);

      conversationHistory.push({
        role: 'tool',
        tool_call_id: toolCall.id,
        content: JSON.stringify(result),
      });
    }

    // Recursive call: model may need to call more tools
    // or will generate the final text response
    return chatWithTools(conversationHistory);
  }

  return assistantMessage.content;
}

// Multi-turn conversation
const history = [
  { role: 'system', content: 'You are a dating app assistant.' },
  { role: 'user', content: 'Improve my bio: "I like coffee"' },
];

const response1 = await chatWithTools(history);
console.log(response1);
// "Here's your improved bio: ..."

// User continues the conversation
history.push({ role: 'user', content: 'Now give me 3 openers for someone who likes rock climbing' });
const response2 = await chatWithTools(history);
console.log(response2);
// "Here are 3 conversation starters: ..."

13. Token Cost of Tool Calling

Tool definitions consume tokens. Every API call that includes tools pays for the tool schemas as part of the input.

Approximate token costs for tool definitions:

Single tool with 3 parameters:    ~100-200 tokens
Three tools with 3 params each:   ~400-600 tokens
Ten tools with 5 params each:     ~1500-2500 tokens

These tokens are charged on EVERY API call that includes tools.
With 100,000 calls/day at $2.50/1M input tokens:
  600 extra tokens x 100,000 = 60M tokens = $150/day just for tool schemas

Optimization strategies:

  • Only include tools relevant to the current context.
  • Keep descriptions concise but clear.
  • Minimize the number of parameters.
  • Use tool_choice: 'none' when you know no tools are needed.

14. Key Takeaways

  1. The tool calling flow is: define -> send -> model decides -> execute -> return result -> model responds.
  2. Tools are defined using JSON Schema with name, description, and parameters.
  3. The model returns tool_calls with a function name and arguments as a JSON string --- always JSON.parse() the arguments.
  4. Tool results are sent back using the tool role with a matching tool_call_id.
  5. The model can make parallel tool calls --- return results for all of them.
  6. Always validate parsed arguments and handle unknown function names gracefully.
  7. Tool definitions consume tokens on every call --- include only what is needed.

Explain-It Challenge

  1. Trace through the complete message array (all roles) for this scenario: user says "Check if 'call me at 555-1234' is okay to send" with the moderateText tool available. Write out every message object in order.
  2. The model returns arguments: '{"currentBio": "I like dogs"' (note: malformed JSON --- missing closing brace). What happens if you call JSON.parse() directly? How should you handle this?
  3. Your tool calling setup has 15 tools, each with 5 parameters and detailed descriptions. What is the approximate per-call token overhead, and how would you reduce it?

Navigation: <- 4.7.b --- When to Use Tool Calling | 4.7.d --- Hybrid Logic ->