Function Calling and Structured Outputs with the OpenAI API

This is the second post in our OpenAI API fundamentals series. Start with your first completion if you're joining partway through.

A plain text response is fine for a chatbot a human reads directly. It's a genuinely fragile foundation for anything where your own code needs to parse the model's output and act on it — a prompt that says "please respond in JSON" produces output that's usually valid JSON, which is a meaningfully different guarantee than always. Function calling and structured outputs are the API's actual mechanisms for closing that gap.

Function calling: letting the model request an action

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_course_availability",
            "description": "Check available seats for a specific training course",
            "parameters": {
                "type": "object",
                "properties": {
                    "course_name": {"type": "string", "description": "Name of the course"},
                    "date": {"type": "string", "description": "Preferred start date, YYYY-MM-DD"},
                },
                "required": ["course_name"],
            },
        },
    }
]
 
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Is there space in the Terraform course starting in March?"}],
    tools=tools,
)
 
message = response.choices[0].message
if message.tool_calls:
    for call in message.tool_calls:
        print(call.function.name, call.function.arguments)

get_course_availability {"course_name": "Terraform", "date": "2026-03-01"}

Critically, the model never actually calls the function itself. It decides a function call is warranted, and returns the function's name along with arguments matching the schema you defined — your own code is what actually runs get_course_availability(...), using those extracted arguments. This is the entire mechanism: the model is reliably turning unstructured natural language ("is there space in the Terraform course starting in March?") into a structured, schema-conforming function call your code can execute directly.

Completing the loop: feeding the result back

import json
 
# Your own actual function
def get_course_availability(course_name, date=None):
    return {"course_name": course_name, "seats_available": 4, "next_cohort": "2026-03-09"}
 
tool_call = message.tool_calls[0]
args = json.loads(tool_call.function.arguments)
result = get_course_availability(**args)
 
follow_up = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "user", "content": "Is there space in the Terraform course starting in March?"},
        message,
        {
            "role": "tool",
            "tool_call_id": tool_call.id,
            "content": json.dumps(result),
        },
    ],
)
 
print(follow_up.choices[0].message.content)

Yes — there are 4 seats available in the Terraform course, with the
next cohort starting March 9th, 2026.

The second call feeds the function's actual result back in as a role: "tool" message, tied to the original tool_call_id — this is what lets the model produce a natural-language answer grounded in your system's real data, rather than a plausible-sounding guess. This two-step round trip (model requests a call → your code executes it → result goes back to the model) is the actual pattern behind every "AI assistant that can look things up or take actions" system, not a special separate capability layered on top of chat completions.

Structured outputs: guaranteeing schema conformance directly

Function calling is the right tool when the model needs to request an action. For simpler cases — extracting structured data from text, with no actual function to call — structured outputs enforce a JSON schema directly on the model's response:

from pydantic import BaseModel
 
class CourseInquiry(BaseModel):
    course_name: str
    preferred_date: str | None
    urgency: str  # "low", "medium", "high"
 
response = client.chat.completions.parse(
    model="gpt-4o",
    messages=[{"role": "user", "content": "I need Kubernetes training ASAP, ideally next month"}],
    response_format=CourseInquiry,
)
 
inquiry = response.choices[0].message.parsed
print(inquiry.course_name, inquiry.urgency)

Kubernetes high

Unlike a prompt that merely asks for JSON, response_format with a defined schema (a Pydantic model, here) makes the API guarantee the response conforms to that exact structure — not "usually valid JSON matching roughly what I asked for," but a genuine contract your code can rely on without a try/except around a json.loads() call that might fail unpredictably.

Choosing between the two

Function calling: the model needs to trigger a real action or look up real data from your system — checking availability, sending an email, querying a database.
Structured outputs: you need reliably-shaped data extracted from a response — classifying a support ticket's urgency, extracting fields from a user's free-text message — with no actual function being called at all.

Both replace "ask for JSON in the prompt and hope" with an actual, enforced contract — they solve two related but genuinely distinct problems.

What to actually remember from this post

The model never calls a function itself — it returns a name and matching arguments; your own code executes the actual function.
Feed a tool's result back in as a role: "tool" message, tied to the original tool_call_id, to complete the loop and get a grounded natural-language answer.
Structured outputs guarantee schema conformance directly — a genuinely different guarantee than a prompt merely asking for JSON.
Function calling is for triggering actions; structured outputs are for extracting reliably-shaped data — related, but solving different problems.