Getting Started with the OpenAI API: Your First Completion

This is the first post in our OpenAI API fundamentals series. Later posts cover function calling, context/token/cost management, RAG, and fine-tuning vs. prompting.

Every OpenAI API tutorial starts with essentially the same five-line example, and it's genuinely a reasonable place to start — the API's core shape really is that simple. What most tutorials skip past is everything around that example that actually matters once you're building something real.

The basic request

from openai import OpenAI
 
client = OpenAI()  # reads OPENAI_API_KEY from the environment
 
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "system", "content": "You are a helpful assistant for a technology training company."},
        {"role": "user", "content": "Explain what a load balancer does, in two sentences."},
    ],
)
 
print(response.choices[0].message.content)

A load balancer distributes incoming network traffic across multiple
servers so no single server becomes overwhelmed, improving both
reliability and response times. If one server fails, the load balancer
routes traffic to the remaining healthy servers automatically.

The messages array is the entire conversation the model sees, every time — there's no server-side memory of previous requests at all. Three roles matter here: system sets behavior and context for the whole conversation, user is the actual prompt, and assistant (not used yet in this first example) is how the model's own previous replies get fed back in for multi-turn conversations, covered directly in the context management post later in this series.

Never hardcode an API key

import os
from openai import OpenAI
 
client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])

The official SDK actually reads OPENAI_API_KEY from the environment automatically, so OpenAI() with no arguments already does the right thing — the explicit version above is worth showing once just to make clear where that value has to come from. An API key hardcoded into source code is the exact same category of mistake as a database password committed to git, covered repeatedly throughout this blog for Terraform, Ansible, and Kubernetes: it ends up in version control history, in logs, in a screen-share, permanently.

Parameters that actually change behavior

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[...],
    temperature=0.2,
    max_tokens=300,
)

• temperature controls randomness in the model's output. Lower values (0.0–0.3) produce more consistent, deterministic-leaning output — appropriate for factual tasks, structured extraction, code generation. Higher values (0.7+) produce more varied, creative output — appropriate for brainstorming or creative writing, not for a task where you need the same input to reliably produce a similar answer.
• max_tokens caps how long a response can be. Leaving it unset risks an unexpectedly long (and correspondingly expensive) response for an open-ended prompt; setting it too low truncates a genuinely necessary answer mid-sentence. Getting this right for a specific use case usually means testing against real example prompts, not guessing.

Streaming: responding before the full answer is ready

stream = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Write a short paragraph about Kubernetes."}],
    stream=True,
)
 
for chunk in stream:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)

Without streaming, a user waits for the entire response to generate before seeing anything at all — for a long response, that can be several real seconds of a blank screen. stream=True yields the response incrementally, token by token, as it's generated — the mechanism behind the "typing" effect in every chat interface built on this API, and a genuinely significant perceived-latency improvement even though the total generation time is identical either way.

Handling errors and rate limits properly

import time
from openai import RateLimitError, APIError
 
def get_completion(messages, retries=3):
    for attempt in range(retries):
        try:
            return client.chat.completions.create(model="gpt-4o", messages=messages)
        except RateLimitError:
            wait = 2 ** attempt
            time.sleep(wait)
        except APIError as e:
            if attempt == retries - 1:
                raise
            time.sleep(1)
    raise RuntimeError("Exhausted retries")

A production application calling this API needs to handle rate limits (a RateLimitError, resolved by backing off and retrying, ideally with exponential backoff as shown) and transient API errors as a matter of course, not as an edge case discovered after a real outage. A demo script that calls the API once and prints the result never needs this; anything running unattended, at real volume, needs it from the first version.

What to actually remember from this post

• messages is the entire conversation state, sent fresh on every request — the API itself has no memory between calls.
• temperature and max_tokens are the two parameters worth deliberately setting for almost any real use case, not leaving at their defaults.
• Streaming improves perceived latency, not actual total generation time — genuinely worth it for anything with a human watching in real time.
• Handle rate limits and transient errors from the start — with exponential backoff, not a single unguarded try.

Next in the series: Function Calling and Structured Outputs with the OpenAI API, where the model stops just returning text and starts reliably returning data your code can act on directly.