This is the fifth and final post in our OpenAI API fundamentals series, building on everything from your first completion through function calling, context management, and RAG.
Fine-tuning sounds like the more serious, production-grade option — training a model on your own data feels like it should outperform "just writing a better prompt." In practice, it solves a narrower set of problems than that instinct suggests, and reaching for it before exhausting what prompting (with the techniques covered throughout this series) can do is one of the more common, avoidable expenses in real applications.
What fine-tuning actually changes
Fine-tuning trains the model further on examples you provide — pairs of input and desired output — nudging its behavior toward matching that pattern going forward.
{"messages": [{"role": "system", "content": "You are a customer support agent for NovuSpark."}, {"role": "user", "content": "Do you offer refunds?"}, {"role": "assistant", "content": "Yes, we offer full refunds within 30 days of purchase for any training program."}]}
{"messages": [{"role": "system", "content": "You are a customer support agent for NovuSpark."}, {"role": "user", "content": "Can I reschedule my session?"}, {"role": "assistant", "content": "Absolutely — sessions can be rescheduled up to 48 hours in advance at no charge."}]}client.fine_tuning.jobs.create(
training_file="file-abc123",
model="gpt-4o-mini-2024-07-18",
)What this actually teaches the model is closer to style, tone, and response format consistency than new factual knowledge. Feed it hundreds of examples of your support team's actual tone and typical response structure, and the fine-tuned model produces new responses matching that pattern more consistently than prompting alone reliably achieves — a genuinely narrower, more specific outcome than "the model now knows things it didn't know before."
What fine-tuning does not do well
It is a poor mechanism for adding new factual knowledge. Fine-tuning on a set of documents does not reliably teach a model to accurately recall specific facts from them on demand — it's closer to influencing style than installing a queryable knowledge base. This is precisely the job Retrieval-Augmented Generation, covered in the previous post in this series, actually does well: retrieving the exact relevant passage and handing it to the model as context, rather than hoping a fact survived training and can be accurately recalled later. A remarkably common, expensive mistake is fine-tuning to try to teach a model facts, when RAG was the appropriate tool for that specific problem the entire time.
What prompting alone can already do, before reaching for fine-tuning
- Few-shot examples directly in the prompt — showing 2-3 examples of desired input/output pairs, in the system or user message itself, often gets a model most of the way to a fine-tuned model's consistency, with zero training cost and no separate deployment step:
messages = [
{"role": "system", "content": "Respond in the style of these examples:\n\n"
"Q: Do you offer refunds?\nA: Yes, full refunds within 30 days.\n\n"
"Q: Can I reschedule?\nA: Yes, up to 48 hours in advance, no charge."},
{"role": "user", "content": "What if I need to cancel entirely?"},
]- A well-structured system prompt, defining tone, constraints, and format explicitly, resolves a large share of the "responses are inconsistent" complaints that first prompt people toward fine-tuning.
- RAG, as covered in the previous post, resolves the "the model doesn't know our specific facts" problem — a distinctly different complaint from "the responses don't sound consistent," and one fine-tuning doesn't actually address.
A rough decision framework
- Inconsistent tone or format, few-shot prompting hasn't fully fixed it → fine-tuning is a reasonable next step.
- The model doesn't know your specific facts, documents, or current data → RAG, not fine-tuning.
- The model needs to take actions or return structured data reliably → function calling and structured outputs, covered earlier in this series, not fine-tuning.
- Nothing has actually been tried on the prompting side yet → start there. It's dramatically cheaper to iterate on, requires no training pipeline or evaluation dataset to maintain, and resolves a genuinely large share of real problems people initially assume need fine-tuning.
The real cost most teams underweight
Fine-tuning isn't just a one-time training cost — it's an ongoing commitment: a new base model version means re-evaluating whether the fine-tune should be redone, evaluating fine-tuned output quality requires its own held-out test set maintained deliberately over time, and every fine-tuned model version is one more artifact someone has to track, version, and eventually retire. That ongoing maintenance burden is the actual reason most teams should treat fine-tuning as a considered, occasionally-necessary decision — not a default reached for before confirming prompting genuinely can't solve the specific problem at hand.
What to actually remember from this series
- Fine-tuning primarily influences style and format consistency — it is not a reliable mechanism for teaching new facts.
- RAG is the right tool for "the model doesn't know our specific information" — a genuinely different problem than tone consistency.
- Few-shot prompting and a well-structured system prompt solve a large share of consistency problems before fine-tuning becomes necessary at all.
- Fine-tuning carries real ongoing maintenance cost — evaluation datasets, re-training as base models update — not just a one-time training expense.
That closes out our OpenAI API fundamentals series — from your first completion through function calling, context and cost management, RAG, and now fine-tuning. If your team is building real production AI applications, this is exactly the kind of hands-on work we build our AI & Generative AI training around.