Fine-Tuning

Permanently teach a model new behaviour. Learn how fine-tuning differs from prompting and RAG, when each is the right tool, instruction-tuning, LoRA, and how to source knowledge.

14.1 The Optimization Spectrum

Out of the box, LLMs are generalists. To build effective applications we guide them. There are three levels of optimization:

The context window

Everything in prompting and RAG happens inside the context window — the limited amount of text the model can consider at once. It holds the system instructions, conversation history, user input and any retrieved knowledge. A huge system prompt leaves less room for the conversation.

14.2 Fine-Tuning vs Prompting (and RAG)

RAG vs Fine-Tuning for knowledge

14.3 Situations for Fine-Tuning

Fine-tune when you need:

• A specialised behaviour or style — e.g. teaching a model to consistently write in legal language ("legalese") or a brand voice.
• A consistent output format that few-shot prompting cannot reliably enforce.
• Lower cost & latency — bake long instructions into the weights so prompts can be short (saving tokens on every call).
• Small-model performance — fine-tune a small model (e.g. Llama-7B) so it performs as well as a large model on one specific task.

14.4 Instruction-Tuning & LoRA

Instruction-Tuning

The dataset — JSONL format

Fine-tuning needs a high-quality dataset, usually in JSONL (one JSON object per line). Quality matters more than quantity.

{"messages": [{"role": "user", "content": "Refund my order"},
              {"role": "assistant", "content": "I'm sorry to hear that. Could you share your order ID?"}]}
{"messages": [{"role": "user", "content": "Where is my package?"},
              {"role": "assistant", "content": "Let me check — please provide your tracking number."}]}

For style transfer, as few as 50–100 high-quality examples can work; for complex reasoning you might need 1,000. Always: quality > quantity.

PEFT & LoRA

from openai import OpenAI
client = OpenAI()

# 1. Upload the JSONL training file
file = client.files.create(
    file=open("training_data.jsonl", "rb"),
    purpose="fine-tune"
)

# 2. Start the fine-tuning job
job = client.fine_tuning.jobs.create(
    training_file=file.id,
    model="gpt-4o-mini-2024-07-18"
)
print("Fine-tuning job started:", job.id)

14.5 Knowledge Sourcing

The knowledge gap

LLMs are frozen in time — they only know what they learned during pre-training. They do not know your private company data or news from yesterday, and they can hallucinate details about specific documents. The right knowledge source fixes this:

• Pre-trained knowledge — general world knowledge; free but static and possibly outdated.
• RAG (retrieved knowledge) — for facts that change or are private; instantly updatable, grounded, citable.
• Fine-tuned knowledge — for deeply embedding behaviour/style; permanent but slow to update.
• State-of-the-art: combine them — fine-tune a model to be excellent at using retrieved context (instruction-tuning), then use RAG to feed it the latest data.

Knowing when to prompt vs RAG vs fine-tune is the most-tested idea here.

{"messages": [{"role": "user", "content": "My order is late"},
 {"role": "assistant", "content": "I'm sorry for the delay. Could you share your order ID so I can check?"}]}
{"messages": [{"role": "user", "content": "I want a refund"},
 {"role": "assistant", "content": "I understand. I'd be happy to help - may I have your order number?"}]}

from openai import OpenAI
client = OpenAI()

# Step 1: upload the dataset
training_file = client.files.create(
    file=open("data.jsonl", "rb"),
    purpose="fine-tune"
)

# Step 2: create the fine-tuning job
job = client.fine_tuning.jobs.create(
    training_file=training_file.id,
    model="gpt-4o-mini-2024-07-18"
)
print("Job ID:", job.id)