When a hosted API is the right call
For the large majority of use cases, a general-purpose API from a major provider is faster, cheaper, and more capable than anything most teams would train themselves. It's the sensible default and often the permanent answer.
- You want to ship in weeks, not quarters
- Your task is general: writing, extraction, classification, or Q&A
- You don't have a large, clean, labeled dataset
- Quality on your task is already good enough with prompting
- You'd rather not own model training, hosting, and upkeep
When a custom or fine-tuned model earns its keep
Custom or fine-tuned models make sense for narrower reasons than vendors often imply. Usually it's a specialized domain, strict data or cost constraints, or a repetitive task where a smaller tuned model beats a big general one.
Even then, we typically try prompting and retrieval first, because they're cheaper to change.
- A specialized domain or format the general models handle poorly
- Data that can't leave your environment for privacy or contract reasons
- High, steady volume where a smaller self-hosted model lowers cost
- A narrow, repetitive task where fine-tuning improves consistency
- Latency or offline requirements a hosted API can't meet
The middle ground most teams miss
It's rarely a binary. Retrieval (feeding the model your data at request time) and prompt engineering close most gaps without training anything, and they're far easier to maintain. We usually exhaust these before recommending a custom model.
- Retrieval-augmented generation to inject your knowledge without training
- Structured prompting and examples for consistent output
- Fine-tuning only when prompting plateaus on a well-defined task
- A clear-eyed look at total cost, including ongoing maintenance
More on ai integrations
Frequently asked questions
Isn't a custom model always better and more "ours"?
Not usually. General APIs are extremely capable, and a custom model adds cost, data requirements, and maintenance you'll own forever. We recommend one only when a specific constraint or quality gap makes it worth that burden.
We're worried about sending data to a third-party API. What are our options?
Several, from providers with strict data controls and no-training guarantees to models you host in your own environment. The right choice depends on your specific privacy and compliance requirements, which we map before recommending anything.
How do we know if fine-tuning will actually help?
By measuring first. We set up an evaluation on your task, exhaust prompting and retrieval, and only fine-tune if there's a clear, measured gap left. That way the decision is based on your numbers, not a hunch.