AI Foundation Models: What They Are and How to Use Them
AI foundation models explained: what they are, how they work, real business use cases, hidden costs, and a practical roadmap for enterprise adoption.
October 27, 2026
In 2023, U.S.-based institutions produced 61 notable AI models, compared with 21 from the European Union and 15 from China, according to Stanford University's 2024 AI Index findings. For organizations adopting AI, the important question isn't whether to join that model-building race. It's how to use someone else's general-purpose model while keeping control of business data, decisions, and costs.
Table of Contents
- What a Foundation Model Is and Why It Matters
- How Foundation Models Learn Before Anyone Uses Them
- What Foundation Models Do Well and Where They Fail
- Where Organizations Are Deploying Foundation Models Today
- Choosing Between Hosted, Fine-Tuned and Self-Hosted Models
- What Running a Foundation Model Really Costs
- Evaluation and Governance Practices That Hold Up in Production
- Pick One Workflow and Measure It First
What a Foundation Model Is and Why It Matters
A foundation model is a large AI model pretrained on broad collections of data, such as text, code, or images, that developers can adapt to many tasks. Think of it as a general-purpose engine that can power different vehicles, rather than a custom engine built for a particular car.
The GPT family, Claude, Gemini, Llama, and IBM Granite 4.1 are examples of foundation-model families. They differ in capabilities and deployment options, but the shared idea is reuse: the same underlying model can support applications with very different purposes.
The engine analogy helps separate the model from the tools built around it.
The model isn't the whole application
A traditional spam filter or credit-scoring model typically learns from labeled examples for a specific prediction task. A new task usually requires a separate training effort, even if developers reuse parts of the original system.
AI foundation models offer a broader starting point. Developers can adapt them through instructions, retrieved documents, or additional training instead of building every capability from scratch.
That expensive shared training effort can support chat, search, summarization, coding assistance, and classification. Adaptation is comparatively lightweight, but building a dependable business application still requires engineering and evaluation.
Treat the foundation model as shared infrastructure, not as a finished business process.
Say a manufacturer wants help answering maintenance questions. The model supplies language capability; document search supplies approved manuals; application code checks employee permissions; and a maintenance specialist decides when an answer needs review.
Those responsibilities belong to different layers of the stack. Understanding the model matters, but so does deciding who controls the surrounding data, software, infrastructure, and governance.
How Foundation Models Learn Before Anyone Uses Them
Think of pretraining as reading a vast library, followed by an apprenticeship in responding to requests. The analogy has a limit: a model doesn't read with human intentions or maintain a reliable catalog of everything it encounters.
For a language model, next-token prediction means learning to predict the next piece of text from the pieces that precede it. A token might be a word, part of a word, or punctuation.
This diagram gives a high-level view of that progression.
From general patterns to useful responses
By repeatedly predicting text across different contexts, the model learns representations that support translation, code generation, explanation, and other tasks. Later training changes how it uses those capabilities, including whether it follows instructions and declines unsuitable requests.
Training stages of a foundation model
| Stage | Purpose | Typical data | What it adds |
|---|---|---|---|
| Pretraining | Learn language, facts and reasoning patterns | Very large web, code and book corpora | General capability base |
| Instruction tuning | Teach the model to follow requests | Curated instruction and answer pairs | Useful, task-oriented responses |
| Preference tuning | Prefer helpful and safe answers | Ranked responses from people or AI judges | Better tone, refusals and format |
| Domain adaptation | Specialise for one field | Company or sector examples | Domain vocabulary and output format |
These are useful categories, not a compulsory sequence for every model. Domain adaptation, for example, might help a model produce maintenance reports in an organization's terminology, but it doesn't automatically make every technical statement correct.
What scaling can predict
The ICLR paper on scaling laws for sparsely connected foundation models describes how validation loss generally follows a power-law relationship with model size, training data, and compute, including formulations for sparse models. Validation loss measures prediction error on held-out material; it's not the same thing as success in a business workflow.
Teams can use smaller checkpoints to estimate whether a larger training run is worthwhile. They should fit those curves within the same model family and data mixture, rather than extrapolating between unrelated architectures.
Sparse models activate selected parts of their networks for a given input. Compare dense and sparse alternatives at matched active-parameter and compute budgets, because total parameter counts alone don't describe the computational work performed per token.
For an accompanying explanation of model training, watch the video below.
Size helps predict training quality within a consistent model family. It isn't a universal ranking of usefulness.
What Foundation Models Do Well and Where They Fail
The expensive misconception is that buying access to a larger model removes the need to test the application. A stronger general model can still misunderstand your documents, mishandle an exception, or produce an answer that looks correct but violates your process.
Foundation models are useful candidates for drafting and summarizing text, extracting document fields, translating, classifying support tickets, generating and reviewing code, and answering questions over supplied material. These tasks become easier to manage when employees can inspect the source material and check the result.
Match each weakness to a control
A model generates plausible output; it doesn't independently verify every claim. That distinction explains why a fluent answer can contain an invented policy, citation, or calculation.
Common limitations, causes and controls
| Limitation | Why it happens | Practical control |
|---|---|---|
| Hallucination | The model predicts plausible text, not verified facts | Retrieval with citations and a verification step |
| Stale knowledge | Training data ends at a cutoff date | Retrieve current documents at query time |
| Context limits | Each request has a fixed input window | Chunk documents and summarize in stages |
| Inconsistent output | Sampling introduces variation between runs | Structured output schemas and fixed settings for tests |
| Benchmark gap | Public benchmarks differ from the real workflow | Test on a set of the organization's own cases |
Consider a benefits assistant that invents a reimbursement rule. Requiring an exact passage from the current policy gives the reviewer something to verify, although a citation alone doesn't prove the answer follows from it.
For stale knowledge, retrieve the current policy rather than asking the model what it remembers. For a large contract collection, select relevant passages instead of assuming that every uploaded document will receive equal attention.
- Unstable formatting: Validate extracted invoice fields against a JSON schema and reject missing required values.
- Misleading benchmark results: Test ticket routing against your actual queue, including ambiguous requests and escalation cases.
- Fragile calculations: Send arithmetic to a calculator or executable code rather than accepting generated totals.
Long multi-document inputs, low-resource languages, precise arithmetic, and rare domain terms deserve explicit test cases. A model may handle the ordinary examples well while failing on precisely the exceptions that require the most employee time.
Trust a capability only after testing it on representative organizational cases, including difficult examples and cases where the correct behavior is to abstain.
Where Organizations Are Deploying Foundation Models Today
Employees don't necessarily encounter foundation models through a chat window or a custom-built application. They can meet them inside software they already use: a coding assistant, support platform, document editor, or enterprise search product.
That distinction changes the adoption decision. An operations leader may be choosing whether to enable an existing feature, while an engineering leader may be deciding how much application logic to build around direct model access.
Look for bounded work with checkable outputs
Useful deployment candidates share practical characteristics: text-heavy inputs, an identifiable reviewer, and an output that can be assessed against a business rule. The following are workflow examples, not claims about a particular company's results.
| Business function | Candidate workflow | What a person can check |
|---|---|---|
| Operations | Extract exceptions from delivery reports | Whether each exception appears in the source |
| Software engineering | Draft a code migration and associated tests | Whether tests pass and behavior is preserved |
| Customer service | Prepare a reply from approved support material | Whether the answer matches policy |
| Finance | Categorize invoice descriptions | Whether the category and supporting fields are correct |
| Manufacturing | Retrieve maintenance instructions | Whether the procedure matches the machine and manual |
The model's role should be narrower than the department's responsibility. A finance assistant can propose an invoice category without receiving authority to release payment.
Remove friction without removing accountability
Consider a hypothetical engineering team migrating Scala code to Java. A coding assistant could draft equivalent classes, explain unfamiliar APIs, and propose regression tests, while engineers retain responsibility for behavior, dependencies, and security.
The deployment work matters as much as the prompt. IT needs to distribute approved software, configure authentication, establish repository permissions, and decide whether code can leave the organization's environment.
For a support assistant, the equivalent work includes connecting approved documents, enforcing customer-data access, and routing uncertain answers to an employee. A polished interface won't compensate for a missing escalation path.
Deployment test: Identify the person who can reject the output before deciding how much autonomy to give the model.
Speed is a useful early outcome to measure because teams can compare cycle time before and after adoption. Pair it with rework and acceptance measures: a faster draft isn't a successful deployment if reviewers spend longer correcting it.
Choosing Between Hosted, Fine-Tuned and Self-Hosted Models
AI sovereignty doesn't necessarily mean building a model from scratch. It means identifying which dependencies an organization must control and making those controls enforceable.
The Linux Foundation's discussion of sovereign AI reports that more than 70% of respondents in global research wanted to protect sensitive data from external appropriation and avoid opaque systems, while 94% considered international collaboration essential to sovereign AI. Control and collaboration aren't opposites.
Choose the layer you need to own
Separate control over data, infrastructure, model weights, applications, and governance. Hosting a model yourself changes some of those responsibilities, but it doesn't automatically establish training-data provenance or remove hardware dependencies.
Foundation-model deployment patterns compared
| Pattern | Control over data and model | Typical upfront effort | Main ongoing cost |
|---|---|---|---|
| Hosted API with prompting | Low model control; contractual data controls | Hours to days | Per-token usage fees |
| Retrieval-augmented generation | Medium application and data control; underlying model unchanged | Weeks to build the data pipeline | Embeddings, vector storage and retrieval calls |
| Fine-tuned model (hosted or open) | Medium to high | Weeks to months, with labeled data | Training runs plus serving |
| Self-hosted open-weight model | High operational control within license terms | Months for GPUs, serving stack and staffing | GPU hours, ML staff, patching and security |
| Custom pretraining | Full control over your training process and resulting weights | Many months and a large research budget | Compute and a dedicated research team |
The effort descriptions are planning guides, not delivery commitments. Retrieval and fine-tuning also aren't successive levels on a universal ladder: they solve different problems and can be combined.
Retrieval-augmented generation, or RAG, supplies relevant documents at query time. Fine-tuning changes model behavior through training, which can help with recurring terminology, style, or output requirements; the discussion of domain-specific language models helps distinguish specialization from merely supplying documents.
For document-connected applications, Yellow.ai's security considerations for RAG and LLMs offer useful context for reviewing access controls and the risks introduced by retrieved content. A document shouldn't become trusted instruction merely because the retrieval system found it.
Make the constraint explicit
Start with a hosted API plus retrieval when contractual data handling, deployment location, and service availability meet the workload's requirements. This lets the team concentrate on application quality rather than GPU operations.
Self-hosting becomes a stronger candidate when offline operation, residency constraints, customization, or supply-chain requirements make managed access unsuitable. Before committing, check the model license, provenance documentation, support ecosystem, hardware needs, and your ability to patch and monitor the service.
Open weights don't automatically mean independence. Dependence can shift to cloud GPUs, specialized hardware, scarce ML staff, or upstream projects.
The decision should name both the constraint and its owner: for example, an offline service requirement owned by operations, supported by an infrastructure team that can maintain it.
What Running a Foundation Model Really Costs
GPT-4-class output costs fell from about $60 to below $1.50 per million tokens between early 2023 and early 2025, a reduction of more than 97%, yet total enterprise AI spending increased as usage expanded faster than unit costs declined, according to the State of Enterprise AI analysis. That analysis also estimated that building a model becomes economically defensible only around $2.1 billion in annual AI spending, a model-dependent estimate rather than a universal purchasing threshold.
Cheaper inference doesn't guarantee better return on investment. Teams may automate more steps, include longer documents, add agents, and retain more logs because each individual call appears inexpensive.
Price the completed workflow
Say a service team replaces manual reply drafting with an assistant. The bill includes more than the draft: document retrieval, permission checks, model calls, retries, employee approval, and correction of any answer that reaches a customer.
Cost layers in a foundation-model workload
| Cost layer | What to measure | Commonly missed |
|---|---|---|
| Model usage | Tokens per completed case | Longer context and retries |
| Retrieval and embeddings | Index size and calls per request | Re-indexing when documents change |
| Orchestration and agents | Steps and loops per task | Runaway agent steps without stopping rules |
| Human review | Minutes per output reviewed | Review time that grows with volume |
| Evaluation and monitoring | Test runs and drift checks per month | Ongoing regression testing after model updates |
| Cost of wrong outputs | Error frequency, severity, and remediation cost | Downstream fixes and customer impact |
Integration work belongs in the calculation too. Authentication, legacy-system connectors, fallback behavior, and incident handling consume engineering time even when the model itself is inexpensive.
Use quality-adjusted cost, such as cost per successfully completed case. Define success before the pilot: an accepted answer, a correctly processed document, or a resolved request without avoidable rework.
Compare like-for-like outcomes
Count failed attempts and escalations in the numerator rather than excluding them without notice. Keep the success criteria consistent across model candidates so a cheaper but less reliable system doesn't appear economical.
A total economic impact assessment can help organize that broader comparison. Include adoption costs and employee time alongside usage fees, and distinguish demonstrated savings from capacity employees haven't yet redirected.
Budget rule: Cap the work a system may attempt, not just the price of each model call.
Route straightforward classification and extraction to smaller models when they pass the same acceptance tests. Give agents explicit stopping conditions, spending limits, and escalation rules.
For many deployments, controlling demand and unnecessary steps will matter more than choosing between similarly capable models.
Evaluation and Governance Practices That Hold Up in Production
Increasing concurrency from 8 to 48 reduced total completion time from 2,034 seconds to 774 seconds in a foundation-model inference cost study, but pushing beyond the optimal range reduced throughput and sharply increased time-to-first-token. That result describes the tested setup, not a recommended concurrency setting for every application.
Batching spreads fixed GPU overhead across requests until memory bandwidth, compute, or scheduling becomes the bottleneck. A model that responds well to an isolated request may behave differently under the traffic your employees generate.
The illustration below emphasizes the concurrency trade-off; use measured results from your own deployment for budgeting and service targets.
Test the operating point
- Reproduce expected traffic. Load-test candidates across realistic prompt lengths, output lengths, and concurrency levels.
- Find the operating point that meets both latency targets and quality thresholds, including peak demand.
- Compare the same cases. Score total cost, latency, throughput, and accuracy together.
- Repeat the test after model or prompt changes, keeping a versioned record of the configuration.
A practical AI model evaluation process should include ordinary requests, difficult exceptions, and cases that require refusal or escalation. Keep a separate holdout set so repeated prompt adjustments don't merely optimize for familiar examples.
Assign operational responsibility
- Named owner: Assign someone responsibility for each deployed model and its application.
- Controlled logging: Record inputs and outputs under retention, redaction, and access rules.
- Review thresholds: Require human approval for high-stakes outputs.
- Supply-chain checks: Review licenses and available training-data provenance.
- Incident handling: Define how to pause the service, investigate errors, and notify affected teams.
For teams formalizing these responsibilities, AI risk management with Freeform Company provides additional governance context. Translate governance principles into release checks and incident procedures that employees can execute.
Reporting only tokens per second or benchmark accuracy can conceal an unfavorable cost-quality trade-off. Production readiness requires acceptable behavior, affordable delivery, and clear accountability together.
Pick One Workflow and Measure It First
Choose a bounded workflow before settling on an ambitious AI architecture. A document classification queue or internal policy assistant gives the team a clearer starting point than a general instruction to automate operations.
- Select a checkable task. Choose text inputs, an output a person can verify, and an existing baseline for time and cost.
- Build an evaluation set from past cases. Record cycle time, employee effort, exceptions, and the criteria for an acceptable result.
- Pilot the simplest viable pattern. Start with managed access and retrieval where appropriate, then load-test at expected volume before expanding access.
- Expand only when quality-adjusted cost per completed case beats the baseline and governance owners are named.
Treat the model as shared infrastructure: the pilot helps show which parts of the stack your organization needs to control. Managed access can simplify the start, while retrieval can connect a model to relevant information without replacing the underlying model. Measure the full workflow, since cheaper tokens do not necessarily mean lower total spend if usage grows or additional systems and review are needed.
Use implementation evidence to choose the next task
Applied is one option for examining deployment patterns alongside vendor documentation and your own pilot results. Its curated library includes implementations across finance, retail, healthcare, technology, manufacturing, and media, with company context, tools, and measured outcomes.
Creating a free account gives you access to the full library, advanced filters, and ongoing research. Use those examples to shortlist workflows, then test whether an approach fits your data, employees, and constraints.
Explore foundation-model implementations, industry-specific tools, and business outcomes at Applied. Create a free account to access the library and ongoing research, or discuss consulting support for AI strategy, education, and hands-on implementation.