Small language models.

1. 1. What Small Language Models Are

Small language models (SLMs) are compact, often task-specialised language models that require less compute, cost, and infrastructure than frontier-scale LLMs. There is no fixed size limit, but SLMs typically range from hundreds of millions to the low tens of billions of parameters.¹

They are often optimized, distilled, fine-tuned, or quantized for specific tasks such as document classification, information extraction, summarization, structured output, tool selection, and omain-specific workflows. 

The shift is simple: use the smallest model that meets the required quality threshold. 

2. Scaling Alone Is No Longer Enough 

For years, AI progress was driven by a simple assumption: more data, more compute, and larger models would produce better results. That approach is now showing diminishing returns. Researchers increasingly argue that future progress will also depend on better architectures, training methods, context, and specialisation. 

Yann LeCun has long argued that simply scaling transformers will not be enough. Kian Katanforoosh, CEO of Workera, similarly expects AI to move from an age of scaling towards an age of research. ²

The implication is important: better AI does not automatically mean bigger AI. 

3. Enterprise alternative 

A model extracting contract clauses, classifying supplier documents, or routing service requests does not need to understand everything. It needs to perform one task reliably, quickly, and at scale.  AT&T chief data officer Andy Markus expects fine-tuned SLMs to become increasingly important because they can match larger models on specific enterprise tasks while offering significant advantages in cost and speed.³ SLMs are therefore not replacements for LLMs, but an increasingly important alternative for specialised workloads. ⁴

4. Why SLMs Work So Well 

Four forces are reinforcing one another:

Economics: Repetitive workloads—classification, extraction, document routing, FAQ response, structured reporting—can generate enormous request volumes. A lower-cost model has a compounding financial advantage when it is called millions of times. 

Latency: A model hosted near the workload, or directly on a device, avoids network round trips and can deliver faster, more predictable response times. That matters for operational interfaces, real-time assistants, call-centre workflows, and machine/edge scenarios. 

Data sovereignty and privacy: In domains such as healthcare, financial services, government, regulated manufacturing, and logistics, sensitive documents or operational data may not be permitted—or may be undesirable—to send to an external general-purpose model API. Local or private SLM deployment provides a stronger control boundary. 

Specialisation: For a well-defined task, a fine-tuned or carefully prompted compact model can be more reliable than a broad model because it sees a smaller action space and follows a limited schema. For example, an SLM trained to extract carrier name, lane, rate, surcharge, validity date, and exclusions from freight quotations may outperform a general model on consistency, even if the general model is stronger at open-ended reasoning. 

5. Use the Right Model for the Right Task 

The future enterprise AI architecture will likely combine both small and large models. 

SLMs can handle high-volume tasks such as document intake, metadata extraction, classification, routing, and structured data generation. Larger models can be reserved for complex reasoning, negotiation analysis, strategic questions, and executive synthesis. 

The objective is no longer to use the largest model available. 

It is to use the right model, with the right data, context, and logic, for the right task.