Large language overkill: How SLMs can beat their bigger, resource-intensive cousins

Two years after ChatGPT’s public release, conversations about AI are inescapable as businesses across industries seek to leverage major language models (LLM) to transform their business processes. Yet as powerful and promising as LLMs are, many business and IT leaders have come to rely too much on them and neglect their limitations. This is why I anticipate a future in which specialized language models, or SLMs, play a larger complementary role in enterprise computing.

SLMs are more generally called “small language models” because they require less data and training time and are “more streamlined versions of LLMs.” But I prefer the word “specialized” because it better expresses the ability of these purpose-built solutions to perform highly specialized work with more precision, consistency, and transparency than LLMs. By complementing LLMs with SLMs, organizations can create solutions that leverage the strengths of each model.

Trust and the LLM “black box” problem

LLMs are incredibly powerful, but they are also known to sometimes “lose the plot” or deliver results that diverge due to their generalist training and massive data sets. This trend is made more problematic by the fact that OpenAI ChatGPT and other LLMs are essentially “black boxes” that do not reveal how they arrive at an answer.

This black box problem is going to become a bigger problem in the future, especially for businesses and mission-critical applications where accuracy, consistency, and compliance are paramount. Think of healthcare, financial services, and law as prime examples of professions where inaccurate answers can have huge financial and even life-or-death consequences. Regulators are already taking note and will likely begin requiring explainable AI solutionsespecially in industries that rely on data privacy and accuracy.

While companies often deploy a “human in the loop” approach to mitigate these issues, an overreliance on LLMs can lead to a false sense of security. Over time, complacency can set in and errors can go undetected.

SLM = greater explainability

Fortunately, SLMs are better suited to address many of the limitations of LLMs. Rather than being designed for general tasks, SLMs are developed with a narrower focus and trained on domain-specific data. This specificity allows them to meet nuanced linguistic requirements in areas where precision is essential. Rather than relying on large and heterogeneous datasets, SLMs are trained on targeted information, giving them the ability to contextual intelligence to provide more coherent, predictable and relevant responses.

This offers several advantages. First, they are more explainable, making it easier to understand the source and rationale for their results. This is essential in regulated sectors where decisions must be traced back to a source.

Second, their small size means they can often run faster than LLMs, which can be a crucial factor for real-time applications. Third, SLMs provide businesses with more control over data privacy and security, especially if they are deployed internally or designed specifically for the business.

Additionally, although SLMs may require specialized training initially, they reduce the risks associated with using third-party LLMs controlled by external providers. This control is invaluable in applications that require strict data management and compliance.

Focus on developing expertise (and be wary of vendors who overpromise)

I want to be clear on this LLM and SLM are not mutually exclusive. In practice, SLMs can augment LLMs, creating hybrid solutions in which LLMs provide broader context and SLMs ensure precise execution. It is also still early, even when it comes to LLMs, so I always advise technology leaders to continue exploring the many possibilities and benefits of LLMs.

Additionally, while LLMs may scale well to a variety of problems, SLMs may not transfer well to certain use cases. It is therefore important to understand from the start the use cases to be addressed.

It is also important that business and IT managers devote more time and attention to developing the specific skills required for training, tuning, and testing SLMs. Fortunately, there is plenty of free information and training available through popular sources such as Coursera, YouTube, and Huggingface.co. Executives need to ensure their developers have ample time to learn and experiment with SLMs as the battle for AI expertise intensifies.

I also advise leaders to vet their partners carefully. I recently spoke with a company who asked my opinion on the claims of a certain technology vendor. In my opinion, they were either exaggerating their claims or were simply outdated in terms of understanding the capabilities of the technology.

The company wisely took a step back and implemented a controlled proof of concept to test the vendor’s claims. As I suspected, the solution simply wasn’t ready for prime time, and the company was able to walk away with relatively little time and money invested.

Whether a company is starting with a proof of concept or a live deployment, my advice is to start small, test often, and build on early success. I have personally experienced working with a small set of instructions and information, only to find that the results veer off course when I then feed more information to the model. This is why a slow and steady approach is a prudent approach.

In summary, while LLMs will continue to provide increasingly valuable capabilities, their limitations are becoming increasingly evident as businesses rely more heavily on AI. Supplementing with SLMs offers a way forward, especially in high-stakes areas that demand precision and explainability. By investing in SLM, businesses can future-proof their AI strategies, ensuring their tools not only drive innovation, but also meet requirements for trust, reliability and control.

AJ Sunder is co-founder, CIO and CPO at Responsive.