ServiceNow open sources Fast-LLM to help businesses train AI models 20% faster

Teaching a master’s degree (LLM) is one of the most expensive and time-consuming endeavors in business. The new open source version being released today is ServiceNow can make a big difference, with the promise of training 20% ​​faster, saving businesses time and money.

Fast-LLM technology is already in development within the company, helping ServiceNow accelerate its LLM efforts. Fast-LLM helped train ServiceNow’s StarCoder 2 LLMwhich the company released earlier this year. StarCoder itself is an open source project, as well, which benefits from the contributions of Hugging Face, Nvidia and others. ServiceNow also uses Fast-LLM for large-scale training, multi-trillion-tolerance operations from existing models, and maintenance operations.

Because it is an open technology, anyone can use Fast-LLM to help advance AI learning, including better performance. The idea is that it can be a replacement for the existing AI training pipeline with minimal changes. A new open-source project aims to diversify the commonly used AI systems, including the Open-source PyTorchand a series of new features about data parallelism and memory management.

“When you’re dealing with compute clusters that cost hundreds of millions and training sessions that cost millions of dollars, 20% can be a huge savings in dollars and time and overall CO2,” Nicholas Chapados, VP of research. at ServiceNow, he told VentureBeat.

Innovations that enable Fast-LLM to accelerate AI education

AI companies understand the challenge of training AI well. VentureBeat Transform 2024 had a group that discussed this issue, it explains in detail the ways in which you can increase the scaling.

The Fast-LLM approach is not about building infrastructure; and about optimizing the quality of the teaching materials available.

“We looked carefully at all the tasks necessary to teach a large variety of languages, especially large language languages,” said Chapados. “We carefully plan how computing is distributed to individuals within the GPU, and how memory is used by those models.”

Fast-LLM’s competitive advantage stems from two primary factors that contribute to differentiation. The first is the Fast-LLM method for ordering computation, which defines how computations are performed in AI training. Chapados explained that Fast-LLM uses a new technology that ServiceNow calls “Breadth-First Pipeline Parallelism.”

“This is an important scientific capability in terms of how computations are done, both within a single GPU and across multiple GPUs,” Chapados said.

The second major innovation focuses on memory management. In large-scale teaching activities, memory fragments over time. This means that memory is broken down over time as learning progresses. This fragmentation causes memory to fail, preventing training groups from using all available memory.

“We have been very careful in the way we design Fast LLM to solve the problem of memory fragmentation when teaching these large languages,” said Chapados.

How businesses can use Fast-LLM today to improve learning

The Fast-LLM framework is designed to be accessible and sustainable for businesses. It works as a replacement for the PyTorch environment and integrates with existing tutorials.

“For any developer or any researcher, it’s a simple configuration file that allows you to specify all the architectures that are important,” said Chapados.

Running a rapid training program has a number of benefits and can allow businesses to experiment more.

“It reduces the risk of major studies,” Chapados said. “It prepares users, researchers and model builders to be interested in training large runs, because they won’t have to fear that it will be too expensive.”

Looking to the future, the hope is that as an open project, Fast-LLM will be able to grow rapidly, benefiting from external contributions. ServiceNow has already had success with this approach with StarCoder.

“Our goal is to be transparent and responsive to the community’s input into the implementation of the plan,” Chapados said. We are still receiving early feedback on what people like, what they can do with it and our goal is to expand this. “