2025 playbook for successful business AI, from assistants to evals

2025 is poised to be a very important year for business AI. Last year saw a lot of innovation, and this year will see the same. This has made it more important than ever to get back home AI ways to stay competitive and create value for your customers. From developing AI assistants to maximizing value, here are five critical areas that businesses should prioritize in their AI strategy this year.

1. Agents: the next generation of automation

AI agents are no longer imaginary. In 2025, they are the most important tools for businesses that want to improve operations and improve customer interactions. Unlike traditional software, agents supported by large-scale language models (LLMs) can create multiple options, handle multiple complex tasks, and seamlessly integrate with tools and APIs.

By early 2024, agents weren’t ready for the big time, making frustrating mistakes like fake URLs. They started to do better when the models of the major languages ​​started to improve.

“Let me put it this way,” said Sam Witteveen, co-founder of Red Dragon, a company that creates corporate sponsors, and recently reviewed the 48 sponsors they created last year. “The interesting thing is, what we built at the beginning of the year, most of them worked well at the end of the year because the species started well.” Witteveen shared this in this video podcast that we recorded to discuss these five major trends in detail.

Models are getting better and better looking, and are being trained to do aging jobs. Another thing that the sample providers are looking for is the way to use the LLM as a judge, and if the samples are cheap (something we will discuss below), companies can use three or more samples to choose the best products to make a decision. on.

Another part of the secret sauce? Retrieval-augmented generation (RAG), which allows agents to store and reuse information efficiently, is gaining momentum. Imagine a travel assistant bot that not only organizes trips but books flights and hotels in real time based on preferences and budgets.

Take away: Businesses need to identify situations where affiliates can provide higher ROI – whether it’s customer service, sales, or internal traffic. The use of tools and advanced thinking skills will define the winners in these areas.

2. Evals: the foundation of reliable AI

Evaluation, or “evals,” are the backbone of any robust AI deployment. This is the way to choose an LLM – among the hundreds available – to use in your career. This is important for accuracy, and aligning AI results with business goals. A good evaluation ensures that the chatbot understands the tone, the recommendation system provides the right options, and the predictive model avoids costly mistakes.

For example, an evaluation of a company’s customer support chatbot might include metrics such as uptime, response accuracy, and customer satisfaction.

Many companies have been spending a lot of time planning inputs and outputs to match the company’s expectations and workflows, but this can take a lot of time and resources. As models improve, many companies are saving effort by relying more on models that do the job, so choosing the right one becomes more important.

And this process forces clear communication and good decisions. “When you understand more about how to evaluate the results of a product and what you want, not only does it make you better with LLMs and AI, it makes you better with people,” Witteveen said. “When you can make it clear to someone: This is what I want, this is what I want it to look like, this is what I’m hoping for. Once you talk about it, people suddenly do a lot better.”

Witteveen has seen company executives and other designers tell him: “Oh, you know, I’ve gotten really good at giving advice to my team for being good at technical skills or just being good at, you know, looking at writing. The right model values.”

By writing clearly, businesses force themselves to define goals – the success of humans and machines.

Take away: Creating high quality evals is important. Start with clear indicators: accurate response, time to fix, and alignment with business goals. This ensures that your AI not only reacts but adapts to your brand’s needs.

3. Cost effective: expand AI without breaking the bank

AI is getting cheaper, but system deployment remains important. Management at any LLM level results in significantly lower costs. Intense competition among LLM providers, and from open source competitors, results in frequent price drops.

Meanwhile, graduate program options are making LLMs more successful.

Competition from new hardware vendors such as Groq’s LPUs, and changes to legacy GPU supplier Nvidia, are significantly reducing costs, making AI available for many applications.

The real achievements come from optimizing the way the models are used in the application, which is the time of knowledge, not the time of training, when the models are first built using the data. Other methods such as model distillation, along with new hardware, mean that companies can achieve more with less. It’s no longer whether you can afford to buy AI — you can do more work for less this year than six months ago — but how you scale it.

Take away: Create a cost analysis for your AI projects. Compare hardware options and explore methods like distillation to reduce cost without compromising performance.

4. Memory Update: improving AI for your users

Customization is no longer optional – it’s expected. In 2025, memory-based AI machines are making this a reality. By remembering users’ preferences and past experiences, AI can provide personalized and useful experiences.

Personalization is not widely discussed or publicized because users are often apathetic to AI programs that store personal information to improve performance. There are privacy concerns, but also an interesting feature when the photographer spits out answers that show they know a lot about you – for example, how many children you have, what you do, and what you like. OpenAI, for one, protects ChatGPT user information in its own memory – which can be turned off and deleted, even if it’s only turned on by default.

Although businesses using OpenAI and other brands that are doing this may not feel the same, what they can do is create their own memory system using RAG, to ensure that the data is safe and effective. However, businesses must tread carefully, balancing personalization and privacy.

Take away: Create a clear process for personalization. Access systems and transparent policies can build trust while providing value.

5. Computing is a time of experimentation: new skills and the limits of thinking

Inference is where AI meets the real world. In 2025, the goal is to make the process faster, cheaper and more powerful. Sequential thinking – where models break down tasks to make sense – is changing the way businesses approach complex problems. Tasks that require critical thinking, such as strategic planning, can now be successfully solved by AI.

For example, the OpenAI version of o3-mini is expected to be released later this month, followed by the full version of o3 later. They introduce advanced thinking that breaks down problems into manageable parts, thus reducing the guesswork of AI and improving decision-making accuracy. This cognitive improvement works in areas such as mathematics, writing, and applied science where increased reasoning can help – although in other areas, such as synthetic language, progress may be limited.

However, this change will also come with increased computing power, and higher operating costs. The O3-mini is meant to provide the highest level of performance.

Take away: Identify work processes that can benefit from advanced analytical methods. Establishing a unique concept for your company, and choosing the best models, can give you the edge here.

Conclusion: Turning information into action

AI in 2025 isn’t just about new tools; it’s about making smart decisions. Whether it’s recruiting agents, refining evals, or increasing value, the key to success lies in smart implementation. Businesses need to follow this with a clear and focused approach.

For more on what’s going on, check out the full podcast between Sam Witteveen and myself here: