Unlocking value from data: How AI agents will conquer 2024

If 2023 was the year of AI-powered chatbots and search engines, 2024 was all about AI assistants. What started from Devin earlier this year has grown into a phenomenon, giving businesses and individuals a way to transform the way they work in a variety of areas, from software and development to personal tasks such as planning and booking vacation tickets.

Among these various activities, we also saw the rise of data agents this year – AI-powered assistants that perform a variety of tasks on a set of data objects. Some worked on data integration while others worked on low-level tasks, such as analysis and pipeline management, making things simpler and easier for business users.

The benefit was efficiency and cost reduction, which makes many ask themselves: How will things change for data groups in the coming years?

Gen AI Agents took over data operations

Although technology has been around for a while, allowing businesses to do more important things, the rise of generative AI has taken things completely to another level.

With next gen AI natural language skills and tools, agents can go beyond simple reasoning and respond by planning multi-level actions, autonomously communicating with digital systems to complete actions while working with other agents and people at the same time. They also learn to improve their behavior over time.

Devin of Cognition AI It was the agent’s first major contribution, supporting engineering projects on a large scale. Then, the big players started to offer highly reviewed businesses and affiliates with the help of their brands.

In an interview with VentureBeat earlier this year, Google Cloud’s Gerrit Kazmaier said he heard from customers that their users were constantly facing challenges including manual handling of data groups, reducing cycle time for data pipelines and analytics and improving data management. Basically, the teams didn’t come up with ideas on how to make value out of their data, but they didn’t have time to act on those ideas.

To fix this, Kazmaier explained, Google also updated BigQuery, which provided its main tools, with Gemini AI. The resulting capabilities not only provide businesses with the ability to discover, clean and organize low-level data – breaking down data silos and ensuring quality and consistency – but also help manage and analyze pipelines, freeing teams to focus on high-value activities.

Several businesses today use Gemini’s capabilities in BigQuery, including a fintech company More thanwhich made it possible for Gemini to understand complex tools in order to create quizzes. A Japanese IT company Don’t worry it also leverages Gemini SQL’s capabilities in BigQuery to help its data groups deliver insights faster.

But, identifying, planning and assisting in the analysis was just the beginning. As startup models evolved, even granular operations—initiated by startups focused on their domains—were overseen by deep, agent-driven systems.

For example, AirByte and to eat created headlines in the data integration category. The first introduced an agent that created data connections from a link to the API documentation in seconds. Meanwhile, the latter has advanced its development services with agents who create business-class APIs – either reading or writing information on any topic – using natural language.

From San Francisco Ultimate AIfor its part, it looks at various data processes including documentation, testing and editing, and the new DataMates technology, which used an AI agent to pull stories from the entire data stack. A few other basics, including Redbird and RapidCanvashas done a similar job, saying that they offer AI assistants that can handle up to 90% of the data needed in AI and analytics pipelines.

Agents support RAG and more

Beyond big data applications, the technology’s potential has also been explored in areas such as retrieval-augmented generation (RAG) and the downscaling of automated navigation. For example, the group behind the vector database Weaviate recently discussed the idea of RAG agenta process that allows AI agents to access multiple tools – such as web searches, calculators or API programs (eg Slack/Gmail/CRM) – to retrieve and validate data from multiple sources to increase the accuracy of responses.

Also, towards the end of the year. Snowflake Intelligence appeared, giving businesses the opportunity to set up data systems that will not only use business information stored in their Snowflake system, but also standard and unmodified data on third-party tools – such as sales in databases, records in information. such as SharePoint and more in productivity tools such as Slack, Salesforce and Google Workspace.

With these additional words, agents display relevant information by answering natural language queries and taking action on the generated information. For example, a user can ask the data provider to enter the information displayed in a customizable form and upload the file to their Google Drive. They can be forced to write to the Snowflake tables and make changes as needed.

More to come

While we may not have explained anything about the data usage that was seen or announced this year, one thing is for sure: The technology will continue to exist. As gen AI models continue to evolve, the adoption of AI agents accelerates, many organizations, regardless of their sector or size, are choosing to outsource repetitive tasks to specialized agents. This will directly translate into efficiency.

As evidence of this, in a recent survey of 1,100 technology executives conducted by Capgemini82% of respondents said they want to integrate AI assistants into their teams within the next 3 years – up from 10% currently. More importantly, about 70 to 75% of respondents said they would trust an AI assistant to analyze and generate information on their behalf, as well as perform tasks such as creating and managing iteratively.

This provider-driven change could also mean significant changes in how data groups operate. At present, the results of the agent are not designed, which means that a person must manage the service to meet their needs. However, with a little progress in the coming years, this gap will disappear – giving AI support teams that can be fast, accurate and not prone to the mistakes that are often made by people.

So, to summarize, the roles of data scientists and experts that we see today can change, where users can go to the AI ​​control center (where they can control the actions of AI) or the valuable tasks that the system does. can struggle to do.