Citi Gen AI Summit 2025 Takeaways: LLM Operations and Infrastructure

Everyone is excited about Gen AI, from its many applications and use cases, but before the technology can truly take off and reach its widest adoption, industry leaders need to address issues related to data infrastructure, hallucinations, and scaling.

I had the opportunity to dig into these challenges and brainstorm potential solutions at the Citi Gen AI Summit with a distinguished panel of experts that included: Yash Sheth, Co-Founder & COO, Galileo; Jobi George, Global VP of Business Development, Weaviate; Debanjan Saha, CEO, DataRobot; and Barry Dauber, Vice President of AI GTM, Databricks.

During our discussion, we looked at the hurdles for successfully bringing AI, particularly LLMs and AI agents, into production. These included ensuring security and privacy, measuring and capturing ROI, obtaining high-quality and relevant data, and using humans in governance and risk management.

Here are some of my key takeaways from the discussion.

Concerns about Gen AI security, privacy and accuracy remain

Panelists noted the lingering concerns among everyone from CIOs to engineers related to risks around full-scale deployment of Gen AI in enterprises. They noted that Gen AI’s promise can only be truly realized if implementers address the fundamental challenges around data, governance, and evaluation/measurement. Brands could suffer if something goes wrong or sensitive information gets leaked, so many companies have focused their initial deployment efforts on internal-facing apps and building from there.

Deploying is the “easy” part — managing AI agents is an even greater challenge

Enterprises will have to dedicate resources to coordinate AI agent workflows and maintain them as business operations evolve. In other words, agents are anything but “set it and forget it,” especially when it comes to tracing and logging an agent’s performance. Issues like model drift and data drift can degrade an agent’s performance. Panelists noted the many challenges associated with debugging agents once deployed, so companies will either need to build in-house expertise or work with evaluation companies like Galileo to manage issues that arise.

Enterprise data stack will need to change to allow Gen AI to flourish

The opportunity to capitalize on proprietary structured and unstructured data is greater than ever before, but to take advantage of that opportunity and unlock insights, companies will first need to rebuild their internal data architecture with innovations like lakehouses, data meshes, and vector databases. From production systems to data access control, companies will need to reexamine their legacy data stack and see where improvements can be made.

Humans will remain “in the loop” but gradually recede from low-risk operations

Panelists had a variety of viewpoints on how, and where, humans should interact with AI agents. One good suggestion was that enterprises identify low-risk opportunities for agents to shine and fully automate those tasks. Then, engineers could design AI agents for higher-risk use cases and incorporate human involvement. Over time, companies could gradually take off the AI agent’s “training wheels” and let them take over more controls from the humans in the loop. Ultimately, though, panelists said it was important to have humans accountable for agent actions across the board.