Nvidia, in collaboration with DataStax, unveiled a groundbreaking technology that significantly reduces storage requirements for businesses implementing generative AI systems. This new offering, named the Nvidia NeMo Retriever, integrates seamlessly with DataStax’s AI platform and boasts the capability to decrease data storage volume by 35 times compared to traditional methods.
The announcement was made in light of predictions that enterprise data is set to exceed 20 zettabytes by 2027, which highlights the pressing need for efficient data management solutions. Kari Briski, Vice President of Product Management for AI at Nvidia, noted in an interview with VentureBeat that today’s enterprise unstructured data stands at around 11 zettabytes. This figure is comparable to 800,000 copies of the Library of Congress, with 83% of this data being unstructured, of which half consists of audio and video formats. Briski remarked, “Significantly reducing these storage costs while enabling companies to effectively embed and retrieve information becomes a game changer.”
The NeMo Retriever technology is expected to revolutionise the scalability of generative AI across various enterprise applications. The tech has already shown impressive results for the Wikimedia Foundation, which leveraged the Nvidia and DataStax solution to expedite the processing time for 10 million Wikipedia entries from an arduous 30 days to just under three days. This system is equipped to manage real-time updates to hundreds of thousands of entries that are regularly modified by 24,000 volunteers globally.
Chet Kapoor, CEO of DataStax, emphasized the necessity of integrating context from existing enterprise data alongside large language models for content management. “You can’t just rely on large language models for content — you need context from your existing enterprise data,” he stated. He further explained how their hybrid search capability merges semantic search with traditional text search, enhanced by Nvidia’s re-ranker technology, which produces the most pertinent results in real time across a global scale.
Security of enterprise data, particularly concerning sensitive information, remains an essential priority. Kapoor also highlighted the complexity of making vast repositories of private data accessible to AI systems without compromising security. He pointed out that for companies like FedEx, which store approximately 60% of their data within DataStax’s products, the preservation of sensitive personal details is paramount: “That’s not going to Gemini or OpenAI anytime soon, or ever.”
The newly launched technology has begun to see early adoption, particularly within the financial services sector, where firms have rapidly adapted to these innovations despite existing regulatory challenges. Kapoor cited examples from Commonwealth Bank of Australia and Capital One, highlighting their advanced positions within the landscape of AI application in finance.
Nvidia is also looking towards the future, planning to extend the capabilities of its technology to accommodate more intricate document formats, particularly through multimodal document processing. Briski expressed enthusiasm for tackling this challenge, stating, “We’re seeing great results with multimodal PDF processing — understanding tables, graphs, charts and images and how they relate across pages. It’s a really hard problem that we’re excited to tackle.”
The new Nvidia NeMo Retriever technology is set to address the challenge faced by many enterprises that grapple with vast amounts of unstructured data, aiming to prepare their information assets for effective AI deployment while ensuring data security and maintaining economical storage solutions. This offering is accessible immediately via the Nvidia API catalog, which includes a 90-day free trial license.
The ongoing developments in this sector underscore an increasing emphasis on building robust enterprise AI infrastructure, reflecting a shift in focus from mere experimentation to expansive deployment, where efficient data management and cost-effectiveness are pivotal for success.
Source: Noah Wire Services