Fractile, a burgeoning name in the tech industry, has taken a significant step forward in AI development by licensing the Andes AX45MPV RISC-V vector processor. This move is set to revolutionise AI model inference – the process of deploying trained AI models for practical use – which is becoming the most substantial part of computational expenses, surpassing even the cost of model training. The integration of Andes’ processor technologies with Fractile's innovations marks a strategic attempt to enhance AI inference efficiency and performance.
Founded in 2022, Fractile has spent two years in "stealth mode," dedicated to crafting advanced chip designs that could potentially transform how AI workloads are handled. Their recent partnership with Andes Technology involves combining Andes' automated custom extension (ACE) technology and domain library with Fractile's advanced in-memory computing architecture. This collaboration is poised to yield an AI inference accelerator set to debut in Fractile’s first-generation data centre.
Fractile's new architecture optimises operations by carrying out 99.99% of necessary actions within the chip's own memory, effectively eliminating the need to continuously transfer model parameters between processor chips. This seamless integration not only promises to boost energy efficiency significantly, boasting impressive trillions of operations per second per watt (TOPS/W), but also ensures that latency is drastically reduced, fostering instantaneous responses in AI applications.
The push towards improving inference capabilities comes at a time when the AI industry is increasingly focusing on scaling inference tasks. This trend is highlighted by OpenAI's recent launch of their latest large language model, o1, which demands far greater inference computational power than its predecessors. Fractile aims to meet such escalating demands head-on with their novel approach.
Fractile's CEO, Walter Godwin, emphasises the inherent challenges in AI hardware development, noting the rapid pace at which leading AI models evolve compared to the slower process of chip development. He highlights the adaptability of software-programmable vector processors like those offered by Andes as crucial for maintaining relevance amidst these dynamically changing demands. According to Godwin, many current systems, like Nvidia's CUDA, require additional computational steps because GPUs are not inherently optimised for AI workloads. Fractile, by eliminating these inefficiencies, aims to streamline software, making it both simpler and faster.
The momentum behind Fractile's developments has been bolstered by a successful investment seed round. Key investors include Kindred Capital, the defence alliance NATO's innovation fund, and Oxford Science Enterprises, which led the funding. Participation also came from Cocoa, Innovia Capital, and a group of notable angel investors with roots in AI and semiconductor industries, reflecting strong confidence in Fractile’s potential impact on the AI landscape.
As the company progresses, their innovations could hold substantial implications for energy consumption, efficiency, and AI model responsiveness, a sector rapidly prioritising optimisation to accommodate ever-growing computational demands.
Source: Noah Wire Services