Hugging Face has introduced its latest offering in the arena of Artificial Intelligence services: Hugging Face Generative AI Services (HUGS), positioning itself as a competitor to Nvidia's Inference Microservices (NIMs). The announcement, made this week, highlights Hugging Face's intent to provide a more versatile solution for deploying Large Language Models (LLMs) and other AI models across a broader spectrum of hardware platforms.

Similar to Nvidia's NIMs, which are containerised model images, HUGS facilitate the deployment of models by incorporating all necessary components within a container image. This alleviates the need for intricate preparations such as optimising large language models like vLLM or TensorRT LLM for scale. Instead, users can deploy these pre-configured containers via platforms like Docker or Kubernetes and interface with them through standard OpenAI API calls.

Built on Hugging Face's open-source Text Generation Inference (TGI) and Transformers frameworks, HUGS extends compatibility to a variety of hardware, including Nvidia and AMD GPUs. The service plans to expand its support to specialized AI accelerators such as Amazon's Inferentia and Google's TPUs, although integration with Intel's Gaudi is not currently on the horizon.

While these services are based on open-source technologies, they are not offered for free. Deploying HUGS on cloud services such as AWS or Google Cloud incurs a cost of approximately $1 per container hour. In comparison, Nvidia's NIMs are priced at $1 per hour per GPU when deployed in the cloud, with an on-premises option costing $4,500 annually per GPU. This pricing strategy makes HUGS a more cost-effective option, especially for deploying larger models like Meta's Llama 3.1 405B, which requires the use of eight GPUs.

For smaller-scale deployments, Hugging Face has made these images available on DigitalOcean's cloud platform without extra charge for the service itself, though users will still need to cover compute costs. DigitalOcean has recently introduced GPU-accelerated Virtual Machines (VMs) utilising Nvidia's H100 accelerators, priced between $2.5 and $6.74 per GPU hour based on usage or a commitment plan.

Enterprise users subscribed to Hugging Face's Enterprise Hub for $20 a month per user have the flexibility to deploy HUGS on their private infrastructure, offering further adaptability in model deployment.

Hugging Face maintains a strategic focus on some of the market’s most popular open models, including Meta's Llama 3.1 available in 8B, 70B, and 405B versions, Mistral AI’s Mixtral models, Nous Research's Hermes fine-tunes, Google’s Gemma 2, and Alibaba’s Qwen 2.5. The company plans to extend support for additional models in the near future, potentially including Microsoft's Phi-series of LLMs.

For users averse to the notion of purchasing packaged open-source software, Hugging Face suggests that individuals can create their own containerised models using open-source tools like vLLM, Llama.cpp, TGI, or TensorRT LLM. However, the main value proposition offered by HUGS, and similarly by Nvidia's NIMs, is the optimisation and tuning of these containers for peak performance, which can significantly reduce time and effort for enterprises looking to utilise AI models effectively.

Source: Noah Wire Services