AMD Unveils AMD-135M Small Language Model with Innovative Speculative Decoding

By Luisa Crawford

In a significant advancement for artificial intelligence technology, AMD has introduced its first small language model dubbed AMD-135M, which employs speculative decoding to enhance the efficiency and performance of AI models. This development, announced on AMD's official platform, marks a milestone in AMD's AI initiatives, aiming to address some of the challenges associated with large language models.

Details of the AMD-135M Model

The newly launched AMD-135M is part of the Llama family and represents AMD's initial foray into the small language model (SLM) landscape. The training of this model was conducted using AMD Instinct™ MI250 accelerators and involved processing 670 billion tokens. The resulting work led to the creation of two distinct models: the AMD-Llama-135M and the AMD-Llama-135M-code.

  • AMD-Llama-135M: Trained on general data, this model involved a six-day training period on four MI250 nodes.
  • AMD-Llama-135M-code: Designed with a focus on code data, this variant underwent an additional four days of fine-tuning with 20 billion more tokens specifically curated for code.

A notable aspect of AMD's approach is its commitment to transparency and community collaboration; all training code, datasets, and model weights for the AMD-135M are open-sourced. This enables developers worldwide to replicate the model and contribute to future advancements in both small and large language models (LLMs).

Speculative Decoding and Its Benefits

One of the groundbreaking features of the AMD-135M is the inclusion of speculative decoding, a method that addresses the inefficiencies inherent in traditional autoregressive language models. Conventional models typically suffer from low memory access efficiency, generating only one token per forward pass. Speculative decoding mitigates this limitation by using a smaller draft model to produce candidate tokens, which are subsequently verified by a larger target model. This innovation permits the generation of multiple tokens each forward pass, thereby enhancing memory access efficiency and reducing inference times.

Enhanced Inference Performance

AMD has rigorously tested the performance of the AMD-Llama-135M-code model as a draft model for CodeLlama-7b across various hardware configurations, including both the MI250 accelerator and Ryzen™ AI processor. These tests have demonstrated substantial improvements in inference speeds when speculative decoding is applied, highlighting the model's capability to streamline the training and inferencing processes on selected AMD platforms.

Future Prospects

AMD's release of the AMD-135M is not just a standalone technological milestone but a step towards fostering greater innovation within the AI community. By making the model's framework open-source, AMD seeks to encourage further research and development in the AI field. The company invites developers to engage with and contribute to the ongoing evolution of AI technologies.

For those interested in more comprehensive technical details regarding the AMD-135M model, AMD has published a full technical blog available on their official website.

This announcement underscores AMD's commitment to advancing artificial intelligence by introducing innovative solutions that leverage the power of both hardware and software to enhance AI model efficiency.

Source: Noah Wire Services