A recent report titled "The Economic Potential of Generative AI: The Next Productivity Frontier," released by McKinsey & Company, has projected that generative AI could deliver a substantial economic boost, potentially adding between $2.6 trillion to $4.4 trillion in value to the global economy. Notably, this value is anticipated to be most significant in four sectors: customer operations, marketing and sales, software engineering, and research and development (R&D).
As organisations increasingly recognise the potential of generative AI, many are beginning to develop applications within the Amazon Web Services (AWS) ecosystem. A crucial topic for many product managers and enterprise architects is understanding the associated costs and optimisation strategies for deploying these AI applications effectively.
The AWS Blog outlines key considerations for cost and performance optimisation in generative AI applications, particularly those utilising Retrieval Augmented Generation (RAG) frameworks. RAG is increasingly recognised for its ability to leverage language models effectively, enhancing precision in responses by grounding them in corporate data.
Key pillars of cost optimisation are highlighted, including model selection, token usage, and analysis of inference pricing plans. The process begins with model selection, whereby organisations must identify the most appropriate model for their specific use cases. Subsequent steps include benchmarking these models against high-quality datasets to determine their suitability. Model choice further entails recognising that different models may carry varying performance and pricing structures, while model customisation focuses on adapting foundation models (FMs) with business-specific training data for enhanced efficacy.
Token usage is another vital aspect, as the cost of employing generative AI models directly correlates with the number of tokens processed. Strategies such as caching frequently asked questions can help to minimise costs while improving performance. In terms of cost structures, AWS offers two pricing options: On-Demand, which is versatile for most models and charges based on token usage, and Provisioned Throughput, which may be more suited for high-demand workloads at a higher cost.
Other miscellaneous factors influencing cost include security measures, such as content filters, and the implications of employing vector databases for storing generative AI application data. These databases are essential for managing increasing data loads effectively, which can exacerbate costs over time. The AWS Blog also emphasised the importance of chunking strategies in optimising data handling and related expenses.
The discussion extends to a detailed evaluation of directional costs across various scenarios based on organisational size and application usage. For example, the estimated costs for a generative AI application servicing between 500,000 to 7 million input queries per month ranged from approximately $12,577 to $134,252 annually, depending on the setup and resource demands.
Specific functionalities of Amazon Bedrock, a fully managed service offering access to a variety of leading AI models, are delineated as well. Users can engage with various LLMs through a unified API, utilising different models for tasks such as generating text embeddings to facilitate the RAG process.
The report delves into advanced technical considerations impacting both performance and cost, such as input and output token management, vector embedding costs, and the operational efficiency of deployed models. It also highlights that generative AI applications could be designed to include guardrails, which help filter inappropriate content and enhance data security.
In future posts, the AWS Blog aims to further uncover methods for estimating the business value of generative AI applications and the factors that influence this potential. The blog provides insights from AWS specialists, with particular emphasis on the ongoing evolution within the generative AI landscape and its implications for businesses striving to harness its capabilities effectively.
Source: Noah Wire Services