As the adoption of Generative AI in the enterprise accelerates, one question that will be on Management’s mind: “What does AI cost?” The answer, like most things in business, is nuanced: it depends on the specific needs of the enterprise.
For a rough estimates, you can look at comparable businesses. For example, small enterprises with limited budgets might begin with AI-powered chatbots to automate customer support, freeing up existing staff for more complex tasks. But leaving at rough estimates is not a good approach.
Underestimating the importance of proper budgeting for adopting and operationalizing AI in the enterprise can be disastrous. A cautionary tale comes from cloud adoption, where unforeseen costs have triggered an exodus of businesses from cloud back to on-premises infrastructure.
Many sources, like those in Reference 1, meticulously list and explain the various cost factors involved. In a diagram here, these costs have been mapped onto different stages of Generative AI adoption. I haven’t elaborated on all stages in the diagram because some warrant their own detailed illustration.
Let us look at why a particular cost matters in the indicated stage. The assumption here is that the enterprises aim to fully implement and manage Gen AI themselves. The understanding would need little tweaking, but would hold good even when the enterprise decides to partially or completely outsources this activity.
Consultant Cost:
- This is the cost of a consultant or consulting firm who will provide guidance and support throughout the AI adoption process.
- In the diagram where this cost is not indicated, it is minimal compared to other costs at that stage.
Talent Cost:
- Primarily encompasses the costs associated with reskilling current staff and hiring new talent.
- Exercise caution, as advised by Hugo Huang, regarding the specific skills and headcount required for both AI solution implementation and ongoing maintenance.
- Meticulous planning and budgeting are essential to prevent cost overruns.
- While not explicitly indicated in the diagram in some stages, staff costs are assumed to be integrated within other categories such as Software Development Cost, Data Preparation Cost, and Rollout costs.
Cloud Cost:
- Initial cloud costs will arise during the training phase, gradually scaling up to the target level during Rollout.
- Carefully anticipate and plan for these costs, which are distributed across multiple stages.
- If opting for an on-premises setup instead of cloud-based infrastructure, accurately factor in the equivalent costs.
- Transitioning to on-premises infrastructure may necessitate a comprehensive review of existing infrastructure, potentially requiring additional efforts and budget allocation.
Inference Cost:
- Initial inference costs will begin during training, escalating significantly during the three stages of Rollout. During steady state operations, this will be a major contributor to the on-going cost.
Data Preparation Cost:
- Encompasses costs associated with data scientists, data analysts, and computing infrastructure (either cloud-based or on-premises).
- Covers tasks such as cleansing, organizing, processing, and labelling data before it’s ready for training.
- Expect a considerable time and money investment for this stage.
- Additional costs may arise for implementing scalable and efficient data storage and data management systems.
Software Development Cost:
- Involves costs related to building and testing applications that facilitate user interaction with the deployed GenAI solution.
- Includes expenses for IT talent, licenses, and necessary infrastructure.
Fine-Tuning Cost:
- Accounts for costs of personnel and infrastructure.
- If synthetic data generation is planned using the trained model, include the related costs as well – such as personnel cost, inference cost, cloud cost etc..
- Budget for this cost only if Fine-Tuning is part of the strategy.
Prompt Engineering Cost:
- Allocate a budget for this cost if Prompt Engineering is chosen instead of, or in conjunction with, Fine-Tuning.
- Primarily consists of costs associated with trained personnel.
Integration Cost:
- This is for the cost of integrating newly built solutions with existing systems to ensure seamless user experience.
- Involves the time and expertise of staff who manage these existing systems, even if not directly involved in GenAI implementation.
- May necessitate changes to existing systems, requiring additional budget allocation.
Operations Cost:
- Covers costs associated with the deployment and ongoing maintenance of the entire solution.
HBR’s Hugo Huang suggests management strategies for CEO/CIO for controlling costs. The CEO/CIO will constitute teams to carry out the GEN Ai adoption. While reviewing and signing-off the budgets / costs, the understanding of where the different costs occur will help.
Maryam Ashoori, in her article gets to the nuts-bolts of how to calculate different costs. A combination of similar approaches will help the teams constituted by CEO/CIO to make sure that costs well budgeted and under control.
To maintain competitive advantage, AI adoption in the enterprise is an unavoidable event. Cost estimation and control is one of the key pillars upon which a successful adoption of AI rests.
References:
- What CEOs Need to Know About the Costs of Adopting GenAI by Hugo Huang
https://hbr.org/2023/11/what-ceos-need-to-know-about-the-costs-of-adopting-genai?ab=HP-latest-image-2 - Decoding the True Cost of Generative AI for Your Enterprise by Maryam Ashoori
https://www.linkedin.com/pulse/decoding-true-cost-generative-ai-your-enterprise-maryam-ashoori-phd/