AI – Standards, Guidelines, Frameworks – an Overview

With the release of ISO/IEC 42001:2023, there has been a noticeable excitement regarding the establishment of a management system designed to effectively oversee the implementation of AI and its associated applications. It is also known that prior to release this management system there were quite a few standards, Guidelines, technical papers were released in last two years and there are a few which are under development.

Also, it is apparent that the standards, methods, and frameworks pertaining to AI must encompass aspects of data and security.

This article delves into the prominent AI/Data/Security related standards, particularly focusing on key ISO/IEC standards (Note: the list provided is for illustrative purpose only and may not be exhaustive).

A further examination reveals that these standards encompass various types of documentation, including Management Systems, Guidelines, Standards, Frameworks, Technical Specifications/Reports, Vocabulary, and more (additional details on the document types can be found on the ISO.ORG website).

(This write-up refers to all such documents as standards for the ease of use.)

Kudos to International Standards Organizations like ISO/IEC and national bodies such as BSI, ANSI, DIN, etc., for their commendable efforts in ensuring the establishment of such standards, contributing significantly to the robustness and trustworthiness of these systems.

Following diagram illustrates these key standards.

The recent release of the AI Management System is noteworthy, given the plethora of guidelines and technical specifications available since 2021 for organizational use. The application of Generative AI, along with the use of Large Language Models (LLMs) in business functions, underscores the need for organizations to adopt these models judiciously. It is crucial to strategically approach the adoption and carefully consider the associated risks.

The risks linked to adoption, particularly concerning data/model bias and system implications, outweigh the benefits. Some of these established standards outline prerequisites such as Management commitment, appropriate structure, relevant tools, enhanced controls, deliberate efforts to reduce bias, ethical practices, improved testing and assessment mechanisms, and robust quality assurance processes.

From a security standpoint, robust cybersecurity practices and effective risk management are imperative. The ISO 27001 (ISMS) standard has played a significant role in recent years, gaining added importance in the context of AI adoption. Standards addressing risks related to AI adoption are pivotal considerations.

On the data front, standards defining the Data Life Cycle Framework or ensuring Data Quality for Analytics in Machine Learning, as well as those pertaining to Process Management for Big Data, are fundamental. These standards play a key role in establishing comprehensive data management practices across the entire data life cycle.

Below is the compiled list of standards

Standards	Institution	Type	Domain	Stage
ISO 31000:2018	ISO/IEC	Guideline	Risk management Guidelines	Published – Feb 2018 (under Review now)
ISO/IEC FDIS 5339	ISO/IEC	Guideline	Guideline for AI Applications	Under Development
ISO/IEC 42001:2023	ISO/IEC	Management System	Information Technology — Artificial intelligence — Management system	Published – Dec 2023
ISO/IEC 24029-2:2023	ISO/IEC	Assessment	Assessment of the robustness of neural networks Part 2	Published – Aug 2023
ISO/IEC 8183:2023	ISO/IEC	Framework	Artificial intelligence – Data life cycle framework	Published – Jul 2023
ISO/IEC 25059:2023,	ISO/IEC	Quality Models	Quality Models for AI	Published – June 2023 To be revised
ISO/IEC 23894:2023	ISO/IEC	Guideline	guidelines for risk management of AI applications.	Published – Feb 2023
AS ISO/IEC 24668:2022	ISO/IEC	Framework	Process management framework for big data analytics	Published – Nov 2022
ISO/IEC 2382:2015	ISO/IEC	Vocabulary / Terminology	Information technology Vocabulary	Published – May 2015, Corrected version Oct 2022 (Site states that it is under review currently)
ISO/IEC 27001:2022	ISO/IEC	Management System	Information security management systems Requirements	Published – Oct 2022
ISO/IEC 22989	ISO/IEC	Vocabulary / Terminology / Taxonomy	AI concepts and Terminology	Published – July 2022 Amendment is in progress
ISO/IEC 23053:2022	ISO/IEC	Framework	Framework for AI Systems Using ML	Published – June 2022
ISO/IEC 38507:2022	ISO/IEC	Guideline	Governance implications of the use of artificial intelligence by organizations	Published – Apr 2022
ISO/IEC 27701	ISO/IEC	Framework	Privacy Information Management – extension to ISO/IEC 27001	Published – Aug 2019
IEC 62243:2012	IEC	Standard	Artificial Intelligence Exchange and Service Tie to All Test Environments (AI-ESTATE)	Published – Jun 2012
ISO/IEC DIS 5259-1	ISO/IEC	Vocabulary / Terminology / Taxonomy	Data quality for analytics and machine learning	Under Development
ISO/IEC CD 12792	ISO/IEC	Vocabulary / Terminology / Taxonomy	AI Transparency Taxonomy	Under Development
ISO/IEC TR 27563:2023	ISO/IEC	Technical Report	Use Cases – Security & Privacy in AI	Published – May 2023
ISO/IEC TR 24368:2022	ISO/IEC	Technical Report	overviews of ethical and societal considerations	Published – Aug 2022
ISO/IEC TR 24372:2021	ISO/IEC	Technical Report	Overview of computational approaches for AI systems	Published – Dec 2021
ISO/IEC TR 24027:2021	ISO/IEC	Technical Report	Bias in AI systems and AI aided decision making	Published – Nov 2021
ISO/IEC TR  24030:2021	ISO/IEC	Technical Report	AI Use Cases	Published – May 2021- being revised to ISO/IEC DTR 24030
ISO/TR 24291:2021	ISO/IEC	Technical Report	Applications of ML in Imaging and Medical Applications	Published – Mar 2021
ISO/TR 22100-5:2021	ISO/IEC	Technical Report	Safety of machinery Relationship with ISO 12100 Part 5: Implications of artificial intelligence machine learning	Published – Jan 2021
ISO/IEC TR 24028:2020	ISO/IEC	Technical Report	Overview of trustworthiness in artificial intelligence	Published – May 2020
ISO/IEC 4213:2022	ISO/IEC	Technical Report	Assessment of machine learning classification performance	Published – Oct 2022
ISO/IEC DTS 25058	ISO/IEC	Technical Specification	Guidance for Quality of AI	Under Publication – 60.00
ISO/IEC TS 12791	ISO/IEC	Technical Specification	Unwanted Bias Treatment of unwanted bias in classification and regression machine learning tasks	Under Development Stage: 50.20
ISO/IEC CD TS 8200	ISO/IEC	Technical Specification (DTS)	Controllability of automated AI systems	Under Development

                                                                                                                                                                                                                                                                                                                                                                                                (Stage as of 31^st Dec 2023)

Conclusion:

Standards play a pivotal role in implementing strong mechanisms throughout the entire process of planning, establishing an organization, designing, building, assessing, deploying, and continuously monitoring AI systems within an organization. The combination of standards, architectural and technical frameworks, tools, as well as adherence to governing laws and regulations, collectively contributes to the development of resilient, ethical AI solutions that not only benefit humanity at large but also enhance productivity and foster innovation. It is an exciting time and we look forward to development in the area of standards in the coming years.

Acknowledgements:

ISO.Org, AiThougts.Org, AI Standards.org