ISO-IEC 25059:2023

ISO/IEC 25059 establishes a standardized framework for evaluating AI system quality, addressing AI-specific characteristics including learning capabilities, incomplete data handling, probabilistic reasoning, explainability, and fairness. It extends the SQuaRE standards to ensure thorough quality assessments specifically designed for artificial intelligence applications.

Overview of ISO/IEC 25059

ISO/IEC 25059 represents a significant advancement in AI system evaluation standardization. Unlike traditional software quality standards, it directly addresses the unique challenges of AI technologies by providing:

• A structured quality model specifically for AI systems
• Clear, consistent terminology for describing and assessing AI system quality
• A comprehensive framework for verifying quality requirements completeness

This new international standard acknowledges that AI systems operate fundamentally differently from conventional software, requiring specialized evaluation approaches that consider their adaptive, probabilistic nature.

Historical Context and Development

The development of ISO/IEC 25059 stemmed from recognizing that existing software quality standards inadequately addressed AI's unique characteristics. Traditional software operates on explicit instructions, while AI systems employ complex learning mechanisms and evolve based on data inputs.

The standard was developed through collaboration between the International Standards Organization (ISO) and the International Electrotechnical Commission (IEC), bringing together experts from AI research, software engineering, quality assurance, and ethics. This multidisciplinary approach ensures the standard addresses technical quality aspects while considering broader societal implications. ISO/IEC 25059 builds upon the foundation established by the SQuaRE series, particularly ISO/IEC 25010, but extends these principles to address AI-specific characteristics. This integration creates a cohesive framework for AI regulatory compliance that organizations can implement alongside existing quality management systems.

Key AI-Specific Characteristics

Learning and Adaptation

ISO/IEC 25059 examines how effectively an AI system learns and evolves over time. It looks at how efficiently the system acquires new knowledge, how well it adapts to unfamiliar situations, and whether it can maintain stable performance while integrating new information. The standard also considers how systems avoid reinforcing undesirable patterns, ensuring that learning processes do not introduce or amplify bias. Together, these measures help organizations determine whether an AI system’s learning behavior meets required quality thresholds and remains reliable throughout its lifecycle.

Handling Incomplete Data

​Because real‑world data is often messy, inconsistent, or incomplete, ISO/IEC 25059 evaluates how well AI systems cope with imperfect inputs. This includes their tolerance for missing data, their ability to manage uncertainty, and their robustness when faced with noisy or partial information. The standard also looks at how gracefully a system degrades when data quality declines. These considerations ensure that AI systems remain dependable even when operating under less‑than‑ideal data conditions, which is the norm in most practical applications.

Probabilistic Reasoning

​Many AI systems produce probabilistic outputs rather than deterministic answers. ISO/IEC 25059 assesses how well these systems calibrate their probabilities, communicate confidence levels, evaluate risks, and determine appropriate thresholds for decision‑making. By addressing these aspects, the standard helps prevent systems from overstating or understating their certainty. This supports sound decision‑making by both automated processes and human operators and aligns with the growing emphasis on transparency as a competitive advantages in AI adoption.

Explainability and Transparency

Trust in AI depends on users understanding how and why systems reach particular conclusions. ISO/IEC 25059 emphasizes the importance of decision transparency, interpretability, traceability, and technical accessibility. These elements mirror regulatory expectations such as those in the EU AI Act, which requires clear transparency mechanisms for high‑risk AI systems. The standard reinforces the idea that explainability is not optional but foundational to responsible AI deployment.

Fairness and Bias Mitigation

​Bias in AI can lead to harmful or inequitable outcomes. ISO/IEC 25059 addresses this by focusing on bias detection, fairness metrics, disparate impact assessment, and strategies for mitigation. By embedding fairness directly into its quality characteristics, the standard recognizes that ethical considerations are essential components of AI system quality rather than secondary concerns.

2025 Updates and Recent Developments

As of 2025, ISO/IEC 25059 has evolved to address emerging technologies and applications, including:

• Edge AI considerations for quality assessment on devices with limited resources
• Evaluation frameworks for multimodal AI integrating various data types
• Enhanced guidance for continuous learning systems that adapt during deployment
• New quality characteristics for human-AI collaboration

The standard has gained significance with the implementation of the EU AI Act and other global AI regulations. According to the ISO, compliance with ISO/IEC 25059 has become a key mechanism for demonstrating conformity with regulatory requirements across multiple jurisdictions. Its relevance is reinforced by recent research published in IEEE Transactions on AI, which indicates that organizations implementing ISO/IEC 25059 report 43% fewer critical AI system failures and significantly higher user trust scores compared to those using ad-hoc quality assessment approaches.

Implementation Guidelines

Implementing ISO/IEC 25059 requires a systematic approach:

1. Define clear, measurable quality requirements specific to your AI application
2. Select appropriate measurement methodologies for each quality characteristic
3. Implement structured evaluation procedures
4. Document assessment processes and results thoroughly

Organizations should integrate the standard into their AI development lifecycle—from conception through deployment and maintenance. The U.S. National Institute of Standards and Technology recommends pairing ISO/IEC 25059 implementation with organizational AI governance frameworks to ensure consistent application. Effective implementation requires strengthening capabilities for comprehensive AI assurance, including technical expertise, appropriate tooling, and organizational commitment to quality principles.

Industry Impact and Challenges

​ISO/IEC 25059 is shaping practices across a wide range of industries. In healthcare, it supports the development of reliable and explainable diagnostic AI systems. Financial institutions use it to evaluate models involved in risk assessment and fraud detection, while the transportation sector applies the standard to assess the safety and performance of autonomous vehicle technologies. Public‑sector organizations rely on it to help ensure fairness in AI systems used for government service delivery. Across these domains, organizations adopting the standard report more structured development processes and a stronger ability to address stakeholder concerns about reliability, transparency, and trustworthiness.

​Despite its comprehensive scope, implementing ISO/IEC 25059 is not without challenges. Some quality characteristics, such as explainability, are inherently subjective and difficult to measure consistently. Thorough assessments can require substantial resources, and organizations must keep pace with rapidly evolving AI capabilities. There is also the ongoing need to balance trade‑offs between competing quality attributes, such as accuracy versus interpretability or robustness versus efficiency. As a result, organizations must tailor their use of the standard, focusing on the characteristics most relevant to the purpose, context, and risk profile of each AI system.

Advancing Trustworthy AI

ISO/IEC 25059 represents a significant milestone in AI technology maturation, providing a structured framework specifically tailored to AI's unique challenges. As AI systems increasingly integrate into critical infrastructure and daily life, rigorous quality assessment becomes essential. This standard offers a comprehensive approach to ensuring AI systems meet expectations for reliability, fairness, transparency, and overall quality. Expert guidance from Nemko Digital can help organizations turn the principles of ISO/IEC 25059 into actionable processes that strengthen the quality, reliability, and overall trustworthiness of their AI systems.