ISO/IEC TR 24029-1:2021
This standard provides a guide to assessing the robustness of an AI system, with a particular focus on neural networks.
As the usage of neural networks proliferates within industry, government, and academia, it becomes key to develop approaches to risk management that suit these new models. ISO/IEC TR 24029-1:2021 develops methods to measure and manage robustness in the context of neural networks.
How does ISO/IEC TR 24029-1:2021 define robustness?
Robustness refers to the ability of a system to maintain its quality of performance even when faced with various challenges. Methods have long existed for measuring robustness in non-AI systems. However, AI systems, especially neural networks, present a new set of challenges. Neural networks have a non-linear nature that can make it hard to cage in their behavior. ISO/IEC TR 24029-1:2021 considers this difference in order to formulate up-to-date methods for testing an AI’s robustness.
Traditional methods for generating system robustness focus on “strength” that prevents the system from being threatened by external factors and maintaining a consistent circuitry of its parts. Neural network robustness focuses more on “agility” in which the system can respond well to internal fluctuations that occur when presented with new data.
Robust neural networks are flexible yet internally consistent. They need to be able to cope with:
● Noise and Perturbations: Neural networks that are robust can handle imperfect “noisy” data or small variations in input without significant degradation in performance. This means that even if sensor errors, image noise, or slight mislabeling causes the data to be mildly corrupted, the model should still be able to produce reliable outputs.
● Adversarial Attacks: A robust neural network can resist adversarial attacks —deliberately crafted inputs designed to trick the model into making incorrect predictions. For example, adding a small, human-imperceptible noise to an image can cause a model to misclassify it, but a robust model should either resist such manipulations or at least not fail catastrophically.
● Out-of-Distribution Data: A robust network performs well not only on the data it was trained on but also on A) in-distribution data that is slightly different or on B) out-of-distribution data that might deviate from the training set. It carries a high capacity for generalization by preventing an overfit to the training data.
● Model Stability: A robust network exhibits stability in its predictions despite small alterations in hyperparameters, training procedures, or other factors. This means that even if the model undergoes slight variations during training, such as variations in batch size or rate of learning, it should not lead to large fluctuations in system performance.
A system’s ability to respond to these threats can be measured through three types of methods: statistical, formal, and empirical. These three categories are detailed in ISO/IEC TR 24029-1:2021. It is key to use a wide variety of approaches to validate a network’s robustness in order to promote safety, accuracy, and resilience.
The introduction of ISO/IEC TR 24029-1:2021 demonstrates how quickly our technologies are evolving. The resultant need for new testing procedures is a challenging one of utmost priority, but it also offers organizations a chance to stand out. By using ISO/IEC TR 24029-1:2021 to create a future-oriented approach to combating threats, your organization’s systems can provide an example for innovation-friendly robustness.