As systems based on artificial intelligence (AI) have become the backbone of various public services such as transportation, finance, medicine, security, academics, medicine, and entertainment, the reliability and trustworthiness of AI artifacts/products, including robustness, algorithmic fairness, explainability, and transparency, is very important in every stage of the AI product life cycle: data preparation, algorithm design, development, and deployment. Trustworthy AI promotes machine learning (ML), private learning, and adversarial learning. Research in this area is multi-disciplinary. This review paper, written for AI practitioners, lists 394 references. The authors’ objective is to provide a comprehensive guide.

The paper starts with an introduction to the theoretical framework for AI trustworthiness. It defines and explains associated terms such as robustness, generalization, explainability, transparency, reproducibility, fairness, privacy preservation, and accountability. The authors point out that the terms and definitions in the AI trustworthiness domain defy universally accepted definitions, hence a systematic approach is required that spans entire system life cycle and development and deployment practices: data collection and preprocessing; robustness, explainability, generalization, fairness, and privacy for algorithm design; functional testing, performance benchmarking, simulation, and verification during development; and anomaly monitoring, human-AI interaction, fail-safe mechanisms, and hardware security for deployment. Smooth workflow and management requires documentation, auditing, cooperation, and ML operational standards. The paper reports the results of a comprehensive survey and analysis.

The paper concludes: trustworthy AI requires long-term research of the aforementioned topics, and end users should be aware of its many aspects. This will require international cooperation.