This chapter is from the book
This chapter is from the book
This chapter is from the book
1.6 Dealing with Uncertainty
Non-symbolic AI models are inherently probabilistic. In other words, there is some probability (hopefully small) that the model output is incorrect. Whether this is acceptable depends on your organization’s risk tolerance in terms of the system you are building. This consideration leads to an increasing emphasis on AI risk assessment, responsible AI, AI safety, standard conformance, and regulatory compliance.
Because of the inherent uncertainty in AI systems, your organization should conduct a risk assessment focused on the use of the system being constructed. This risk assessment should look at human values and AI safety, among other factors.
1.6.1 Human Values
Some AI systems are unpredictable and autonomous. Their unpredictable behavior necessitates that risk assessments must evaluate the impact, in terms of both consequence and likelihood, if these systems violate ethical behavior.
In Chapters 7–11, we will focus on key quality attributes, such as reliability, security, privacy, and fairness. In any system, these qualities are balanced against one another. Design decisions for a system must reflect system priorities and tradeoffs should be explicitly considered.
1.6.2 Safety
Safety is an evolving concern in AI systems. Initially centered on physical and psychological safety to individuals, AI safety now encompasses concerns about AI systems with dangerous capabilities, such as chemical, biological, radiological, and nuclear (CBRN) concerns; misuse potential including cybersecurity, broader societal harms via misinformation/disinformation at scale, AI controllability, and even existential threats for humanity.
This change reflects that the meaning of safety is now more about the severity of AI system risks rather than the types of risks. That is why we have not included a chapter on AI safety, as we believe the architectural and DevOps approaches to evaluating and achieving quality attributes are the ultimate way to mitigate AI system risks and reduce them to acceptable levels, given the risk appetite for an organization or wider society.
