Introduction
- 1.1 What We Talk about When We Talk about Things: Terminology
- 1.2 Achieving System Qualities
- 1.3 Life-Cycle Processes
- 1.4 Software Architecture
- 1.5 AI Model Quality
- 1.6 Dealing with Uncertainty
- 1.7 Summary
- 1.8 Discussion Questions
- 1.9 For Further Reading
This chapter is from the book
This chapter is from the book
This chapter is from the book
Science is about knowing; engineering is about doing.
—Henry Petroski
Software development is a team sport. Different skills and perspectives are essential to build high-quality software.
—Martin Fowler
ARTIFICIAL INTELLIGENCE (AI) is the topic of our time. But let’s face it: Not everyone is an expert in both software engineering (SE) and AI. Even among AI experts, not all of the concepts that were developed for “narrow machine learning” apply to emerging new technologies like foundation models. Yet, the behavior of systems depends on all components. That’s why it’s important to get all of it right: the AI parts and the non-AI parts, the architecture, the Dev and the Ops, and all relevant quality requirements. We need to engineer our AI systems—that is, we need to apply SE to systems that incorporate AI or rely on it for their core functionality. That’s what this book is about.
Our lead quotes provide a frame for this book. Henry Petroski is saying that knowledge alone is not enough to build systems; engineering must be applied as well. This is a book about engineering systems that are based on AI, not a book about extending the science of AI. Martin Fowler is saying that a wide variety of knowledge specialties must be applied to build software. In this book, we cover software architecture, DevOps, testing, quality control, and monitoring in the context of AI. Knowledge from all of these areas goes into creating a successful system that incorporates AI.
A system that incorporates AI has, roughly speaking, two portions: the AI portion and the non-AI portion. Most of what you will read elsewhere about AI systems focuses on the AI portion and its construction. That is important, but equally important are the non-AI portion and the means by which the two portions are integrated. The quality of the overall system depends on the quality of both portions and their interactions. All of that, in turn, depends on a great many decisions the designer must make, only some of which have to do with the AI model chosen and how it is trained, fine-tuned, tested, and deployed.
Accordingly, in this book we focus on the overall system perspective and aim to provide a holistic picture of engineering and operating AI systems. The goal is for you, your SE and AI teams, your company, and your users to get good value out of them, with effective management of risks.
To ensure you, the reader, and we, the authors, are on the same page about the core terms, we start this chapter with a few definitions. Subsequently, we discuss the qualities of a system and how quality is influenced. Specifically, system quality depends on AI model quality, the software architecture, and the processes used to build the system. We explore each of these elements in turn. Given the diverse backgrounds we expect our readers to have, some of the text in this chapter may not go into enough detail for some readers. If that’s you, and you cannot follow something we say here, worry not: The material in the subsequent chapters will help.
1.1 What We Talk about When We Talk about Things: Terminology
The book is about architecture and DevOps for AI-based systems, and the definitions we use for those terms are what we start with. Some of the definitions are direct quotes; you can find the details of their sources either in the footnotes or in the “For Further Reading” section at the end of this chapter. We will define other related terms as needed. We begin by defining what it means to be an AI system:
An AI system is a machine-based system that, for explicit or implicit objectives, infers, from the input it receives, how to generate outputs such as predictions, content, recommendations, or decisions that can influence physical or virtual environments. Different AI systems vary in their levels of autonomy and adaptiveness after deployment.1
The definition of AI systems has evolved over the years. For example, producing “content” as output was a recent addition in response to the rise of generative AI. While it has been updated recently, this definition shares a lot of aspects with other current definitions. The core differentiation in comparison to non-AI systems is that AI systems “infer” output, rather than computing it based on (often deterministic) explicitly coded algorithms. Note that we use the terms AI system and AI-based system synonymously in this book.
Next we define software architecture:
The software architecture of a system is the set of structures needed to reason about the system, which comprise software elements, relations among them, and properties of both.2
Although this definition stands for itself, we would like to point out that properties play a significant role. Hence, in this book we cover the various quality attributes—including reliability, performance, security, privacy and fairness, and observability—in a total of five chapters. We discuss achieving qualities after this section.
Let’s continue with the definition of DevOps:
DevOps is a set of practices intended to reduce the time between committing a change to a system and the change being placed into normal production, while ensuring high quality.3
This definition focuses on the goal of DevOps, which is speed in placing changes into production while maintaining quality—that is, getting new code into production, but not at the cost of lower quality. Typical practices include automation in testing, quality assurance, setting up and configuring environments, monitoring, and more. We discuss DevOps in more depth in Chapter 2.
The focus of this book is AI engineering, which we define next:
This definition of AI engineering emphasizes the integration of established software engineering practices into the development life cycle of AI systems. It recognizes that AI systems, despite their unique characteristics, can still benefit from the rigorous methodologies and best practices that have evolved in traditional software engineering.
As promised, we now discuss the achievement of system qualities.
