How to Shape Responses to API Calls
Many solutions exist to optimize API calls concerning lean messaging (aka data parsimony). Five API design patterns addressing this issue are presented: PAGINATION, WISH LIST, WISH TEMPLATE, CONDITIONAL REQUEST, and REQUEST BUNDLE.
From the book
From the book
From the book
Modern software systems are distributed systems. Mobile and Web clients exchange information with backend API services, often hosted by a single or even multiple cloud providers, and various backends trigger activities in each other. Independent of the technologies and protocols used, request and response messages travel through one or several API clients and their service providers in such systems. In another article, we introduced the notion of API quality and discussed what the right service granularity in API designs is.
In this article, we are specifically interested in the following question, also referred to as response shaping:
How to avoid unnecessary data transfer in API and message design?
Many solutions exist to optimize API calls concerning lean messaging (aka data parsimony). Here, we discuss widely used ones in the form of five API design patterns, PAGINATION, WISH LIST, WISH TEMPLATE, CONDITIONAL REQUEST, and REQUEST BUNDLE.
For more details on these patterns (and many more), please consult our book Patterns for API Design: Simplifying Integration with Loosely Coupled Message Exchanges.
Option 1: Pagination
PAGINATION is a widely used option to avoid unnecessary data transfer in an API. Sometimes complex data elements can contain large amounts of repetitive data (for instance, data records). If an API client only requires a subset of this information at a time, it might be better to send the information in small chunks rather than in one large transmission.
For example, consider hundreds of person records being contained in the data, but a client displaying the information page by page (with 20 records per page) and requiring user input to advance to the next page. Showing only the current page, and maybe prefetching one or two pages in either stepping direction, might be far more efficient in terms of performance and bandwidth use than downloading all available records before even starting to display the data. In such design situations, you can consider the PAGINATION pattern.
| Pattern: PAGINATION | |
| Problem | How can an API provider deliver large sequences of structured data without overwhelming clients? |
| Solution | Divide large response data sets into manageable and easy-to-transmit chunks (also known as pages). Send one chunk of partial results per response message, and inform the client about the total and remaining number of chunks. Provide optional filtering capabilities to allow clients to request a particular selection of results. For extra convenience, include a reference to the next page from the current one. |
The message exchanges resulting from applying PAGINATION are as follows (three pairs of requests and replies are shown in the figure, as well as some of the required control metadata:
Click to view full-sized image
The pattern comes in variants such as offset-based pagination, cursor-based pagination, and time-based pagination. The general idea is easy to understand; that said, implementing the pattern is usually harder than it seems. Additional design concerns include where, when, and how to define the page size, how to order results, where and how to store intermediate results, how long to store it (deletion policy, timeouts), how to deal with request repetition, and how to correlate pages with the initial, previous, and next requests.