Paging indices vs Continuation tokens

Developers coming from the world of relational databases are well familiar with indexed paging. Paging is rather straightforward:

• Each row gets assigned a unique integer, starting from 0, and going with +1 increment for each additional row.
• The query is said to be paged, because constraints specifies that only the row assigned an index greater or equal to N and lower than N+PageSize are retrieved.

I call such as pattern a chunked enumeration: instead of trying to retrieve all the data at once, client app is retrieving chunks of data, potentially splitting a very large enumeration is into a very large number of much small chunks.

Indexed paging is a client-driven process. Indeed, it is the client code (aka the code retrieving the enumerated content) that decides how big each chunk is supposed to be. Indeed, it’s the client code that is responsible for incrementally updating indices from one request to the next. In particular, the client code might decide to make a fast forward read on the data.

Although indexed paging is well established pattern, I have found that it’s not such a good fit for cloud computing. Indeed, client-driven enumeration is causing several issues:

• Chunks may be highly heterogeneous in size.
• Retrieval latency on the server-side might be erratic too.
• If a chunk retrieval fails (chunk too big), client code has no option but to initiate a tedious trial-and-error process to gradually go for smaller chunks.
• Chunking optimization is done on the client side, injecting replicated logic into every single client implementation.
• Fast forward may be completely impractical to implement on the server side.

For those reasons, on the continuation tokens are usually favored in a cloud computing situation. This pattern is simple too:

1. Request a chunk, passing a continuation token if you have one (not for the first call)
2. Server returns an arbitrarily sized chunk plus an eventual continuation token.
3. If no continuation token is retrieved, then the enumeration is finished.
4. If a token is returned, then go back to 1, passing the token in the call.

Although, this pattern looks similar to the indexed paging, constraints are very different. Continuation tokens are a server-driven process. It’s up to the server to decide how much data should be send at each request, which yield many benefits:

• Chunk size, and retrieval latency can be made much more homogeneous.
• Server has (potentially) much more local info to wisely choose appropriate chunk sizes.
• Clients hold no more complex optimization logic for data retrieval.
• Fast-forward is disabled which leads to a simpler server-side implementation.

Then, there are even more subtle benefits:

• Better resilience against denial of service. If the server suffer an overload, then, it can optionally delay the retrieval by returning nothing but the current continuation token (a proper server-driven way of saying busy, try again later to the client).
• Better client resilience to evolution. Indeed, the logic that optimize the chunking process might evolve on the server-side over time, but client code is not impacted and implicitly benefits from those improvements.

Bottom line: unless you specifically want to offer support for fast-forward, you are nearly always better off relying continuation tokens in your distributed computing patterns.