diff --git a/doc/pagination-and-sorting.md b/doc/pagination.md similarity index 81% rename from doc/pagination-and-sorting.md rename to doc/pagination.md index 73f7262..14cbc04 100644 --- a/doc/pagination-and-sorting.md +++ b/doc/pagination.md @@ -1,17 +1,15 @@ -# Pagination and Sorting - -## Pagination +# Pagination When implementing pagination, you **MUST** use either Cursor Pagination (preferred) or Offset Pagination, on the formats detailed below. -### Offset Pagination +## Offset Pagination This strategy is based on these query parameters: | Parameter | Type | Description | |-----------|---------|----------------------------------------------------------------------------| -| `offset` | integer | Zero-based index of the first item to retrieve. **MUST** be named `offset` | -| `limit` | integer | Number of items to get. **MUST** be named `limit` | +| `offset` | integer | Zero-based index of the first item to retrieve. | +| `limit` | integer | Number of items to get. | Implementations **SHOULD** implement and document default and max values for `limit`. @@ -21,24 +19,43 @@ Implementations **SHOULD** implement and document default and max values for `li GET /api/v1/bus-stops?city=Oslo&offset=10&limit=20 ``` -#### Response format +### Response format The response **MUST** contain the following fields: | Parameter | Type | Description | |--------------|---------|----------------------------------------------------------------------------| -| `items` | array | **MUST** be named `items`. | -| `totalItems` | integer | The total number of items across all pages. **MUST** be named `totalItems` | +| `items` | array | Returned items. | +| `totalItems` | integer | The total number of items across all pages. | | `limit` | integer | The requested `limit`, or max limit if given `limit` was over max. | -### Cursor / Keyset Pagination +**Example** + +```json +{ + "items": [ + { + "id": "100", + "name": "Item 100" + }, + { + "id": "101", + "name": "Item 101" + } + ], + "totalItems": 2, + "limit": 100 +} +``` + +## Cursor / Keyset Pagination This strategy is based on these query parameters: | Parameter | Type | Description | |-----------|---------|----------------------------------------------------------------------------------------| -| `cursor` | string | An opaque string identifying the next page of items to get. **MUST** be named `cursor` | -| `pageSize` | integer | Number of items per page. **MUST** be named `pageSize` | +| `cursor` | string | An opaque string identifying the next page of items to get. | +| `pageSize` | integer | Number of items per page. | Cursor-based pagination is based on a `cursor` that is created when handling requests from the client. The cursor is returned to the client in the response body. The cursor points to the next page of items. Sorting parameters, `pageSize` and filters **MAY** also be embedded in the cursor. @@ -59,7 +76,7 @@ The response includes a cursor for the next page. To fetch the next page: GET /api/v1/bus-stops?city=Oslo&pageSize=20&cursor=eyJpZCI6MTAwfQ ``` -#### Cursor key selection +### Cursor key selection The cursor **MUST** encode a value (or set of values) that uniquely and stably identifies a position in the sorted result set. @@ -78,22 +95,40 @@ Example cursor key for encoding a single value (e.g. database id): ``` -#### Encoding +### Encoding The cursor **MUST** be URL-safe (no URL-encoding required). Because the cursor should be opaque to the client and may contain internal details, it **MAY** be Base64 encoded. For cursors with multiple values, a common solution is to have JSON in string value and then Base64-encode the string. If the cursor contains data that you do not want to expose, the cursor **MAY** be encrypted and then Base64 encoded. -#### Response format +### Response format The response **MUST** contain the following fields: | Parameter | Type | Description | |-----------|---------|-------------------------------------------------------------------------------------------------------------------------------------| -| `items` | array | **MUST** be named `items`. | -| `cursor` | string | An opaque string pointing to next item to get. If no more items, cursor value is not returned to client. **MUST** be named `cursor` | +| `items` | array | Returned items. | +| `cursor` | string | An opaque string pointing to next item to get. If no more items, cursor value is not returned to client. | -### Choosing a Strategy +**Example** + +```json +{ + "items": [ + { + "id": "100", + "name": "Item 100" + }, + { + "id": "101", + "name": "Item 101" + } + ], + "cursor": "eyJpZCI6MTAwfQ" +} +``` + +## Choosing a Strategy Use the comparison table below to select the pagination strategy that best fits your use case. @@ -107,14 +142,6 @@ Use the comparison table below to select the pagination strategy that best fits As a rule of thumb, cursor pagination **SHOULD** be used unless: offset pagination DB queries are not too heavy and inserts and deletes are infrequent OR jumping to a specific position must be supported. ## Sorting -Sorting **MAY** be implemented without pagination, but when using pagination you **MUST** also use sorting. - -:eyes: If you implement sorting, you **MUST** use query parameter `sort`. -You **MAY** also allow sorting on multiple levels, and allow specifying sort order (desc / asc). -In your service, always use a secondary sorting on a unique id, so that two entries with the same primary sorting -(e.g. created date) are always sorted in the same order. -Example: -```http -GET /api/v1/bus-stops?city=Oslo&sort=name,asc&sort=something,desc -``` \ No newline at end of file +When using pagination you **MUST** return elements in a deterministic order. Without one, the boundary between pages is undefined: the same item may appear on multiple pages or be skipped entirely as the client paginates, and cursors can no longer reliably point to the next page. +See [sorting](sorting.md) for more details. diff --git a/doc/sorting.md b/doc/sorting.md new file mode 100644 index 0000000..ee973b2 --- /dev/null +++ b/doc/sorting.md @@ -0,0 +1,139 @@ +# Sorting + +All returned collections **MUST** be sorted, which it means that the order is deterministic: the same +request **MUST** return items in the same order. + +## Client-controlled sorting + +Client-controlled sorting **MUST** use the query parameter `sort`. + +| Parameter | Type | Description | +|-----------|--------|-----------------------------------------------------| +| `sort` | string | The field(s) to sort on. | + +Example: +```http +GET /api/v1/bus-stops?sort=name +``` + +### Sort field and direction + +A `sort` value is a field name, optionally followed by a comma (`,`) and a +sort direction, either `asc` (ascending) or `desc` (descending): + +```http +GET /api/v1/bus-stops?sort=name,asc +``` + +- If the direction is omitted, the direction **MUST** default to `asc`. +- The direction tokens `asc` and `desc` **MUST** be treated as + case-insensitive. + +### Sorting on multiple fields + +You **MAY** allow sorting on multiple fields by repeating the `sort` parameter: + +```http +GET /api/v1/bus-stops?sort=name&sort=created +``` + +Each field **MAY** have its own direction: + +```http +GET /api/v1/bus-stops?sort=name,asc&sort=created,desc +``` + +### Allowed sort fields + +You **MUST** document which fields are sortable. + +### Case sensitivity, collation and null ordering + +- For string fields, the API **SHOULD** document whether sorting is + case-sensitive and which collation/locale is used. For Norwegian data, + sorting **SHOULD** order the letters `æ`, `ø` and `å` according to Norwegian + collation rules. +- The API **SHOULD** document where `null` or missing values are placed + (sorted first or last). + + +### Tiebreaking +To achieve a deterministic order, the sorting implementation **MUST** append a unique +field with a stable value (typically `id`) as a final tiebreaker. For example, if a +client requests sorting on `name` and `name` is not unique across all items, a secondary +sort **MUST** also be applied on a unique field. + +**Example** + +```http +GET /item/v1/items?sort=name,asc +``` + +```json +{ + "items": [ + { + "id": "100", + "name": "Item A" + }, + { + "id": "101", + "name": "Item A" + }, + { + "id": "99", + "name": "Item B" + } + ] +} +``` + +In this example, the client requested items sorted on `name` ascending. There +are two items with the name "Item A" but the backend applies a secondary sort on `id`, so they are +always returned in the same order (`100` before `101`). The item with `id` 99 +is still returned last, because the primary sort is on `name` and "Item B" +sorts after "Item A". + +### Documenting in OpenAPI + +The documentation requirements above (allowed sort fields, default order, case +sensitivity, collation and null ordering) **MUST** be expressed on the `sort` +query parameter in your OpenAPI specification. + +**Example** +```json +{ + "parameters": [ + { + "name": "sort", + "in": "query", + "description": "Field(s) to sort on. Repeat the parameter to sort on multiple fields. Each value is a field name optionally followed by a direction: `,`.\n\n**Sortable fields:** `name`, `created`.\n\n**Collation:** String fields are sorted case-insensitively using Norwegian collation.\n\n**Null ordering:** Items with a `null` or missing value for the sort field are placed last, regardless of sort direction.", + "schema": { + "type": "array", + "items": { + "type": "string", + "pattern": "^(name|created)(,(asc|desc))?$" + }, + "default": ["name,asc"], + "example": ["name,asc", "created,desc"] + } + } + ] +} +``` + +## Sorting not controlled by client +When returning a collection, there is no requirement that the client can control the sorting. When sorting is not controlled by the client, you **MUST** however document the order on the response field for the collection: + +**Example** +```json +{ + "items": { + "type": "array", + "description": "Sorted by `name` ascending, with `id` as a tiebreaker.", + "items": { + "$ref": "#/components/schemas/Item" + } + } +} +``` diff --git a/guidelines.md b/guidelines.md index 10ac66b..bca0df6 100644 --- a/guidelines.md +++ b/guidelines.md @@ -520,10 +520,17 @@ A "de-facto" standard for correlating a request throughout a microservice archit GET /api/v1/bus-stops?city=Oslo&cursor=eyJpZCI6MTAwfQ&pageSize=20 ``` -:eyes: If you implement sorting, you **MUST** use query parameter `sort`: +See [pagination](doc/pagination.md) for more details. + +All returned collections **MUST** be sorted, which it means that the order is deterministic: the same +request **MUST** return items in the same order. + +:eyes: If you implement client-controlled sorting, you **MUST** use query parameter `sort`: > GET /api/v1/bus-stops?city=Oslo&sort=name,asc&sort=something,desc -See [pagination and sorting](doc/pagination-and-sorting.md) for more details. + +See [sorting](doc/sorting.md) for more details. + ### 6.2 Partial Responses - :eyes: You **MAY** let clients choose which fields to include to reduce data transfer