Introduce cursor-based pagination

[Venefilyn]

Description

Subtask of #2567

This would involve displaying the API differently and preferably done as a major version change. Which is something that should ideally be done during the move to FastAPI as mentioned in #2567.

With it you could have a data style such as

{
  "data": [...], // the requested data
  "cursor": "abc123" // pointing to the next data in the pagination
}

Benefit

Cursor-based pagination makes sure we have an efficent pagination while also ensuring that the data will not move, such as with page or offset-based pagination where a new entry would shift the whole set by one, meaning you could see the same data or worse miss something if the new entry is not added in the beginning.

Showing the cursor and data results this way also ensures that it is clear the endpoint is paginated.

Importance

Medium.

What is the impacted category (job)?

General

Workaround

• There is an existing workaround that can be used until this feature is implemented.

Participation

• I am willing to submit a pull request for this issue. (Packit team is happy to help!)