API Performance - Backend

Overview

In general, our assumption is this will run in a cloud native environment, in which network lag plays a significant role.

Response Times

Heavy requests <500ms P95
Many of our requests and responses are fairly "heavy". We still generally expect them to return within 500 ms. If the endpoint is above this then it's a candidate to be optimized. In general we try to avoid adding new endpoints that exceed this limit.
Examples

• Get a list of objects
• Update a significant system part

Light requests <100ms
Examples of a light request: get single object.

Lists

Limits

In general we limit list items returned to 100.

1. We expect that all internal functions will default to an appropriate limit, which may be less then than 100.
2. As possible further check or reduce this limit - eg if you never expect more then 10 results, then put a limit of 10.
3. Make it clear to the requesting interface (UI, SDK). Eg a user should never be "silently limited". Otherwise this can cause unexpected issues eg when exporting.

Example of Why Default Limits

Case:

1. User requests TaskList for project without passing a file.
2. Task.list() internally goes to fetch (without limit) and returns all (eg 8,000) tasks in the project.
3. This causes it to attempt to marshal 8,000 objects, which spikes the memory of the instance and exceeds memory limits of the instance, hard crashing it. (And the response to time out)

• Of course you could always have more memory - but the base assumption is the default service can operate with marginal memory - about 2 GB per core.

Takeaways
Keep in mind the assumption was that a file ID would always be present. While the frontend should always validate it in theory, it may not. By forcing a default limit on the backend prevent this from causing cascading issues.

Example

Python

class File():
  def file_list(limit=25):
    query = session.query(File)
    #... more filters and conditions
    query = query.limit(limit)	# usually near end
    return query.all()