Network client rate limiter

[twitu]

The rate limiting bug #780 can be fixed comprehensively by implementing a middleware rate limiter for the newly written network module #1098. There are are many variations of rate limits across different exchanges. Most exchanges have different rate limits for different endpoints and return rate limit exceeded like errors when the limits are exceeded. However repeated violations can lead to suspension, so it's best to have decent rate limiting built into the client itself.

Exchange	HTTP Rate limit style
Deribit	token bucket
Bitmex	token bucket, layered, response header info
Okx	token bucket
Kraken	point system
Bitfinex	req/s
ByBit	req/s, layered
Binance	point system, response header info
DyDx	req/s, response header info

• token bucket - req/s with burstly allowance
• layered - limiter for large group of urls + sub-limiters for sub-group of those urls
• response header info - Request response returns info about remaining rate limits
• point system - point based system with different points for different kinds of order operations

In the current network client design req/s, token bucket and layered rate limiters can be supported as a middleware. Point system rate limiters are more complex and are not well suited to a middleware. The middleware will need to intercept responses (which may be out of order) and update it's state based on header values. It might be easier to enforce simple conservative req/s rate limiting on these endpoints or leave the specific rate limiting logic to the exchange specific client.

[rsmb7z]

Hi,
Adding the IB's rate limit requirement for reference here because we may potentially migrate to use this new socket client.

• API's limitation of 50 messages per second for all messages.
• Maximum of 50 orders per second being sent to the TWS
• The maximum number of simultaneous open historical data requests from the API is 50
• Making identical historical data requests within 15 seconds.
• Making six or more historical data requests for the same Contract, Exchange and Tick Type within two seconds.
• Making more than 60 requests (historial or live subscription) within any ten minute period.

[twitu]

API's limitation of 50 messages per second for all messages

This can be enforced by the rate limiting middleware.

To enforce other conditions the client needs to maintain a lot of state, or run logic based on the contents of the request. Moreover to enforce conditions like "The maximum number of simultaneous open historical data requests from the API is 50" the client need to keep track of open requests and get information from responses or by calling IB API. Such logic is best implemented in the IB client itself.

[twitu]

A layered, token bucket rate limiter with different quota per key will cover the rate limiting needs for most exchanges. Since there is no progress on boinkor-net/governor#193, we'll have handroll the implementation borrowing relevant parts from the governor repo.

This is a draft of the design I'm considering. We can reuse these modules from governor.

• Quota - A representation of the req/s and burst capacity that needs to be enforced
• Gcra - Genetic cell rate algorithm is the actual logic that decides if rate limit is exceeded or not and what the next rate limit state should be
• MonotonicClock uses Instant to give monotonic time
• InMemoryState stores the current state of the rate limiting algorithm for a particular key
• DashMapStateStore - Maps a key to the rate limiting state (in this case the InMemoryState). It's represented as type DashMapStateStore<K> = DashMap<K, InMemoryState>;

RateLimiter {
  default_quota: Gcra,
  quota_store: DashMap<String, Gcra>,
  state_store: DashMapStateStore,
}

impl RateLimiter {
    pub fn check_key(&self, key: &K) -> Result<PositiveOutcome, NegativeOutcome> {
        let gcra = self.quota_store.get_gcra(key).unwrap_or(self.gcra);
        let state = self.state_store.get(key).expect("A key needs to have a state");
        gcra.test_and_update::<K, C::Instant, S>(self.start, key, &self.state, self.clock.now());
    }

    pub fn check_key_parts(&self, key: &K) -> Result<PositiveOutcome, NegativeOutcome> {
         let parts = key.into_parts();
         parts.for_each(|part| { self.check_key(part)? })
    }
}

There are a few key differences between this and the governor implementation -

• Instead of K and V we use the specific types for nautilus use case where Key is String and the value is either Quota or rate limiting state. This rate limiter doesn't need the abstract interface of governor library
• It allows different keys to have different quotas. This is not currently supported by governor as tracked in above linked issue.
• It makes layered rate limiting possible by breaking a key into multiple parts. For e.g. a req for "post/v1/createOrder" can have two parts "post" and "post/v1/createOrder" with different limits enforced on both parts.

[cjdsellers]

The initial rate limiter for the base HttpClient is now in place, many thanks to @twitu 👏 and we've begun adding some rate limiting into the BinanceHttpClient.

Initially this is a default quota of 6000/minute for Spot and 3000/minute for Futures (where a normal message is worth 1), with reference to the following specs:
https://www.binance.com/en/support/faq/rate-limits-on-binance-futures-281596e222414cdd9051664ea621cdc3
https://www.binance.com/en/support/announcement/binance-spot-api-to-increase-request-weight-limits-9820396bf54644c39e666b4780622846

We're also experimenting with the hierarchical limiting for the heavier weight "request all order status" endpoints (20 times the weight of a normal message). The basic approach here is to allow right up to the specified API limits for standard weight messages, with the idea that the limiter will hold back requests just before Binance would otherwise respond with a 429 -- and in the case of heavier weight requests joining the message bursts, this would result in limiting from Binance.

The plan is we'll gradually proceed to add additional finer grain "keyed" limiting where it makes sense, and based on user feedback, and our own testing and observations from "the wild".

[cjdsellers]

Since the basic implementation is now in place and starting to be used, closing this issue in favor of more specific enhancement requests / bug reports as they may occur.