Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

routing by packet lookup #257

Merged
merged 17 commits into from
Sep 18, 2023
Merged

routing by packet lookup #257

merged 17 commits into from
Sep 18, 2023

Conversation

michaeldjeffrey
Copy link
Contributor

This PR changes the way packets get their routing determined, for the purpose of having more freedom in how we store SessionKey Filters.

The when of packets

The first time a packet is see, we take a lock on the phash and start finding if it's routable.
Any packets that come in during that time will queue themselves.
Once the routing is figured out, the initial packet, and all the queued packets are routed, and the routing information is put into ETS.
Any packet coming along after routing has been determined will immediately retrieve it's routes.

Multi-buy is always checked.

But there's a lot of changes to SKF storage...

The last_used timestamp for an SKF is removed. Worst case scenario is the applicable SKF is at the end of an ETS table, but that lookup is only done one time per hash.

This means we can keep SKF unique by session_key, and delete in constant time.

Other things of note

  • We no longer assume that if 2 routes have a valid SKF, the latest one wins. Now, both routes will be routed to.
  • There's a cleanup job for phash routing. I based it loosely off of packet-purchaser unique packet hash counter.
    • It crawls the table more often than the removal window to keep the table from getting bigger than can be traversed in a single window.
    • packet-purchaser was doing this while being a default router.
  • There as a multi-buy test that was relying on a race condition.
    • There was a max_copies of 3, and we were simulating 2 servers for every request. The test asserted that 3 requests were sent, but that could only happen if request 2 and 3 were sent at the same time.
    • It's been updated to wait until multi-buy has received more than max_copies worth of function calls, and makes sure it's sent less than the expected requests.

do not check how many requests have been sent until the multi-buy
function has been called enough times that it could be more than what we expect.
When it packet comes into routing...
1. It's never been seen before and we need to figure out the routing
   information for it.
2. We've seen it before, and have not yet determined the routing, in
   which case it get's queued.
3. We've seen it before, and already know what to do.
SKF tables are not sets with no timing information. We can do direct
inserts and deletes for greater performance.
SKF are unique in their route tables since the last_used key has been
removed. If more than one route has an applicable skf, let's send to it.
every 15 seconds, anything older than 2 minutes will be cleaned out.
src/hpr_routing.erl Outdated Show resolved Hide resolved
src/hpr_multi_buy.erl Show resolved Hide resolved
src/hpr_routing.erl Outdated Show resolved Hide resolved
src/hpr_routing.erl Outdated Show resolved Hide resolved
src/hpr_routing.erl Outdated Show resolved Hide resolved
src/hpr_routing.erl Outdated Show resolved Hide resolved
src/hpr_routing.erl Outdated Show resolved Hide resolved
src/hpr_routing.erl Outdated Show resolved Hide resolved
Do not let more than 1 cleanup routine start.
Check if the current cleanup is registered, and bail if there's another
PID claiming to be the cleanup.
when more than one packet is told to establish routing, the first will
get the lock and continue. The second will see it's not the first entry,
and go back to looking up routing again. Most likely to queue itself.
inserts and deletes are from a set table with a known key, not an
ordered_set with a timestamp as the key.
@michaeldjeffrey
Copy link
Contributor Author

The current largest DevAddr used has ~13.5k Session Key Filters.

Providing a packet that will never match any of the SKFs. (running with a single core locally).

Number of SKfs Worst case time
100 4ms
1,000 27ms
10,000 217ms
20,000 431ms
50,000 1027 ms
100,000 2048ms

@michaeldjeffrey michaeldjeffrey merged commit fedc2ea into main Sep 18, 2023
15 checks passed
@michaeldjeffrey michaeldjeffrey deleted the mj/packet-lookup branch September 18, 2023 20:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants