Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace allocator with mimalloc #323

Closed
aumetra opened this issue Aug 27, 2023 · 4 comments · Fixed by #350
Closed

Replace allocator with mimalloc #323

aumetra opened this issue Aug 27, 2023 · 4 comments · Fixed by #350

Comments

@aumetra
Copy link
Member

aumetra commented Aug 27, 2023

We originally used mimalloc as the allocator for Kitsune and the search service via the mimalloc crate and then switched to jemalloc as kind of a band-aid fix for unsoundness inside the mimalloc crate (they return wrongly aligned pointers. Big no no.)

There is the mimalloc-rust crate though which provides bindings that are actually sound.

I personally prefer mimalloc over jemalloc since it's smaller, faster to build, and (according to their benchmarks) outperforms a lot of other allocators.

Right now we spend ~40-50sec of our total build time on building jemalloc. We can probably reduce that with mimalloc.

@erlend-sh
Copy link

Has an issue about the unsoundness been raised in mimalloc? It appears to be more regularly updated.

@aumetra
Copy link
Member Author

aumetra commented Aug 27, 2023

Yes, the issue is here: purpleprotocol/mimalloc_rust#96

Didn't seem to have gotten much attention from the maintainers though. If they fix it, I guess we can still switch.
Allocators are luckily not the hardest thing to switch in Rust.

@aumetra
Copy link
Member Author

aumetra commented Aug 27, 2023

I'm just doing some little PRs on the mimalloc-rust crate right now. Their code seems largely fine, just a few small things here and there.

And I'm looking into just upgrading it to v2.x since version 2 brings improvements in memory usage and fragmentation (according to the mimalloc README)

@aumetra
Copy link
Member Author

aumetra commented Aug 27, 2023

I just read through the issue again, apparently it's just an issue with the mimalloc crate when compiled with the secure feature?
That sounds a bit weird. I didn't think page protections, layout randomizations, etc. could cause unaligned pointers..

Edit: Yeah, it's probably not the secure mode (see purpleprotocol/mimalloc_rust#98)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants