Multiple .so files?

[paddor]

While trying to reduce the size of a snap (snapcraft) package, I noticed that sqlite3, among a few other gems with extensions (nokogiri, ffi) have multiple .so files. After bundling into vendor/bundle, snapping, and installing the snap, this is what's in it:

$ find /snap/my-ruby-app/current/app/vendor/bundle/ruby/3.3.0/gems/  -name '*.so'|xargs du -sh
...
1.5M    /snap/my-ruby-app/current/app/vendor/bundle/ruby/3.3.0/gems/sqlite3-2.1.1-x86_64-linux-gnu/lib/sqlite3/3.1/sqlite3_native.so
1.5M    /snap/my-ruby-app/current/app/vendor/bundle/ruby/3.3.0/gems/sqlite3-2.1.1-x86_64-linux-gnu/lib/sqlite3/3.2/sqlite3_native.so
1.5M    /snap/my-ruby-app/current/app/vendor/bundle/ruby/3.3.0/gems/sqlite3-2.1.1-x86_64-linux-gnu/lib/sqlite3/3.3/sqlite3_native.so
...

It seems there's an .so file for each recent Ruby version like 3.3, 3.2, 3.1?
(Same goes for gems installed on the system itself under ~/.gem/ruby/3.3.3/gems/)

What's the purpose of these? I can't think of a reason, given that they're installed by and for a single Ruby version (3.3.5 in this case)?

Maybe off-topic, but I also noticed that some gems have copies (verified with sha1sum) of their .so files under ~/.gem/ruby/3.3.5/extensions/. What's up with that? Just asking in case anyone knows. I can't find any other info on this.

Thanks for any info or pointers.

[flavorjones]

@paddor thanks for asking. The sqlite3 gem has a C extension (this is what we call the ".so" file) to call into libsqlite. The project releases "native" gem packages that include precompiled extensions, to make installation faster and easier. Each minor version of Ruby has a distinct "binary API" (called an "ABI" for application binary interface) that requires the extension to be compiled specifically for it. Rubygems and bundler today only support one gem file for each gemversion+rubyplatform combination, meaning that it's necessary to put a precompiled extension for every supported Ruby version in each native gem.

So, for example, the native gem package for sqlite3 v2.3.0 for linux x86_64 gnu is named "sqlite3-2.3.0-x86_64-linux-gnu.gem" and contains extensions for ruby 3.0 – 3.3 inclusive.

This is explained much more in depth in https://github.com/flavorjones/ruby-c-extensions-explained and also in a couple of conference talks that I gave on the subject, linked to from that project's README.

If you want to optimize for a smaller gem file or a smaller install size (trading off speed of installation), you can install the "vanilla" gem file ("sqlite3-2.3.0.gem") that will compile from source at installation time. This project's INSTALLATION.md file will point you in the right direction if that's something you want to do.

given that they're installed by and for a single Ruby version

It would be great if rubygems had a feature for "conditionally install this file based on the version of Ruby", but it doesn't. Maybe this is an opportunity to start a conversation on the rubygems project?

I also noticed that some gems have copies (verified with sha1sum) of their .so files under ~/.gem/ruby/3.3.5/extensions/

This again is the behavior of rubygems with respect to extensions. I believe there are ways to "clean up" unnecessary files but I'm unfamiliar personally with the mechanics.

[paddor]

Thanks a lot. That cleared it up. Thanks for the clear explanation.