Plugin doesn't work with Logstash 8.10+

[mashhurs]

Description

TL;DR: Ruby 3.1+ (even upgrading mail dependency to 2.8) breaks the plugin behavior.

Details:
Logstash 8.10+ versions use Ruby 3.1. Ruby 3.1 extracts bundled gems such as net-smtp, net-imap, etc.. dependencies. When running imap plugin with Logstash 8.10+ versions, we get LoadError: no such file to load -- net/smtp error.
In order to solve this error, mail dependency of this plugin needs to explicitly include net-smtp dependency which happened in mail-v2.8.
However, the big issue, when this plugin upgrades mail to 2.8+ (upgrade PR), plugin's behaviour is completely broken. The test cases show,

• email content type is null eligible
• attachments are automatically decoded

There is no workaround for now. We need more investigation and plan to bring this plugin to working state.

[ceeeekay]

Is there any progress here? One of my Logstash nodes is still pinned on 8.9.2, as the data it collects is only available via an IMAP account.

[yaauie]

Moving forward with confidence with this one is going to be very difficult, primarily because the specs of this plugin use Mail.new(&block) to dynamically compose the mail Message objects that we test against, and newer versions of mail compose different raw messages than older versions. Because this plugin is used to parse messages, we need to ensure it behaves the same when encountering the exact same sequence of bytes.

Therefore, we would need to do the following in order:

1. with an old logstash, capture the on-the-wire form of the messages in specs as fixtures and rework the spec setup to load each message from its fixture without modifying any of the specs themselves.
2. come up with a patch for the implementation that allows the specs-from-fixtures to pass with mail ~> 2.8 without modifying the specs or the fixtures

[ceeeekay]

Is anything at all happening here? It's been well over a year now and this plugin is completely unusable.

I still have a production node pinned on LS 8.9 which is starting to raise some eyebrows in our security team.