ANSI codes in replies to HTML mails

[mjg]

Describe the bug
Replying to HTML messages produces ANSI codes

Software Versions

• Python version: 3.7.7
• Notmuch version: 0.29
• Alot version: 530addc

To Reproduce
Steps to reproduce the behaviour:

1. Use the suggested text/html-entry in mailcap (elinks) to produce ANSI codes
2. reply to HTML message
3. observe the quoted text:

XYZ venit, vidit, dixit 2020-06-21 14:26:49:
> ^[[38;5;0m^[[48;5;0m^[[38;5;15m   ^[[38;5;250mLieber Herr Gruber,

I might be missing some additional config to convert ANSIfied HTML back to txt or so, but ANSIfied bofy text certainly is not intended to be quoted in replies as is.

[pazz]

Ah yes, that makes sense. What you want in this case are two different html->text converters. One for displaying in alot's UI and one for creating the reply template. I'm not sure how to implement this nicely (in terms of user configs).. Suggestions welcome!

[johslarsen]

I have the same problem. My far from perfect, but still usable workaround is to strip away ANSI escape codes in the hook for quoting text. This can be achieved with a hook like:

ANSI_ESCAPE = re.compile(r'\x1B(?:[@-Z\\-_]|\[[0-?]*[ -/ ]*[@-~])')
def text_quote(message):
  ANSI_ESCAPE.sub('', message)

[mjg]

Thanks for the hint.

My workaround is to avoid the feature alltogether since interpret_ansi_background = false does not catch those cases where some combination of stupid styles and elinks mode produces blinking text (for the whole e-mail body), which is completely nuts. Yes, I know this is foreground, but for me there are too many issues (startup performance, blinking foreground, escapes in replies) with ANSItext and too little time to hunt them down (in config or code).

I'm just wondering if there's anyone not having a problem with replies to HTML e-mails when using ANSI conversion - this appears to be by design, not by coincidence in specific configurations.

[pazz]

I agree that the conversion from html to plaintext and back is error-prone, which is why I also don't use this feature. Unfortuntely we cannot assume that all html emails have sensible plaintext alternatives to fall back on. I am also quite annoyed by the inlined links in text created by elinks. Perhaps one can use pandoc do interpret html into markdown (and sanitize this somewhat to remove tables etc?). Quoting Michael J Gruber (2020-07-08 09:00:31)

…

Thanks for the hint. My workaround is to avoid the feature alltogether since interpret_ansi_background = false does not catch those cases where some combination of stupid styles and elinks mode produces blinking text (for the whole e-mail body), which is completely nuts. Yes, I know this is foreground, but for me there are too many issues (startup performance, blinking foreground, escapes in replies) with ANSItext and too little time to hunt them down (in config or code). I'm just wondering if there's anyone not having a problem with replies to HTML e-mails when using ANSI conversion - this appears to be by design, not by coincidence in specific configurations. — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.*

[aplund]

Shouldn't the mailcap 'compose' field be used to specify how to format the text correctly?

[pazz]

@aplund I've never heard of that one, but yes, it looks like we should be respecting that. PRs welcome :)