Wrapping an AsyncRead + AsyncWrite stream

[link2xt]

I have a AsyncRead + AsyncWrite network stream that I want to compress, like a TcpStream or TlsStream.

Is there a way to wrap such stream to get a single CompressedStream with async-compress?

If I use tokio::io::split and then wrap each part in e.g. DeflateEncoder and DeflateDecoder separately, I get two separate objects and not a single stream anymore. Besides, split adds overhead by placing the inner stream behind a mutex. In async-email/async-imap#112 I manually placed the stream behind a mutex to pass it into DeflateEncoder and DeflateDecoder, so at least I have an access to inner stream, but now I cannot have a get_ref() and get_mut() API on the resulting stream because it is behind a mutex internally.

What I want is some API like a compressed stream builder which gets encoder, decoder and underlying stream and constructs a single object that allows direct access to underlying stream via get_ref() and get_mut(). Is it impossible with async-compression?

[link2xt]

The other option to fix this without changing API is to do like BufWriter and have pass-through trait implementations like impl<W: AsyncWrite + AsyncBufRead> AsyncBufRead for Decoder<W>.

Then I will be able to just wrap the whole stream into DeflateEncoder<DeflateDecoder<...>> and still use it as an AsyncRead + AsyncWrite stream.

[link2xt]

I am going to pass through AsyncRead and AsyncWrite where I can in #297

[NobodyXu]

You can use TcpStream::into_split, which is reversible, and does not involve any Mutex or RwLock.

It does use Arc but I think it's acceptable.

[link2xt]

This works if you have plain TcpStream, but in deltachat-core-rust I have TlsStream<TcpStream>> or TlsStream<Socks5Stream<TcpStream>> or some other encrypted and proxified streams that don't have into_split().

[NobodyXu]

cc @link2xt how about TcpStream::into_split?

Can it solve your problem efficiently?

[link2xt]

In async-imap I even have just AsyncRead + AsyncWrite stream without any assumptions about the underlying stream, it should be able to run IMAP over whatever stream library user provides and also compress it with deflate if supported.

[NobodyXu]

This works if you have plain TcpStream, but in deltachat-core-rust I have TlsStream<TcpStream>> or TlsStream<Socks5Stream<TcpStream>> or some other encrypted and proxified streams that don't have into_split().

Can you call TcpStream::into_split before creating TlsStream?

Or, can you pass a reference to the *Decoder::new method?

[NobodyXu]

In async-imap I even have just AsyncRead + AsyncWrite stream without any assumptions about the underlying stream,

Hmmm I was mostly worried about having mismatched functionality, since bufread type should be AsyncBufRead.

But then I realized that if we consider async-compression to be a decorator that adds a layer to decompress/compress, it does make sense

[link2xt]

https://docs.rs/tokio/latest/tokio/io/struct.BufReader.html passes AsyncWrite through and https://docs.rs/tokio/latest/tokio/io/struct.BufWriter.html passes AsyncRead, AsyncBufRead etc. through, so it seems to be already common.

Same for futures::io::BufReader, it passes AsyncWrite through.

[NobodyXu]

Yeah it does seem to be a reasonable pattern.

The buf reader/tls/compress/decompress can be seen as an decorator, so they should pass through everything trait that isn't decorated.