How to use Multi for streaming a zip file?

[softshipper]

Hi all,
I am trying to stream a big zip file with Multi through GRPC service as follows:

@GrpcService
class HelloGrpcService : HelloGrpc {
    override fun source(request: Empty?): Multi<SourceResponse> {
        val file = File("/Users/developer/Downloads/Archive.zip")
        val res = SourceResponse.newBuilder().setData(ByteString.readFrom(file.inputStream())).build()
        return Multi.createFrom().item(res)
    }
}

Unfortunately, I have received the following exception:

System.ArgumentException: The JSON value of length 212507417 is too large and not supported.

My goal is to stream the file and not to send it once.
The question is how to stream a big file in GRPC?

Here is the proto file:

syntax = "proto3";

import "google/protobuf/empty.proto";

option java_multiple_files = true;
option java_package = "io.acme";
option java_outer_classname = "HelloGrpcProto";

package hello;


service HelloGrpc {
  rpc Source(google.protobuf.Empty) returns (stream SourceResponse) {}
}

message SourceResponse {
  bytes data = 1;
}

Best regards

[jponge]

This doesn't look like a Mutiny question, and it's not clear what libraries/frameworks you're using.

[softshipper]

@jponge I agree. The problem was not explain well at all.
Let consider the following example:

    Multi.createFrom().emitter<ByteBuffer>(Consumer { em: MultiEmitter<in ByteBuffer> ->
        val stream = File("/Users/developer/Downloads/Archive.zip").inputStream()
        val buffer = ByteBuffer.allocate(100)
        while (stream.read(buffer.array()) != -1) {
            em.emit(buffer)
            buffer.clear()
        }
        em.complete()
    }).subscribe().with({ st -> println(st) }, { err -> println(err.message) })

What the code does is pretty straightforward. It streams or emits 100 bytes chunk of data from the inputstream to the sink until end of file. In the example, the file is pretty big approximately 100 MB.
Every time when I want to stream a big inputstream I have to implement it by my own. The question is, does it exist a method with parameters inputstream and bytebuffer, e.g.

Multi.createFrom().stream(inputstream, bytebuffer)

for reading huge inputstream.

I hope my question is now clear another let me know.

Best regards

[jponge]

No we don't have that, Mutiny is orthogonal to the IO sources.

…

On Wed, Oct 12, 2022 at 9:48 PM softshipper ***@***.***> wrote: @jponge <https://github.com/jponge> I agree. The problem was not explain well at all. Let consider the following example: Multi.createFrom().emitter<ByteBuffer>(Consumer { em: MultiEmitter<in ByteBuffer> -> val stream = File("/Users/developer/Downloads/Archive.zip").inputStream() val buffer = ByteBuffer.allocate(100) while (stream.read(buffer.array()) != -1) { em.emit(buffer) buffer.clear() } em.complete() }).subscribe().with({ st -> println(st) }, { err -> println(err.message) }) What the code does is pretty straightforward. It streams or emits 100 bytes chunk of data from the inputstream to the sink until end of file. In the example, the file is pretty big approximately 100 MB. Every time when I want to stream a big inputstream I have to implement it by my own. The question is, does it exist a method with parameters inputstream and bytebuffer, e.g. Multi.createFrom().stream(inputstream, bytebuffer) for reading huge inputstream. I hope my question is now clear another let me know. Best regards — Reply to this email directly, view it on GitHub <#1070 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAGK2PHUXXRWCXWVUCADV3WC4IWPANCNFSM6AAAAAARAMATBE> . You are receiving this because you were mentioned.Message ID: ***@***.*** com>

[softshipper]

Which means I need to do by my own, right?

[jponge]

Yes

…

On Thu, Oct 13, 2022 at 1:53 PM softshipper ***@***.***> wrote: Which means I need to do by my own, right? — Reply to this email directly, view it on GitHub <#1070 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAGK2MGYWRLBVZYFT3RVQTWC7Z43ANCNFSM6AAAAAARAMATBE> . You are receiving this because you were mentioned.Message ID: ***@***.*** com>