Possible memory consumption issue

[AlejandroCabeza]

Memory consumption scales with message size being injected, regardless of whether the messages are being sent or not due to node message size limit.
E.g.: AFAIK node can only send messages of size <15KB, if I configure the simulator to send 1000KB messages, even if those messages are not sent, the node memory usage increases.

[alrevuelta]

send messages of size <15KB

This is configurable but should default to 150kb. https://github.com/waku-org/nwaku/blob/master/waku/factory/external_config.nim#L108

if I configure the simulator to send 1000KB messages, even if those messages are not sent, the node memory usage increases.

Can you share more info about this? Is the memory increase significant? Whats the rate of messages that you send? I assume you are injecting the messages via the API? Can you confirm they are being rejected by the API?

[AlejandroCabeza]

send messages of size <15KB

This is configurable but should default to 150kb. https://github.com/waku-org/nwaku/blob/master/waku/factory/external_config.nim#L108

if I configure the simulator to send 1000KB messages, even if those messages are not sent, the node memory usage increases.

Can you share more info about this? Is the memory increase significant? Whats the rate of messages that you send? I assume you are injecting the messages via the API? Can you confirm they are being rejected by the API?

This is the base configuration I'm using:

=================================================================================================
                                       Summary of Parameters
=================================================================================================
- Nwaku Image:                 harbor.status.im/wakuorg/nwaku:v0.30.1
- Nodes
  | Total:                     10
  | Traffic Injection:         10
  | Spam Injection:            0
- Message Publishing Delay:    6s
- Spam Publishing Delay:       10s
- Message Size:                10KB
- Private Key:                 0xac0974bec39a17e36ba4a6b4d238ff944bacb478cbed5efcae784d7bf4f2ff80
- ETH From:                    0xf39fd6e51aad88f6f4ce6ab8827279cfffb92266
- RLN
 | Contract Messages Limit:    300
 | Relay Epoch:                60s
 | RLN Relay Messages Limit:   200
=================================================================================================

I've played with the number of nodes and the message sizes:

+-------+--------------------+-----------------------+
| Nodes | Message Size (KB)  | Memory (MB)/Container |
+-------+--------------------+-----------------------+
| 10    | 10                 | 3.5 - 6.5             |
| 10    | 150                | 50                    |
| 10    | 151                | 50                    |
| 10    | 500                | 30                    |
| 10    | 1000               | 55                    |
| 10    | 2000               | 120                   |
| 10    | 5000               | 210                   |
| 2     | 10                 | 3.5                   |
| 2     | 150                | 10                    |
| 2     | 151                | 25                    |
| 2     | 500                | 25                    |
| 2     | 1000               | 55                    |
| 2     | 2000               | 100                   |
| 2     | 5000               | 210                   |
+-------+--------------------+-----------------------+

And yes, they're being rejected by the API, as there're no valid messages shown in the graphs (when the message size >150KB.

[alrevuelta]

Interesting, so eg 5MB sizes are being rejected by the API but increase the mem consumption of the container? In that case that would be a nwaku issue. Can we reproduce it with a single nwaku node?