High ram consumption

[t-kutenkova]

Hi. Could you please help to investigate how to lower RAM consumption? Every new mocked hostname greatly increases mock server RAM consumption. Statistics is the following:

6 mocked hostnames: 2.056GiB (which is already quite high)
8 mocked hostnames: 2.404GiB
10 mocked hostnames: 2.955GiB

The whole statistics:
6 hostnames

CONTAINER ID    NAME                                                      CPU %    MEM USAGE / LIMIT     MEM %     NET I/O                  BLOCK I/O              PIDS
b1cdfc449c6a   autotests-mock-server-1   2.16%     1.879GiB / 7.667GiB   24.51%    4.56MB / 138kB   182MB / 272MB   253

8 hostnames
ce33554a0a74 autotests-mock-server-1 3.18% 2.404GiB / 7.667GiB 31.36% 4.54MB / 115kB 0B / 282MB 313

10 hostnames
cd9bb0bd2bfb autotests-mock-server-1 4.26% 2.955GiB / 7.667GiB 38.54% 4.54MB / 128kB 0B / 283MB 374

[t-kutenkova]

I started basic wiremock container - it consumes only ~100MB.

docker pull wiremock/wiremock:latest
git clone https://github.com/wiremock/wiremock-docker.git
docker run -it --rm \
  -p 8080:8080 \
  -v $PWD/wiremock-docker/samples/hello/stubs:/home/wiremock \
  wiremock/wiremock:latest

CONTAINER ID   NAME                              CPU %     MEM USAGE / LIMIT     MEM %     NET I/O          BLOCK I/O    PIDS
1f79477ce171   wiremock                  0.23%     112.1MiB / 7.667GiB   1.43%     1.39kB / 0B      0B / 246kB   34

[wtertius]

Hello! I'll take a couple of weeks for investigation, what component is responsible for such excessive consumption.

[t-kutenkova]

Thanks for the response. Looking forward for the results of the investigation! We are planning to increase the number of the mocked hostnames so currently it looks like a blocker for us :(
Btw do you see such RAM consumption on you env? How many hostnames do you usually have?

[t-kutenkova]

Hello, is there any progress?

[wtertius]

Hi! I've explored RAM consumption on my env - Ubuntu Linux, docker CE - and I've got these results:

5 hostnames (~140 MiB per host):

docker stats grpc-wiremock-container
CONTAINER ID   NAME                CPU %     MEM USAGE / LIMIT   MEM %     NET I/O           BLOCK I/O      PIDS
c49764c44088   grpc-wiremock-container   0.37%     693MiB / 13.53GiB   5.00%     58.5kB / 12.7kB   332MB / 19MB   177

13 hostnames (~113 MiB per host):

docker stats grpc-wiremock-container
CONTAINER ID   NAME                CPU %     MEM USAGE / LIMIT   MEM %     NET I/O           BLOCK I/O      PIDS
fbf3f42dcee3   grpc-wiremock-container   1.12%     1.476GiB / 13.53GiB   10.91%    52.5kB / 13.2kB   16.5MB / 6.68MB   482

It's looks quite close to original wiremock RAM consumption.

Could you please answer

1. What is your env: OS, docker backend, etc
2. How many files do you have in mapping folder for the hostnames
   I've noticed some correlation between the number of mapping files and memory consumption
3. Please attach results of this command for your container:
   docker exec -it grpc-wiremock-container top -c -n 1 -o RES

[t-kutenkova]

How to explain that every new hostname adds 150M of RAM? GRPC Wiremock creates new wiremock instance for every hostname?

Regarding your questions:

1. I have ARM64 M1 "Apple Silicon" Mac MacOS 13.6.7 (22G720), Docker Desktop 4.24.0 (122432). My colleagues have other Mac configurations.
2. The thing is that there is no mappings at all. The container consumes so much memory at the time it starts. I attached the screenshots with /contracts and /home folders content.

[t-kutenkova]

May be there is some way to place all contracts to just one wiremock instance?

[wtertius]

GRPC Wiremock creates new wiremock instance for every hostname?

That's right - GRPC Wiremock creates new wiremock instance for every hostname. It allows us to not change mapping files and have separate GUI for every hostname.

May be there is some way to place all contracts to just one wiremock instance?

Unfortunately it requires the change of GRPC Wiremock architecture. I can see two ways:

1. The most user friendly - fork wiremock itself and develop it to support multiple hostnames
   The downside is that you will have to merge new wiremock code into the fork from time to time. The wiremock company sells multiple hostnames as the adventure of their cloud solution, so I don't think they will accept such fork changes.
2. Another option - use single wiremock instance inside grpc-wiremock, add hostname rule to every mapping file and distinct hostname by the Host header.
   The downsides are:
   • Changing of user mapping files directly OR complicated sync mechanism for keeping synchronised user mapping files with automatically patched ones
   • Single GUI for all hostnames

If you have resources to implement it, I can help with architecture decisions.

[wtertius]

Also consider to use orbstack - I've heard it drammatically speeds up docker containers on MacOS.