docs: document how to integrate with Ryuk

[mdelapenya]

What does this PR do?

It describes the integration points to be implemented by Testcontainers libraries in order to communicate with Ryuk.

Why is it important?

Better documentation is always good

[kiview]

Suggested change

	- Identify test session semantics for the Testcontainers library. For example, a test session could be a single test method, a test class, or a test suite. As reference, please consider taking a look at Go's implementation [here](https://golang.testcontainers.org/features/test_session_semantics/). This unique identifier for the test session semantic, is referenced as `SESSION_ID` from now on.
	- Identify test session semantics for the Testcontainers library. For example, a test session could be a single test method, a test class, or a test suite. As reference, please consider taking a look at Go's implementation [here](https://golang.testcontainers.org/features/test_session_semantics/). This unique identifier for the test session semantic, is referenced as `SESSION_ID` from now on.
	As an implementation hint, consider how an atomic user interaction with the intent of running tests should generally lead to one single session (i.e. run tests from within IDE).

[kiview]

Suggested change

	- Bind the Docker socket to the Ryuk container, so that it can communicate with the Docker daemon. This is necessary to be able to create and remove resources.
	- Bind the Docker socket to the Ryuk container, so that it can communicate with the Docker daemon. This is necessary to be able to create and remove resources. Be aware that this needs to be the Docker socket accessible to the container, which might be a different one from Docker socket accessible by host processes.

[kiview]

Suggested change

	- Use a TCP connection to send Ryuk the message. The connection must be established to the address of the Ryuk container and the port specified in the `RYUK_PORT` environment variable.
	- Use a TCP connection to send Ryuk the message. The connection must be established to the address of the Ryuk container and the (mapped) port specified in the `RYUK_PORT` environment variable.

[kiview]

Suggested change

	- Whenever a resource is removed by the Testcontainers library, send a termination signal to Ryuk using a TCP connection in the same way as seen above; this way Ryuk can identify the test session is about to finish and start the cleanup process. Ryuk uses `RYUK_CONNECTION_TIMEOUT` and `RYUK_RECONNECTION_TIMEOUT` to determine when to start the cleanup process.
	- Whenever all resources are removed by the Testcontainers library, send a termination signal to Ryuk using a TCP connection in the same way as seen above; this way Ryuk can identify the test session is about to finish and start the cleanup process. Ryuk uses `RYUK_CONNECTION_TIMEOUT` and `RYUK_RECONNECTION_TIMEOUT` to determine when to start the cleanup process.

Correct?
Also @eddumelendez , do we still do in tc-java, or did we had to revert it because of the Gradle daemon issue?

[eddumelendez]

not anymore

[HofmeisterAn]

question: Are you really doing this for every resource? .NET sends it only once at the start and then assigns the same value and label the resources.

[mdelapenya]

I need to double check, but at first sight yeah, once a container is created, it notifies Ryuk. I think it could be a legacy situation, where each container had its own reaper, which is not possible anymore 🤔

@stevenh we should take a look at this, as we could possibly optimise the communication with Ryuk

[HofmeisterAn]

In .NET, we follow these steps:

1. Create and start Ryuk.
2. Connect to Ryuk.
3. Send the filter.
4. Maintain the connection.

Then we use the ID from the filter and label every resource we create with it.

[stevenh]

Technically you only need to send a new filter if its different however it's connections being removed which trigger clean up, so unless you connect for each resource it could be slower to clean up and might even remove in use resources if the last connection was removed while another resource is still in use which matches the filter.

Does that make sense?

[HofmeisterAn]

Sorry, I do not fully understand. .NET establishes only one connection to Ryuk. Each test process has its own instance. Each test process creates Ryuk, connects, sends the filter, and maintains the connection only once for all running tests (within the process).

[stevenh]

Yep I understood you meant Testcontainers .NET. What I'm trying to understand is what in testcontainers .NET is responsible for setting up ryuk and connecting to it. You seemed to infer it was something outside of just creating a container with testcontainers .NET, is that the case?

How testcontainers-go works is every resource created checks to ensure ryuk is running, if not it creates it and in either way connects to it, so ryuk knows there is an additional dependency.

ryuk monitors these connections and when the last one disconnects, it runs the clean up. This means that the clean up should be quick as its a triggered event but also it means that if there is an issue along the way and something triggers an unexpected shutdown the failing connections would still trigger a clean up.

For the go implementation this is import as the test infrastructure has a global timeout and if that triggers it doesn't gracefully clean up test resources.

In the wider use, a test container could be used for a single test in a suite, so having each container do this validation helps to ensure ryuk is run and orphaned resources only run while needed.

[HofmeisterAn]

You seemed to infer it was something outside of just creating a container with testcontainers .NET, is that the case?

No, we start Ryuk with the first container resource.

How testcontainers-go works is every resource created checks to ensure ryuk is running, if not it creates it and in either way connects to it, so ryuk knows there is an additional dependency.

This part is slightly different in .NET. Every resource we create checks if Ryuk is running. If it is not, we create it; if it is, we skip the creation. We do not establish another connection to Ryuk. The connection is created with the first resource that creates Ryuk (the resource does not hold the connection; it is held by the test process).

[stevenh]

So to confirm, if there was a test that ran for a period of time after all docker resources were unneeded, these would be keep available as its not until the process exits that the connection is removed?

Example in sudo code

func testWithContainer() {
    container = tc.Run("myimage"....)
    // Do things with container....
}

func main() {
      testWithoutContainer()
      testWithContainer()
      // Resources not cleaned up yet...
      moreTestsWithoutContainer() 
      // Connection to ryuk still open, container not cleaned up until main exits?
}

[HofmeisterAn]

Connection to ryuk still open, container not cleaned up until main exits?

In theory, that is true, but .NET provides a concept for releasing unmanaged resources: Dispose. Typically, when a test completes, the Dispose method cleans up the container if it is no longer needed. Ryuk serves as a fallback, guaranteeing cleanup in cases when the test process crashes. Thank you for the pseudocode example; I now understand your perspective ✅.

[stevenh]

Yep that's the same with testcontainers-go, its a fall back for tests written correctly, so having an extra delay is not a big deal.

However documenting that you can use a connection per resource to ensure timely clean up is still of benefit IMO, thoughts?

[eddumelendez]

use of env var should be optional and it doesn't matter when talking about container because the exposed port is random from Testcontainers POV

[stevenh]

Yes the configuring any RYUK_ prefixed environment variables is optional, including RYUK_PORT, if however the user specifies it, it should be honoured, so we ensure connections are made to that port.

[eddumelendez]

not anymore