Limit thread amount in PyTorch

[cube1324]

For the evaluation on the server the agent is only supposed to use 4 threads. This is however not activly limited. If the agent spawns more threads there will be a significant performance loss because of the overhead of the threads interleaving.

Torch can be a source for this problem because it will use all available cores by default. In Torch there is Inter-op parallelism and Intra-op parallelism which both have to be limited. The optimal thread distribution between the two depends on the application, one example is:

torch.set_num_threads(1)
torch.set_num_interop_threads(4)

More information about torch threading can be found here.