[bug report (unsolved)/question] SRL training slows down per epoch (Memory leak?).

[ncble]

Describe the bug
The training of SRL algorithm slows down by a few seconds per epoch. The first epoch cost 31s then, it grows almost linearly to 60s at the end of 30 epochs. Besides, I also took a look the memory usage, at the beginning it uses around 3GB of GPU memory, but it grows to 5GB at the end of first epoch. Is it normal ?

Code example
The following code can reproduce the problem.

• Under robotics-rl-srl/
  $ python -m environments.dataset_generator --env MobileRobotGymEnv-v0 --name mobile2D_fixed_tar_seed_0 --seed 0 --num-cpu 8

• Under srl_zoo/
  $ python train.py --data-folder mobile2D_fixed_tar_seed_0 --losses autoencoder

System Info

• GPU: GTX2080Ti
• Python 3.6.8
• Tensorflow version = 1.13.1

[araffin]

also took a look the memory usage, at the beginning it uses around 3GB of GPU memory, but it grows to 5GB at the end of first epoch. Is it normal ?

looks like a memory leak...