Code

WIP Celery Code Impact#97

Code

Draft

DaanZ wants to merge 138 commits intoci-group:celeryci-group/revolve:celeryfrom SamFerwerda:masterSamFerwerda/revolve:masterCopy head branch name to clipboard

+3,076-71

Commits

Commits on Dec 16, 2019

• added celery basics

  SamFerwerda

  committed
  d24bd8d

Commits on Dec 17, 2019

• celery works

  SamFerwerda

  committed
  0fbcf7f

• changed readme file

  SamFerwerda

  committed
  d31d70d

• Added a standard robot simulation to celery

  SamFerwerda

  committed
  54e9152

Commits on Dec 29, 2019

• Added functionality for robot queues and insertion

  SamFerwerda

  committed
  5a62a7f

Commits on Jan 3, 2020

• CeleryController added

  SamFerwerda

  committed
  66d3bec

Commits on Jan 27, 2020

• Added some extra dependencies, that are needed for celery + cpp

  SamFerwerda

  committed
  aed0ac6

Commits on Jan 29, 2020

• added a simulator queue for celery and changed the manager

  SamFerwerda

  committed
  4e6b037

Commits on Jan 30, 2020

• Evaluate_robot (task) now handles multiple robots at a time, by waiting on running one to finish.

  SamFerwerda

  committed
  ca8dc87

Commits on Feb 1, 2020

• finished evaluate_robot and adjusted population class to fit celery

  SamFerwerda

  committed
  8f3f8f0

• Backup before major changes in tasks.py file

  SamFerwerda

  committed
  e881ebd

Commits on Feb 2, 2020

• removed evaluate_robot and made simulator_worker, added a put_in_queue task which puts a robot in queue instead of busy waiting.

  SamFerwerda

  committed
  ad3d620

• Comments added to all functions.

  SamFerwerda

  committed
  80e550a

Commits on Feb 3, 2020

• limited celery workers to one task at the time, added evaluate_robot instead of queue and simulator_worker. celery worker will get a task directly from the celery queue now.

  SamFerwerda

  committed
  e223718

Commits on Feb 7, 2020

• Every celery worker will now also analyze the robot, and added speed records.

  SamFerwerda

  committed
  c23601c

Commits on Feb 11, 2020

• Implemented the basics of message consuming in C++.

  SamFerwerda

  committed
  0d9968a

Commits on Feb 16, 2020

• Simple celery c++ task is set up

  SamFerwerda

  committed
  54bfa8c

• globalized celery channel and consumer tag

  SamFerwerda

  committed
  e8388f5

Commits on Feb 18, 2020

• Bug was caused by printing the sdf string, also made some variables global instead of redefined every update iteration.

  SamFerwerda

  committed
  a8a7b6a

Commits on Feb 20, 2020

• c++ part is now handling actual robots, however just returning 1, None for now. Should be behavorial movement.

  SamFerwerda

  committed
  37915a1

Commits on Feb 22, 2020

• Celery is now fully implemented on python and cpp side. However, currently normal simulations without celery cant be run due to changes in worldcontroller.cpp. Will fix this soon.

  SamFerwerda

  committed
  99cfdc0

• Some comments and error handling.

  SamFerwerda

  committed
  399e9ad

Commits on Mar 4, 2020

• Starting implementing contacts as result

  SamFerwerda

  committed
  cf71c19

Commits on Mar 5, 2020

• Basic locking system added and standard value of 10 added to every contact call

  SamFerwerda

  committed
  5a1498b

• real contacts are send through celery know. However, contacts are calculated in the cpp part now, not the robotmanager part. For details go to worldcontroller.cpp function OnContacts().

  SamFerwerda

  committed
  dc035f4

• celery messages are converting to robot_manager with the function update_from_celery(), this means everything is implemented now.

  SamFerwerda

  committed
  b82f052

Commits on Mar 13, 2020

• Adding a function to restart analyzer simulator. Currently in test phase.

  SamFerwerda

  committed
  15a6da3

Commits on Mar 17, 2020

• Added a timeout on the get request of a celery message. Such that if a robot gets lost, a result of None is read instead of a crash.

  SamFerwerda

  committed
  f1b1f15

Commits on Mar 20, 2020

• Added a way to reset celery and gazebo if a get request fails or 10 generations passed by. Still studying the effect of restarting.

  SamFerwerda

  committed
  029a4ec

• Changed the readme description

  SamFerwerda

  committed
  1ed7ec1

• Added changed files and problems with celery

  SamFerwerda

  committed
  0aa4489

Commits on Mar 23, 2020

• Old revolve is now accessible again with the normal world controller. To use celery, an additional command needs to be place (--world worlds/celeryplane.world)

  SamFerwerda

  committed
  876ed5d

Commits on Mar 27, 2020

• Update manager_pop.py

  SamFerwerda

  authored
  5ef39d7

Commits on Apr 21, 2020

• Make changes required for running on ripper machines

  DaanZ

  committed
  85f67d8

• celery_pool_asyncio no longer needs patch

  SamFerwerda

  authored
  7850549

• Update manager.py

  SamFerwerda

  authored
  d2d6c23

• Update celery.py

  SamFerwerda

  authored
  606de95

• Adding logging so we can find some errors or bugs

  SamFerwerda

  committed
  10c65fc

Commits on Apr 23, 2020

• Commented pkill because not sure about effects on other ripper users.

  SamFerwerda

  committed
  4ecff5c

• This file isnt needed.

  SamFerwerda

  authored
  a69db25

• Old comments and imports deleted.

  SamFerwerda

  authored
  2994824

Commits on Apr 24, 2020

• Update readme.txt

  SamFerwerda

  authored
  48e0ecc

• Rename readme.txt to README.md

  SamFerwerda

  authored
  e77dd5e

• Updated the readme file for better reader experience.

  SamFerwerda

  authored
  5a84078

• Update README.md

  SamFerwerda

  authored
  6b06ff9

• Update README.md

  SamFerwerda

  authored
  3d6e87f

• Update README.md

  SamFerwerda

  authored
  33065d4

• Update README.md

  SamFerwerda

  authored
  84e9c2b

• Update README.md

  SamFerwerda

  authored
  f40267d

• Update README.md

  SamFerwerda

  authored
  3a69a95

• Update README.md

  SamFerwerda

  authored
  7936037

• Update README.md

  SamFerwerda

  authored
  116ca43

• Update README.md

  SamFerwerda

  authored
  55b5fad

• Update celery.py

  SamFerwerda

  authored
  a85758a

• Update CMakeLists.txt

  SamFerwerda

  authored
  9ebd6d6

• Got fatal error that there is no such file json/json.h

  SamFerwerda

  authored
  a903b1b

• Last commit didnt fix issue

  SamFerwerda

  authored
  81de1d2

Commits on Apr 28, 2020

• changed cmake file to use pkgconfig for simpleamqp, fixed jsoncpp

  SamFerwerda

  committed
  923dae3

Commits on Apr 29, 2020

• Currently failing due to not finding libSimpleAmqpClient.so.2

  SamFerwerda

  committed
  cf4945d

• Added celery and celery pool asyncio to the python requirements.

  DaanZ

  committed
  7352b4f

Commits on May 1, 2020

• Changed Cmake to add library to build/lib folder.

  SamFerwerda

  committed
  bd4205c

• Merge branch 'ripper' of https://github.com/SamFerwerda/revolve into ripper

  SamFerwerda

  committed
  0a1f6cd

Commits on May 4, 2020

• Works on local machine, not yet on Ripper

  SamFerwerda

  committed
  dc7d807

• Changed pkg to find_library, works locally, testing on ripper

  SamFerwerda

  committed
  e925771

Commits on May 5, 2020

• Celery runs with logs

  DaanZ

  committed
  82cf6be

Commits on May 6, 2020

• Added speedtesting requirements.

  SamFerwerda

  committed
  5bf11f2

Commits on May 8, 2020

• added speedup checks to analyze speedup

  SamFerwerda

  committed
  2d200ab

Commits on May 11, 2020

• Added timing points to find bottleneck of revolve

  SamFerwerda

  committed
  572b886

• Fixed wrong position of lists

  SamFerwerda

  committed
  e6a49a3

• reversed location of list again

  SamFerwerda

  committed
  2c8a842

• fixes

  SamFerwerda

  committed
  f9b0f72

• speed data

  SamFerwerda

  committed
  5f08887

• speed data

  SamFerwerda

  committed
  bfca4c7

• changed speeddata

  SamFerwerda

  committed
  4e25224

• added data

  SamFerwerda

  committed
  1a7c630

• added data

  SamFerwerda

  committed
  299dade

Commits on May 12, 2020

• changes in manager_pop

  SamFerwerda

  committed
  a5cd902

• Update manager for writing data.

  SamFerwerda

  authored
  0a83f4a

• Commit the revolve log for data

  SamFerwerda

  committed
  56c47cb

Commits on May 14, 2020

• Speed data

  SamFerwerda

  committed
  09214b0

• Changed location of time measure.

  SamFerwerda

  authored
  30c82d2

Commits on May 15, 2020

• data

  SamFerwerda

  committed
  31947ba

Commits on May 18, 2020

• Data for 100 gen

  SamFerwerda

  committed
  2031c0e

• Added extra data writing points.

  SamFerwerda

  authored
  a832b50

Commits on May 21, 2020

• some celery data

  SamFerwerda

  committed
  16cef67

Commits on May 22, 2020

• export is now done parallel to evaluating

  SamFerwerda

  committed
  c041889

• Merge branch 'speedup' of https://github.com/SamFerwerda/revolve into speedup

  SamFerwerda

  committed
  2c4723e

Commits on May 25, 2020

• possible fix to nonetype error

  SamFerwerda

  committed
  dc68210

Commits on May 26, 2020

• speed data

  SamFerwerda

  committed
  27f6539

Commits on May 27, 2020

• Implemented speed checks to analyze revolve

  SamFerwerda

  committed
  4b5b2e9

Commits on May 28, 2020

• speeddata

  SamFerwerda

  committed
  1273bb1

• added start_experiment.py for quick parallel experiments

  SamFerwerda

  committed
  c829e4e

• Changed layout of speedlogging

  SamFerwerda

  committed
  93d0e3a

Commits on May 29, 2020

• Speed data of 15 normal runs

  SamFerwerda

  committed
  4d2aa47

• deleted revolve.log

  SamFerwerda

  committed
  1b29d56

• changes in start_experiment and manager

  SamFerwerda

  committed
  845ac27

• Merge branch 'speedup' of https://github.com/SamFerwerda/revolve into speedup

  SamFerwerda

  committed
  44e40ba

• Deleted the timeout of 120 seconds

  SamFerwerda

  committed
  bf4e329

• small changes in manager file

  SamFerwerda

  committed
  fc82bb5

Commits on May 30, 2020

• speed data of all 30 old revolve

  SamFerwerda

  committed
  c92c4bb

• delete data

  SamFerwerda

  committed
  a6166d8

Commits on May 31, 2020

• Queues are now unique for different experiments

  SamFerwerda

  committed
  83aa70e

• fixed wrong implemententation in population.py

  SamFerwerda

  committed
  151caa2

Commits on Jun 1, 2020

• Test for multi experiments with celery

  SamFerwerda

  committed
  db57b35

• Backup

  SamFerwerda

  committed
  0ab9832

• Unique queues for different experiments

  SamFerwerda

  committed
  3d86b58

• Merge branch 'master' into multiCelery

  SamFerwerda

  committed
  2b0450c

• added script for consecutive runs

  SamFerwerda

  committed
  82e514e

Commits on Jun 3, 2020

• extra data 32 cores

  SamFerwerda

  committed
  ac94e40

• removed worker pids

  SamFerwerda

  committed
  1f5063f

• changes for consecutive runs

  SamFerwerda

  committed
  e46fd3c

• data for 32 cores

  SamFerwerda

  committed
  b4e3364

Commits on Jun 8, 2020

• Unnecesarry file

  SamFerwerda

  authored
  830f63e

• Data for 16 cores

  SamFerwerda

  committed
  2384e1e

Commits on Jun 10, 2020

• data for 16 cores

  SamFerwerda

  committed
  0a806b1

• Merge branch 'master' of https://github.com/SamFerwerda/revolve

  SamFerwerda

  committed
  eb6bd87

• all data for 16 cores

  SamFerwerda

  committed
  e9a1cd5

• speeddata upload and script for parallel experiments

  SamFerwerda

  committed
  ab8d232

• Revert "data for 16 cores"

  Show description for 764a26d
  SamFerwerda

  committed
  764a26d

Commits on Jun 12, 2020

• added world to start_experiments

  SamFerwerda

  committed
  dc5a03f

• Added rotation

  SamFerwerda

  committed
  0c32d0a

• deleted print statement

  SamFerwerda

  committed
  d4f774d

Commits on Jun 13, 2020

• changed plugin

  SamFerwerda

  committed
  1d1fe80

• data

  SamFerwerda

  committed
  8533057

• merge

  SamFerwerda

  committed
  d30684e

• fixed abnormal tabs.

  SamFerwerda

  authored
  40f3c54

• Deleted duplication

  SamFerwerda

  authored
  55e6507

Commits on Jun 15, 2020

• data

  SamFerwerda

  committed
  ad6238b

• Testing possible problems

  SamFerwerda

  authored
  e98081b

• merge

  SamFerwerda

  committed
  5736985

• merge

  SamFerwerda

  committed
  df62dfe

• Increased waiting time

  SamFerwerda

  committed
  5d139c6

• Adding other way to parallelize experiments

  SamFerwerda

  committed
  5909bf3

Commits on Jun 16, 2020

• speed data

  SamFerwerda

  committed
  d4dec49

• cleaned data file

  SamFerwerda

  authored
  a4de28b

Commits on Jun 18, 2020

• Data of 16 and 8 cores

  SamFerwerda

  committed
  1018892

Commits on Jun 19, 2020

• Data 4 cores

  SamFerwerda

  committed
  f8451fb

Commits on Jun 22, 2020

• Merge pull request #1 from ci-group/master

  Show description for 750a2e3
  SamFerwerda

  authored
  750a2e3