Integrating a high fidelity ship maneuvering simulator with ROS2 : a path-following case study

Undergraduate Thesis for the obtention of the Mechatronics Engineering degree from the University of São Paulo (USP).

This project develops the pydyna_simple package that encapsulates pydyna's maneuvering simulator in a ROS2 node and enables taking advantage of it in the ROS2 environment. There, it can be combined with multiple other packages natively and in modular fashion. In this structure, the project can also scale very well to a complex robotic system when more modules are added. The package proposition is done in the context of a proof-of-concept case study, where the aim is to provide enough demonstration of the capabilities that come along with the ROS2 environment. It is a path-following scenario, similar to the ones in fast-time simulations done by TPN. The case study is encapsulated in the path_following package. Therefore, this repo holds two packages. An overview of these two packages is given below.

pydyna_simple package: The pydyna_simple package has the objective of integrating pydyna to ROS2. This package contains the pydyna_simple node, which runs the simulation. The package also includes a launch file that should be used to launch the node, and consequently, the simulation. A config folder contains the necessary archives to support pydyna's functioning and installation (the main files are the .p3d files of the vessels, and the .whl file of the pydyna library).

path_following package: The path_following package implements a ship that follows a path defined by waypoints. Each waypoint is a tuple (x, y, u) where x and y define the location in the 2D plane and u is the desired velocity when passing through the aforementioned location. It is a complement to pydyna_simple package as it integrates a GNC (Guidance-Navigation-Control) architecture to pydyna's dynamical system. This package not only complements pydyna as a way of implementing it into the architecture of a path following vessel, but also includes a GUI to visualize the vessel. Given the starting state and the desired waypoints, the vessel will then follow a path given by straight lines connecting the waypoints positions, following also the surge velocity given by the waypoints velocities.

Example of the ship following a zigzag path (gif with speed increase).

ROS2 terminology

• Node: independent processes that are able to send and receive data from each other;
• topic: one of the means in which nodes can send (by publishing to a topic) and receive (subscribing to a topic) data;
• service: another mean of communication between nodes with a request and response pattern. Can be asynchronous or synchronous. In our application only asynchronous services are used;
• msg and srv files: msg files define the data structure for topics and srv files for services. There are standard libraries with basic data types and structures available for use, but custom implementations are also needed in the application;
• rosbag: tool that subscribes to topics and writes a bag file with the contents of all messages published on those specified topics. A bag file is essentially a sqlite database.

Features

pydyna_simple package

rqt graph of the pydyna_simple package. Topics are rectangles and nodes are elipses.

With this package, the user is able to start the simulation with a request using a service containing the initial state of the vessel, propeller rotation and yaw angle. After the simulation is initialized, the user can give two inputs to pydyna: propeller rotation and the rudder angle. The pydyna node subscribes to these two inputs as topics, runs one step of the simulation only when it has received both of these inputs, and publishes the next state of the vessel to the state topic. Ending the simulation is also an option and can be done with one of two topics: end or shutdown. The last relates to the path_following package that will presented afterwards.

To start a simulation, a request has to be sent using the service InitValues.srv, shown in Source Code 1. This service is a custom srv file. This file contains a request and response in yaml format. Request and response are separated by a dashed line. The request contains four properties: initial_state, waypoints, surge and yaw. The initial_state property contains the initial state of the vessel. The waypoints property is not used in this case. The surge and yaw properties are the inital values for propeller rotation and rudder angle, respectively. The values to the right are the default values for the properties.

The three, mentioned earlier, topics are: propeller_rotation, rudder_angle and state; and are defined by their msg files. Propeller rotation and rudder angle use the standard library Float32 msg file. The state, seen in Source Code 2, uses a custom yaml-styled msg file that contains time, position and velocity properties, which are msg files by themselves. Position is a set of three Float32 properties: x, y and theta, seen in Source Code 3. Velocity, in the same way, is defined by having u, v and r, seen in Source Code 4. The values to the right are the default values for the properties.

All custom msg and srv files are defined and in a separate package for flexibility. This package is called path_following_interfaces. After building the aforementioned package, these data structures can be imported in nodes as Python objects.

Source Code 1 - InitValues.srv

#request
State initial_state
Waypoints waypoints
float32 surge 0.0
float32 yaw 0.0
---
#response 
float32 surge 0.0
float32 yaw 0.0

Source Code 2 - State.msg

# 3DOF state of the craft
Position position 
Velocity velocity
float32 time 0.0

Source Code 3 - Position.msg

# positions (earth-fixed reference frame)
float32 x 0.0
float32 y 0.0
# 1.57079632679 radians = 90 degrees
float32 theta 1.57079632679 

Source Code 4 - Velocity.msg

# velocities (craft-fixed reference frame)
float32 u 1.0
float32 v 0.0
float32 r 0.0

path_following package

GNC architecture of the path_following package.

With the nodes active, it’s possible to visualize the vessel in http://localhost:6150. Using the packages’ HTTP API, then send requests to start the simulation, the initial state of the vessel, and the desired waypoints to http:localhost:5000.

In order to run the simulation with desired parameters, the user must send two POST and one GET request. Send the initial state to /initial_condition, illustrated in Figure X, and desired waypoints to /waypoints, illustrated in Figure X. In /waypoints, the “from_gui” property, for the time being, is always zero. This is because the waypoints are given solely through the HTTP Client, however, in future work the value 1 will say to the backend that it should ignore the waypoints in the payload, and instead get them from the GUI’s application server. Then, the user sends a GET request to /start, illustrated in Figure X.

The user can also end the simulation in two ways: killing only pydyna node or killing all nodes except for backend node. The first one is achieved with a GET request to /end, illustrated in Figure X; while the second option is achieved with a GET request to /shutdown, illustrated in Figure X.

POST request to /initial_condition

POST request to /waypoints

GET request to /start

GET request to /end

GET request to /shutdown

Requirements

• Windows 10;
• Python 3.6.8;
• Python public libraries specified in project_requirements.txt;
• pydyna (pydyna is a private Python library made by TPN);
• venus (venus is a private Python library made by TPN);
• ROS2 galactic;
• Chocolatey;
• vcredist2013;
• vcredist140;
• OpenSSL;
• Visual Studio 2019;
• DDS implementation*¹;
• OpenCV;
• CMake;
• Qt5;
• Graphviz;
• xmllint;

Folder structure

• / → root directory:
  • /helpers → contains complementary files:
    • /helpers/payloads → contains json payloads to be used with the project's HTTP API:
      • comments.md → comments on the payloads;
      • initialCondition.json → initial condition json payload;
      • waypointsHexagon.json → waypoints, that form an hexagon, json payload;
      • waypointsLinear.json → waypoints, that form an straight line, json payload;
      • waypointsSantos.json → waypoints, to be used at the port of Santos, json payload;
      • waypointsZigZag.json → waypoints, that form a zigzag, json payload;
    • client.bat → sends initial condition, waypoints and runs the path following simulation through curl HTTP requests, using payloads in /helpers/payloads;
    • configpublic.bat → configuration file, required to use the scripts in this folder;
    • overlay.bat → batch script that sources the workspace;
    • path.bat → batch script that runs the path_following package;
    • pydyna.bat → batch script that runs the pydyna_simple package;
    • vsbuildall.bat → batch script that builds all packages;
  • /notes → contains useful notes on pydyna or ROS2:
    • /notes/pydyna_docs → pydyna documentation;
    • /notes/ros2_notes → our notes on some important ROS2 commands;
  • /readme_resources → contains static files used in README.md;
  • /reports → contains documents that explain the project, which were developed for the obtention of our Mechatronics Engineering degree:
    • monograph.pdf → monograph in ABNT like format (contains some diffferences);
    • paper.pdf → paper in IEEE format;
    • slides.pdf → slides developed in LaTex;
  • /src → contains project code:
    • /src/experiments → experiments made to develop some modules of the system:
      • /src/experiments/notch-filter → notch filter experimentation and development;
      • /src/experiments/surge-control → surge controller experimentation and development;
      • /src/experiments/yaw-cotrol → yaw controller experimentation and development;
    • /src/main_ws → ROS2 worskpace:
      • /src/main_ws/install → overlay ROS2 installation;
        • /src/main_ws/install/lib → contains installed ROS2 packages;
        • /src/main_ws/install/share → contains files used by, or generated by, installed packages. Plots, logs and data (of the ROS2 packages) go here;
      • /src/main_ws/src → source code of the ROS2 packages;
        • /src/main_ws/src/path_following → path_following package : ship that follows a path defined by waypoints;
        • /src/main_ws/src/path_following_interfaces → path_following_interfaces package: provides msg and srv files to the other packages;
        • /src/main_ws/src/pydyna_simple → pydyna_simple package : ship dynamical system. Subscribes to rudder angle and propeller rotation, and publishes the state of the ship;
  • project_requirements.txt → requirements file for the project.

Getting started

pydyna_simple package

To install ROS2 and pydyna, follow the steps in the ROS2 Galactic docs and TPN page (need VPN access) respectively. Then, clone this repository.

1. To build packages, run the following command in src/main_ws/src with the x64 Prompt for Visual Studio 2019 terminal as admin:

~/tcc-autonomous-ship/src/main_ws/src> colcon build --merge -install

2. On the newly created install directory, source the workspace:

~/tcc-autonomous-ship/src/main_ws/install> call setup.bat

3. Next, run the launch file. This will create rosbags in the same directory. Therefore, the recommendation is to run it in src/main_ws/install/share/pydyna_simple/db, which is intended to store rosbags.

~/tcc-autonomous-ship/src/main_ws/install/share/pydyna_simple/db> ros2 launch pydyna_simple pydyna_simple.launch.py

4. In order to verify topics, the user may run:

~> ros2 topic list -t

5. To get a graph of topics and nodes, the user may run:

~> rqt_graph

6. With the pydyna_simple node active, the user can, from the command line, start the simulation with the init_simul service, send topics for rudder_angle and propeller_rotation and listen to state; each one of them in a separate terminal. In the example below, "{}" means taht the default values for InitValues will be used.

   a. start simulation by making a request to init_simul:

   ~> ros2 service call /init_simul path_following_interfaces/srv/InitValues "{}"

   b. publish 0 (example) to rudder_angle:

   ~> ros2 topic pub --once /rudder_angle std_msgs/msg/Float32 "{data: 0}"

   c. publish 1 (example) to propeller_rotation:

   ~> ros2 topic pub --once /propeller_rotation std_msgs/msg/Float32 "{data: 1}"

   d. subscribe to state:

   ~> ros2 topic echo /state

path_following package

To install ROS2 and pydyna, follow the steps in the ROS2 Galactic docs and TPN page (need VPN access) respectively. Then, the user must clone this repository. The first two steps are identical to the ones in the previous section.

1. To build packages, run the following command in src/main_ws/src with the x64 Prompt for Visual Studio 2019 terminal as admin:

~/tcc-autonomous-ship/src/main_ws/src> colcon build --merge- install

2. On the newly created install directory, source the workspace:

~/tcc-autonomous-ship/src/main_ws/install> call setup.bat

3. Next, run the launch file. This will create rosbags in the same directory. Therefore, the recommendation is to run it in src/main_ws/install/share/path_following/db which is intended to store rosbags.

~/tcc-autonomous-ship/src/main_ws/install/share/path_following/db> ros2 launch path_following path_following.launch.py

4. In order to verify topics, the user may run:

~> ros2 topic list -t

5. To get a graph of topics and nodes, the user may run:

~> rqt_graph

6. Just like for the pydyna_simple node, the user can publish and subscribe to different topics, or call services; to verify if they are working accordingly.

7. With all the nodes working, the user can visualize the vessel through Venus on http://localhost:6150

8. Once the server is up and running, the user can send HTTP requests, giving the initial conditions and waypoints, for example, using Insomnia client. Once this is done, the user can send a request to start the simulation, in the same way. This is explained and illustrated in this section

Logs, plots and data

After running a simulation using a package, go to the package's installation and enter the share folder. There, you will find generated logs, data and eventually generated plots, of the last simulation. Below, the folders that contain these files are presented for each package.

pydyna_simple package

• / → pydyna_simple installation share directory:
  • /logs → contains custom logs, logs generated by the pydyna_simple node and pydyna reports;
  • /db → contains rosbags.

path_following package

• / → path_following installation share directory:
  • /logs → contains custom logs, logs generated by each node and pydyna reports;
  • /plots → contains generated plots;
  • /db → contains desired waypoints and rosbags.

Useful scripts

We made some scripts to facilitate the process of setting up the environment, building and running the packages. In order to use these scripts, it is necessary to follow these two steps:

1. In helpers/configpublic.bat: fill all placeholder values with the actual paths;
2. Change the name configpublic to config.

Source Code 5 - configpublic.bat

@REM ROS2 installation directory, where setup.bat is located
set underlay_path="placeholder"
@REM ROS2 worskpace source directory
set ws_src_path="placeholder"
@REM ROS2 workspace install directory, where local_setup.bat is located
set overlay_path="placeholder"
@REM ROS2 workspace directory
set main_ws_path="placeholder"

@REM pydyna_simple share directory: db folder
set pydyna_pkg_db_path="placeholder"
@REM pydyna_simple share directory: db>rosbags folder
set pydyna_pkg_rosbags_path="placeholder"

@REM path_following share directory: logs folder
set path_pkg_logs_path="placeholder"
@REM path_following share directory: db folder
set pydyna_pkg_db_path="placeholder"

@REM directory where http json payloads are being kept (only when using curl)
set payloads_path="placeholder" 

pydyna_simple package

To build all the packages, run the following inside X64 Native Tools Comand Prompt for VS:

   ~/tcc-autonomous-ship/helpers> call vsbuildall.bat

To source the worskpace (set environment variables relative to your workspace), always run the following when starting a new terminal (process):

   ~/tcc-autonomous-ship/helpers> call overlay.bat

To run the pydyna_simple package, run the following in any terminal:

   ~/tcc-autonomous-ship/helpers> call pydyna.bat

path_following package

To build all the packages, run the following inside X64 Native Tools Comand Prompt for VS:

   ~/tcc-autonomous-ship/helpers> call vsbuildall.bat

To source the worskpace (set environment variables relative to your workspace), always run the following when starting a new terminal (process):

   ~/tcc-autonomous-ship/helpers> call overlay.bat

To run the path_following package, run the following in any terminal:

   ~/tcc-autonomous-ship/helpers> call path.bat

Project details

Our monograph, paper and slides are inside the reports folder.

Monograph here. Paper here. Slides here.

Important notes

*¹ DDS stands for Data Distribution Service. ROS2 uses a DDS implementation to do it's communication. In theory, ROS2 already comes with a default DDS and there would be no necessity of further investigation. In practice however, we encountered problems and had to use a third-party DDS. We used RTI Connext DDS. To use it, we asked for a student's licence and followed the instructions provided.

Additional material on ROS2

ROS2 documentation and tutorials can be found here. A ROS2 CLI cheatsheet can be found here. Our ROS2 notes can be found here.

Authors

Bruno Scaglione and Pedro Marzagão are the developers of the project. Here is Bruno's linkedin profile. Here is Pedro's linkedin profile.