Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(velodyne): implement generic velodyne decoder #144

Closed
wants to merge 45 commits into from

Conversation

ike-kazu
Copy link
Contributor

@ike-kazu ike-kazu commented May 3, 2024

PR Type

  • Improvement

Related Links

Description

Previously, there was a separate Velodyne decoder for each sensor. However, since the sensors function very similarly, this leads to large amounts of repeated code that is unnecessary and difficult to maintain.
This PR implements a generic decoder to handle all Velodyne sensors.

Design

VelodyneDecoder<SensorT>
Generic decoder using templating to get sensor-specific information
VLP16
Class holding VLP16 specific information
VLS128
Class holding VLS128 specific information

Review Procedure

  1. colcon test --event-handlers console_cohesion+ --packages-above nebula_common
  2. code review

Remarks

Current Progress:

Task Done? Notes
Add VLP16 and VLS128 support to generic decoder
Add VLP32 support to generic decoder Tests are broken, see below.
General refactor
Validate on data live
Evaluate performance

Pre-Review Checklist for the PR Author

PR Author should check the checkboxes below when creating the PR.

  • Assign PR to reviewer

Checklist for the PR Reviewer

Reviewers should check the checkboxes below before approval.

  • Commits are properly organized and messages are according to the guideline
  • (Optional) Unit tests have been written for new behavior
  • PR title describes the changes

Post-Review Checklist for the PR Author

PR Author should check the checkboxes below before merging.

  • All open points are addressed and tracked via issues or tickets

CI Checks

  • Build and test for PR: Required to pass before the merge.

@ike-kazu ike-kazu marked this pull request as draft May 3, 2024 09:25
@ike-kazu
Copy link
Contributor Author

ike-kazu commented May 3, 2024

@bgilby59

@ike-kazu
Copy link
Contributor Author

ike-kazu commented May 3, 2024

Thank you @bgilby59 for refactoring velodyne drivers!
I tested this and report the result about it below. Please confirm it!

Pointcloud Output

Field Type
azimuth screenshot_from_2024-05-01_17-34-49_720
channel screenshot_from_2024-05-01_17-37-36_720
distance screenshot_from_2024-05-01_17-40-35_720
elevation screenshot_from_2024-05-01_17-40-54_720
intensity screenshot_from_2024-05-01_17-41-18_720
return_type screenshot_from_2024-05-01_17-41-53_720
timestamp screenshot_from_2024-05-01_17-42-03_720

🟢Pointcloud Output is looking same as before! It is great refactoring!!

Decoder Performance

timing_comparison_720
🟢Performance is looking great! It is faster than before with your work!!

@ike-kazu ike-kazu marked this pull request as ready for review May 8, 2024 07:43
@ike-kazu
Copy link
Contributor Author

ike-kazu commented May 13, 2024

Progress Update: Reason why all vlp tests are broken

It has 2 main reasons

  • coordinates type is double. We need to put them in float.
  • Getting Azimuth code has bag (see below)
Correct output Now output
Screenshot from 2024-05-13 13-24-06 Screenshot from 2024-05-13 13-23-41

🔴 Pointcloud Shape is looking good. It has bag on just azimuth angle. Maybe getAzimuthCorrected in (each lidar).hpp file is broken

@mojomex
Copy link
Collaborator

mojomex commented May 14, 2024

Thank you @ike-kazu! The point cloud fields are looking good, and it's great the performance became better already.
As for the test cases, I advised @bgilby59 to set the fields to double to test if that makes it better but it seens you're right, we should set them back to float. See 0195ef0 for reference.

Once everything is float again, the tests for VLP16 and VLS128 should pass.

The azimuth angle is interesting! It looks like it's just rotated, right? It might only be that case for VLP32 as its decoder was quite different from the other two.

@ike-kazu ike-kazu force-pushed the feat/add-generic-velodyne-decoder branch from eb653c4 to c528884 Compare May 20, 2024 03:06
@ike-kazu
Copy link
Contributor Author

Progress Update

Thank you @mojomex for your advises! I tried to change all double field into float. Then, i can confirm that test is succeeded in vlp128, but it is failured in vlp16 and vlp32.
Furthermore, i tried test and confirm the failures at vlp32 and vlp128 in this commit 0195ef0. So, I think that I can solve the vlp16 test failure with past commit.
So, I will follow the steps below.

  1. fix the vlp16 decoder with seeing commits between current and 0195ef0.
  2. fix the vlp32 decorder by myself.

@ike-kazu
Copy link
Contributor Author

Progress Update

I tried the test with changing EXPECT_EQ into ASSERT_EQ at these parts in each lidar test file to debug about pointclouds mount differences.
I confirm that the number of pointclouds is different in each lidar. (See below)

# about vlp16 test
Expected equality of these values:
3:   pp1->points.size()
3:     Which is: 14768
3:   pp2->points.size()
3:     Which is: 14736
# about vlp32 test
Expected equality of these values:
5:   pp1->points.size()
5:     Which is: 26196
5:   pp2->points.size()
5:     Which is: 26216

@codecov-commenter
Copy link

codecov-commenter commented May 27, 2024

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

Attention: Patch coverage is 66.82028% with 72 lines in your changes missing coverage. Please review.

Project coverage is 11.52%. Comparing base (eb5e1b7) to head (244bed7).
Report is 6 commits behind head on main.

Files Patch % Lines
...ers_velodyne/decoders/velodyne_generic_decoder.hpp 77.57% 14 Missing and 23 partials ⚠️
...ders/nebula_decoders_velodyne/decoders/vls_128.hpp 41.66% 9 Missing and 5 partials ⚠️
...oders/nebula_decoders_velodyne/decoders/vlp_32.hpp 16.66% 8 Missing and 2 partials ⚠️
...oders/nebula_decoders_velodyne/decoders/vlp_16.hpp 50.00% 3 Missing and 2 partials ⚠️
...ula_decoders_velodyne/decoders/velodyne_sensor.hpp 0.00% 3 Missing ⚠️
...s/src/nebula_decoders_velodyne/velodyne_driver.cpp 0.00% 0 Missing and 3 partials ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@            Coverage Diff            @@
##            main     #144      +/-   ##
=========================================
+ Coverage   4.84%   11.52%   +6.67%     
=========================================
  Files        249       72     -177     
  Lines      19210     7118   -12092     
  Branches    1075     1064      -11     
=========================================
- Hits         931      820     -111     
+ Misses     17579     5508   -12071     
- Partials     700      790      +90     
Flag Coverage Δ
differential 11.52% <66.82%> (?)
total ?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@ike-kazu
Copy link
Contributor Author

@mojomex
hello! I have one question about nebula refactoring.
i don't know why this function is needed in each velodyne lidar. In this function, true azimuth angle whch can pass nebula_test is selected. I create for nebula_test. To pass this test, I should refactor codes to be not using calculated accurate azimuth angles and using before caluculated in only vlp32. But other velodyne lidar vlp16 and vls128 test are passed with using calculated accurate angles. i checked vlp16 and vlp32 velodyne user manual and confirm that calculating accurate angle is needed. I think that vlp32c test data is suspicious. i would like to ask your opinion.
thank you

@mojomex mojomex self-requested a review June 10, 2024 05:04
Copy link
Collaborator

@mojomex mojomex left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @ike-kazu, thank you for your great work!
I have reviewed the changes and found a few more points that still need a bit of tweaking (a few of them I missed in my initial review of #138). Could you please implement the requested changes?

Thank you very much!

* Raw Velodyne packet constants and structures.
*/
static const int SIZE_BLOCK = 100;
static const int RAW_SCAN_SIZE = 3; // TODO: remove
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove

static const int SIZE_BLOCK = 100;
static const int RAW_SCAN_SIZE = 3; // TODO: remove
static const int RAW_CHANNEL_SIZE = 3;
static const int SCANS_PER_BLOCK = 32; // TODO: remove
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove

static const int BLOCK_DATA_SIZE = (SCANS_PER_BLOCK * RAW_SCAN_SIZE);

static const double ROTATION_RESOLUTION = 0.01; // [deg]
static const uint16_t ROTATION_MAX_UNITS = 36000u; // [deg/100]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please change to 360 * 100u to make it clearer to the reader

Suggested change
static const uint16_t ROTATION_MAX_UNITS = 36000u; // [deg/100]
static const uint16_t ROTATION_MAX_UNITS = 360 * 100u; // [deg/100]

static const uint16_t RETURN_MODE_LAST = 56;
static const uint16_t RETURN_MODE_DUAL = 57;

const int PACKET_SIZE = 1206;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please rename to PACKET_SIZE_BYTES so the unit is clear

{
uint16_t header; ///< UPPER_BANK or LOWER_BANK
uint16_t rotation; ///< 0-35999, divide by 100 to get degrees
uint8_t data[BLOCK_DATA_SIZE];
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Please rename data to units (so it is easier to compare to Hesai's pandar_packet.hpp).
  • Please change the data type from uint8_t to raw_unit and create a raw_unit struct:
    struct raw_unit {
      uint16_t distance;
      uint8_t reflectivity;
    };

Comment on lines 4 to 5
#include "nebula_decoders/nebula_decoders_velodyne/decoders/vlp_16.hpp"
#include "nebula_decoders/nebula_decoders_velodyne/decoders/vls_128.hpp"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The generic decoder should not use any sensor-specific headers. Please remove.

Comment on lines 178 to 198
uint bank_origin = 0;
// Used to detect which bank of 32 lasers is in this block.
switch (current_block.header) {
case VLS128::BANK_1:
bank_origin = 0;
break;
case VLS128::BANK_2:
bank_origin = 32;
break;
case VLS128::BANK_3:
bank_origin = 64;
break;
case VLS128::BANK_4:
bank_origin = 96;
break;
default:
RCLCPP_ERROR(
rclcpp::get_logger("VelodyneDecoder"),
"Invalid bank origin detected in packet. Skipping packet.");
return; // bad packet: skip the rest
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is the same for all Velodyne sensors, move this code and the constants BANK_1 etc. to velodyne_sensor.hpp. If it is different for sensors, make a virtual function in velodyne_sensor.hpp and override it in the sensor .hpp files.

Comment on lines 242 to 244
for (int channel = 0;
channel < velodyne_packet::CHANNELS_PER_BLOCK && channel < SensorT::channels_per_firing_sequence;
channel++, k += velodyne_packet::RAW_CHANNEL_SIZE) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Iterate over raw_unit_ts here, for example:

for (size_t unit_idx = 0; unit_idx < velodyne_packet::UNITS_PER_BLOCK; ++unit_idx) {
  const auto & unit = current_block.units[unit_idx];
  // ...
}

Comment on lines 248 to 256
// Distance extraction.
current_return.bytes[0] = current_block.data[k];
current_return.bytes[1] = current_block.data[k + 1];
if (dual_return) {
other_return.bytes[0] =
block % 2 ? raw->blocks[block - 1].data[k] : raw->blocks[block + 1].data[k];
other_return.bytes[1] =
block % 2 ? raw->blocks[block - 1].data[k + 1] : raw->blocks[block + 1].data[k + 1];
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These should all access unit.distance instead (which is already uint16_t so only one line will be needed instead of two)


float last_azimuth_diff = 0;
uint16_t azimuth_next;
const uint8_t return_mode = velodyne_packet.data[velodyne_packet::RETURN_MODE_INDEX];
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should now use the return_mode field introduced in velodyne_packet.hpp

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove this as a git submodule

@ike-kazu
Copy link
Contributor Author

ike-kazu commented Jul 8, 2024

@mojomex
I finished to code with your advices, Thank you! Please review.

Copy link

sonarcloud bot commented Sep 30, 2024

@mojomex
Copy link
Collaborator

mojomex commented Nov 18, 2024

Will be continued in #228.

@mojomex mojomex closed this Nov 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants