fix(-): non uniformity of concatenated pointcloud publishing speed

[emuemuJP]

Description

The number of point clouds to be publish is suddenly reduced, which has negative effects such as unstable ROI shape.

pointpainting_roi_cluster_fusion_objects_before.webm

Related links

Parent Issue:

• fix(shape_estimation): unintented roi shape estimation #8842

How was this PR tested?

• Test with this ros bag data.

Notes for reviewers

None.

Interface changes

None.

Effects on system behavior

None.

[github-actions]

Thank you for contributing to the Autoware project!

🚧 If your pull request is in progress, switch it to draft mode.

Please ensure:

• You've checked our contribution guidelines.
• Your PR follows our pull request guidelines.
• All required CI checks pass before marking the PR ready for review.

[emuemuJP]

If I check the number of point clouds actually published, the top point cloud is excluded from the point clouds to be concatenated.

[emuemuJP]

Node: /perception/object_recognition/detection/object_association_merger_araya_AERO_16_YE5_6871_436549973956238849
topic: /perception/object_recognition/detection/clustering/camera_lidar_fusion/objects
↑
Node: /perception/object_recognition/detection/clustering/camera_lidar_fusion/detected_object_feature_remover
topic: /perception/object_recognition/detection/clustering/camera_lidar_fusion/objects
↑
Node: /perception/object_recognition/detection/clustering/camera_lidar_fusion/shape_estimation
topic: /perception/object_recognition/detection/clustering/camera_lidar_fusion/objects_with_feature
↑
Node: /perception/object_recognition/detection/clustering/camera_lidar_fusion/roi_cluster_fusion
topic: /perception/object_recognition/detection/clustering/camera_lidar_fusion/clusters
↑
Node: /perception/object_recognition/detection/clustering/cluster_merger
topic: /perception/object_recognition/detection/clustering/clusters
↑
Node: /perception/object_recognition/detection/clustering/roi_cluster/roi_pointcloud_fusion
topic: /perception/object_recognition/detection/clustering/roi_cluster/clusters
↑
Node: /perception/object_recognition/detection/voxel_based_compare_map_filter
topic: /perception/object_recognition/detection/pointcloud_map_filtered/pointcloud
↑
Node: /perception/object_recognition/detection/voxel_grid_downsample_filter
topic: /perception/obstacle_segmentation/pointcloud_map_filtered/downsampled/pointcloud
↑
Node: /perception/obstacle_segmentation/occupancy_grid_based_outlier_filter
topic: /perception/obstacle_segmentation/pointcloud
↑
Node: /perception/obstacle_segmentation/common_ground_filter
topic: /perception/obstacle_segmentation/single_frame/pointcloud
↑
Node: /perception/obstacle_segmentation/crop_box_filter
topic: /perception/obstacle_segmentation/range_cropped/pointcloud
↑
Node: /sensing/lidar/concatenate_data
topic: /sensing/lidar/concatenated/pointcloud
↑
Node: /sensing/lidar/{xxx}/ring_outlier_filter
topics:
/sensing/lidar/left/pointcloud
/sensing/lidar/rear/pointcloud
/sensing/lidar/right/pointcloud
/sensing/lidar/top/pointcloud
↑
Node: /sensing/lidar/{xxx}/distortion_corrector_node
topic: /sensing/lidar/{xxx}/rectified/pointcloud_ex
↑
Node: /sensing/lidar/{xxx}/crop_box_filter_mirror
topic: /sensing/lidar/{xxx}/mirror_cropped/pointcloud_ex
↑
Node: /sensing/lidar/{xxx}/crop_box_filter_self
topic: /sensing/lidar/{xxx}/self_cropped/pointcloud_ex
↑
Node: /sensing/lidar/{xxx}/velodyne_driver_ros_wrapper_node
topic: /sensing/lidar/{xxx}/pointcloud_raw_ex
↑
Node: rosbag
topic: /sensing/lidar/{xxx}/velodyne_packets

[emuemuJP]

Here, the waiting time for each topic is set.
The longer the timeout, the more it is integrated.
But of course, this is undesirable because the frequency of topic publication will decrease.

https://github.com/tier4/aip_launcher/blob/6a26e822243ed8703638b1e7e6c3ba4aae73befd/aip_xx1_launch/launch/pointcloud_preprocessor.launch.py#L46C1-L52C64

[emuemuJP]

It is possible that the data in velodyne_packets does not contain any contents, so I will investigate including that point.

[Owen-Liuyuxuan]

Please put the ticket/bag link to the PR description, so that other internal member can also track the issue.

[emuemuJP]

Top lidar and other lidars are not same.
[component_container_mt-29] SensorModel::VELODYNE_VLS128
[component_container-31] SensorModel::VELODYNE_VLP16
[component_container-32] SensorModel::VELODYNE_VLP16
[component_container-30] SensorModel::VELODYNE_VLP16

[emuemuJP]

Cases where point clouds exist in rosbag and have been converted, but have not been integrated.

[component_container-31] velodyne_scan->packets.size() = 75
[component_container-31] [INFO] [1730101252.751180432] [vls16_decoder]: Points count - scan_pc: 8538, overflow_pc: 0
[component_container-30] velodyne_scan->packets.size() = 76
[component_container-32] velodyne_scan->packets.size() = 76
[component_container-30] [INFO] [1730101252.752043478] [vls16_decoder]: Points count - scan_pc: 8287, overflow_pc: 0
[obstacle_pointcloud_based_validator_node-55] [WARN] [1730101252.756735050] [perception.object_recognition.detection.obstacle_pointcloud_based_validator_node]: cannot receive pointcloud
[component_container_mt-29] velodyne_scan->packets.size() = 603
[component_container_mt-29] [INFO] [1730101252.769706788] [vls128_decoder]: Points count - scan_pc: 190069, overflow_pc: 354
[component_container_mt-1] publishing concatened pointcloud
[component_container_mt-1] num points: 16825
[component_container_mt-1] publishing /sensing/lidar/left/pointcloud
[component_container_mt-1] num points: 8287
[component_container_mt-1] publishing /sensing/lidar/right/pointcloud
[component_container_mt-1] num points: 8538

[emuemuJP]

The left velodyne packets had no delay, but the Top velodyne packets had an interval of 200 ms.

データ数: 296
平均差分: 0.099997 秒

最大差分: 0.109516 秒 (109.52 ms)

データ数: 295
平均差分: 0.100320 秒

最大差分: 0.196633 秒 (196.63 ms)

On the other hand, the node interpreting velodyne packets(velodyne_decoder_ros_wrapper.cpp) often takes about 200 ms, which may be the reason for the lack of integration.
Case: Top Velodyne packets

Message 1 -> 2: 99.775 ms
Message 2 -> 3: 98.518 ms
Message 3 -> 4: 203.668 ms
Message 4 -> 5: 199.875 ms
Message 5 -> 6: 200.336 ms
Message 6 -> 7: 97.608 ms
Message 7 -> 8: 99.380 ms
Message 8 -> 9: 99.105 ms
Message 9 -> 10: 99.951 ms
Message 10 -> 11: 99.971 ms
Message 11 -> 12: 100.056 ms
Message 12 -> 13: 100.037 ms
Message 13 -> 14: 99.806 ms
Message 14 -> 15: 100.111 ms
Message 15 -> 16: 101.071 ms
Message 16 -> 17: 98.582 ms
Message 17 -> 18: 104.835 ms
Message 18 -> 19: 198.319 ms
Message 19 -> 20: 96.594 ms
Message 20 -> 21: 204.447 ms
Message 21 -> 22: 197.809 ms
Message 22 -> 23: 98.411 ms
Message 23 -> 24: 100.012 ms
Message 24 -> 25: 100.156 ms
Message 25 -> 26: 99.877 ms
Message 26 -> 27: 105.081 ms
Message 27 -> 28: 95.789 ms
Message 28 -> 29: 201.195 ms
Message 29 -> 30: 97.990 ms
Message 30 -> 31: 204.392 ms
Message 31 -> 32: 95.462 ms
Message 32 -> 33: 100.424 ms
Message 33 -> 34: 199.443 ms
Message 34 -> 35: 99.929 ms
Message 35 -> 36: 99.928 ms
Message 36 -> 37: 102.224 ms
Message 37 -> 38: 197.744 ms
Message 38 -> 39: 200.854 ms
Message 39 -> 40: 199.044 ms
Message 40 -> 41: 112.927 ms
Message 41 -> 42: 88.444 ms
Message 42 -> 43: 98.881 ms
Message 43 -> 44: 100.654 ms
Message 44 -> 45: 200.173 ms
Message 45 -> 46: 202.625 ms
Message 46 -> 47: 97.698 ms
Message 47 -> 48: 98.824 ms
Message 48 -> 49: 103.562 ms
Message 49 -> 50: 96.340 ms
Message 50 -> 51: 199.901 ms
Message 51 -> 52: 100.036 ms
Message 52 -> 53: 100.310 ms
Message 53 -> 54: 99.721 ms
Message 54 -> 55: 99.838 ms
Message 55 -> 56: 200.053 ms
Message 56 -> 57: 100.839 ms
Message 57 -> 58: 99.316 ms
Message 58 -> 59: 99.904 ms
Message 59 -> 60: 100.021 ms
Message 60 -> 61: 100.091 ms
Message 61 -> 62: 100.085 ms
Message 62 -> 63: 99.824 ms
Message 63 -> 64: 105.368 ms
Message 64 -> 65: 94.720 ms
Message 65 -> 66: 100.105 ms
Message 66 -> 67: 103.452 ms
Message 67 -> 68: 196.647 ms
Message 68 -> 69: 100.128 ms
Message 69 -> 70: 100.078 ms
Message 70 -> 71: 199.982 ms
Message 71 -> 72: 100.229 ms
Message 72 -> 73: 100.992 ms
Message 73 -> 74: 98.988 ms
Message 74 -> 75: 99.919 ms
Message 75 -> 76: 99.789 ms
Message 76 -> 77: 100.023 ms
Message 77 -> 78: 99.747 ms
Message 78 -> 79: 99.868 ms
Message 79 -> 80: 99.808 ms
Message 80 -> 81: 109.514 ms
Message 81 -> 82: 90.400 ms
Message 82 -> 83: 199.956 ms
Message 83 -> 84: 99.923 ms
Message 84 -> 85: 100.169 ms
Message 85 -> 86: 99.821 ms
Message 86 -> 87: 307.337 ms
Message 87 -> 88: 93.210 ms
Message 88 -> 89: 100.040 ms
Message 89 -> 90: 200.156 ms
Message 90 -> 91: 100.048 ms
Message 91 -> 92: 100.023 ms
Message 92 -> 93: 100.140 ms
Message 93 -> 94: 205.035 ms
Message 94 -> 95: 95.696 ms
Message 95 -> 96: 104.123 ms
Message 96 -> 97: 94.439 ms
Message 97 -> 98: 203.762 ms
Message 98 -> 99: 96.119 ms
Message 99 -> 100: 100.532 ms
Message 100 -> 101: 99.498 ms
Message 101 -> 102: 104.032 ms
Message 102 -> 103: 96.133 ms
Message 103 -> 104: 303.662 ms
Message 104 -> 105: 96.795 ms
Message 105 -> 106: 203.852 ms
Message 106 -> 107: 96.115 ms
Message 107 -> 108: 100.091 ms
Message 108 -> 109: 100.876 ms
Message 109 -> 110: 98.959 ms
Message 110 -> 111: 100.201 ms
Message 111 -> 112: 99.840 ms
Message 112 -> 113: 99.748 ms
Message 113 -> 114: 100.916 ms
Message 114 -> 115: 99.063 ms
Message 115 -> 116: 106.786 ms
Message 116 -> 117: 101.950 ms
Message 117 -> 118: 91.039 ms
Message 118 -> 119: 203.210 ms
Message 119 -> 120: 98.633 ms
Message 120 -> 121: 98.223 ms
Message 121 -> 122: 99.960 ms
Message 122 -> 123: 100.086 ms
Message 123 -> 124: 99.893 ms
Message 124 -> 125: 203.589 ms
Message 125 -> 126: 197.886 ms
Message 126 -> 127: 104.039 ms
Message 127 -> 128: 98.087 ms
Message 128 -> 129: 96.614 ms
Message 129 -> 130: 100.050 ms
Message 130 -> 131: 108.533 ms
Message 131 -> 132: 91.338 ms
Message 132 -> 133: 99.960 ms
Message 133 -> 134: 204.837 ms
Message 134 -> 135: 95.155 ms
Message 135 -> 136: 103.149 ms
Message 136 -> 137: 218.477 ms
Message 137 -> 138: 81.427 ms
Message 138 -> 139: 96.871 ms
Message 139 -> 140: 99.986 ms
Message 140 -> 141: 99.987 ms
Message 141 -> 142: 204.454 ms
Message 142 -> 143: 199.772 ms
Message 143 -> 144: 96.028 ms
Message 144 -> 145: 99.993 ms
Message 145 -> 146: 100.008 ms
Message 146 -> 147: 100.022 ms
Message 147 -> 148: 202.217 ms
Message 148 -> 149: 97.805 ms
Message 149 -> 150: 100.485 ms
Message 150 -> 151: 99.271 ms
Message 151 -> 152: 100.042 ms
Message 152 -> 153: 100.153 ms
Message 153 -> 154: 99.835 ms
Message 154 -> 155: 100.132 ms
Message 155 -> 156: 204.012 ms
Message 156 -> 157: 204.665 ms
Message 157 -> 158: 97.261 ms
Message 158 -> 159: 93.846 ms
Message 159 -> 160: 202.366 ms
Message 160 -> 161: 97.616 ms
Message 161 -> 162: 101.453 ms
Message 162 -> 163: 98.522 ms
Message 163 -> 164: 100.280 ms
Message 164 -> 165: 99.987 ms
Message 165 -> 166: 101.282 ms
Message 166 -> 167: 98.717 ms
Message 167 -> 168: 100.026 ms
Message 168 -> 169: 102.866 ms
Message 169 -> 170: 97.149 ms
Message 170 -> 171: 204.656 ms
Message 171 -> 172: 95.231 ms
Message 172 -> 173: 99.999 ms
Message 173 -> 174: 99.905 ms
Message 174 -> 175: 100.122 ms
Message 175 -> 176: 99.930 ms
Message 176 -> 177: 104.667 ms
Message 177 -> 178: 95.199 ms
Message 178 -> 179: 100.079 ms
Message 179 -> 180: 100.052 ms
Message 180 -> 181: 99.882 ms
Message 181 -> 182: 100.775 ms
Message 182 -> 183: 100.702 ms
Message 183 -> 184: 99.977 ms
Message 184 -> 185: 100.029 ms
Message 185 -> 186: 98.734 ms
Message 186 -> 187: 201.293 ms
Message 187 -> 188: 100.046 ms
Message 188 -> 189: 98.649 ms
Message 189 -> 190: 100.020 ms
Message 190 -> 191: 200.059 ms
Message 191 -> 192: 99.857 ms
Message 192 -> 193: 100.014 ms
Message 193 -> 194: 99.854 ms
Message 194 -> 195: 100.839 ms
Message 195 -> 196: 203.047 ms
Message 196 -> 197: 96.901 ms
Message 197 -> 198: 100.660 ms
Message 198 -> 199: 112.644 ms
Message 199 -> 200: 85.785 ms
Message 200 -> 201: 200.101 ms
Message 201 -> 202: 102.956 ms
Message 202 -> 203: 96.978 ms
Message 203 -> 204: 113.553 ms
Message 204 -> 205: 87.034 ms
Message 205 -> 206: 99.535 ms
Message 206 -> 207: 99.952 ms
Message 207 -> 208: 100.080 ms
Message 208 -> 209: 99.955 ms
Message 209 -> 210: 200.057 ms
Message 210 -> 211: 99.962 ms
Message 211 -> 212: 211.369 ms
Message 212 -> 213: 88.740 ms
Message 213 -> 214: 109.622 ms
Message 214 -> 215: 90.279 ms
Message 215 -> 216: 101.347 ms
Message 216 -> 217: 100.520 ms
Message 217 -> 218: 198.135 ms
Message 218 -> 219: 100.089 ms
Message 219 -> 220: 311.861 ms
Message 220 -> 221: 202.478 ms
Message 221 -> 222: 89.657 ms
Message 222 -> 223: 96.544 ms
Message 223 -> 224: 99.270 ms
Message 224 -> 225: 204.828 ms
Message 225 -> 226: 95.113 ms
Message 226 -> 227: 99.908 ms
Message 227 -> 228: 104.887 ms
Message 228 -> 229: 96.854 ms
Message 229 -> 230: 98.443 ms
Message 230 -> 231: 100.257 ms
Message 231 -> 232: 99.700 ms
Message 232 -> 233: 100.154 ms
Message 233 -> 234: 204.460 ms
Message 234 -> 235: 105.243 ms
Message 235 -> 236: 90.326 ms
Message 236 -> 237: 100.658 ms
Message 237 -> 238: 101.268 ms
Message 238 -> 239: 101.303 ms
Message 239 -> 240: 98.121 ms
Message 240 -> 241: 101.190 ms
Message 241 -> 242: 97.511 ms
Message 242 -> 243: 108.754 ms
Message 243 -> 244: 90.979 ms

Statistics:
Mean: 120.978 ms
Std: 45.160 ms
Min: 81.427 ms
Max: 311.861 ms
Median: 100.042 ms

Case: Left Velodyne Packets

Message 1 -> 2: 99.503 ms
Message 2 -> 3: 99.530 ms
Message 3 -> 4: 301.367 ms
Message 4 -> 5: 99.358 ms
Message 5 -> 6: 99.698 ms
Message 6 -> 7: 100.855 ms
Message 7 -> 8: 100.355 ms
Message 8 -> 9: 98.808 ms
Message 9 -> 10: 100.707 ms
Message 10 -> 11: 99.523 ms
Message 11 -> 12: 99.480 ms
Message 12 -> 13: 101.020 ms
Message 13 -> 14: 99.435 ms
Message 14 -> 15: 99.528 ms
Message 15 -> 16: 100.821 ms
Message 16 -> 17: 99.433 ms
Message 17 -> 18: 99.622 ms
Message 18 -> 19: 201.247 ms
Message 19 -> 20: 98.645 ms
Message 20 -> 21: 101.153 ms
Message 21 -> 22: 99.823 ms
Message 22 -> 23: 100.273 ms
Message 23 -> 24: 99.599 ms
Message 24 -> 25: 99.541 ms
Message 25 -> 26: 100.889 ms
Message 26 -> 27: 100.010 ms
Message 27 -> 28: 201.355 ms
Message 28 -> 29: 98.086 ms
Message 29 -> 30: 100.606 ms
Message 30 -> 31: 99.803 ms
Message 31 -> 32: 302.661 ms
Message 32 -> 33: 96.639 ms
Message 33 -> 34: 200.712 ms
Message 34 -> 35: 99.317 ms
Message 35 -> 36: 100.920 ms
Message 36 -> 37: 99.484 ms
Message 37 -> 38: 104.438 ms
Message 38 -> 39: 97.775 ms
Message 39 -> 40: 97.874 ms
Message 40 -> 41: 99.528 ms
Message 41 -> 42: 100.786 ms
Message 42 -> 43: 99.516 ms
Message 43 -> 44: 99.494 ms
Message 44 -> 45: 100.795 ms
Message 45 -> 46: 199.111 ms
Message 46 -> 47: 100.799 ms
Message 47 -> 48: 99.500 ms
Message 48 -> 49: 99.553 ms
Message 49 -> 50: 101.079 ms
Message 50 -> 51: 99.410 ms
Message 51 -> 52: 99.451 ms
Message 52 -> 53: 100.839 ms
Message 53 -> 54: 99.567 ms
Message 54 -> 55: 99.532 ms
Message 55 -> 56: 200.472 ms
Message 56 -> 57: 101.005 ms
Message 57 -> 58: 99.419 ms
Message 58 -> 59: 99.472 ms
Message 59 -> 60: 299.852 ms
Message 60 -> 61: 100.848 ms
Message 61 -> 62: 99.588 ms
Message 62 -> 63: 99.487 ms
Message 63 -> 64: 101.050 ms
Message 64 -> 65: 99.324 ms
Message 65 -> 66: 99.456 ms
Message 66 -> 67: 100.980 ms
Message 67 -> 68: 99.559 ms
Message 68 -> 69: 101.644 ms
Message 69 -> 70: 99.328 ms
Message 70 -> 71: 98.955 ms
Message 71 -> 72: 100.815 ms
Message 72 -> 73: 100.426 ms
Message 73 -> 74: 199.588 ms
Message 74 -> 75: 99.675 ms
Message 75 -> 76: 101.308 ms
Message 76 -> 77: 99.033 ms
Message 77 -> 78: 101.493 ms
Message 78 -> 79: 98.305 ms
Message 79 -> 80: 99.974 ms
Message 80 -> 81: 99.649 ms
Message 81 -> 82: 99.360 ms
Message 82 -> 83: 100.889 ms
Message 83 -> 84: 100.055 ms
Message 84 -> 85: 200.004 ms
Message 85 -> 86: 99.389 ms
Message 86 -> 87: 100.978 ms
Message 87 -> 88: 99.491 ms
Message 88 -> 89: 199.019 ms
Message 89 -> 90: 200.506 ms
Message 90 -> 91: 101.121 ms
Message 91 -> 92: 99.213 ms
Message 92 -> 93: 99.385 ms
Message 93 -> 94: 101.052 ms
Message 94 -> 95: 99.403 ms
Message 95 -> 96: 99.543 ms
Message 96 -> 97: 200.458 ms
Message 97 -> 98: 99.491 ms
Message 98 -> 99: 100.787 ms
Message 99 -> 100: 99.542 ms
Message 100 -> 101: 99.675 ms
Message 101 -> 102: 103.490 ms
Message 102 -> 103: 96.912 ms
Message 103 -> 104: 200.392 ms
Message 104 -> 105: 99.485 ms
Message 105 -> 106: 100.805 ms
Message 106 -> 107: 99.618 ms
Message 107 -> 108: 99.483 ms
Message 108 -> 109: 100.870 ms
Message 109 -> 110: 100.481 ms
Message 110 -> 111: 98.601 ms
Message 111 -> 112: 200.387 ms
Message 112 -> 113: 99.531 ms
Message 113 -> 114: 100.819 ms
Message 114 -> 115: 101.028 ms
Message 115 -> 116: 98.071 ms
Message 116 -> 117: 100.900 ms
Message 117 -> 118: 99.480 ms
Message 118 -> 119: 101.061 ms
Message 119 -> 120: 99.344 ms
Message 120 -> 121: 99.556 ms
Message 121 -> 122: 100.912 ms
Message 122 -> 123: 99.550 ms
Message 123 -> 124: 200.392 ms
Message 124 -> 125: 99.600 ms
Message 125 -> 126: 99.320 ms
Message 126 -> 127: 101.308 ms
Message 127 -> 128: 99.160 ms
Message 128 -> 129: 99.596 ms
Message 129 -> 130: 100.878 ms
Message 130 -> 131: 198.897 ms
Message 131 -> 132: 100.984 ms
Message 132 -> 133: 99.416 ms
Message 133 -> 134: 99.709 ms
Message 134 -> 135: 100.949 ms
Message 135 -> 136: 99.441 ms
Message 136 -> 137: 99.490 ms
Message 137 -> 138: 100.734 ms
Message 138 -> 139: 99.589 ms
Message 139 -> 140: 100.915 ms
Message 140 -> 141: 198.890 ms
Message 141 -> 142: 101.129 ms
Message 142 -> 143: 99.420 ms
Message 143 -> 144: 200.531 ms
Message 144 -> 145: 200.607 ms
Message 145 -> 146: 99.668 ms
Message 146 -> 147: 99.223 ms
Message 147 -> 148: 99.881 ms
Message 148 -> 149: 100.360 ms
Message 149 -> 150: 99.697 ms
Message 150 -> 151: 99.323 ms
Message 151 -> 152: 100.940 ms
Message 152 -> 153: 99.764 ms
Message 153 -> 154: 99.276 ms
Message 154 -> 155: 100.845 ms
Message 155 -> 156: 99.399 ms
Message 156 -> 157: 100.936 ms
Message 157 -> 158: 99.557 ms
Message 158 -> 159: 101.033 ms
Message 159 -> 160: 198.780 ms
Message 160 -> 161: 99.671 ms
Message 161 -> 162: 100.813 ms
Message 162 -> 163: 99.431 ms
Message 163 -> 164: 102.261 ms
Message 164 -> 165: 98.247 ms
Message 165 -> 166: 99.600 ms
Message 166 -> 167: 99.410 ms
Message 167 -> 168: 100.876 ms
Message 168 -> 169: 99.659 ms
Message 169 -> 170: 100.012 ms
Message 170 -> 171: 100.177 ms
Message 171 -> 172: 99.613 ms
Message 172 -> 173: 100.193 ms
Message 173 -> 174: 100.289 ms
Message 174 -> 175: 99.751 ms
Message 175 -> 176: 100.658 ms
Message 176 -> 177: 99.530 ms
Message 177 -> 178: 99.480 ms
Message 178 -> 179: 100.803 ms
Message 179 -> 180: 100.020 ms
Message 180 -> 181: 199.921 ms
Message 181 -> 182: 99.458 ms
Message 182 -> 183: 99.736 ms
Message 183 -> 184: 100.879 ms
Message 184 -> 185: 99.273 ms
Message 185 -> 186: 99.587 ms
Message 186 -> 187: 100.927 ms
Message 187 -> 188: 99.756 ms
Message 188 -> 189: 100.880 ms
Message 189 -> 190: 99.457 ms
Message 190 -> 191: 99.583 ms
Message 191 -> 192: 99.265 ms
Message 192 -> 193: 100.957 ms
Message 193 -> 194: 99.497 ms
Message 194 -> 195: 101.350 ms
Message 195 -> 196: 99.190 ms
Message 196 -> 197: 99.464 ms
Message 197 -> 198: 100.814 ms
Message 198 -> 199: 99.439 ms
Message 199 -> 200: 99.651 ms
Message 200 -> 201: 100.858 ms
Message 201 -> 202: 99.456 ms
Message 202 -> 203: 99.516 ms
Message 203 -> 204: 100.933 ms
Message 204 -> 205: 99.402 ms
Message 205 -> 206: 99.671 ms
Message 206 -> 207: 100.818 ms
Message 207 -> 208: 99.545 ms
Message 208 -> 209: 100.931 ms
Message 209 -> 210: 99.437 ms
Message 210 -> 211: 99.446 ms
Message 211 -> 212: 201.228 ms
Message 212 -> 213: 98.864 ms
Message 213 -> 214: 101.014 ms
Message 214 -> 215: 99.363 ms
Message 215 -> 216: 99.513 ms
Message 216 -> 217: 101.141 ms
Message 217 -> 218: 99.293 ms
Message 218 -> 219: 100.031 ms
Message 219 -> 220: 100.255 ms
Message 220 -> 221: 99.856 ms
Message 221 -> 222: 99.443 ms
Message 222 -> 223: 100.781 ms
Message 223 -> 224: 99.558 ms
Message 224 -> 225: 99.583 ms
Message 225 -> 226: 200.280 ms
Message 226 -> 227: 101.102 ms
Message 227 -> 228: 99.263 ms
Message 228 -> 229: 99.531 ms
Message 229 -> 230: 101.030 ms
Message 230 -> 231: 99.434 ms
Message 231 -> 232: 99.578 ms
Message 232 -> 233: 100.709 ms
Message 233 -> 234: 99.946 ms
Message 234 -> 235: 200.457 ms
Message 235 -> 236: 98.890 ms
Message 236 -> 237: 99.673 ms
Message 237 -> 238: 100.965 ms
Message 238 -> 239: 99.412 ms
Message 239 -> 240: 99.806 ms
Message 240 -> 241: 100.703 ms
Message 241 -> 242: 100.782 ms
Message 242 -> 243: 98.164 ms
Message 243 -> 244: 101.068 ms
Message 244 -> 245: 99.393 ms
Message 245 -> 246: 100.940 ms
Message 246 -> 247: 100.453 ms
Message 247 -> 248: 98.448 ms
Message 248 -> 249: 100.965 ms
Message 249 -> 250: 99.485 ms
Message 250 -> 251: 200.860 ms
Message 251 -> 252: 99.155 ms
Message 252 -> 253: 99.510 ms
Message 253 -> 254: 100.860 ms
Message 254 -> 255: 99.809 ms
Message 255 -> 256: 99.096 ms
Message 256 -> 257: 101.053 ms
Message 257 -> 258: 99.400 ms
Message 258 -> 259: 99.718 ms
Message 259 -> 260: 200.185 ms
Message 260 -> 261: 99.834 ms
Message 261 -> 262: 101.378 ms
Message 262 -> 263: 98.777 ms
Message 263 -> 264: 100.841 ms
Message 264 -> 265: 99.443 ms

Statistics:
Mean: 111.361 ms
Std: 35.278 ms
Min: 96.639 ms
Max: 302.661 ms
Median: 99.727 ms

[emuemuJP]

The way I got the above display was to output the topic header
ros2 topic echo /sensing/lidar/top/pointcloud_raw_ex --field header > output.txt
and then parsed it later with a simple python script.

[emuemuJP]

I analyzed following published time of topic.
/sensing/lidar/top/pointcloud/debug/published_time

I still think there is a major spike lag. Is it a problem with the execution environment?
OS: Ubuntu 22.04.5 LTS
CPU:12th Gen Intel(R) Core(TM) i9-12900HK
GPU: GeForce RTX 3080 Ti
Memory: 64GB

[Owen-Liuyuxuan]

I also have this problem in my environment, even I replayed the ROS bag slowly.

[emuemuJP]

The processing delay in the decoder of Velodyne packets is affecting the processing in the latter stage.

・Pipeline nodes in the top point cloud

・Only velodyne packets node

・2nd trial

[emuemuJP]

Reference:
・Pipeline nodes in the left point cloud

[Owen-Liuyuxuan]

It seems that decoder is slow, I will also ask the component team for ideas.

[knzo25]

Sorry for butting in.
I could not follow the whole conversation, but I may have some insight of some of the issues you are facing.

• If there are time differences of ~200ms between packet messages, there was a packet loss
• The velodyne drivers (nebula and the one we used before), do not use the whole packet to produce a pointcloud. A few degrees worth of data is saved for the next pointcloud (synchronization concerns)
• The next packet will use the previously saved points in addition to the decoded pointcloud from the current packet (save the last few degrees worth of data)
• This means that when a packet loss occurs, the following pointclouds will have a dt in their timestamps of about 200ms

[mojomex]

I'm one of the Nebula maintainers, I'll quickly take a look at the bag to see if it's packet loss or an issue with Nebula!

[mojomex]

I ran the bag (just top LiDAR packets) locally and

• all scan messages have around 603 packets, which is good
• there is one scan message missing in the bag, see screenshot below
• with the latest Nebula release, v0.2.2, there are no other issues than that one hiccup caused by the missing packet.

Which version of Nebula are you currently using @emuemuJP?

[Owen-Liuyuxuan]

@mojomex
The test should be on this commit.

which is part of nebula/v0.01 on April 3rd.

It is the nebula used in XX1:beta/v0.45.0

[mojomex]

@Owen-Liuyuxuan
I see, thank you for the information. This is quite an old version by now, so I would suggest updating to v0.2.1 or newer, since there have been a lot of stability and performance improvements since then.

[emuemuJP]

Thank you everyone!
I'll try again with nabula v0.2.1.

[emuemuJP]

@mojomex
I was able to successfully build with v0.2.2.
However, this version does not seem to publish pointcloud2 as expected by the original node pipeline.
https://github.com/tier4/nebula/blob/18af521fc3a3ab1ce9c17ec94c2a0a8633317de4/nebula_ros/src/velodyne/velodyne_decoder_ros_wrapper.cpp#L35
How can this be used to receive Velodyne packets and publish pointcloud2?

Node info

araya@araya-AERO-16-YE5:~/data$ ros2 node info /velodyne_ros_wrapper_node
/velodyne_ros_wrapper_node
  Subscribers:
    /clock: rosgraph_msgs/msg/Clock
    /parameter_events: rcl_interfaces/msg/ParameterEvent
  Publishers:
    /parameter_events: rcl_interfaces/msg/ParameterEvent
    /rosout: rcl_interfaces/msg/Log
  Service Servers:
    /velodyne_ros_wrapper_node/describe_parameters: rcl_interfaces/srv/DescribeParameters
    /velodyne_ros_wrapper_node/get_parameter_types: rcl_interfaces/srv/GetParameterTypes
    /velodyne_ros_wrapper_node/get_parameters: rcl_interfaces/srv/GetParameters
    /velodyne_ros_wrapper_node/list_parameters: rcl_interfaces/srv/ListParameters
    /velodyne_ros_wrapper_node/set_parameters: rcl_interfaces/srv/SetParameters
    /velodyne_ros_wrapper_node/set_parameters_atomically: rcl_interfaces/srv/SetParametersAtomically
  Service Clients:

  Action Servers:

  Action Clients:

Then, after waiting for a while after launch, the following error message appeared.

[component_container_mt-1] [INFO] [1731391483.526020920] [IoContext::IoContext]: Thread(s) Created: 1
[component_container_mt-1] [ERROR] [1731391619.607506574] [pointcloud_preprocessor.velodyne_node_container]: Component constructor threw an exception: Could not initialize HTTP client: Http Connection Error
[ERROR] [launch_ros.actions.load_composable_nodes]: Failed to load node 'velodyne_ros_wrapper_node' of type 'VelodyneRosWrapper' in container '/pointcloud_preprocessor/velodyne_node_container': Component constructor threw an exception: Could not initialize HTTP client: Http Connection Error

[mojomex]

@emuemuJP Hi, it looks like a network error to me. Could you confirm your IP settings? There is a config file for each sensor in nebula_ros/config/lidar/velodyne, please compare the settings there with your network setup, Wireshark etc.

[mojomex]

Ah, are you trying to run off a rosbag?
Could you include the param dump of Nebula and how you launched it?

[emuemuJP]

@mojomex
Thanks for the reply!
I had a chance to talk to @Owen-Liuyuxuan today and asked him about it, and it seems that the problem was that the launch_hw option was set to true (I was trying to play rosbag...), so I changed this to false and now it works.
Thank you very much!

[mojomex]

Thank you for the confirmation! We've only updated the Nebula documentation to cover this a few days ago, so it's understandable that people haven't seen it yet.

If there's any further trouble, the new documentation provides some useful info 🙂

[emuemuJP]

@Owen-Liuyuxuan
I tried again to make debug/published_time outputable and looked at the time, but there still seems to be some spike lag.
Top point clouds published time

Reference:
Left point clouds published time

[Owen-Liuyuxuan]

@mojomex
Can you also reproduce this result? It seems that the problem is still similar in the new nebula?

[mojomex]

@Owen-Liuyuxuan With the new Nebula, I was not able to reproduce the result: comment from above.

• What is the overall setup you are using? Just default Autoware?
• Is the graph always the same, even when re-running the experiment?
• On which hardware is this running and what CPU/(network) usage is there?

[emuemuJP]

@mojomex

• Using this branch: https://github.com/tier4/pilot-auto.xx1/tree/v0.45.0
• It's not same. second run results.
  
• Spec is following.
  OS: Ubuntu 22.04.5 LTS
  CPU:12th Gen Intel(R) Core(TM) i9-12900HK
  GPU: GeForce RTX 3080 Ti
  Memory: 64GB

CPU/(network) usage is around 50%.

cpu_usage.webm

[mojomex]

@emuemuJP Hm, this indeed looks like a performance issue then.
Since you are running this on a laptop, and the CPU usage never goes above 50%, it could be the case that it's throttling or in energy saving mode.

When using tools like s-tui (sudo apt install s-tui, then type s-tui in the terminal), do the temperature bars ever get red? Is the CPU frequency stable and reasonably high (2-3 GHz)?

To be honest, Velodyne is not high on our priority list and the code is known to be quite........, so I cannot spend a lot of time on it unless the above factors are confirmed.
There is a PR from our part timer tier4/nebula#228 which should bring some simplifications and improvements. I don't have a real sensor to test with, but if you are able to give it a run, it could help. 🙇

[emuemuJP]

@mojomex
I see, thank you very much.
I'll give it a try s-tui and discuss future actions with @Owen-Liuyuxuan .

Thank you for your kind reply!

[emuemuJP]

My laptop was balanced mode.
In this case, CPU usage is around 50-60%.
balanced_720.webm

I changed mode from balanced to performance.
In this case, the temperature gets red but CPU usage gets around 40% and results don't change.
performance_720.webm

[emuemuJP]

I tried running rosbag with the playback speed set to 0.1, but there still seems to be a spike lag

[emuemuJP]

packet timestamps are here.

---
stamp:
  sec: 1671592125
  nanosec: 480910336
frame_id: velodyne_top
---
stamp:
  sec: 1671592125
  nanosec: 581353472
frame_id: velodyne_top
---
stamp:
  sec: 1671592125
  nanosec: 682152960
frame_id: velodyne_top
---

Processing log is following.
component_container_mt-29] [INFO] [1732642358.710178859] [sensing.lidar.top.velodyne_ros_wrapper_node]: Packet Time:1671592125480910336
[component_container_mt-29] [INFO] [1732642358.711319341] [sensing.lidar.top.velodyne_ros_wrapper_node]: Queue size:602
[component_container_mt-29] [INFO] [1732642358.715947324] [VelodyneDriver]: Set PointCloud
[component_container_mt-29] [INFO] [1732642358.716007630] [sensing.lidar.top.velodyne_ros_wrapper_node.VelodyneDecoder]: Publish Pointcloud
[component_container_mt-29] [INFO] [1732642358.721642034] [sensing.lidar.top.velodyne_ros_wrapper_node.VelodyneDecoder]: Publish Done.
[component_container_mt-29] [INFO] [1732642358.721658863] [sensing.lidar.top.velodyne_ros_wrapper_node.VelodyneDecoder]: aw_points_ex_pub: 1, 1
[component_container_mt-29] [INFO] [1732642358.723958816] [sensing.lidar.top.velodyne_ros_wrapper_node.VelodyneDecoder]: Publish Done.
[component_container_mt-1] [WARN] [1732642358.753091319] [sensing.lidar.concatenate_data]: transformed_raw_points[/sensing/lidar/rear/pointcloud] is nullptr, skipping pointcloud publish.
[component_container-31] [INFO] [1732642359.149074109] [sensing.lidar.right.velodyne_ros_wrapper_node]: Packet Time:1671592125570002688
[component_container-31] [INFO] [1732642359.149552204] [sensing.lidar.right.velodyne_ros_wrapper_node]: Queue size:75
[component_container-31] [INFO] [1732642359.150501465] [VelodyneDriver]: Set PointCloud
[component_container-31] [INFO] [1732642359.150583794] [sensing.lidar.right.velodyne_ros_wrapper_node.VelodyneDecoder]: Publish Pointcloud
[component_container-31] [INFO] [1732642359.150701272] [sensing.lidar.right.velodyne_ros_wrapper_node.VelodyneDecoder]: aw_points_ex_pub: 1, 1
[component_container-31] [INFO] [1732642359.151064993] [sensing.lidar.right.velodyne_ros_wrapper_node.VelodyneDecoder]: Publish Done.
[component_container-30] [INFO] [1732642359.151091944] [sensing.lidar.left.velodyne_ros_wrapper_node]: Packet Time:1671592125570457600
[component_container-30] [INFO] [1732642359.151619186] [sensing.lidar.left.velodyne_ros_wrapper_node]: Queue size:75
[component_container-30] [INFO] [1732642359.152181731] [VelodyneDriver]: Set PointCloud
[component_container-30] [INFO] [1732642359.152221832] [sensing.lidar.left.velodyne_ros_wrapper_node.VelodyneDecoder]: Publish Pointcloud
[component_container-30] [INFO] [1732642359.152242958] [sensing.lidar.left.velodyne_ros_wrapper_node.VelodyneDecoder]: aw_points_ex_pub: 1, 1
[component_container-30] [INFO] [1732642359.152510889] [sensing.lidar.left.velodyne_ros_wrapper_node.VelodyneDecoder]: time_diff: 101005312 ns (0.101 sec)
[component_container-30] [INFO] [1732642359.152606233] [sensing.lidar.left.velodyne_ros_wrapper_node.VelodyneDecoder]: Publish Done.
[component_container-32] [INFO] [1732642359.153738575] [sensing.lidar.rear.velodyne_ros_wrapper_node]: Packet Time:1671592125570585856
[component_container-32] [INFO] [1732642359.154116385] [sensing.lidar.rear.velodyne_ros_wrapper_node]: Queue size:75
[component_container-32] [INFO] [1732642359.154426321] [VelodyneDriver]: Set PointCloud
[component_container-32] [INFO] [1732642359.154459919] [sensing.lidar.rear.velodyne_ros_wrapper_node.VelodyneDecoder]: Publish Pointcloud
[component_container-32] [INFO] [1732642359.154480593] [sensing.lidar.rear.velodyne_ros_wrapper_node.VelodyneDecoder]: aw_points_ex_pub: 1, 1
[component_container-32] [INFO] [1732642359.154503223] [sensing.lidar.rear.velodyne_ros_wrapper_node.VelodyneDecoder]: time_diff: 100801024 ns (0.101 sec)
[component_container-32] [INFO] [1732642359.154593345] [sensing.lidar.rear.velodyne_ros_wrapper_node.VelodyneDecoder]: Publish Done.
[component_container_mt-1] [WARN] [1732642359.516961149] [sensing.lidar.concatenate_data]: transformed_raw_points[/sensing/lidar/rear/pointcloud] is nullptr, skipping pointcloud publish.
[component_container_mt-1] [WARN] [1732642359.517098037] [sensing.lidar.concatenate_data]: transformed_raw_points[/sensing/lidar/top/pointcloud] is nullptr, skipping pointcloud publish.
[component_container-31] [INFO] [1732642359.656119619] [sensing.lidar.right.velodyne_ros_wrapper_node]: Packet Time:1671592125669756672
[component_container-31] [INFO] [1732642359.656483409] [sensing.lidar.right.velodyne_ros_wrapper_node]: Queue size:76
[component_container-31] [INFO] [1732642359.657357716] [VelodyneDriver]: Set PointCloud
[component_container-31] [INFO] [1732642359.657416970] [sensing.lidar.right.velodyne_ros_wrapper_node.VelodyneDecoder]: Publish Pointcloud
[component_container-31] [INFO] [1732642359.657449541] [sensing.lidar.right.velodyne_ros_wrapper_node.VelodyneDecoder]: aw_points_ex_pub: 1, 1
[component_container-31] [INFO] [1732642359.657854687] [sensing.lidar.right.velodyne_ros_wrapper_node.VelodyneDecoder]: Publish Done.
[component_container-30] [INFO] [1732642359.659414026] [sensing.lidar.left.velodyne_ros_wrapper_node]: Packet Time:1671592125670132992
[component_container-30] [INFO] [1732642359.659782201] [sensing.lidar.left.velodyne_ros_wrapper_node]: Queue size:75
[component_container-30] [INFO] [1732642359.660443153] [VelodyneDriver]: Set PointCloud
[component_container-30] [INFO] [1732642359.660497105] [sensing.lidar.left.velodyne_ros_wrapper_node.VelodyneDecoder]: Publish Pointcloud
[component_container-30] [INFO] [1732642359.660519308] [sensing.lidar.left.velodyne_ros_wrapper_node.VelodyneDecoder]: aw_points_ex_pub: 1, 1
[component_container-30] [INFO] [1732642359.660862380] [sensing.lidar.left.velodyne_ros_wrapper_node.VelodyneDecoder]: Publish Done.
[component_container-32] [INFO] [1732642359.661364242] [sensing.lidar.rear.velodyne_ros_wrapper_node]: Packet Time:1671592125670335744
[component_container-32] [INFO] [1732642359.661962531] [sensing.lidar.rear.velodyne_ros_wrapper_node]: Queue size:75
[component_container-32] [INFO] [1732642359.662317902] [VelodyneDriver]: Set PointCloud
[component_container-32] [INFO] [1732642359.662366609] [sensing.lidar.rear.velodyne_ros_wrapper_node.VelodyneDecoder]: Publish Pointcloud
[component_container-32] [INFO] [1732642359.662394383] [sensing.lidar.rear.velodyne_ros_wrapper_node.VelodyneDecoder]: aw_points_ex_pub: 1, 1
[component_container-32] [INFO] [1732642359.662442520] [sensing.lidar.rear.velodyne_ros_wrapper_node.VelodyneDecoder]: Publish Done.
[component_container_mt-29] [INFO] [1732642359.716414545] [sensing.lidar.top.velodyne_ros_wrapper_node]: Packet Time:1671592125682152960
[component_container_mt-29] [INFO] [1732642359.722460738] [sensing.lidar.top.velodyne_ros_wrapper_node]: Queue size:603
[component_container_mt-29] [INFO] [1732642359.730159674] [VelodyneDriver]: Set PointCloud
[component_container_mt-29] [INFO] [1732642359.730213580] [sensing.lidar.top.velodyne_ros_wrapper_node.VelodyneDecoder]: Publish Pointcloud
[component_container_mt-29] [INFO] [1732642359.736736482] [sensing.lidar.top.velodyne_ros_wrapper_node.VelodyneDecoder]: Publish Done.
[component_container_mt-29] [INFO] [1732642359.736756737] [sensing.lidar.top.velodyne_ros_wrapper_node.VelodyneDecoder]: aw_points_ex_pub: 1, 1
[component_container_mt-29] [INFO] [1732642359.739376713] [sensing.lidar.top.velodyne_ros_wrapper_node.VelodyneDecoder]: time_diff: 201242624 ns (0.201 sec)
[component_container_mt-29] [INFO] [1732642359.739403830] [sensing.lidar.top.velodyne_ros_wrapper_node.VelodyneDecoder]: Publish Done.

Packet
stamp:
sec: 1671592125
nanosec: 581353472
wasn't processed.

[mojomex]

Interesting, the queue size (I assume that's Nebula's mt_queue?) is way larger than expected...
We are getting rid of it soon (already did for Hesai in tier4/nebula#226), so there is one less failure point.
I can push that change to some branch and we can see if that fixes it, what do you think @emuemuJP?

[emuemuJP]

@mojomex
Yes, I was checking queue size here
https://github.com/tier4/nebula/blob/214373ae4ee131aa6c281d3e7f377c2ed1522ebc/nebula_ros/src/velodyne/velodyne_ros_wrapper.cpp#L167
That change could improve the problem but as far as I check sometimes the callback function wasn't called(as I shown on the log , processing packets only takes around 5ms.). https://github.com/tier4/nebula/blob/214373ae4ee131aa6c281d3e7f377c2ed1522ebc/nebula_ros/src/velodyne/velodyne_ros_wrapper.cpp#L64C5-L65C5
Also, I don't know if it is a publish or subscribe issue, but the packets are not handled correctly.
I used ros2 topic echo for this check, so it should not be a queue issue.

Originally, It had only one packet loss but while processing there were three packets lost.

[emuemuJP]

I made the same change and checked it.