How MINS dealwith multi thread problem？

[jiachenglihxl]

In the code， each sensor callback will use function "try to update" to update system state. And they all use StateHelper::EKFUpdate in the end.
My question is, what if two sensor come together, and call EKFUpdate at the same time?
Will this leads to coredump or memory corruption？

[aidadgy]

Hello， When I ran roslaunch mins simulation.launch cam_enabled:=true lidar_enabled:=true, the following problems occurred:
========Final Status========
Total procesing time: 21s
Total traveling time: 48s

RMSE average: 0.257, 0.117 (deg,m)
NEES average: 7.133, 2.380 (deg,m)
simulation: /usr/include/boost/thread/pthread/recursive_mutex.hpp:108: void boost::recursive_mutex::lock(): Assertion `!posix::pthread_mutex_lock(&m)' failed.
================================================================================REQUIRED process [mins_simulation0-2] has died!
process has died [pid 1327580, exit code -6, cmd /home/ubantu/catkin_ws/devel/lib/mins/simulation __name:=mins_simulation0 __log:=/home/ubantu/.ros/log/d0e201e6-885e-11ef-8f8b-41555d88daad/mins_simulation0-2.log].
log file: /home/ubantu/.ros/log/d0e201e6-885e-11ef-8f8b-41555d88daad/mins_simulation0-2*.log
Initiating shutdown!
[mins_simulation0-2] killing on exit
[rosout-1] killing on exit
[master] killing on exit
shutting down processing monitor...
... shutting down processing monitor complete
done
I wonder if the author can give some suggestions，thanks

[WoosikLee2510]

In the code， each sensor callback will use function "try to update" to update system state. And they all use StateHelper::EKFUpdate in the end. My question is, what if two sensor come together, and call EKFUpdate at the same time? Will this leads to coredump or memory corruption？

Hi, @jiachenglihxl. I'm sorry for my (extremely) late reply. The current implementation is not truly multi-threaded. This means that even if two different sensor measurements come to the system at the same time, whichever calls the callback function fist will be processed first, and the other one will wait until it can be processed.

[WoosikLee2510]

Hello， When I ran roslaunch mins simulation.launch cam_enabled:=true lidar_enabled:=true, the following problems occurred: ========Final Status======== Total procesing time: 21s Total traveling time: 48s

RMSE average: 0.257, 0.117 (deg,m) NEES average: 7.133, 2.380 (deg,m) simulation: /usr/include/boost/thread/pthread/recursive_mutex.hpp:108: void boost::recursive_mutex::lock(): Assertion `!posix::pthread_mutex_lock(&m)' failed. ================================================================================REQUIRED process [mins_simulation0-2] has died! process has died [pid 1327580, exit code -6, cmd /home/ubantu/catkin_ws/devel/lib/mins/simulation __name:=mins_simulation0 __log:=/home/ubantu/.ros/log/d0e201e6-885e-11ef-8f8b-41555d88daad/mins_simulation0-2.log]. log file: /home/ubantu/.ros/log/d0e201e6-885e-11ef-8f8b-41555d88daad/mins_simulation0-2*.log Initiating shutdown! [mins_simulation0-2] killing on exit [rosout-1] killing on exit [master] killing on exit shutting down processing monitor... ... shutting down processing monitor complete done I wonder if the author can give some suggestions，thanks

Hi, @aidadgy. I suspect the error comes from LiDAR, the only place I use multi-thread for fast parallel computation. However, I cannot produce your error on my end, so it is hard to debug where the error is coming from. One thing I noticed is your Total traveling time: 48s. If you ran the default simulation, the total travel time should be 48 s, indicating that you may get your error while terminating the system at the end of the run. Could you provide more information about your problem? It will be helpful to debug the issue.

[aidadgy]

Hello， When I ran roslaunch mins simulation.launch cam_enabled:=true lidar_enabled:=true, the following problems occurred: ========Final Status======== Total procesing time: 21s Total traveling time: 48s
RMSE average: 0.257, 0.117 (deg,m) NEES average: 7.133, 2.380 (deg,m) simulation: /usr/include/boost/thread/pthread/recursive_mutex.hpp:108: void boost::recursive_mutex::lock(): Assertion `!posix::pthread_mutex_lock(&m)' failed. ================================================================================REQUIRED process [mins_simulation0-2] has died! process has died [pid 1327580, exit code -6, cmd /home/ubantu/catkin_ws/devel/lib/mins/simulation __name:=mins_simulation0 __log:=/home/ubantu/.ros/log/d0e201e6-885e-11ef-8f8b-41555d88daad/mins_simulation0-2.log]. log file: /home/ubantu/.ros/log/d0e201e6-885e-11ef-8f8b-41555d88daad/mins_simulation0-2*.log Initiating shutdown! [mins_simulation0-2] killing on exit [rosout-1] killing on exit [master] killing on exit shutting down processing monitor... ... shutting down processing monitor complete done I wonder if the author can give some suggestions，thanks

Hi, @aidadgy. I suspect the error comes from LiDAR, the only place I use multi-thread for fast parallel computation. However, I cannot produce your error on my end, so it is hard to debug where the error is coming from. One thing I noticed is your Total traveling time: 48s. If you ran the default simulation, the total travel time should be 48 s, indicating that you may get your error while terminating the system at the end of the run. Could you provide more information about your problem? It will be helpful to debug the issue.

Dear author, I am glad to receive your reply and thank you for it. I put a screenshot of the last run terminating below as well, and I noticed when I was runningroslaunch mins rosbag.launch config:=kaist/kaist_LC path_gt:=urban30.txt path_bag:=urban30.bag， have the same termination problem。

I think the source code should be fine, but I also noticed that when I compiled the code with catkin build, a lot of warnings appeared, as follows：The first warning is related to Anaconda library, and the second one seems to be related to lidar. I don't know whether these warnings are related to program termination. I look forward to your reply, thank you.

[aidadgy]

Hello， When I ran roslaunch mins simulation.launch cam_enabled:=true lidar_enabled:=true, the following problems occurred: ========Final Status======== Total procesing time: 21s Total traveling time: 48s
RMSE average: 0.257, 0.117 (deg,m) NEES average: 7.133, 2.380 (deg,m) simulation: /usr/include/boost/thread/pthread/recursive_mutex.hpp:108: void boost::recursive_mutex::lock(): Assertion `!posix::pthread_mutex_lock(&m)' failed. ================================================================================REQUIRED process [mins_simulation0-2] has died! process has died [pid 1327580, exit code -6, cmd /home/ubantu/catkin_ws/devel/lib/mins/simulation __name:=mins_simulation0 __log:=/home/ubantu/.ros/log/d0e201e6-885e-11ef-8f8b-41555d88daad/mins_simulation0-2.log]. log file: /home/ubantu/.ros/log/d0e201e6-885e-11ef-8f8b-41555d88daad/mins_simulation0-2*.log Initiating shutdown! [mins_simulation0-2] killing on exit [rosout-1] killing on exit [master] killing on exit shutting down processing monitor... ... shutting down processing monitor complete done I wonder if the author can give some suggestions，thanks

Hi, @aidadgy. I suspect the error comes from LiDAR, the only place I use multi-thread for fast parallel computation. However, I cannot produce your error on my end, so it is hard to debug where the error is coming from. One thing I noticed is your Total traveling time: 48s. If you ran the default simulation, the total travel time should be 48 s, indicating that you may get your error while terminating the system at the end of the run. Could you provide more information about your problem? It will be helpful to debug the issue.
When I run roslaunch mins simulation.launch cam_enabled:=false lidar_enabled:=true，Result display：process has finished lwanly。

[aidadgy]

When I run roslaunch mins simulation.launch cam_enabled:=true lidar_enabled:=false, the same problem occurs when I change lidar to false，but When I run roslaunch mins simulation.launch cam_enabled:=false lidar_enabled:=true，Result display：process has finished lwanly，I suspect that the imu time stamp and the camera time stamp do not align, do you think this is the problem
In addition, I have a problem like you mentioned, when I set the parameter to debug in simulation.launch file, some messages are output as follows: