Reproducing Tianjin University's YOPO End-to-End Planning Algorithm in ProSim
Based on Tianjin University open-open-source YOPO end-to-end trajectory planning algorithm, we are on the SU17 developer engineering reproduction has been completed on the version. The SU17 developer version is oriented to R&D scenarios, has strong scalability and rich custom development interfaces, facilitates data collection, model training and joint debugging and verification, and seamlessly connects with the ProSim (PrometheusSim) simulation platform.
This article focuses on ProSim(PrometheusSim) platform provides straightforward training and simulation reproduction tutorials, laying a solid foundation for subsequent real machine deployment.
There are also links to papers and codes at the end of the article, allowing you to access them directly with one click.
Model training tutorial based on ProSim platform
01 preparation stage
Open wslgui, Prometheus ground control station, ProSim connection is successful
02 data collection
Ground control station unlocks for takeoff
ground control station operating depth obstacle avoidance
Enter wslgui and run the data collection script
In the directory in the picture, run conda activate YOPO to activate the environment
After activation, run python collection_realworld.py to start collecting data
The collection is completed as shown in the figure below and is saved in the yopo_simulation directory under the run directory:
After organizing the data according to the following structure, put it into the path shown in the figure below:
Things to note
Saved in the folder are img_*.tif pictures and label.npz labels;
Pointcloud-*.ply is the corresponding point cloud map;
The names of the collected pictures and point cloud maps should be one-to-one correspondence, such as: folder 0 contains the pictures and tags collected in Pointcloud-0.ply.
03 map collection
ground control station operates radar obstacle avoidance
Fly the points in Rviz and try to scan every corner to make the map complete. The scanned point cloud map is saved in pcd format in the following directory:
Point cloud map format conversion: pcd to ply
Place the scans.pcd file generated above into the utils directory under the run directory:
Open the terminal and run conda activate YOPO to activate the YOPO environment
Run python pointcloud_clip.py to start format conversion
The converted map will be displayed when the conversion is complete
The generated ply name is trans_scans.ply
Move it to the yopo_sim directory under the run directory and rename it to the corresponding pointcloud-*.ply
)
Now open the ply file and change the double in the first few lines to float.
Open the terminal in the run directory, run conda activate YOPO to activate the YOPO environment, and run python run_yopo.py to start training with your own data.Note that you need to install open3d for the first time, command: pip install open3d to install it.
Simulation based on ProSim platform
01 Environment deployment
Establish YOPO's python environment, python version 3.8
conda create -n yopo python=3.8
Install pytorch
pip install torch==2.4.1 torchvision==0.19.1 torchaudio==2.4.1 —index-url
download:https://download.pytorch.org/whl/cu118
Install opencv-python
pip install opencv-python
Install gym, stable-baseline3
pip install gym stable-baselines3
Install arithmetic package
pip install scipy==1.10.1 scikit-build==0.18.1 ruamel-yaml==0.17.21 numpy==1.22.3 tensorboard==2.8.0 empy catkin_pkg
02 Build flightlib
Go to the flightlib/build directory and run the command: cmake..
The correct configuration is as shown below:
Start compiling
make -j4
Solve the error
Solution:
Instructions: pip install pyyaml
Run make -j4 again to compile
03 Install flightgym module
Run the following command, and be careful not to delete the setup.py file in the build directory by mistake.
pip install .
04 Configure flightpolicy
Run the following command to refresh the environment variables. After refreshing, enter the base environment. Pay attention to reactivating the YOPO environment.
echo "export PYTHONPATH=$PYTHONPATH:~/YOPO" >> ~/.bashrc
source ~/.bashrc
05 planning
Go to the run directory to run the py program and start planning.
python test_yopo_ros_local.py
Solve the error
Solution:
Command: pip install rospkg
Run the py program again
06 ProSim simulation
Open the ground control station and emulator and establish a connection
Enter the run directory and run the command
bash yopo.sh
The successful operation is as follows:
Unlock and take off at the ground control station to start testing.
In typical scenarios such as forests, white pillars, and warehouses on the ProSim simulation platform, YOPO end-to-end planning enables the quadcopter to stably complete obstacle avoidance flight. After the model is optimized and deployed by TensorRT, the inference speed of the model is improved. It can be seen that the inference speed before and after the model conversion has been improved.3 times improvement.
In the test environment, by flying to a point in Rviz, the UAV only uses the depth map obtained by the depth camera as well as the speed and acceleration as model input, and finally obtains a feasible motion trajectory.
Resource Express
and wiki link
Past recurrence tutorials:
No need to envy Tesla FSD, you can also reproduce visual end-to-end on UAV | Ubuntu reproduction tutorial
Paper analysis:
Tianjin University end-to-end planning algorithm open-source! Autonomous flight no longer lags/recurrence video attached
Paper link:
https://ieeexplore.ieee.org/document/10528860
open-source code:
