YOPO End-to-End Real-Drone Reproduction Tutorial: SU17 Developer Edition Quick-Start Guide
Using the SU17 Developer Edition (including the Orin optional package), we completed an engineering reproduction of Tianjin University's YOPO end-to-end planning algorithm and compile a set of practical tutorials from "zero to run-through".
YOPO does not rely on online mapping, coupling perception, trajectory search and optimization into the same network, and a single forward inference can provide safe and feasible trajectories; with the help of Guidance Learning training strategy, millisecond-level latency can be achieved in simulation and real machine testing.
This tutorial is applicable to the SU17 developer version (including the Orin optional package); in order to reduce the cost of getting started, we will launch the corresponding function installation package in the future, which can be used out of the box and quickly deployed.
01 Prepare and connect
Use NoMachine to connect the SU17-Orin optional package through the same LAN and enter the desktop.
Install conda; put the YOPO project in the directory shown in the figure below.
02 Create and activate Python environment
Establish a YOPO environment through conda (can be connected to a wired network or USB shared network)
conda create -n yopo python=3.8
Activate YOPO environment
conda activate yopo
03 Install PyTorch
View CUDA version
nvcc -V
The Orin option package provided with the SU17 Developer Edition is based on the NVIDIA ARM architecture, so NVIDIA's official PyTorch that adapts to Jetson Orin NX needs to be installed.
Official website link: PyTorch for Jetson - Jetson & Embedded Systems / Announcements - NVIDIA Developer Forums
You need to install the Orin downloaded in the previous step in the corresponding YOPO environment.
pip install torch-2.1.0a0+41361538.nv23.06-cp38-cp38-linux_aarch64.whl
04 Install basic dependencies
Install OpenCV-python
pip install opencv-python
Install gym, stable-baseline3
pip install gym stable-baselines3
Install arithmetic package
pip install scipy==1.10.1 scikit-build==0.18.1 ruamel-yaml==0.17.21 numpy==1.22.3 tensorboard==2.8.0 empy catkin_pkg
05 Building and compiling flightlib
Go to the flightlib/build directory and run the command: cmake..
The construction is completed as shown in the figure:
Start compiling
make -j4
Solve the error
Solution:
Instructions: pip install pyyaml
Run make -j4 again to compile
06 Install flightgym
# Run in the directory containing setup.py (the docs recommend the build directory; do not delete setup.py by mistake)
pip install .
07 Install flightpolicy
echo "export PYTHONPATH=$PYTHONPATH:~/YOPO" >> ~/.bashrc
source ~/.bashrc
(Used to refresh environment variables, enter the base environment after refreshing, remember to reactivate the YOPO environment)
08 Run YOPO
Go to the run directory and run the py program to start planning.
python test_yopo_ros_new.py
Solve the error
Solution:
Command: pip install rospkg
Run the py program again
python test_yopo_ros_new.py means it runs successfully
09 RVizVisualization
# Load the preset RViz configuration from the YOPO root directory
rviz -d yopo.rviz
If there is no image at this time, it means that the camera and depth map compression module have not been turned on.
10 Depth map link up
Use the ground control station to connect to the aircraft. As shown in the figure below, the connection is successful.
Start four-view localization; right-click on RViz to switch the localization source and display valid That is, the switch is successful and the camera is turned on;
Enable depth map compression
Start the depth map decoder on Orin:
# In the workspace that contains devel
source devel/setup.bash
roslaunch su17_experiment su17_rvl_decoder.launch
Back to RViz: the depth map should be displayed in the lower left corner, and the trajectory planned by YOPO is on the right.
11 Field test
-
test_yopo_ros_new.py will continue to send control commands; the flight controller will not execute when the aircraft is not unlocked.
-
It is recommended to terminate the script by pressing Ctrl+C on the corresponding terminal after completing the visual inspection on the ground.
- After arriving at the test site, repeat the process according to Sections 8–10, and finally run:
python test_yopo_ros_new.py
12 Related configuration items
Modify the maximum flight speed in the traj_opt.yaml file in the YOPO/flightlib/configs directory.
Resource Express
Past recurrence tutorials:
Tianjin University YOPO end-to-end planning algorithm recurrence tutorial | ProSim article
No need to envy Tesla FSD, you can also reproduce visual end-to-end on UAV | Ubuntu reproduction tutorial
Paper analysis:
Tianjin University end-to-end planning algorithm open-source! Autonomous flight no longer lags/recurrence video attached
Paper link:
open-source code:
https://github.com/TJU-Aerial-Robotics/YOPO
