Documentation/P600 UAV WikiEnglish · Allspark2
Browse documentation
Additional documentation

GX40 Target Click-to-Track

Familiarity with outdoor position hold flight is required first. The following demonstration uses Windows.

5 min read · English documentation
  • Familiarity with outdoor position-hold flight is required first.
  • The following demonstration uses Windows.
Note:
Due to the publication date of this tutorial and upgrades to product features and code:
1. The ground station version may differ from the version shown in this tutorial.
2. The names and order of custom demos in the function scripts may differ slightly.
3. The following tutorial assumes that the communication link, MID360, and Prometheus ground station are ready.
4. Special note:
  • For operations using MID360 positioning, testing this function indoors is not recommended due to the risk of a crash. A site of at least 50x50 m is recommended.

Check Whether the UAV Status Is Normal

Start MID360 Positioning

  • Connect to the UAV in Connection Settings (the positioning source must be switched to MID360 positioning).
  1. Under Function Scripts, click P600_MID360 Positioning.
  • The onboard command is: /home/amov/p600_experiment/src/p600_experiment/scripts/mid360_location.sh
  1. The status bar displays the current UAV status (whether the UAV flight controller is connected, the current flight mode, control status, MID360 positioning status, whether the UAV is armed, etc.).
  2. A red Prometheus status indicator means that the status is abnormal. The figure below shows a normal status.
  3. The x, y, and z positions should be around 0, indicating that the positioning data is normal.

6d1590aa0c1f2846597b3862f29ac84.png

Check the Data

  1. It is normal for the initial position data to be close to 0.
  2. An excessive error in the initial position data, greater than 0.5, is abnormal (usually caused by an initial data-fusion error or accumulated error after multiple flights).

MID360 positioning is normal

image.png

Note:
1. The following two scripts, ekf2 stop and ekf2 start, reinitialize EKF2 fusion for the flight controller's position data when the initial data is abnormal or the error is excessive under MID360 positioning.
2. Run these scripts while checking the data before takeoff. Do not click or use them during flight, as doing so may cause loss of positioning and a crash. (This will be updated later to prevent accidental activation; refer to the official release notes for the applicable behavior.)

Initial Data

  1. Click Function Scripts in the ground station.
  2. Click ekf2 stop.
  3. Click ekf2 start.

image.png

Start the Function Demo

  1. Click Function Scripts.
  2. Click GX40 Gimbal Visual Tracking.
  • The onboard command is: /home/amov/p600_experiment/src/p600_experiment/scripts/gimbalx_detection_with_tracking.sh
  1. A message confirming that the corresponding function demo has started appears.

image.png

  1. After the function demo runs, the list of ROS nodes currently running on the onboard computer is displayed.
  2. The current CPU usage and CPU temperature are displayed.

image.png

Retrieve the GX40 Gimbal Video

View the Stream URL

  1. Click System Settings.
  2. Click Streaming Settings.
  3. The default video stream URL is: rtsp://192.168.144.60:8554/live
    • The default configuration can be used.
Description

Display the Image in Video Monitoring

  1. Click Main Interface.
  2. Click Video Monitoring.
  3. Click Play to retrieve the video stream.
  4. The video stream is retrieved successfully.

image.png

Take Off

  1. Move SW1 to the bottom position to arm the UAV. SW2, SW3, and SW4 should be in the top position at this time.
  2. The current Prometheus control status is INIT, which corresponds to the flight controller's POSITION mode.
  3. The UAV video is normal, and general-object recognition begins.

image.png

  • Move SW2 to the middle position.
  • The current Prometheus control status is RC_POS_CONTROL, which is the flight controller's OFFBOARD mode.

image.png

  1. Move SW2 to the bottom position. SW3 and SW4 should be in the top position at this time.
  2. The current Prometheus control status is COMMAND_CONTROL, which is also the flight controller's OFFBOARD mode.
  • The UAV will hover at its current position.

image.png

Note:
1. After entering Prometheus-controlled COMMAND_CONTROL mode, the UAV responds to control commands from the onboard computer. You do not need to operate the remote controller at this time and can set it aside while continuing to monitor the flight.
2. If the UAV status becomes abnormal, take manual control with the remote controller, move SW2 to the top position, and land the UAV manually.

Run the GX40 Gimbal Target Click-to-Track Function

  1. Click Click.
  2. Recognized targets are enclosed in boxes and labeled person. (The system is in recognition mode at this point and has not started tracking.)

image.png

  • Click a recognized target. Its label changes to ID:1. The UAV then enters tracking mode, moves toward the target, and maintains the default distance from it.

image.png

  • When the target moves, the UAV tracks it while maintaining a certain distance.

image.png

Cancel the Gimbal Target Click-to-Track Function

  1. Select Click.
  2. Right-click in the video to cancel tracking. The UAV will hover at its current position.

image.png

  • After tracking is canceled, the recognition program continues to run normally. You can click another recognized target to track it.

image.png

Land the UAV

Land Using Prometheus Control

Landing Method 1: Send a Landing Command from the Prometheus Ground Station

  1. The UAV must currently be in COMMAND_CONTROL.
  2. Click Data Monitoring.
  3. Click Land.

image.png

Landing Method 2: Send a Landing Command with the H16 Remote Controller

  • The UAV must currently be in COMMAND_CONTROL with a normal status. Move SW4 to the bottom position to perform a Prometheus-controlled landing.

Landing Method 3: Land Manually

  1. Move SW2 to the top position to enter flight-controller-controlled POSITION mode. The UAV now responds to the remote controller.
  • Slowly move the left stick all the way down to control the UAV's landing manually.
  • After the UAV lands, wait for it to disarm automatically, or move SW3 to the bottom position to stop the propellers with one switch action.

image.png

Gimbal Control

  1. Click Manual Control.
  2. Double-click Stop to recenter the gimbal.

image.png

Handling Abnormal UAV States

False Recognition While Tracking

  1. Select Click.
  2. Right-click in the video to cancel tracking, and then click the target to track it again.

image.png

Function Parameters

  • The script used by this function is:
`/home/amov/p600_experiment/src/p600_experiment/scripts/gimbalx_detection_with_tracking.sh`
  • The script contents are as follows:
gnome-terminal --window -e 'bash -c "roslaunch p600_experiment gimbalx_server.launch; exec bash"' \
--tab -e 'bash -c "sleep 2; roslaunch p600_experiment gimbalx_detection_with_tracking.launch; exec bash"' \
--tab -e 'bash -c "sleep 4; roslaunch p600_experiment gimbalx_yolov5_tracking.launch; exec bash"' \

Gimbal Driver

  • The contents of gimbalx_server.launch are as follows:
  • File path:

/home/amov/p600_experiment/src/p600_experiment/launch_spirecv/gimbalx_server.launch

<launch>
    <node pkg="spirecv_ros" type="gimbal_server" name="gimbal_server" output="screen">
        <param name="Type" type="string" value="GX40" />
        <param name="Port" type="string" value="/dev/ttyTHS1" />
        <param name="camera" type="string" value="192.168.144.108" />
        <param name="FPS" type="int" value="30" />
        <param name="H" type="int" value="720" />
        <param name="W" type="int" value="1280" />
    </node>
</launch>

Gimbal Tracking

  • The contents of gimbalx_detection_with_tracking.launch are as follows:
  • File path:

/home/amov/p600_experiment/src/p600_experiment/launch_spirecv/gimbalx_detection_with_tracking.launch

<launch>
    <node pkg="spirecv_ros" type="gimbal_detection_with_tracking" name="gimbal_detection_with_tracking" output="screen">
        <param name="uav_id" type="int" value="1" />
        <param name="camera_parameters" type="string" value="$(find spirecv_ros)/camera_params/calib_webcam_1280x720.yaml" />
        <param name="local_saving_path" type="string" value="$(find spirecv_ros)" />
        <param name="input_image_topic" type="string" value="/uav1/gimbal/image_original" />
	    <param name="output_image_topic" type="string" value="/uav1/spirecv/image_det" />
    </node>
</launch>

UAV Tracking

  • The contents of gimbalx_yolov5_tracking.launch are as follows:
  • File path:

/home/amov/p600_experiment/src/p600_experiment/launch_spirecv/gimbalx_yolov5_tracking.launch

<launch>
    <node pkg="p600_mission" type="gimbal_yolov5_tracking" name="gimbal_yolov5_tracking" output="screen">
		
		<param name="uav_id" value="1" />
		
            	<param name="kp_x" type="double" value="0.3" />
            	
            	<param name="kp_z" type="double" value="0.1" />
            	
            	<param name="kp_gimbal" type="double" value="1.0" />
            	
            	<param name="max_velocity" type="double" value="2.5" />
            	
            	<param name="max_yaw_rate" type="double" value="40" />
            	
            	<param name="tracking_distance" type="double" value="10.0" />	
	</node>
</launch>