FlyCore Recognition and Planning Computer Application Guide
Before you begin, confirm that the following conditions are met:
9 min read · English documentationPreparations Before Use
Before you begin, confirm that the following conditions are met:
Hardware Preparation
-
One Windows computer
-
Ethernet cable
-
FlyCore system
Network and Account Information
The recognition and planning computer uses the following address and credentials:
Recognition and planning computer IP: 192.168.1.66
Username: amov
Password: amov
Hardware Information
External hardware interfaces of the recognition and planning computer
The recognition and planning computer has two external hardware interfaces.
Interface ①: Primarily the external serial port of the recognition and planning computer, used mainly for serial communication with the flight controller.
Interface ②: Primarily the MIPI camera interface and serial interfaces for the gimbal.
Logging In to the RK System
Connecting the Development Board
Use an Ethernet cable to connect the development board to your host computer.
Logging In via SSH
Run the following command in a terminal:
ssh amov@192.168.1.66
Enter the password when prompted:
amov
After a successful login, you will enter the terminal of the recognition and planning computer.
Checks When SSH Login Fails
Check the following first:
Whether the host and the recognition and planning computer are on the same subnet
Whether the Ethernet cable is connected properly
Whether the target IP is reachable
Whether the recognition and planning computer has started normally
Whether the username or password has been changed
You can first run the following command on the host:
ping 192.168.1.66
PTP Time Synchronization
Purpose
This step configures the recognition and planning computer as a slave to synchronize its time with the mapping and localization onboard computer.
Command
Run the following command in the terminal of the recognition and planning computer:
sudo ptpd2 -s -i eth0 -C
Determining Success
If the host IP appears in the output, the PTP configuration has usually succeeded and the recognition and planning computer has begun synchronizing its time with the host.
Troubleshooting Suggestions
If synchronization fails, focus on checking the following:
Whether eth0 is the network interface actually in use
Whether the host is providing a PTP master clock
Whether the network link is functioning normally
Whether the device's time synchronization service is occupied or disrupted by another program
Video Streaming, Stream Playback, and Image Capture
First, use the MIPI interface to connect to interface ② shown above.
Implementing Streaming
Logging In to the Recognition and Planning Computer
First, log in to the recognition and planning computer via SSH:
ssh amov@192.168.1.66
Starting mediamtx
Enter the tool directory and start it:
cd Push_flow_Tool
./mediamtx &
Viewing Camera Device Numbers
Run:
v4l2-ctl --list-devices
Note: The /dev/videoX number may differ among device batches, camera modules, or system versions. Use the actual query result as the device number in subsequent commands.
Infrared Camera Streaming Example
gst-launch-1.0 v4l2src device=/dev/video20 io-mode=4 ! video/x-raw,width=1280,height=512 ! mpph265enc ! rtspclientsink protocols=udp latency=0 location=rtsp://127.0.0.1:8554/1
Visible-Light Camera Streaming Example (IMX586)
gst-launch-1.0 v4l2src device=/dev/video11 io-mode=4 ! video/x-raw,width=1920,height=1080 ! mpph265enc ! rtspclientsink protocols=udp latency=0 location=rtsp://127.0.0.1:8554/1
Notes:
The device=/dev/videoX values used for the infrared and visible-light cameras are not necessarily fixed.
By default, the stream is sent to the local mediamtx instance at:
rtsp://127.0.0.1:8554/1
Playing the Stream on the Host Computer
Use VLC on the host computer to open the network stream:
rtsp://192.168.1.66:8554/1
Notes:
4000x3000 is an example resolution.
Replace the device number with the actual camera node.
Camera Sample Programs
The original document includes OpenCV-based camera examples for the color camera and infrared camera. These examples capture a single-frame image and save it as a JPG file.
For this user application guide, you only need to understand how to run the examples; there is no need to read the source code first.
Color Camera Example
cd rgb
mkdir build
cd build
cmake ..
make
./rgb_camera_demo
After the program runs successfully, the following file will be generated in the current directory:
rgb_capture.jpg
Infrared Camera Example
cd ti
mkdir build
cd build
cmake ..
make
./ti_camera_demo
After the program runs successfully, the following file will be generated in the current directory:
ti_capture.jpg
Usage Notes
By default, the program opens a /dev/videoX device internally.
If the default device number is incorrect, pass the actual device number as a command-line argument at runtime.
The color camera example supports changing the resolution; the example sets it to a maximum of 4000x3000.
Serial Port Description
When connected to interface ②, the recognition and planning computer controls the gimbal through the following serial ports:
ttyS7
ttyS8
Users can select a gimbal solution according to their requirements and define their own communication protocol to control the gimbal through the serial port.
Communicating with the Flight Control Computer (BSA-PX4)
Flight controller configuration: First configure the correct baud rate and serial port number on the flight controller. For example, when using DDS for communication through the TELEM2 interface, set SER_TEL2_BAUD to 921600 in the QGC parameters.
Recognition and planning computer configuration and verification: In the terminal of the recognition and planning computer, set the baud rate for the corresponding serial port (for example, ttyS6): sudo stty -F /dev/ttyS6 921600
Purpose
View the data received over the serial port in hexadecimal format to confirm whether the recognition and planning computer can continuously receive MAVLink data sent by the flight controller.
Command
sudo cat /dev/ttyS6 | xxd
Normal Output
The terminal should continuously output hexadecimal data. Continuous data indicates that the serial link is connected. If there is no output, check the serial port number, baud rate, wiring, and flight controller port configuration.
Subsequent Development
After the link has been verified, users can communicate with the flight controller and perform secondary development through this serial port (/dev/ttyS6), using the DDS or MAVLink protocol to transmit localization, planning, and control data.
Obtaining Mapping and Localization Information
The mapping and localization onboard computer continuously sends map and localization information to the recognition and planning computer. Through UDP broadcast, the recognition and planning computer can obtain map and localization data by listening on port 8888. This data is used for UAV navigation and planning.
The following is ROS 2 sample code. It assumes that the ROS 2 packages are installed on the recognition and planning computer. If the ROS 2 packages are not installed by default, it is recommended to use AI (Cursor or Codex) to take over this onboard computer by logging in through SSH with a non-interactive public key. After establishing passwordless SSH access, the AI can automatically detect the libraries that the system requires and install the corresponding ROS 2 packages, greatly improving development efficiency.
The following uses Cursor as an example to explain how to use AI to take over this onboard computer.
1: Log in to the recognition and planning computer from Cursor's terminal.
2: Copy the local public key to the recognition and planning computer. If the local machine has no key, generate one first. AI can guide both non-interactive key generation and copying the public key, so these procedures are not described here. Ultimately, you can use passwordless SSH in the AI dialog to log in to the recognition and planning computer, obtain the highest system privileges, and automatically write code. After the product is released, close these SSH ports to avoid security risks.
Sample code:
#include <iostream>
#include <cstring>
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <unistd.h>
#include <cmath>
#include <pcl/point_cloud.h>
#include <pcl/point_types.h>
#include <pcl/io/pcd_io.h>
#include <pcl_conversions/pcl_conversions.h>
// #include "bsaslam2/udp/crcLib.h"
// #include "bsaslam2/utils/tic_toc.h"
// ROS2 includes
#include <rclcpp/rclcpp.hpp>
#include <sensor_msgs/msg/point_cloud2.hpp>
#include <geometry_msgs/msg/pose_stamped.hpp>
#include <nav_msgs/msg/odometry.hpp>
// Eigen includes
#include <Eigen/Dense>
#include <Eigen/Geometry>
typedef pcl::PointXYZINormal PointType;
typedef pcl::PointCloud<PointType> PointCloudXYZI;
// ROS 2 node class
class UDPPointCloudReceiver : public rclcpp::Node
{
public:
UDPPointCloudReceiver() : Node("udp_pointcloud_receiver"),uav_id_(1)
{
// Create the point cloud topic publisher
pointcloud_pub_ = this->create_publisher<sensor_msgs::msg::PointCloud2>(
"/uav" + std::to_string(uav_id_) + "/prometheus/local_points", 10);
// Create the pose topic publisher
pose_pub_ = this->create_publisher<geometry_msgs::msg::PoseStamped>(
"/uav" + std::to_string(uav_id_) + "/prometheus/pose_slam", 10);
// Create the odometry topic publisher
odom_pub_ = this->create_publisher<nav_msgs::msg::Odometry>(
"/uav" + std::to_string(uav_id_) + "/prometheus/odom_slam", 10);
RCLCPP_INFO(this->get_logger(), "UDP PointCloud Receiver started");
}
// Publish point cloud data
void publishPointCloud(const PointCloudXYZI::Ptr& cloud, double timestamp)
{
if (cloud->empty()) return;
sensor_msgs::msg::PointCloud2 cloud_msg;
pcl::toROSMsg(*cloud, cloud_msg);
// Set the message header
cloud_msg.header.stamp = this->now();
//cloud_msg.header.frame_id = "/world";
cloud_msg.header.frame_id = "/uav" + std::to_string(uav_id_) + "/lidar_link";
// Log the number of points in the point cloud
RCLCPP_INFO(this->get_logger(), "Publishing pointcloud with %zu points", cloud->size());
// Publish the point cloud
pointcloud_pub_->publish(cloud_msg);
}
// Publish pose data
void publishPose(const Eigen::Vector3d& pos, const Eigen::Quaterniond& quat, double timestamp)
{
geometry_msgs::msg::PoseStamped pose_msg;
pose_msg.header.stamp = this->now();
pose_msg.header.frame_id = "/uav" + std::to_string(uav_id_) + "/base_link";
pose_msg.pose.position.x = pos.x();
pose_msg.pose.position.y = pos.y();
pose_msg.pose.position.z = pos.z();
pose_msg.pose.orientation.x = quat.x();
pose_msg.pose.orientation.y = quat.y();
pose_msg.pose.orientation.z = quat.z();
pose_msg.pose.orientation.w = quat.w();
pose_pub_->publish(pose_msg);
}
// Publish odometry data
void publishOdometry(const Eigen::Vector3d& pos, const Eigen::Quaterniond& quat,
const Eigen::Vector3d& vel, double timestamp)
{
nav_msgs::msg::Odometry odom_msg;
odom_msg.header.stamp = this->now();
odom_msg.header.frame_id = "/world";
// Position
odom_msg.pose.pose.position.x = pos.x();
odom_msg.pose.pose.position.y = pos.y();
odom_msg.pose.pose.position.z = pos.z();
// Orientation
odom_msg.pose.pose.orientation.x = quat.x();
odom_msg.pose.pose.orientation.y = quat.y();
odom_msg.pose.pose.orientation.z = quat.z();
odom_msg.pose.pose.orientation.w = quat.w();
// Velocity
odom_msg.twist.twist.linear.x = vel.x();
odom_msg.twist.twist.linear.y = vel.y();
odom_msg.twist.twist.linear.z = vel.z();
odom_pub_->publish(odom_msg);
}
private:
rclcpp::Publisher<sensor_msgs::msg::PointCloud2>::SharedPtr pointcloud_pub_;
rclcpp::Publisher<geometry_msgs::msg::PoseStamped>::SharedPtr pose_pub_;
rclcpp::Publisher<nav_msgs::msg::Odometry>::SharedPtr odom_pub_;
int uav_id_;
};
const uint32_t tmp_Crc32Table[256] = {
0x00000000, 0x04C11DB7, 0x09823B6E, 0x0D4326D9, 0x130476DC, 0x17C56B6B,
0x1A864DB2, 0x1E475005, 0x2608EDB8, 0x22C9F00F, 0x2F8AD6D6, 0x2B4BCB61,
0x350C9B64, 0x31CD86D3, 0x3C8EA00A, 0x384FBDBD, 0x4C11DB70, 0x48D0C6C7,
0x4593E01E, 0x4152FDA9, 0x5F15ADAC, 0x5BD4B01B, 0x569796C2, 0x52568B75,
0x6A1936C8, 0x6ED82B7F, 0x639B0DA6, 0x675A1011, 0x791D4014, 0x7DDC5DA3,
0x709F7B7A, 0x745E66CD, 0x9823B6E0, 0x9CE2AB57, 0x91A18D8E, 0x95609039,
0x8B27C03C, 0x8FE6DD8B, 0x82A5FB52, 0x8664E6E5, 0xBE2B5B58, 0xBAEA46EF,
0xB7A96036, 0xB3687D81, 0xAD2F2D84, 0xA9EE3033, 0xA4AD16EA, 0xA06C0B5D,
0xD4326D90, 0xD0F37027, 0xDDB056FE, 0xD9714B49, 0xC7361B4C, 0xC3F706FB,
0xCEB42022, 0xCA753D95, 0xF23A8028, 0xF6FB9D9F, 0xFBB8BB46, 0xFF79A6F1,
0xE13EF6F4, 0xE5FFEB43, 0xE8BCCD9A, 0xEC7DD02D, 0x34867077, 0x30476DC0,
0x3D044B19, 0x39C556AE, 0x278206AB, 0x23431B1C, 0x2E003DC5, 0x2AC12072,
0x128E9DCF, 0x164F8078, 0x1B0CA6A1, 0x1FCDBB16, 0x018AEB13, 0x054BF6A4,
0x0808D07D, 0x0CC9CDCA, 0x7897AB07, 0x7C56B6B0, 0x71159069, 0x75D48DDE,
0x6B93DDDB, 0x6F52C06C, 0x6211E6B5, 0x66D0FB02, 0x5E9F46BF, 0x5A5E5B08,
0x571D7DD1, 0x53DC6066, 0x4D9B3063, 0x495A2DD4, 0x44190B0D, 0x40D816BA,
0xACA5C697, 0xA864DB20, 0xA527FDF9, 0xA1E6E04E, 0xBFA1B04B, 0xBB60ADFC,
0xB6238B25, 0xB2E29692, 0x8AAD2B2F, 0x8E6C3698, 0x832F1041, 0x87EE0DF6,
0x99A95DF3, 0x9D684044, 0x902B669D, 0x94EA7B2A, 0xE0B41DE7, 0xE4750050,
0xE9362689, 0xEDF73B3E, 0xF3B06B3B, 0xF771768C, 0xFA325055, 0xFEF34DE2,
0xC6BCF05F, 0xC27DEDE8, 0xCF3ECB31, 0xCBFFD686, 0xD5B88683, 0xD1799B34,
0xDC3ABDED, 0xD8FBA05A, 0x690CE0EE, 0x6DCDFD59, 0x608EDB80, 0x644FC637,
0x7A089632, 0x7EC98B85, 0x738AAD5C, 0x774BB0EB, 0x4F040D56, 0x4BC510E1,
0x46863638, 0x42472B8F, 0x5C007B8A, 0x58C1663D, 0x558240E4, 0x51435D53,
0x251D3B9E, 0x21DC2629, 0x2C9F00F0, 0x285E1D47, 0x36194D42, 0x32D850F5,
0x3F9B762C, 0x3B5A6B9B, 0x0315D626, 0x07D4CB91, 0x0A97ED48, 0x0E56F0FF,
0x1011A0FA, 0x14D0BD4D, 0x19939B94, 0x1D528623, 0xF12F560E, 0xF5EE4BB9,
0xF8AD6D60, 0xFC6C70D7, 0xE22B20D2, 0xE6EA3D65, 0xEBA91BBC, 0xEF68060B,
0xD727BBB6, 0xD3E6A601, 0xDEA580D8, 0xDA649D6F, 0xC423CD6A, 0xC0E2D0DD,
0xCDA1F604, 0xC960EBB3, 0xBD3E8D7E, 0xB9FF90C9, 0xB4BCB610, 0xB07DABA7,
0xAE3AFBA2, 0xAAFBE615, 0xA7B8C0CC, 0xA379DD7B, 0x9B3660C6, 0x9FF77D71,
0x92B45BA8, 0x9675461F, 0x8832161A, 0x8CF30BAD, 0x81B02D74, 0x857130C3,
0x5D8A9099, 0x594B8D2E, 0x5408ABF7, 0x50C9B640, 0x4E8EE645, 0x4A4FFBF2,
0x470CDD2B, 0x43CDC09C, 0x7B827D21, 0x7F436096, 0x7200464F, 0x76C15BF8,
0x68860BFD, 0x6C47164A, 0x61043093, 0x65C52D24, 0x119B4BE9, 0x155A565E,
0x18197087, 0x1CD86D30, 0x029F3D35, 0x065E2082, 0x0B1D065B, 0x0FDC1BEC,
0x3793A651, 0x3352BBE6, 0x3E119D3F, 0x3AD08088, 0x2497D08D, 0x2056CD3A,
0x2D15EBE3, 0x29D4F654, 0xC5A92679, 0xC1683BCE, 0xCC2B1D17, 0xC8EA00A0,
0xD6AD50A5, 0xD26C4D12, 0xDF2F6BCB, 0xDBEE767C, 0xE3A1CBC1, 0xE760D676,
0xEA23F0AF, 0xEEE2ED18, 0xF0A5BD1D, 0xF464A0AA, 0xF9278673, 0xFDE69BC4,
0x89B8FD09, 0x8D79E0BE, 0x803AC667, 0x84FBDBD0, 0x9ABC8BD5, 0x9E7D9662,
0x933EB0BB, 0x97FFAD0C, 0xAFB010B1, 0xAB710D06, 0xA6322BDF, 0xA2F33668,
0xBCB4666D, 0xB8757BDA, 0xB5365D03, 0xB1F740B4};
uint32_t tmp_checkCRC32(uint8_t *pData, uint32_t Length)
{
uint32_t nReg;
uint32_t nTemp = 0;
uint16_t i, n;
nReg = 0xFFFFFFFF;
for (n = 0; n < Length; n++)
{
nReg ^= (uint32_t)pData[n];
for (i = 0; i < 4; i++)
{
nTemp = tmp_Crc32Table[(uint8_t)((nReg >> 24) & 0xff)];
nReg <<= 8;
nReg ^= nTemp;
}
}
return nReg;
}
int main(int argc, char** argv)
{
// Initialize ROS 2
rclcpp::init(argc, argv);
// Create the ROS 2 node
auto node = std::make_shared<UDPPointCloudReceiver>();
// Create the UDP socket
int server_socket = socket(AF_INET, SOCK_DGRAM, 0);
if (server_socket == -1) {
RCLCPP_ERROR(node->get_logger(), "Failed to create socket.");
return -1;
}
// Bind the address and port
sockaddr_in server_address{};
server_address.sin_family = AF_INET;// IPv4 protocol
server_address.sin_port = htons(8888);// Port 8888
server_address.sin_addr.s_addr = INADDR_ANY;// Listen on all network interfaces
if (bind(server_socket, (struct sockaddr*)&server_address, sizeof(server_address)) == -1) {
RCLCPP_ERROR(node->get_logger(), "Failed to bind socket.");
close(server_socket);
return -1;
}
RCLCPP_INFO(node->get_logger(), "UDP server started on port 8888");
// TicToc t_pose, t_cloud;
try {
while (rclcpp::ok()) {
// Process ROS 2 callbacks
rclcpp::spin_some(node);
// Receive data
sockaddr_in client_address{};
socklen_t client_address_size = sizeof(client_address);
PointCloudXYZI::Ptr save_cloud(new PointCloudXYZI());
double timestamp = -1.0;// Timestamp; -1 indicates that it is uninitialized
bool last_bag = false; // Whether the final packet has been received
while(!last_bag)
{
uint8_t buffer[1223] = {0};// Data buffer
int points_received = recvfrom(server_socket, buffer, sizeof(buffer), 0, (struct sockaddr*)&client_address, &client_address_size);
if(points_received > 0)
{
// Check the frame header
bool bCorrupted = false;
if(*buffer != 0xEB || *(buffer+1) != 0x90)
{
bCorrupted = true;
char *response = "The bag is corrupted";
sendto(server_socket, response, strlen(response), 0, (struct sockaddr*)&client_address, client_address_size);
continue;
}
if(*(buffer+2) == 0xFF && *(buffer+3) == 0xFF)
{
uint64_t timestamp;
{
uint64_t *ptr = (uint64_t *)(buffer+4);// Timestamp
timestamp = *ptr;
}
double time_stamp = timestamp * 1e-9; // Convert to seconds
Eigen::Vector3d pos;// Position data
{
int32_t *ptr = (int32_t *)(buffer+12);
pos(0) = *ptr * 1e-6;
ptr = (int32_t *)(buffer+16);
pos(1) = *ptr * 1e-6;
ptr = (int32_t *)(buffer+20);
pos(2) = *ptr * 1e-6;
}
Eigen::Quaterniond quat; // Quaternion orientation
{
int32_t *ptr = (int32_t *)(buffer+24);
quat.x() = *ptr * 1e-6;
ptr = (int32_t *)(buffer+28);
quat.y() = *ptr * 1e-6;
ptr = (int32_t *)(buffer+32);
quat.z() = *ptr * 1e-6;
ptr = (int32_t *)(buffer+36);
quat.w() = *ptr * 1e-6;
}
Eigen::Vector3d velocity_linear;
{
int32_t *ptr = (int32_t *)(buffer+40);
velocity_linear(0) = *ptr * 1e-6;
ptr = (int32_t *)(buffer+44);
velocity_linear(1) = *ptr * 1e-6;
ptr = (int32_t *)(buffer+48);
velocity_linear(2) = *ptr * 1e-6;
}
unsigned int num_front, num_back;// Point count statistics
{
uint16_t *ptr = (uint16_t *)(buffer+52);
num_front = *ptr;
ptr = (uint16_t *)(buffer+54);
num_back = *ptr;
}
float re_project;// Reprojection error
{
uint32_t *ptr = (uint32_t *)(buffer+56);
re_project = (float)(*ptr)/1e6;
}
// Sensor status
uint8_t lidar_state = *(buffer+60);
uint8_t imu_state = *(buffer+61);
uint8_t cam_state = *(buffer+62);
// CRC check
uint32_t *crc = (uint32_t *)(buffer+63);// CRC checksum
uint32_t crc_compute = tmp_checkCRC32(buffer, 63);
if(*crc == crc_compute)
{
printf("timestamp: %.6f, front_num: %u, back_num: %u, reprojection: %.6f, lidar: %x, imu: %x, cam: %x\n",
time_stamp, num_front, num_back, re_project, lidar_state, imu_state, cam_state);
// Publish pose and odometry data to ROS 2 topics
node->publishPose(pos, quat, time_stamp);
node->publishOdometry(pos, quat, velocity_linear, time_stamp);
}
}
else
{
if(*(buffer+2) == 0x0B)
last_bag = true; // Mark this as the final packet
// Read the current packet length
uint16_t *bag_size = (uint16_t *)(buffer+4);
// If the point cloud size is not initialized, read the number of points in this frame
if(save_cloud->empty())
{
uint16_t *ptr = (uint16_t *)(buffer+6);
unsigned int point_num = (unsigned int)*ptr;
save_cloud->reserve(point_num);
}
// If the point cloud timestamp is not initialized, read the timestamp
if(timestamp == -1.0)
{
uint64_t *ptr = (uint64_t *)(buffer+8);
timestamp = *ptr * 1e-9;
}
// Read the number of points in the current packet
unsigned int bag_point_size;
{
uint16_t *ptr = (uint16_t *)(buffer+17);
bag_point_size = (unsigned int)*ptr;
}
// Parse the coordinates of each point
for(unsigned int i = 0; i < bag_point_size; i++)
{
int8_t *ptr = (int8_t *)(buffer+19+3*i);
int x = (int)*ptr;
ptr = (int8_t *)(buffer+19+3*i+1);
int y = (int)*ptr;
ptr = (int8_t *)(buffer+19+3*i+2);
int z = (int)*ptr;
PointType pt;
pt.x = (float)x * 5.0f * 1e-2;
pt.y = (float)y * 5.0f * 1e-2;
pt.z = (float)z * 5.0f * 1e-2; // Coordinate conversion: multiply by 5.0f * 1e-2 = 0.05 to convert integer coordinates to meters
pt.normal_x = 0.0f;// Initialize the normal vector
pt.normal_y = 0.0f;
pt.normal_z = 0.0f;
pt.intensity = 0.0f;// Initialize the intensity
save_cloud->points.push_back(pt);
}
// Read the CRC checksum
uint32_t *crc = (uint32_t *)(buffer+(*bag_size));
uint32_t crc_compute = tmp_checkCRC32(buffer, *bag_size);
}
}
else
{
char *response = "The bag is corrupted";
sendto(server_socket, response, strlen(response), 0, (struct sockaddr*)&client_address, client_address_size);
}
if(last_bag)
{
printf("\ncloud size: %ld\n===========\n", save_cloud->size());
// Publish point cloud data to the ROS 2 topic
if (!save_cloud->empty()) {
printf("Publishing pointcloud with %ld points\n", save_cloud->size());
node->publishPointCloud(save_cloud, timestamp);
} else {
printf("Warning: Point cloud is empty, not publishing\n");
}
}
}
}
}
catch (...) {
std::cerr << "An error occurred in the server loop." << std::endl;
}
// Clean up resources
close(server_socket);
rclcpp::shutdown();
return 0;
}
The code above receives point-cloud and position data. After it runs successfully, enter the ROS message list command in the terminal of the recognition and planning computer. The following topics will be displayed:
$ ros2 topic list
/parameter_events
/rosout
/uav1/prometheus/local_points
/uav1/prometheus/odom_slam
/uav1/prometheus/pose_slam
The system can display the point-cloud and position topics in real time. Printing the position information confirms that this data is exactly the same as the position information sent to the flight control computer. This position information fuses multisource sensor data from vision, LiDAR, IMU, RTK, and other sources. It serves as input for UAV planning and control and, together with the flight-controller communication interface, forms a complete foundation for control.
Updating the Recognition and Planning Computer System
This section describes how to update the recognition and planning computer and restore it to factory settings. It provides customers with an update method and allows them to reflash and restore the system for continued development after a system failure.
Installing the RK Driver
On the included USB flash drive, open the FlyCore planning computer development materials, select "Android and Linux Image Flashing Tools - RKDevTool and Drivers," and then open the folder shown below.
Use an extraction tool to extract DriverAssitant_v5.12.zip. In the extracted folder, locate and open the DriverInstall.exe executable.
After opening DriverInstall.exe, install the Rockchip driver as follows:
Click the "Install Driver" button.
After a short wait, a window will indicate that the driver was installed successfully. Click the "OK" button.
Installing the RK Download App
Next, extract RKDevTool_Release_v3.15.zip. This software does not require installation. Locate RKDevTool in the extracted folder and open it.
Downloading Firmware
After opening the RKDevTool flashing tool, the lower-left corner will display "No Devices Found" because the computer is not yet connected to the development board with a Micro_USB-Type_A cable.
Next, begin flashing the AMOV image to the recognition and planning computer.
(1) First, connect the development board to the Windows computer through Micro_USB-Type_A. The location of the development board's USB flashing port is shown below:
(2) Ensure that no other power source is connected to the development board.
(3) Press and hold the MaskROM button on the development board. Its location on the board is shown below:
(4) Connect power to the development board and power it on. You can then release the MaskROM button.
(5) If the preceding steps were successful, the development board will enter MASKROM mode and the flashing tool interface will display "Found One MASKROM Device."
(6) Select firmware upgrade.
(7) Click Firmware and select the firmware.
(8) Select the firmware.
(9) Click Upgrade and wait for the upgrade to complete.
Troubleshooting Common Issues
Unable to Connect via SSH
Possible causes:
Incorrect IP address
Abnormal Ethernet cable connection
The recognition and planning computer has not finished starting
The username or password has been changed
Run the following command first:
ping 192.168.1.66
Camera Device Is Not Visible
First run:
v4l2-ctl --list-devices
If the expected /dev/videoX device is not listed, check the following:
Whether the camera is connected properly
Whether the driver loaded normally
Whether the current image includes support for the corresponding camera
Stream Playback Fails
Check the following:
Whether mediamtx is running
Whether the streaming command is running
Whether the RTSP address is correct
Whether the host computer and the recognition and planning computer can communicate over the network
Whether device=/dev/videoX is correct
PTP Synchronization Fails
Check the following:
Whether the PTP master clock is enabled on the host
Whether the network interface on the RK side is eth0
Whether the network is directly connected or configured correctly
Whether another time synchronization service is causing a conflict
Recommended Sequence of Operations
For first-time use, follow this recommended sequence:
Prepare the tools, drivers, and firmware
Flash the firmware in Windows
Connect the recognition and planning computer to the host with an Ethernet cable
Log in to the system via SSH
Enable PTP time synchronization as needed
Start mediamtx
Query the camera device number
Stream video, play the stream, or capture images
Quick Reference for Key Commands
SSH Login
ssh amov@192.168.1.66
PTP Synchronization
sudo ptpd2 -s -i eth0 -C
View Camera Nodes
v4l2-ctl --list-devices
Start mediamtx
cd Push_flow_Tool
./mediamtx &
Infrared Streaming
gst-launch-1.0 v4l2src device=/dev/video20 io-mode=4 ! video/x-raw,width=1280,height=512 ! mpph265enc ! rtspclientsink protocols=udp latency=0 location=rtsp://127.0.0.1:8554/1
Visible-Light Streaming
gst-launch-1.0 v4l2src device=/dev/video11 io-mode=4 ! video/x-raw,width=1920,height=1080 ! mpph265enc ! rtspclientsink protocols=udp latency=0 location=rtsp://127.0.0.1:8554/1
