The exploration of extraterrestrial objects is an important way of space science and technology innovation, which is also one of the development priorities in the current and future aerospace field. Surface mobile exploration of the extraterrestrial objects is an effective way to expand the exploration breadth and depth. In the future, the demand for surface exploration range and efficiency of lunar and Mars exploration missions will be significantly improved. The exploration range will be expanded from kilometers to more than 100 km, and the moving speed per hour will be increased from 100 m level to more than one kilometer level. It is required that extraterrestrial mobile probes have stronger adaptability to extraterrestrial environment and higher exploration efficiency. One of the key factors determining the ability of extraterrestrial surface exploration is the ability of autonomous environment perception, manipulation and control. Facing the needs of major engineering tasks of deep space exploration in China in the future, this paper summarizes and analyzes the development status and trend of autonomous perception, manipulation and control technologies of extraterrestrial surface exploration unmanned system.

The unmanned system in interplanetary exploration has the characteristics of limited storage, computing power, energy and so on. The perception, localization and decision making algorithms based on deep neural network can effectively improve the intelligence level, but these algorithms generally require huge computing power, which is difficult to be directly applied to unmanned systems. Therefore, this paper reviews the existing lightweight methods including pruning and quantization, and makes a quantitative analysis on public dataset. Furthermore, this paper proposes pruning and quantization solutions, establishes a lightweight computing software StarLight, realizes rapid lightweight and evaluation of deep neural network, and solves the problem that the deep model is difficult to be directly applied to resource constrained systems. Finally, based on StarLight, various models used in the Mars rover are compressed, and deployed in the embedded platform; under the premise of ensuring performance, the power≤15 W, CPU frequency≤1.2 GHz and storage≤1 TB. Experiments show that the software can meet the lightweight requirements of resource constrained systems, and builds a foundation for further improving the intelligent level of unmanned systems for interplanetary exploration.

In extraterrestrial detection, multi sensor data synchronous acquisition is necessary. A novel multi sensor hardware time synchronization method without GPS (global positioning system) is proposed to solve the problem that the GPS signal cannot be achieved in extraterrestrial detection. The embedded MCU (micro control unit) is employed as the core of hardware system, so that a high precision synchronous acquisition can be realized. A method combining simulated GPS time service, PPS (pulse per second) clock correction and pulse trigger from MCU is employed to get the time synchronization between multi camera IMU (inertial measurement unit) and LIDAR. Finally, a hardware platform is established in an explorer and some new proof schemes are designed. The experimental results validate the effectiveness of the proposed method. The synchronous precision is about 2 ms, which is significantly improved compared with the existing extraterrestrial detector. This method can be used as a reference for the data acquisition, autonomous navigation and obstacle avoidance of the next generation of extraterrestrial detector.

Target detection technology is widely used in transportation, medical treatment, security, aerospace and other fields. At present, target detection technology faces challenges such as weak target, complex background and blocked target. Aiming at the problems of low detection and recognition rate and high false recognition rate of small rock and stone targets in the images collected from unstructured simulated terrain of star catalog, the YOLOv5 target detection algorithm with the best effect and lightweight model is studied. On the basis of it, a rock target detection algorithm meeting the real time requirements is improved via improving the optimizer and optimizing the repeated detection effect of detection frame. In particular, the feature description ability and classification accuracy of YOLOv5 network model are improved by introducing empty labels as negative sample, combining SGD optimization model and nonmaximum suppression parameter adjustment method. Using complex terrain images collected in ground test sites as data sets, and using mAP, FPS and PR curves as performance indexes, the proposed target detection network is verified by experiments. The experimental results show that the improved network proposed in this paper has higher accuracy and lower false reputation rate, while maintaining the real time performance of the original algorithm.

Due to the problem of the original natural partial surface of Mars, the single color and the high texture similarity, it is difficult to achieve accurate positioning of the binoculars. We propose a stereo matching method of Martian partial surface image fused with depth information. The spatial pyramid feature extraction module is used to aggregate context information of different scales and locations, and then a multi scale matching cost volume is constructed through a hierarchical stereo matching architecture, and the batch normalization layer is replaced by conditional cost volume normalization. In the cost regularization stage of the stereo matching network, the depth information is used as the condition to modulate the cost volume features, thereby reducing the amount of calculation, improving the inference speed, and generating a high precision disparity map. Finally, combining the disparity value of the target and the camera parameters, the three dimensional coordinates of the target point are obtained in the specified coordinate system to realize the positioning task. The disparity map on the Mars simulation dataset achieves a three pixel error of less than 0.017%, and the comparison with GCNet+CCVNorm and other methods shows the advantages of the proposed method in the Martian partial surface scene.

Accurate positional estimation and real time perception of obstacles in the surrounding environment are the basis for autonomous roving on the Mars. However, the Mars rover is limited by its own weight, volume, energy supply and other factors. The computational resources and device power are severely constrained, which poses a challenge to the design and implementation of the perception system. In this paper, we design an intelligent perception system based on vision inertial multi sensor filter fusion to address the problem of severely limited computing resources of Mars rovers. The system includes two main modules. A visual inertial odometry estimation module based on MSCKF (multi state constrained Kalman filter) algorithm achieves a high localization accuracy with relative error less than 1.5% An elevation map construction algorithm using GPU acceleration achieves real time construction of dense terrain maps. The elevation map and robot poses are fused in a probability optimal way to ensure the probability consistency of the perception system. Compared with existing perception systems, the proposed method can achieve the robot’s pose estimation and the construction of the elevation map of the surrounding environment only using binocular vision and IMU, and finally achieve 400 Hz poses’ output frequency and 4.2 Hz maps’ output frequency in 30W SoC through the rational algorithm design and GPU hardware acceleration.

3D bin packing problem is a combinatorial optimization problem that needs packing a certain number of objects and maximizing the volume utilization under the constraints of volume limit and stability limit. 3D packing problem is a NP hard problem. Heuristic algorithm is usually used to find the best position to place the object. When robot is used for packing, manipulation uncertainties should be handled. For example, the collisions between the manipulator and the surroundings, and the planning errors of the manipulator motion trajectories may make some optimal poses infeasible. Thus, the object can only be dropped from a higher place or placed near the optimal pose. The uncertainties of robot in grasping, recognition and placing also lead to the error between the real object position and the planned one. Therefore, an optimization method for robot 3D packing via pushing is proposed based on deep reinforcement learning. Aiming at minimizing the score of the heuristic algorithm for the positions of objects in the bin, robot can reorganize the positions of placed objects via pushing. Meanwhile, the objects are compressed towards a corner to make more space and improve the volume utilization rate of packing.

In the Mars exploration mission, in order to ensure that the rover can obtain large scale surface information, the drone group carried by the rover needs to be able to effectively cover the designated area. In this paper, a coverage control method is proposed based on multi agent deep reinforcement learning that aims to maximize the coverage object area and subject to drone communication net and energy efficiency. By adopting the CRITIC parameter sharing mechanism, the training efficiency is improved. The parameter permute invariant property is obtained by utilizing the graph net. The simulation results show that the algorithm proposed in this paper is better than two baseline methods in terms of coverage area, energy efficiency, and connectivity maintenance.

The development of GANs (generative adversarial networks) has brought great progress to many traditional fields including image generation. By training GANs with a small number of samples, the features of specific image categories can be learned, and then the generated samples can be applied to a variety of tasks such as scene testing and training other networks. This work explores the automatic generation of Mars image samples from limited training datasets. A cooperative clustering training generating method for Mars sample generation is proposed based on progressive growing GANs, which alleviates the problems that the features of samples are not obvious, and the types of features are insufficient when directly adopting GANs to sample generation. Our results show that the generated samples are improved compared with the baseline original progressive growing GANs.

The visual simulation of the Mars surface environment is very important for the demonstration and verification of Mars exploration scheme. In this paper, according to the real Martian scene data, a 3D visual model of the Martian surface environment is designed and constructed based on unreal Engine 4 digital simulation software, which meets the requirement of modeling the Martian surface digital environment. The designed simulation model not only includes sand slope, sand pit, hills and other terrain, but also adds lighting, sand and dust conditions, which makes the model closer to the real environment and meets the requirements of visual effect and authenticity.