融合视觉与力的机器人反馈控制开题报告

1. 研究目的与意义（文献综述包含参考文献）

Introduction: The topic that I will be working on is a scientific research project, mainly to design and realize the shaft hole assembly simulation of vision and force feedback of the joint virtual simulation platform CoppeliaSim and Python. Shaft can be defined as it is a member which can fit into another part with proper assembly of the part. A shaft can either rotate or be stationary. The shaft will be applied to a member which has to fit even if there is a restriction between two spaces.Hole can be defined as a member who can house or fit the shaft. A hole can also rotate or be stationary. A hole can be applied to space between two restrictions in which a member has to fit. If an assembly is made of two parts whether both of them are similar or not similar, one part is known as a male part (outer elements of the part), and the other part is known as female part (inner element of the part) surface. The female part is known as the hole and the male part is known as the shaft.Here, my work is to assemble shaft and hole; and I have to design its simulation of vision force feedback of the joint virtual simulation platform CoppeliaSim and Python. The subject needs to design a type of manipulator shaft hole assembly simulation system based on vision-force feedback. The assembly process is divided into three stages: shaft hole coarse positioning, shaft hole fine positioning, and position adjustment in the hole, combined with monocular vision sensors and six The Weili sensor senses external environment information and workpiece information, and finally adopts the admittance feedback control strategy to adjust the posture of the robot end. This topic describes the construction of virtual multi-sensor data acquisition, processing and control strategies, which enables the realization of high-precision shaft hole assembly experiments. Based on actual needs, examine students' independent design ability, analysis ability and practical ability of complex engineering.Figure: Cylindrical Shaft in Hole Assembly(a) The grasp of a shaft. (c) The axis pose estimation for the hole. (c) The smooth admittance of the shaft into the hole.Research design and planning: Im going to have to use CoppeliaSim; formerly known as V-REP, is a robot simulator used in industry, education and research. I will use it in order to build shaft hole assembly simulation and then I will use Python programming to perform the simulation. I have to use vision sensors. There is an example given:Fig: Using Vision Sensor Fig: Python Robot Simulation using CoppeliaSim It focuses on the problem of Robot control based on vision and force fusing, especially for the task of problem solving. Propose effective algorithms based on low-rank matrix recovery for image restoration. It is a study on the Shaft hole assembly problem and learn about the state-of-the-art methods in this area. It is based on existing methods, develop a Robot control based on vision and force fusing simulation. It performs experiments using the developed algorithm, compare it with existing algorithms. My proposal: For the whole project I used simulation of Shaft Hole Assembly Robot using CoppeliaSim and Pycharm for Python coding; It is going to be a simulation of Vision and Force feedback.Its going to follow shaft hole coarse positioning, shaft hole fine positioning, position adjustment in the hole combined with monocular vision sensors and six the Weili sensor which senses external environment information and workpiece information and finally adopts the admittance feedback control strategy to adjust the posture of the robot end. It is a novel coarse-to-fine method for object pose estimation coupled with admittance control to promote robotic shaft-in-hole assembly. Considering that traditional approaches to locate the hole by force sensing are time-consuming, I employ 3D vision to estimate the axis pose of the hole. Thus, robots can locate the target hole in both position and orientation and enable the shaft to move into the hole along the axis orientation. In this method, first, the raw point cloud of a hole is processed to acquire the keypoints. Then, a coarse axis is extracted according to the geometric constraints between the surface normals and axis. Lastly, axis refinement is performed on the coarse axis to achieve higher precision. Practical experiments verified the effectiveness of the axis pose estimation. The assembly strategy composed of axis pose estimation and admittance control was effectively applied to the robotic shaft-in-hole assembly.

2. 研究的基本内容、问题解决措施及方案

文 献 综 述Introduction: The topic that I will be working on is a scientific research project, mainly to design and realize the shaft hole assembly simulation of vision and force feedback of the joint virtual simulation platform CoppeliaSim and Python. Shaft can be defined as it is a member which can fit into another part with proper assembly of the part. A shaft can either rotate or be stationary. The shaft will be applied to a member which has to fit even if there is a restriction between two spaces.Hole can be defined as a member who can house or fit the shaft. A hole can also rotate or be stationary. A hole can be applied to space between two restrictions in which a member has to fit. If an assembly is made of two parts whether both of them are similar or not similar, one part is known as a male part (outer elements of the part), and the other part is known as female part (inner element of the part) surface. The female part is known as the hole and the male part is known as the shaft.Here, my work is to assemble shaft and hole; and I have to design its simulation of vision force feedback of the joint virtual simulation platform CoppeliaSim and Python. The subject needs to design a type of manipulator shaft hole assembly simulation system based on vision-force feedback. The assembly process is divided into three stages: shaft hole coarse positioning, shaft hole fine positioning, and position adjustment in the hole, combined with monocular vision sensors and six The Weili sensor senses external environment information and workpiece information, and finally adopts the admittance feedback control strategy to adjust the posture of the robot end. This topic describes the construction of virtual multi-sensor data acquisition, processing and control strategies, which enables the realization of high-precision shaft hole assembly experiments. Based on actual needs, examine students' independent design ability, analysis ability and practical ability of complex engineering.Figure: Cylindrical Shaft in Hole Assembly(a) The grasp of a shaft. (c) The axis pose estimation for the hole. (c) The smooth admittance of the shaft into the hole.Research design and planning: Im going to have to use CoppeliaSim; formerly known as V-REP, is a robot simulator used in industry, education and research. I will use it in order to build shaft hole assembly simulation and then I will use Python programming to perform the simulation. I have to use vision sensors. There is an example given:Fig: Using Vision Sensor Fig: Python Robot Simulation using CoppeliaSim It focuses on the problem of Robot control based on vision and force fusing, especially for the task of problem solving. Propose effective algorithms based on low-rank matrix recovery for image restoration. It is a study on the Shaft hole assembly problem and learn about the state-of-the-art methods in this area. It is based on existing methods, develop a Robot control based on vision and force fusing simulation. It performs experiments using the developed algorithm, compare it with existing algorithms. My proposal: For the whole project I used simulation of Shaft Hole Assembly Robot using CoppeliaSim and Pycharm for Python coding; It is going to be a simulation of Vision and Force feedback.Its going to follow shaft hole coarse positioning, shaft hole fine positioning, position adjustment in the hole combined with monocular vision sensors and six the Weili sensor which senses external environment information and workpiece information and finally adopts the admittance feedback control strategy to adjust the posture of the robot end. It is a novel coarse-to-fine method for object pose estimation coupled with admittance control to promote robotic shaft-in-hole assembly. Considering that traditional approaches to locate the hole by force sensing are time-consuming, I employ 3D vision to estimate the axis pose of the hole. Thus, robots can locate the target hole in both position and orientation and enable the shaft to move into the hole along the axis orientation. In this method, first, the raw point cloud of a hole is processed to acquire the keypoints. Then, a coarse axis is extracted according to the geometric constraints between the surface normals and axis. Lastly, axis refinement is performed on the coarse axis to achieve higher precision. Practical experiments verified the effectiveness of the axis pose estimation. The assembly strategy composed of axis pose estimation and admittance control was effectively applied to the robotic shaft-in-hole assembly. References: [1]Wang Z, Fu T, Yang X, et al. A Hierarchical Force Guided Robot Assembly Method Using Contact State Model[C]//International Conference on Intelligent Robotics and Applications. Springer, Cham, 2021: 186-197.[2]Deng W, Zhang C, Zou Z, et al. Peg-in-hole assembly of industrial robots based on Object detection and Admittance force control[C]//2021 36th Youth Academic Annual Conference of Chinese Association of Automation (YAC). IEEE, 2021: 672-677.[3]Wang J, Jiang Y, Lin S, et al. Geometric model-based joint angle selection criterion for force parameter identification there is a lot of works to do. For the beginning I just downloaded CoppeliaSim software and getting to know it. I have to use CoppeliaSim to simulate the Robot. Firstly I will use Vision sensor for UR5 robot or any robot. Then using CoppoliaSim I will I have to give force control assembly to the particular UR5 robot or the one I will be using. In that case the force control will showcase the work that the robot is capable of doing; while the vision control will visualize just the way I have shown in the picture upward.The programming language I have to use for the simulation is Python. Only Python will be used in this process of experiment. As I have already knowledge about Python; thanks to the GOOGLE IT Automation course. Using that knowledge I am going to use Python to simulate the Robot. Vision sensor will visualize the whole simulation from position x,y,z axis and orientation a,b,c. The simulation time will be around 50.0 ms. I will finish the whole experiment with all the conditions that I have given, with using 3 positioning status that was mentioned I will complete the experiment. For force control I can do Peg-in hole or shaft hole.In the background of precision peg-in-hole assembly of industrial robot, the problem of unknown assembly target position is solved by object detection and the position error calculation method and admittance force control are used together to solve the position error caused by vision control, improve the success rate of assembly and accelerate the assembly speed.Working flow for the project:[Three stages of assembly process]