外文翻譯-果樹采摘機(jī)器人及控制系統(tǒng)研制【文獻(xiàn)翻譯中英文】

外文翻譯-果樹采摘機(jī)器人及控制系統(tǒng)研制【文獻(xiàn)翻譯中英文】,文獻(xiàn)翻譯中英文,外文,翻譯,果樹,采摘,機(jī)器人,控制系統(tǒng),研制,文獻(xiàn),中英文 Chen XueFu Received in revised form Published online 6 August 2011 control strategy. The spoon-shaped end-effector with the pneumatic actuated gripper was requirements for harvesting apple. The harvesting robot autono- ent of hours, still depend on manual labour (Xu Sakai, Osuka, Maekawa, Sarig, 1993; Van Henten, Hemming, Van Tuijl, Kornet, Meuleman, 2002). In addition, several relevant studiesonagriculturalrobotsingreenhouseshavebeencarried * Corresponding author. Tel.: 86 511 82028322; fax: 86 511 82028322. Available at biosystems engineering 110 (2011) 112e122 E-mail address: (J. Wei). Harvesting is expected to be automated because the farming population is gradually decreasing in China. In addition, since the fruit trees are tall, harvesting work has to be con- ducted using step ladders, which makes manual harvesting dangerous and inefficient. Therefore, there is a strong desire Pellenc, Journeau, and Aldon (1987), developed an apple- harvesting robot. Since then, their pioneering studies were followed by many research papers covering several aspects (e.g., ;Edan, Rogozin, Flash, Foglia Hwang Kondo Muscato, and the continuous adjustment of planting structures, fruit cultivation areas, such as apple, citrus and pear, have reached8-9millionhasince1993,accountingforone-quarter of the total fruit cultivation area in the world. However, fruit harvesting tasks, which take 50%e70%ofthetotalworking instance, the machine needs to detect fruit, calculate the position of the fruit and then pick it without damaging the pericarp or the fruit tree. Research on fruit harvesting robots took place in the 1980s. Kawamura, Namikawa, Fujiura, and Ura (1984) first developed 1. Introduction In China, with the rapid developm 1537-5110/$ e see front matter Crown Copyright doi:10.1016/j.biosystemseng.2011.07.005 machine with radial basis function, the fruit recognition algorithm was developed to detect and locate the apple in the trees automatically. The control system, including industrial computer and AC servo driver, conducted the manipulator and the end-effector as it approached and picked the apples. The effectiveness of the prototype robot device was confirmed by laboratory tests and field experiments in an open field. The success rate of appleharvestingwas77%,andtheaverageharvestingtimewasapproximately15sperapple. Crown Copyright 2011 Published by Elsevier Ltd on behalf of IAgrE. All rights reserved. the rural economy harvesting, which is commonly used, requires sophisticated robotic technology. In short, it is necessary to design an intelligent robot withhuman-likeperceptive capabilities. For Accepted 17 July 2011 mouslyperformeditsharvestingtaskusingavision-basedmodule.Byusingasupportvector 4 July 2011 designed to satisfy the Article history: Received 9 February 2011 structurewasgeometricallyoptimisedtoprovidequasi-linearbehaviourandtosimplifythe Research Paper Design and control of an apple Zhao De-An, Lv Jidong, Ji Wei*, Zhang Ying, School of Electrical and Information Engineering, Jiangsu University, article info A robotic device consisting control system was developed journal homepage: www.elsevi 2011 Published by harvesting robot Yu Road No.301, Zhenjiang, Jiangsu Province 212013, PR China of a manipulator, end-effector and image-based vision servo for harvesting apple. The manipulator with 5 DOF PRRRP Elsevier Ltd on behalf of IAgrE. All rights reserved. biosystems engineering 110 (2011) 112e122 113 out; for instance, tomato harvesting (Monta et al., 1998), cucumber harvesting (Van Henten, Van Tuijl, Hemming, Kornet, Bontsema q 2 ;q 3 Joint angles of waist, major arm and minor arm. u, v Image plane coordinates horizontal and vertical axes u o , v o Image centre coordinate x g , y g Projection centre coordinate of target fruit ex, ey The difference of target fruit image feature between x g , y g and u o , v o M C2 N Image plane pixels of video camera jex max j;jey max j Maximum of ex and ey Dq 1 ; Dq 2 ; Dq 3 Jointdeviationanglesofwaist,majorarmand minor arm k 1 , k 2 Control parameters of arms furtherresearchanddevelopmenttoimprovetheperformance and reduce the initial set-up costs of these robots. Based on the concepts above, this study intends to develop and evaluate a competitive low price device for automatic harvesting, i.e., an apple-harvesting robot. Firstly, a detailed description on the components of the robot including the manipulator,theend-effectorandtheimage-basedvisionservo control system is described. Secondly, the geometrically opti- misation of the manipulator to gain a quasi-linear behaviour andsimplifythecontrol strategy isdescribed. Thirdly, theend- effectorwiththepneumaticactuatedgripperdesignedtosatisfy the requirements for harvesting apple is described. Based on this design, the harvesting robot autonomously performs its harvesting task using a vision-based module to detect and locate the apple in the trees, and control system conducts the manipulator and the end-effector to approach and pick apple. To verify the validity of the developed harvesting robot, the laboratory tests and field experiments in an open field were performed. The experimental results are the important contri- bution of this paper. The paper is organised as follows: in section 2 the main components of the robot are presented in detail, i.e., the manipulator, the end-effector and the image-based vision servo control system, respectively; in section 3 the experi- mental results are discussed to show the feasibility of the robot system proposed; finally, in section 4 conclusions are drawn and suggestions for future research are made. 2. Material and methods 2.1. Mechanical structure of apple harvesting robot A prototype model of the apple harvesting robot is designed forbothefficiencyandcosteffectiveness. Itmainlyconsistsof an autonomous vehicle, a 5 degree of freedom (DOF) manip- Dd The angle to adjust for the movement of a pixel with unit of degree per pixel. Abbreviations AC Alternating Current A/D Analog, Digital CCD Charge Coupled Devices D/A Digital, Analog DC Direct Current. DOF Degree of Freedom GPS Global Position System HIS Hue, Intensity, Saturation IBVS Image-Based Vision Servo PBVS Position-Based Vision Servo PRRRP Prismatic Revolute Revolute Revolute Prismatic RBF Radial Basis Function RST Rotation Scale, Translation SVM Support Vector Machine USB Universal Serial Bus VFW Video for Windows ulator, an end-effector, the sensors, the vision system and control system. The mechanical structure of fruit harvesting robot self-developed in this paper is shown in Fig. 1. 2.1.1. The autonomous mobile vehicle A crawler type mobile platform was selected as the mobile vehicle. It carried the power supplies, pneumatic pump, electronic hardware for data acquisition and control, and the manipulator with the end-effector for cutting the fruit. Global position system (GPS) technology was used for autonomous navigation of the mobile vehicle, whose typical speed was 1.5 ms C01 . 2.1.2. The manipulator Compared with other structures, as described in Sakai, Michihisa, Osuka, and Umeda (2008), joint structure is effec- tive for any position and orientation in three-dimensional space. The operation of a harvesting robot is a random large space distribution, where a lot of obstacles may exist around the robot. A joint manipulator with multi-degrees of freedom has an arbitrary curve fitting function. It is therefore easy to avoid obstacles by operating the corresponding joints when the end-effector reaches the object position. Therefore, a harvesting robot manipulator with 5 DOF prismatic- revolute-revolute-revolute-prismatic (PRRRP) structure to be mounted on autonomous mobile vehicle was designed. The first DOF was used for uplifting the whole manipulator. The of biosystems engineering 110 (2011) 112e122114 Fig. 1 e Schematic diagram middle three DOF were for rotation, among which, the second driving arm was designed to rotate around the waist, and the thirdandfourthoneswererotationaxesto movetheterminal operator up and down. This DOF allowed the end-effector to move towards an arbitrary direction in the work space. The fifth,andlast,DOFwasflexibleandusedforelongation,which made the end-effector reach the target location according to the robot control commands, thus achieving the harvesting of fruit (Zhao, Zhao, Zhao, Zhao, Position Sensor Pressure Sensor Vision Sensor Collision Sensor Table 1 e Motion parameters of manipulator mechanical structure. Joint Motion parameters Lift platform 0 me0.8 m Rotation joint of waist C0180 C14 e180 C14 Rotation joint of major arm C080 C14 e80 C14 Rotation joint of minor arm C080 C14 e80 C14 Flexible joint 0 me0.8 m biosystems engineering 110 (2011) 112e122 115 2.2. The sensors The non-structural and uncertain features of the operating environment, and the individual differences and random nature of the operating objects, determines that fruit har- vesting robots should have intelligent sensibility to their complexenvironment(Edanetal.,2000;Zhao,Zhao, Zhao, Zhao, Qiao, Wu, Liu, Zhang, Plebe Zhao, Yang, Mariottini, Oriolo Dd is the angle to be adjusted for the movement of a pixel with unit of degree per pixel. Then, the host computer sent instructions to the flexible jointtospread.Aftertheobjectfruitenteredintothegripperof end-effector, the flexible joint stopped spreading. The gripper wasthenclosedandtheelectricalcuttercutofftheapplestalk. Finally, the flexible joint backed to its initial position. Thereafter, the gripper was opened and fruit slid along the flexible tube into the basket. To achieve continuous picking the above steps were repeated. 2.4.4. System software design AWindowsXPsystemwasemployedasanoperatingplatform for its good stability and security. Visual C 6.0 was selected as programming development tool for the host computer. In the system, multiply tasks needed to be processed simulta- neously. Noting that a single-thread might lead to data communication jams and not guarantee real-time control, a multi-threading event-driven approach was adopted for the program control system software. The main thread was responsible for the management of visualisation control interface, system initialisation; the sub-thread was respon- sibleforcommunicationandsynchronisation.Thesub-thread biosystems engineering 110 (2011) 112e122 119 flexible joint contracted in the minor arm during the process of searching for target fruit. Therefore, the harvesting robot manipulator can be regarded as a three-joint robot manipu- lator,andtherelationshipbetweencameracoordinatesystem and robot coordinate system can be obtained according to geometrical relation shown in Fig. 9. The camera coordinates axes (X c ,Y c ,Z c ) parallel to corresponding axes in robot coor- dinates (X o ,Y o ,Z o ). L 1 , L 2 , L 3 are the lengths of the waist, major arm and minor arm respectively, and q 1 ;q 2 ;q 3 are the joint angles of the second, third and fourth DOF. Apples with radius of 40 mm (average radius of the apples) were considered as research objectives. Their projection was a circle on the image captured by video camera. Perspective projection of a fruit in 3-D space is shown in Fig. 10, and formed in the video camera. Feature information of target fruitin imageplaneis shownin Fig.11. Fora two-dimensional image captured by a video camera, the origin is a point in the upper right corner. Symbols of u and v denote horizontal and vertical axes respectively. The image feature of target fruit is characterize as ex and ey, which are the errors between projection centre coordinate (x g , y g ) and image centre coordi- nate (u o , v o ). During joint control of harvesting robot manip- ulator, image feature of ex varies along with the change of waist joint angle q 1 , and image feature of ey varies along with the change of major arm joint angles q 2 and minor arm joint angles q 3 . It can be seen that the manipulator with 5 DOF PRRRP mechanicalstructurewasgeometricallyoptimisedtosimplify the control strategy, and the control algorithm designed to avoid complicated jacobian operations. At the same time, the vision systems software gave only planar information of the target fruit in our robotic system. The distance information between target fruit and camera was unknown. Hence the manipulatorjacobiancouldnotbedirectlyusedinoursystem. The process of picking target fruit can be presented as follows. Firstly, each module of harvesting robot was ini- tialised, and the manipulator made to approachthe fruit trees at a proper location. Then the video obtained image infor- mation of target fruit, and the recognition and location were obtained by image processing software such that the centroid coordinate x g , y g of target in image and the errors ex and ey obtained by comparison with the image centre coordinate u o and v o . Secondly,therobotwascontrolledtomovewithsmallstep according to the calculated deviations ex and ey, and eventu- ally it drove them to be zero. Assuming that image plane pixels of video camera are M C2 N, then jex max jM=2 and jey max jN=2. The flowchart of the small step transformation algorithm is shown in Fig. 12. When the deviations of the smallstepmovementsofthewaist,majorarmandminorarm were zero, then the centroid of target fruit was coincident with image centre. During the process of eliminating devia- tions ex and ey, each joint angle was required to move. This was calculated according to Eq. (1) Dq 1 exC2Dd Dq 2 k 1 C2eyC2Dd Dq 3 k 2 C2eyC2Dd (1) where Dq 1 ; Dq 2 ; Dq 3 are joint angles of waist, major arm and minor arm respectively; k 1 ,k 2 are the control parameters of Fig. 13 e Main program flowchart of robot harvesting task. 3. Experiment results 3.1. Laboratory tests biosystems engineering 110 (2011) 112e122120 3.1.1. Recognition and Location experiment For the control system of the fruit harvesting robot, live video windowsonthecontrolsoftwareinterfacewasusedtodisplay the real-time process of picking. Target recognition windows In this section, the results of a feasibility study of the system performed throughlaboratorytestsalongwithfieldvalidation arepresented.Thelaboratoryexperimentswereperformedon the prototype operating in simulation working conditions. This stage was helpfulto set up and optimisethe components of our system. Finally, the performance of the harvesting robot was verified in field tests. system involved video capture, motion control, elongation test of flexible joint and extraction test of prism sub-threads. The main program flowchart for the fruit harvesting robot is shown in Fig. 13. Fig. 14 e Recognition and location showed the accuracy of target recognition, where red “” implied image centre and blue “” implied the centroid of the object fruit. The position of object fruit could be easily shown in the images. In the target location windows, the track the centroid of target fruit with regard to image centre during the location process was marked with a blue line. Table 2 e Dynamic images recognition time. Image frame 123456789101 Recognition time(ms) 235 235 315 390 315 310 390 390 310 390 390 Image frame 26 27 28 29 30 31 32 33 34 35 36 Recognition time(ms) 310 390 390 310 315 390 390 315 310 390 390 Image frame 51 52 53 54 55 56 57 58 59 60 61 Recognition time(ms) 315 390 395 310 390 390 310 315 390 390 315 Image frame 76 77 78 79 80 81 82 83 84 85 86 Recognition time(ms) 315 390 310 315 390 395 310 390 390 310 390 Recognition and location test results of object fruit can be seen in Fig. 14. It is obvious that in the figure, accurate recognition and smooth location track made following fruit grabbing possible, which verified that the designed robot has good tracking performance to meet the requirements of accurate real-time recognition and location. During the process of picking operations, video image signals needed to be acquired dynamically and continuously, and handled frame by frame. In the video, the size of one frame of dynamic images was 320 C2 240 pixels. The recogni- tion time for 100 continuous and dynamic images is shown in Table 2. From Table 2, the average recognition time of 100 frame images was 352 ms. From these results, it was concluded that the developed recognition algorithm met the requirements of real-timeoperationandthatthesystemcouldbeusedtoguide a robot manipulator as it approached an apple in real-time. 3.1.2. Harvesting experiments A photograph of the fruit harvesting robot operating during laboratory simulation harvesting tests is shown in Fig. 15. Underlaboratoryconditions,appleswithradiusabout40mm, were hung on fresh branches in different directions. The results in laboratory tests. periodofimageacquisitionwas100ms.100pickingtestswere carried out in 10 different positions. The test results were as follows: successful picking occa- sions 86, and failed occasions 14. Therefore the success rate was 86%. Without regard to the set-up time, the average time of picking one apple is 14.3 s. This was high enough to meet 121314151617181920212232425 310 390 310 235 315 390 390 310 390 390 390 315 390 395 37 38 39 40 41 42 43 44 45 46 47 48 49 50 310 390 390 310 390 390 310 315 390 315 310 390 390 390 62 63 64 65 66 67 68 69 70 71 72 73 74 75 310 390 390 310 390 390 310 390 390 310 315 390 395 310 87 88 89 90 91 92 93 94 95 96 97 98 99 100 390 310 315 390 390 310 315 390 390 315 390 390 315 390 not clamping tightly. After calculation, the mean recognition time for picking was 15.4 s and the picking success rate was biosystems engineering 110 (2011) 112e122 121 requirements of continuous harvesting operations. The main reasons for failure could be attributed to the experimental environment, where the soft foliage and apple vibration during operation resulted in a decrease in precision posi- tioning.Inaddition,occasionallythecuttingknifefailedtocut the apple stalk. 3.2. Field tests To further verify the reliability and adaptability of harvesting robot system, field tests were carried out in the Beijing Changping orchard during October 2009. The recognition result in the orchard is shown in Fig. 16. There 7 apples were well recognised, which indicated that the recognition algorithm could identify apples efficiently. Where apples are behind branches and leaves and apples cover each other, the apples cannot be picked directly. Those without label “”inFig. 16, wouldbe recognised after pickinga certain number of apples. Fig. 15 e Harvesting experiments in laboratory tests. In practice, once an image such as that in Fig. 16 was acquired, the vision system of robot located and picked the apple which had the minimum distance from the image centre of the visible-field of the camera. The vision system of robot located the next target fruit. Continuously picking Fig. 16 e Apple recognition results in an orchard. 77%, which indicates that the prototype machine and control system could be used to carry out the picking operation outdoors. 4. Conclusions and future research A self-developed fruit harvesting robot and its control system was developed. The main components of the robot, i.e., the manipulator, the end-effector and the image-based vision servo control system, have been described in detail. The experiments (shown in Fig. 17) were carried out in an orchard with a complex environment. In 10 min, 39 apples were rec- ognised, of which 30 apples were picked and put into the container successfully. Six apples failed to be picked since their image was blocked by branches. Three were picked but fell down to the ground due to their small size and the gripper Fig. 17 e Harvesting experiments in an orchard. manipulator was geometrically optimised to gain a quasi- linear behaviour and simplify the control strategy, and the end-effector with the pneumatic actuated gripper was designed to satisfy the requirements for the harvesting of apples. The harvesting robot autonomously performed its harvesting task using a vision-based module to detect and localise the apple in the trees, and c