The robotic micromanipulation system is built on a motorized inverted microscope (Eclipse Ti-U, Nikon) as it is shown in Fig.1 (a).  Other main components include a pair of 3-DOF mechanical micromanipulators , an X-Y linear translational stage, a holding micropipette, a biopsy micropipette, a CMOS camera with maximum frame rate of 90 FPS, two motorized linear stages for controlling the microinjectors, and two multi-channel motion controllers. Extra rotational degrees of freedom may be added by installing rotary motors on the 3-DOF micromanipulators, which will be used to adjust the orientation of the micropipette. As it is shown in Fig.1. (a), there is a customized 4-DOF micromanipulator on the right-hand side, which is able to adjust the inclined angle that the micropipette is held. Its rotation resolution is around 0.0002 with the motor running at microsteps.

Fig.1(b) shows the control panel of the vision guided micromanipulation system. It enables the system to control the motion of the micromanipulators either in the teleoperation or the automated mode via the visual feedback acquired from the CMOS camera.

Firstly, an automatic micropipette alignment algorithm is proposed to precisely align the micropipette tip with the platform. It underpins the basis for the following micromanipulation operations. Assisted by the adaptive control algorithms, the micropipette tip can be aligned well with the platform with the vertical difference of 2.6 m along the 250 m length of the tip. The algorithm can also cope with the parameter uncertainties caused by the frequent replacement of the micropipette. [see in Fig. 2]

Fig. 1  Hardware and software of the vision guided robotic micromanipulation system. (a) System setup for the hardware part. (b) The control panel is built with PyQT. It shows a scenario of the embryo biopsy. An opening is created on the ZP of a 2.5 dpc (days post coitum) embryo by the integrated laser.

Fig. 2 The alignment result of the micropipette tip after the calibration and adjustment.  

Then a robotic micromanipulation method with the friction force -based cell rotation control is designed. The friction generated on the cell in contact with the micropipette and petri dish can be directly utilized to rotate cells. Robotic micromanipulation shall make the operation procedures stable and efficient in a deterministic way. For the successful trials, the rotation resolution can reach up to .

To further improve the robustness of the embryo rotation technique, fluid flow control is also introduced. Even for embryos that are not  perfect spheres or those whose mechanical properties may change a lot from one to one, utilizing the simultaneous perturbation stochastic approximation (SPSA) method, the system can robustly rotate embryos to the target position and make them ready for the coming biopsy operations.

All the control algorithms are aided by the visual information extracted from the microscopic images. It not only can track the positions of  embryos and  micropipette tip-ends, it can also outline the contour of the micropipette tip and the segment the inner cell structure of the embryo [see in Fig. 3].

Fig. 3 Intracellular structure segmentation of mouse embryos at different developmental stages.