Pupil software controlling a mouse and a robotic arm

Pupil software controlling a mouse and a robotic arm


First 8 seconds are random blackness (happened after conversion) Calibrating Pupil software with “Natural Features” method. Note world cam mounted on the robotic arm Checking if calibration was sufficient Starting streaming the gaze co-ordinates on the tcp server by pressing “s” Starting up terminal to start the mouse control script Starting up script now, notice the mouse pointer moving in accordance with the gaze tracker Demonstrating the blink-to-click function A blink of about 1.5 seconds counts as a click (long blink to eliminate random clicks resulting from human reflex) This ubuntu install was setup to substitute single clicks for double clicks, meaning folders can open with a single click Demonstrating the clicking again with the eye window to make the blink to click more apparent. Ending the running mouse control script Starting other script in terminal for controlling the robotic arm Keep in mind it is a simple python script and there is no gui The user has to imagine that the screen is now divided into 6 equal square regions, blinking in one region activates the robotic arm action associated with that region a quick blink in another region stops the arm The 6 robotic arm movements we mapped to the 6 regions were up-down, left-right, and gripper open-close Changing to world cam feed to demonstrate optical feedback from the arm By only looking at the screen, the user is able to know where the arm is headed and control it, without having to look away from the screen Making it grab and lift a pen, attempt number 1… Attempt 1 failed, pen wasn’t placed in the grippers… Opening grippers for attempt no. 2 and attempt 2 is a success… The screen showing the user where the arm is headed LOL….clumsy

2 thoughts on “Pupil software controlling a mouse and a robotic arm

  • HELLO AGAIN, when you assembled it, which film did you use so the webcam could see the IR light ? and what exactly I have to remove from the camera to I can put the film on ?

Leave a Reply

Leave a Reply

Your email address will not be published. Required fields are marked *