ME 5286 Robotics: CNN-based robotic tool sorting

For our final lab in ME 5286 Robotics, we implemented an automated tool sorting procedure on a UR5 robotic arm. Tensorflow (keras) was used to design and train a convolutional neural network for identifying different types of tools. Corresponding bins for each tool were marked with unique april tags. A camera and Robotiq gripper were mounted on the end effector of the UR5. Starting with tools laid on a table, the UR5 used the pre-trained CNN to identify each tool. The gripper picked up each identified tool and placed it in its corresponding bin, referring to the unique april tag identifying the proper bin. Our procedure was implemented using RoboDK and its corresponding Python libraries, in conjunction with the Tensorflow and OpenCV Python libraries. A demonstration of the sorting process is shown below, along with still images from the UR5-mounted camera during the sorting process.