How the robot sees the world

For me Computer Vision is one of the most exciting and important topics in robotics. Humans get 80% of all information about the world using their eyes. So the ability to collect visual information is a big advantage for any robot. The basic visual information our brain supplies to our mind is what objects are around and where they are. Let’s consider how robots do that.

Computer vision object detection consists of the following major steps:

test31. Image preprocessing – improve the quality of the image: reduce noise, adjust the light and contrast.

2. Extracting points that are potentially related to the looking object. This robot applies a gradient filter to detect potential borders – the green dots on the picture.

3. Segmenting the extracted points. Removing small segments. Roughness of the surfaces also have its mini-borders on the image. The “real” object have a long solid border that helps to distinguish it.

4. Constructing objects from the extracted segments using specific patterns. The robot is interested only in objects he can pick up by his gripper, the objects should not be too small or too large. Also black gripper parts that are on the bottom right and left of the image should not be detected as objects.

test45. When the objects are detected, the robot knows the coordinates on the image – these coordinates need to be translated into the angles – deviation from the main direction of the camera.

6. Knowing the configuration of the robot’s neck the robot calculates position and direction of the camera, using that, he calculates the location of the detected objects on the floor. The accuracy of the detection by this robot is some millimeters!

Calculating the location of the objects the robot can easily locate his head to grab them.

Follow my posts, you will read the details of every step of this process.

See you!



Welcome to my creative blog!

With this blog I am going to show my home projects, share some best practices and what I had been learned in robotics, electronics, artificial intelligence, astro-photo and shooting.