A new version of algorithm is based on Reinforcement Learning.
Let me demonstrate a new 3-stages algorithm of picking up objects.
1. At the first stage the robot make a picture of area ~5×5 inches to see if there is an object or part there. If a whole object is detected, the second stage can be skipped. The robot already knows the location and dimensions of the object.
2. If there is only part of an object in the first picture, on the 2nd stage robot moves his head to a position to see the detected corner on the opposite side next picture. If the the object still not fully fit the the image, a new attempt is made. Totally he does up to 3 attempts.
3. When the object fits to the image, the position and dimensions can be captured, robot can pick it up on the 3rd stage.
Let’s watch the video
Let me show an easy way I use to measure speed of robot’s movements. It is important to do the measurements with accuracy ~0.1sec to optimize delays between commands.
I use my camera, make video of the movements, and then convert the video file into set of pictures with the command like this:
ffmpeg -ss 00:00:04 -t 40 -i ../MVI_3219.MOV G_%04d.png
ffmpeg is a GNU licensed cross platform utility
-ss <position> – start position
-t <duration> – limit the duration of data read from the input file
-i <input_file> – the video file
G_%04d.png – is an example of C-printf template for the generated images. %04d is for substitution by frame number in format 0000, 0001, 0002, ….
The generated pictures are looked like this:
We can see that the transition lasts 7 frames.
With the displayed metadata, we can get the video frame rate:
Metadata: major_brand : qt minor_version : 537331968 compatible_brands: qt CAEP encoder : Lavf56.36.100 Stream #0:0(eng): Video: png, rgb24, 1920x1080, q=2-31, 200 kb/s, 23.98 fps, 23.98 tbn, 23.98 tbc (default)
It is 23.98 frames per second in this case (that is almost 24). So the transition lasts:
7f / 24fps = 0.29sec
Let’s talk about programming of robot behavior. Looks promising?
And you can start doing that spending only $8. Wow!
There is no upper limit for the price 😉 however you can buy 900 MHz Quard-Core 1Gb RAM microcomputer for less than $50 and control your robot with Linux applications!
Visit this Wikipedia page to see a long list of the boards you can use. Let’s consider what kind of brains your robot does really need.
First let’s figure out the difference between micro-controllers and microcomputers.
Microcomputers are really full functional computers, the most popular one is Raspberry Pi. Micro-controllers are boards that specifically designed to control devices and collect data from sensors, the most popular is Arduino family boards.
Let me summarize the key features you need to know to make the decision which kind of board to use. I will provide numbers for: Raspberry Pi 2 model B – the most powerful in Raspberry Pi family, Raspberry Pi A+ – a light version, Arduino Due – the most powerful micro-controller in Arduino family, Arduino Nano – a light version.
The perf difference is huge. If you need high perf tasks, like Computer Vision, complex logic, machine learning – there is no choice, you need microcomputer.
|Raspberry Pi 2 model B||900MHz||4||1GB||up to 64GB (see details)|
|Raspberry Pi A+||700MHz||1||256MB||up to 64GB (see details)|
|Arduino Due||84 MHz||1||96KB||512KB|
Microcomputer is a full functional computer with an operation system like Linux, Android, or Windows. You really don’t need another computer for the programming. You have the whole spectrum of programming languages and IDEs. You microcomputer can be connected to the Internet, and you can develop web applications as like.
Micro-controller will require another computer for its programming, and you may have only one specific C-like programming language. The programming itself will be lower level. However you will have a full control over the processor, so you can easier develop real-time applications, that is a problem on a computer under non real time operation system.
The micro-controller start up time is much faster. In some seconds your last uploaded sketch is running. The microcomputer operation system booting process takes some tens seconds, then you probably need to run your application.
The most of DIY applications are not so complex, and micro-computers capabilities are more than enough. You can find hundreds of robotics or smart-home projects on the Interned based on Arduino micro-controller.
Microcomputers usually have some USB ports, you can easily connect any number of USB devices (you need powered USB host if you need more ports). You can connect keyboard, mouse, USB WiFi, Bluetooth, IR port if they are not already on the board. And all these devises will be controlled by Linux. On micro-controllers you can extend the board with USB port, but that will require specific programming.
Microcomputers usually have network socket. Micro-controllers require a specific extension board for it.
Both microcomputers and micro-controllers have a number of digital General Purpose Input Output (GPIO) pins you can control external devices. But there is a difference, in micro-controllers (in Arduino at least) connectors are buffered, so if you do any mistake most probably you will not break the board. Microcomputer’s connectors, on other hand, can be directly connected to the chip (on Raspberry Pi at least).
Micro-controllers usually have a number of analog I/O pins that extends the applications and devices the board can directly communicate.
Power consumption, size and weight
|Raspberry Pi 2 model B||5V||4W||85.60 mm × 56.5 mm||45 g|
|Raspberry Pi A+||5V||1W||65 mm × 56.5 mm||23 g|
|Arduino Due||7-12V||~0.25W||101.52 mm x 53.3 mm||36 g|
|Arduino Nano||7-9V||~0.15W||45 mm x 18 mm||5 g|
|Raspberry Pi 2 model B||$42.25 + micro SD card|
|Raspberry Pi A+||$21 + micro SD card|
For me Computer Vision is one of the most exciting and important topics in robotics. Humans get 80% of all information about the world using their eyes. So the ability to collect visual information is a big advantage for any robot. The basic visual information our brain supplies to our mind is what objects are around and where they are. Let’s consider how robots do that.
Computer vision object detection consists of the following major steps:
2. Extracting points that are potentially related to the looking object. This robot applies a gradient filter to detect potential borders – the green dots on the picture.
3. Segmenting the extracted points. Removing small segments. Roughness of the surfaces also have its mini-borders on the image. The “real” object have a long solid border that helps to distinguish it.
4. Constructing objects from the extracted segments using specific patterns. The robot is interested only in objects he can pick up by his gripper, the objects should not be too small or too large. Also black gripper parts that are on the bottom right and left of the image should not be detected as objects.
6. Knowing the configuration of the robot’s neck the robot calculates position and direction of the camera, using that, he calculates the location of the detected objects on the floor. The accuracy of the detection by this robot is some millimeters!
Calculating the location of the objects the robot can easily locate his head to grab them.
Follow my posts, you will read the details of every step of this process.