Avans Smart Assistant

Aka, A SmartAss:

This last semester I studied the minor “Embedded Systems” on my university, in this minor you learn a lot about most of the thing I count as my hobby; Electronics and programming. The minor takes a half a year and you have to do a certain project that takes about a half a year. The project I and a friend chose was a bit different, but a really cool one; The Avans Smart Assistant!The project was initiated by our mentor, but this was because there was a school challenge written out by the HKU, the university of art in Utrecht, the Netherlands. There is also a facebook page with some more information about the challenge but I’ll keep it short. The challenge is called Robotmatch 2012 and is held on sept. 11 2012. You have to create a robot which purpose is to be a student assistant and help student keep track off their planning and it has to be interactive and humanoid. There where not much restrictions on the match so one could build a really cool robot.

For us, making a robot is off course really cool but we wanted to make it also useful, so that it wont become a project that gets into the trash can after. So we asked the department of Social studies what could be done to make the robot more useful.  They came up with the idea to help, in particular, autistic students at our university. Social studies did a study on this subject so they could help us designing the robot. So we did. In our case the robot has to be able to print the agenda of the person which asks for it, and help them plan their activities. Besides that it also has to be able to print out their school schedule for that day or week, by voice command. Another feature is that the robot has to be able to talk back, follow the person in front, can be driven by a Wiimote, assist at a presentation with its beamer, have a Skype chat and a lot more…

Below is a sketch of what the robot was going to be like and look like;

So the main parts of the robot will be;

• Pan-tilt unit
• Kinect 3d camera
• VeeaR – SmartVR-DK PRO
• Pioneer 3-DX robotic platform
• Touch screen display
• Pico projector beamer
• PC
• Printer

These parts are the base of the robot, all these peripherals have to be connected to the PC which is the main core of the robot. Below is another impression of the robot, in a bit more color full drawing.

So this is what we have done!;

The frame

At the picture you can see the frame on which all the parts are going to be connected into one robot, we were luckily enough to have connections (actually the university had these connections), to let the stainless steel frame made at a company called CSI, for free! They lasered, bend and drilled all the parts and welded them together, super! The big box at the bottom is the box for the battery’s which make the robot cordless. This whole frame if build upon the robotic platform called Pioneer 3-DX which is a common used base platform for mobile robotic applications. Below some more pictures of the frame;

Beamer and Webcam

The beamer and webcam are the “head” of the robot, this “head” must be able to look around and follow people with its webcam. This can be used for Skype for example. We used a pretty heavy Pan-Tilt unit on which we made a aluminum plate with holes to connect the little beamer and webcam. The pan-tilt unit it controlled by a microcontroller (Arduino) which is connected via serial connection to the PC. Here are some more pictures of the “head” and pan-tilt unit;

Kinect 3d camera

The Kinect is a very cheap 3d camera with image processing in it, it can be connected to a computer and it has its own SDK developing suite. This makes it easy to use in an other application as using it as game controller. We wanted to use the Kinect for tracking and recognizing people, so we did this in combination with the pan tilt unit. The “eyes” of the robot (the webcam and beamer) follows the person who is in front of the robot. The Kinect is connected to the squire pipe with a custom made part I printed on my 3d printer 🙂 The part is based on this design on Thingiverse. Below some more pictures;

Voice module

Voice recognition is a great feature that this robot has, it makes sure the student easily can use the functions the robot has, like printing their agenda or schedule for that day, or ask them about certain information.

The Tigal Voice GP DK-T2SI voice recognition module we used is a module that can recognize voice commands but also speak, this is a feature which is very cool about it since the user can get feedback in a normal voice. The development board we got is heavily featured with a lot of peripherals. The module is actually a big microcontroller which you can program with voice recognition. Besides the recognition a lot of GPIO and peripherals like a I2C, PWM (or DAC) are available. This way you can also control servo motors with it or make analog voltages. Another great feature is the SD-card socket, the board has a SPI interface peripheral so you can, for example, play music files from the card which can be used for speech!

At this point I want to thank Tigal for their awesome robotic and embedded products and for letting me review their Voice GP development kit. This was, for us, in our robot the base of the voice recognition and speech synthesis.

Combine all the parts

The previous chapters where more about the individual parts, when these parts where done we had (almost) a complete robot, at least the mechanical and electrical part. The last things we added where the printer and the touch screen. When these where connected we could get to program the whole robot, which is always the biggest part on embedded/robotic projects. With the peripherals the robot consists of, there was and still is a lot of programming to do to make the robot do exactly what we want.

At the time writing this article the robot can do almost all the tasks specified, and even more, much more since other students are on it right now. This projects is definitely not finished yet since we only have made a robot which can work, but it doesn’t have a body yet, only the mechanical part and electronic part and a bit of the programming has been done. This is why other students of the department of electronics, programming (TI), mechatronics and arts are working on the robot right now! Really cool! this way the robot gets cooler and cooler and will become a worthy opponent to other university’s at the RobotMatch 2012!

Unfortunately I am not at liberty to tell all the features and progress the robot has made as well as showing video’s or sharing code because of the challenge which has not taken place yet, but I hope you enjoy reading about it already.
Update: Here’s a small video we made from a presentation given some time ago. It is still in development but these are some small results;