GoodRobot Blog

I wanted to resolve the Arduino crashes problem, that occurred after my arduino ethernet code for goodrobot became slightly huge (>10K). It used to hang/crash at arbitrary places and removing some code – here and there – seems to make it work, only to create problems somewhere else. After some debugging to make sure, no memory was being overwritten, due to faulty coding, I realized the following.

Arduino has very limited RAM, and thus was causing severe problems, when you exceed that. The worst problem is that we don’t know, that we are overwriting  heap and stack. So read these URLs:

http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1213583720/all

http://www.arduino.cc/playground/Code/AvailableMemory

http://www.arduino.cc/en/Reference/PROGMEM

This blog is based on an email exchange, discussing this topic and edited portion is given below:

First, the architecture. There are Features, each having multiple “data tags”.  Each Feature is of type ‘contr ol’ or ‘resource’ and hence all its data tags are also of the same type. One ore more features form together a “component”.  Currently, there is a limitation that all features within a component have to be of the same type (control or resource).  In addition, the component names are fixed – “control”, “control2″, “control3″ (all are type “control”), and “resource”, “resource2″, “resource3″ (type ‘resource’) and so on.

When any data comes for component, say ‘control’, it is directed to ‘resource’ and vice-versa. Similarly data from ‘control2′ is sent to ‘resource2′ and so on. This is hard mapping. I could have created another configuration mapping and let the admin configure it, but there were already two more mappings – and I did not want to complicate things further.(Update: This is however implemented now, an additional Routing Controls that allows the admin to route data from any component to any component)

A hardware device, such as RobotClient or Arduino, identifies itself as the component name (such as ‘control’, ‘resource’, ‘control2′) to the server, while connecting using the API. You might have noticed that, while updating arduino text “PUT /api/Toronto3.control” – and that key “control” in that identifies it as component name “control”. This would tell the server, it is coming from ‘control’ named component and it knows what features are configured in this, and what tags to expect, what data massaging needs to be done and where to redirect the data. etc.

Also, other than some minor details (which I listed in earlier mail and produced below), the server does not “semantically” differentiate between control/resource data tags. The only thing it bothers about is which data tag of ‘control’ should be mapped (sent) to which data tag of ‘resource’. The actual differences (three golden differences) are:

• We check for the readiness of the ‘resource’, before ‘control’ can send data and allow only one control to access a ‘resource’ at a given time (but this does not make sense for sensor data, which has ‘input’ direction).
• All ‘resource’ data are candidates for historic storage – while it is not  currently for ‘control’ data.
• What you said about last reading to retrieve is true for any ‘resource data (whether that data is of direction “output” — such as feedback from motor controls, or it has the direction “input” — such as data coming from sensors). The same thing is not true for ‘control’ data, which is deleted after being passed to the resource.

But is robot sensor data (e.g. distance reading from a sensor) handled differently than the control data sent by the arduino (e.g. when it sends joystick commands)?  Because it seems that the arduino can both receive data as a resource (since it received commands that could drive the robot), yet the arduino can also send sensor data (in the case where I used the analog pins to send joystick control signals). So on the arduino, it seems that both are possible. Or am I mistaken that it is possible to do both simultaneously. E.g. that the same arduino could not both send out commands based on analog readings yet also be a “resource” that receives data to act upon?

The above conclusions are correct. We did not attempt to do both simultaneously. It seems that the architecture does not support a hardware device to act as both control as well as resource at the same time. However, we can overcome that by make it pose as “control” or “resource” component – by appropriately placing the component name in the PUT request (along with passing the correct secret key as well). So an arduino can send its robot motors data tags as being sent from ‘resource’ component and send its joystick data as being sent from ‘control2′ component. And whoever needs to pick up its data needs to specify their component names as “control” and “resource2″ respectively (either a single hardware device or two different hardware devices). Notice, if you pose yourself as “resource” and as well as “control” component names, you would simply get all the data back (works as a loopback!) – which we should avoid).

I see, this is very interesting. I like it because one of the things it would allow us to do is to have a computer attached to the internet which could control a robot entirely (instead of a human), but is not on the robot itself. So for example, ff my robot has an arduino (call it RobotArduino) can “poses” as both control and resource it could send certain data (like compass information or distance measures) to a computer that is the artificially intelligent AI controller (call it ComputerAI), that is a resource, but also able to pose as a control too. This ComputerAI receive the data readings sent out as control data from the RobotArduino and then make decisions about what actions to take and then the ComputerAI could also send a control signal back to the RobotArduino to tell it what to do. This way, the “brains” of a robot could be located anywhere in the world.

Ok, for the reason above, it might be nice to eventually have a robot capable of acting as both “control” and “resource”, so that an intelligent control loop (or brain) for the robot could be located elsewhere. Today, the only option is to have the “brain” and decisions be located in the robot. Eventually, it would be very nice if we might be able to develop smart control algorithms ourselves that operate over a network and could give remote robots various behaviours (e.g. “explore”, “sentry/guard”, “play” etc).  This has been in the back of my mind for awhile as something for future phases. But I guess allowing a robot to pose as a control and resource at the same time (even if it is just adjusting the PUT and secret key lines in the code) would be a first step toward that. Maybe that could be a priority to make control/resource in one machine for the next phase of work.

Let us extend this further. Some possible scenarios are given and notes on how to implement.

• One-to-One: (One ‘control’ device to one ‘resource’ device). Such as web-to-arduino. There are only two devices and and data will simply be exchanged with each other. So it does not matter, even if you send joystick data (which we take it to mean as “control” data) as part of the resources data from arduino. The server will simply pass the data to the web, with the corresponding mapping. (We just need to make sure the Feature which defines the joystick control does not enable ‘data storage’ for historic storage purposes, unless we want it so).

• Multi-to-One: Multiple ‘control’ devices to one ‘resource’ device. Such as web, twitter to arduino. In both cases, web as well as twitter pose itself as component name “control”. And arduino will pose itself as component name ‘resource’. Both web and twitter can control the arduino resource – but if it is done at the same time, resource locking ability will take care to allow only one at the same time. In addition, the web or twitter (or pachube etc) can send the same data (like speed/direction) or web can send one set of data and twitter can send another set of data (which resource accepts) and it should work. But again not at the same time.
• One-to-Multi: One control device to multiple ‘resource’ devices (all posing as ‘control’ or ‘resource’). This will NOT work as expected, as the server would be sending some ‘control’ commands to one resource and some other to another resource, depending on which resource connects to the server and gets the data at that moment.

• Multi-to-Multi: Multiple control devices to multiple resource devices. This will only work to expected results, if each devices poses itself correctly as ‘control’, ‘control2′, ‘resource’, ‘resource2′ as its component name. They will be routed appropriately. They will follow the rules of One-to-One and Multi-to-One.
  • As an example, take the case of web, arduino (driving a motor as well as joystick) and phidgets to robotic arms (controlled by arduino joystick).  It will all work as expected, if web poses as ‘control’, arduino poses as ‘resource’ (for motors) and ‘control2′ (for joystick), while the phidgets will say itself as ‘resource2′. There are only three devices, but posing themselves as four endpoints. And they will all be routed appropriately.

So, I need to make changes to arduino, RobotClient and as well web to ensure that all of these can act as both control and resource. In the case of web, I can accordingly pose as “control” and “control2″ — so that the data from web can be sent to multiple resources (specifically multiple resource devices).

Update: All the above todo changes are implemented as of now.

The objective was to provide virtual joystick controls to drive a robot or a car, that works like a “tank”.

Tank drive relies only on two wheels, left and right, to drive forward, backward, left or right. These left and right wheels can move independently of the other and hence this is also known as differential drive, where the speed difference between its left and right wheels determines the turn and how much to turn.

A joystick control allows one to control the movements in forward, backward, left or right. A virtual joystick simulates a joystick on a web page, by the use of mouse and/or keyboard. For all practical purposes, this is similar to a XY graph or what is known as rectangular or cartesian co-ordinate system.

As I mentioned above, the need was to map the joystick controls to a tank-drive movements. Alan Majer gave the initial inputs for the mapping, via this simple and excellent chart that depicts the mapping values for boundary conditions. Thanks Alan!

Tank drive joystick controls

In simple terms, this meant the XY values available from the graph needs to be converted to left and right, the values appropriately mapped. This post describes the logic needed to do the mapping, for every value of x and y. In the following description, x and y indicates the XY values and left, right indicates the to-be-mapped tank drive values.

This conversion must have been pretty obvious or easy, but it was not for me Sigh! After few retries, I got the mapping done and here is how one would go about it.

Before we go further, here are some more details on tank drive.

• To move the tank forward/backward, keep both the left and right motors at the same speed.
• To turn right at 90 deg, you will move the left motor at 100% speed and the right motor at 100% reverse speed. Visualize how the tank would just rotate about itself in the right direction.
• Also, if you intend to go to the NE direction, then you would keep the left motor at some speed and keep the right motor off.

First, a few fundamentals to internalize. After having looked at the image, you will realize the following aspects:

• As with any joystick control, x represents “turn” and y represents forward/backward movements. In other words, the user who controls the joystick (or the virtual joystick) varies x to indicate turns and y to indicate movements.
• You may remember this is a differential drive, so if there is no difference between left and right motors, the tank/robot moves forward/backward without any turn. Further, if the difference between left and right are maximum, then the robot just rotates about itself without any forward/backward movement.
• Translated here, if x is zero(0) then there is no difference between left and right values, it would just follow y. If y is zero(o), then the difference between left and right are maximum (which is 200 in this co-ordinate system).
• Further, within each quadrant, if the x and y values are same (in other words, if the modulus of x and y are equal), then one of the left/right motors would stop rotating and the other motor will take the value of y.
• You will observe that in the first and third quadrant (movements intended in the NE and SW direction), roughly the right motor can be said to control the turn aspect and in the other quadrants, the left motor can be said to control the turn aspect.
• Finally, the most important aspect. Draw any straight line on the XY graph. You will notice that amount of turn is same, across all along that straight line ie. the “turn co-efficient” is same for a given straight line, with the same angle from the x-axis.
• For a simple example, if the straight line is x-axis itself, the “turn co-efficient” is maximum at 1 or -1. If the straight line is y-axis, the “turn co-efficient” is 0. And hence turn co-efficient varies from -1 to 1 within a span of 90 deg. And at 45deg, the turn co-efficient is 0 and hence the motor that is said to controlling the turn aspect would be off.

There it is. We should be able to write the algorithm, now and here it is. It is in javascript.

<pre>// Tankdrive function, that converts the x and y values
// to store as left/right
function tankdrive(x, y) {
    // first Compute the angle in deg

    // First hypotenuse
    var z = Math.sqrt(x*x + y*y);
    // angle in radians
    rad = Math.acos(Math.abs(x)/z);
    // and in degrees
    angle = rad*180/Math.PI;

    // Now angle indicates the measure of turn
    // Along a straight line, with an angle o, the turn co-efficient is same
    // this applies for angles between 0-90, with angle 0 the co-eff is -1
    // with angle 45, the co-efficient is 0 and with angle 90, it is 1
    var tcoeff = -1 + (angle/90)*2;
    var turn = tcoeff * Math.abs(Math.abs(y) - Math.abs(x));
    turn = Math.round(turn*100)/100;

    // And max of y or x is the movement
    var move = Math.max(Math.abs(y),Math.abs(x));

    // First and third quadrant
    if( (x >= 0 && y >= 0) || (x < 0 &&  y < 0) ) {
        left = move;
        right = turn;
    } else {
        right = move;
        left = turn;
    }

    // Reverse polarity
    if(y < 0) {
        left = 0 - left;
        right = 0 - right;
    }
}</pre>

Back in January, I posted the first video of our prototype Internet-controlled robot (thanks to Steve Guengerich for the mention of the video on the Wikinomics blog):

This fully-functional demo was controlled via a web page that used a virtual “joystick” which I used to control and steer the robot. These commands were sent to our web server (Amazon EC2) and then the robot received them wirelessly via a wifi connection in my home.  Video was handled via an IP camera that served its own video, and this video stream was embedded into the “control” web page as well.  For those who would like to know more about the robot, here’s some more detail on how this prototype worked and some updates on more recent improvements.

First, here’s some photos of the main robot itself, which is actually the lower half of a power wheelchair. After much experimenting with motors/gears/chassis, I found that used wheelchairs are one of the most affordable sources of a robust motorized platform that offered the precise control and high power that I was looking for.  In the above video, all the hardware is loosely stacked on top of the robot. I didn’t like the look of that, so I now have an enclosure that can be used for the battery and electronics that looks much better. You many recognize this “enclosure” as a piece of carry-on luggage (note the wheels). I liked the smooth look of it, low weight, and the color matched the lower part of the robot almost perfectly. Here’s some photos of it:

The robot’s “brain” is actually a low cost linux PC called an Eee PC made by Asus. These tiny computers have flash memory (making them more impact resistant), can be ordered w/ Linux, and don’t consume too much power. You can also order special super-size batteries for them too (check ebay), which will give your robot many hours of running time. My Eee PC also came with a built-in webcam (meaning you can use skype video for navigation using the Eee PC if you wish instead). Using Skype also makes it easy for your robot to have a two-way dialogue with people in the vicinity too. The most important function of the EeePC is to connect to the Amazon web server to get the movement commands (the EeePC also has built in wifi), and then relay those control commands to the robot.  Here’s a photo of the Eee PC I’m using:

The Eee PC communicates to the robot using a USB input/output device made by Phidgets.com. This company makes a wide variety of USB connected hardware that’s useful for real-world control (via relays, motors, actuators) and sensing (distance measurement, motion encoders, buttons etc). In this case, we use a PWM output signal to control the left-right and forward-backward movement of the robot. So we purchased a Low-Voltage motor control phidget that would accept commands from the Eee PC and then send them to the robot:

One of the next modifications we’d like to test is to generate a PWM control signal via a much cheaper set of hardware. From some limited testing we’ve done with the fantastic arduino microcontroller (about $25 on ebay),  it looks like we ought to be able to get away with a much more stripped down “brain” using an arduino – especially if we use an ethernet shield to connect the arduino directly to the Internet.

The PWM signal generated by the phidget is then converted into a voltage control signal that the wheelchair can “understand”. At the moment, that task is handled by this circuit board:

But I’ve just ordered some more professional PCBs that’ll be a little more durable, and look at lot nicer too (happy to upload the schematic and details for anyone who is interested):

The voltage control signal then goes into something called an Omni +. P&G drives (the company that makes the motor controllers for many wheelchairs) makes the Omni + in order to accommodate other types of wheelchair controls beyond the standard joystick. That way, people with different disabilities can still control their wheelchair – so for example there’s one type of supported control that operates by breath pressure on the end of a straw, or you could add button controllers, or just about anything you can dream up. So in our case, this Omni + controller is an ideal way to hook up our own proprietary control system by sending the right set of voltage commands via our custom circuit board. The Omni + is also very picky about the ranges of voltages it accepts and to check that it’s always connected – and while this requires more careful design of the circuit, it’s designed with safety in mind which is just what’s needed for tele-robotic applications too. Here’s the photo of the Omni +:

The Omni + also has a connection for an emergency stop switch. While I (unwisely) didn’t make use of that during my early testing, I’ve since purchased a wireless emergency stop button that will activate it in the case of a runaway robot. Always good to default your switch settings so that shutdown happens automatically if the switch gets low on batteries or gets out of range. Here’s the wireless switch I chose for the task (it offers up to four remotely controlled relays, so I may have to think up uses for the other 3):

Last, is the video camera I used for the over-the-web navigation. In this case I tried a few different IP cameras that come with their own min-web server that can send video directly over the Internet. We then embedded this video stream into the control page for the robot (included the virtual joystick, and the live video from the robot). Because I wanted the video to also work wirelessly I chose a wireless Linksys PTZ camera for this task. The pan/tilt/zoom feature was nice, latency was moderate, and price wasn’t prohibitive. Here’s a photo of the camera:

The camera itself was mounted on a microphone stand that sat atop the wheelchair. I’ve since added a flexible gooseneck to the top of that microphone stand and that’s a dramatic improvement since it allows the camera to be easily twisted to get the desired angle. Overall though, this is the feature that I’m currently least happy with. The latency of the camera (time it takes between the moment of video capture and the time it arrives on the control web page) is still too high for my liking. And that makes it a bit difficult to steer and move around. The other limitation is that it’s very very difficult to perceive depth using a camera like this, and you really have a hard time knowing just how far things are in front of you or how much room you have to maneuver. It was difficult to guess the extent of this limitation until you actually tried navigating from afar. It does get better with practice however, but I think we still need more improvement in this area.  That’s a big area of focus for the work we’re doing now, so the next version should be much more intuitive.

In my next blog post, I’ll describe some of the new work that’s been going on in the creation of our general-purpose robot control architecture, and where we’d like to take GoodRobot.com in the future.

The objective of writing these posts, in addition to tracking using Tickets, is to make sure information is not lost or corrupted (time corrupts memory, human memory that is). Ticket #10 tracks this problem.

As explained in this post, data tags need to be unique across all the features. That would mean we cannot have the same feature twice. Say, we have two motors connected to a Phidget, which is handled by motorControl Feature. Suppose I want to connect another phidget with another two motors to it, we need to create the second instance of the the same Feature and this is not possible as of now. In the first place, adding the same Feature in the same Robot is not possible. Secondly, even if the feature was added, the same data tags will be present twice and that violates the “unique” requirement.

There are alternatives for resolving the issue. One simple method is to create a new feature (which is a copy), with the name motorControl2,  except that the data tag names are modified to “speed_2″ and “direction_2″ (that _2 indicating the second instance’s data). And the third instance with _3 suffix and so on. But, as one can see, this is not elegant solution.

This is how we can address this problem:

1. Create a instance field in the RobotFeatures model, so that one can add the same feature multiple times, with the instance field varying and pointing out the number of the instance.

2. The data tags will be sent in duplicates, but EEML structure allows us to “multiple” tags for the same datum. Something like this:

<data id=”0″>
<tag>speed</tag>
<tag>velocity</tag>
….
</data>

Here “speed” and “velocity” are synonyms for the same data set that is sent. We will use the same mechanism to identify the “instance” of the data tag, that is being sent. The additional data tag will identify the instanceth number like this:

<data id=”0″>
<tag>speed</tag>
<tag>instance:1</tag
</data>

<data id=”0″>
<tag>speed</tag>
<tag>instance:2</tag>
</data>

If there is no additional tag with “instance:n” present, it is considered to be the first or the instance-1 th parameter. (Also, this is the default for those single instance feature). If the additional tag “instance:n”, then that tag should match the “nth” instance tag, as was specified along with Features, while added to Robot.

This will help solve this problem. However, we won’t address this as late as Milestone 7 or later.

This is a quick post. Each Feature has data tags, that is used in data communication with the server, using the API. Within a component, there can be multiple Features (such as a phidgetMotorControl, phidgetDistanceSensor etc) but when it communicates with the GoodRobot server, all data tags of all features are sent across together.

That would mean that all Features, that would go together in a single component, need to have unique data tags. I don’t suppose this is an issue, but people who write Feature XML definition (developers) need to keep this mind.

That’s it for now.

Well, the GoodRobot server is aimed to provide “any” to “any” connectivity – some sort of “switchboard” to connect and pass data between a “resource” (a robot, switches, relays, sensors) and a “control” element (a web page, twitter, pachube, any user control). A resource accepts data that can be used to control a robot, relay, switch or any real world element, which data is provided by a control component. At the same time, the resource provides sensor data, which is consumed by a control component.

There is a need to provide some mechanism that can act as a “glue” between a resource or a control component. For example, we should be able to pass the forward control movement to the appropriate motor of the resource component. And such data of mapping resource and control tags need to come from the user.

There are two types of mapping that is required and this post describes that. Prior to that, a brief on Feature and Plugin.

A Component is the end point node, an element that talks to the GoodRobot server using the API. Each component has multiple Features, with each Feature describing the capabilities that it brings to the table. The Features actually don’t have the implementation, for which it relies on the Plugins. An example: a motorsensor feature would provide the details of forward and backward movement, as well as directional turns such as left or right. That motorsensor can be implemented using any of the plugins such as phidgets, arduino, nerd kit or even a virtual one. Conversely, using the same plugins, different types of applications (features) can be used.

So, back to the two types of data tag mappings, that need to be provided by the user:

Plugin Mappings: (Hardware Mappings)

A plugin that represents the hardware, such as IO pins or motors, have different inputs and outputs physically. Which hardware pin is connected for which purpose is a simple requirement. Ex: which input pin out of 16 is connected to a temperature sensor is to be specified by the robot owner. In this case, the plugin tag pin16 may be mapped to a feature data tag tempSensor.

This configuration data is provided by the robot owner, for each Robot, on the GoodRobot server using the Admin pages. Default values are provided and pre-configured, when a Robot is added and the Robot owner can go there and modify the values anytime. For now, the screen to go is Robot Plugin Mappings.

Feature Mappings: (Control-Resource Mappings)

These are the data tag mappings between a control element and a resource element. A resource feature may have a data tag as speed (indicating the forward/backward movement of a robot) and this may need to be mapped to yaxis data tag of mouseGrid Feature (that is of type control). The robot owner provides such mappings on the Admin screen of the GoodRobot server, for each Robot. For now, the screen to go to is Robot Feature Mappings.

Each resource or control feature will provide sufficient hints in its definition XML, so that data tags can be automatically mapped for default configuration, in most cases.

Thus, we can have “any” control feature to “any” resource feature mapping, including any individual data tags of control feature mapped to resource feature. The other plugin mapping allows proper hardware configuration.

The initial design had Component playing one of the important role in the internal design and architecture of GoodRobot.  The description of Component is here. In short, Component are end-point nodes that interact with the server using the API. Typical examples include a Robot (resource) or web page (control).

Each component had a definition XML (CoCap XML), that included data tags that will be sent and received and as well definitions for configuration data. For sake of granularity, these included Feature definitions — which had a similar structure and data/config items as Component — so that the same Feature definitions can be included in multiple Components (and hence no need to repeat the same data/config items in more than one Component).

However, internally the administering this setp had the following problems:

• The single most important problem being that these Component definitions cannot be modified by administrator or user, as they are stored in XML files, and created by software developer.
• For ex, one may have a component, that defines the Phidget motor plugin and is specified in the capability XML. If we need to add another functionality (say add a sensor using another phidget IO), the developer needs to create a new component definition including these two functionality. As may be understood, it is difficult to create various combination of components, each with a specific set of features and functionality.
• There are other problems as well, about the ability to modify a component in an existing Robot configuration — which disturbs several settings in the database.

So it was decided to change the design, in order to avoid the above problems. As per the new design:

• The term Component still exists (end-point node that communicates with the server using API), there will be no CoCap XML (Component Capability XML is being done away with).
• When a Robot is defined, the user selects the list of Features (Feature definition is similar in contents to CoCap XML) via the admin screen. Based on hints from the Features file (based on control or resource), the Component name is automatically chosen (default as “control” or “resource”) and saved. Typically that specifies there are two components — one is the Robot itself (”resource”), the other being the Web component (”control”)
• If a robot requires more than these two components, in the Advanced admin screen, the user can specify new component names (may be “control2″, “resource2″ or any other name).
• In other words, each Feature is configured to “belong” to a specific component name, which has normal default values, but can be modified in the admin screens.

The initial design of the API was completely based on EEML XML schema. EEML is used by Pachube and is meant to update to environment data (input and output feeds), which mostly mirrored our requirement of Robots providing resources with web providing controls – though feeds do not really correspond to “controls” and “resources” directly. In any case, the EEML structure was considered to be sufficient for GoodRobot requirements as well.

However, the config data and now the feature mapping data clearly don’t fit the EEML structure and using hacks (like sending data tags beginning with “underscore” to mean config data etc) is proving to be ugly. I did not earlier opt for a different XML schema, other than EEML, just to keep to standards.

But what struck me now is that we can as well have both XML schema support, EEML and as well as EEMLx. Robot components, who do not require lot of config data can simply send EEML in the API and who require them can send EEMLx. The server will support both XML schemas, and will find which schema is being sent by the xmlns attribute in the XML file sent as part of the API.

EEMLx XML schema will have the following additional parameters:

• config element, similar to that exists in Feature XML schema
• a new mapping element, that can provide mechanisms to describe Feature Mapping aspects
• data tag will have additional parameters to specify default value etc

The EEMLx schema should be located at http://web.goodrobot.com/xsd/eemlx-01.xsd