When I got two new Rascal prototypes back from the assembler last month, I was pretty excited. The new version, Rascal 1.1, cleans up a bunch of little annoyances and adds new features. The previous batch had sold out quickly and, even better, the Rascal was living up to my vision of a relatively easy-to-use board for connecting stuff to the internet.

After some testing, all the new features-- USB ports, the new JTAG port, and the like-- appeared to work well. Unfortunately, there was a problem with the Rascal's RAM, which meant that when the Rascal tried to write data to certain addresses, the data was lost. If that data turned out to be code that was later executed, the Rascal would choke and reset when it tried to execute whatever random data happened to be there before the failed write.

This all seems straightforward in retrospect. In futuro-spect, it took me about a week to narrow down the problem to failing RAM writes.

Figuring out the cause

After watching the board reset a few times, I tried logging the boot messages to see when resets occurred. Some tedious tallying yielded this table of last boot messages from the Linux kernel before reset occurred.

Reps  Message
----  ------------------------------------------------  
1     Uncompressing Linux... done, booting the kernel.
4     RPC: Registered tcp NFSv4.1 backchannel transport module.
2     INIT: version 2.86 booting
1     mount: mounting /dev/mmcblk0p1 on /mnt/root failed: . . .
4     Setting up IP spoofing protection: rp_filter.
2     Lease of 192.168.10.190 obtained, lease time 43200
1     udevd (403): /proc/403/oom_adj is deprecated . . .
1     * Starting Avahi mDNS/DNS-SD Daemon: avahi-daemon...done.
2     Populating dev cache
1     INIT: Entering runlevel: 5
14    rascal14 login:

My fellow Artisan's Asylum denizen, Edison, noticed that the resets seemed to only occur after U-boot passed control to the Linux kernel. He suggested that we try making the board reset under U-boot. Looking through the list of U-boot commands, the memory test command, mtest, seemed like it might generate a decent load on the processor, so we gave it a try.

As it turns out, mtest revealed that the processor resets are probably caused by memory write errors in the RAM on the board. The idea is that the kernel gets written to RAM incorrectly when it is copied out of the serial flash. Faced with an invalid instruction in RAM, the processor does the only thing it can do, which is reset itself. Similar failures occurred on both new boards but did not occur with an older board.

The memory errors reported by U-boot looked like this:

Pattern 0000001E  Writing...            Reading...
Pattern FFFFFFE1  Writing...            Reading...
Pattern 0000001F  Writing...            Reading...
Pattern FFFFFFE0  Writing...            Reading...
Mem error @ 0x2124A33C: found 0049D711, expected FFB6D711
Mem error @ 0x2124A340: found 0049D710, expected FFB6D710

There's a pattern to the errors: the two lower bytes are correct, but the two upper bytes are the inverse of what they should be (0x0049 instead of 0xFFB6, and note that 0xFFB6 + 0x0049 = 0xFFFF). This pattern appeared most of the time. Sometimes, all four bytes of the found value were exactly the inverse of the expected value.

The mtest program writes to every RAM address alternating values from each end of the range of 32 bits, i.e. this sequence: 0xFFFFFFFF, 0x00000000, 0xFFFFFFFE, 0x00000001, . . ., but incrementing the value written by 1 for each memory location in the address space. The memory test code is on Github.

The fact that the data read back is the inverse of what we expect, with errors along byte boundaries, rather than random corruption, suggests that the problem is a timing issue, rather than a data or address problem. It looks like two bytes are failing to be written from time to time, so we read back whatever was written during the previous cycle of the memory test, which explains the inverse values. This leads to looking at signal integrity.

This matter called "signal integrity"

"Signal integrity" means making sure that the digital pulses on a circuit board pass between chips without distortion in time or voltage. Sharp-edged pulses, as used in most digital communication, get rounded off because PCB traces have a little bit of capacitance and a little bit of resistance, which together make a low-pass filter. For most signals, this is not a problem, but when you send pulses that are faster than around 100 MHz over distances of more than an inch or so, you have to start being careful. With slow signals, if the sharp edges of your signals get rounded off for a few nanoseconds, you don't care. On the other hand, if you're signaling at 100 MHz, your signals are only 10 nanoseconds long, so you can't afford a few nanoseconds of sagging. Debugging this kind of problem is made more difficult because the average oscilloscope is too slow to capture 100 MHz signals accurately; all the sharp corners get rounded off whether your pulses are getting distorted or not.

There are more problems beyond getting sharp transitions. As signals propagate down PCB traces, they can be reflected back wherever the impedance of the trace changes, just like a wave of water reflects off the side of a tub. These reflections settle out in a few nanoseconds, but that's still a problem for high frequency signals.

Also, you want all of your signals to arrive at the same time. Electrical signals propagate through copper at around half the speed of light in a vacuum, or around 6 inches per nanosecond. This means that for two traces that differ by an inch in length, you get a timing error of around 0.17 ns.

Signal integrity on the Rascal

Here's a cross-section of the Rascal circuit board. There are 4 layers of copper separated by 3 layers of fiberglass.

The thickness of the green layer (which is actually yellow in real life) means that a 5 mil wide trace has a characteristic impedance of 80-85 ohms. In theory, I could match this impedance with termination resistors near the memory chips to insure that signals don't reflect in nasty ways, but because the memory errors look like timing problems rather than noise from reflections on a few data lines and more parts mean higher costs, I decided not to add them at this point.

My friend Michael has an extremely fast oscilloscope. Using his scope, I was able to take a look at the signals on some of the memory control lines. Here's what they look like. (Apologies for the poor photo-- I wasn't intending to show it to the world.)

The whole screen shows shows two signals for 1/20,000,000th of a second. The green line is the 133 MHz memory clock pulse. The yellow line is the write-enable signal, which pulses low to signal that a write is taking place.

So what the hell does this picture mean? A rough summary is, "That yellow line looks like a noisy mess." In more precise terms, after the green signal crosses the minimum logical high voltage of 2.0 V (AKA V_IH,MIN), we need the yellow signal (write enable) to stay below the maximum legal logical low voltage, 0.8 V (V_IL,MAX) for 0.8 ns. You can't tell exactly from this picture, but the yellow signal rises right when the required hold time elapses. In theory, it should work, but given how noisy the signal is, it seems likely that it might slip to the wrong side of the line some of the time.

How do we fix it?

To make the memory timing work right, I wanted to do two things at the same time-- I wanted to make sure all the signals arrive simultaneously and that they're not jammed so close together that currents in one trace induce noise in adjacent traces.

To make the traces the same length, I delved into the fetid depths of the Altium API via Jscript. After Altium emitted the length of each leg, I added them together with a quick Python script. (I'll include the code at the bottom of the post.) This gave me a CSV file that allowed me to calculate the average and standard deviation of the net lengths.

With the help of this tool, I redid the connections between the RAM and the processor with even trace lengths. For the original Rascals, I laid the board out without particular attention to signal integrity. The average memory trace was 2.91 inches with a standard deviation of a whopping 1.24 inches. For Rascal 1.2, the average trace was reduced to 2.15 inches and the standard deviation was 0.23 inches. The picture below shows the original Rascal in blue on the left and the new version in red on the right. You can see that the blue version looks crazy; in the red version, the longest mismatch relative to the clock line roughly 5x better at around 0.7 inches, which corresponds to a delay mismatch of 0.1 nanoseconds.

I also rearranged the decoupling capacitors to minimize the length of their connections to the ground and power planes. I tried to space traces at least 6 mil apart from each other to minimize crosstalk between them. I could have done a more serious analysis of the trace spacing to minimize crosstalk, but as with termination resistors, I opted not to do it until I know I have to. My original, naive Rascal design worked, even though my traces were drastically different lengths, crammed together, and varying in impedance, so my hope is that a layout that doesn't do anything really stupid will succeed.

Rascal 1.2 on the way

The Rascal 1.2 PCBs have been sent out for assembly in Colorado; I should have the new Rascals back later this week. If they work, I'll have a larger batch up for sale in a few weeks as I already have the PCBs. Otherwise, Rascal 1.2 be cursed, and on to Rascal 1.3!

UPDATE

I got the new Rascals today (May 2, 2012) and they work. I haven't tested them 100% yet, but they both booted correctly, passed a few minutes of memory testing (~1 trillion write/read cycles without error) and loaded a bunch of files in the editor without a hitch.

Code

Here's the Jscript code for talking to Altium; feel free to do whatever you want with it. I'd recommend deleting it and then destroying all storage media it may have tainted.

var Board; //IPCB_Board;
var Net;
var Iterator;
var ReportFile;
var address_lines = ["A0","A1","A2","A3","A4","A5","A6","A7","A8","A9","A10","A11","A13","A14"]; // A12, A15+ omitted deliberately
var data_lines = ["D0","D1","D2","D3","D4","D5","D6","D7","D8","D9","D10","D11","D12","D13","D14","D15","D16","D17","D18","D19","D20","D21","D22","D23","D24","D25","D26","D27","D28","D29","D30","D31"];
var control_lines = ["CKE","CLK","CS","CAS","DQMH1","DQMH2","RAS","WE"];
var term_res_lines = ["NetRA3_1","NetRA3_2","NetRA3_3","NetRA3_4","NetRA4_1","NetRA4_2","NetRA4_3","NetRA4_4","NetRA5_1","NetRA5_2","NetRA5_3","NetRA5_4","NetRA6_1","NetRA6_2","NetRA6_3","NetRA6_4"]; // NetRA1* and NetRA2* omitted deliberately
var more_res_lines = [,"NetRA7_1","NetRA7_2","NetRA7_3","NetRA7_4","NetRA8_1","NetRA8_2","NetRA8_3","NetRA8_4","NetRA9_1","NetRA9_2","NetRA9_3","NetRA9_4","NetRA10_1","NetRA10_2","NetRA10_3","NetRA10_4","NetRA11_1","NetRA11_2","NetRA11_3","NetRA11_4","NetRA12_1","NetRA12_2","NetRA12_3","NetRA12_4","NetRA13_1", "NetRA13_2","NetRA13_3", "NetRA13_4", "NetRA14_2", "NetRA14_3"];
var critical_nets = address_lines.concat(data_lines, control_lines, term_res_lines, more_res_lines);

if (!Array.prototype.indexOf) {
   Array.prototype.indexOf = function(item) {
      var i = this.length;
      while (i--) {
         if (this[i] === item) return i;
      }
   }
}

FileName = "C:\Net_Length_Report.Txt";

fso = new ActiveXObject("Scripting.FileSystemObject");
ReportFile = fso.CreateTextFile(FileName, true);

function ShowBusLength(){
    Board = PCBServer.GetCurrentPCBBoard;
    Iterator = Board.BoardIterator_Create;

    Iterator.AddFilter_ObjectSet(MkSet(eNetObject));
    //Iterator.AddFilter_NetClass("U5-signals");
    Iterator.AddFilter_LayerSet(AllLayers);
    Iterator.AddFilter_Method(eProcessAll);
    Net = Iterator.FirstPCBObject;

    while (Net != null) {
        if (critical_nets.indexOf(Net.Name) >= 0) {
            ReportFile.WriteLine(Net.Name + "," + CoordToMils(Net.RoutedLength));
        }
        Net = Iterator.NextPCBObject;
    }
    ReportFile.Close();

    ReportDocument = Client.OpenDocument("Text", FileName);
    if(ReportDocument != null)
        Client.ShowDocument(ReportDocument)
}

Here's the Python script for summing lengths of multi-leg nets.

import csv

r = csv.reader(open('net-length-report.csv', 'rb'))
d = {}
for line in r:
     d[line[0]] = line[1]
d['zero'] = '0.0'

outfile = open('output.csv', 'wb')

buses = [
    ('A0','NetRA11_1'),
    ('A1','NetRA11_2'),
    ('A2','NetRA11_3'),
    ('A3','NetRA11_4'),
    ('A4','NetRA12_1'),
    ('A5','NetRA12_2'),
    ('A6','NetRA12_3'),
    (you get the idea-- there were more nets listed here)
]

for leg1, leg2 in buses:
    total = float(d[leg1]) + float(d[leg2])
    outfile.write(leg1 + ',' + str(total) + '\n')

outfile.close()

In reference to last year's similar event for 3D printing, Michael says,

"If you're a legislator and the first time you ever hear about 3-D printing is someone coming into your office saying, 'This horrible pirate box is ruining my business,' you would have one world view of 3-D printing. If the first time you hear about 3-D printing is someone saying, 'Wow, look at all these new businesses and people who are coming together around technology and creating all these amazing things,' you have a different world view about the subject."

Cat explains that with this approach, "Public Knowledge doesn’t have to spend the first half of a meeting explaining what it is they’re talking about."

The event consisted of two panels about open source hardware, followed by a demo session in the Rayburn Foyer. I watched the panels and had a table in the demo session. The demo I brought for the event is a motor that can have its speed controlled through the internet; the picture below shows its I-am-not-a-terrorist guise.

The two panels occurred in a small room on the third floor of the Rayburn office building. Each panel was 4 or 5 people. There were around 100 people there; at least 40 of them were open hardware folks; the remainder were either policy people or random geeks who wanted to see the action.

My hometown of Somerville is in Rep. Mike Capuano's district (he used to be mayor of Somerville), but Capuano and his aides were too busy to come to the event.

Where we stand

The panelists answered questions from the moderators, Alicia Gibb and Michael Weinberg. The event was filmed, so I won't try to reproduce all the questions and answers here, but it was definitely interesting and informative. Bunnie Huang had a great rant about how everyone involved in the production of electronics should visit the electronics markets in Shenzhen and feel the energy of commerce. (If that intrigues you, read his great blog post about Shenzhen from a few years ago.) All hardware, with some level of reverse engineering, is open, he pointed out. There are huge markets that don't have the strong IP rights enforcement we have in the US.

To me, the point is that we are making open source hardware even if we don't want to. This forces us to adopt openness by default, so that we get the benefits as well as the detriments.

I was encouraged to see that we have a bunch of smart people on our side. Bunnie, AnnMarie Thomas, David Mellis, and Nathan Seidle are articulate and funny. The demos were impressive. My favorite moment of the day was talking to a janitor who was drawn in by a RepRap 3D printer that was running unattended before the event officially started. Noticing that she was staring at the machine and seemed puzzled, I pointed out the printed-out plastic rabbits on the table and explained that the RepRap was halfway through printing another one. I showed her the pair of new Makerbot Replicators that can print two colors at once. She had the 3D printing epiphany and said something like, "This thing can make you anything you want!" She grabbed a few Makerbot fliers and as she left, she said, "I gotta tell people about this!" She had the altered-world urgency of someone who has just learned that Kennedy is still alive, or discovered that soylent green is people, or seen Nathan's secret tattoo. (Don't bother asking. He just pretends it doesn't exist.) I'm glad that we could help spread the enthusiasm.

The demos went well; I'd estimate that 50-100 people passed through, which is pretty good for a sunny Friday afternoon. I hadn't seen the very cool squishy circuits before; somehow, they managed to attract the children in the crowd, which is impressive for an event aimed at adults in suits. The Rascal demo worked perfectly, though I did have a couple moments of terror while setting up. First, it seems that in double-checking my demo, I managed to delete my DHCP server's configuration file. Fortunately, it was easy to switch the Rascal to a static IP. I guess I'm prepared for idiots like me.

Then, in my motor control demo, some of the Javascript code was linked from external servers. When I tested it the night before without an internet connection, the Rascal used cached versions of the code. 12 hours later, the cached versions expired, which left me scrambling for a wireless connection to pull down the code I needed. After some quick modifications to use local code, the demo worked quite well. Several people stopped by to ask when new Rascals would be available for sale, which made my day. (Thanks, Shawn!)

Still, my overall impression was that as a group, we open hardware folks are weak on the policy front. We don't know what policies we want. Worse, we don't even know what outcomes we want. Some of us really don't care about licenses; others think we need them desperately; and still others think licensing hardware is hopeless. We're just setting up the first open source hardware nonprofit organization. Our most experienced professionals are people like Nathan and Bunnie, who were in college 10 years ago. Generally, we seem like people much more interested in building stuff than public policy. In some contexts, our political inexperience is alarming; Michael Weinberg put it more optimistically: "No, that just means you've made prudent life choices!"

Thanks to Public Knowledge for organizing the event.

(More details about the state of the new Rascal boards coming soon.)

Whenever I build a new version of the Rascal, the big question is, "Have I made some horribly stupid error that has turned all of the parts I sent away to the assembler into landfill stuffing?" So far, I haven't found any landfill-worthy errors.

Here's the list of changes.

1. Added second USB host port and stacked USB connector
2. Put in a new Ethernet controller (Micrel KSZ8051RNL)
3. Moved the I²C/TWI pins to the Arduino-compatible location
4. Added I²S port for streaming audio
5. Changed JTAG footprint to work with pogo pins more easily
6. Switched to black Samtec headers rather than the blue ones, which were chintzy

What works and what doesn't?

The new Rascal has two USB host ports. (Earlier Rascals have only one, and with a worse connector.) I plugged in an old Logitech webcam I picked up at the MIT flea market a few months ago and it was correctly identified by the kernel, as you can see in the command line snippet below.

[root@rascal1.1$:~]: lsusb
Bus 001 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 001 Device 002: ID 046d:08b0 Logitech, Inc. QuickCam 3000 Pro [pwc]

Both USB ports are at least wired correctly, but I haven't tested them more thoroughly yet.

The hairiest change in this rev of the Rascal was the change to a new ethernet PHY chip, the Micrel KSZ8051. The new chip costs less, uses less power, requires fewer external caps and resistors to make it work, and, most importantly, won't be obsolete any time soon. The macb driver in the Linux kernel correctly identifies the controller, as you can see in the snippet of boot spew below, but the link doesn't come up. I'm not sure where the problem is yet.

MACB_mii_bus: probed
eth0: Atmel MACB at 0xfffc4000 irq 21 (02:71:82:06:00:14)
eth0: attached PHY driver [Micrel KS8051] (mii_bus:phy_addr=ffffffff:00, irq=-1)

I haven't tested the I²S or I²C ports yet.

The new JTAG interface worked for programming the kernel and bootloaders into the serial flash on the Rascal. I had to make a new programming fixture, pictured below. The beige part was printed on the Uprint 3D printer at Artisan's Asylum. The brass pins are pogopins, i.e. their tips are spring-loaded like (upside-down) pogosticks. The 20-pin black connector at lower right will connect to an Atmel JTAG pod, which then connects via USB to a PC.

Having access to high-quality 3D printing on the cheap (that part cost around $10) is definitely changing the way I design stuff. For low volume plastic parts like this, it's a huge improvement over sending stuff out to be machined out of Delrin with a 5 day turn for $500.

Here's one last photo, which shows the Rascal in the fixture. The pogopins are under the PCB under the clamp.

If I can get the ethernet port working under Linux and no other problems emerge, I'll send the 20 remaining PCBs out for assembly, and they'll be available in the store for sale when I get them back. After that will come the first batch of 100 Rascals, which will be a huge milestone.

So far, so good!

Here's a screenshot of the final layout. (Click for a substantially larger version.)

Thanks to Jinbuhm Kim of Wiznet for the last-minute suggestion to add pin labels on the underside of the PCB (so you can see them after an Arduino shield is plugged in on top).

A full list of the hardware changes appears in the previous blog post. The software will be updated as well-- a new Linux kernel, a new driver to call Python from hardware interrupts, an improved web editor, and some awesome new HTML5 demos from the volunteer sharpshooting development team in England.

If you think the Rascal might be right for your next ridiculous internet machine, sign up for the announcement list to get alerted when they're ready.

The next batch of Rascals will be nearly identical to the previous batch. The hardware changes are:

1. Added second USB host port and stacked USB connector
2. Cheaper, newer, less power-hungry Ethernet controller (Micrel KSZ8051RNL)
3. Moved the I2C/TWI pins to the Arduino-compatible location
4. Added I2S port for streaming audio
5. Changed JTAG and serial debug footprints to work with pogo pins more easily

The software will also be improved for the next batch, in part thanks to the people who have been tweaking the code on their Rascals. The editor will have some new features and the set of tested libraries on the Rascal will be much larger. Because the hardware changes are minor, it's likely that the new software will run fine on old Rascals.

The next batch will be delayed because of Chinese New Year, which runs from January 20 - February 5 this year. In the meantime, if you're interested in buying a Rascal, sign up for the announcement list to get alerted when they're ready.

Making the Rascal work with Color Kinetics lights

Most of my time has been devoted to PCB layout, but I've also been working on some Rascal-controlled light displays around Boston with a group of public artists operating out of Artisan's Asylum. Dan Taub of CEMMI was kind enough to lend me a Color Kinetics light for testing. I've been using a Python library written by Giles Hall that wraps DMX512 data in UDP packets using the KiNet protocol. I created a quick demo page consisting of a color picker on a web page. You pick a color and the Rascal tells the light to turn that color.

Here's a short video that shows the demo in action. If you happen to be on Artisan's Asylum's wireless network, you're welcome to test it out yourself-- just look for the light near the Rascal Micro space (#16).

Read/listen to the interview.

A brief rant about firewalls and port forwarding

One topic that I had meant to discuss, but forgot about in the heat of battle, is the growing asymmetry of internet connections as the industry matures. The original design of the internet made each computer on the network equal to every other. At least in the US, this is no longer the case. Most incoming connections to our homes and businesses are filtered to prevent people from running their own servers.

Instead of assigning unique addresses to each computer, we use what's called "Network Address Translation," which allows each connection to build its own network behind a firewall, while masquerading as a single computer to the rest of the internet. This works pretty well for surfing the web, but when you want to take off the mask to build the internet of things, it's a problem. Additionally, some connections are filter out incoming requests to ports 80 (HTTP) and 25 (email). This is the case for the least-of-evils, at-least-they're-not-Comcast cable connection from RCN I have at home.

In theory, you can request a block of IP addresses from the American Registry of Internet Numbers, but the minimum fee for IP addresses is $1250, plus various fees for other details (registration and maintenance of an Autonomous System number and such) and ARIN won't give you addresses without a convincing explanation of why you need them. Even if they relented, you'd still need to pay someone who owns a bunch of telephone poles to run a wire from your house to your local internet exchange or negotiate some kind of deal with the local government to allow you to run the cable yourself.

There are a few workarounds that you can use. If you have control of your router, you can pick a few external ports that aren't blocked by your ISP and forward them to devices on your internal network. If you live in a big apartment building where all you get is wireless access, this won't work unless your landlord likes you a lot and nobody else wants to fight you for those ports. The other option is creating encrypted software tunnels to external servers, but that requires either that you run a server yourself or get access from someone who does.

Anyway, it's a bit of a mess. I wish I had more ideas about how to fix the situation, but I don't. I wish I had grown up in the good old days, when men were men and IP addresses were globally routed.

Now that consumer-grade 3D printers like the Makerbot Thing-O-Matic are available, printing custom brackets to hold your circuit board is plausible. I've taken a first cut at the design and uploaded the source files to Thingiverse. If you'd like some brackets for your Rascal and you have access to a 3D printer, you're welcome to download the files and print out some brackets. The print time is around 8 minutes per bracket. They'll work with either #4-40 or M3 fasteners. If you're not sure, use M3 because the metric system is better.

If you'd like to try this out, but you can't find a 3D printer, you might try checking with your local hackerspace, which you can find on the Hackerspaces.org map.

Drilling and tapping 3D printed brackets

The holes in the brackets won't have threads in them straight out of the printer. To make threads in the plastic, you need to drill out the holes to the right diameter and then use a tap to cut the threads.

It would be nice if you didn't need to predrill the holes before tapping. In the model, the holes are set to 2.26 mm in diameter, which theoretically shouldn't need predrilling, but with parts printed on my local Thingomatic, the actual diameter was smaller, so predrilling was required.

M3 fasteners

For M3 fasteners, I use a 0.100" drill (AKA drill size #39). You could certainly use a 2.5 mm drill. I suspect that a 3/32" drill would also work. For mounting the Rascal, a M3 x 6 stainless steel socket head cap screw, such as McMaster part number 91292A111, works great.

#4-40 fasteners

For #4-40 fasteners, I use a 0.089" drill (AKA drill size #43). I suspect that a 3/32" drill would also work. For mounting the Rascal, a #4-40 x 1/4" stainless steel socket head cap screw, such as McMaster part number 92196A106, works great.

I'm working on a project to control the light blades on the Greenway in Boston through text messages. While I was filming the video with my phone, Dan was crouched in "The Vault," a concrete bunker under the Greenway plaza. We had a Rascal plugged into the light controller with a serial cable. Dan used a Python script I wrote to generate HTTP POSTs on my laptop. In theory, the POSTs are the same requests that we'll eventually receive from Twilio. (See this Rascal-and-Twilio tutorial for the details on how that works.) The Rascal parses the messages Dan sends and decides what show to play. The shows are still a bit of a mess, but the Rascal is working great.

I don't want to say too much more about what we have planned until it's closer to fruition, but I think it will be cool.

Below is a better still shot from Eric Kilby that gives a better idea of what the light blades look like in person. There are 12 of them in all; they're in the park between Faneuil Hall and the Aquarium.

The Rascal beta testers have done their job. As of today, there are 11 Rascals available for sale in the Rascal store. The hardware is the same as the Rascals in the beta test, but the operating system and web interface have been upgraded with all the improvements from the last 6 months of testing and software development.

You're a good candidate to buy a Rascal today if:

1. You want to connect something to the internet, and you'd like my help in doing it. I don't want these things stuck in a drawer!
2. You're willing to put up with some rough edges. The web interface works, but it's still rudimentary.
3. You have some technical skills. While not necessary, some vague familiarity with SSH would be helpful if we need to debug stuff.

Historic firsts

As far as I know, the sale of the Rascal to the general public marks new progress for our species in a few ways.

• First small hacker board to ship with a modern Python web stack (Flask, uWSGI, and Nginx) pre-installed. This is the original target of the Rascal achieved.
• First small hacker board to ship with a web-based editor that you can use to edit web pages on the board.
• First Linux hacker board to ship with Arduino shield headers. (The Leaflabs Maple has the headers, but no operating system. I hear there was a beta version of the Chumby Hackerboard that had Arduino headers, but they were removed in the production release.)

"Getting started" video tutorial

Now that the hardware and software have reached a level of basic functionality, I've been working on video tutorials that explain how to use the Rascal. The first tutorial covers the web interface.

Detonate the fireworks now,

Brandon

The talks weren't recorded and my slides (1.7 MB PDF) contain only an outline of what I talked about, so I thought it would be useful to summarize what I talked about, while also adding a few links and details that I omitted in the 18-minute spoken format.

What I said at Sketching In Hardware

Introductory fact: I want to connect weird stuff to the internet.

Because I'm an embedded systems engineer, idle conversation often leads to someone saying this to me: "Oh, you know about computers? OK, here's what I want to do. I want to somehow take this thing and connect it to the internet."

"This thing" is usually something weird. Recent examples include:

• A chicken coop
• Some kind of sausage factory
• Hot tub
• Thermostat for the boiler in a box factory
• A lighting display that reflects the anger level of the owner, two friends, and a guest

I love this kind of project, but I used to have an attitude about the proper hardware for the job.

In grad school, I used to scoff at the BASIC Stamp. Running into a friend buying a BASIC Stamp in Fry's, I'd challenge them: "You know that's just a $3 PIC with a BASIC interpreter on it, right? Why would you pay $50 for that?"

They would have some reasonable response like: "Look, Brandon, I just want to make this valve keep my tomato plants watered. What would you have me do?"

I'd think: "Why, just learn C, PCB layout, and some basic electrical engineering! . . . Er. Um. I guess I'm being unreasonable. But it's a $3 PIC!" I'd say something dumb, and then storm off to the lab.

Now, after a few years working with lots of different microcontrollers and industrial automation systems, I'm on the other side. Yes, I know C, and I can lay out PCBs, but for simple tasks, I want a simple tool.

This is where the Arduino enters the picture. The Arduino folks have done an incredible job of making a simple microcontroller board that is easy to use for people who are new to programming. As a result, a huge community has developed around the board. The Arduino is not the ultimate solution for all problems, but it does a great job at simple automation tasks.

Moving beyond the Arduino

Of course, as you try to do more complex stuff with the Arduino, you start to hit its limits: 16 MHz clock, no file system, no threading, 2 kB RAM, 32 kB of flash, no internet connection, no display. Marginally capable of audio, no hope of video.

You start cobbling together an operating system with different modules. You try to squeeze a webserver, network driver, and threading module into 32 kB.

(Note: Dave Mellis was nodding knowingly and smiling at this point in the talk.)

Then you end up in the unfortunate situation where you need to jump up to a more powerful system. There are lots of choices, but probably you end up with something like Gumstix or the Beagleboard-- an embedded Linux system. Suddenly, instead of the friendly Arduino program, you're off in the weeds. The friendly programming environment is gone. The sea of helpful community has drained back through the Bosphorus, and you're standing in a field of muddy, dying fish. This is not pleasant.

(To speak in defense of the manufacturers of Linux boards for a moment: they're targeting professional engineers who want cheap, reliable hardware. They're deliberately not optimizing for amateur tinkerers, because tinkerers don't place large orders. Makes sense to me.)

This brings us to the point of this presentation (er, blog post). I've come up with 8 barriers to normal people connecting weird stuff to the internet. There are certainly more, but these are what I see as the big ones. I'll list each problem and try to explain what I see as our best solutions are, or at least what direction we should be going. (Throughout this discussion, I'll refer to a "Linux board," but it could be running some other operating system and most or all of the same problems would exist.)

Problem 1: “I can’t find the damn thing’s IP address!”

You plug a Linux board into your network. It receives an IP address from the DHCP server in your firewall, but you have no idea what it is. If you're a normal person, you probably don't even know that your firewall has a DHCP server, and you definitely don't know how to access its admin interface to check what IPs it has dispensed. You're left in the ridiculous situation where you can see the board sitting on your desk, but you can't find it on your network.

The best solution we have at present is what is called "zero configuration networking" or "zeroconf." Apple calls their implementation "Bonjour," while the Linux people call theirs "Avahi." Apple has a zeroconf implementation for Windows, but it is not installed by default.

I propose that all Linux boards ship with Avahi set to announce its existence on the network. For the board I'm working on, the Rascal, I'd give it the hostname "rascal". If you read the serial number 246 on the bottom of the board, you can open a web browser and go to rascal246.local, and the Rascal will respond with a page that gives you basic information about the board (such as the damn IP address).

Problem 2: SSH is baffling.

The next two problems are connected.

Once you know the board's IP address, you have to log into it using a secure shell (SSH) terminal like Putty. Unfortunately, because SSH traffic is encrypted, and configuring encryption keys is confusing, SSH baffles people. The standard SSH warnings are good, because we want our systems to be secure, but they presume a level of sophistication that is not present in the normal person who is trying to connect a chicken coop to the internet.

Problem 3: Console editors are weird, alienating, and ugly.

If you do manage to figure out SSH, you then have to use an ugly console-based text editor like Nano that doesn’t support cut and paste by any conventional means.

I think we can solve the previous two problems together in one stroke. I think we need what I call "self-serve programming tools." (I've written about this in more detail previously.) Since we know these boards can connect to the internet, we should make them serve browser-based editors. On Github, I've started an open-source project to do this, based around Codemirror and the jQuery Filetree plugin. Note that I'm not talking about stuff like the cloud compiler for the Mbed, though I do think that's cool. I'm talking about hosting the code editor directly on the Linux board.

Problem 4: Embedded C, Makefiles, and gcc are scary.

I like programming in embedded C as much as the next Linux zealot, but most people prefer to sacrifice some performance in order to use higher-level languages like Python or Ruby. I think we need to have hardware APIs in multiple higher-level languages that can be called through web requests. (In the long run, I think we need server software based around event frameworks like Twisted, but I didn't actually say that in the talk.)

Problem 5: GUI builders

Often you need to use a custom GUI builder for a new embedded device with display (e.g. GTK, Qt, …). I actually like GUI builders, but the market has been badly fragmented for so long. As a result, HTML and CSS have won. As Mark Pilgrim has said: “[HTML] is the format of our age.”

There is a large swarm of amateurs who, when faced with HTML and CSS, will say, “I don’t really know how this works, but I can cut and paste it real well...” This pattern of example modification is something that tinkerers thrive on. (TCP handshake with Björn Hartmann for the apt phrase, "example modification.") Rather than resisting HTML and CSS, we should embrace them, particularly with the burgeoning acceptance of HTML5 and rapidly improving Javascript libraries like jQuery.

Problem 6: No users, permissions, sessions.

After you get your weird thing connected to the internet, you often realize that you want to have some features that aren't accessible to everyone, while still letting the general public have some access. This involves writing a bunch of tedious code, and doing it securely is not easy.

Fortunately, these problems were crushed a few years ago by web frameworks like Ruby on Rails or Django (“Python on Rails”).

Problem 7: Low fault tolerance

“I edited some files, and now it won’t boot.”

Our Linux boards should have removable media so that they can be reset to factory defaults. A normal person can probably handle downloading a default file system from the web onto an SD card and sticking that into the board.

Problem 8: Dynamic IP addresses

In the US, at least, the IP addresses of most home internet connections change periodically, which breaks the domain name system. (I'm told IPs change less in Japan.) If you want your IP address to remain the same, you have to pay around $15 per month to the people selling you the connection.

The only solution I know of to this is to use a dynamic DNS service, and a router running open source firmware like Tomato or DD-WRT. After the talk, people suggested a few other possibilities (Yaler, some VPN service like Hamachi, and something involving IPv6), but I don't have any intelligent opinions about them yet.

That's all I talked about during the presentation. If you're interested in making more powerful physical computing devices easier to use, email me at brandon at rascalmicro. If complete sentences aren't your style, try me @rascalmicro.