An Enchanting Idea

Update: Please see the vision on the Enchanting website.

---

In my previous post, where I discussed creating custom code-tiles in a variant of Scratch and using them to control devices external to the computer — namely an Arduino and an NXT robot — I finished off by saying that I would discuss where I want to go to from here.

An Enchanted Vision

I want to make a graphical code-creation tool to make it easier for kids to program robots—so that they can cast a magic spell, as it were, that makes a robot sing and dance and do as they’ve instructed.

My current plan is to build off of Scratch, which my kids enjoy working with. (The same can not be said for the NXT-G programming environment which comes with the LEGO Mindstorms NXT kits).

I envision two modes of operation:

1. Interactive Mode

In interactive mode, your robot is instructed step-by-step on what to do by the computer, and can interact with other sprites and such in Scratch.

You could, for example:

• have a robot with an ultrasonic sensor on it that spins in place, with a sonar-like display on-screen.
• adjust parameters (like speed, or degrees to turn) with sliders on screen as a program is running.
• use the NXT as an input device; for example, one could build a crank to spin a motor, and on the computer, read the motor’s encoder position to, say, reel in a fish on screen
• have a robot (or robots) be in a play with virtual actors who appear on screen.
• see what your sensors are seeing, so as to better understand and correct a program.
• or just use this method to prototype programs.

2. Autonomous Mode

When you are ready to try a program that the robot runs directly, you export the code (see this video tutorial for Scratch For Second Life to get an idea of what I mean), and from there, put it onto the robot.

The reasoning behind this is two-fold:

1. Plenty of good programming environments already exist that can be used to program the NXT (and other potentially supported platforms), and there is no sense re-inventing the wheel
2. I want to expose kids to the idea of text-based programs, so that they are able to transition to them when the time is right.

Having said that, though, I want the code to look nice so that someone could study and learn from it.

Inclusive Support

My primary focus at the moment is to help kids program their NXTs. I’d like the same environment, though, to be able to target other systems, such as the RCX, the Arduino, and perhaps things like VEX or OLLO.

Next, I’d like the software to be multi-platform and open-source. It should run on Windows, Mac OS X, and Linux.

Lastly, I’d like the software to be accessible to as wide an audience as possible. If I were designing it from square one, I’d want to ensure that you could do everything with the keyboard, everything with the mouse, and that you can make the text really large so that people with poor visual acuity can use it. I also want to ensure that text can be translated into other languages. The software should also allow users to do advanced things but in a comprehensible manner.

Where Am I At?

I’m close to the beginning of this journey. My previous post gives a proof-of-concept peek into what I want to do. I’m reading through Squeak By Example and lurking in the Scratch Advanced Topics Forum to learn how to hack on Scratch. (I should mention that I programmed video-games professionally for three years, but Squeak (the language Scratch is written in) is not widely used and different from C++ and Python).

How Can You Help?

I’m not expecting much by way of coding help (but if anyone is interested, do let me know!), and I’m nowhere near ready for testing help, but I could certainly use some thoughts on design, specifically, on what sort of code tiles would be useful to have and how to organize them.

My current though is that, if you are programming for an NXT, perhaps you should use a special NXT sprite, which supports all options available for use in programming the NXT, and omits those which are not applicable (such as changing costumes, setting the graphic effect, or determining the distance to another sprite). That being the case, though, perhaps it is better to have different categories of tiles altogether, although I’m not really sure what those categories should be. I’d love to hear your ideas on this.

I’m also quite happy to hear suggestions on what language(s?) should be used as an intermediate language for autonomous programs. I’m partial to leJOS NXJ, and certainly there is something to be said for NXC ; I can’t help but think that NXTalk would be ideal, but it is also in an embryonic state and working on Enchanting itself is more important to me. I’d like to balance ease of use, readability of code, and ease of installation, and yes, there are many, many languages to choose from.

What of the Arduino? It might be easier to get it up and working with Scratch—especially as Scratch will already talk to a PicoBoard over a serial port, and the language to output to is obvious and readable. It looks to me like the Modkit project might have things covered, and the NXT is my primary focus at this time.

Feel free to contact me or add a comment below.

Code Blocks, Electronic Bricks, and LEGO Beams

Wouldn’t it be cool if you were to be able to write a program for the Arduino in Scratch, using code something like this:

or, how about if you could program an NXT robot with code sort of like this:

Well, you can … sort of. This was what I worked on for the Western Canadian Robotics Society annual games — I entered an Art Bot that did just that, using an Arduino and some Seeed Studio Electronic Bricks and an NXT robot called Chelonius. I do have some code to share, but it’ll take more effort before it is usable by the target audience.

A Bridge Between Scratch and the Outside World

The lifelong kindergarten group at MIT who created Scratch added a really cool and useful feature: Enable Remote Sensor Connections, also known as Scratch Connections. Basically, you can right-click on the “[slider] sensor value” block (in the “Sensing” palette in Scratch) and an advanced option comes up: “enable remote sensor connections”. When you do this, Scratch opens a network socket and uses a simple protocol to communication messages. You can easily write a program in the language of your choice that will receive broadcast messages from Scratch, changes to global variables, and can tell Scratch that a remote sensor exists and when its value changes. The Scratch Connections site has details on how to activate the feature and communicate with Scratch from Python, Processing, or Java.

Chalkmarrow wrote one of these helper applications that bridges Scratch and the Arduino. The application is called Catenary. Unfortunately, to use it from Scratch, you have to create some ugly broadcast commands which are not easily editable at all.

And that’s where BYOB (Build Your Own Blocks) saves the day. Jens Mönig has taken the source code for Scratch and extended it so that you can create your own code blocks. With Brian Harvey, he’s been working towards BYOB version 3.0, which will be used to support an introductory computer science course at Berkeley. (That is so cool! I can’t say enough good things about Scratch, and BYOB really extends the possibilities of what can be done with it.)

Throw In Some Custom Blocks

The beautiful thing about this is that I can now create code blocks for Scratch that look like regular code blocks, but make it way easier to communicate with Catenary (or any other program like it). I have a sample application designed to run in BYOB version 2.99.007. As BYOB is undergoing rapid changes and is still in alpha, and so that you can better experiment with creating blocks of your own, I’d like to show you the blocks I’ve created — both what the block looks like, and what the procedure is inside the block.

The core block is this one:

and (although the end user really doesn’t need to know or care), the procedure inside looks like this:

It takes a list of commands and a filter character. Catenary filters out (ignored) all broadcast messages, unless they start with a caret (’^’) character, in which case, Catenary knows it is a special command for it to act upon. Thus, the caret is the filter character.

At a slightly higher level of abstraction, we have this block:

which simply takes the list of commands supplied and puts the appropriate filter character in front, like this:

[Below, you’ll see that I used a very similar command, with a different filter, for sending instructions to a custom robot.]

Here, then, are all the blocks that I’ve created for use with the Arduino, via Catenary.

Block	Procedure

All in all, it is pretty slick. I did want to note that I’ve had trouble reading inputs. When you, say, set pin 10 to be an input, then you are supposed to be able to read the value via a sensor block:

I had some success with digital inputs, but not with analog inputs. It is not clear to me if I’ve made a mistake or need to study the Catenary code further.

Five other points with using Catenary:

1. Before using it, you’ll need to load the StandardFirmata sketch onto your Arduino board.

2. You will need to know the port number that Catenary uses to address the Arduino. You can either run Catenary within Processing, where it prints a list of the serial ports, or, look at the menu within the Arduino environment, and count up from zero until you find your serial port (as indicated by the numbers in red). For me, the Arduino shows up on port 1 (and hence my programs all start with “control arduino on port 1”).

3. After you’ve uploaded the StandardFirmata sketch and figured out which port you are using, close the Arduino environment. (This frees up the serial port so that Catenary can talk to it.)

4. Finally, make sure that BYOB is running and then run Catenary. It will say that it has connected to Scratch and that the Arduino port has not been specified. (If it does not say that it connected to Scratch, make sure that Remote Sensor Connections are enabled).

5. I’ve hard better success running Catenary within Processing than from running the stand-alone executables that come with it. (I’ve only tried on a machine running OS X 10.5, mind you). If you opt to run it that way, you’ll need to download Processing (which is what the Arduino environment is based on, so it shouldn’t be too unfamiliar), and download and install the library that let’s Processing talk to an Arduino.

You should now be able to double-click on blocks and have things happen on your Arduino.

The NeXT Step

I also wanted to be able to control my LEGO Mindstorms NXT from this Scratch-like environment, and the LeJOS environment — which allows you to program your NXT using Java, and allows you to communicate with it from your computer in Java — is an excellent way to go.

The project had three main parts:

1. The BYOB Code Tiles for the robot
2. A custom Java program, called “TalkToBot” which uses Dr. Andrew Davison’s library for talking to Scratch from Java, and
3. A custom LeJOS application, called ArtBot, running on my robot, Chelonius.

By all means, download the source code. I make no claims that the code is any good, but I did want to get it up here for anyone else who wanted to explore. Also note that it is not general purpose; it only works with a robot running the specific program that is included. You will need to have LeJOS NXJ installed on your computer and the LeJOS firmware on your robot, to make any use of this code.

As my time was very limited, I developed a simple communication method: all of the blocks would issue a command and specify two (floating-point) numbers. They’d also move the sprite on screen a little bit.

There, we have the following code tiles:

One thing I really like about the LeJOS libraries for the NXT is that they easily allow you to tell the robot to go a certain distance or turn a certain amount. (Mind you, it really requires some tweaking to get things exact.)

How To Run The Application

These instructions assume:

1. You have BYOB 2.99.007 (or newer?) set up on your computer.
2. You have LeJOS NXJ 0.85 (or newer?) set up on your computer and NXT.
3. You aren’t scared of using a command prompt
4. You have paired your NXT and computer for bluetooth communication.

Here, then, are the instructions.

1. Start BYOB and load the script found in BYOB/Base Commands.ypr in the source archive.

2. Compile and upload the program for the robot.

Issue the following commands, at a command prompt, starting in the root directory of the source archive. (Skip any lines starting with a #).

# Go into the NXT code directory
cd NXT
# Compile the ArtBot program
nxjc ArtBot.java
# Upload the ArtBot program to your robot (via USB or Bluetooth)
nxj ArtBot
# Return to the main directory
cd ..

3. Run the computer program.

From a command prompt,

cd PC
# compile the TalkToBot application and Scratch connection libraries
nxjpcc *.java
# run the TalkToBot program
nxjpc TalkToBot

The program should indicate that it has connected to Scratch, and then it’ll try to make a Bluetooth connection to the NXT.

After that happens, every time you double-click on a block with ‘robot’ in the name in the Scratch-like environment, you’ll see a message in your terminal window and on the robot, and the robot will execute the command.

I’m sorry that I don’t have a video to post, as seeing it in action is way better than taking my word for it. Unfortunately, my robot Chelonius has already been dismantled for use in my next project.

In my next post, I’ll discuss where things go from here.