Introduction
Last year O'Reilly Media announced a new platform for interactive, online tutorials called Oriole. Oriole combines video, text and code into a cohesive, engaging medium for learning about coding. O'Reilly offers a free example to show you how it all works. When you start an Oriole, you're immediately presented with a video at the top of the page. As the video progresses, the video slides to the side of the page (while still playing) and scrolls to the relevant content that the video is talking about. Eventually, the video gets to talking about code and you're brought to an interactive IPython widget that can run Python scripts right there in your browser. As the video is teaching you something about coding, you can be right there with the narrator coding up your own script. What a great way to learn!
At the time, I was working at the Children's Museum of Bozeman in their STEAMlab teaching many classes about Arduino. After seeing the Oriole demonstration, I immediately thought about how useful this medium could be for teaching Arduino. I'd heard of Codebender, a site that allowed you to program your Arduino from the browser using a Chrome or Firefox plugin, but hadn't really tried it out at the time. Could it be used for this type of Oriole experience? The IPython widgets were great because you could run the code and see your program's output. With Arduino, though, your output is either physical results such as turning on an LED or printing data over serial communication. This was going to be the test. Could I get access to serial data using Codebender's plugin. I immediately started hacking on it to see what I could do. I got pretty far with my tests running on my local machine.
When uploading the example to a live site, though, it broke. This made sense to me. I was sure there was some kind of domain restrictions on the Chrome plugin to ensure Codebender was the only site that could access their plugin. I reached out to Codebender to see if they had some kind of public API that I could use, but only had a generic support email to go through. I didn't receive a response, but I recorded the video above and made some emails to O'Reilly (I've worked there before, so I knew who to contact directly). I was able to get in contact with the guys managing Oriole, Paco Nathan and Brian Jepson. They were thrilled with the idea of combining Oriole with Arduino and they were able to put me in contact with the CEO of Codebender, Vasilis Georgitzikis. Yay!
It turned out Codebender was working on a public API and they were going into beta very shortly (this was just about 1 year ago). I tabled the idea at that point, and periodically reached out to Codebender for updates. Back in October, Vasilis sent me the bad news, Codebender was shutting down codebender.cc, and was focusing their business efforts on more profitable markets. Unfortunately, that meant the most important piece of this puzzle was just lost. Without codebender.cc to provide a clean web IDE and their plugin to perform the upload and serial communication, this became a lot more work.
Arduino Create was in beta about this time last year and was an alternative to using Codebender. Arduino Create is the official Arduino web IDE that does essentially what codebender.cc did but uses very different technology. Instead of a browser plugin that provides access to the serial port, Arduino Create runs a local web server called the Arduino Create Agent, which is a fork of John Lauer's, Serial Port JSON Server (SPJS). Using WebSockets, you can communicate with the web server and get access to the data on the serial port as well as upload programs to a connected Arduino. Unfortunately, Arduino Create doesn't provide the functionality I was looking for. You can embed code that you have stored on Arduino Create, but you can't interact with that code, or compile it without signing up for an account with Arduino and heading over to their site. When I asked about this on their forums, I was told this is for security reasons and to avoid abusive behavior by anonymous users.
Fast forward to now. I've launched my own blog with intent to create plenty of Arduino tutorials and would like to have an easy way to upload code to an Arduino and use it to interact with widgets all from my blog posts. Codebender is shutting down, so that's out. Arduino Create lets me embed code on my site, but doesn't provide a way to interact with it or upload it to a connected Arduino without leaving my site. What I really want, though, is to have everything nice and cohesive on my site just like O'Reilly's Orioles. I was going to have to just do it myself. It was more work than I initially hoped for, but I managed to get a proof of concept up and running.
The Plan
I had a rough idea of what needed to happen:
1) Create a Heroku app
A Node.js app with two processes. One web process that would accept and report on the status of Arduino script compilations. The web process will communicate with a worker process that will perform the compilations. This seemed totally doable if I could leverage the Arduino executable to do all the work for me (I don't really want to get into all the details of avrgcc when one arduino command will compile an entire project). I just had to configure my Heroku app to ship with the latest Arduino binaries, and give it a thin REST wrapper so I could tell it what to do.
2) Implement a simple code editor
The Ace code editor is easy enough to set up and provides syntax highlighting for many different languages. I just needed to add some additional controls to select an Arduino and a button to trigger an upload. Ideally, there will also be some error checking for when users have compilation errors. That seems doable given the arduino command will generate the errors, I just need to report them back to the user. Once the compilation succeeds, the generated hex file just needs to be sent to the Arduino some how. Which brings us to:
3) Leverage Arduino Create Agent for upload and serial communication
Ideally, I could leverage the Arduino Create Agent that many people already have installed to perform the upload. Worst case, though, I could fork the project and adjust it as necessary and deploy my own version of it.
First Steps
The other day I decided to see where I could get with that plan in mind. I started by creating a a Heroku app using the Node Articles NLP example as a starting point (you can see all my code here). The Node Articles NLP app had all the infrastructure in place for a work queue. A web process adds to the queue and a worker process does all the heavy lifting. After a couple days of hacking on the Heroku app and the Arduino Create Agent, I was able to get demo from the video above working. It's definitely rough around the edges, but the proof of concept is there. In order for the demo to work for you, you'll need to have the Arduino Create Agent installed and an Arduino Uno or and Arduino Micro (I plan to allow more boards, but this is what I managed to get up and running initially and are what I have to test with). You'll also want to wire up 2 potentiometers to analog pins 0 and 1.
You can download the Arduino Create Agent installer for your OS here. Once it's installed, you'll also need to edit its config.ini file to give Allwine Designs permission to access it. On Mac, the file is located at ~/Applications/ArduinoCreateAgent/ArduinoCreateAgent.app/Contents/MacOS/config.ini. I've only tried this on my Mac so far, so I'm not sure where to find it for Windows and Linux, but I imagine if you're still reading this you'll be up for a bit of exploration (if you find it, let me know and I'll update this post). The file is called config.ini and you'll need to edit the origins line to look like this:
origins = http://webide.arduino.cc:8080, https://arduino-compiler.herokuapp.com, https://allwinedesigns.github.io
Then you'll have to restart the Arduino Create Agent. The two domains that you added to the list are my GitHub Pages domain where I plan to store widgets for interacting with Arduinos and the Heroku app's domain that serves the code editor and uploads the code.
Update 3/1/2017: I've implemented the Arduino editor in React. In doing so, I made it much clearer which port you're connecting to and provide some indicators as to what's happening when you click Update. I'd say we've moved into alpha for the editor, but I still need to revamp the demo as well.
Make sure you only have a single device connected. Right now I'm assuming the first listed device is the one you want to connect with. I know that's a bad assumption to make, but in my experience that's the way it is most of the time. Select the Arduino that you have hooked up, either an Uno or a Micro. Then click Upload. There isn't any feedback currently, but your device's TX and RX LEDs should start flashing within a few seconds. This is worse than alpha software, though, so no promises. Once it finishes uploading continue to the widget below. Just click the Connect button (try a couple times if it doesn't change to a Disconnect button). If all goes well, you should be able to turn your potentiometers and see the green bars move up and down. You can also try editing the code above (just make sure you don't make any syntax errors, as it won't tell you about them yet). You'll need to make sure you've disconnected the widget first. Try switching the 0 and 1 in the analogReads to reverse how they affect the green bars.
This is just the beginning of this process. I plan to flesh out these demos so they're ready for more widespread use on my site, but there's still plenty to do before that can happen. If you try it out, let me know how it goes!