A while ago we implemented something we called mux-mode for controllers, where the Crazyflie can be controlled from multiple controllers at once. Initially it was implemented the day before a “bring-your-kids-to-work” day at Minc (where our office is). The idea was that the kids would control roll/pitch from one controller and we would control thrust/yaw from another controller. But we would also have the possibility to take over roll/pitch by holding a button on our controller. It was a big hit and let’s just say the “take-over” functionality came in handy :-)

A couple of months later we started working with the Kinect v2 with the goal of automatically piloting the Crazyflie using it. Again the input-mux feature came in handy. Instead of having the kids controlling the roll and pitch, the autopilot was now doing it. This enabled us to work on one problem at a time, first roll/pitch then yaw and finally thrust. When we were finished the autopilot was controlling all of the axes and we just used the “take-over” functionality when things got out of control.

So far this functionality has been disabled by default, but last week we fixed it up and enabled the code. With this change we’ve renamed the feature to teacher/student and also changed the way mappings are selected for the client. Below is a screenshot of the new menu, but have a look at the wiki for more details. If you want to try it out, pull the latest version on the development branch. It’s a great feature if you know someone that want’s to try flying for the first time!

On a side-note we tried some other ways to mix the controllers, like one we called “mix-mux”. This would take the input from two devices and add them together, so if both give 25% thrust the total would be 50%. It was really fun to try, but impossible to fly (maybe we need to work on our communication skills…).

Recently I got a Chromebook, mostly out of curiosity for this odd “computer that runs only a web browser”. While playing with Google dev tools I quickly saw a possibility to make a Crazyflie client as a Chrome app: the Chrome API provides USB connectivity,and HTML5 has a javascript gamepad API. A chrome app is designed to look and feel like a native application: the app does not require internet connection and is launched in its own window.

This week-end I finally got around to test it, it’s not pretty but it works :-)

The current functionality is:

• Channel and datarate can be changed
• Read input from a gamepad, the mapping is fixed to mode 3 and the sensitivity is fixed
• Sends set-points to Crazyflie 33 times per seconds.

I haven’t had time to do any layout work on it (that is pretty obvious in the screenshoot :) ), but the plan is to use Angular Material to handle the GUI.

This is only a proof of concept but we are seeing a lot of potential: the Chrome app runs on Linux, Mac, Windows and Chromebook, is easy to install and is written in HTML/CSS/Javascript which seems to be a very popular platform nowadays.

I have pushed the code on Github so if anyone is interested in helping to shape up the app head to the forum to read the discussion about it.

Lately we have merged a new input subsystem to the Crazyflie python client, it allows for more flexible input configuration like connecting more than one gamepad in training mode, or having external input like the LeapMotion appear as standard input. To test the flexibility of the new input system, on Friday, we implemented a ØMQ input driver. ØMQ is a library that permits to easily transmit messages between program. It is high speed, low latency and extremely easy to use. It has binding and implementation for a lot of programming language, which opens Crazyflie control to a lot more people and application.

The way it currently work is: You connect the graphical Crazyflie client and select ZMQ as input device. You connect the ØMQ socket and send json to it containing pitch/roll/yaw/thrust. And voila, you are controlling Crazyflie from your own program.

We just got a Kinect 2 and started playing around with it, suddenly the new ØMQ control became very useful. We already worked with the kinect long time ago so we had control code from back then. Also we wanted to quickly do a proof of concept so we started working with the Microsoft SDK to see what is possible, we are going to use libfreenect2 on Linux later. Finally we know how hard it was to control the thrust so we wanted a way to test pitch/roll first as a proof of concept. At the end of the day we ended up with this architecture in the office:

The C++ implementation in visual studio detects the Crazyflie and sends coordinate to the Controller python script. The controller script is a stripped down version of or previous kinect experiment and only runs PID control loops to control pitch and roll, trying to keep Crazyflie at a fixed coordinate in space. Finally the Crazyflie client is setup in training mode with ZMQ handling pitch and roll and the gamepad handling thust and yaw. The gamepad can take-over completely in case of problem.

Thanks to ZMQ resiliance we can stop any part of the system and start it again, the connections automagically reappears. So we ended up with the following workflow:

• Fly in the Kinect detection area
• hand-over pitch/roll to the controller loop
• When the Crazyflie starts oscillating or going away, take over control and land
• Stop controller, modify things, restart

This was quite painless and nice. We ended-up connecting together very different systems and they just worked. We got so interested by the experiment that we now have a full ZMQ Crazyflie server on the work that would allow other program to do what is possible with the standard Crazyflie API: scan for Crazyflie, connect, read and write log and params.

When the schools closed for vacation last week Minc (the company accelerator where we have our office) hosted a “bring-your-children-to-work” day. The original idea came from Fredrik who works at the Swedish Arduino office (in the same building as us). One of the goals whas to let children see cool technology and to get a chance to try it out. We quickly jumped on this since we thought it would be fun for the children to fly. But we realized there was a bit of a safety problem letting Crazyflies go wild all around the place. So we dug up an old idea we had been discussing (yes, there’s lots of those..), the input-device MUX.

The idea is to allow combinations of input-devices to work together at the same time. There’s a couple of use-cases for this that we have been discussing in the past, like combining Kinect with a controller so you can take control if you loose tracking. But the most obvious one is a “learning” mode, where you could control roll/pitch from one device and thrust/yaw from another. Or having the possibility to take over control “on-the-fly” (no pun intended…) from one device to another. For us it makes it a lot easier to give a controller to people and say “Try it out!” if we can take over control just in case things start going bad.

So we started hacking around a bit and got it working, but there’s still lots of work to be done. We revised the architecture a bit for the input-device layer in the Crazyflie Python client from what we posted before. The main change is the multiplexer, which enables the user to open multiple input-devices at the same time and combine them. The other change is to connect other devices “higher-up” in the architecture, like the Leapmotion. Instead of connecting it as an input-device with a mapping, it is connected directly to the MUX and will give well scaled values for controlling. The same goes for other devices such as the Kinect and network connections (which would allow to control Crazyflie throw the client from any other software).

The hardest part will not be to code this, it will be to design the UI and configuration of this functionality. We have some sketches, but any ideas are welcome! In its current state you just select a MUX, then the devices that you want to use (master first, then slave). But there’s no way of configuring what values are taken from what device.

Check out this album to see a few photos from the “bring-your-children-to-work” day!