Category: Software

DWM1000 nodes
Last hacking-Friday we have had some time to put together the DWM1000 boards we ordered during the summer. The DWM1000 from Decawave is an ultra-wide-band ieee802.15.4 radio transceiver that can very precisely timestamp packets arrival and departure. More simply it means that it is a standard and it can be used to implement a real time local positioning system: this could be really handy for the Crazyflie. We soldered all the boards and we got some basic ranging working on the nodes. The next step is to implement an opensource driver to be able to implement the ranging in the Crazyflie. We will keep you updated on the progress but in the mean time here is a photo of the prototypes:

DWM1000 nodes and deck

 

Things happening on the firmware side
Recently the commit rate for the Crazyflie 1.0/2.0 firmware has increased a lot. Some of it because of pull requests, great work, and some because we are starting to move in hacks and such on feature branches into the master branch. Our new college Kristoffer has taught us that having stuff on feature branches can be a bad idea, they tend to stay there. It is then better to have them compile switched and in the master branch as it is more visible and get a better chance of getting in for real.

Github crazyflie firmware contributions

 

Here are some of the recent thing going on:

  • A situation awareness framework originating from this pull request by fredgrat. It allows the Crazyflie 1.0/2.0 to react to triggers. Currently there is a free-fall, tumbled and at-rest detection. He recently also submitted an auto-takeoff functionality. Enable the functionality with the define here.
  • The beginning of a Arduino like API for the deck port. Currently GPIO and ADC are the only functions there but more will come.
  • Possibility to fly in rate (acrobatic) mode committed here. Support in the cfclient for this is being developed so currently one have to change the parameters to activate it manually.
  • Carefree, plus and X-mode implemented in firmware. There is also support for this being added to the cfclient.
  • Automatically switch to brushless driver. Motor driver being rewritten so it can be dynamically configured. This means that if the Crazylfie 2.0 is attached to the big-quad-deck it can automatically switch over to the brushless driver during power on.

Summertime are good times, less administration and more time to develop! As soon as things has been integrated and fully tested we will do a new release of the firmware and the cfclient :).

A while ago we implemented something we called mux-mode for controllers, where the Crazyflie can be controlled from multiple controllers at once. Initially it was implemented the day before a “bring-your-kids-to-work” day at Minc (where our office is). The idea was that the kids would control roll/pitch from one controller and we would control thrust/yaw from another controller. But we would also have the possibility to take over roll/pitch by holding a button on our controller. It was a big hit and let’s just say the “take-over” functionality came in handy :-)

A couple of months later we started working with the Kinect v2 with the goal of automatically piloting the Crazyflie using it. Again the input-mux feature came in handy. Instead of having the kids controlling the roll and pitch, the autopilot was now doing it. This enabled us to work on one problem at a time, first roll/pitch then yaw and finally thrust. When we were finished the autopilot was controlling all of the axes and we just used the “take-over” functionality when things got out of control.

So far this functionality has been disabled by default, but last week we fixed it up and enabled the code. With this change we’ve renamed the feature to teacher/student and also changed the way mappings are selected for the client. Below is a screenshot of the new menu, but have a look at the wiki for more details. If you want to try it out, pull the latest version on the development branch. It’s a great feature if you know someone that want’s to try flying for the first time!

On a side-note we tried some other ways to mix the controllers, like one we called “mix-mux”. This would take the input from two devices and add them together, so if both give 25% thrust the total would be 50%. It was really fun to try, but impossible to fly (maybe we need to work on our communication skills…).

Continuing from last Monday post where the hardware wiring part was discussed we now move on to the software side. The brushed motors are controlled with a normal PWM  where the duty cycle will adjust how much power goes into the motors. A brushless motor on the other hand needs more complicated controlling and uses it’s own micro-controller to handle this. These brushless motor controllers (BLMC or Brushless ESC) comes in many flavors and sizes but what is pretty common is how they are interfaced/controlled. This is inherited from how R/C receivers are controlling servos and how the receiver gets updates from the transmitter. This is a PWM where the width of the high pulse define the duty cycle. 1ms equals min and 2ms equals max and this is repeated every 20ms, thus giving an update rate of 50Hz.ServoPwm

This way of interfacing the BLMC is currently the most common way but interfacing with I2C, CAN, etc is getting more common.

To generate the servo PWM on the Crazyflie we have just reconfigured the timer a bit using a conversion macro so that setting the motor ratio of zero will result in a 1ms high pulse and setting it to max (uint16) will result in a 2ms pulse. The period time can be set with the BLMC_PERIOD define in motors.h. The standard period time of 20ms is actually a big drawback as it adds a latency from when a new output is calculated to when it is actually set. Therefore many motor controllers allow to shrink this period down to 2.5ms (400Hz) which result in lower latency and better flight stability.

Brushless prototype board

First you need to put together the brushless prototype board from the last post. The output will be generated on the pins marked BLMC 1,2,3,4 which should be connected in the same position an rotational direction as the brushed M1, M2, M3, M4 respectively. The output signal will be 0v – 3.0v which should work fine with 5V BLMC but it might be worth keeping that in mind.

BL proto descr

Building the brushless firmware

The code which contains the brushless functionality is currently on the bigmerge branch so start by pulling the latest changes and switching to that branch.

git pull
git checkout bigmerge

Then to activate the brushless functionality enable the brushless defines

BRUSHLESS_MOTORCONTROLLER
BRUSHLESS_PROTO_DECK_MAPPING

This can be done by by either creating defines in config.h or by creating  aconfig.mk file in the same directory as the Makefile with the content:

CFLAGS += -DBRUSHLESS_MOTORCONTROLLER
CFLAGS += -DBRUSHLESS_PROTO_DECK_MAPPING

The period time BLMC_PERIOD is by default set to 2.5ms (400Hz) so change that if needed.

Then build the firmware (make sure to clean first)

make clean
make

It will build for the Crazyflie 2.0 and for wireless bootloading by default. Put the Crazyflie 2.0 in bootloader mode by holding the power button until the blue led start to blink, then flash it with the wireless bootloader (Crazyradio required).

make cload

Causion!

You are now probably dealing with powerful and dangerous stuff so make sure to take precautions. E.g. don’t have propellers mounted when you test! When you have taken all the safety precautions do your first test. Remember the Crazyflie firmware has not yet been developed for big quads, it is all at your own risk! And another thing, even though the Crazyflie 2.0 can be controlled using a mobile device we don’t recommend this, use a Crazyradio.

Tuning

With that cautions being said it is great seeing a big quad fly and we will soon put out a video showing some of our builds. Our bigger quads have flown quite well with the stock tuning (PID parameters) but tuning should be done as well. Plenty of guides can be found on ways how to do it so I will not go into details here. The values can be found in pid.h and can be updated (but not saved) live using the cfclient. To save them the pid.h file must be changed and the firmware flashed again.

Recently I got a Chromebook, mostly out of curiosity for this odd “computer that runs only a web browser”. While playing with Google dev tools I quickly saw a possibility to make a Crazyflie client as a Chrome app: the Chrome API provides USB connectivity,and HTML5 has a javascript gamepad API. A chrome app is designed to look and feel like a native application: the app does not require internet connection and is launched in its own window.

This week-end I finally got around to test it, it’s not pretty but it works :-)

crazyflie_chrome_client

The current functionality is:

  • Channel and datarate can be changed
  • Read input from a gamepad, the mapping is fixed to mode 3 and the sensitivity is fixed
  • Sends set-points to Crazyflie 33 times per seconds.

I haven’t had time to do any layout work on it (that is pretty obvious in the screenshoot :) ), but the plan is to use Angular Material to handle the GUI.

This is only a proof of concept but we are seeing a lot of potential: the Chrome app runs on Linux, Mac, Windows and Chromebook, is easy to install and is written in HTML/CSS/Javascript which seems to be a very popular platform nowadays.

I have pushed the code on Github so if anyone is interested in helping to shape up the app head to the forum to read the discussion about it.

 

As many out there think is is more fun to fly the Crazyflie rather then develop upon it (like we do :-)) we have quickly looked at ways to pilot it with a RC transmitter. This forum thread is a good starting point for a developer discussion. To summarize it a bit there are many ways of implementing it which all require more or less development and has different pros/cons

  • Attach the Crazyradio PA to the expansion module/port of an RC transmitter.
  • Implement the E-Sky transmitter protocol or other nRF24L01+ protocols in the nRF51.
  • Using the DeviationTx code and a nRF24L01+ module.
  • Attaching a RC receiver to the deck (expansion port) interface.

Today I will write a bit more about the DeviationTx as it is a great open source project and that it has support for the Crazyflie. It was over a year ago we got contacted by Victor who wanted to implement the Crazyflie protocol in the DeviationTx code base, which he did pretty quickly. We feel ashamed for taking so long to try it out. So during the weekend I freed up some time and gave it a shot. The DeviationTx project replaces the firmware in Walkera transmitters with a better one which has the possibility to support a great amount of RC models. However many of them use different transceivers modules so this must be added to the hardware. Well a bit of hardware hacking is always fun and we had a nRF24L01+PA module laying around. They have a module installation document but I found it easier follow this guide as I had the same type of module and transmitter (Devo7e).

devo7e nRF24L01 module

The module installation was done pretty quickly but what took time was to update the firmware. The instructions tells one to download the walkera update tool but I just couldn’t find it on their website, nor the original walkera devo7e firmware (which they recommend to test with first). Thankfully I could find one using google and ended up using this link. Next thing was to fire up a windows 7 virtual machine to install it in which it worked without problems. So did the flashing of the DeviationTx firmware, I flashed the nightly build 4.0.1 and copying the file system according to the instructions. What I forgot though was to edit the hardware.ini file to enable the module which I understood when the protocol selection had a star in front of the name <*CFlie> (OK, I admit, I thought the hardware wasn’t working at first…). Then I setup a new model using the <CFlie> and I used the Fixed ID to setup the address, The data rate and channel are combined in the fixed id using channel as lowest two decimal digits and the rate the first were 0, 1, and 2 for 250kbit/s, 1Mbit/s, and 2Mbit/s respectively. So channel 80 on 2Mbit/s is encoded as 280 and channel 80 on 250kbit/s as 80.

And boy I was happy when I saw the green led (radio com led) blink on the Crazyflie 2.0, but nothing happened when I pushed the trust… Then I remembered, we recently implemented a lock so that a zero thrust must be sent at least once for it to unlock (to prevent Crazyflie to fly away if the gamepad is not setup correctly and constantly sends 100% thrust :). Looking into the source code one could see the trust was at minimum 5535 which was never unlocking the thrust. Removing this lock in the Crayflie 2.0 firmware and it was flying! but in plus mode… The Deviation cflie module code seem to rotate the pitch/roll setpoints.

Next step will be to do some modifications to the Deviation cflie module for Crazyflie 2.0, adding code to unlock the thrust and disabling the axis rotation, then make a pull request to the DeviationTx project for everyone to enjoy.

A big thanks to the DeviationTx project and to Victor for the cflie module implementation! Ohh, and by the way, they are making a universialTx module that will support almost any RC model out there, including Crazyflie, can’t wait to see that.

Lately we have merged a new input subsystem to the Crazyflie python client, it allows for more flexible input configuration like connecting more than one gamepad in training mode, or having external input like the LeapMotion appear as standard input. To test the flexibility of the new input system, on Friday, we implemented a ØMQ input driverØMQ is a library that permits to easily transmit messages between program. It is high speed, low latency and extremely easy to use. It has binding and implementation for a lot of programming language, which opens Crazyflie control to a lot more people and application.

The way it currently work is: You connect the graphical Crazyflie client and select ZMQ as input device. You connect the ØMQ socket and send json to it containing pitch/roll/yaw/thrust. And voila, you are controlling Crazyflie from your own program.

We just got a Kinect 2 and started playing around with it, suddenly the new ØMQ control became very useful. We already worked with the kinect long time ago so we had control code from back then. Also we wanted to quickly do a proof of concept so we started working with the Microsoft SDK to see what is possible, we are going to use libfreenect2 on Linux later. Finally we know how hard it was to control the thrust so we wanted a way to test pitch/roll first as a proof of concept. At the end of the day we ended up with this architecture in the office:

zmq_kinekt

The C++ implementation in visual studio detects the Crazyflie and sends coordinate to the Controller python script. The controller script is a stripped down version of or previous kinect experiment and only runs PID control loops to control pitch and roll, trying to keep Crazyflie at a fixed coordinate in space. Finally the Crazyflie client is setup in training mode with ZMQ handling pitch and roll and the gamepad handling thust and yaw. The gamepad can take-over completely in case of problem.

Thanks to ZMQ resiliance we can stop any part of the system and start it again, the connections automagically reappears. So we ended up with the following workflow:

  • Fly in the Kinect detection area
  • hand-over pitch/roll to the controller loop
  • When the Crazyflie starts oscillating or going away, take over control and land
  • Stop controller, modify things, restart

This was quite painless and nice. We ended-up connecting together very different systems and they just worked. We got so interested by the experiment that we now have a full ZMQ Crazyflie server on the work that would allow other program to do what is possible with the standard Crazyflie API: scan for Crazyflie, connect, read and write log and params.

 

When the schools closed for vacation last week Minc (the company accelerator where we have our office) hosted a “bring-your-children-to-work” day. The original idea came from Fredrik who works at the Swedish Arduino office (in the same building as us). One of the goals whas to let children see cool technology and to get a chance to try it out. We quickly jumped on this since we thought it would be fun for the children to fly. But we realized there was a bit of a safety problem letting Crazyflies go wild all around the place. So we dug up an old idea we had been discussing (yes, there’s lots of those..), the input-device MUX.

The idea is to allow combinations of input-devices to work together at the same time. There’s a couple of use-cases for this that we have been discussing in the past, like combining Kinect with a controller so you can take control if you loose tracking. But the most obvious one is a “learning” mode, where you could control roll/pitch from one device and thrust/yaw from another. Or having the possibility to take over control “on-the-fly” (no pun intended…) from one device to another. For us it makes it a lot easier to give a controller to people and say “Try it out!” if we can take over control just in case things start going bad.

So we started hacking around a bit and got it working, but there’s still lots of work to be done. We revised the architecture a bit for the input-device layer in the Crazyflie Python client from what we posted before. The main change is the multiplexer, which enables the user to open multiple input-devices at the same time and combine them. The other change is to connect other devices “higher-up” in the architecture, like the Leapmotion. Instead of connecting it as an input-device with a mapping, it is connected directly to the MUX and will give well scaled values for controlling. The same goes for other devices such as the Kinect and network connections (which would allow to control Crazyflie throw the client from any other software).

input-arch-mux

 

The hardest part will not be to code this, it will be to design the UI and configuration of this functionality. We have some sketches, but any ideas are welcome! In its current state you just select a MUX, then the devices that you want to use (master first, then slave). But there’s no way of configuring what values are taken from what device.

Check out this album to see a few photos from the “bring-your-children-to-work” day!

The Crazyflie 2.0 supports two types of radio protocols out of the box: ESB (Enhanced ShockBurst) for Crazyradio/Crazyradio PA USB dongle compatibility and  Bluetooth LE for mobile devices. All Crazyflie 2.0’s are shipped with a radio bootloader, able to update virtually all the different parts of the firmware running in Crazyflie 2.0. We thought we would do a post to explain a bit further how it all fits together. Even though the Crazyflie 2.0 supports Bluetooth LE, the preferred way is to use the Crazyradio/Crazyradio PA when doing development. Flying via Bluetooth is very practical since there’s no additional hardware needed except for a mobile device, but the Crazyradio/Crazyradio PA has minimal latency, more bandwidth and works easily with computer. This makes it ideal for development and other advanced usage.

(On a side note, some of our products are currently out of stock (like the Crazyradio PA and LED-ring). We have started the next batches, so they should arrive soon. Until then have a look at our distributor page to find our products.)

Crazyflie 2.0 architecture

For the radio of Crazyflie 2.0 we choose the nRF51822. The great thing with this chip is that it integrates a radio compatible with our existing Crazyradio USB dongle as well as with Bluetooth Low Energy. The nRF51 also integrates an ARM Cortex-M0 MCU enabling radio protocols to be implemented directly in the chip. However this MCU is not powerful enough to be used by itself in our quadcopter development platform, so we put a powerful STM32F4 Cortex-M4 168MHz on the side to do the heavy work.

The nRF51 mainly handles radio and power management and the STM32F4 handles all the rest: flight control, expansion port, log, param, etc. As far as the radio is concerned the nRF51 act as a bridge: CRTP packets are received by the nRF51 and sent unmodified to the STM32F4 and the STM32F4 sends raw CRTP packets to the nRF51 that transmit it by the radio. This means that the STM32 firmware is unaware of the physical communication protocol, it can be Bluetooth, Shockburst or something else, it makes no difference it is still CRTP packets. CRTP is the protocol used to control Crazyflie, it encapsulates all commands and messages exchanged between the client and the Crazyflie.

Enhanced Shockburst (ESB)

Enhanced Shockburst is a radio physical protocol implemented in some Nordic Semiconductor 2.4GHz radio chips. In the nRF24 chip, used by Crazyradio (PA) and the original Crazyflie, this protocol is implemented in hardware and we have little control over it. The nRF51 used in Crazyflie 2.0 offers more control on the radio physical packet and ESB since it’s implemented in software instead.

ESB handles ack and retries: The Crazyradio sends a packet on a given channel and waits for an ack. If a Crazyflie receives the packet without error it sends an ack packet. If an ack is received, the Crazyradio can send the next packet. If no ack is received Crazyradio will automatically retry by sending the same packet again. The current implementation will retry to send a packet forever, so all packet are guaranteed to be transmitted as long as a Crazyflie is in range. Packets are sent back by the Crazyflie by adding a data payload to the ack packet. Close to 80 different channel can be used and every packet is sent with an access address, so more than one Crazyflie could share the same channel.

Enhanced Shockburst using the Crazyradio (PA) offers the lowest latency that can be expected. The minimum latency to send a packet is estimated to about 2ms (1ms minimum for USB and 1ms measured latency for the radio at 2Mbps without any retries). With the Crazyflie 2.0 and the Crazyradio PA it also offers the maximum range with 20dBm power output.

For the original Crazyflie there is an implementation of the E-sky RC transmitter protocol. This permits controlling the Crazyflie directly from an RC transmitter. It is technically possible to implement this protocol in the Crazyflie 2.0 nRF51, but it hasn’t been done yet.

Bluetooth Low Energy

In the Crazyflie 2.0 Bluetooth low energy is implemented as a CRTP bridge: CRTP packet are written and read using Bluetooth. This simplifies greatly the implementation, but does not make full use of the Bluetooth LE functionality. It is planned to add more Bluetooth capabilities for things like communicating the battery status and being able to switch ON or OFF the Crazyflie with a mobile devices.

On the nRF51 side the Bluetooth functionality is implemented using the Nordic Semiconductor S110 Bluetooth stack. This stack runs independently of the firmware. Nordic calls this a softdevice and it’s a fitting description of how it works. It almost looks like a hardware device from the firmware point of view as it is not linked into the main firmware.

However we do not currently have the right to distribute the supporting files for the stack, so unfortunately you will have to download them yourself if you want to compile your own nRF51 Bluetooth firmware. To do so you must own a Nordic Semiconductor development kit. Even though some of the kits are pretty cheap, we do not like this situation at all so we are working on solving it. However we made sure that it is possible to compile the nmRF51 firmware without Bluetooth, to avoid the added dependencies. This means that you can still build your own nRF51 firmware to work with a Crazyradio/Crazyradio PA.

Bootloader/firmware upgrade

The Crazyflie 2.0, like the original Crazyflie, is upgradable wirelessly with radio. The radio bootloader has been enhanced to work both with ESB and Bluetooth LE. Both nRF51 and STM32F4 can be upgraded and it is even possible to upgrade the nRF51 Bluetooth stack and bootloader.

Currently the bootloader is working with Crazyradio on ESB. Bluetooth implementation for iPhone and Android is still work in progress. The STM32F4 can also be upgraded via USB and so, for people that does not have a Crazyradio, the first Crazyflie 2.0 firmware upgrade has been released for USB DFU as well as the classical radio bootloader update package.

The production of the first Crazyflie 2.0 batch is still on track. Currently all the components are being purchased and when they are all in, the assembly will start. We are still busy preparing things for the production, but we have now slowly shifted over to software and firmware development. The last week has been spent implementing the bootloader and implementing our communication protocol (CRTP) over USB support.

The bootloader has gone from a simple piece of firmware to something that’s more complex. For the Crazyflie 2.0 we have a dual-MCU architecture, which means that we have to flash two MCUs instead of one. Aside from that we now also support bootloading via Bluetooth LE as well as the enhanced shockburst protocol. Another fact that complicates things is that the nRF51 uses a closed binary for it’s Bluetooth LE support, which means we have to take special care when upgrading it. But the trickiest part of all this is to make it safe so that it’s always possible to rescue the system without needing a JTAG/SWD dongle if something goes wrong. It’s important to get it right, since the bootloader is flashed during production it can’t be updated at a later stage without using JTAG/SWD.

The Crazyflie 2.0 only has one button and it’s connected to the nRF51. Instead of directly switching on/off the power to the platform like on the Crazyflie 1.0, the button is now multi-functional. Pressing it will turn on/off the power and put the nRF51 to sleep, but holding it down when powering on the Crazyflie 2.0 will affect the start-up behavior. By holding it down 1.5 seconds the bootloader will launch and by holding it down 5s the STM32F4 will start in DFU mode (Device Firmware Upgrade). One of the reasons for designing in the DFU functionality was to use it as a last resort to rescue the system in case we couldn’t make the nRF51 bootloader secure. Another reason for it is that we always try to design in lots of possibilities, even if we can’t find a use for it we are hoping someone else will.

Crazyflie 2.0 boot arch

CRTP over USB is something that’s been on our TODO list for at least 2 years, but we have never gotten around to it. One idea we had was for building a bigger quadcopter using a Raspberry Pi and a Crazyflie connected to it via USB. The Crazyflie would still be used as the real-time control board, but would receive commands on the USB. This would allow the Crazyflie to become a quadcopter control board using our normal Python API for commands from Linux. This way we could for instance create a higher level autonomous system. But now we needed it for one of our test-rigs, so we had to sit down and get it done. For the implementation we use a raw USB device (i.e class 0) where we use the end-points to send CRTP packets back and forth from the PC. Since a CRTP packet is smaller than a USB packet, there’s no need for flow control or any extra protocols, we just receive new packets on the OUT requests and send a new packets back on the IN requests (if there’s any available). This fits nicely into the current architecture of the Crazyflie Python Lib and the firmware, but it still needs some cleaning up. Once everything is done it will be merged into the main firmware as well.

It’s been a while since we summed up things happening in the community so here’s some of the things that are happening. There’s lots of more things, so if you think we are missing something, then post it in the comments below.

Ralph has been doing some work on an semi-automatic flip feature in the client. There’s more info on the forum and video below.

Last week we tested some modifications made by otto for a headfree mode (i.e yaw only rotates the platform, not the referance direction). It’s a really nice feeling just rotating without taking care of the direction you are going in :-) There’s more information and links to code on the forum.

The SHERPA project have been working on swarm algorithms using a vision system and the Crazyflie.

Geof from Centeye have been working on optical flow stabilization using the Crazyflie. He has a prototype board working and there’s lots of information in the forum about this build. To see the results have a look at the video below.

Thanks to Victor the Deviation firmware for Devo-7e (custom firmware for Devo RC-controllers) now has support for the Crazyflie (needs hardware hack). If you would like to give it a try have a look at the code or grab one of the nightlies.

Researchers at the University of Tokyo have been testing a new concept for a HoverBall using the Crazyflie. Imagine throwing a ball into the air that doesn’t come down (well not right away at least..). Here’s some more info and a picture.

We have also seen some nice stand alone controllers for the Crazyflie, one by  MidLifeCrisis (more info here) and one by ivandevel (video below) . There’s also more info in the forum.

There also some updates on the work done by Oliver on the Kinect tracking of the Crazyflie. A demo video is shown below (it looks great!) and there’s more information on the forum.

And finally here’s a nice video we found on Youtube showing position control of the Crazyflie using a VICON system.