Author: Kristoffer Richardsson

As mentioned in this blog post, we added the possibility to write apps for the Crazyflie firmware a while ago. Now we have added more functions in the Firmware to make it possible to use apps for an even wider range of tasks.

The overall idea of the app API is to mirror the functionality of the python lib. This will enable a user to prototype an application in python with quick iterations, when everything is working the app can easily be ported to C to run in the Crazyflie instead. The functions in the firmware are not identical to the python flavour but we have tried to keep them as close as possible to make the translation simple.

An app is also a much better way to contain custom functionality as the underlying firmware can be updated without merging any code. The intention is that the api API will be stable over time and apps that work one version of the firmware also should work with the next version.

Improvements

We used our demo from IROS and ICRA (among others) with a fairly autonomous swarm as a driver for the development. The demo used to be implemented in a branch of the firmware with various modifications of the code base to make it possible to do what we wanted. The goal of the exercise was to convert the demo into an app and add the required API to the firmware to enable the app to do its thing. The new app is available here.

The main areas where we have extended the API are:

Log and parameters framework

The log framework is the preferred way for an app to read data from the firmware and this has been working from the start. Similarly the parameter framework is the way to set parameters. Even though this has worked, it broke a basic assumption in the setup with the client, that only the client can change a parameter. Changing a parameter from an app could lead to that the client and Crazyflie had different views of the state in the Crazyflie, but this has now been fixed and the client is updated when needed.

High level commander

The high level commander was not accessible from an app earlier and the functions in the python lib have been added to make it easy to handle autonomous flight.

Custom LED sequences

It is now possible to register custom LED sequences to control the four LEDs on the Crazyflie to signal events or state.

Lighthouse functionality

Functions for setting base station geometry data as well as calibration data have been added. These functions are also very useful for those who are using the lighthouse system as it now can be done from an app instead of modifying lighthouse_position_est.c.

Remaining work

We have taken a step forward with these changes but there is more to be done! The two main areas are support for custom CRTP packets and memory mapping through the memory sub system. There might be more, let us know if there is something you are missing. The work will continue and there might even be some documentation at some point :-)

Tutorial

One reason for doing this API work now was to prepare for the tutorial about the lighthouse 2 positioning system, swarm autonomy and the demo app that we will run this Wednesday on-line, don’t miss out! You can read more about the event here.

We’re happy to announce that we have taken an important step forward in the development of the lighthouse positioning system, we have improved the calibration compensation. The changes improves the correctness of the coordinate system, especially for lighthouse V2 base stations.

As mentioned in this blog post one of the remaining areas to solve was handling of calibration data and this is what we have addressed lately. In the manufacturing process mechanical elements are mounted within some tolerances but since the precision of the system is so good, also a very fine tolerances makes a big difference in the end result. Each base station is measured in the factory and the calibration data describing these imperfections are stored in the base station. The calibration data is transmitted in the light sweeps to enable a receiver to use it to correct for the errors in the measured angles.

As with everything else related to lighthouse, there is no official information of how to interpret the calibration data so we (and the community) had to make educated guesses.

Lighthouse 1

The compensation model for lighthouse 1 has been known for quite long, see the Astrobee project by Nasa and Libsurvive. The most important parameter is the phase and until now this is the only part of the calib data that we have used in the firmware. In the new implementation we use all parameters.

The parameters of the lighthouse 1 calibration model are phase, tilt, gib mag, gib phase and curve.

Lighthouse 2

The compensation data for lighthouse 2 is similar to lighthouse 1 but there are two new parameters, ogee mag and ogee phase. It also seems as some parameters that are sharing names between lighthouse 1 and 2 have different meanings, for instance curve.

Libsurvive has implemented compensation for lighthouse 2 but we have unfortunately not managed to use their work with good results, instead we have tried to figure out what the model might look like and match it to measurements. We have managed to get good results for the phase, tilt, gib mag and gib phase, while we don’t know how to use curve and ogee mag and ogee phase. The solution seems to be pretty good with a subset of the parameters and we have decided to leave it at that for now.

Use of calibration data

The way we have used the calibration data so far has been to apply it to the measured angles to get (more) correct sweep angles that have been fed into the position estimation algorithms. The problem is that the compensation model is designed the other way around, i.e. it goes from correct angles to measured angles, and an iterative approach is required to apply it to the measured angles. A better way (most likely by design) is to apply it in the kalman estimator instead where it simply becomes part of the measurement model.

Currently we do calculate the corrected angles as well and expose them as log data, but it is not required for the standard functionality of the lighthouse system. We may make it possible to turn it on/off via a parameter in the future to save some CPU power.

Functional improvements

So what kind of improvements will the calibration add?

The first improvement is the base station geometry estimation. With more correct angles the estimated base station position and orientation will be better. This is important to be able to get a good estimation of the Crazyflie position since poor geometry data will give the position estimator conflicting data.

Secondly more correct angles will straighten the coordinate system. With angular distortion the position estimator will not be able to estimate the correct position and the coordinate system will be warped, bent or stretched. The improvement can be seen when flying parallel to the floor at constant height for instance.

Thirdly the stability will hopefully be improved. When the angles from two base stations match better, the estimated position will change less when one base station is occluded and generally make life easier for the position estimator. We will take a look at the outlier filter to see if it can be improved as well.

Remaining problems

The calibration data is transmitted as a part of the sweeping light planes with a low bitrate. For lighthouse 1 the decoding process works well and all calibration data is usually received within 20-30 seconds. For lighthouse 2 it does not work as well in our current implementation it takes (much) longer before all data has been received correctly from both base stations.

It is possible to get the calibration data via the USB port on lighthouse 2 and we are considering storing the calibration data in the Crazyflie somehow instead. This will be even more important when we support larger systems (2+ base stations) and all base stations are not within range at startup.

It has almost been 2.5 years since we last made a compilation video about what has been done with the crazyflie 2.0 (see this blogpost). During this time we released several new products and many of you were able to achieve very cool and awesome applications so we thought it was time for a new research compilation video!

We have seen quite a lot of projects with swarms of Crazyflies, ranging from close proximity flight to autonomous exploration in a building. Some research groups have also been experimenting with controlling the Crazyflie with our own hands, either to control its position or to reach another state of mind entirely. Others have created their own deck in order to add their own sensors or cameras necessary for their application. One of those even led to the new AI-deck that we introduced in early release before the summer! Last but not least: we were shown that the low level control can be further improved and multiple crazyflies can be linked together and still fly!

We were overwhelmed by all the awesome things that the community showed us of what possible with the Crazyflie and this will inspire others to think of new things to do as well. We hope that we can continue with helping you to make your ideas fly, so that we are soon to be forced to make another completion video ;)

Here is a list of all the research that has been included in the movie:

  • Close proximity flight of sixteen quadrotor drones, CalTech: B. Rivière, W. Hoenig, Y. Yue, and S.-J. Chung (video, paper)
  • Pointing gestures, IDSIA: B. Gromov, J, Guzzi, L. M. Gambardella, A. Giusti (video, project)
  • Yaw actuation, Modlab UPenn: B. Gabrich G. LiM. Yim (video, paper)
  • Learning to seek, Harvard University: B. P. Duisterhof , S. Krishnan, J. J. Cruz, C. R. Banbury, W. Fu, A. Faust, G. C. H. E. de Croon, V. Janapa Reddi (video, paper)
  • Drone Chi, Exertion Labs and RMIT: J. La Delfa, M. Baytas, E. Luke, B. Koder, F. Mueller (video, project)
  • Generalization through simulation, Berkeley AI Research (BAIR): K. Kang, S. Belkhale, G. Kahn, P. Abbeel, S. Levine (video, paper)
  • Swarm exploration, TU Delft: K.N. McGuire, C. De Wagter, K. Tuyls, H. Kappen, G.C.H.E. de Croon (video, paper)
  • PULP-based autonomous drone, ETH Zürich: D. Palossi, F. Conti, and L. Benini (video, paper)
  • Networked Autonomous Aerial Vehicles, University of Klagenfurt, Austria: Research by Karl Popper Kolleg (video)
  • GLAS for Multi-Robot Motion Planning with End-to-End Learning, CalTech: B. Rivière, W. Hoenig, Y. Yue, and S.-J. Chung (video, paper)
  • Resilience by reconfiguration, USC: R. K. Ramachandran, J. A. Preiss, G. S. Sukhatme (video, paper)

We are happy to annouce that we have released a new version of the Crazyflie firmware, version 2020.09. It is available for download from Github.

The new firmware solves an old compatibility issue when using the LPS and Flow deck at the same time and also improves stability. A list of all the issues that have been fixed can be found on the release page.

For users that have a LPS system, we have also made some improvements to the LPS node firmware and are releasing version 2020.09.

If you are building the Crazyflie or LPS firmware from source, remember to update the libdw1000 git submodule using

git submodule update

We are working on a release of the python client as well, but still have a few issues to fix so stay tuned.

We’re happy to announce the availability of the 2020.06 release! The release includes the Crazyflie firmware, the Crazyflie NRF firmware and the python library (0.1.11). You can find the full package in the Crazyflie Release repository, to be used for flashing through the python client.

More RAM

The major event of this release is the use of the Core Coupled Memory in the Crazyflie. The CCM is a 64k RAM memory bank and by moving memory blocks from the standar RAM to the CCM, we have freed up 64k of RAM! The 128k of RAM was almost full so an extra 50% is good news.

One might ask why this has not been done earlier and the answer is that the CCM has some special properties that has to be taken into account. It is RAM, just like the “normal” RAM, but it is connected to a different internal bus in the STM MCU. The most notable difference is that it can not be used in DMA operations that are commonly used when accessing sensors, and if a pointer to CCM memory is passed to a sensor driver things will go bad. To make it clear where the memory is located, we have introduced a macro to be used when explicitly moving a memory area to the CCM, otherwise it will end up in normal RAM.

Hopefully the chosen design will have very little impact on the “normal” firmware programmer. We have moved a bunch of memory blocks to the CCM that are “safe”, and most programmers can happily forget about the CCM and just enjoy the new 64k of available RAM!

Battery temperature

In release 2020.02 we introduces a battery temperature check do not charge the battery if it is too warm. Lithium batteries likes to be charged within 0-45 deg Celsius. To do this we used the temp sensor within the nRF51822 which is mounted just under the battery. It hover turned out that the temp measurement is way to biased and as a result stops charging to early. So in this release we did more measurements and increased the allowed charging range.

The Lighthouse V2 implementation has been simmering away for a long time in the Bitcraze kitchen and in this blog post we will give you an update on the current status and what is remaining for a full release of this tasty dish.

Crazyflie 2.1 and Lighthouse V2 base station

We believe we have solved most of the major technical hurdles (last famous words) on the way to a working implementation that uses Lighthouse V2 base stations for positioning, now it is mostly work to implement the functionality that is remaining. As described in this post we now have a new FPGA binary that has the ability to decode both V1 and V2 base stations, and this was a major step forward. This new binary is used in the Crazyflie firmware master branch, and if the Lighthouse deck is used with the latest Crazyflie firmware, the new FPGA binary will automatically be flashed to the deck.

What has changed?

The new FPGA binary uses a different UART protocol to communicate with the Crazyflie. This protocol has been implemented in the firmware and hopefully there is no functional difference compared to the previous FPGA binary when using Lighthouse V1 base stations.

We have added a first version of Lighthouse V2 base station decoding, but it is still a bit limited. As a start we decided to “emulate” V1 base stations to be able to reuse as much of the existing code as possible. For now we support only 2 base stations and they must use channel 1 and 2 (used to be called modes). The V2 angles are transformed into V1 angles and fed into the old positioning logic and are handled exactly the same way as before. Even though this works, it is not the optimal solution and we hope to be able to refine it later on.

We have also written a python script to estimate base station geometry (positions and orientation) using the Lighthouse deck. This removed the requirement to use software from Steam which should simplify the set up process. Please see the (still limited) documentation. Note that this calibration method only supports the basestation V1… for now!

There is a lot of code that has been modified and the FPGA implementation is completely new, it is not unlikely that there is functionality that is unstable or broken, or configurations that are not supported. If you happen to notice any bugs, please let us know!

What is remaining?

The functional areas that needs to be implemented or cleaned up before we leave the Early Access stage is the following:

Calibration data

The calibration data is embedded in the modulated light from the base stations and describes imperfections from the manufacturing process for each individual. This data is not read yet for V2 and will increase the precision when available.

Support for more than 2 base stations

Lighthouse V2 base stations are designed for systems with more than 2 base stations. The Crazyflie firmware needs to be extended for this functionality to work, including handling of geometry data, logging, memory management and some other bits and pieces.

Native V2 positioning

The angles from the V2 base station should be fed directly into the kalman filter for positioning, instead of first being transformed into V1 angles. This will increase robustness and reduce data loss.

Client support

We want to add a tab in the python client where a Lighthouse system can be monitored, configured and managed. It should, for instance enable the user to configure and visualize base station geometry.

FPGA binaray management

Currently the FPGA binary is included in the Crazyflie firmware and it is automatically uploaded to the deck when booted. This is not a viable long term solution and we hope to be able to find a more generic way of handling deck binaries.

Conclusions

As can be seen, there is still quite some work to be done before the Lighthouse V2 stew is ready to be served, but we are definitely starting to smell some nice flavours from the kitchen!

Finally a view from Kristoffer’s home lab, currently in the summer house. Three base stations are set up as a Fun Friday hack to see what it would take to use more than 2. Luckily it did not take too much time to get this to work :-)

3 Lighthouse V2 base stations

We have mentioned the Active Marker deck in an earlier blog post, and are now happy to announce that it has been released and is available in our store.

Crazyflie with Active marker deck

By changing the passive, reflective markers to active, IR-LEDs, it is possible to improve the detection of markers in the cameras. There are two main reasons: the area of the marker is smaller and easier to separate from other markers close by, and the LEDs are emitting light and can be detected further away.

The deck has been developed in collaboration with Qualisys and together with the QTM system, it utilizes their Active Marker technology. An ID is assigned to each marker, and since the identity of each marker can be detected by the MoCap system, it is possible to estimate the full body pose of the Crazyflie without unique marker positions or known starting positions. IDs are easily assigned using the parameter sub system of the Crazyflie.

Even though the deck mainly is intended to be used with Qualisys MoCap systems, the LED markers can also be configured to be on or off which we hope might be useful in other applications as well.

There has been some work done earlier to use the Crazyflie for generating images, for instance the dot-drawing by Paul Kry and light painting. I wanted to see if it is possible to put a brush or pen on a Crazyflie and use it to draw lines on a paper. I decided to use a fun Friday to try it out. The idea is simple: mount a pen on the Crazyflie, put a paper on a wall, write a script to draw a figure, fly!

The setup

The first thing I looked into was to investigate if a Crazyflie can fly with a brush or pen mounted on it. I wanted to keep the weight down and my initial approach was to use a cotton swab (0.6 g) dipped in paint. I found one that was long enough to extend in front of the propellers and I mounted it by squeezing it between the battery and the PCB. Flying was no problem with such low extra weight.

For positioning I decided to use the Lighthouse system. It is very accurate, simple to use and the easiest way to get started. I mounted a piece of cardboard in the YZ-plane of our lighthouse coordinate system, where I could attach a drawing paper. The idea of setting up the drawing surface parallell to the YZ-plane was to make the scripting easier. I (of course) used the Crazyflie and lighthouse system to measure that the cardboard was mounted at the right position.

Finally I wrote a simple python script that utilized the high level commander to move towards the drawing surface and yawing at the right position to draw a stroke on the paper. It sort of worked, but the cotton swab has to be “refilled” before each stroke which took a lot of time, and the results were a bit random.

I decided to try out a pen instead. The upside is that it does not require refill, on the other hand it is much heavier which makes the Crazyflie a bit sluggish when flying. I mounted the pen on the top side of the PCB, squeezed under the Lighthouse deck, and moved the battery to under the Crazyflie to distribute the weight.

Initial tests – both cotton swab and pen

The script was updated to draw the outline of the Bitcraze logo. I had a couple of variations where I tried to draw the full outline in one long stroke, as separate strokes, going up or going down and some other flavours.

So was it successful? Currently the Crazyflie is not a new Picasso, but the painting skills could maybe be improved with some more work. I think the main problems were:

  1. the pen is too heavy and requires too much force on the paper
  2. the controller cannot handle the situation in a good way. In essence I set the set point a few millimeters “into” the paper to push the pen against the surface which seems to be confusing as the controller can not reach the set point.
  3. Flying that close to the drawing surface creates an air flow that disturbs the flight.

Video showing the Crazyflie drawing the logo

The Bitcraze logo (17×17 cm), drawn by the Crazyflie

The Crazyflie supports wireless communication using both the Crazyradio PA and BLE (Bluetooth Low Energy https://en.wikipedia.org/wiki/Bluetooth_Low_Energy). BLE is used with the mobile phone apps while Crazyradio PA usually is used together with a PC.

The lower levels of the radio communication in the Crazyflie is handled by the nRF51 that is capable of handling both types of communication. When using the Crazyradio we are using the manufacturers, Nordic Semiconductor, proprietary Enhances ShockBurst protocol (ESB) which makes it simple to send packages, up to 32 bytes, between each other. When communicating over BLE we are using Nordic Semis S110 SoftDevice which is a BLE stack developed by Nordic Semi to simplify implementation.

When we designed the first Crazyflie, the Crazyflie 1.0/Nano, we choose to use the nRF24L01+ that uses the ESB protocol because of simplicity, good range and low latency. Then came the Crazyflie 2.0 and we wanted BLE for mobile client support. Luckily Nordic released the nRF51 which could handle both. However there is a small drawback, both protocols can’t run concurrently and has to be interleaved. For BLE this has never been any problem as this protocol has the priority, but for ESB it means that when BLE is running there will be a small amount of packet loss.

The CRTP protocol we developed that runs on top of the ESB, handles the packet loss fairly well but as more and more Crazyflies are added we have been seeing communication issues. So last week we dived in to this problem and after some digging we understood that BLE was one of the problems. Therefor we added a switch which disables BLE as soon as a ESB packet is received. This improved the ESB connection and it now seems more stable. If you have the possibility we suggest you to get the latest from the crazyflie2-nrf-firmware master branch, try it out and give us feedback.

This change will hopefully provide more stable communication between the Crazyradio PA and the Crazyflie. From a functionality point of view, most users will not see any difference, but we would like to point out that if you have communicated with your crazyflie using the Crazyradio PA, it will not be possible to connect with a mobile phone until the Crazyflie has been re-booted. Note that a simple radio scan with the python client has the same effect and disables BLE.

2020 has arrived! We are back after a long and nice holiday and are ready for new awesome year. So what’s happening in 2020 at Bitcraze?

Frequently we will go through a hectic period of releasing new hardware and adding new features that’s followed by a period of stabilization and maintenance. Since last fall we released a number of new products and added lots of new functionality to the Crazyflie, this spring it’s time to look back, finish up and make things more stable.

First up this year is to release a few of the products we didn’t manage to get out during the fall, the AI-deck and the Active marker deck. The first batches should be ready during the beginning of 2020 and we are really excited to see what our users will do with the new hardware.

The AI-deck

Secondly we will be focusing on getting the Lighthouse deck out of Early Access. After the release last year we’ve added lots of functionality such as full pose using only a single base-station, but we’re still missing Lighthouse V2 support. In order to finish up we need to make the system easier to use and to add the V2 support.

The third thing we will be focusing on is stability and usability. We’re really happy about the ecosystem that exists around the Crazyflie platform. With lots of hard work and help/contributions from our community it’s become rather extensive over the years. But with the ecosystem and feature-set continuing to grow it’s not always easy to keep up.

The last thing we will be looking at is to continue adding more swarm-related functionality to simplify the task of flying multiple Crazyflies at the same time. Some of the features we’ve been discussing are: improved positioning, improved communication, drone health, trajectories and so on.

In order to handle the growing product portfolio and Crazyflie platform we’ll also continue growing the team during 2020. Hopefully there will more news on this already during the coming weeks :-)

With a long list of new customers, a growing ecosystem and lots of interesting things to come, we’re really looking forward to an exciting and fun year at Bitcraze! Oh, and as always there’s a few new hardware prototypes cooking… Happy New Year!