Author: Kimberly McGuire

For the users that have subscribed to our github repository this does not come as an surprise, but for the rest, we have released a new version of our Crazyflie firmware (both STM and NRF) last week!

We know that it is quite close to our last release in February, but we had so many changes and contribution that we deemed it necessary to add a stamp to this current version. In this blog-post, we will give an overview on which features to expect in this update.

UART communication

With courtesy of Saarland University, it is now possible to connect the Crazyflie through its UART to a port on your raspberry pi or through an FTDI cable directly to your computer. This is an extra port for communicating with CRTP will open up new possibilities to interact with your crazyflie.

This is compatible with CFlib version 0.1.10, however there was a fix implemented in the current master (see the ticket here). Please see the ticket for the UART communication here if you are interested in the implementation details.

Lighthouse

It is now possible to get the lighthouse geometry (the position and orientation of the base stations) without SteamVR. We made a script based on the latest stable release of openCV, to calculate the base station geometry based on the received sweep angles on the lighthouse deck. Check these full instructions on how to use this new script. It is a very new and fresh implementation, so if you are experiencing any trouble, please leave an issue on this page or leave a comment on the forum.

Also, FPGA v4 is now integrated in 2020.04, which support Basestation v2. This is still in a very early phase and not yet fully integrated in the firmware, so please keep an eye on this ticket for the implementation process in the latest master of the crazyflie-firmware. There was also a blogpost a few weeks ago about the current state of the lighthouse v2 development.

Bluetooth management

We also provided an update of the bluetooth management of the Crazyflie communication by the NRF chip. Before, it was (unintentionally) possible to connect to the Crazyflie over Bluetooth while it also connected to the CFclient through the crazyradio PA. This caused a lot of unwanted elements such as package loss and unresponsiveness. Now, whenever a Crazyradio packet has been received, Bluetooth will automatically be disabled. The same goes for the peer-2-peer packet, so the NRF firmware no longer needs to be flashed without Bluetooth support. The Crazyflie needs to be restarted after connecting through the CF dongle or P2P in order to connect to it again with the Crazyflie mobile app.

General fixes and improvements

Here are the general fixes and improvements listed that has been fixed in release v2020.04:

  • BMI088 (IMU of the CF2.1) has an self-test now.
  • Fixed memory issue with the Micro SD card deck.
  • High-level commander improvements.
  • Documentation improvements.
  • LPS TDoA (2 and 3) improvements.

See the release notes of the crazyflie-firmware and crazyflie-nrf-firmware to see the full list of improvements and issues that were fixed in 2020.04. The zip files for the firmware for both the roadrunner (tag) and crazyflie (cf2) can be found here.

In this blog-post we wanted to give you guys an overview of our running projects and a general update of the status of things! We got settled in our home-labs and are working on many projects in parallel. There are a lot of development happening at the moment, but the general feeling is that we do miss working with each other at our office! With our daily slack Bitcraze sync meetings and virtual fikapause (Swedish for coffee breaks), we try to substitute what we can. In the mean time, we are going on a roll with finishing all our goals we have set at our latest quarterly meeting, so here you can read about those developments.

AI-deck

Crazyflie with AI-deck

The last time we gave an update about the AI-deck was in this blog post and in the final post of our intern Zhouxin. Building on his work, we are now refocusing on getting the AI-deck ready for early release. The last hurdle is mostly software wise on which we are considering several approaches together with the manufacturer of the Gap8 chip Greenwaves technologies. Currently we are preparing small testing functions as examples of the different elements of the AI-deck in our repo, which are all still in a very primarily phase.

Even though we still need some time to finalize the AI-deck’s early release, we will consider sending an early version of the AI-deck if you are willing to provide feedback while working with it. Please fill in the form and we will get back to you.

Lighthouse

We have made quite some progress on the development for the lighthouse V2. Kristoffer has been working hard from his homelab to get a seamless integration of both V1 and V2 in our firmware (check out this github issue for updates). Currently it is still very untested and very much in progress, however we do have a little preview for you to enjoy.

Crazyflie with LH basestation v2

Documentation

Right now, we are also doing a lot of revamping of the large web of documentation. Unfortunately this is a lot of work! As you noticed by now, we have added overview pages to guide the reader to the right information. We also have moved the tutorials to another part of the menu to avoid clutter on our website. In general we try to go through the repository docs to see if there is any information missing or outdated, however please let us know if you have encountered an error in any description or are missing crucial elements.

Our latest task is revamping the product pages as well, by putting all the necessary information about the hardware in just one place. Also, we are planning to make (video) tutorials soon about many elements of the Crazyflie and how to work with it. More about that later!

Production and Shipment

Production at our manufacturers in China are slowly starting up again. Although it is not yet back at full force, it does enable us to already start ordering to replenish our stock and to get started with finishing our test rigs. Moreover, we are also negotiating to resolve the propeller issue we mentioned earlier, but there is no update on that so far.

As mentioned in this blogpost, we are still shipping orders about twice a week. Both DHL and Fedex are functioning as normal, but we do notice that there is a delay of a few extra days on some deliveries. Please keep that in mind when ordering at our webshop.

Many people in the world have now settled in the reality of working from home. We have also taken precautions ourselves by not go to our office as normal and only ship out packages a few times per week instead of every day (see this blogpost). This also means that we do not have full access to our lab with all our equipment and positioning systems in our big 10 x 10 meter flight lab at the office. In this blogpost we will show how we manage to keep on developing and flying, even in the current situation.

Crazyflie flying in a kitchen with the lighthouse deck

In(light)house positioning

Currently we started to use the Lighthouse positioning system to setup up the remote home lab at our houses. As of recent additions to the Crazyflie firmware, it has been made easy to get the geometry data from the base station. Now the only items we need for indoor flight are just two (or only one) lighthouse basestations V1’s and a Crazyflie, and that is it! There is no need for an HTC Vive headset or hub, or third-party software like SteamVR and the setup is finished in 2 minutes! Check out the new documentation here if you want to know more about the new setup of the lighthouse positioning system.

Also, we recently got a very primarily version of the lighthouse V2 working (see here) and we of course want to keep the momentum going! We will be working on full compatibility from our homes so stay tuned. For now, see this video of the Crazyflie flying with just a single base-station, taken from one of our team-member’s home lab.

Remote Lecture Hall and Practicals

We were invited by Dario Floreano and Fabrizio Schiano from the EPFL-LIS laboratory to do a lecture for the ‘Aerial Robotics’ Course as part of EPFL’s Master’s program in Robotics. Due to the virus, we had to cancel our trip to go there physically… but luckily we were able to do the lecture remotely anyway!

Screenshots of the lectures

The lecture consists of two parts. In the first hour we mostly explained about the Crazyflie ecosystem, hardware and sensors. In the second hour we focused on how the stabilization module worked, including the controllers and the state estimation. During both sessions, we alternated between the theory slides with actual hands-on demos. The lighthouse positioning system was setup in a kitchens, so that we were able to show full flights and practicals with the Crazyflie. At the end there was also the push-demo with just the flowdeck and multiranger, which didn’t use any external positioning at all.

The lectures can be found below and the documentation has been updated as well with the covered material (see here). Be sure to check out the controller tuning presented in part 2 of the lecture (25:00 – Cascaded PID controller).

Other Home labs

Home lab with Crazyflie

We know that there are currently users that are moving their flight lab from their university or company to their homes to be able to continue their work. We would love to hear about your experience and your home lab! Send us an email with your story to contact@bitcraze.io, drop us a message on forum.bitcraze.io, or mention us in your Twitter, Linkedin, Facebook or Reddit post. Also, if you want to setup your own home lab and you need any advice or help, please let us know!

Two weeks ago, we had a blogpost about the state estimators that are available within the Crazyflie. So once the Crazyflie knows where it is, it would need to be determined where it wants to go, by means of the high level commander (implemented as part of the crazyswarm project) or set-points given by CFclient or directly from scripts using Crazyflie python lib. But exactly how would the crazyflie get to those desired positions in the first place? The differences between the current state estimates and the desired state, will need to be transformed to inputs given to the motors. Unfortunately, quadrotors like the Crazyflie do not have easy dynamics to maintain, so if you want to learn more, see this blogpost to read more about it!

Controlling the Crazyflie

So in order use the thrust of the motors in an useful way to get the Crazyflie to do what you want to do, there are several controllers to consider, which you can see on this quick overview here underneath. It shows the different control paths that can be taken from the high level commander all the way to the power distribution of the motors. Bear in mind that these are still simple representations and that the actual implementation is of course a bit more complicated, but at least it will give you a rough idea of which paths are possible to pursue.

Possible controller pathways

PID Controller

So the default settings in the Crazyflie firmware is the proportional integral derivative (PID) control for all desired state aspects. So the High Level Commander (HLC) will send desired position set-points to the PID position controller (which used to be done off-board, so outside of the Crazyflie firmware before this blogpost). These result in desired pitch and roll angles, which are sent directly to the attitude PID controller. These determine the desired angle rates which is send to the angle rate controller (which is… you guessed… also a PID controller). This is also called Cascaded PID controller. That results in the desired thrusts for the roll pitch yaw and height that will be handled by the power distribution by the motors. (Note that height is mostly handled by the position controller)

INDI Controller

So the Incremental Nonlinear Dynamic Inversion (INDI) controller is an controller that immediately deal with the angle rates to determine the trust. This is a very new addition to the Crazyflie firmware by one of our community members and is based on the implementation of this paper. Currently, the position control is still handled by the same PID controller mentioned in the last paragraph, Nevertheless for handling the angles, it should be faster than the attitude and rate PID controller combined. We have not yet fully tested this out but if you do, let us know how you like it on the Bitcraze forum!

Mellinger Controller

As part of the Crazyswarm project, the controller designed by Daniel Mellinger has been implemented in the Crazyflie firmware as well. Please see this paper about the details of the Mellinger controller. It is a sort of “all in one”: based on the desired position and velocity vectors towards those position, it will calculate right away what the desired thrusts are that need to be distributed to all the motors. This results in a much smoother controlled trajectory of the high level commander and therefore advised to use when the Crazyflie has a precise position estimate (lighthouse and mocap). However, as it is so aggressive, any position estimate of a lesser quality (flowdeck or LPS) will not be sufficient for this controller. See some examples of mellinger controlled flights here and here.

Let us know what you think!

So do you have experience working with these controllers or want to know more about them, please drop us a message on the forum! We are currently working on stabilization and documentation of multiple aspects of the Crazyflie and the controllers is one of them, so we are really interested what your experiences are!

How does a Crazyflie manage to fly and stay in the air in the first place? Many of us tend to take this for granted as much research tend to happen on the application level. Although we try to make the low level elements of flight as stable as possible, it might happen that whatever you are trying to implement on the application level actually effects the Crazyflie on the low level controls and estimation. We therefore would like to focus a little bit on the inner-workings of the autopilot of the Crazyflie, starting with state estimation. The state estimation is part of the stabilizer loop in the Crazyflie, an overview of is was made in a previous blog post.

State estimation is really important in quadrotors (and robotics in general). The Crazyflie needs to first of all know in which angles it is at (roll, pitch, yaw). If it would be flying at a few degrees slanted in roll, the crazyflie would accelerate into that direction. Therefore the controller need to know an good estimate of current angles’ state and compensate for it. For a step higher in autonomy, a good position estimate becomes important too, since you would like it to move reliably from A to B.

There are two types of state estimators in the crazyflie firmware, namely a Complementary Filter and an Extended Kalman Filter.

Complementary Filter

The complementary filter is consider a very lightweight and efficient filter which in general only uses the IMU input of the gyroscope (angle rate) and the accelerator. The estimator has been extended to also include input of the ToF distance measurement of the Zranger deck. The estimated output is the Crazyflie’s attitude (roll, pitch, yaw) and its altitude (in the z direction). These values can be used by the controller and are meant to be used for manual control. If you are curious how this code is implemented exactly, we encourage you to checkout the firmware in estimator_complementary.c and sensfusion6.c. The complementary filter is set as the default state estimator on the Crazyflie firmware.

Schematic overview of inputs and outputs of the Complementary filter.

Extended Kalman Filter

The (extended) Kalman filter is an step up in complexity compared to the complementary filter, as it accepts more sensor inputs of both internal and external sensors. It is an recursive filter that estimates the current state of the Crazyflie based on incoming measurements (in combination with a predicted standard deviation of the noise), the measurement model and the model of the system itself. We will not go into detail on this but we encourage people to learn more about (extended) Kalman filters by reading up some material like this.

Schematic overview of inputs and outputs of the Extended Kalman Filter

Shortly said, because of the more state estimation possibilities, we preferred the Kalman filter in combination with several decks: Flowdeck, Loco positioning deck and the lighthouse deck. If you look in the deck driver firmware (like for instance this one), you see that we set the required estimator to be the Kalman and that is of course because we want position/velocity estimates :). Important though is that each input of the measurement effects the quality of the position, as positioning of the Lighthouse deck (mm precision) is much more accurate that the loco positioning deck (cm precision), which has all to do with the standard deviation of the measurement of those values. Please check out the content of estimator_kalman.c and kalman_core.c to know more about the implementation. Also good to know that the Kalman filter has an supervisor, which resets if the position or velocity estimate is gets out of hand.

Of course this blogpost does not show the full detailed explanation of state estimation, but we do hope that it gives some kind of overview so you know where to look if you would like to improve anything. The Kalman filter can easily be extended to accept more inputs, or the models on which the estimates are based can be improved. If you would like implement your own filter, that would be perfectly possible to do so too.

It would be great if you guys could share your thoughts and questions about the state estimation on the crazyflie on the forum!

Happy holidays to all our users, community members and friends! We are happy to announce our 2019 Christmas video which we have made in collaboration with Ben Kuper! It is starring 7 Crazyflies, the lighthouse positioning system, our office Christmas tree and a whole lot of holiday spirit, so go ahead and take a look!

Here are some words from Ben how it was to work on this year’s Christmas video at our office:

Coming to Bitcraze’s HQ and working with them has been once more a wonderful experience, technically and humanly ! The main goal of this session was to test and implement the new lighthouse tracking system in the tool suite I’m creating, and it was an amazing surprise to witness for real the uncanny stability of the drones when they’re on lighthouse tracking !

Of course, my first reaction was to push the limit and see what can be done with this new power, this is why I created this choreography : to see what can be done in a limited amount of time (1 and a half day to create the full choreography, the official video shows the first part only), and trying to go at the limit of the current possibilities. As the team was working on occlusion recovery, we decided to have the drone fly around the tree as a fun test, and it works !

In the new year we will have a followup blog-post going into detail on how exactly we made this video. Until then, happy holidays and have an awesome new year!

For the last four years of doing my PhD at the TU Delft and the MAVlab, we were determined to figure out how to make a swarm/group of tiny quadcopters fly through and explore an unknown indoor environment. This was not easy, as many of the sub-challenges that needed to be solved first. However, we are happy to say that we were able to show a proof-of-concept in the latest Science Robotics issue! Here you can see the press release from the TU Delft for general information about the project.

Since we used the Crazyflie 2.0 to achieve this result, this blog-post we wanted to mostly highlight the technical side of the research, of the achievements and the challenges we had to face. Moreover, we will also explain the updated code which uses the new features of the Crazyflie Firmware as explained in the previous blogpost.

A swarm of drones exploring the environment, avoiding obstacles and each other. (Guus Schoonewille, TU Delft)

Hardware

In the paper, we presented a technique called Swarm Gradient Bug Algorithm (SGBA), which borrows (as the name suggests) navigational elements from the path planning technique called ‘Bug Algorithms’ (see this paper for an overview). The basic principle is that SGBA is a state-machine with several simple behavior presets such as ‘going to the goal’, ‘wall-following’ and ‘avoiding other Crazyflies’. Here in the bottom you can see all the modules were used. For the main experiments (on the left), the Crazyflie 2.0’s were equipped with the Multiranger and the Flowdeck (here we used the Flow deck v1). On the right you see the Crazyflies used for the application experiment, were we made an custom Multiranger deck (with four VL53L0x‘s) and a Hubsan Camera module. For both we used the Turnigy nanotech 300 mAh (1S 45-90C) LiPo battery, to increase the flight time to 7.5 min.

Hardware used in the experiments. Adapted from the science robotics paper.

Experiments

With this, we were able to have 6 Crazyflies explore an empty office floor in the faculty building of Aerospace engineering. They started out in the middle of the test environment and flew all in different preferred directions which they upheld by their internal estimated yaw angle. With the multi-rangers, they managed to detect walls in their, and followed its border until the way was clear again to follow their preferred direction. Based on their local odometry measurements with the flowdeck, the Crazyflies detected if they were flying in a loop, in order to get out of rooms or other situations.

A little before half way of their battery life, they would try to get back to their initial position, which they did by measuring the Received Signal Strength Intensity of the Crazyradio PA home beacon, which was located at their initial starting position. During wall-following, they measured the gradient of the RSSI, to determine in which directions it increases or decreases, to estimate the angle back the goal.

While they were navigating, they were also communicating with each-other by means of broadcasting messages. Based on those measurements of RSSI, they could sense other Crazyflies approaching, which they first of all used for collision avoidance (by letting the low priority CFs move out of the way of the high priority CFs). Second of all, during the initial exploration phase, they communicated their preferred direction as well, so that one of them can change its exploration behavior to not conflict with the other. This way, we tried to maximize the explored area by the Crazyflies.

One of those experiments with 6 Crazyflies can be seen in this video for better understanding:

We also showed an application experiment where 4 crazyflies with the camera modules searched for 2 dummies in the same environment.

Challenges

In order to get the results presented above, there were many challenges to overcome during the development phase. Here is a list that explains a couple of the elements that needed to work flawlessly:

  • Single CF robustness: We used the Flowdeck v1, for the ‘deadlock’ detection and the basic velocity control, which was challenging in the testing environment because of low lighting conditions and texture. Therefore the Crazyflies were flying at 0.5 meters in order to ensure robustness. The wall-following was performed solely using the Multiranger. This was tested out in many situations and was able to handle a lot of type of obstacles without any problem. However the limited FOV of the laser range finder can not detect all types of obstacles, for instance thin ones or irregular ones such as plants. Luckily these were not encountered in the environment the Crazyflies flew in, but to increase robustness, we will need to consider adding a camera to the navigational drive as well.
  • Communication base-station. SGBA by essence only needs one base-station Crazyradio PA, since all the behavior is completely on board. However, in order to show results in the paper, it was necessary for the CF to communicate information back, like odometry, state and such. As this was a two way communication (CFs needed RSSI to get back) each Crazyflie needed 1 base-station. Also, they all needed to be on different channels to avoid package collisions and RSSI accumulation.
  • Communication Peer to Peer. At development time, P2P didn’t exist yet, so we had to implement broadcast communication between the Crazyflies. Since the previous pointer required them to listen on different channels, the NRF had to be configured to send separate broadcast messages on all those channels as well. In order to time this properly, the home beacon had to sync the Crazyflies accordingly by sending out a timer. Even so, the avoidance maneuvers were done very conservatively to try to prevent inter-drone collisions.

Many of the issues, especially the communication challenges, will be solved with the updated code implementation as explained in the next section.

Updated code

The firmware that the Crazyflies used to fly in the experiments showed in the paper, can all be found in this public repository. However, the code is based quite an old version the current Crazyflie firmware, as it was forked almost a year ago. The implementation of the SGBA state machine and the P2P broadcasting were not generic enough to integrate this back to the development cycle, therefore the current code is only suitable for the old Crazyflie 2.0.

Therefore, we developed two major changes in the latest firmware which will make it much easier for me (and other ideas as well we hope!) to implement SGBA and the P2P communication in a way that should be compatible with any version of the firmware (and hardware) from here and on. We implemented SGBA as an app-layer and also handled all the broadcast messaging directly from this layer as well. Please check out this Github repository with this new app layer implementation of SGBA.

The Crazyflie Bolt and the Crazyflie 2.1 with the lighthouse deck are coming to Madrid!

Only one week away until the start of the big Bitcraze Conference frenzy, with the first stop… Madrid! We will visit the Micro Air Vehicle Competition and Conference, which is a robotics event that is more specialized in (as the name implies) MAVs! So it should be right up our alley. This is the first time that we attend as Bitcraze, although the writer of this blog post has experienced fun times at the conference and the competition as a participant with her previous lab, the MAVlab.

The IMAV has been around for almost 12 years, starting in Toulouse, France in 2007. Although it initially mostly was held in various places in Europe, in 2016 into a more worldwide phenomenon by making it’s tribute in Beijing, China and Melbourne, Australia in 2018. It hosts a conference to which researchers can send their work in anything related to MAVs, from autonomous navigation, state estimation and design.

IMAV is mostly know for hosting big indoor and outdoor competitions for MAVs. The outdoor competitions can range from survey tasks to finding a hidden person or object. This year the focus will be on the delivery of packages from one place to another. The judges will look at how many packages that can be safely delivered and if the drone is able to detect certain objects in the outdoor environment. The indoor competition is oriented around the application of MAVs in a warehouse. They should be able to take off autonomously, monitor boxes in shelves and make an innovatory, and pickup packages to release them in their designated location. 40 teams of 28 universities will show their awesome implementation in these difficult tasks.

We will have a booth at the main company fair at the conference and indoor competition, and will also be present at the outdoor competition day as well. We will bring the lighthouse positioning system and show the awesome swarming demo we developed. Also we want to bring the new Crazyflie Bolt with us, which we are sure of that the regular IMAV crew will love. If you are at the IMAV between the 30th of September to the 4th of October, come by and say hello!

We have a busy schedule in the fall, since we are planning to travel all around the world for all kinds of awesome conferences and symposia. Here is a quick overview of where we are planning to go in October and November.

IMAV Madrid 1/10-4/10

We will attend the IMAV 2019, and this time it is in Madrid (Spain) from the 1th to the 4th of October. This is a conference for anything related to micro-aerial vehicles and also includes an awesome indoor and outdoor competition. Arnaud and Kimberly will be present at both the indoor and outdoor fair, where we will show some demos with the lighthouse system and show off our new Crazyflie Bolt!

https://imav2019.org/

Symposium Seoul 14/10 – 15/10

In collaboration with our partner CLRobur, we will have a symposium in Seoul (Korea) on the 14th and 15th of October. Kristoffer will explain the attendees all about positioning systems, and show off the LPS and our new lighthouse deck.

Sign up here or send an email to clrobur@naver.com

IROS Macau 4/11-8/11

Last but not least, we will be there at IROS 2019 as well! This is one of the flagship robotics conferences of IEEE. In Macau (China) from the 4th till the 8th of November, Marcus and Tobias will show some awesome demos with the lighthouse system and hopefully we will present the new AI deck!

https://www.iros2019.org/

A lot of you are using our forum on a regular basis. From our point of view, we prefer giving support and guidance on our forum instead of by email, since solutions we provide to our users can be found by others as well. However, we are experiencing some spam-related issues with the forum!

Somewhere last week, our forum started getting some weird posts, including some links to websites that are…. not exactly Bitcraze or Crazyflie related… We didn’t think about it too much, so we cleaned the forum, deleted the users responsible and moved on. But the very next day more spam post appeared, and it was getting so bad that we changed our sign-up from the regular security question to Captcha. Unfortunately, spambots are way too smart these days, so that didn’t work either… By the end of the weekend we had to remove over an hundred spam accounts and 500 spam posts!

The forum is experiencing a major spam attack at the moment and the usual security measures are not helping. Since yesterday, we enabled manual activation of new users (meaning that we physically need to approve each account), since currently this is the only way to catch the dodgy new accounts (we already caught over 100 in one day alone !!). This week we will look into other measures and plugins to better secure our forum, so we don’t have to manually go through all the newly registered users to see if they are not spam bots. We heard about Cleantalk by one of our users, so we probably will give that a go!

However, please be contact us if the following occurred or is occurring to you in our spam removing frenzy:

  • If we accidentally removed a post of you
  • If we removed or banned your account
  • If we are not activating your new account

Also make our lives a bit easier this week, and send us an email straight away if you have just have made a new account on the forum. Send an email to: contact@bitcraze.io with your username and a sentence about why you are using the Crazyflie (so we know that you are real).

This is only a temporary solution, so hopefully at the end of this week or the next we will not need to worry about this anymore :).