Category: Software

Two weeks ago we posted about the demo we did for our new office move-in party. There has been multiple requests to share the script but unfortunately this is a hacked old script that is not going to be useful at all as an example. So, last week, we made an example that could run a synchronized swarm sequence.

The example has been pushed in the example folder of the Crazyflie-lib-python project. It is called synchronizedSequence.py. Running this example unmodified with 3 Crazyflies in a positioning system will give you this result. (Like the previous demo, this was done in a lighthouse system.)

One of the key design of the example is that it is based on a single control loop that can be synchronized with an outside system: in this example, there is a simple sleep of one seconds between each step of the sequence but it could for example be changed into a midi clock receiver to synchronize the sequence with music.

The example was developed with the help of Victor, a student we have hired to help-out during the summer. He has then played around a little bit to make a 9 Crazyflies sequence that is more impressive:

I uploaded Victor’s sequence in a github gist as it can be good for inspiration. One bit of warning though: as is, the sequence contains some vertical movements that are quite aggressive and the part where Crazyflies fly directly on top of each-other is more to be considered as a stress test.

Summer is here and temperatures are rising. Since many of us will be on holidays, we will focus this quarter on a special summer clean up! See here what we are working on:

  • Fixing issues: This time we are aiming to close many of the issue tickets in our Github repositories, so that after the summer everything will run much more smoothly (we hope!). Definitely our test rig will come in very hand to sniff out more issues in terms of radio communication as well. You can help as well! Everybody who is using and developing on with the STM-firmware, NRF-firmware or python library, or anything else and is noticing issues, please make a ticket in that same Github repository (if you are familiar with the code) or post about it on our forum (if you do not know exactly what is going on). Together we can make the code better.
  • Lighthouse calibration: In March we released our lighthouse deck for positioning with the HTC Vive base stations. We did feel that the setup process could be improved further, since currently, the Crazyflies’ firmware must contain hardcoded information of the Steam VR’s base station position. We will try to apply the factory calibration direct from the Base stations itself. This will enable us to do 2 additional things: (1) The Crazyflie with the LH deck itself could be used to setup the Lighthouse system, so that SteamVR would not be necessary anymore. (2) Only 1 base station is needed for positioning instead of 2, which will improve the robustness in case of loss of visual-line-of-sight of one base station.
  • Documentation: We try to provide all the possible information for everybody to be able do anything they want with their Crazyflie. But with high flexibility comes great responsibility…. for proper documentation! We are planning to restructure all of our media outlets and try to improve the flow and level of detail for our users. We hope to make it easier for beginning developers to get started and more advanced developers to gain better understanding of the system in order to implement their own awesome ideas. So our very first step is to restructure and clean up the Bitcraze wiki and see where we can add more content.
  • Products: We have a lot of products coming out in the 2nd half of the year
    • AI Deck: We are working hard to get the AI deck all ready for production and we are estimating that they will be available for early access in late autumn. Keep a look out on our forum for regular updates on the progress!
    • Lighthouse breakout board: We made our first working prototype of the lighthouse breakout board, which should make it easier for the lighthouse positioning system to also work on other platforms than the Crazyflie.
    • Active Marker Deck: We are very much on on track with the Qualisys active marker deck! It should be available in the Autumn.
    • Crazyflie Bolt: This has been send off to production for the early access version, which should be available in the Autumn!

Many of our users are flying larger and larger swarms and we’ve been getting some feedback that there’s communication issues when connecting to many Crazyflies. So during the last weeks we’ve been looking at this. Among the things we’re doing is building a test rig where we can automate the communication testing (last weeks blog post). We’ve also fixed a few communication issues listed below.

One of the issues causing problems is dropping packages coming in to the Crazyflie. If the flow of packages was too high to one CRTP port these would start dropping. This has now been fixed by increasing the length of the queues for each port. (GitHub issue)

Another issues has been logging data piling up after disconnect. The detection for the radio disconnection was boken so logging data would continue to be generated and pushed into the communication stack. This has now been fixed so logging will be reset which should clear up he congestion on the next connect. (GitHub issue)

Lastly we also fixed the USB communication issue with dropped packages and crashing when the USB was disconnected. (GitHub issue)

We’ve already noticed a few other issues when using the rig so there should be more fixes coming soon. In the meantime test out the new firmware and let us know if there’s still issues.

While running our ICRA demo, we came across a bug in the Crazyflie python-lib radio handling, limiting the number of Crazyflie that could be controlled using one Crazyradio PA. Communication with many Crazyflies is crucial as flying swarms is becoming more of an interesting topic for research and education. So we decided to take the problem at hand and create a radio test-bench:

To make the test-bench we have attached 10 roadrunner boards to a plank of wood together with USB switches that can provide enough power to the roadrunners. We used the roadrunner because it is mechanically easier to use in this context and it has an identical architecture to the Crazyflie 2.1 when it comes to the radio implementation.

Initially we will use the test-bench to run test scripts that pushes the communication to its limits and that consistently test the communication stack functionalities. This should allow us to find bug and verify that we solve them as well as discovering and documenting limitations.

Eventually we want to connect a raspberry-pi to the test-bench and run tests for each commit and pull-request to the crazyflie-firmware, crazyflie2-nrf-firmware and crazyflie-lib-python projects. This will guarantee that we do not introduce new limitations in the communication stack. The test-bench will also be very useful in implementing new functionalities like direct crazyflie-to-crazyflie P2P communication.

As a final note, the Crazyswarm project is not affected by the Crazyflie-lib bug since it is using the C++ implemented crazyflie-ros driver. Hence Crazyswarm can control more Crazyflies per Crazyradio PA, so it is still the preferred way to fly a swarm mostly when using a motion capture system. Though, with the progress made on LPS and Lighthouse positioning, running swarms, using the python API directly is a probably a more lightweight alternative.

Hi everyone, here at the Integrated and System Laboratory of the ETH Zürich, we have been working on an exciting project: PULP-DroNet.
Our vision is to enable artificial intelligence-based autonomous navigation on small size flying robots, like the Crazyflie 2.0 (CF) nano-drone.
In this post, we will give you the basic ideas to make the CF able to fly fully autonomously, relying only on onboard computational resources, that means no human operator, no ad-hoc external signals, and no remote base-station!
Our prototype can follow a street or a corridor and at the same time avoid collisions with unexpected obstacles even when flying at high speed.


PULP-DroNet is based on the Parallel Ultra Low Power (PULP) project envisioned by the ETH Zürich and the University of Bologna.
In the PULP project, we aim to develop an open-source, scalable hardware and software platform to enable energy-efficient complex computation where the available power envelope is of only a few milliwatts, such as advanced Internet-of-Things nodes, smart sensors — and of course, nano-UAVs. In particular, we address the computational demands of applications that require flexible and advanced processing of data streams generated by sensors such as cameras, which is beyond the capabilities of typical microcontrollers. The PULP project has its roots on the RISC-V instruction set architecture, an innovative academic and research open-source architecture alternative to ARM.

The first step to make the CF autonomous was the design and development of what we called the PULP-Shield, a small form factor pluggable deck for the CF, featuring two off-chip memories (Flash and RAM), a QVGA ultra-low-power grey-scale camera and the PULP GAP8 System-on-Chip (SoC). The GAP8, produced by GreenWaves Technologies, is the first commercially available embodiment of our PULP vision. This SoC features nine general purpose RISC-V-based cores organised in an on-chip microcontroller (1 core, called Fabric Ctrl) and a cluster accelerator of 8 cores, with 64 kB of local L1 memory accessible at high bandwidth from the cluster cores. The SoC also hosts 512kB of L2 memory.

Then, we selected as the algorithmic heart of our autonomous navigation engine an advanced artificial intelligence algorithm based on DroNet, a Convolutional Neural Network (CNN) that was originally developed by our friends at the Robotic and Perception Group (RPG) of the University of Zürich.
To enable the execution of DroNet on our resource-constrained system, we developed a complete methodology to map computationally-intense deep neural networks on the PULP-Shield and the GAP8 SoC.
The network outputs two pieces of information, a probability of collision and a steering angle that are translated in dynamic information used to control the drone: respectively, forward velocity and angular yaw rate. The layout of the network is the following:

Therefore, our mission was to deploy all the required computation onboard our PULP-Shield mounted on the CF, enabling fully autonomous navigation. To put the problem into perspective, in the original work by the RPG, the DroNet CNN enabled autonomous navigation of big-size drones (e.g., the Bebop Parrot). In the original use case, the computational power and memory was not a problem thanks to the streaming of images to a remote base-station, typically a laptop consuming 30-100 Watt or more. So our mission required running a similar workload within 1/1000 of the original power.
To make this work, we combined fixed-point arithmetic (instead of “traditional” floating point), some minimal modification to the original topology, and optimised memory and computation usage. This allowed us to squeeze DroNet in the ultra-small power budget available onboard. Our most energy-efficient configuration delivers 6 frames-per-second (fps) within only 64 mW (including all the electronics on the PULP-Shield), and when we push the PULP platform to its limit, we achieve an impressive 18 fps within just 3.5% of the total CF’s power envelope — the original DroNet was running at 20 fps on an Intel i7.

Do you want to check for yourself? All our hardware and software designs, including our code, schematics, datasets, and trained networks have been released and made available for everyone as open source and open hardware on Github. We look forward to other enthusiasts contributions both in hardware enhancement, as well as software (e.g., smarter networks) to create a great community of people interested in working together on smart nano-drones.
Last but not least, the piece of information you all were waiting. Yes, soon Bitcraze will allow you to enjoy of our PULP-shield, actually, even better, you will play with its evolution! Stay tuned as more information about the “code-name” AI-deck will be released in upcoming posts :-).

If you want to know more about our work:

Questions? Drop us an email (dpalossi at iis.ee.ethz.ch and fconti at iis.ee.ethz.ch)

As mentioned earlier, we will be attending ICRA 2019 and have started to work on the demo that we will run in our booth. The main features we are going to show this time are the Lighthouse positioning deck and the Crazyflie 2.1.

One Crazyflie flying and 5 re-charging on their pads

Running a demo with flying Crazyflies at a conference, usually means a lot of work with changing and charging batteries, starting demos and so on. This takes time and leads to less time to talk to all the interesting people that visit our booth. This year we are aiming at making the demo as fully automated as possible. We will have 8 Crazyflie 2.1 with Lighthouse and Qi charger decks, each with a charging pad. A computer will orchestrate the Crazyflies and make sure one is flying at all times while the others re-charge their batteries. When the battery of the flying Crazyflie is depleted it goes back to its pad while another one takes over.

Most of the functionality is implemented in the Crazyflie firmware and it is pretty much autonomous after the trajectory is started. It will fly the trajectory over and over until it is low on battery, when it goes back to where it started from for recharging. Even though the Lighthouse positioning is very good, it sometimes slips off the charging pad when landing, so we have added a reposition feature to take off again and land if it is not charging after landing.

The orchestration computer (we call it the Control tower) is just keeping track of the state of the flying copter and starts a new one when the flying one goes back the the pad.

We are reusing the spiral trajectory from IROS last year and it has the property that it is possible to run up to 4 copters at the same time without colliding, if they are started at the correct times. There is a swarm feature in the Control tower that runs up to 4 Crazyflies with continuous replacements. The chargers are not fast enough to keep it going and it does not work as expected every time, but it is exciting with a few more Crazyflies buzzing around!

There are some bits and pieces left to implement, like crash detection, but the demo is mostly functional. If you want to play with the (slightly hackish) code, it is available at https://github.com/bitcraze/crazyflie-firmware-experimental/tree/icra-2019

You can find us in booth 101 at ICRA 2019, May 20 – 22. Drop by and say hi, check out the demo and tell us what you are working on. We love to hear about all the interesting projects that are going on. See you there!

We attended the Innovation Week at Lund University on Thursday last week. Primarily we wanted to talk to students and possibly find future colleagues (yes, we are hiring) but it was also a good opportunity to get some demo time with the Lighthouse positioning system.

The demo setup. A bit blurry, sorry!

As mentioned in an earlier blog post, we are going to ICRA in May and we have started to think about what to demo. The main feature will of course be the Lighthouse deck. The setup at Innovation week also served the purpose of a first iteration for the ICRA setup.

We reused an old cage that we created for another fair a couple of years ago, built from a garden tent. It turned out to be fairly wobbly and a bit heavy (steel tubing) considering we will bring it in our luggage to Canada. We probably have to rethink the construction a bit and see if we can change to aluminium.

We put the Lighthouse base stations on tripods, which worked like a charm in our flight lab. We found that we had a lot of problems calibrating the system, not to mention flying the Crazyflie, at the Innovation week fair though. It turned out that the floor was not as stable as one might expect and that the tripods were swaying when people walked by. We solved the problem by adding a tube to the top of tripod that was pushed against the ceiling and thus minimizing the movement. Experience from the real world is always useful!

The general idea for the demo at ICRA is to automate as much as possible to give us more time with visitors. With the high precision of the Lighthouse system, it should be easy to land the Crazyflies on Qi chargers to avoid changing batteries. We hope to set up 6-8 Crazyflies where one is always flying while the others are charging, and have the possibility to temporarily fly more Crazyflies for small swarms. It is still just ideas and we will not see the end result until we are at ICRA, but it will be fun to build!

We have just released the Crazyflie Lighthouse deck as Early Access! It is now available in our web store.

The lighthouse deck allows the Crazyflie to estimate its position using the HTC Vive tracking base-station normally used for Virtual Reality. The positioning is done by tracking the timing of rotating infra-red laser beams emitted from the base-stations. This system has the advantages of having a very good precision and of allowing the Crazyflie to acquire its position autonomously: once the Crazyflie knows the position and orientation of the base-station, it can calculate its own position without the help of any external systems.

The release as Early Access means that we have finished the hardware and we are confident that the hardware is working properly. Though we have not yet finished all the software and firmware, by releasing the hardware early we can get the hardware into the hands of users quickly to try it out. In return we hope we can get some help making the software better.

Current state

  • The Crazyflie can calculate its position from the received Vive Base-Station V1 signals.
  • Direct line of sight should be kept to both base-stations. The Lighthouse deck has 4 receivers so in the future it will be possible to get a position from seeing only one base station.
  • Base-Station V2 support is still being worked-on, it will only require a software update.
  • The Base-station position is hard-coded in the Crazyflie and found using SteamVR. Ideally this should be sent from the ground and the Crazyflie should calculate the positions of the Base-Stations automatically.
  • The previous point means that a full VR system or at least two base stations and a controller or tracker is required to setup the system. In the future we hope to setup the system with only a Crazyflie and two base stations.
  • Since this version of the deck only has horizontal sensors, it is important that the base-stations are placed above the flight space and the Crazyflies should fly ~40cm bellow the base-stations

As long as the deck is in early access, the main documentation will be the lighthouse positioning page in the wiki. This page is going to be updated a lot in the near future and will track the progress in development.

Demo

We have written a small demo script that allows to set the position of the Crazyflie using a Vive controller. It is a good demo to experiment with the precision of the system and the ability to mix VR and Crazyflie since they are in the same tracking space:

In this demo, a python script connects to two Crazyflies and acquire the controller position using OpenVR and makes the Crazyflies take-off above the controller. Then, when the controller trigger is pushed, the setpoint to the closest Crazyflie is changed to follow the controller movement, the Crazyflies are flying autonomously only getting position setpoints from the python script. The position estimation and control is handled onboard.

We are pretty excited by this release since we think this positioning technology will be very useful for a lot of use-case. Let us know what you think and do not hesitate to contribute if you want to improve the system :).

While Crazyflie is nowaday mostly used connected to a computer, we have mobile clients that can be used to fly a Crazyflie using either Bluetooth Low Energy or a Crazyradio with an Android device or an iphone.

The Android client is currently the most advanced one with support for some decks. The goal of these mobile clients is to at least allow to fly a Crazyflie manually, though a lot more could be done by supporting the various decks of the Crazyflie (for example using the flow deck, one might imagine drawing a trajectory on the phone and having the Crazyflie following it :-).

As for development, we have not been very active in the development of the mobile clients and are relying mostly on contributions. So if you are interested into adding functionalities do not hesitate to drop by the Github page of the Android or iOS clients and to propose functionalities and pull requests.

Android client

In 2018, Fred the maintainer of the Android client, has worked hard to stabilize the current app and solve the last few bugs and problem in the current app. A new version was released last week that incorporate all the fixes.

Last years the Android client has seen big internal changes including separating all Crazyflie protocol handling in a separate java library. All these changes will make it easier to implement new functionality in the future and to make the functionality available to android as well as, to any Java program using the Crazyflie java lib.

Iphone client

The iPhone client has seen much less activity in 2018. It has been kept updated with the new versionsd of the Swift language and have seen some bugfixes, all thanks to Github contributors.

There have been reports of a couple of pretty bad bugs that have appeared in the latest release, as soon as these bugs are fixed we plan to release a new version of the iPhone client. The new version will also include the possibility to control the Crazyflie by tilting the phone, and with the bug fixes in place we should be off for a good start of 2019.

Windows client

The Windows UAP Crazyflie client is the least advance of all the mobile Clients. It has the particularity to work on Windows 10 for computers as well as for Phones. This makes it the only implementation of a Bluetooth low energy Crazyflie client for computers. However, Windows 10 for phones being pretty much dead now, the future of this client might be more on the Computer side if any.

Anyway, if anyone is interested in improving the Windows client, we will gladly test and merge pull requests when they come.

Last week we have been focusing on making a release for nearly all our firmware and software. This was done mainly to support the new products we will release this fall but it also contains a lot of other functionality that have been added since the previous release. In this blog-post we will describe the most important features of this release.

New Loco Positioning status and configuration tab

New deck support

The Crazyflie firmware and Crazyflie client 2018.10 adds support for a range of new decks that are about to be released:

  • Flow deck V2 and Z-Ranger V2: New versions of the flow and Z-Ranger deck that uses the new VL53L1 distance sensor. Drivers are implemented in the Crazyflie firmware and the client has been updated to allow flying up to 2 meter in height hold and hover modes when the new decks are detected.
  • Multiranger deck: Diver for the new Multiranger deck is implemented in the Crazyflie firmware, support code is now present in the lib as well as an example implementing the push demo that makes the Crazyflie fly in hover mode using the flow deck and move away from obstacles:

The Flow deck V2 is already available in our webstore. The Z-Ranger V2 and Multiranger will be available in the following weeks, stay tuned on the blog for updated information.

Crazyswarm support

During the year, functionality implemented for the Crazyswarm project has been merged back to the Crazyflie firmware master branch. Practically it means that the Crazyflie firmware 2018.10 is the first stable version to support Crazyswarm. The main features implemented by Crazyswarm are:

  • Modular controller and estimator framework that allows to switch the estimator or the controller at runtime. Practically it means that it is not required to recompile the firmware to use a different controller anymore.
  • Addition of a high-level commander that is able to generate setpoints for the controller from within the Crazyflie. The high-level commander is usable both from Crazyswarm and from the Crazyflie python library. It currently has commands to take-off, land, go to a setpoint and follow a polynomial trajectory. It is made in such a way that it can be extended in the future.
  • Addition of the Mellinger controller: a new controller that allows to fly much tighter and precise trajectories than the PID controller. It is tuned pretty tight so it is currently mostly usable using a motion capture or lighthouse as positioning and togeather with the high-level commander.

Improved and more stable Loco Positioning System

A lot of work has been put in the Loco Positioning System (LPS) this summer. The result of this work is the creation of a new ranging mode: TDoA3. TDoA3 allows to fly as many Crazyflie as we want in the system and to add as many anchors are needed, see our previous blog-post for more information. With this release TDoA 3 is added as a stable ranging mode for LPS. The added features related to LPS are:

  • Added TDoA3 as a ranging mode in the LPS-Node-firmware, the Crazyflie 2.0 firmware and the Crazyflie client
  • Revampted the Crazyflie client LPS tab and communication protocol to handle more than 8 anchors
  • Implementation of a new outlier detector for TDoA2 and TDoA3 that drastically improve positioning noise and flight quality

Release notes and downloads

As usual the release build and release note is available on Github. The Crazyflie client and lib are also available as python pip package as cfclient and cflib.