Category: Random stuff

As you have noticed, we talk about the lighthouse positioning a lot these last couple of months ever since we got it out of early release. However, it is good to realize that it is not the only option out there for positioning your Crazyflie! That is why in this blog-post we will lay out possible options and explain how they are different/similar to one another.

The four possible ways to position the crazyflie

Absolute Positioning / Off-board Pose Estimation

Absolute Positioning and External Pose Estimation with the MoCap System

The first we will handle are the use of motion capture systems (MoCap), which resolves around the use of InfraRed cameras and Markers. We use the Qualysis camera ourselves but there are also labs out there that use Vicon or Optitrack. The general idea is that the cameras have an IR-light-emitting LED ring, which are bounced back by reflective markers that are supposed to be on the Crazyflie. These markers can therefore be detected by the same cameras, which pass through the marker positions to an external computer. This computer will have a MoCap program running which will turn these detected markers into a Pose estimate, which will in turn be communicated to the Crazyflie by a Crazyradio PA.

Since that the positioning is estimated by an external computer instead of onboard of the crazyflie, a MoCap positioning system is categorized as an off-board pose estimation using an absolute positioning system. For more information, please check the Motion Capture positioning documentation.

Absolute Positioning / On-board Pose Estimation

Absolute Positioning and Internal Pose Estimation with the Lighthouse and Loco Positioning System

The next category is a bit different and it consists of both the Loco positioning system and the Lighthouse positioning system. Even though these both use beacons/sensors that are placed externally of the Crazyflie, the pose estimation is done all on-board in the firmware of the Crazyflie. So there is no computer that is necessary to communicate the position back to the Crazyflie. Remember that you do need to communicate the reference set-points or high level commands if you are not using the App layer.

Of course there are clear differences in the measurement type. A Crazyflie with the Locodeck attached takes the distance to the externally placed nodes as measured by ultra wide band (UWB) and the Lighthouse deck detects the light plane angles emitted by the Lighthouse Base Stations. However the principle is the same that those raw measurements are used as input to the Extended Kalman filter onboard of the Crazyflie, and outputs the estimated pose after fusing with the IMU measurements.

Therefore these systems can be classified as absolute positioning systems with on-board pose estimation. To learn more please read the Loco and Lighthouse positioning system documentation!

Relative Positioning / On-board Pose Estimation

Relative Positioning and Internal Pose Estimation with the Flowdeck V2.

It is not necessary to have to setup an external positioning system in your room in order to achieve a form of positioning on the Crazyflie. With the Flowdeck attached, the Crazyflie can measure flows per frame with an optical flow sensor and the height in millimetres with a time of flight sensor. These measurements are then fused together with the IMU within the Extended Kalman filter (see the Flow deck measurement model), which results in a on-board pose estimation.

The most important difference here to note is that positioning estimated by only the Flowdeck, will not result in a absolute positioning estimate but a relative one. Instead of using an external placed system (like MoCap, Lighthouse and Loco) which dictate where the zero position is in XYZ, the start-up position the Crazyflie determines where the origin of the coordinate system is. That is why the Flowdeck is classified as a Relative Positioning System with On-board Pose Estimation.

IMU-only On-board Pose Estimation ?

Oh boy… that is a different story. Theoretically it could be possible by using the onboard accelerometers of the crazyflie and fusing those in some short of estimator, however practice has shown that the Crazyflie’s accelerometers are too noisy to result in any good pose estimation… We haven’t seen any work that has been successfully to achieve any stable hover on only the IMU of the Crazyflie, but if you have done/see research that has, please let us know!

And if you would like to give a go yourself and build an estimator that is able to do this, please check out the new out of tree build functionality for estimators. This is still work in progress so it might have some bugs, but it should enable you to plugin in your own estimator separate from the Crazyflie firmware ;)

Documentation

We try to keep keep all the information of all our positioning systems on our website. So check out the positioning system overview page to be referred to more details if you would be interested in a particular system that fits your requirements!

Now that the Lighthouse deck is out of early access and we have made it easier to setup a lighthouse positioning system, we are currently at the next stage: showing how awesome it is! We feel that there are not enough people out there that know about the Lighthouse positioning system and sometimes confuse it even with the Loco position system (to be honest, the abbreviation LPS makes it challenging). But we are confident that the Lighthouse system is a good alternative for those that want to do drone research but are on a tight budget.

The area of the data collection. from the paper

Lighthouse Dataset

During Wolfgang Hönig‘s time here at Bitcraze, one of the bigger projects we worked together on was to generate a dataset comparing the positioning quality of the Lighthouse system with a Motion Capture (MoCap) system. You could imagine that would be a difficult task, since as the lighthouse basestations transmit infrared light sweeps and MoCap cameras by default also emit IR light which are reflected back by markers. However, with the Active marker deck for the Qualysis system, we were able to use the MoCap and Lighthouse positioning without too much interference.

Moreover, Wolfgang also helped out with improving the logging quality on the Micro-SD-card deck which also enabled us to get as much data real-time as possible. He wrote a blogpost about event-based logging a few weeks ago which is a new approach to record data on the Crazyflie at a fast pace. With the Active Marker Deck, the Micro-SD-card deck and of course the Lighthouse deck, … the Crazyflie turn into a full-blown positioning data-collection machine!

The configuration of the Crazyflie with the Micro-SD-card deck, the Lighthouse-deck from the lighthouse dataset paper

Paper

About this whole process, we wrote the following paper:
Lighthouse Positioning System: Dataset, Accuracy, and Precision for UAV Research,
A.Taffanel, B. Rousselot, J. Danielsson, K. McGuire, K. Richardsson, M. Eliasson, T. Antonsson, W. Hönig, ICRA Workshop on Robot Swarms in the Real World, Arxiv 2021

This paper contains an short explanation of the lighthouse system, how we set up the data collection and an analysis of the results, where we compared both Lighthouse V1 and V2 with the Crossing beam (C.B.) method and the extended Kalman filter. In all cases, the mean and median Euclidean error of the Lighthouse positioning system are about 2-4 centimeters compared to our MoCap system as ground truth.

Check out the lighthouse dataset paper to read all the details of the experiments!

The Euclidean Error of both LH1 and LH2 with Mocap as ground-truth taken from the dataset paper.

ICRA Swarm Workshop

Our paper is selected for a poster presentation at the ICRA 2021 Workshop: Robot Swarms in the Real World. So if you have any questions about the paper, please join and ask us in person! The workshop will be held on the 4th of June.

Moreover, we also are sponsoring the event by giving away a Lighthouse Swarm Bundle to whomever wins the best video-demonstration award! So to all the participants, the best of luck! We are super curious to what you’ll have to show us.

During the last year we, like many others, have had to deal with new challenges. Some of them have been personal while some have been work related. Although it’s been a challenge for us to distribute the team, we’ve tried to refocus our work in order to keep making progress with our products.

Our users have of course had challenges of their own. Having a large user-base in academia, we’ve seen users having restricted physical access to hardware and purchasing procedures becoming complex. Some users have solved this by moving the lab to their homes using the Lighthouse positioning or the Loco positioning systems. Others have been able to stay in their labs and classrooms although under different circumstances.

At Bitcraze we have been able to overcome the challenges we’ve faced so far, largely thanks to a strong and motivated team, but now it seems as if one of our biggest challenges might be ahead of us.

Semiconductor sourcing issues

Starting early this year lead-times for some components increased and there were indication that this might become a problem. This has been an issue for other parts like GPUs and CPUs for a while, but not for the semiconductors we use. Unfortunately sourcing of components now has become an issue for us as well. As far as we understand the problems have been caused by an unforeseen demand from the automotive industry combined with a few random events where the production capacity of certain parts has decreased. Together it has created a global semiconductor shortage with large effects on the supply chain.

For us it started with the LPS Node where the components suddenly cost twice as much as normally. The MCU and pressure sensor were mainly to blame, incurring huge price increases. Since these parts were out of stock in all the normal distributor channels, the only way to find them was on the open market. Here price is set by supply and demand, where supply is now low and demand very high. Prices fluctuate day by day, sometimes there’s very large swings and it’s very hard to predict what will happen. To mitigate this for the LPS Node we started looking for an alternative MCU, as the STM32F072 was responsible for most of the increase (600% price increase). Since stock of the LPS Node was getting low we needed a quick solution and found the pin-compatible STM32L422 instead, where supply and price was good through normal channels. The work with porting code started, but after a few weeks we got word that importing this part to China was blocked. So after a dead end we’re back to the original MCU, with a few weeks of lead-time lost and a very high production price.

Unfortunately this problem is not isolated to the LPS Node, the next issue we’re facing is the production of the Crazyflie 2.1 where the STM32F405, BMI088, BMP388 and nRF51822 are all affected with increases between 100 and 400 % in price. These components are central parts of the Crazyflie and they can not easily be switch to other alternatives. Even if they could, a re-design takes a long time and it’s not certain that the new parts are still available for a reasonable price at that time.

Aside from the huge price increases in the open market we’re also seeing price increases in official distributor channels. With all of this weighed together, we expecting this will be an issue for most of our products in the near future.

Planning ahead

An even bigger worry than the price increase is the risk of not being able to source these components at all for upcoming batches. Having no stock to sell would be really bad for Bitcraze as a business and of course also really bad for our customers that rely on our products for doing their research and classroom teaching.

To mitigate the risk of increasing price and not being able to source components in the near future, we’re now forced to stock up on parts. Currently we are securing these key components to cover production until early next year, hoping that this situation will have improved until then.

Updated product prices

Normally we keep a stable price for a product once it has been released. For example the LPS Node is the same price now as it was in 2016 when it was released, even though we’ve improved the functionality of the product a lot. We only adjust prices for hardware updates, like when the Crazyflie 2.0 was upgraded to Crazyflie 2.1. But to mitigate the current situation we will have to side-step this approach.

In order for us to be able to continue developing even better products and to support those of you that already use the Crazyflie ecosystem, we need to keep a reasonable margin. From the 1st of May we will be adjusting prices across our catalog, increasing them with 10-15%. Although this doesn’t reflect the changes we are seeing in production prices at the moment, we believe the most drastic increases are temporary while the more moderate ones will probably stick as we move forward.

Even though times are a bit turbulent now, we hope the situation will settle down soon and we think the actions we are taking now will allow us to focus on evolving our platform for the future.

Approximately two month ago we wrote a blog post presenting our planned master thesis. Time flies and we have now reached a sufficient state where the results are presentable and possible to use. We have used the Renode framework and created a platform for the Crazyflie 2.1. In Bitcraze’s Github repository there now is a Renode fork with a custom Renode-infrastructure submodule. To get Renode up and running on your computer check the README found there.

On the Renode branch crazyflie there are two new REPL (Renode platform) files describing the platform. An example of the syntax is given below.

// I am a comment
peripheralname: Namespace.ClassName @ parent 0x08000000
    numericConstructorField: 0x100000
    stringConstructorField: "template"
    Interrupt -> interrupthandler@3

In the cf2.repl all the external peripherals are connected while the stm32f405.repl contains the STM32F405 peripherals. Note that only the peripherals used by the current Crazyflie 2.1 have been added since they are the only ones that can be tested using the Crazyflie firmware.

When running Renode a RESC (Renode script) file is loaded. There are currently two RESC files for the Crazyflie, one that only loads the Crazyflie plattform and one that can be used for testing. The one for testing automatically starts the simulation and it also has a hook to exit Renode once the self test has passed.

Successful startup!

As mentioned, Renode is usable both for automatically and interactively testing firmware. The current version of automatic testing is based on the firmware passing the self test. The plan is to incorporate this in the CI pipeline.

When used interactively it is possible to pause the emulation whenever the user wants to, either manually via a Renode command or by connecting to GDB. This allows reading (as well as writing to) memory addresses. Want to read the DMA status at a specific line of code or mess with the system by randomly flipping bits? Doable in the emulator without risking your Crazyflie crashing.

In the platform there also are our customized sensors to which data can be loaded. The data can be loaded either manually or via a file and then sent to the STM32. The scope of this master thesis however has been on firmware testing, not getting a simulated Crazyflie to fly in a virtual environment.

Emulation of hardware is not a trivial task, there are still improvements to be made and the everlasting question whether the emulation actually represents the real system.

One of the Crazyflie features that had to be simplified was the syslink and connection to the nRF microcontroller, which in the emulation simply sends messages back to the STM32. The most exciting part about it currently is how the STM32 receives a signal that no expansion decks are connected via the 1-wire when a scan is executed. Further improvements would be to emulate the radio, power management and support expansion decks, either via an external program or through Renode.

Of course there are other things to improve as well, there will always be someone who thinks of better ways to implement features and only time will tell how this emulator is going to evolve.

Josefine & Max

One crucial aspect of any research and development is to record and analyze data, which then can be used for quantifying performance, debugging strange behavior, or guide us in our decision making. We have been trying to improve the way that this is done to help all of you researcher out there with their work, so this blog post will explain an alternative, better, method to record whatever is happening on the Crazyflie in real-time.

Example data collection of received Lighthouse angles over time. The y-axis contains a unique measurement ID (16 in total for 4 sensors * 2 basestations * 2 sweeps/basestation). Thus, each dot represents the time when a certain kind of measurement was received. We do not receive all angles in fixed intervals, because of the interference between basestations.

Existing Logging Approaches

So far there have been two principle ways of recording data with the Crazyflie:

  1. Logging: In the firmware, one can define global variables that can be streamed out at a fixed frequency, using logging configurations. Variables have a name and data type, and the list of all available variables can be queried from the firmware.
  2. Debug Prints: It is possible to add DEBUG_PRINT(…) in any place that contains a string and possibly some variables. These are asynchronous, but not timestamped, so they are mostly useful to notify the user of some status change.

Both approaches have multiple backends. The logging variables can be streamed over USB, via the Crazyradio, or on a Micro-SD-card using a specialized deck. When using the radio or USB, the frequency is limited to 100 Hz and only a few variables can be streamed at this speed due to bandwidth limitations. The Micro-SD-card deck, on the other hand, allows to log a lot more logging variables at speeds up to 1 kHz, making it possible to access high-speed sensor information, e.g., the IMU data. The Debug Prints can be used over the radio or USB, or with a SEGGER J-Link and a Debug Adapter, over J-Link Real-Time Transfer (RTT). The former is very bandwidth limited, while the latter requires a physical wire connection, making it impossible to use while flying.

New Approach: Event-Based Logging

Event-based logging combines the advantages of the existing approaches and offers a third alternative to record data. Similar to debug prints, a user can trigger an event anywhere in the code and include some mandatory variables that describe this event (so-called payload). Similar to logging, these events are timestamped and have fewer bandwidth limitations. Currently, there is just one backend for the Micro-SD-card deck, but it would be possible to add support for radio and USB as well.

One of our first test cases of the event-based logging is to analyze the data we get from the Lighthouse deck. Here, we trigger an event whenever we received a raw sweep. This allows us to visually see the interference that happens between two LH2 basestations. We also use the event-based logging for time synchronization of the Crazyflie and a motion capture system: when we enable the IR LEDs on the active marker deck, we record an event that contains the Crazyflie timestamp. On the PC side, we record a PC timestamp when the motion capture system first detects the IR LEDs. Clock drifts can be computed by using the same logging mechanism when the IR LEDs are turned off.

Adding event-based logging had some other good side effects as well: the logging is now generally much faster, there is more user feedback about the correct buffer size usage, and the binary files are smaller. More details are in the documentation.

Future Work

We are working on adding event triggers to the state estimator. This will enable us to record all the sensor information during real flights with different positioning systems, so that we can tune and improve the Kalman state estimator. It will also be interesting to add support for events in CRTP, so that it can be used over USB and radio.

Hi all! I am Jonas Danielsson and I started at Bitcraze on the first of March. I am a software developer with experience of Linux-, open source- and embedded development. Sometimes all at once.

Jonas and Bosse out hiking

I have worked with embedded systems of different sizes since 2007 and look forward to getting to know the Crazyflie and all of its ecosystem.

I am interested in and care a lot about software- and product maintenance. At Bitcraze I hope to be involved both in developing new features, fixing bugs and finding ways of keeping the product and code in nice shape. To find ways of working that allow us to add a bunch of cool stuff as fast as we want to, without endangering the cool stuff we already have.

Also, I am interested in getting to know the Crazyflie community and to work together with you to create the best development experience we can achieve. Do not be afraid to reach out with ideas or suggestions on how we can improve!

Communicating with your Crazyflie is an important pillar for its operation. As more robots are controlled, the reliability of this communication link becomes more and more important, as the probability that there is no failure at any of the Crazyflies decreases exponentially with the number of robots. We have written about the low-level radio link before. Today, we focus on past, on-going, and future improvements to make the communication more reliable.

Reliability Challenges

As part of doing research with the Crazyswarm, I noticed several issues:

  1. Sometimes commands do not seem to arrive at the Crazyflie, especially when using unicast (i.e., direct) communication with a specific Crazyflie.
  2. Sending ‘too much’ data while the Crazyflie is flying can cause catastrophic crashes. One example is trying to upload a trajectory, while flying in a motion capture space. However, this occasionally even happens if just trying to update a parameter.

The radio link has a feature called Safelink, which essentially guarantees that packets are send and received in order and no packet gets dropped. This feature never worked reliably in the crazyflie_cpp implementation and is therefore not used in the Crazyswarm. However, it is the default mode for cflib. The Crazy RealTime Protocol (CRTP) also has the notion of different communication ports in order to prioritize important messages such as control commands over less important ones such as trajectory upload. However, this prioritization was never implemented in any of the clients.

Native Link Implementations

Another, non reliability related, issue always was that the performance of cflib is not stellar when connected to many Crazyflies. This is mostly because (C)Python has a Global Interpreter Lock (GIL), which prevents true multi-threaded operation. A common solution to allow true parallelism is multiprocessing or implementations in a language that compiles to machine code (native code).

Crazyflie-link-Cpp

The first native implementation is written in C++ and includes Python bindings using pybind11. The overall API is simple: A connection can be created given a URI, and data can be send and received. Internally, this library implements Safelink to guarantee packets are in order, uses priority queues to prioritize messages on important CRTP ports, and handles all multi-radio and multi-thread related synchronization issues. Unlike prior implementations, the bandwidth is shared uniformly between all connections, i.e., there is no race between different Crazyflies communicating over the same radio. We measured a 50% lower CPU utilization when connecting with the CF client, a 20% higher bandwidth, and a 20% lower latency compared to the pure Python implementation.

For now, this feature is experimental and needs to be enabled using an environment variable. If you want to give it a try, you can use

pip install cflinkcpp
USE_CFLINK=cpp cfclient

once everything is merged to master in the next couple of days.

Crazyflie-link-rs

The second native implementation is written in Rust. As it turns out, writing multi-threaded code in C++ is very difficult and error-prone, because it is up to the user to use the various synchronization primitives correctly. Rust, on the other hand, provides extensive compile-time checks to increase reliability by design. This implementation does not have quite as many features as the C++ version yet and requires a bit more work to be used with the client but it is going to be worked-on on Fridays. The goal is to reach functional parity with the other link implementations and be able to use the rust-implemented link with “USE_CFLINK=rs cfclient” in the future.

Future Work

As part of the development process, we also wrote many system tests that verify correctness and measure performance of the various link implementations. We already found and fixed several firmware bugs (681 and 688). The major open issue is that there is no flow control on the system link (the connection between the NRF51 and the STM32), which still causes packet loss in some cases.

Hi!

Since the middle of January, Bitcraze has had two additional guests: us! We, Josefine & Max, are currently doing our master thesis at Bitcraze during our final semester at LTH.

Unfortunately, the pandemic means remote working which could have caused some difficulties with hardware and equipment accessibility. Fortunately, for us, our goal with the thesis is to emulate the Crazyflie 2.1 hardware in the open source software Renode. We can therefore do the work at home.

Since this is the first time either of us have tried to emulate hardware it is exciting to see if it will even be possible, especially as we do not know what limitations Renode might have. Thus far, four weeks in, there have been some hiccups and crashes but also progress and success. One example was when we got the USART up and running and it became possible to start printing debug messages and another was when the LED lights were connected and it was possible to see when they were turned on and off. 

Example of output from partially implemented hardware.

If all goes well, the emulated hardware could be used as a part of the CI pipeline to automatically hunt for bugs. Academically, it would be used to further study testing methods of control firmware.

Getting a glimpse into the workings at Bitcraze and the lovely people working here has been most interesting and we are looking forward to the time ahead. Until next time, hopefully with a working emulation, Josefine & Max.

The lithium polymer battery we use, as with basically all rechargeable batteries, suffers from degradation. That means that when using it, and as time goes by, its energy capacity as well as performance will degrade. The performance, which is very related to the batteries internal resistance, will result in that the Crazyflie will not be able to produce the same maximum thrust and it will not be able to carry as much payload. The loss of capacity is due to ageing and charge cycles, results in that the flight time will decrease. A common solution to monitor the degradation is to have a BMS, or Battery Management System, that constantly monitors the battery health. For the small type of battery that is used in the Crazyflie, this is not yet viable, but maybe there is something we can do to test part of the battery health anyway?

Theory

Since the internal resistance will result in a larger voltage drop during load we can exploit this property and measure it. We will however not only measure the batteries internal resistance but the resistance of the complete power path as a result of the components we have at hand on the Crazyflie.

Power path block diagram

So what we do is to activate the switch (mosfet) so the load (motor) will pull power from the battery. The power drawn will result in a voltage drop compared to a no-load situation, which we can measure and compare to a healthy setup. Since the measurement point is at the PCB traces, any of the components before that point can be causing the voltage drop, however the battery and connector are most likely of doing so as they are most prone to ageing.

Implementation

The load is achieved by, for a very short time, activate the motors at full thrust. We don’t want the Crazyflie to fly away as that would be a bit unhandy. Before activating the motors we measure the idle voltage and during load we measure the minimum voltage so we can calculate the voltage drop. This is pretty easy to do, the problem is to find a good level where we can distinguish a good battery from bad battery. Therefore this feature is pretty experimental. We tested many batteries and good batteries tend to yield a voltage drop between 0.60V – 0.85V while bad batteries go above 1.0V. Therefore the current threshold is set to 0.95V but it would be good to have more data so if you use this feature please give us feedback if the level is wrong. The testing was run on a “stock” setup with the standard battery, propeller and motors, and it is for these the level is set. A different setup will probably not work well and needs a different threshold. Also keep in mind that the connector can also be a “bad” guy as oxide can build up and result in a higher resistance. Often this can be solved with some e.g. WD-40 solvent or un-connecting/connecting the connector several times.

Usage

This is not in the 2021.01 release so one would have to run the latest on the master branch on both the crazyflie-firmware and the Crazyfie-clients-python. The simplest way to test this feature is to launch the cflient, connect to the Crazyflie, open the console tab and press the battery test button.

cfclient console tab after running battery test

When pressing the button the propellers will shortly spin and there will be an output in the console as highlighted in red. If the sag value is below 0.95V it will yield [OK] and if it is above it will say [FAIL].

A probably more useful use case is to test this automatically before taking of with e.g. a swarm. This can be done by setting the parameter health.startBatTest to 1 and after around 0.5s readout the result in the log variable health.batteryPass to check that it is set to 1. The health.batterySag log variable will contain the latest sag (voltage drop) measurement. Hopefully this experimental feature will be a good way of increasing reliability of flights.

We’re happy to announce the availability of the 2021.01 release! The release includes the Crazyflie firmware (2021.01), the python library (0.1.13.1) and the python client (2021.1). The firmware package can be downloaded from the Crazyflie release repository (2021.01) or can be flashed directly using the client bootloader window.

Most of the improvements have been done in the Crazyflie firmware and include:

The App API in the Crazyflie firmware has been extended and improved to be able to handle a wider range of applications. The goal is to enable a majority of users to implement the functionality they need in an app instead of hacking into the firmware it self.

We have improved the Lighthouse support in the firmware and both V1 and V2 are now working well. Even-though everything is not finished yet, we have taken a good step towards official Lighthouse positioning.

A collision avoidance module has kindly been contributed by the Crazyswarm team.

A persistant storage module has been added to enable data to be persisted and available after the Crazyflie is power cycled. It will initially be used to store Lighthouse system information, but will be useful for many other tasks in the future.

Basic arming functionality has been added for platforms using brushless motors.

In the client the LPS tab now has a 3D visualization of the positioning system and a new tab has been added to show the python log output.

Unfortunately we have run into some problems for the Windows client build which is not available for this release.

Finally we have fixed bugs and worked to improve the general stability.

We hope you enjoy it!