Category: Software

One of the pain points when setting up the Loco Positioning system is to measure the anchor positions and enter them into the system. I wanted to see if I could automate this task and let the system calculate the positions, and if so understand what kind of precision to expect. I have spent a few Fun Fridays playing with this problem and this is what I have found so far.

The problem can be broken down into two parts:
1. How to calculate the anchor positions. What data is required?
2. How to define the coordinate system. To make it useful the user must to be able to define the coordinate system in a simple way.

Anchor and ruler

How to calculate the anchor positions

The general idea of how to calculate the anchor positions is to set up a system of equations describing the distances between the anchors and/or the Crayzflie and solve for the anchor positions. The equations will be non linear and the (possibly naive) plan is to use the Gauss Newton method to solve the system.

To understand how to calculate the anchor positions we must first take a look at the data that is available. The Loco Positioning system can be run in two different modes: Two Way ranging (default mode) and TDoA.

Two way ranging

In the Two Way ranging mode we measure the distance between each anchor and the Crazyflie and to get enough data we must record ranging data for multiple positions. The anchor positions are unknown, and for each new Crazyflie position we add yet a new unknown position, on the other hand we measure the ranges to the anchors so these are knowns. 

The equations used are simply to calculate the distance between the assumed position of each anchor and the Crazyflie and then subtracting it from the measured distance.

TDoA

In TDoA we measure the Time Difference of Arrival, that is the difference in distance to two anchors from the Crazyflie’s position. It is probably possible to use this information, but I was looking for a different solution here. In our new TDoA implementation that we have been playing with a bit, we get the distance between all anchors (calculated in the anchors) as a side effect. 

In this case the Crazflie is not really needed and the equations describe the distance between assumed anchor positions versus measured distances.

How to define the coordinate system

To get a useable positioning system, the coordinate system must be well defined and oriented in a practical direction. For example when writing a script you probably want (0, 0, 0) to be at some specific spot, the X-axis pointing in a certain direction, the Z-axis to point up and so on. My initial idea was to use the anchors to define the coordinate system, use anchor 0 as (0, 0, 0), let the X-axis pass through anchor 1 and so on. Just by looking at our flight lab I realised that this would be too limiting and decided that the coordinate system should be completely disconnected from the anchor positions, but still easy to define. I also realised that a really good way to tell the system about the desired coordinate system would be to move the Crazyflie around in space to show what you want. The solution is to place the Crazyflie at certain positions and click a button to record data at these positions. The steps I have chosen are:

  1. Place the Crazyflie at (0, 0, 0)
  2. Place the Crazyflie on the X-axis, X > 0
  3. Place the Crazyflie in the XY-plane, Y > 0
  4. Move the Crazyflie around in the space with continuous recording of data

In this scheme the XY-plane is typically the floor.

Results

I have written basic implementations for both the Two Way ranging and TDoA modes and they seem to work reasonably well in simulations. I have also tested the Two Way ranging algorithm in our flight lab with mixed results. The solution converged in most cases but not always. When converging the estimated anchor positions ended up in the right region but some were off by up to a meter. Finally I did run the algorithm and fed the result into the system and managed to fly using the estimated positions which I find encouraging.

I will continue to work on this as a Friday Fun project and maybe it will make its way into the client code base at some point in the future? There are probably better ways to estimate the anchor positions and more clever algorithms, feel free to share them in the comments.

 

 

A few weeks ago we wrote about a new prototype that we call “the obstacle avoidance deck”. Basically it’s a deck fitted with multiple VL53L0x ToF distance sensors that measures the distance front/back, right/left and up of the Crazyflie 2.0. Combined with the Flow deck this gives you an X/Y/Z robot that you can program fly around avoiding obstacles which doesn’t need any external positioning system.

After implementing firmware support for the deck (see #253 and #254) we’ve finally had a chance to do some initial testing, see the video below. In the current implementation we’re doing the measurements in the firmware but using the logging framework to get all the distances into a Python script which does the movement control. Since we have the Flow deck attached we can control the Crazyflie 2.0 in velocity mode, which means we can say things like “Go forward with 0.5 m/s until the forward sensor shows a distance lower than 50cm” or “Go forward 1 m/s for 1s and rotate to measure the distance to all objects”. Since there’s no real-time requirements we can move the complexity of the algorithm from the firmware into external scripting which makes it a lot easier to develop. Now we’re really eager to start setting up obstacle courses and time how fast we can move though them 🙂

The results from the testing shows that our two main concerns aren’t an issue: The sensors doesn’t seem to interfere with each other and we can sample them all at high-enough frequency without occupying the bus too heavily (currently we’re doing 20Hz). The next step is figuring out the requirements (i.e how many VL53L0x sensors are needed, do we really need the back one?) and a mechanical solution for attaching the sensors in production. If there’s any feedback let us know now and we’ll try to get it into the design. Also, we really need a new name for the board. Any suggestions?

We have had an implementation of a Time Difference of Arrival algorithm (TDoA) in the Crazyflie 2.o and the Loco Positioning System for quite a long time. The coolest feature of the TDoA algorithm is that it can be used to position virtually unlimited number of Crazyflies concurrently as opposed to the standard Two Way Ranging algorithm that is limited to one (or very few Crazyflies). The original implementation is working pretty well but contains some flaws that we are not completely happy with, hence we have not released it officially and are still calling it experimental. Since support for tracking multiple objects is a requirement for flying swarms and we like swarms, we have started to iron out the problems. 

A small swarm using the old TDoA algorithm, from February 2017.
 

In the current implementation each anchor transmits the time of the transmission and the times of the latest reception of transmissions from all other anchors about every 16 ms. From this information it is possible to calculate the difference in time of flight for the radio waves from two anchors to the Crazyflie. When we know the difference in time of arrival, by multiplying with the speed of light we get the difference in distance and can calculate the position of the Crazyflie. This all sounds fine and dandy but the set up has some problems, the biggest one being error handling. If one or more packets are lost, either from anchor to Crazyflie or anchor to anchor, there is no deterministic way to detect it in some cases. The current algorithm relies on sanity checking the calculated result and discarding data that looks suspicious, which is usually easy as the distances quickly gets unrealistic (several thousands of meters!). We suspect that some erroneous values slip through the check though and we would like to be able to really understand when data is valid or not.

TDoA 2.0

What we are working on now is to add a sequence numbering scheme to enable the receiving party to understand when a packet has been lost. With this information it will be possible to discard bad data as well as use the available information better. While we are re-writing the code we are also moving a part of the algorithm from the Crazyflie to the anchors, after all there is a CPU in the anchors that is not fully utilized. The idea is to let each anchor continuously calculate the distance to all other anchors and add this information to the messages it transmits, which will reduce the work in the Crazyflie.

This is work in progress and we are not completely sure where we will end up, but we are aiming at making the TDoA mode part of the official release at some point.

Sensor fusion

Related to the Loco Positioning system is our line of other positioning sensors; the Z-ranger deck and the Flow deck. The Flow deck has really good precision at low altitudes but can not provide absolute positioning while the Loco Positioning system does not have the same precision but absolute position capabilities. So what if we fuse the information from the Flow deck with the Loco Positioning system? We have tried it out and it works pretty well, we can get the best of two worlds! The Z-ranger can also be used in the same way to improve the Z component of the estimated position when flying bellow ~1m.

Even though it works using multiple positioning sensors at the same time, there is room for improvements and some tweaking will be required to make it rock solid.

 

 

The Crazyflie, the original one, usually called Crazyflie 1 to avoid confusion, was the first commercially available open source nano quadcopter back in 2013. After getting feedback on the platform and having a lot of ideas of things that could be improved, we developed the Crazyflie 2.0 during 2014 and released the same fall. We decided we wanted to keep full backward compatibility with Crazyflie 1, both in the firmware project and in the different clients and support libraries even though we now had more processing power and RAM.

But during the last year we’ve almost exclusively been adding functionality that is Crazyflie 2.0 specific, while still trying to stay inside the constraints of the Crazyflie 1. We’ve also seen a decline in the discussions and interest of the Crazyflie 1. So this week when we once again broke the build because we run out of RAM in the Crazyflie 1, we decided to remove the Crazyflie 1 support from the Crazyflie firmware project. It’s of course with a heavy heart we do this, but we feel that in order for the Crazyflie 2.0 to move forward it’s a must. The last release that’s compatible with the Crazyflie 1 is 2017.6.

But if you’re still using the Crazyflie 1, don’t worry we’re not completely dropping it, we will continue to carry spare parts and if anyone wants to continue firmware development we will be happy to assist. If anyone is motivated, the code can be branched from the last release and we could make a new repository to host the Crazyflie 1 code.

It is summer again in Sweden and things are now starting to slow down and people are going to vacation. The last couple of years we have used the summer to look back and clean-up the technical dept accumulated during the year: when trying to get things done we have to prioritize which means that some things have to be left on the side (at least until we invent a way to add more hours to each day). This year is no exception, the last couple of weeks we have been working very hard to get the Flow products out and now the production is hopefully on rails so the cleanup can begin.

There is a lot of things we could do but here is a sneak peek of what we are currently looking at:

  • Crazyflie Client gamepad handling and configuration: The current input device handling is complicated and the architecture is hard to work with. There is a lot that can be done both in the front-end and the back end to make it easier to use and to work with.
  • Loco positioning system support for multiple Crazyflie, we have two mode implemented for that, TWR-TDMA and TDoA, both are very experimental and need some more work.
  • Cleanup of the webpage, information and documentation: we already have done a lot of work to make better documentation but there is always margin for improvement.
  • Cleaning up and improving the Crazyradio firmware: the Crazyradio starts to show its limits when flying swarms of Crazyflie. There is some improvement that could be done in the Firmware to make it more efficient. The first step is clean up the current implementation.

If you have any ideas of areas you feel we should focus on, even better if you want to help with some things and fix it together with us, just tell us in the comment.

On a side note, the manufacturing of the Flow products is still on progress and it should soon be on the Bitcraze shop and the Seeedstudio bazaar, stay tuned.

This week’s Monday post is a guest post written by members of the Computer Science and Artificial Intelligence Lab at MIT.

One of the focuses of the Distributed Robotics Lab, which is run by Daniela Rus and is part of the Computer Science and Artificial Intelligence Lab at MIT, is to study the coordination of multiple robots. Our lab has tested a diverse array of robots, from jumping cubes to Kuka youBots to quadcopters. In one of our recent projects, presented at ICRA 2017, Multi-robot Path Planning for a Swarm of Robots that Can Both Fly and Drive, we tested collision-free path planning for flying-and-driving robots in a small town.

Robots that can both fly and drive – in particular wheeled drones – are actually somewhat of a rarity in robotics research. Although there are several interesting examples in the literature, most of them involve creative ways of repurposing the wings or propellers of a flying robot to get it to move on the ground. Since we wanted to test multi-robot algorithms, we needed a robot that would be robust, safe, and easy to control – not necessarily advanced or clever. We decided to put an independent driving mechanism on the bottom of a quadcopter, and it turns out that the Crazyflie 2.0 was the perfect platform for us. The Crazyflie is easily obtainable, safe, and (we can certify ourselves) very robust. Moreover, since it is open-source and fully programmable, we were able to easily modify the Crazyflie to fit our needs. Our final design with the wheel deck is shown below.

A photo of the Crazyflie 2.0 with the wheel deck.

A model of the Crazyflie 2.0 with the wheel deck from the bottom

The wheel deck consists of a PCB with a motor driver; two small motors mounted in a carbon fiber tube epoxied onto the PCB; and a passive ball caster in the back. We were able to interface our PCB with the pins on the Crazyflie so that we could use the Crazyflie to control the motors (the code is available at https://github.com/braraki/crazyflie-firmware). We added new parameters to the Crazyflie to control wheel speed, which, in retrospect, was not a good decision, since we found that it was difficult to update the parameters at a high enough rate to control the wheels well. We should have used the Crazyflie RealTime Protocol (CRTP) to send custom data packages to the Crazyflie, but that will have to be a project for another day.
The table below shows the mass balance of our miniature ‘flying car.’ The wheels added 8.3g and the motion capture markers (we used a Vicon system to track the quads) added 4.2g. So overall the Crazyflie was able to carry 12.5g, or ~44% of its body weight, and still fly pretty well.


Next we measured the power consumption of the Crazyflie and the ‘Flying Car.’ As you can see in the graph below, the additional mass of the wheels reduced total flight time from 5.7 minutes to 5.0 minutes, a 42-second or 12.3% reduction.

Power consumption of the Crazyflie vs. the ‘Flying Car’

 

The table below shows more comparisons between flying without wheels, flying with wheels, and driving. The main takeaways are that driving is much more efficient than flying (in the case of quadcopter flight) and that adding wheels to the Crazyflie does not actually reduce flying performance very much (and in fact increases efficiency when measured using the ‘cost of transport’ metric, which factors in mass). These facts were very important for our planning algorithms, since the tradeoff between energy and speed is the main factor in deciding when to fly (fast but energy-inefficient) versus drive (slow but energy-efficient).

Controlling 8 Crazyflies at once was a challenge. The great work by the USC ACT Lab (J. A. Preiss, W. Hönig, G. S. Sukhatme, and N. Ayanian. “Crazyswarm: A Large Nano-Quadcopter Swarm,” ICRA 2017. https://github.com/USC-ACTLab/crazyswarm) has made our minor effort in this field obsolete, but I will describe our work briefly. We used the crazyflie_ros package, maintained by Wolfgang Hönig from the USC ACT Lab, to interface with the Crazyflies. Unfortunately, we found that a single Crazyradio could communicate with only 2 Crazyflies at a time using our methods, so we had to use 4 Crazyradios, and we had to make a ROS node that switched between the 4 radios rapidly to send commands. It was not ideal at all – moreover, we had to design 8 unique Vicon marker configurations, which was a challenge given the small size of the Crazyflies. In the end, we got our system to work, but the new Crazyswarm framework from the ACT Lab should enable much more impressive demos in the future (as has already been done in their ICRA paper and by the Robust Adaptive Systems Lab at Carnegie Mellon, which they described in their blog post here).

We used two controllers, an air and a ground controller. The ground controller was a simple pure pursuit controller that followed waypoints on ground paths. The differentially steered driving mechanism made ground control blissfully simple. The main challenge we faced was maximizing the rate at which we could send commands to the wheels via the parameter framework. For aerial control, we used simple PID controllers to make the quads follow waypoints. Although the wheel deck shifted the center of mass of the Crazyflie, giving it a tendency to slowly spin in midair, overall the system worked well given its simplicity.

Once we had the design and control of the flying cars figured out, we were able to test our path planning algorithms on them. You can see in the video below that our vehicles were able to faithfully follow the simulation and that they transitioned from flying to driving when necessary.

Link to video

Our work had two goals. One was to show that multi-robot path planning algorithms can be adapted to work for vehicles that can both fly and drive to minimize energy consumption and time. The second goal was to showcase the utility of flying-and-driving vehicles. We were able to achieve these goals in our paper thanks in part to the ease of use and versatility of the Crazyflie 2.0.

A couple of weeks ago Qualisys visited us at the Bitcraze office, they came with the Miqus motion capture system that they installed temporary in the office. This gave us the opportunity to play with their motion capture system and the Crazyflie 2.0 :-).

It was our first hands-on experience with a motion capture system. We were eager to try the algorithms that have been developed for the loco positioning system with the more precise position information offered by the motion capture. The result was above our expectations. You get a bit amazed when it is just sitting there in the air. The normally difficult in-air photo shoot became a breeze since you suddenly have plenty of time to focus and shoot.

After running a couple of simple stabilized flight demos, we endeavored to run the ICRA demo with motion capture instead of our loco positioning system. As the loco positioning deck isn’t needed it was removed and instead the measured position was sent using Crazyradio. Doing so made the demo work pretty much out of the box. The ICRA demo had 2 buttons, one for playing a pre-recorded trajectory and one for recording a path and playing it back as soon as the Crazyflie is dropped. Both modes worked seamlessly without requiring any code change. We tried the path recording and playback functionality and were pretty impressed by the precision:

Link to video

We look forward of meeting and working more with Qualisys. One goal is to provide better information, documentation and tools to get started with Crazyflie in a motion capture system.

We exhibited at the IEEE International Conference on Robotics and Automation in Singapore a couple of weeks ago.

We had a booth where we demoed autonomous flight with the Crazyflie 2.0 and the Loco Positioning system, without any external computer in the loop. The core of the demo was that the Crazyflie had an onboard trajectory sequencer that enabled it to fly autonomously along a path, based on the position from the Loco Positioning system.

We had a pre programmed path that we used most of the time, since it enabled us to start the demo and the leave the Crazyflie without any further manual interference from our side (except changing battery). The other option was to manually record a path for the Crayzflie to retrace by moving it around in the flying space. When we dropped it (detecting zero gravity) the onboard sequencer and controller took over to replay the recorded path. This mode was very useful when showing the accuracy and performance of the system by recording a short sequence of one point and just leaving the Crazyflie to hover. We had mounted a deck with two buttons on the Crazyflie that we used to chose which mode to use.

The code used for the demo is available at github for anyone to play with.

Optical flow

We also showed our brand new Flow deck that we will release soon. It is a deck that is mounted underneath the Crazyflie with a downwards facing optical flow sensor. The sensor is in essence what is used in an optical mouse but with a different lens that enables it to track motion further away. The output from the deck is delta X and Y for the motion of the Crazyflie and can be used by the onboard controller to control the position. We will publish more information in this blog soon.

We had a great time talking to all you interesting, bright and awesome people. Thanks for all feedback, sharing ideas and telling us about your projects!

A couple of weeks ago we played with recording and retracing trajectory directly from the Crazyflie using Loco Positioning System. The result was quite nice and resulted, a first for us, in a fully autonomous Crazyflie, no computer or controller required:

We decided to expand on this experiment for our demo at ICRA. We have modified the retracing code to accepts multiple modes, including running pre-programmed sequence. The plan for the demo is to have Crazyflies that can:

  • Record and retrace a manual trajectory
  • Record and replay in a loop a manual trajectory
  • Play a pre-defined trajectory in a loop
  • Land automatically when the battery level is low

With this we should be able to demonstrate quite well the capabilities of both the Crazyflie and the Loco Positioning system, and since we do not require a computer in the loop it simplifies a lot running the demo. Of course we keep the possibility to connect the Crazyflie with the Crazyflie client and with ROS while the crazyflie is flying.

Having a completly autonomous Crazyflie is also new to us and it brings its share of problems: how to we choose the working mode and how to we stop the flight if something happens (things tends to happen …).

To solve the former we have made a button deck that adds 2 push-button to the Crazyflie. One means “Start autonomous sequence”. The second means “Record trajectory”. If the recorded trajectory is a loop (if the end point is close to the start point) then the loop is played back as soon as the crazyflie is dropped, otherwise Crazyflie retraces the trajectory and stop.

We solved the later problem by making an autonomous emergency stop button that sends a radio watchdog signal. If the signal stops to be sent or if an emergency stop signal is sent (ie. by pressing the button), the Crazyflie will stop all motors and drop. The button is implemented using a Raspberry pi, a Crazyradio and an Arduino to interface the button:

If you are curious about code, we have created a github repos where we push all code we are making for this demo. As usual, this conference is an opportunity for us to hack new functionalities, though not everything can be done in the master branch. Later some things can be merged, others (like the retrace trajectory recorder/player that looks more like a user app.) will need much more though if we want to merge it in the Crazyflie firmware.

The past

The Crazyradio has been designed as a radio dongle to control the original Crazyflie 1. It is based on a Nordic Semiconductor nRF24LU1. It is basically the radio from Crazyflie 1, the nRF24L01, with a microcontroller and a USB device peripheral. Crazyradio has been designed from the beginning to be extended for other used: the code is completely open-source, can be powered with 12V and has an expansion port with possibility of implementing a serial port to communicate with the radio.

When we designed Crazyflie 2.0, we extended Crazyradio to make Crazyradio PA. It is basically the same hardware but we just added a power amplifier to make sure Crazyradio is transmitting with the same power as competing radio in the same 2.4GHz band: Wifi and Bluetooth. Crazyflie 2.0 is not using an nRF24 chip anymore but an nRF51822 which integrates a microcontroller and implements bluetooth low energy as well as nRF24 compatible radio.

 

The present

The Crazyradio (PA) has been used for a couple of hack like our glove controller. Another popular use has been by security researcher to experiment with the security of wireless mouse and keyboard. Indeed the nRF24 serie of chip is extensively used in wireless mouse, keyboard and even quadcopters ;).

We are often very quiet about the Crazyradio since ‘it just works’. It does not means that it is finished or perfect, there is actually a lot that was planned to make the radio link more efficient, to be able to control more Crazyflie per Crazyradio, etc… Some of this work has been done in the context of the Crazyswarm project by Wolfgang from USC but a lot more could be done. One of the main blockers for Crazyradio development is that it is based on a very old microcontroller, an 8051, and that it does not have a safe bootloader or (open) debugging access. It means that each modification is potentially a big commitment in time and since the radio already work quite well it is hard to put much time in it.

The absence of a safe bootloader means that if you flash a firmware that crashes, you will need an SPI programmer to recover the radio in working mode. This makes it quite stressful to work with the radio. However there are a couple of programmers made for this and we recently published a Hackster project about using a Raspberry Pi to recover the bootloader:

The future

We have been looking at making a Crazyradio with an nRF51-series chip, these chips have a Cortex-M0 CPU which means that they are much easier to program using a modern development environment. However none of the chip in the nRF51 series have USB which forced us to prototype the concept with an added microcontroller for USB. This creates a bigger and more populated Crazyradio:

We did not like this design very much since having 2 microcontrollers is always much more haste to program, debug and maintain. Thankfully Nordic semiconductor will release a nRF52 chip with USB support. Adding to that a powerful Cortex-M4 microcontroller, a lot of ram, still Bluetooth low energy and nRF24 radio compatibility but they also added iEEE802.15.4 2.4GHz (ie. the protocol used by Zigbee). This new chip is a very good candidate for the next Crazyradio.

We are very much in the pre-study phase for the next Crazyradio (and mass production for the new nRF52 is planned for Q4 2017 anyway…), so if there is functionality that you would like to see in a future Crazyradio it is time to speak up! Please tell us in the comment section bellow or in the forum ;-).