Category: Loco Positioning

 

We have been writing a couple of times already about the new TDoA2 algorithm for the Loco Positioning System. A TDoA mode has been experimental from the day we released the LPS but we are now proud to announce that TDoA is an official positioning mode for the Loco Positioning System and the Crazyflie.

Practically it means that the Loco Positioning System now has an officially supported mode to locate and fly a swarm of Crazyflie 2.0.

We have worked these last weeks at updating documentation, the “Getting started” tutorial and releasing all the affected firmware and software. One of our goals was to make the new TDoA mode as seamless and as easy as possible to work with, this meant having everything working without having to recompile the Crazyflie or any other part of the system. The Crazyflie is now detecting the LPS mode automatically and it is possible to configure the anchors position and ranging mode remotely from the within Crazyflie client LPS tab.

What we have just released is:

If you have 8 anchors and want to convert your local positioning system to TDoA, this can be done very easily by following the new version of the getting started with loco positioning system guide.

If you want more information about the different positioning modes, we have also updated the system description.

 

This week we have a guest blog post by Ben, enjoy!

I’m Ben Kuperberg and i’m a digital artist, artist-friendly software developer and orchestra conductor. Being a juggler, I’ve decided to focus some of my work on the intersection between juggling and technology, and i’ve since been working more and more with jugglers, my last project being “Sphères Curieuses” from Le Cirque Inachevé, created by Antoine Clée. While the whole project is not focused on drones, a part of it involves synchronized flight of multiple drones and precise human interaction with those drones. Swarm flight is something already out there and some solutions already exist but the context of this project added some challenges to it.

Most work on drone swarms have been done by research group or school. They use high-grade expensive motion-capture system able to track precisely the drones and able to assign their absolute positions. While the quality of the result is undeniable, it’s not fit for stage shows : the setup is taking a lot of time which we can’t always have when the show is on the road. Moreover, the mocap system is too invasive for the stage if you want to be able to “hide” a bit the technology and let the spectator focus on what the artist wants you to see. Not to mention it costs an arm and a leg and Antoine needs both to juggle.

So we had to find other ways to be able to track multiple drones. That’s when we found out the [amazing] team at Bitcraze was working on the TDoA technology, which allows precise-enough tracking of a virtually unlimited amount drones, at reduced cost and with a fast and clean setup.

After some work we managed to have a first rough version of our swarm server made by Maxime Agor that allowed to connect and move multiple drones using the TDoA system, controlled from a Unity application.

While we were able to present a decent demo with this system, we were facing a major problem of reactivity. When working with artists and technology, reactivity is a key component to creativity. Because it can be frustrating and tense to stop each 2 minutes to make changes or fix problems. My first priority was therefore to prepare and design softwares that will allow me to spend most of the “creation time” on the actual creation aspect and not on technical parts. It is also essential that the artist performing in front of the audience can entirely focus on the performance and by fully confident in this technology. The last challenge is that as I focus my work on the creation and not touring, all my work needs to be easily understood and modified by both the artists and the technicians who will take over my work for the tour.

With all of that in mind, I decided to create a software with a high-end user interface called “La Mouche Folle” (« The Crazy Flie » in french) that allows to control multiple drones and have an overview of all the drones, their battery/charging/alert states, auto-connect / auto-reboot features, external control via OSC, and a Unity client to view and actually decide how to move the drones. All my work is open-source, so you can find the software on github.

There only is a Windows release for now but it should compile just fine on OSX and Linux, the software is made with JUCE, depends on OrganicUI and lib-usb. Feel free to contact me if you want more information on the software. Many thanks Wolfgang Hoenig for the support and the great work on the crazyflie cpp library i’m relying on.

So this is the basic setup of our project, but we needed more than that to control the drone. We wanted to be able to control them in the most natural way possible. We quickly decided to go with glove-base solutions, and have been working with Specktr to get our hands – pun intended – on developer versions of the glove. The glove is good but can’t give us absolute position of the hand, so we added HTC Vive trackers with the lighthouse technology and then were able to get both natural hand control and sub-millimeter precision of the tracked hand.

Then it was a matter of connecting everything together : for other projects for Theoriz Studio, I already developed MrTracker (used in the MixedReality project) that acts as a middleware between the Vive trackers and Unity.

I used Chataigne to easily connect and route the Specktr Glove data to Unity as well so we would have maximum flexibility to switch hardware or technology without breaking whole setup if we needed to.

 
A video of the final result
 

 

In the past years, i’ve come to work on a lot of different projects, with different teams, which i like very much, because each project leads to discover new people, new ways of working and new challenges to overcome. I’m having a great time working on this project and especially sharing everything with the guys at Bitcraze and the community, everyone has been so cool and nice. I’ve planned to go at the Bitcraze studios to work for few weeks with them and i’m sure it’ll be a great experience !

The Loco Positioning System (LPS) default working mode is currently Two Way Ranging (TWR), it is a location mode that has the advantage of being pretty easy to implement and gives good positioning performance for most use cases and anchor setups. This was a very good reason for us to start with it. Though, TWR only supports positioning and flying of one or maybe a couple of Crazyflies, while it is not a solution to fly a swarm.

One solution to fly a swarm is an algorithm called Time Difference of Arrival (TDoA). We have had a prototype implementation for a while but we experienced problems with outliers, most of them where due to the fact that we where loosing a lot of packets and thus using bad data.

To solve these issues, TDoA2 makes two changes:

  • Each packet has a sequence number and each timestamps is associated with the sequence number of the packet it has been created from
  • The distances between anchors are calculated and transmitted by the anchors

A slightly simplified explanation follows to outline why this helps (a more detailed explanation of how TDoA works is available in the wiki).

We start by assuming that all timestamps are available to the tag, this is done by transmitting them in the packets from the anchors to the copter.

The end goal is to calculate the difference of time of arrival between two packets from two different anchors. Assuming we have the transmission time of the packets in the same clock, all we need to do is to subtract the time between the two transmissions with the time between the two receptions:

0 – anchor 0, 1 – anchor 1, T – Tag (that is the LPS deck on the Crazyflie)

To do so we need to have the time it took for the packet to travel between the two anchors, this will enable us to calculate the transmit time of P2 in anchor 1, this can be done by calculating the TWR time of flight between the two anchors, this would require the tag to receive 3 packets in sequence:

So now for the part where TDoA2 helps: previously we had to have the 3 packets in sequence in order to calculate a TDoA, if any one of these where missing the measurement would fail or worse, it could give the wrong result. Since we did not have sequence numbers, it was hard to detect packet loss. Now that we have sequence numbers, we can understand when a packet is missing and discard the faulty data. We also do not have to calculate the distance between anchors in the tag anymore, it is calculated by the anchors themselves. This means that we can calculate a TDoA with only two consecutive packets which increases the probability of a successful calculation substantially.

To reduce packet loss even more, we have also added functionality to automatically reduce the transmission power of the NRF radio (the one talking to the Crazyradio dongle) when the LPS deck is detected. It has turned out that the NRF radio transmissions are interfering with UWB radio reception, and since most indoor use cases does not require full output power we figured that this was a good trade-off.

The results we have seen with the new protocol is quite impressive: TDoA is usually very sensitive to the tag being inside the convex hull, so much so that with the first TDoA protocol we had to start the Crazyflie from about 30cm up to be well within the convex hull. This is not required anymore and the position is still good enough to fly even a bit outside of the convex hull. The outliers are also greatly reduced which makes this new TDoA mode behave very close to the current TWR mode, but with the capability to locate as many Crazyflies as you want:

Added to that, we have also implemented anchor position handling in the TDoA2 protocol and this means that it is now as easy to setup a system with TDoA2 as with TWR:

We are now working on finishing the last functionality, like switching between algorithms (TWR and TDoA) and on writing a “getting started guide”. When that is done TDoA will become an official mode for the LPS.

In the mean time, if you are adventurous, you can try it yourself. It has been pushed in the master branch of the Crazyflie firmware and the LPS node firmware. You should re-flash the Crazyflie firmware, both STM32 and nRF51, from master and the anchors from master too.

As I wrote about in a previous blog post, I have been working on an anchor position estimation algorithm in the Crazyflie Client. The algoritm uses ranging data from the Loco Positioning system to estimate where the anchors are located, and thus remove the need to measure their positions in the room. I have finally reached a point where I think it is good enough to let it out from the lab and it has been pushed to the client repository.

A button has been added to the Loco Positioning tab that opens a wizard. In the wizard the user is asked to place the Crazyflie in certain positions to record ranges and define the coordinate system. If all goes well, the estimated anchor positions are transfered to the anchor position fields in the Loco Positioning tab. If the user is happy with the result the next step is to write the positions to the anchors and start flying!

Now to the disclaimer: the results may not always be perfect – surprise! We have not tested the algorithm a lot but it seems to give decent results, at least it can be useful as a base for manual corrections and sanity checks. Some of the estimated positions are pretty good, while others might be a meter or so off. The conclusion is that you should not trust it blindly, check that the estimated positions seem reasonable before flying.

Currently the system only supports Two way ranging, but extending it to TDoA should not be too complicated. There are probably many possible improvements that can be done, and we hope that everyone that finds this interesting and have ideas of how to do it will give it a go. After all, it is open source and we would love to see contributions refining the functionality, now that there is a base to build from.

Any feed back is welcome, let us know if it works or not in your setup!

One of the pain points when setting up the Loco Positioning system is to measure the anchor positions and enter them into the system. I wanted to see if I could automate this task and let the system calculate the positions, and if so understand what kind of precision to expect. I have spent a few Fun Fridays playing with this problem and this is what I have found so far.

The problem can be broken down into two parts:
1. How to calculate the anchor positions. What data is required?
2. How to define the coordinate system. To make it useful the user must to be able to define the coordinate system in a simple way.

Anchor and ruler

How to calculate the anchor positions

The general idea of how to calculate the anchor positions is to set up a system of equations describing the distances between the anchors and/or the Crayzflie and solve for the anchor positions. The equations will be non linear and the (possibly naive) plan is to use the Gauss Newton method to solve the system.

To understand how to calculate the anchor positions we must first take a look at the data that is available. The Loco Positioning system can be run in two different modes: Two Way ranging (default mode) and TDoA.

Two way ranging

In the Two Way ranging mode we measure the distance between each anchor and the Crazyflie and to get enough data we must record ranging data for multiple positions. The anchor positions are unknown, and for each new Crazyflie position we add yet a new unknown position, on the other hand we measure the ranges to the anchors so these are knowns. 

The equations used are simply to calculate the distance between the assumed position of each anchor and the Crazyflie and then subtracting it from the measured distance.

TDoA

In TDoA we measure the Time Difference of Arrival, that is the difference in distance to two anchors from the Crazyflie’s position. It is probably possible to use this information, but I was looking for a different solution here. In our new TDoA implementation that we have been playing with a bit, we get the distance between all anchors (calculated in the anchors) as a side effect. 

In this case the Crazflie is not really needed and the equations describe the distance between assumed anchor positions versus measured distances.

How to define the coordinate system

To get a useable positioning system, the coordinate system must be well defined and oriented in a practical direction. For example when writing a script you probably want (0, 0, 0) to be at some specific spot, the X-axis pointing in a certain direction, the Z-axis to point up and so on. My initial idea was to use the anchors to define the coordinate system, use anchor 0 as (0, 0, 0), let the X-axis pass through anchor 1 and so on. Just by looking at our flight lab I realised that this would be too limiting and decided that the coordinate system should be completely disconnected from the anchor positions, but still easy to define. I also realised that a really good way to tell the system about the desired coordinate system would be to move the Crazyflie around in space to show what you want. The solution is to place the Crazyflie at certain positions and click a button to record data at these positions. The steps I have chosen are:

  1. Place the Crazyflie at (0, 0, 0)
  2. Place the Crazyflie on the X-axis, X > 0
  3. Place the Crazyflie in the XY-plane, Y > 0
  4. Move the Crazyflie around in the space with continuous recording of data

In this scheme the XY-plane is typically the floor.

Results

I have written basic implementations for both the Two Way ranging and TDoA modes and they seem to work reasonably well in simulations. I have also tested the Two Way ranging algorithm in our flight lab with mixed results. The solution converged in most cases but not always. When converging the estimated anchor positions ended up in the right region but some were off by up to a meter. Finally I did run the algorithm and fed the result into the system and managed to fly using the estimated positions which I find encouraging.

I will continue to work on this as a Friday Fun project and maybe it will make its way into the client code base at some point in the future? There are probably better ways to estimate the anchor positions and more clever algorithms, feel free to share them in the comments.

 

 

We have had an implementation of a Time Difference of Arrival algorithm (TDoA) in the Crazyflie 2.o and the Loco Positioning System for quite a long time. The coolest feature of the TDoA algorithm is that it can be used to position virtually unlimited number of Crazyflies concurrently as opposed to the standard Two Way Ranging algorithm that is limited to one (or very few Crazyflies). The original implementation is working pretty well but contains some flaws that we are not completely happy with, hence we have not released it officially and are still calling it experimental. Since support for tracking multiple objects is a requirement for flying swarms and we like swarms, we have started to iron out the problems. 

A small swarm using the old TDoA algorithm, from February 2017.
 

In the current implementation each anchor transmits the time of the transmission and the times of the latest reception of transmissions from all other anchors about every 16 ms. From this information it is possible to calculate the difference in time of flight for the radio waves from two anchors to the Crazyflie. When we know the difference in time of arrival, by multiplying with the speed of light we get the difference in distance and can calculate the position of the Crazyflie. This all sounds fine and dandy but the set up has some problems, the biggest one being error handling. If one or more packets are lost, either from anchor to Crazyflie or anchor to anchor, there is no deterministic way to detect it in some cases. The current algorithm relies on sanity checking the calculated result and discarding data that looks suspicious, which is usually easy as the distances quickly gets unrealistic (several thousands of meters!). We suspect that some erroneous values slip through the check though and we would like to be able to really understand when data is valid or not.

TDoA 2.0

What we are working on now is to add a sequence numbering scheme to enable the receiving party to understand when a packet has been lost. With this information it will be possible to discard bad data as well as use the available information better. While we are re-writing the code we are also moving a part of the algorithm from the Crazyflie to the anchors, after all there is a CPU in the anchors that is not fully utilized. The idea is to let each anchor continuously calculate the distance to all other anchors and add this information to the messages it transmits, which will reduce the work in the Crazyflie.

This is work in progress and we are not completely sure where we will end up, but we are aiming at making the TDoA mode part of the official release at some point.

Sensor fusion

Related to the Loco Positioning system is our line of other positioning sensors; the Z-ranger deck and the Flow deck. The Flow deck has really good precision at low altitudes but can not provide absolute positioning while the Loco Positioning system does not have the same precision but absolute position capabilities. So what if we fuse the information from the Flow deck with the Loco Positioning system? We have tried it out and it works pretty well, we can get the best of two worlds! The Z-ranger can also be used in the same way to improve the Z component of the estimated position when flying bellow ~1m.

Even though it works using multiple positioning sensors at the same time, there is room for improvements and some tweaking will be required to make it rock solid.

 

 

It is summer again in Sweden and things are now starting to slow down and people are going to vacation. The last couple of years we have used the summer to look back and clean-up the technical dept accumulated during the year: when trying to get things done we have to prioritize which means that some things have to be left on the side (at least until we invent a way to add more hours to each day). This year is no exception, the last couple of weeks we have been working very hard to get the Flow products out and now the production is hopefully on rails so the cleanup can begin.

There is a lot of things we could do but here is a sneak peek of what we are currently looking at:

  • Crazyflie Client gamepad handling and configuration: The current input device handling is complicated and the architecture is hard to work with. There is a lot that can be done both in the front-end and the back end to make it easier to use and to work with.
  • Loco positioning system support for multiple Crazyflie, we have two mode implemented for that, TWR-TDMA and TDoA, both are very experimental and need some more work.
  • Cleanup of the webpage, information and documentation: we already have done a lot of work to make better documentation but there is always margin for improvement.
  • Cleaning up and improving the Crazyradio firmware: the Crazyradio starts to show its limits when flying swarms of Crazyflie. There is some improvement that could be done in the Firmware to make it more efficient. The first step is clean up the current implementation.

If you have any ideas of areas you feel we should focus on, even better if you want to help with some things and fix it together with us, just tell us in the comment.

On a side note, the manufacturing of the Flow products is still on progress and it should soon be on the Bitcraze shop and the Seeedstudio bazaar, stay tuned.

We exhibited at the IEEE International Conference on Robotics and Automation in Singapore a couple of weeks ago.

We had a booth where we demoed autonomous flight with the Crazyflie 2.0 and the Loco Positioning system, without any external computer in the loop. The core of the demo was that the Crazyflie had an onboard trajectory sequencer that enabled it to fly autonomously along a path, based on the position from the Loco Positioning system.

We had a pre programmed path that we used most of the time, since it enabled us to start the demo and the leave the Crazyflie without any further manual interference from our side (except changing battery). The other option was to manually record a path for the Crayzflie to retrace by moving it around in the flying space. When we dropped it (detecting zero gravity) the onboard sequencer and controller took over to replay the recorded path. This mode was very useful when showing the accuracy and performance of the system by recording a short sequence of one point and just leaving the Crazyflie to hover. We had mounted a deck with two buttons on the Crazyflie that we used to chose which mode to use.

The code used for the demo is available at github for anyone to play with.

Optical flow

We also showed our brand new Flow deck that we will release soon. It is a deck that is mounted underneath the Crazyflie with a downwards facing optical flow sensor. The sensor is in essence what is used in an optical mouse but with a different lens that enables it to track motion further away. The output from the deck is delta X and Y for the motion of the Crazyflie and can be used by the onboard controller to control the position. We will publish more information in this blog soon.

We had a great time talking to all you interesting, bright and awesome people. Thanks for all feedback, sharing ideas and telling us about your projects!

We have been working with our indoor positioning system for over a year now and during that time the interest for the Loco positioning system have continuously been growing. Universities all over the world are already using our system and the attention we have received have been very encouraging. But the system has been a bit hard to set up and lacking in documentation. So we decided that this had to be fixed before we could leave early access.

So now we’re happy to announce that the Loco positioning system has reached a state where it’s out of “Early access” and we’re excited to see even more customers starting to use the system. We would like to thank everybody in the community that has been contributing to the project, this is what makes open source great!

Below are some of the steps we’ve taken to make it easier to use.

 

Getting started guide

We have created a brand new “Getting started with Loco positioning” tutorial that will replace the old “Getting started” video that we made. The tutorial contains a step-by-step guide for flying one Crazyflie 2.0 autonomously using two way ranging. The more advanced features such as TDoA is still considered as experimental and won’t be covered in this tutorial. We are very happy with the outcome of this tutorial but please help us out and try the tutorial and give us feedback.  

LPS tool

In order to easily configure and upgrade the firmware for the Loco positioning nodes we’ve created the LPS tool. With it you can set the address of the node, the mode and also upgrade the firmware easily. 

 

 

LPS tab

Up until lately there hasn’t been any support for the Loco positioning system in our Crazyflie Python client, instead we’ve been using ROS. It’s a great tool and very powerful tool, but ROS can be a bit complicated with specific requirements on the environment in which it runs. So we wanted to be able to perform basic use-cases directly in our client, like setting up the system, debugging it and flying with it. In order to achieve this we created the Loco positioning tab and a new flight mode called Position hold.

The Loco positioning tab is used to set-up the system and view it’s status. You can configure the position of the anchors, easily see which anchors you’re close to and if all the anchors are responding as they should. It’s also possible to see the layout of the system and the Crazyflies position in it.

For flying the Crazyflie we’ve added the Position hold mode. When enabled this will translate gamepad input into velocity control set-points that are followed using the positioning.

 

Wiki documentation

We’ve added technical documentation for the Loco positioning system to the wiki and we’ve also created product pages with schematics and details on the hardware for the Loco positioning node and deck. In the documentation you can find useful things like protocol documentation and details on our reference set-up that we use in our lab. Sine the system is still continuously evolving this is a working document that will continue to be improved over time and we hope to push the boundaries further for swarm flying and robotics in general.

ICRA

Finally a reminder, if your are attending 2017 IEEE International Conference on Robotics and Automation in Singapore you can meet us in booth C08 from 30/5 – 01/6. We will show autonomous flight with the Crazyflie 2.0 enabled by the the Loco Positioning System. Come by have a chat with us and see the Loco positioning in action!

You will also be able to attend the presentation of the paper “Crazyswarm: A Large Nano-Quadcopter Swarm” by researchers from USC during Wednesday 31/5 at 11:15.

Releases

To support the features mentioned above we’ve released version 2017.05 of the Crazyflie Python client and Crazyflie firmware.

 

 

A couple of weeks ago we played with recording and retracing trajectory directly from the Crazyflie using Loco Positioning System. The result was quite nice and resulted, a first for us, in a fully autonomous Crazyflie, no computer or controller required:

We decided to expand on this experiment for our demo at ICRA. We have modified the retracing code to accepts multiple modes, including running pre-programmed sequence. The plan for the demo is to have Crazyflies that can:

  • Record and retrace a manual trajectory
  • Record and replay in a loop a manual trajectory
  • Play a pre-defined trajectory in a loop
  • Land automatically when the battery level is low

With this we should be able to demonstrate quite well the capabilities of both the Crazyflie and the Loco Positioning system, and since we do not require a computer in the loop it simplifies a lot running the demo. Of course we keep the possibility to connect the Crazyflie with the Crazyflie client and with ROS while the crazyflie is flying.

Having a completly autonomous Crazyflie is also new to us and it brings its share of problems: how to we choose the working mode and how to we stop the flight if something happens (things tends to happen …).

To solve the former we have made a button deck that adds 2 push-button to the Crazyflie. One means “Start autonomous sequence”. The second means “Record trajectory”. If the recorded trajectory is a loop (if the end point is close to the start point) then the loop is played back as soon as the crazyflie is dropped, otherwise Crazyflie retraces the trajectory and stop.

We solved the later problem by making an autonomous emergency stop button that sends a radio watchdog signal. If the signal stops to be sent or if an emergency stop signal is sent (ie. by pressing the button), the Crazyflie will stop all motors and drop. The button is implemented using a Raspberry pi, a Crazyradio and an Arduino to interface the button:

If you are curious about code, we have created a github repos where we push all code we are making for this demo. As usual, this conference is an opportunity for us to hack new functionalities, though not everything can be done in the master branch. Later some things can be merged, others (like the retrace trajectory recorder/player that looks more like a user app.) will need much more though if we want to merge it in the Crazyflie firmware.