Blog

It is summer again in Sweden and things are now starting to slow down and people are going to vacation. The last couple of years we have used the summer to look back and clean-up the technical dept accumulated during the year: when trying to get things done we have to prioritize which means that some things have to be left on the side (at least until we invent a way to add more hours to each day). This year is no exception, the last couple of weeks we have been working very hard to get the Flow products out and now the production is hopefully on rails so the cleanup can begin.

There is a lot of things we could do but here is a sneak peek of what we are currently looking at:

  • Crazyflie Client gamepad handling and configuration: The current input device handling is complicated and the architecture is hard to work with. There is a lot that can be done both in the front-end and the back end to make it easier to use and to work with.
  • Loco positioning system support for multiple Crazyflie, we have two mode implemented for that, TWR-TDMA and TDoA, both are very experimental and need some more work.
  • Cleanup of the webpage, information and documentation: we already have done a lot of work to make better documentation but there is always margin for improvement.
  • Cleaning up and improving the Crazyradio firmware: the Crazyradio starts to show its limits when flying swarms of Crazyflie. There is some improvement that could be done in the Firmware to make it more efficient. The first step is clean up the current implementation.

If you have any ideas of areas you feel we should focus on, even better if you want to help with some things and fix it together with us, just tell us in the comment.

On a side note, the manufacturing of the Flow products is still on progress and it should soon be on the Bitcraze shop and the Seeedstudio bazaar, stay tuned.

This week’s Monday post is a guest post written by members of the Computer Science and Artificial Intelligence Lab at MIT.

One of the focuses of the Distributed Robotics Lab, which is run by Daniela Rus and is part of the Computer Science and Artificial Intelligence Lab at MIT, is to study the coordination of multiple robots. Our lab has tested a diverse array of robots, from jumping cubes to Kuka youBots to quadcopters. In one of our recent projects, presented at ICRA 2017, Multi-robot Path Planning for a Swarm of Robots that Can Both Fly and Drive, we tested collision-free path planning for flying-and-driving robots in a small town.

Robots that can both fly and drive – in particular wheeled drones – are actually somewhat of a rarity in robotics research. Although there are several interesting examples in the literature, most of them involve creative ways of repurposing the wings or propellers of a flying robot to get it to move on the ground. Since we wanted to test multi-robot algorithms, we needed a robot that would be robust, safe, and easy to control – not necessarily advanced or clever. We decided to put an independent driving mechanism on the bottom of a quadcopter, and it turns out that the Crazyflie 2.0 was the perfect platform for us. The Crazyflie is easily obtainable, safe, and (we can certify ourselves) very robust. Moreover, since it is open-source and fully programmable, we were able to easily modify the Crazyflie to fit our needs. Our final design with the wheel deck is shown below.

A photo of the Crazyflie 2.0 with the wheel deck.

A model of the Crazyflie 2.0 with the wheel deck from the bottom

The wheel deck consists of a PCB with a motor driver; two small motors mounted in a carbon fiber tube epoxied onto the PCB; and a passive ball caster in the back. We were able to interface our PCB with the pins on the Crazyflie so that we could use the Crazyflie to control the motors (the code is available at https://github.com/braraki/crazyflie-firmware). We added new parameters to the Crazyflie to control wheel speed, which, in retrospect, was not a good decision, since we found that it was difficult to update the parameters at a high enough rate to control the wheels well. We should have used the Crazyflie RealTime Protocol (CRTP) to send custom data packages to the Crazyflie, but that will have to be a project for another day.
The table below shows the mass balance of our miniature ‘flying car.’ The wheels added 8.3g and the motion capture markers (we used a Vicon system to track the quads) added 4.2g. So overall the Crazyflie was able to carry 12.5g, or ~44% of its body weight, and still fly pretty well.


Next we measured the power consumption of the Crazyflie and the ‘Flying Car.’ As you can see in the graph below, the additional mass of the wheels reduced total flight time from 5.7 minutes to 5.0 minutes, a 42-second or 12.3% reduction.

Power consumption of the Crazyflie vs. the ‘Flying Car’

 

The table below shows more comparisons between flying without wheels, flying with wheels, and driving. The main takeaways are that driving is much more efficient than flying (in the case of quadcopter flight) and that adding wheels to the Crazyflie does not actually reduce flying performance very much (and in fact increases efficiency when measured using the ‘cost of transport’ metric, which factors in mass). These facts were very important for our planning algorithms, since the tradeoff between energy and speed is the main factor in deciding when to fly (fast but energy-inefficient) versus drive (slow but energy-efficient).

Controlling 8 Crazyflies at once was a challenge. The great work by the USC ACT Lab (J. A. Preiss, W. Hönig, G. S. Sukhatme, and N. Ayanian. “Crazyswarm: A Large Nano-Quadcopter Swarm,” ICRA 2017. https://github.com/USC-ACTLab/crazyswarm) has made our minor effort in this field obsolete, but I will describe our work briefly. We used the crazyflie_ros package, maintained by Wolfgang Hönig from the USC ACT Lab, to interface with the Crazyflies. Unfortunately, we found that a single Crazyradio could communicate with only 2 Crazyflies at a time using our methods, so we had to use 4 Crazyradios, and we had to make a ROS node that switched between the 4 radios rapidly to send commands. It was not ideal at all – moreover, we had to design 8 unique Vicon marker configurations, which was a challenge given the small size of the Crazyflies. In the end, we got our system to work, but the new Crazyswarm framework from the ACT Lab should enable much more impressive demos in the future (as has already been done in their ICRA paper and by the Robust Adaptive Systems Lab at Carnegie Mellon, which they described in their blog post here).

We used two controllers, an air and a ground controller. The ground controller was a simple pure pursuit controller that followed waypoints on ground paths. The differentially steered driving mechanism made ground control blissfully simple. The main challenge we faced was maximizing the rate at which we could send commands to the wheels via the parameter framework. For aerial control, we used simple PID controllers to make the quads follow waypoints. Although the wheel deck shifted the center of mass of the Crazyflie, giving it a tendency to slowly spin in midair, overall the system worked well given its simplicity.

Once we had the design and control of the flying cars figured out, we were able to test our path planning algorithms on them. You can see in the video below that our vehicles were able to faithfully follow the simulation and that they transitioned from flying to driving when necessary.

Link to video

Our work had two goals. One was to show that multi-robot path planning algorithms can be adapted to work for vehicles that can both fly and drive to minimize energy consumption and time. The second goal was to showcase the utility of flying-and-driving vehicles. We were able to achieve these goals in our paper thanks in part to the ease of use and versatility of the Crazyflie 2.0.

A couple of weeks ago Qualisys visited us at the Bitcraze office, they came with the Miqus motion capture system that they installed temporary in the office. This gave us the opportunity to play with their motion capture system and the Crazyflie 2.0 :-).

It was our first hands-on experience with a motion capture system. We were eager to try the algorithms that have been developed for the loco positioning system with the more precise position information offered by the motion capture. The result was above our expectations. You get a bit amazed when it is just sitting there in the air. The normally difficult in-air photo shoot became a breeze since you suddenly have plenty of time to focus and shoot.

After running a couple of simple stabilized flight demos, we endeavored to run the ICRA demo with motion capture instead of our loco positioning system. As the loco positioning deck isn’t needed it was removed and instead the measured position was sent using Crazyradio. Doing so made the demo work pretty much out of the box. The ICRA demo had 2 buttons, one for playing a pre-recorded trajectory and one for recording a path and playing it back as soon as the Crazyflie is dropped. Both modes worked seamlessly without requiring any code change. We tried the path recording and playback functionality and were pretty impressed by the precision:

Link to video

We look forward of meeting and working more with Qualisys. One goal is to provide better information, documentation and tools to get started with Crazyflie in a motion capture system.

This week’s Monday post is a guest post written by members of the Robust Adaptive Systems Lab at Carnegie Mellon University.

As researchers in the Robust Adaptive Systems Lab (Director: Prof. Nathan Michael) at Carnegie Mellon University http://rasl.ri.cmu.edu, we’re interested in developing algorithms for control, planning, and coordination to enable robots to adapt to their environments, and we use the Crazyflie platforms to help us test our theory on real world systems.

We employ Crazyflie quadrotors to evaluate several research objectives in our lab. One research thrust looks at enabling a platform to learn new controllers as it flies, allowing the quadrotor to automatically switch between controllers to handle specific flight conditions. Another objective looks at adapting flight plans for multiple robots in real time, based on what a user tells the robots to do. We use the Crazyflie platform to evaluate our algorithms because the hardware is robust and the user community has helped make firmware available on which we can base our own systems (J. A. Preiss, W. Hönig, G. S. Sukhatme, and N. Ayanian. “Crazyswarm: A Large Nano-Quadcopter Swarm,” ICRA 2017. https://github.com/USC-ACTLab/crazyswarm). At the same time, the power and memory constraints of the Crazyflie force us to pursue efficient algorithms that can handle real world considerations such as limited battery life and computing restrictions.

We’re bringing these research ideas together to make an adaptive multi-robot system that a person can direct online and operate over long periods of time–longer than the lifespan of a single battery. In this blog post, we wanted to share a bit of information about some of the research we’re working on that uses the Crazyflie platforms. We look at single-robot controllers, multi-robot planning systems, and hardware designs to move us towards a persistent, adaptive swarm of Crazyflies.

Control Approaches

We have looked at implementing advanced control strategies onboard the Crazyflie for improved safety and flight performance. This includes an extremely efficient formulation of explicit Model Predictive Control (MPC) developed specifically for systems like the Crazyflie that have severe computational constraints due to size, weight, and power limitations. MPC optimizes the control commands based on current and predicted motion, allowing us to enforce speed limits and avoid motor saturation. The controller is run in a separate task on the STM32F405 MCU, allowing it to maintain a 100 Hz position control rate without blocking other components, such as the state estimator and attitude controller.

In the image above, the vehicle uses MPC control to rapidly and accurately move between two destinations. [V. R. Desaraju and N. Michael, “Fast Nonlinear Model Predictive Control via Partial Enumeration,” ICRA 2016.]

Multi-robot Systems

Another element of our research emphasizes adapting plans for multi-robot systems online, in order to translate a human operator’s intent into dynamically feasible and safe motion plans. We are using a swarm of Crazyflies for hardware validation of the online motion planning algorithms.

Our experimental testbed consists of a Vicon motion capture system that provides the state information (position and orientation) of the robots. The position coordinates and the heading angle are broadcast to the individual Crazyflies using the radios at a rate of 100 Hz. A centralized planner, running on a separate thread, parses user instructions into dynamically feasible and safe motion plans for each individual robot. The user instructions consists of high level information such as robot ids, behavior type (go to a target destination, rotate in a circle), and formation shapes (line, polygon, 3D). Once the feasible and safe motion plans are computed, the desired position setpoints are broadcasted to the Crazyflies at a rate of 30 Hz.

These images show a formation of 30 Crazyflies in a 3D shape and five groups of three Crazyflies, flying around a motion capture arena in response to a user’s direction. The multi-robot planner computes these flight plans online in response to the user’s commands, without any prior planning. [Ellen A. Cappo, Arjav Desai and Nathan Michael. “Robust Coordinated Aerial Deployments for Theatrical Applications Given Online User Interaction via Behavior Composition,” DARS 2016.]

Hardware Considerations 

As we move towards operating for long periods of time, we needed to incorporate the Qi inductive charger to enable the robots to recharge. We also want to retain the LEDs, to keep an aesthetic visual element and act as status indicators. There were two concerns with using both the standard LED and Qi decks: they both mount to the bottom of the Crazyflie, and both decks add weight. We created a combined deck based on the schematics from the Bitcraze wiki, and the final deck is shown below and weighs in at 6-7g.

Even with the combined Qi + LED deck, we observe shorter flight times and motor saturation with the Crazyflie flights, so we increased the battery capacity and flying with different motor and propeller combinations. Pictured below is a Crazyflie with 7×20 motors using 55mm propellers and a 500mAh battery, and carrying a foam mount for the motion capture markers. No frame modifications are required as the motors fit in the mounts and the battery is chosen to fit within the header pins. These changes resulted in an increase in flight time, greater control margin and aggressiveness, and larger payload capacity.

Toward Persistent Operation

The capabilities we mentioned above allow us to work towards long duration operation. We are currently integrating our adaptive control, online planning, and efficient hardware design to enable persistent operation of a swarm of Crazyflies. To achieve this goal, we are developing scheduling algorithms that update flight plans based on battery capacities across a Crazyflie fleet. The flight planner calculates trajectories that allow robots with full batteries to exchange places with robots with expired batteries, landing Crazyflies that need to recharge onto charging stations. In the first image below, 10 robots rotate in a circle formation. The time-lapsed image shows three Crazyflies with full batteries (with blue LEDs) exchange places with three Crazyflies with depleted batteries (with red LEDs). The second photo shows a close-up of our Crazyflie charging station.

In closing, we show a few video clips of Crazyflie flight illustrating the research we discuss here. We show a group of fifteen Crazyflies following online-issued commands to split and merge between formations; a formation of 30 Crazyflies; and an example of our exchange planner, substituting robots into a multi-robot formation and returning robots to ground station locations.

Contributors: Ellen Cappo, Arjav Desai, Matt Collins, Derek Mitchell, and Vishnu Desaraju, students and researchers at the Robust Adaptive Systems Lab (Director: Prof. Nathan Michael), Carnegie Mellon University.

For a while now we have been selling the BigQuad deck which makes it possible to transform the CF2 to control a bigger sized drone. It does so by becoming the quadrotor control board, controlling external brushless motor controllers, which allows to scale up the size. This can be very convenient when trying out/developing new things as it first can be tested on the small CF2 and later scaled up by attaching it to a bigger quad. However for a more permanent setup it is a bit bulky, so we have been playing around a bit and designed something in the middle. The result is a stand alone control board targeting quads around 0.1 – 0.5kg.

We call it the CF-RZR as it is inspired by the smaller sized racers with some fundamental differences. It is designed with a higher level of autonomous functionality in mind and being easy to repair while still being fully compatible with the CF2 firmware and decks. Listing the biggest features of the current prototype:

  • Fully compatible with the CF2 firmware, expansion decks as well as radio.
  • Connectors to attach motor controllers (still possible to solder though) so it is easy to build and repair.
  • Power distributions built into controller board. (Max ~8A per motor controller though)
  • Motor controllers can be switched of by the system so the system can go into deep sleep and consume around 50uA.
  • Voltage input 1S-4S (3V to 17V)
  • Standard mounting (M3 mounting holes placed 30.5mm square)
  • External antenna for increased range

To summarize, the strength of the CF2 but in a little bit bigger package :-). Last week we got a chance to test fly it for the first time. We used a off the shelf racer frame, ESC and motors. At first it did not fly that well at all but after some PID tuning it became pretty stable and we had a lot of fun :-).

We would love your feedback, good/bad idea, what do you like/dislike etc!

Link to video

We exhibited at the IEEE International Conference on Robotics and Automation in Singapore a couple of weeks ago.

We had a booth where we demoed autonomous flight with the Crazyflie 2.0 and the Loco Positioning system, without any external computer in the loop. The core of the demo was that the Crazyflie had an onboard trajectory sequencer that enabled it to fly autonomously along a path, based on the position from the Loco Positioning system.

We had a pre programmed path that we used most of the time, since it enabled us to start the demo and the leave the Crazyflie without any further manual interference from our side (except changing battery). The other option was to manually record a path for the Crayzflie to retrace by moving it around in the flying space. When we dropped it (detecting zero gravity) the onboard sequencer and controller took over to replay the recorded path. This mode was very useful when showing the accuracy and performance of the system by recording a short sequence of one point and just leaving the Crazyflie to hover. We had mounted a deck with two buttons on the Crazyflie that we used to chose which mode to use.

The code used for the demo is available at github for anyone to play with.

Optical flow

We also showed our brand new Flow deck that we will release soon. It is a deck that is mounted underneath the Crazyflie with a downwards facing optical flow sensor. The sensor is in essence what is used in an optical mouse but with a different lens that enables it to track motion further away. The output from the deck is delta X and Y for the motion of the Crazyflie and can be used by the onboard controller to control the position. We will publish more information in this blog soon.

We had a great time talking to all you interesting, bright and awesome people. Thanks for all feedback, sharing ideas and telling us about your projects!

Last week while Kristoffer and Arnaud was in Singapore showcasing our Loco positioning system and Crazyflie at ICRA we initiated moving to a new bigger room here at The Ground. The Ground is a great business collective hosting offices for companies such as Mapilary, Monix, Castle to name a few. It is a very inspirational place to be with a great sharing climate which helps a lot for startups and early phase companies and we are happy to be part of it.

One of the nice things with the new office is that we now can have a dedicated flying area close buy and don’t have to run up and down the basement all the time. Hopefully this will help us speed up development and testing a bit, benefiting not just us, but the whole community :-).

So next time you get an email or forum post written from us you know where it is likely to have been typed.

We have been working with our indoor positioning system for over a year now and during that time the interest for the Loco positioning system have continuously been growing. Universities all over the world are already using our system and the attention we have received have been very encouraging. But the system has been a bit hard to set up and lacking in documentation. So we decided that this had to be fixed before we could leave early access.

So now we’re happy to announce that the Loco positioning system has reached a state where it’s out of “Early access” and we’re excited to see even more customers starting to use the system. We would like to thank everybody in the community that has been contributing to the project, this is what makes open source great!

Below are some of the steps we’ve taken to make it easier to use.

 

Getting started guide

We have created a brand new “Getting started with Loco positioning” tutorial that will replace the old “Getting started” video that we made. The tutorial contains a step-by-step guide for flying one Crazyflie 2.0 autonomously using two way ranging. The more advanced features such as TDoA is still considered as experimental and won’t be covered in this tutorial. We are very happy with the outcome of this tutorial but please help us out and try the tutorial and give us feedback.  

LPS tool

In order to easily configure and upgrade the firmware for the Loco positioning nodes we’ve created the LPS tool. With it you can set the address of the node, the mode and also upgrade the firmware easily. 

 

 

LPS tab

Up until lately there hasn’t been any support for the Loco positioning system in our Crazyflie Python client, instead we’ve been using ROS. It’s a great tool and very powerful tool, but ROS can be a bit complicated with specific requirements on the environment in which it runs. So we wanted to be able to perform basic use-cases directly in our client, like setting up the system, debugging it and flying with it. In order to achieve this we created the Loco positioning tab and a new flight mode called Position hold.

The Loco positioning tab is used to set-up the system and view it’s status. You can configure the position of the anchors, easily see which anchors you’re close to and if all the anchors are responding as they should. It’s also possible to see the layout of the system and the Crazyflies position in it.

For flying the Crazyflie we’ve added the Position hold mode. When enabled this will translate gamepad input into velocity control set-points that are followed using the positioning.

 

Wiki documentation

We’ve added technical documentation for the Loco positioning system to the wiki and we’ve also created product pages with schematics and details on the hardware for the Loco positioning node and deck. In the documentation you can find useful things like protocol documentation and details on our reference set-up that we use in our lab. Sine the system is still continuously evolving this is a working document that will continue to be improved over time and we hope to push the boundaries further for swarm flying and robotics in general.

ICRA

Finally a reminder, if your are attending 2017 IEEE International Conference on Robotics and Automation in Singapore you can meet us in booth C08 from 30/5 – 01/6. We will show autonomous flight with the Crazyflie 2.0 enabled by the the Loco Positioning System. Come by have a chat with us and see the Loco positioning in action!

You will also be able to attend the presentation of the paper “Crazyswarm: A Large Nano-Quadcopter Swarm” by researchers from USC during Wednesday 31/5 at 11:15.

Releases

To support the features mentioned above we’ve released version 2017.05 of the Crazyflie Python client and Crazyflie firmware.

 

 

A couple of weeks ago we played with recording and retracing trajectory directly from the Crazyflie using Loco Positioning System. The result was quite nice and resulted, a first for us, in a fully autonomous Crazyflie, no computer or controller required:

We decided to expand on this experiment for our demo at ICRA. We have modified the retracing code to accepts multiple modes, including running pre-programmed sequence. The plan for the demo is to have Crazyflies that can:

  • Record and retrace a manual trajectory
  • Record and replay in a loop a manual trajectory
  • Play a pre-defined trajectory in a loop
  • Land automatically when the battery level is low

With this we should be able to demonstrate quite well the capabilities of both the Crazyflie and the Loco Positioning system, and since we do not require a computer in the loop it simplifies a lot running the demo. Of course we keep the possibility to connect the Crazyflie with the Crazyflie client and with ROS while the crazyflie is flying.

Having a completly autonomous Crazyflie is also new to us and it brings its share of problems: how to we choose the working mode and how to we stop the flight if something happens (things tends to happen …).

To solve the former we have made a button deck that adds 2 push-button to the Crazyflie. One means “Start autonomous sequence”. The second means “Record trajectory”. If the recorded trajectory is a loop (if the end point is close to the start point) then the loop is played back as soon as the crazyflie is dropped, otherwise Crazyflie retraces the trajectory and stop.

We solved the later problem by making an autonomous emergency stop button that sends a radio watchdog signal. If the signal stops to be sent or if an emergency stop signal is sent (ie. by pressing the button), the Crazyflie will stop all motors and drop. The button is implemented using a Raspberry pi, a Crazyradio and an Arduino to interface the button:

If you are curious about code, we have created a github repos where we push all code we are making for this demo. As usual, this conference is an opportunity for us to hack new functionalities, though not everything can be done in the master branch. Later some things can be merged, others (like the retrace trajectory recorder/player that looks more like a user app.) will need much more though if we want to merge it in the Crazyflie firmware.

We are going to the IEEE International Conference on Robotics and Automation in Singapore. The exhibition is open Tuesday May 30 to Thursday June 1 and we will have a booth, number C08, where we will show demos and discuss our work, positioning technologies and quadcopters.

We have not finalized our demos yet but they will include autonomous flight with the Crazyflie and the Loco Positioning system. We also hope to show our brand new optical flow expansion deck that will enable positioning and autonomous flight when on a tight budget. We also plan to show integration with external computers running ROS or our own python library. If we are lucky there might even be a small swarm, even though the space is very limited.

We love to talk to people that are using our products or just interested in our technology, if you are at the conference please drop by and say hi and tell us what you are working on. We will arrive in Singapore on Saturday morning May 27, and if you want to hook up and say hi and have a coffee during the weekend, drop us an email.

See you in Singapore!