Category: Video

This week’s Monday post is a guest post written by members of the Computer Science and Artificial Intelligence Lab at MIT.

One of the focuses of the Distributed Robotics Lab, which is run by Daniela Rus and is part of the Computer Science and Artificial Intelligence Lab at MIT, is to study the coordination of multiple robots. Our lab has tested a diverse array of robots, from jumping cubes to Kuka youBots to quadcopters. In one of our recent projects, presented at ICRA 2017, Multi-robot Path Planning for a Swarm of Robots that Can Both Fly and Drive, we tested collision-free path planning for flying-and-driving robots in a small town.

Robots that can both fly and drive – in particular wheeled drones – are actually somewhat of a rarity in robotics research. Although there are several interesting examples in the literature, most of them involve creative ways of repurposing the wings or propellers of a flying robot to get it to move on the ground. Since we wanted to test multi-robot algorithms, we needed a robot that would be robust, safe, and easy to control – not necessarily advanced or clever. We decided to put an independent driving mechanism on the bottom of a quadcopter, and it turns out that the Crazyflie 2.0 was the perfect platform for us. The Crazyflie is easily obtainable, safe, and (we can certify ourselves) very robust. Moreover, since it is open-source and fully programmable, we were able to easily modify the Crazyflie to fit our needs. Our final design with the wheel deck is shown below.

A photo of the Crazyflie 2.0 with the wheel deck.

A model of the Crazyflie 2.0 with the wheel deck from the bottom

The wheel deck consists of a PCB with a motor driver; two small motors mounted in a carbon fiber tube epoxied onto the PCB; and a passive ball caster in the back. We were able to interface our PCB with the pins on the Crazyflie so that we could use the Crazyflie to control the motors (the code is available at https://github.com/braraki/crazyflie-firmware). We added new parameters to the Crazyflie to control wheel speed, which, in retrospect, was not a good decision, since we found that it was difficult to update the parameters at a high enough rate to control the wheels well. We should have used the Crazyflie RealTime Protocol (CRTP) to send custom data packages to the Crazyflie, but that will have to be a project for another day.
The table below shows the mass balance of our miniature ‘flying car.’ The wheels added 8.3g and the motion capture markers (we used a Vicon system to track the quads) added 4.2g. So overall the Crazyflie was able to carry 12.5g, or ~44% of its body weight, and still fly pretty well.


Next we measured the power consumption of the Crazyflie and the ‘Flying Car.’ As you can see in the graph below, the additional mass of the wheels reduced total flight time from 5.7 minutes to 5.0 minutes, a 42-second or 12.3% reduction.

Power consumption of the Crazyflie vs. the ‘Flying Car’

 

The table below shows more comparisons between flying without wheels, flying with wheels, and driving. The main takeaways are that driving is much more efficient than flying (in the case of quadcopter flight) and that adding wheels to the Crazyflie does not actually reduce flying performance very much (and in fact increases efficiency when measured using the ‘cost of transport’ metric, which factors in mass). These facts were very important for our planning algorithms, since the tradeoff between energy and speed is the main factor in deciding when to fly (fast but energy-inefficient) versus drive (slow but energy-efficient).

Controlling 8 Crazyflies at once was a challenge. The great work by the USC ACT Lab (J. A. Preiss, W. Hönig, G. S. Sukhatme, and N. Ayanian. “Crazyswarm: A Large Nano-Quadcopter Swarm,” ICRA 2017. https://github.com/USC-ACTLab/crazyswarm) has made our minor effort in this field obsolete, but I will describe our work briefly. We used the crazyflie_ros package, maintained by Wolfgang Hönig from the USC ACT Lab, to interface with the Crazyflies. Unfortunately, we found that a single Crazyradio could communicate with only 2 Crazyflies at a time using our methods, so we had to use 4 Crazyradios, and we had to make a ROS node that switched between the 4 radios rapidly to send commands. It was not ideal at all – moreover, we had to design 8 unique Vicon marker configurations, which was a challenge given the small size of the Crazyflies. In the end, we got our system to work, but the new Crazyswarm framework from the ACT Lab should enable much more impressive demos in the future (as has already been done in their ICRA paper and by the Robust Adaptive Systems Lab at Carnegie Mellon, which they described in their blog post here).

We used two controllers, an air and a ground controller. The ground controller was a simple pure pursuit controller that followed waypoints on ground paths. The differentially steered driving mechanism made ground control blissfully simple. The main challenge we faced was maximizing the rate at which we could send commands to the wheels via the parameter framework. For aerial control, we used simple PID controllers to make the quads follow waypoints. Although the wheel deck shifted the center of mass of the Crazyflie, giving it a tendency to slowly spin in midair, overall the system worked well given its simplicity.

Once we had the design and control of the flying cars figured out, we were able to test our path planning algorithms on them. You can see in the video below that our vehicles were able to faithfully follow the simulation and that they transitioned from flying to driving when necessary.

Link to video

Our work had two goals. One was to show that multi-robot path planning algorithms can be adapted to work for vehicles that can both fly and drive to minimize energy consumption and time. The second goal was to showcase the utility of flying-and-driving vehicles. We were able to achieve these goals in our paper thanks in part to the ease of use and versatility of the Crazyflie 2.0.

A couple of weeks ago Qualisys visited us at the Bitcraze office, they came with the Miqus motion capture system that they installed temporary in the office. This gave us the opportunity to play with their motion capture system and the Crazyflie 2.0 :-).

It was our first hands-on experience with a motion capture system. We were eager to try the algorithms that have been developed for the loco positioning system with the more precise position information offered by the motion capture. The result was above our expectations. You get a bit amazed when it is just sitting there in the air. The normally difficult in-air photo shoot became a breeze since you suddenly have plenty of time to focus and shoot.

After running a couple of simple stabilized flight demos, we endeavored to run the ICRA demo with motion capture instead of our loco positioning system. As the loco positioning deck isn’t needed it was removed and instead the measured position was sent using Crazyradio. Doing so made the demo work pretty much out of the box. The ICRA demo had 2 buttons, one for playing a pre-recorded trajectory and one for recording a path and playing it back as soon as the Crazyflie is dropped. Both modes worked seamlessly without requiring any code change. We tried the path recording and playback functionality and were pretty impressed by the precision:

Link to video

We look forward of meeting and working more with Qualisys. One goal is to provide better information, documentation and tools to get started with Crazyflie in a motion capture system.

For a while now we have been selling the BigQuad deck which makes it possible to transform the CF2 to control a bigger sized drone. It does so by becoming the quadrotor control board, controlling external brushless motor controllers, which allows to scale up the size. This can be very convenient when trying out/developing new things as it first can be tested on the small CF2 and later scaled up by attaching it to a bigger quad. However for a more permanent setup it is a bit bulky, so we have been playing around a bit and designed something in the middle. The result is a stand alone control board targeting quads around 0.1 – 0.5kg.

We call it the CF-RZR as it is inspired by the smaller sized racers with some fundamental differences. It is designed with a higher level of autonomous functionality in mind and being easy to repair while still being fully compatible with the CF2 firmware and decks. Listing the biggest features of the current prototype:

  • Fully compatible with the CF2 firmware, expansion decks as well as radio.
  • Connectors to attach motor controllers (still possible to solder though) so it is easy to build and repair.
  • Power distributions built into controller board. (Max ~8A per motor controller though)
  • Motor controllers can be switched of by the system so the system can go into deep sleep and consume around 50uA.
  • Voltage input 1S-4S (3V to 17V)
  • Standard mounting (M3 mounting holes placed 30.5mm square)
  • External antenna for increased range

To summarize, the strength of the CF2 but in a little bit bigger package :-). Last week we got a chance to test fly it for the first time. We used a off the shelf racer frame, ESC and motors. At first it did not fly that well at all but after some PID tuning it became pretty stable and we had a lot of fun :-).

We would love your feedback, good/bad idea, what do you like/dislike etc!

Link to video

We exhibited at the IEEE International Conference on Robotics and Automation in Singapore a couple of weeks ago.

We had a booth where we demoed autonomous flight with the Crazyflie 2.0 and the Loco Positioning system, without any external computer in the loop. The core of the demo was that the Crazyflie had an onboard trajectory sequencer that enabled it to fly autonomously along a path, based on the position from the Loco Positioning system.

We had a pre programmed path that we used most of the time, since it enabled us to start the demo and the leave the Crazyflie without any further manual interference from our side (except changing battery). The other option was to manually record a path for the Crayzflie to retrace by moving it around in the flying space. When we dropped it (detecting zero gravity) the onboard sequencer and controller took over to replay the recorded path. This mode was very useful when showing the accuracy and performance of the system by recording a short sequence of one point and just leaving the Crazyflie to hover. We had mounted a deck with two buttons on the Crazyflie that we used to chose which mode to use.

The code used for the demo is available at github for anyone to play with.

Optical flow

We also showed our brand new Flow deck that we will release soon. It is a deck that is mounted underneath the Crazyflie with a downwards facing optical flow sensor. The sensor is in essence what is used in an optical mouse but with a different lens that enables it to track motion further away. The output from the deck is delta X and Y for the motion of the Crazyflie and can be used by the onboard controller to control the position. We will publish more information in this blog soon.

We had a great time talking to all you interesting, bright and awesome people. Thanks for all feedback, sharing ideas and telling us about your projects!

A couple of weeks ago we played with recording and retracing trajectory directly from the Crazyflie using Loco Positioning System. The result was quite nice and resulted, a first for us, in a fully autonomous Crazyflie, no computer or controller required:

We decided to expand on this experiment for our demo at ICRA. We have modified the retracing code to accepts multiple modes, including running pre-programmed sequence. The plan for the demo is to have Crazyflies that can:

  • Record and retrace a manual trajectory
  • Record and replay in a loop a manual trajectory
  • Play a pre-defined trajectory in a loop
  • Land automatically when the battery level is low

With this we should be able to demonstrate quite well the capabilities of both the Crazyflie and the Loco Positioning system, and since we do not require a computer in the loop it simplifies a lot running the demo. Of course we keep the possibility to connect the Crazyflie with the Crazyflie client and with ROS while the crazyflie is flying.

Having a completly autonomous Crazyflie is also new to us and it brings its share of problems: how to we choose the working mode and how to we stop the flight if something happens (things tends to happen …).

To solve the former we have made a button deck that adds 2 push-button to the Crazyflie. One means “Start autonomous sequence”. The second means “Record trajectory”. If the recorded trajectory is a loop (if the end point is close to the start point) then the loop is played back as soon as the crazyflie is dropped, otherwise Crazyflie retraces the trajectory and stop.

We solved the later problem by making an autonomous emergency stop button that sends a radio watchdog signal. If the signal stops to be sent or if an emergency stop signal is sent (ie. by pressing the button), the Crazyflie will stop all motors and drop. The button is implemented using a Raspberry pi, a Crazyradio and an Arduino to interface the button:

If you are curious about code, we have created a github repos where we push all code we are making for this demo. As usual, this conference is an opportunity for us to hack new functionalities, though not everything can be done in the master branch. Later some things can be merged, others (like the retrace trajectory recorder/player that looks more like a user app.) will need much more though if we want to merge it in the Crazyflie firmware.

At Bitcraze we have some history with trying to fly our Crazyflie autonomously. The most recent step is the Loco Positioning System that allows us, and you, to fly in a full room. The Loco Positioning system has boosted development of advanced algorithms for onboard position estimation and control.

Our earlier attempts where mostly based on different kinds of cameras, either a 3D camera like Kinect or regular webcams. Though, at that point, we only had the camera for position estimation and where doing the position control on the PC and not onboard the Crazyflie. This has the disadvantage to be brittle and requires a very high quality positioning from the camera: any frame where we loose the Crazyflie has a huge impact on the control behavior since the position controller relies exclusively on the camera detection.

With the Kalman filter and onboard position controller, the Crazyflie can now handle lost position information for at least a couple of seconds without big problems. This has the potential of making webcam-based position detector much more robust!

To test this theory we have grabbed the 2 years old crazyflie-ar-detector from the dawer github, updated it to OpenCV 3.2.0, and fed the position output to the Crazyflie 2.0 external position port. The crazyflie-ar-detector program is using ZeroMQ to communicate position and so we made a simple external position tab for the Crazyflie Client that receives position from ZeroMQ and sends it to the connected Crazyflie.

Using the new position-hold mode recently introduced in the client we can test and fly the Crazyflie under the webcam. We have taken a short video to show the performance. The result is promising and we will continue to play with ways to fly the Crazyflie autonomously.

Early on when we started to work on the Loco Positioning system, we came up with an idea of a Crazyflie autonomously flying into a light box, positioning it self for a few product pictures and then flying out again. The positioning system is now pretty mature and close to leave Early Access and this Friday we finally got around to do it. In this blog post we will share what we did and it also doubles as a brief howto on how to set up the system and fly a simple autonomous sequence.

We used a Crazyflie with a Loco Positioning Deck and eight Loco Positioning Anchors in our setup. Six anchors would have been fine too, but we happened to have eight in our flight lab.

When working with the Loco Positioning system the first step is always to make sure the anchors are set up correctly. We had an experimental version of the anchor firmware so we started out by pulling down the latest stock version of the source code and compiled it into a .dfu file. After that we fired up the brand new lps-tool that is used to flash firmware and configure the anchors. The anchors must be connected with a USB cable to the computer but the lps-tool reduces the flashing and configuration into a few clicks. When all anchors were updated we were ready for the next step.

The positions of the anchors are stored in the anchors them selves and the position is transmitted to the Crazyflie as a part of the ultra wide band messages used for measuring the ranges to the anchors. This way, the Crazyflie gets both anchor positions and ranges in the same process and has all the information needed to calculate its position. The second step is thus to store the positions in the anchors. In our “flight lab” we have fixed mounts for the anchors with known positions, so we could skip measuring the physical positions of the anchors.

We are working on making it possible to remotely configure the anchors to reduce the need to physically connect to them, and the position can now be set from the Crazyflie Client. We simply opened the “Loco Positioning” tab in the client, connected to a Crazyflie (with a Loco Positioning deck mounted), entered the anchor positions and hit the “Write to anchors” button. A few seconds later the anchor positions in the graphs were updated to indicate that the positions have been written to the anchors and then subsequently sent back through the ultra wide band messages to the Crazyflie.

Step three is to verify that the system is working as expected. First thing is to check that we did not mix the anchors up when configuring or placing them. In the “Loco Position” tab in the client, click the “Anchor Identification” button. In this mode anchors are lit up in the graphs when the Crazyflie gets close to them in the physical world. We went from anchor to anchor with the Crazyflie and checked that the correct anchor lit up on the screen. When confident that all was good we changed to “Position estimate” mode and verified that the estimated position matches the physical position of the Crazyflie. We have found that it can be very hard to understand, for instance that two anchors have been mixed up, by looking at the estimated position and that the “Anchor Identification” step simplifies the setup.

At this point we had a fully functioning Loco Positioning system ready for autonomous flight!

Now it was time to script a sequence. The easiest way to script a sequence is to start from the autonomousSequence.py example. Our intern Alfred took over at this point, he updated the uri to the correct settings and crafted a sequence to take off, fly into the light box, wait a while and then fly back out for a stylish landing in his hand!

Now we were ready for the actual photo shoot and Björn came down with the camera to shoot the product pictures. We hope you enjoy the results!

We have spent most of our time working on the two way ranging in the Loco Positioning system lately, mainly on features that are not directly related to the actual ranging but such as making it easier to configure and upgrade the anchors. As a result we have not exercised the TDoA mode in a while, so on our Fun Friday we wanted to play a bit with that and try to fly a small swarm with some sort of coordinated autonomous flight.

Setup

We have a Loco Positioning system set up in the basement, we call it the flight lab to make it sound more fancy! The setup has been 6 anchors with three anchors in the ceiling and three on the floor, configured as triangles pointing in opposite directions. When using two way ranging that is fine as the positioning works pretty well outside the volume defined by the anchors. For TDoA on the other hand, the accuracy of the estimated position degrades rapidly when you go outside the convex hull. We decided to add two more anchors (to a total of eight) and arrange them as the corners of a box. A few hours and mounting/cabling later we were ready to try it out.

We modified the swarmSequence script to suite my (limited) space and set it up to fly four Crazyflies in a square, moving them to the next corner of the square every 5 seconds. Next problem was to find 4 working Crazyflies and Loco Positioning decks. We have a few Crazyflies lying around but a fair number of them have been modified in one way or another but finally we had the hardware we needed and could run the script. After a couple of failed tries (out of battery and such) we shot this video

Lessons learned

So what did we learn from this exercise? Adding two more anchors and changing anchor positions improved the positioning significantly. We have seen earlier that TDoA is less accurate than two way ranging, but better anchor positions reduces the problem. We could also fly the swarm using our example python script (not using ROS) without too much work and trouble, even though the TDoA mode still is very experimental.

In this flight we used the stock controller and just moved the set point to the next desired position for each copter. We are really looking forward to try out the improved controller and trajectory planning that we showed at Fosdem in combination with the TDoA mode, we think it will improve the performance a lot!

For the third year some of us from Bitcraze visited Fosdem, the biggest open-source European conference. Like the other years we enjoyed being there a lot and we had a great time hanging-out with community members like Fred.

Fred presented a great lightning talk about the news in the Crazyflie galaxy, the video and slide are already available. 

Arnaud talked about the Loco positioning system. The talk and the demo, went well. Unfortunately the video from the talk is not available yet, we will tweet it and add it to this post as soon as it is online.

The Loco Positioning talk was a great opportunity for us to test the most recent bleeding edge additions to the Crazyflie autonomous algorithms. We flew the new non-linear controller from Mike Hammer using trajectory generation from Marcus Greiff. The non linear controller uses setpoints not only of position but also of velocity and acceleration to control the Crazyflie. This is where trajectory generation is useful: if you can generate a trajectory and calculate position, velocity and acceleration over time, you can feed all this information to the controller and the controller will be able to do a much better job following your trajectory. This enabled us to fly the Crazyflie fairly aggressively the week before the FOSDEM talk:

In this video the Crazyflie is accelerating to about 2g continuously to keep the trajectory. We were a bit concerned to fly such aggressive maneuvers in public without more testing so we designed a slightly safer demo the night before the talk in our hotel room:

This trajectory was successfully flown in the demo and shows the performance of this new controller. There has been a lot happening with the Crazyflie control algorithms lately: Marcus, Mike and Wolfgang have all made new controllers and Marcus has developed an on-board trajectory generator. There is still some work required in the firmware architecture to merge these into Master, but we hope this can be done in the coming weeks. Follow the Crazyflie firmware commits and github tickets if you are interested in the progress.

During the fall of 2016 fashion designer Maartje Dijkstra have in collaboration with music producer Beorn Lebenstedt (Newk) and engineer Erik Overmeire been working with the creation “TranSwarm Entities”, a dress made out of 3D prints accompanied by autonomously flying Crazyflies. The project was made during the Fashion Fusion Lab, a three-month workshop in which selected teams got to work on their fashion concepts. Maartje and her team used our Loco positioning system to enable 4 Crazyflies to do a “dance” around the dress during the show.

 

Copyright Fashion Fusion

Here is how Maartje describes the creation:

“The sculptural high fashion dress is totally build up out of small fragments (bird skulls), like cells building an organism.The parts are manual 3D printed and after printing all connected together by hand with polyester wires and green leather. The technology part is integrated in a special way. 4 small drones, that have given the same black 3D printed appearance as the dress, fly up from places inside so it looks like parts of the dress are flying away.The drones fly on the beats and melodies of music producer Newk around the model creating a little swarm. The shoes are digital 3D printed but finished manually.”

The finalists from the Fashion Fusion Lab got to compete during the Berlin Fashion Week at the “Fashion Fusion Challenge” and we are happy to announce that Maartje together with her team got the third place

We at Bitcraze are very happy for Maartje and her team and think it’s very exciting to see the Crazyflie 2.0 and the Loco positioning system being used in such a different context. It shows again the potential for future applications and how versatile the Crazyflie and the Loco positioning system is. 

Here is a video showing the dress: