I have returned from my family visit in California, who I’ve haven’t seen them in 3 years due to Covid. To spend the most possible time with them, the plan was that I would still work full time for Bitcraze from my father’s home. The problem became however, that it wouldn’t fit so well in our current way of work as I would miss all the morning stand up meetings due to the large time difference between Sweden and California (-9 hours). That is why we settled that I would work on separate projects/investigations during my time away. So I thought it would be a great opportunity to dig into ROS and 3D simulations again and see what the latest state of that is! So about the simulations is what I’ll be mostly talking about right in this blog post, in terms of what simulators are out there and what simulation development is currently ongoing.
Need for simulation?
Why would it be actually be necessary to have a simulation in our current frame work? Just to give an example, my new colleague Jonas recently tried out his hand on the CFlib swarm class for the first time for the BAMdays tutorials, and simulator would have been great during that initial porcess. Namely, most of the crashes were not necessary due to low batteries or bad communication, but mostly due to the fact that he was not able to double check his script beforehand. If one is able to check if all the programmed positions of the Crazyflies are implemented as they should before an actual flight, this would prevented a lot of broken propellers!
Just to note here that there are a lot of types of simulations that you can think of. Earlier this year had our ex-interns Max and Josephine finish an Renode simulation of the Crazyflie’s microcontrollers. We’ve also seen the word Simulink pop-up multiple times on the forum which indicates that quite some control classes are investigating the dynamic model of the Crazyflie. However, the type of simulation that I’m currently referring to are the 3D simulators in which a robot or quadcopter can move and interact with a virtual environment, with usually an physics engine in effect.
Crazyflie in Gazebo (+ROS)
During some initial investigation there were already some simulations that pop out. First of all I went and looked into what is available for Gazebo at the moment, which is:
CrazyS is based on the RotorS simulation with some additional off-board crazyflie controllers for position control. I wasn’t able to build it for my Ubuntu 20.04 just yet myself, but that there is ongoing work to port CrazyS to ROS Noetic. For now on a virtual machine with ROS melodic it build just fine! Note my laptop did had to work quite hard when I wanted to simulate more than 1 Crazyflie, but the physics and plugins that were made for Gazebo is enabling many to do a lot for their research. Please check out the core papers about CrazyS!
Sim_cf is perhaps a little lesser known, but the project does stand out as it has some interesting features to it. It is for instance, possible to use the actual c-based firmware in software-in-the-loop (SITL) mode, which controls the simulated Crazyflie. It is even possible to use an actual crazyflie with an hardware-in-the-loop (HITL) simulation. Eventhough the project is not actively maintained anymore, I did manage to build it from source for ROS Noetic and Gazebo 11, although I was not able to fly more than 4 do to errors.
Other Simulators
Ofcourse Gazebo is not the only possibility out there. I also had a quick go at another simulator called Webots, which is quite an interesting option indeed as well. Currently there is only one quadcopter model available, so it only makes sense for it to also contain an Crazyflie! They do use their own robotic format, so probably the easiest process would be, is to convert an existing model for Gazebo/ROS into an format that Webots can understand.
So in the future, the current Gazebo in its form will disappear and will be only be part of Ignition. So that is why it made sense for me to start playing with an separate Crazyflie model and plugins for the Ignition frame work instead. Moreover, it seems that quite some elements and plugins based on the RotorS simulation for the original gazebo, are now fully integrated within the Ignition gazebo framework, which should make it more easier to make quadcopter models fly. Currently it’s still work in progress, so right now is only to be found on my personal github repository, but as soon as it becomes more fleshed out and stable, this will probably transferred to Bitcraze’s github repos and we will write a more elaborate blogpost about it. For now, I’ll try to work on it further as my Fun Friday project!
In the mean time, we have started a simulation discussion thread in the Crazyswarm2 repository, which is an ongoing port of Crazyswarm to ROS2. It would be the ideal situation if we would be able to use this simulator for both Crazyswarm and our native CFlib! But I’ve mostly have used Gazebo in the past, so if there are any other simulators that we should try out too, please join the discussion and let us know!
The parameters have default values and up until now have been reset to those default values on each restart of the Crazyflie. This has not been seen as much of a problem (Or has it? Let us know!) since most use of the Crazyflie platform has been session based and the need for persistent parameters has been low. Our work with getting the Crazyflie Bolt out of early access has however changed this need.
The Crazyflie Bolt kit
The Bolt will need different values for the tuning parameters for controllers and being forced to set these on each boot would be pretty annoying. And since we want the Crazyflie Bolt and the Crazyflie 2.1 to share the same firmware, persistent parameters would come in handy.
Fortunately the Crazyflie includes an EEPROM (Electronically Erasable Programmable Read-Only Memory) which we can use to store parameter values and have them survive restarts of the quad.
So this means, with the latest branches of the firmware and the python library, you can set values of a persistent parameter, store it to EEPROM, and the parameter will keep its values across reset and even across firmware upgrades.
Persistent Parameter API
Not all parameters will be able to be persistent and stored in the EEPROM. We have added a way to declare a parameter as persistent in the firmware:
This will allow the new API added to our python library to be used with this parameter. The API consists of three actions:
Getting the state of a persistent parameter
Storing the current value of a persistent parameter
Clearing the stored value of a persistent parameter
The state of a persistent parameter consist of three pieces of information:
Is the value stored or not
The default value of the parameter
The stored value of the parameter
Setting a value, using the regular parameter API, is not enough to store the value to the EEPROM for a persistent parameter, you need to be explicit. A special API call is needed to store the current value to memory. And to stop storing the value you need to use the clear API action.
Python API
The above choices corresponds to three methods in our Python API:
Get the state of the specified persistent parameter. The state will be returned in the supplied callback. The state is represented as a named tuple with members: is_stored, default_value and stored_value. The state is None if the parameter is not persistent or if something goes wrong.
Store the current value of the specified persistent parameter to EEPROM. The supplied callback will be called with True as an argument on success, and with False as an argument on failure.
Clear the current value of the specified persistent parameter from EEPROM. The supplied callback will be called with True as an argument on success and with False as an argument on failure.
Example code
We have also added an example script that will showcase the functionality, it will list the state of all persisted parameters in the firmware, and will store and clear some of them. The output when run against current master is:
$ CFLIB_URI=radio://0/80/2M/E7E7E7E7E8 python3 examples/parameters/persistent_params.py
-- All persistent parameters --
sound.effect: PersistentParamState(is_stored=False, default_value=0, stored_value=None)
ring.effect: PersistentParamState(is_stored=True, default_value=6, stored_value=2)
Set parameter ring.effect to 10
Store the new value in persistent memory
Persisted ring.effect!
The new state is:
ring.effect: PersistentParamState(is_stored=True, default_value=6, stored_value=10)
Clear the persisted parameter
Cleared ring.effect!
The new state is:
ring.effect: PersistentParamState(is_stored=False, default_value=6, stored_value=None)Code language:PHP(php)
You can count on more parameters to be marked as persistent in the near future. Hopefully this will be useful for you! Please report any issues you find!
From Star Wars to Black Mirror, sci-fi movies predict a future where thousands of drones will fill our sky. Curving sharply around trees or soaring over buildings, they fly just like a flock of starlings. To turn this vision into a reality, real drone swarms need to increase their autonomy and operate in a decentralized fashion. In a decentralized swarm, each robot makes its own decision based only on local information. Decentralization not only allows the swarm to be more robust to the failure of single individuals, but also removes the dependency from a single computing unit, thus making the swarm more scalable in terms of size.
We at LIS (EPFL) have shown that predictive controllers can improve the safety of aerial swarms by predicting and optimizing the agents’ future behavior in an iterative process. However, the centralized nature of this method allowed us to only control five drones and prevented us from scaling up to a large number of drones. For this reason, we have worked on a novel decentralized and scalable swarm controller that allows the safe and cohesive flight of aerial swarms in cluttered environments. In our latest article, published in IEEE Robotics and Automation Letters (RA-L), we describe how we designed the controller, show its scalability in size, and demonstrate its robustness to noise. We studied the swarms’ performance and compared how it changes in two different environments: a forest and funnel-like environment.
The Crazyflie 2.1 was the perfect platform for our experiments. They are lightweight, modular, and tough. This quadcopter can survive big hits when things don’t go as planned… and, if you work on swarms, things can go wrong!
The fleet of Crazyflies equipped with a single marker.
With our algorithm, sixteen robots were able to fly through an artificial forest that we set up in our indoor motion capture arena. In our previous work, we installed four markers on each quadcopter and used the rigid body tracking from Motive (the Optitrack software). The large volume of our experimental room required the usage of big markers for long-distance detection, which added considerable weight to the drone. Hence, in our new work, we use a single marker per drone. Tracking is supported by the ‘crazyswarm’ package and communication with the entire swarm only requires two radio links. However, despite our model being decentralized, in our implementation robots relay the information to an external brain, which does the computations for them. In the future, all the necessary code will be embedded onboard, removing the dependency on external infrastructure.
Our predictive swarm of Crazyflies flying among obstacles in our indoor experimental room.
Video about the article
This work is a step forward towards the fully autonomous deployment of drone swarms in our cities. By enabling safe navigation in cluttered environments, drone fleets will be able to integrate with conventional air traffic, search for missing people, inspect dangerous areas, transport injured people to hospitals quicker, and deliver important packages right to our doors.
The last week was epic. We had 3 days of our online conference, the BAM days – I’m sure you’ve heard of them by now.
We are really happy with how everything went down. During those 3 days, 142 people attended, which is a highest number than we could have expected. The Welkom platform we used was stellar, allowing us to use Mibo rooms for very fruitful discussions after each talk.
Quiz and community Q&A
We took the opportunity to talk to our community, which is something we didn’t have the opportunity to do in a long time. Your insights and feedback were greatly appreciated and we have a lot to think about in the next coming weeks on how to best use all the remarks we got.
We also had a short quiz about Bitcraze, and we were quite impressed with how you performed ! And interesting to note that the hardest question for you was how many decks we sell (it’s 16, if you want to cheat on our hypothetical next quiz)
As I’ve mentioned, after each event we gathered in Mibo rooms. Even though attendance there was not as high as we would have liked, we still got quality time with community members, speakers, collaborators, even first-timers that were interested in the Crazyflie. We really love this platform, making us feel almost like meeting in real life. We even had some karaoke in Mibo during the closing party (which MAY be a good excuse to end the day for those who listened)
Content
Our external speakers presented a lot of interesting work. It was a great pleasure and honor to welcome every one of them as they explained their latest work. I have to admit that it’s rewarding to see such smart people doing awesome and cool research with our products.
We did our share too, with workshops and demos. Kristoffer’s autonomous demo using distributed consensus required a lot of work but worked perfectly in the end. Here is a small excerpt:
What now?
Now, we’re feeling as everyone is feeling the day after a party: exhausted, happy, and wondering what to do next. Hopefully we have some plans for that !
If you missed the conference, we created a Youtube playlist where you can watch everything that you missed. During the next few days, we’ll update the event page with BAM’s presentations too so you will have the opportunity to catch up. Some of our workshops will also turn into tutorials or documentations of some kind, but we’re still just cleaning up.
We are so happy with how everything went that we are already thinking about a future BAM. This one was exceptional, of course, since it was at first to celebrate our 10 year anniversary (and I have to admit that we’re all a little bit tired after 3 intense days), so we’re not going to be able to top that. But we are considering making BAM a fixed point in our agenda (and yours, let’s hope). We don’t how, we don’t know when, but one thing is sure: BAM is just beginning.
Kimberly on a different continent
On a totally different note, Kimberly is flying to the US this week: if any of you America-based wants to grab the occasion to have a more time-zone appropriate conversation with one of us, you will have a few weeks to make it possible!
Only 8 days left before the first Bitcraze-organized conference ever. We are really excited about it, and this coming week is pretty much dedicated to organize the conference. We’re working hard on getting the best experience possible for there 3 days, both for us and for you. That means collaborating with great speakers, creating awesome workshops, but also think about the best way to come together – one of the main purpose behind the BAM days.
It’s been a couple of months since we decided that we wanted to organize our own conference. The main idea behind it was that we missed going to conferences, meeting people and talking with a lot of interesting persons.
Expression of the BAMdays
Bitcraze turning 10 felt like a big occasion that we wanted to celebrate. And indeed we did, internally, but we wanted to share this accomplishment with you too. An online conference seemed like the best idea. So when time came to choose a platform for it, we looked for something which put emphasis on social experience. We finally went with the Welkom platform, from the Netherlands. It’s easy and natural to use, with a simple interface, and lots of options for us to organize different activities.
The event platform
The Welkom platform we’ll use hosts two different Mibo rooms. It’s a spatial chat, where your avatar can run around a 3D world, discovering different environments. Your camera is your head, so you’ll need a webcam to join. There are some fun activities around, plenty interactive stuff, from chairs to basketballs. We have used it to for our coffee breaks when we were working from home, and had fun using this world: we’ll be happy to meet you there and hope you’ll find it fun too.
Bitcraze goofing around in Mibo
The plan is to have between each talk or workshop at least 30 minutes in Mibo to talk to the speakers, take a coffee break, discuss the exciting things that’s been happening, and generally catch up in ways we haven’t be able to the past year.
At the end of each day, we’ve programmed social events to gather together. On Tuesday, we’ll host a special quizz. Telling you its content may be a bit cheating, but if we’ve read our blogposts regularly, the questions should not seem hard to you. For the winner, a once-in-a-lifetime reward: a GoldenFlie ! You will be able to put it in your chimney mantle with your others awards, or actually use it – because, yes, it flies!
The GoldenFlie!
The second day, Wednesday, we’ll have a roundtable with Q&A. A perfect occasion for us to get feedback from you, and for you to ask us all the questions you have been burning to ask us.
The last day (Thursday), we’ll talk a little about Bitcraze and its future: where it is headed. We’ll then have a last party – in Mibo of course.
All the program and other useful information is in our event page. We really hope you’ll take some time to come by and say hello !
On September, 1st of 2011, Arnaud, Marcus and Tobias signed a paper in Lund registering their brand-new company: Bitcraze. We’re here, now, 10 years later, hardly believing this gamble paid off so greatly.
Bitcraze’s story actually begins in 2009, when the 3 met in their day job and started dreaming about what will become the Crazyflie. For a long time, they combined a 9-to-5 job with the development of the nanodrone. Monday meetings run from the end of the workday until 2am, and they were exhausted, but the hard work paid off when we know now how many Crazyflies took off after that!
Some things have been added since 2011, of course! For starters, 4 more people joined the team, each changing the face of the company, making it evolve. Lean and self-organized was the way to go, the more people there was. The product itself changed and grew: from the first batch of Crazyflies, that had to be soldered out of the box, we’re now at the 2.1 version. And as years flew by, Bitcraze added to its backpack with new decks and extensions, always wanting to test new things and generally make awesome stuff. From its first flight, we’re now at impressive autonomous swarm flights!
The Crazyflie celebrating its first year in 2012! (and yes, it flew with the candle on it)
In 10 years, we’ve also done things we would never have thought of. Did you know Marcus gave a 15 minutes speech at a conference … entirely in French ? Or that 3 different employees got a cut on their finger for the exact same reason (we’ll let you guess why!) ? We’re still amazed at how much we discovered, learned and shared in those 10 years.
For us, it’s a special moment, looking back at 10 years of awesome things, of hard work, and of passion. We plan on celebrating, of course! First with a few days’ getaway, the first occasion we’ve had in a while to be actually together and enjoy. But this milestone would not be happening without all the support of those around us: our family and friends of course, but also our contributors and our customers. That’s why we’ve decided to organize our own conference, to gather everyone for awesome days of meetup and celebrate with you!
Keep an eye out on our social media too, as this September we’ll look back a little down the road. We’ll post some fond memories (and maybe, just maybe, some embarrassing photos too!) and see how much we’ve accomplished in 10 years.
How to handle our documentation has been always a bit of struggle. For almost 2 years (see this blogpost and this one) we have working on improving the documentation structure, with by transferring information from the wiki, putting information closer to the code and setting up automating documentation. A few months ago, we managed to have automated logging and parameter documentation (see this blogpost).
Even though we think there is some improvement already, it can always be better! We have noticed that some of our users are a bit confused of how to go through our documentation. So in this blogpost we are discussing some navigational strategies of how you can maneuver yourself through the documentation as it is presented on bitcraze.io, which can also be found here.
Ecosystem-based navigation
So more than a year ago, we also started with a Ecosystem overview page, which are meant to take first-timers by the hand through the Crazyflie ecosystem.. This type of overview pages are starting from the three main pillars: the Crazyflie Platform, the Clients and Positioning Technology. This is a type of navigation that we mostly advise to take if you are a beginner Crazyflie user who do not know the structure of the eco system fully.
For those that already have experience with the Crazyflie and its Ecosystem, the previous way of navigating through the docs might be a bit convoluted. With the Ecosystem-based navigation, it takes about 3 scrolls and clicks to reach the STM development documentation, which is a bit to much of a round way if you already know what you are looking for. We have made the repository overview page not for this purpose but we actually started using ourselves a lot within the company, as a direct pathway to the development repository per element. So this is a page that would be useful to other advanced developers as well!
So the repository overview page is split up in 4 main categories: Python-based software, C-based firmware, Other languages and bootloaders. See the navigation tree which of those repositories approximately point too. By the way, have you noticed that repository documentation has a gray header (like this one) and all the overview pages on the web have a green header (like this one)? This are meant to make you aware if you are still on a fluffy overview website page or going in the nitty gritty details of the development documentation.
Repository-based navigation tree
Feature-based navigation ?
Still a remaining problem is that the repository documentation might not be enough to get a good overview. Where do you need to look if you are interested in ‘controllers’ or ‘state estimators’, or how to make an app-layer application? Currently all of this is within the stm32 firmware documentation, as that is the exact location of where all of this is implemented. But how to document spanning features like the CRTP, where not only the STM chip but also the NRF, Crazyradio PA and the Crazyflie python library are also involved? Or how about the loco positioning system, where the Crazyflie communicates through the LPS deck with a separate LPS node?
So perhaps a good way how to present all this information, is to do it feature-based, like ‘controllers’, ‘positioning’, ‘high level commander’, where we present a structure that points to parts of the detailed documentation within the repo-docs. With ecosystem-based, or even repository-based, navigation documentation strategy, it will take for instance 4-7 clicks to come to the specific controller page, as you can verify by looking at the bread-crumb of the header. Perhaps splitting it up based on feature instead of Ecosystem elements or programming language might be a more logical structure of the current state of the Crazyflie documentation.
Feedback
One reason why it is so difficult to do this properly, is that we have a lot of repositories based on each microprocessor of all of our products, which makes our opensource projects quite unique. It is therefore difficult to find another opensource project of which we can take inspiration from. So, let us know what you would prefer for navigating through our documentation in this poll, but we are always open to other suggestions! If you know of any example of a similar opensource software project that is doing it the right way, or have any other tips, send us an email (contact_at_bitcraze.io), contact us on social media platforms or post a comment on this blogpost!
Update 2021-12-21:
The poll is closed and this is the result! Thanks all for responding!
Ever since we released the Lighthouse deck back in 2019, we’ve wanted to offer a bundle with the deck and the base stations. There’s multiple reasons for this, but the main reason was that we wanted users to be able to buy a full swarm (like the Loco Positioning Swarm) directly from us, without having to find the base stations separately. Initially this seemed easy to do, but it turned out to be a bit tricky. This post is about how we finally managed to get the Lighthouse Swarm Bundle finished and into the E-store.
The Lighthouse swarm bundle
When the Lighthouse deck was initially released it only had support for Lighthouse V1 base stations, but Ligthouse V2 was already out. Since the V1 base stations were already in short supply, we wanted to support V2 since this was what would be available in the future. We had started looking at V2 support, but there was still ongoing efforts from us (and others) to reverse engineer the protocol. After some prototyping we had some initial support, but there was still a lot of infrastructure work to be done before it could be released.
In parallell with this work we started trying to buy the Lighthouse V2 base stations. Normally there’s two options here, either buy from local distributors or buy directly from the manufacturer. Buying from local distributors wasn’t a good option for us since these will only have local power plugs and buying directly from the manufacturer often requires very large orders. So this process quickly stalled. But after a couple of months we got an offer to buy a bulk shipment of Ligthouse V2 base stations (without box or power adapters) which we finally decided to accept. And yeah, that’s me looking really happy next to a bunch of base stations…
Marcus looking happy about the base stations
With a bunch of base stations at the office, work with sourcing a power adapter and creating a box started. Unfortunately the number of COVID-19 cases started rising again shortly after receiving the base stations, so we started working more from home again. And with only 2 persons at the office at a time, it’s hard to work with hardware. Different team-members needs access to different resources, like the electronics labs, flight arena or packing orders. So getting box/adapter samples from manufacturers, doing testing and getting input on physical objects from other team-members quickly went from days to weeks.
Finally, after a couple of months of testing, evaluating and learning lots about adapters and cardboard, we had good candidates. But then, literally as we’re ordering the power adapters, it turns out the certification was not good for all the regions we wanted. Thankfully this time around we already had other options so we quickly decided on the second best option (now the best option) and ordered.
In the meantime work was underway finalizing the implementation of Lighthouse V2, including client support, firmware updates of the Lighthouse deck and documentation/videos. Finally in the beginning of 2021 we got documentation and the full implementation (although only for 2 base stations) in place (blog post).
After a bit more than a month of waiting, the power adapters and boxes finally showed up at our office. With all the supplies in place, we started preparing for the packing. Since you can buy base stations for multiple sources, we wanted to keep track of the base stations that we were sending out to be able to debug issues users might have with these units. Also, even though the base stations had already been factory tested, we wanted to quickly test them before shipping them out. So our flight arena was turned into a makeshift assembly line and we had some outside help come in to do the packing.
Finally, the end result! We’re really excited to be able to offer yet another swarm bundle, the Lighthouse swarm bundle. And we’re pretty happy about how the packaging turned out :-)
This week we have a guest blog post from Bart Duisterhof and Prof. Guido de Croon from the MAVlab, Faculty of Aerospace Engineering from the Delft University of Technology. Enjoy!
Tiny drones are ideal candidates for fully autonomous jobs that are too dangerous or time-consuming for humans. A commonly shared dream would be to have swarms of such drones help in search-and-rescue scenarios, for instance to localize gas leaks without endangering human lives. Drones like the CrazyFlie are ideal for such tasks, since they are small enough to navigate in narrow spaces, safe, agile, and very inexpensive. However, their small footprint also makes the design of an autonomous swarm extremely challenging, both from a software and hardware perspective.
From a software perspective, it is really challenging to come up with an algorithm capable of autonomous and collaborative navigation within such tight resource constraints. State-of-the-art solutions like SLAM require too much memory and processing power. A promising line of work is to use bug algorithms [1], which can be implemented as computationally efficient finite state machines (FSMs), and can navigate around obstacles without requiring a map.
A downside of using FSMs is that the resulting behavior can be very sensitive to their hyperparameters, and therefore may not generalize outside of the tested environments. This is especially true for the problem of gas source localization (GSL), as wind conditions and obstacle configurations drastically change the problem. In this blog post, we show how we tackled the complex problem of swarm GSL in cluttered environments by using a simple bug algorithm with evolved parameters, and then tested it onboard a fully autonomous swarm of CrazyFlies. We will focus on the problems that were encountered along the way, and the design choices we made as a result. At the end of this post, we will also add a short discussion about the future of nano drones.
Why gas source localization?
Overall we are interested in finding novel ways to enable autonomy on constrained devices, like CrazyFlies. Two years ago, we showed that a swarm of CrazyFlie drones was able to explore unknown, cluttered environments and come back to the base station. Since then, we have been working on an even more complex task: using such a swarm for Gas Source Localization (GSL).
There has been a lot of research focussing on autonomous GSL in robotics, since it is an important but very hard problem. The difficulty of the task comes from the complexity of how odor can spread in an environment. In an empty room without wind, a gas will slowly diffuse from the source. This can allow a robot to find it by moving up gradient, just like small bacteria like E. Coli do. However, if the environment becomes larger with many obstacles and walls, and wind comes into play, the spreading of gas is much less regular. Large parts of the environment may have no gas or wind at all, while at the same time there may be pockets of gas away from the source. Moreover, chemical sensors for robots are much less capable than the smelling organs of animals. Available chemical sensors for robots are typically less sensitive, noisier, and much slower.
Due to these difficulties, most work in the GSL field has focused on a single robot that has to find a gas source in environments that are relatively small and without obstacles. Relatively recently, there have been studies in which groups of robots solve this task in a collaborative fashion, for example with Particle Swarm Optimization (PSO). This allows robots to find the source and escape local maxima when present. Until now this concept has been shown in simulation [2] and on large outdoor drones equipped with LiDAR and GPS [3], but never before on tiny drones in complex, GPS-denied, indoor environments.
Required Infrastructure
In our project, we introduce a new bug algorithm, Sniffy Bug, which uses PSO for gas source localization. In order to tune the FSM of Sniffy Bug, we used an artificial evolution. For time reasons, evolution typically takes place in simulation. However, early in the project, we realized that this would be a challenge, as no end-to-end gas modeling pipeline existed yet. It is important to have an easy-to-use pipeline that does not require any aerodynamics domain knowledge, such that as many researchers as possible can generate environments to test their algorithms. It would also make it easier to compare contributions and to better understand in which conditions certain algorithms work or don’t work. The GADEN ROS package [4] is a great open-source tool for modeling gas distribution when you have an environment and flow field, but for our objective, we needed a fully automated tool that could generate a great variety of random environments on-demand with just a few parameters. Below is an overview of our simulation pipeline: AutoGDM.
AutoGDM, a fully automated gas dispersion modeling (GDM) simulation pipeline.
First, we use a procedural environment generator proposed in [5] to generate random walls and obstacles inside of the environment. An important next step is to generate a 3D flowfield by means of computational fluid dynamics (CFD). A hard requirement for us was that AutoGDM needed to be free to use, so we chose to use the open-source CFD tool OpenFOAM. It’s used for cutting-edge aerodynamics research, and also the tool suggested by the authors of GADEN. Usually, using OpenFOAM isn’t trivial, as a large number of parameters need to be selected that require field expertise, resulting in a complicated process. Next, we integrate GADEN into our pipeline, to go from environment definition (CAD files) and a flow field to a gas concentration field. Other parts that needed to be automated were the random selection of boundary conditions, which has a large impact on the actual flow field, and source placement, which has an equally large impact on the concentration field.
After we built this pipeline, we started looking for a robot simulator to couple it to. Since we weren’t planning on using a camera, our main requirement was for the simulator to be efficient (preferably in 2D) so that evolutions would take relatively little time. We decided to use Swarmulator [6], a lightweight C++ robot simulator designed for swarming and we plugged in our gas data.
Algorithm Design
Roughly speaking, we considered two categories of algorithms for controlling the drones: 1) a neural network, and 2) an FSM that included PSO, with evolved parameters. Since we used a tiny neural network for light seeking with a CrazyFlie in our previous work, we first evolved neural networks in simulation. One of the first experiments is shown below.
A single agent in simulation seeking a light source using a tiny neural network.
While it worked pretty well in simple environments with few obstacles, it seemed challenging to make this work in real life with complex obstacles and multiple agents that need to collaborate. Given the time constraints of the project, we have opted for evolving the FSM. This also facilitated crossing the reality gap, as the simulated evolution could build on basic behaviors that we developed and validated on the real platform, including obstacle avoidance with four tiny laser rangers, while communicating with and avoiding other drones. An additional advantage of PSO with respect to the reality gap is that it only needs gas concentration and no gradient of the gas concentration or wind direction (which many algorithms in literature use). On a real robot at this scale, estimating the gas concentration gradient or the direction of a light breeze is hard if not impossible.
Hardware
Our CrazyFlie needs to be able to avoid obstacles, execute velocity commands, sense gas, and estimate the other agent’s position in its own frame. For navigation, we added the flow deck and laser rangers, whereas for gas sensing we used a TGS8100 gas sensor that was used on a CrazyFlie before in previous work [7]. The sensor is lightweight and inexpensive, but accurately estimating gas concentrations can be difficult because of its size. It tends to drift and needs time to recover after a spike in concentration is observed. Another thing we noticed is that it is possible to break them, a crash can definitely destroy the sensor.
To estimate the relative position between agents, we use a Decawave Ultra-Wideband (UWB) module and communicate states, as proposed in [8]. We also use the UWB module to communicate gas information between agents and collaboratively seek the source. The complete configuration is visible below.
A 37.5 g nano quadcopter, capable of fully autonomous waypoint tracking, obstacle avoidance, relative localization, communication and gas sensing.
Evaluation in Simulation
After we optimized the parameters of our model using Swarmulator and AutoGDM, and of course trying many different versions of our algorithm, we ended up with the final Sniffy Bug algorithm. Below is a video that shows evolved Sniffy Bug evaluated in six different environments. The red dots are an agent’s personal target waypoint, whereas the yellow dot is the best-known position for the swarm.
Sniffy Bug evaluated in Swarmulator environments.
Simulation showed that Sniffy Bug is effective at locating the gas source in randomly generated environments. The drones successfully collaborate by means of PSO.
Real Flight Testing
After observing Sniffy Bug in simulation we were optimistic, but unsure about performance in real life. First, inspired by previous works, we disperse alcohol through the air by placing liquid alcohol into a can which is then dispersed using a computer fan.
Dispersion of liquid alcohol in flight tests.
We test Sniffy Bug in our flight arena of size 10 x 10 meters with large obstacles that are shaped like walls and orange poles. The image below shows four flight tests of Sniffy Bug in cluttered environments, flying fully autonomously, i.e., without the help from any external infrastructure.
Time-lapse images of real-world experiments in our flight arena. Sniffy was evaluated on four distinct environments, 10 x 10 meters in size, seeking a real isopropyl alcohol source. The trajectories of the nano quadcopters are clearly visible due to their blue lights.
In the total of 24 runs we executed, we compared Sniffy Bug with manually selected and evolved parameters. The figure below shows that the evolved parameters are more efficient in locating the source as compared to the manual parameters.
Maximum recorded gas reading by the swarm, for each time step for each run.
This does not only show that our system can successfully locate a gas source in challenging environments, but it also demonstrates the usefulness of the simulation pipeline. The parameters that were learned in simulation yield a high-performance model, validating the environment generation, randomization, and gas modeling parts of our pipeline.
Conclusion and Discussion
With this work, we believe we have made an important step towards swarms of gas-seeking drones. The proposed solution is shown to work in real flight tests with obstacles, and without any external systems to help in localization or communication. We believe this methodology can be extended to larger environments or even to 3 dimensions, since PSO is a robust, multi-dimensional heuristic search method. Moreover, we hope that AutoGDM will help the community to better compare gas seeking algorithms, and to more easily learn parameters or models in simulation, and deploy them in the real world.
To improve Sniffy Bug’s performance, adding more laser rangers will definitely help. When working with only four laser rangers you realize how little information it actually provides. If one of the rangers senses a low value it is unclear if a slim pole or a massive wall is detected, adding inefficiency to the algorithm. Adding more laser rangers or using other sensor modalities like vision will help to avoid also more complex obstacles than walls and poles in a reliable manner.
Another interesting discussion can be held on the hardware required for real deployment. When working with 40 grams of maximum take-off weight, the sensors and actuators that can be selected are limited. For example, the low-power and lightweight flow deck works great but fails in low-light scenarios or with smoke. Future work exploring novel sensors for highly constrained nano robots could really help increase the Technological Readiness Level (TRL) of these systems.
Finally, this has been a really fun project to work on for us and we can’t wait to hear your thoughts on Sniffy Bug!
If you haven’t visited our store in a while, you may have missed our new addition: the Lighthouse Swarm bundle!
We’ve been working for some time now on improving the Lighthouse decks and its positioning system. Earlier in the year, we have brought the Lighthouse deck out of early access. While working with it, we have seen the great possibilities and the accuracy of this new positioning system. Thanks to Steam’s VR base station that we use as an optical beacon, the Crazyflie calculates its position with an accuracy better than a decimeter and millimeter precision. It gives a tracking volume of up to 5x5x2 meters with sub-millimetre jitter and below 10 cm accuracy while flying. It’s perfect for a swarm, as it’s accurate, precise and autonomous. We’ve flown our Crazyflies with it a number of time and seen some awesome stuff with it!
As an example, here is a demo we’ve shown on a conference back in October. We’ve used 8 Crazyflies equipped with Lighthouse decks and Qi chargers, to make a spiraling swarm. A computer orchestrates the Crazyflies and make sure one is flying at all times, while the others re-charge their batteries on their pads. After a pre-programmed trajectory is finished or when the battery of the flying Crazyflie is depleted, it goes back to its pad while another one takes over. The demo had an all-in mode that runs the trajectory on all Crazyflie with sufficient charge at once, the result is quite impressive and demonstrate the great relative precision of the Lighthouse system:
After the launch signal is sent to the Crazyflies, the computer is not required anymore: the Crazyflie will autonomously estimate its position from the lighthouse’s signals. The Crazyflie can estimate its own X, Y and Z in a global coordinate system.
What’s great with the Lighthouse Swarm is that it allows you to do drone research even if you’re on a tighter budget.
And when we got the opportunity to acquire our own base stations (that are also available in the shop by the way), it seemed only logical to offer a Swarm bundle similar to our Loco swarm bundle. So what’s in it ?
While the positioning will work with one base station, two base stations will allow better coverage of the flight space and better stability; as Kimberly can attest, it’s even possible to set it in your kitchen. The Crazyradios allow communication between the Crazyflies and your computer.
We dedicated a lot of time to the Lighthouse this winter, writing a paper with the help of Wolgangs’ calibration expertise. In this paper, we compared both Lighthouse V1 and V2 with the MoCap system. In all cases, the mean and median Euclidean error of the Lighthouse positioning system are about 2-4 centimeters compared to our MoCap system as ground truth. You can check the paper here, but here is a brief summary we used for our ICRA workshop:
The poster presenting our paper
We are now quite excited to get to see what you will do with this exciting new swarm bundle !
And if you don’t know how to set up the Swarm, you can get started at least with your Lighthouse system in this tutorial or watch Kristoffer explain it in this video:
