Lighthouse V2 status update

2020-04-20 | Kristoffer Richardsson | 3 Comments

The Lighthouse V2 implementation has been simmering away for a long time in the Bitcraze kitchen and in this blog post we will give you an update on the current status and what is remaining for a full release of this tasty dish.

Crazyflie 2.1 and Lighthouse V2 base station

We believe we have solved most of the major technical hurdles (last famous words) on the way to a working implementation that uses Lighthouse V2 base stations for positioning, now it is mostly work to implement the functionality that is remaining. As described in this post we now have a new FPGA binary that has the ability to decode both V1 and V2 base stations, and this was a major step forward. This new binary is used in the Crazyflie firmware master branch, and if the Lighthouse deck is used with the latest Crazyflie firmware, the new FPGA binary will automatically be flashed to the deck.

What has changed?

The new FPGA binary uses a different UART protocol to communicate with the Crazyflie. This protocol has been implemented in the firmware and hopefully there is no functional difference compared to the previous FPGA binary when using Lighthouse V1 base stations.

We have added a first version of Lighthouse V2 base station decoding, but it is still a bit limited. As a start we decided to “emulate” V1 base stations to be able to reuse as much of the existing code as possible. For now we support only 2 base stations and they must use channel 1 and 2 (used to be called modes). The V2 angles are transformed into V1 angles and fed into the old positioning logic and are handled exactly the same way as before. Even though this works, it is not the optimal solution and we hope to be able to refine it later on.

We have also written a python script to estimate base station geometry (positions and orientation) using the Lighthouse deck. This removed the requirement to use software from Steam which should simplify the set up process. Please see the (still limited) documentation. Note that this calibration method only supports the basestation V1… for now!

There is a lot of code that has been modified and the FPGA implementation is completely new, it is not unlikely that there is functionality that is unstable or broken, or configurations that are not supported. If you happen to notice any bugs, please let us know!

What is remaining?

The functional areas that needs to be implemented or cleaned up before we leave the Early Access stage is the following:

Calibration data

The calibration data is embedded in the modulated light from the base stations and describes imperfections from the manufacturing process for each individual. This data is not read yet for V2 and will increase the precision when available.

Support for more than 2 base stations

Lighthouse V2 base stations are designed for systems with more than 2 base stations. The Crazyflie firmware needs to be extended for this functionality to work, including handling of geometry data, logging, memory management and some other bits and pieces.

Native V2 positioning

The angles from the V2 base station should be fed directly into the kalman filter for positioning, instead of first being transformed into V1 angles. This will increase robustness and reduce data loss.

Client support

We want to add a tab in the python client where a Lighthouse system can be monitored, configured and managed. It should, for instance enable the user to configure and visualize base station geometry.

FPGA binaray management

Currently the FPGA binary is included in the Crazyflie firmware and it is automatically uploaded to the deck when booted. This is not a viable long term solution and we hope to be able to find a more generic way of handling deck binaries.

Conclusions

As can be seen, there is still quite some work to be done before the Lighthouse V2 stew is ready to be served, but we are definitely starting to smell some nice flavours from the kitchen!

Finally a view from Kristoffer’s home lab, currently in the summer house. Three base stations are set up as a Fun Friday hack to see what it would take to use more than 2. Luckily it did not take too much time to get this to work :-)

3 Lighthouse V2 base stations

Running Projects and Status Update

2020-04-14 | Kimberly McGuire | 2 Comments

In this blog-post we wanted to give you guys an overview of our running projects and a general update of the status of things! We got settled in our home-labs and are working on many projects in parallel. There are a lot of development happening at the moment, but the general feeling is that we do miss working with each other at our office! With our daily slack Bitcraze sync meetings and virtual fikapause (Swedish for coffee breaks), we try to substitute what we can. In the mean time, we are going on a roll with finishing all our goals we have set at our latest quarterly meeting, so here you can read about those developments.

AI-deck

The last time we gave an update about the AI-deck was in this blog post and in the final post of our intern Zhouxin. Building on his work, we are now refocusing on getting the AI-deck ready for early release. The last hurdle is mostly software wise on which we are considering several approaches together with the manufacturer of the Gap8 chip Greenwaves technologies. Currently we are preparing small testing functions as examples of the different elements of the AI-deck in our repo, which are all still in a very primarily phase.

Even though we still need some time to finalize the AI-deck’s early release, we will consider sending an early version of the AI-deck if you are willing to provide feedback while working with it. Please fill in the form and we will get back to you.

Lighthouse

We have made quite some progress on the development for the lighthouse V2. Kristoffer has been working hard from his homelab to get a seamless integration of both V1 and V2 in our firmware (check out this github issue for updates). Currently it is still very untested and very much in progress, however we do have a little preview for you to enjoy.

Crazyflie with LH basestation v2

Documentation

Right now, we are also doing a lot of revamping of the large web of documentation. Unfortunately this is a lot of work! As you noticed by now, we have added overview pages to guide the reader to the right information. We also have moved the tutorials to another part of the menu to avoid clutter on our website. In general we try to go through the repository docs to see if there is any information missing or outdated, however please let us know if you have encountered an error in any description or are missing crucial elements.

Our latest task is revamping the product pages as well, by putting all the necessary information about the hardware in just one place. Also, we are planning to make (video) tutorials soon about many elements of the Crazyflie and how to work with it. More about that later!

Production and Shipment

Production at our manufacturers in China are slowly starting up again. Although it is not yet back at full force, it does enable us to already start ordering to replenish our stock and to get started with finishing our test rigs. Moreover, we are also negotiating to resolve the propeller issue we mentioned earlier, but there is no update on that so far.

As mentioned in this blogpost, we are still shipping orders about twice a week. Both DHL and Fedex are functioning as normal, but we do notice that there is a delay of a few extra days on some deliveries. Please keep that in mind when ordering at our webshop.

The Home Lab

2020-04-06 | Kimberly McGuire | Leave a comment

Many people in the world have now settled in the reality of working from home. We have also taken precautions ourselves by not go to our office as normal and only ship out packages a few times per week instead of every day (see this blogpost). This also means that we do not have full access to our lab with all our equipment and positioning systems in our big 10 x 10 meter flight lab at the office. In this blogpost we will show how we manage to keep on developing and flying, even in the current situation.

Crazyflie flying in a kitchen with the lighthouse deck

In(light)house positioning

Currently we started to use the Lighthouse positioning system to setup up the remote home lab at our houses. As of recent additions to the Crazyflie firmware, it has been made easy to get the geometry data from the base station. Now the only items we need for indoor flight are just two (or only one) lighthouse basestations V1’s and a Crazyflie, and that is it! There is no need for an HTC Vive headset or hub, or third-party software like SteamVR and the setup is finished in 2 minutes! Check out the new documentation here if you want to know more about the new setup of the lighthouse positioning system.

Also, we recently got a very primarily version of the lighthouse V2 working (see here) and we of course want to keep the momentum going! We will be working on full compatibility from our homes so stay tuned. For now, see this video of the Crazyflie flying with just a single base-station, taken from one of our team-member’s home lab.

Remote Lecture Hall and Practicals

We were invited by Dario Floreano and Fabrizio Schiano from the EPFL-LIS laboratory to do a lecture for the ‘Aerial Robotics’ Course as part of EPFL’s Master’s program in Robotics. Due to the virus, we had to cancel our trip to go there physically… but luckily we were able to do the lecture remotely anyway!

The lecture consists of two parts. In the first hour we mostly explained about the Crazyflie ecosystem, hardware and sensors. In the second hour we focused on how the stabilization module worked, including the controllers and the state estimation. During both sessions, we alternated between the theory slides with actual hands-on demos. The lighthouse positioning system was setup in a kitchens, so that we were able to show full flights and practicals with the Crazyflie. At the end there was also the push-demo with just the flowdeck and multiranger, which didn’t use any external positioning at all.

The lectures can be found below and the documentation has been updated as well with the covered material (see here). Be sure to check out the controller tuning presented in part 2 of the lecture (25:00 – Cascaded PID controller).

Other Home labs

We know that there are currently users that are moving their flight lab from their university or company to their homes to be able to continue their work. We would love to hear about your experience and your home lab! Send us an email with your story to contact@bitcraze.io, drop us a message on forum.bitcraze.io, or mention us in your Twitter, Linkedin, Facebook or Reddit post. Also, if you want to setup your own home lab and you need any advice or help, please let us know!

Drawing with the Crazyflie

2020-03-02 | Kristoffer Richardsson | Leave a comment

There has been some work done earlier to use the Crazyflie for generating images, for instance the dot-drawing by Paul Kry and light painting. I wanted to see if it is possible to put a brush or pen on a Crazyflie and use it to draw lines on a paper. I decided to use a fun Friday to try it out. The idea is simple: mount a pen on the Crazyflie, put a paper on a wall, write a script to draw a figure, fly!

The first thing I looked into was to investigate if a Crazyflie can fly with a brush or pen mounted on it. I wanted to keep the weight down and my initial approach was to use a cotton swab (0.6 g) dipped in paint. I found one that was long enough to extend in front of the propellers and I mounted it by squeezing it between the battery and the PCB. Flying was no problem with such low extra weight.

For positioning I decided to use the Lighthouse system. It is very accurate, simple to use and the easiest way to get started. I mounted a piece of cardboard in the YZ-plane of our lighthouse coordinate system, where I could attach a drawing paper. The idea of setting up the drawing surface parallell to the YZ-plane was to make the scripting easier. I (of course) used the Crazyflie and lighthouse system to measure that the cardboard was mounted at the right position.

Finally I wrote a simple python script that utilized the high level commander to move towards the drawing surface and yawing at the right position to draw a stroke on the paper. It sort of worked, but the cotton swab has to be “refilled” before each stroke which took a lot of time, and the results were a bit random.

I decided to try out a pen instead. The upside is that it does not require refill, on the other hand it is much heavier which makes the Crazyflie a bit sluggish when flying. I mounted the pen on the top side of the PCB, squeezed under the Lighthouse deck, and moved the battery to under the Crazyflie to distribute the weight.

Initial tests – both cotton swab and pen

The script was updated to draw the outline of the Bitcraze logo. I had a couple of variations where I tried to draw the full outline in one long stroke, as separate strokes, going up or going down and some other flavours.

So was it successful? Currently the Crazyflie is not a new Picasso, but the painting skills could maybe be improved with some more work. I think the main problems were:

the pen is too heavy and requires too much force on the paper
the controller cannot handle the situation in a good way. In essence I set the set point a few millimeters “into” the paper to push the pen against the surface which seems to be confusing as the controller can not reach the set point.
Flying that close to the drawing surface creates an air flow that disturbs the flight.

Video showing the Crazyflie drawing the logo

The Bitcraze logo (17×17 cm), drawn by the Crazyflie

Crazyflie flying with Lighthouse V2!

2020-02-24 | Arnaud | 5 Comments

We are happy to announce that we have gotten Crazyflie 2 to fly autonomously using the Lighthouse deck and Lighthouse V2 base-stations. This was a very requested features, and while this is not stable and ready to use yet, it is a great milestone toward Lighthouse V2 support.

There exists two incompatible versions of the Lighthouse positioning system. Version 1 was released with the original HTC Vive VR system. In this system base-station are using two rotating laser beam that sweeps the room, one horizontal and one vertical, and an omnidirectional synchronization flash to allow IR light receiver to be located in the room. One limitation of this version is that up to two base-station can be used and no more, this is mainly due to the fact that beam identification is done using a TDMA scheme: base stations switch-on their laser in a dedicated time-slot one after each-other and adding more time slots for more base-stations will greatly reduce the update rate of the system.

Lighthouse V2, was released with the HTC Vive PRO headset and is also used by the Valve Index. The big change is that laser sweeps now carries modulated data and that there is only one rotor with two angled slit instead of the two rotors for V1. The V2 sweep data is described as ‘Sync on beam’ and contains timing information of how long it has been since the synchronization event (ie. when the rotor crossed 0 degree). The sweep data also allows to identify the base-station that has transmitted the sweep. This removes the need for an omni-directional synchronization pulse and allows more than two base-station to operate at the same time in the same space, since their sweeps can now be identified and timed.

The lighthouse V2 system is very elegant and scalable. However, actually decoding the signal from the sweeps has taken a lot of time since it is not documented and we needed to find-out what the encoding actually was. There has been effort on the internet to understand how the system worked, the most useful one is this github ticket that goes from raw data acquisition to fully unlocking the beam encoding.

I have been working on-and-off for a long time on making an FPGA design for the lighthouse deck to acquire and decode Lighthouse V2. The main blocking point until now was that I had not been able to reliably acquire useful signal from the system in order to allow real-time decoding on the Crazyflie. Added to that, there was some inconsistency between what we though the system was doing and what we could gather from the base-stations debug console. Recently though, the last piece of the puzzle, was to discover that the beam encoding was not Manchester, as we though, but Bi-phase mark code FM1 (BMC). Once this decoding was used everything made sense and worked.

Added to that, I started using SpinalHDL instead of raw Verilog to write the FPGA design which allows for much quicker iteration, much less frustration, and it also allowed me to easily make the design multi-clock which is required to decode the BMC signal: the beam decoder runs at 48MHz, and the rest of the system works at 24MHz. This design is required since the FPGA we use in the lighthouse deck is not fast enough to run everything at 48MHz.

The result, is a new FPGA firmware for the lighthouse deck that receives, identify and decode Lighthouse V2 sweep signal and send them over to the Crazyflie. The Crazyflie still has a little pulse packing to do (putting together pulses from a single sweep received on multiple sensors) and then can use pulse timing information to calculate azimuth and elevation at which the base-station sees the Crazyflie. This information is the same as the one we get from Lighthouse V1 and so the same algorithm can be used to calculate the Crazyflie position.

I hacked a proof of concept was this last fun Friday and it flies!

If anyone is curious the code for this demo has been uploaded as an out-of-tree driver and the code for the FPGA parts is already in the lighthouse-fpga project. The current Crazyflie code is too incomplete to be usable, but it is a nice starting point if anyone wants to play with Lighthouse V2 and the Crazyflie right away ;-).

As a side note, the Bitcraze team will shrink temporary as I, Arnaud, will go in parental leave until mid-August. I look forward to this new adventure and I trust the lighthouse V2 development and the forum will be in good hands in my absence.

Happy Holidays!

2019-12-23 | Kimberly McGuire | 1 Comment

Happy holidays to all our users, community members and friends! We are happy to announce our 2019 Christmas video which we have made in collaboration with Ben Kuper! It is starring 7 Crazyflies, the lighthouse positioning system, our office Christmas tree and a whole lot of holiday spirit, so go ahead and take a look!

Here are some words from Ben how it was to work on this year’s Christmas video at our office:

Coming to Bitcraze’s HQ and working with them has been once more a wonderful experience, technically and humanly ! The main goal of this session was to test and implement the new lighthouse tracking system in the tool suite I’m creating, and it was an amazing surprise to witness for real the uncanny stability of the drones when they’re on lighthouse tracking !

Of course, my first reaction was to push the limit and see what can be done with this new power, this is why I created this choreography : to see what can be done in a limited amount of time (1 and a half day to create the full choreography, the official video shows the first part only), and trying to go at the limit of the current possibilities. As the team was working on occlusion recovery, we decided to have the drone fly around the tree as a fun test, and it works !

In the new year we will have a followup blog-post going into detail on how exactly we made this video. Until then, happy holidays and have an awesome new year!

IROS and Lighthouse yaw

2019-11-04 | Kristoffer Richardsson | Leave a comment

This week we are exhibiting at IROS in Macau. We are running our fully autonomous demo based on the Lighthouse positioning technology and charging pads. We also have brought some prototypes to show, for instance the Crazyflie Bolt, the AI deck and the Active marker deck. You can read more about the demo at the IROS 2019 page.

We’d love to hear what you are working on, discuss issues, possibilities or new products. If you are at IROS, drop by our booth (B34) and say hi!

Lighthouse yaw

We have not only prepared for IROS, we have also been working on improving the lighthouse positioning system. Recently we added a (slightly hackish) solution for updating the yaw with data from the Lighthouse deck. This means that it is not necessary to start the Crazyflie facing the positive X direction when using the Lighthouse deck. The Crazyflie will understand its heading and act accordingly.

Two Crazyflies facing a random direction, take off and rotate to yaw=0.

We are also working on integrating the Lighthouse deck in a better way in the kalman filter. If everything goes according to plan, it will enable a Crazyflie to fly with only one base station, and be more robust when using two base stations.

Excited about IMAV 2019

2019-09-23 | Kimberly McGuire | Leave a comment

The Crazyflie Bolt and the Crazyflie 2.1 with the lighthouse deck are coming to Madrid!

Only one week away until the start of the big Bitcraze Conference frenzy, with the first stop… Madrid! We will visit the Micro Air Vehicle Competition and Conference, which is a robotics event that is more specialized in (as the name implies) MAVs! So it should be right up our alley. This is the first time that we attend as Bitcraze, although the writer of this blog post has experienced fun times at the conference and the competition as a participant with her previous lab, the MAVlab.

The IMAV has been around for almost 12 years, starting in Toulouse, France in 2007. Although it initially mostly was held in various places in Europe, in 2016 into a more worldwide phenomenon by making it’s tribute in Beijing, China and Melbourne, Australia in 2018. It hosts a conference to which researchers can send their work in anything related to MAVs, from autonomous navigation, state estimation and design.

IMAV is mostly know for hosting big indoor and outdoor competitions for MAVs. The outdoor competitions can range from survey tasks to finding a hidden person or object. This year the focus will be on the delivery of packages from one place to another. The judges will look at how many packages that can be safely delivered and if the drone is able to detect certain objects in the outdoor environment. The indoor competition is oriented around the application of MAVs in a warehouse. They should be able to take off autonomously, monitor boxes in shelves and make an innovatory, and pickup packages to release them in their designated location. 40 teams of 28 universities will show their awesome implementation in these difficult tasks.

We will have a booth at the main company fair at the conference and indoor competition, and will also be present at the outdoor competition day as well. We will bring the lighthouse positioning system and show the awesome swarming demo we developed. Also we want to bring the new Crazyflie Bolt with us, which we are sure of that the regular IMAV crew will love. If you are at the IMAV between the 30th of September to the 4th of October, come by and say hello!

Creating trajectories for the High-level Commander with Bezier curves

2019-08-05 | Kristoffer Richardsson | Leave a comment

The High-level Commander has been part of the Crazyflie firmware since the 2018.10 release. In combination with a positioning system, it can fly the Crazyflie along a trajectory that is either defined in the firmware or uploaded through the python lib. It originates from the Crazyswarm project and we have used it in various demos since it is possible to make trajectories that are very fluid and looks really cool. The trajectories are defined as 7th degree polynomials describing segments executed one after each other.

The controller gives full control of position, velocity, acceleration and jerk, the only problem is that it is non-trivial to generate the polynomials. We have wanted to simplify the creation of trajectories for a long time and have finally had some time to play with it. In this blog post we will describe how it can be done with Bezier curves and show some examples.

Each segment in a High-level commander trajectory is defined by four 7 degree polynomials, one for x, y, z and yaw. There is also a scaling parameter that tells the controller the time scale to use when executing the segment. Using polynomials of degree 7 makes it possible to design trajectories that are continuous in position, velocity, acceleration and jerk when changing from one segment to the next, which is important to get a smooth and controlled flight.

Bezier curves are common in many graphics applications and are probably known to most users. They are parametric curves defined by control points, usually three or four. Bezier curves can also be expressed as a polynomial, and this is what we will use in this case. To get a correct mapping to the desired polynomials we need some more control points and will use 8 per segment. The basic idea is to define the trajectory as Bezier curves and make sure the control points are placed in such a way that the continuity requirements are satisfied.

On this page from University of Cambridge, there is a good explanation on continuity across the joins between curves and formulas for c0, c1 and c2 continuity. We also need c3 continuity which can be calculated in the same manner

With these formulas it is possible to set the handles of the Bezier curves to make sure we get a smooth ride.

We have added a python example that implements the ideas above. You can find it in crazyflie-lib-python/examples/positioning/bezier_trajectory.py. The design is based on Nodes that represents the connection points between bezier curves (called Segments). The Nodes has a set of handles that are shared between the Segments that use the Node. If not all handles are set the implementation will set them to appropriate values, see the comments in the code for more details. The Node API only allows the user to set handles on one of the Segments, the handles for the other segment are automatically set to generate a continuous trajectory.

The example uses nodes in the corners of a square and contains three parts:

No velocity in the nodes. The Crazyflie stops in the Nodes. Similar to calling go-to in the HL commander.

Velocity in the nodes. A fluid motion all the way around.

A bit more aggressive settings to get a little action.

Finally a video showing the full sequence, we use the Lighthouse for positioning.

Synchronous swarm example

2019-07-08 | Arnaud | Leave a comment

Two weeks ago we posted about the demo we did for our new office move-in party. There has been multiple requests to share the script but unfortunately this is a hacked old script that is not going to be useful at all as an example. So, last week, we made an example that could run a synchronized swarm sequence.

The example has been pushed in the example folder of the Crazyflie-lib-python project. It is called synchronizedSequence.py. Running this example unmodified with 3 Crazyflies in a positioning system will give you this result. (Like the previous demo, this was done in a lighthouse system.)

One of the key design of the example is that it is based on a single control loop that can be synchronized with an outside system: in this example, there is a simple sleep of one seconds between each step of the sequence but it could for example be changed into a midi clock receiver to synchronize the sequence with music.

The example was developed with the help of Victor, a student we have hired to help-out during the summer. He has then played around a little bit to make a 9 Crazyflies sequence that is more impressive:

I uploaded Victor’s sequence in a github gist as it can be good for inspiration. One bit of warning though: as is, the sequence contains some vertical movements that are quite aggressive and the part where Crazyflies fly directly on top of each-other is more to be considered as a stress test.