Getting started with the AI deck

Introduction

The AI deck  enables WiFi communication with the Crazyflie as well as using the power-efficient GAP8 to run neural networks on board. This tutorial will prepare your AI deck  to be used together with the examples in our GAP8 example repository and for you to start developing your own applications.

Prerequisites

Make sure that you have gone through the Getting started with the Crazyflie 2.X tutorial first.

To run this tutorial and set up the AI deck you will need the following:

Preparing the AI deck and Crazyflie firmware

The first thing to do is to make sure that your firmware is up to date, this includes both the AI deck firmware (ESP32 and the GAP8) as well as the Crazyflie (STM32 and nRF51)

Update Crazyflie and AIdeck firmware

  1. Open up the cfclient on your computer, and make sure that it is version 2022.05+ (‘Help’->’About’)
  2. Make sure that only the AI-deck is attached to the Crazyflie, with no other deck.
  3. Go to ‘Connect’->’bootloader’
  4. Type the address of your crazyflie, press ‘Scan’ and select your crazyflie’s URI. Make sure to choose ‘radio://…’ (not ‘usb://’). Now press ‘Connect’
  5. In the ‘Firmware Source’ section, select 2022.05 from ‘Available downloads’. Make sure to select the right platform (cf2 is for the crazyflie 2.x ).
  6. Press ‘Program’ and wait for the STM, NRF and ESP MCUs the have been reflashed. The crazyflie will restart a couple of times, and the flashing of the ESP (‘bcAI:esp deck memory’) takes about 3 minutes.
  7. Once the status states ‘Idle’ and the Crazyflie is disconnected, double check if the flashing has succeded. In the cfclient, connect to the crazyflie and check in the console tab if you see: ESP32: I (910) SYS: Initialized. Also LED1 should be flashing with 2 hz.
Flashing

NOTE: If you don’t see ESP32: I (910) SYS: Initialized, try go through the steps again.

Gap8 bootloader (for older AIdecks revisions A-C)

If you do not already have a bootloader on the GAP8, which would be the case if your AIdeck 1.1 has an older revision(Rev A, B and C), you will need to flash the bootloader on the GAP8 seperately. This can only be done from a native linux computer or virtual machine (not WSL) and a jtag enabled programmer (Olimex ARM-USB-TINY-H JTAG or Jlink).

You only need to do this once and then you can enjoy the benefits of overair flashing.

Clone, build and flash the bootloader with an Olimex ARM-USB-TINY-H JTAG or a Jlink using the following commands:

$ git clone https://github.com/bitcraze/aideck-gap8-bootloader.git
$ cd aideck-gap8-bootloader
$ docker run --rm -it -v $PWD:/module/ --device /dev/ttyUSB0 --privileged -P bitcraze/aideck /bin/bash -c 'export GAPY_OPENOCD_CABLE=interface/ftdi/olimex-arm-usb-tiny-h.cfg; source /gap_sdk/configs/ai_deck.sh; cd /module/;  make all image flash'

Check out the aideck flashing documentation for more detailed instructions.

Once you see the following it means you were successfull

--------------------------
flasher is done!
--------------------------
--------------------------
Reset CONFREG to 0
--------------------------
GAP8 examine target
RESET: jtag boot mode=3
DEPRECATED! use 'adapter [de]assert' not 'jtag_reset'

NOTE: if you see Error: Burst read failed or something similar, make sure that the 10 pin lint cable is properly attached. The programmer is not able to connect to the aideck properly.

If flashing hangs at Initialising GAP8 JTAG TAP, you probably have to disconnect and reconnect the programmer and restart the Crazyflie.

Flash Wifi Example

  1. Go to the aideck example releases page and download aideck_gap8_wifi_img_streamer_with_ap.bin from the latest release.
  2. With a crazyradio, replace [CRAZYFLIE_URI] with your crazyflie URI in the same form as radio://0/80/2M/E7E7E7E7E7 and type the following in your terminal:
cfloader flash aideck_gap8_wifi_img_streamer_with_ap.bin deck-bcAI:gap8-fw -w [CRAZYFLIE_URI]

Then you will see:

Reset to bootloader mode ...
Could not save cache, no writable directory
Could not save cache, no writable directory
Skipping bcAI:esp, not in the target list
Deck bcAI:gap8, reset to bootloader
| 0% Writing to bcAI:gap8 deck memory
/ 1% Writing to bcAI:gap8 deck memory
...
\ 99% Writing to bcAI:gap8 deck memory
| 100% Writing to bcAI:gap8 deck memory

This only should take 10 seconds for the wifi example.

NOTE: If the over air flashing hangs, try to reflash the firmware and ESP again and in case you flashed the gap8 bootloader, also flash that again with a programmer.

In your wifi list you should see ‘WiFi streaming example’. If you do, please connect to it.

Clone the AIdeck example repository and select this WiFi and run this script.

Then run:

python opencv-viewer.py

to see the following:

wide

Note: This viewer needs opencv-python which you can install with pip install opencv-python. Be aware that the cfclient has a conflict with this library (see this issue), so please install it in a seperate environment or remember to deinstall this library if you want to use the cfclient again.

Setup development environment

For development of the GAP8 chip, you need to have an environment with the Gap SDK installed. To make things easier, we have made a Docker container which should have everything (almost) everything configured.

In order to pull in the GAP8 SDK docker be able to use the autotiler in the GAP8 SDK (Facedetection and Classifcation examples) you will have to manually set it up and accept the license.

Setting up docker and the autotiler

$ docker run --rm -it --name myAiDeckContainer bitcraze/aideck
$ cd /gap_sdk
$ source configs/ai_deck.sh
$ make autotiler

Follow the instructions of the autotiler pull script. Press enter immediately at Enter URL from email: , fill in your information, wait for the email with the URL. Once you receive the email, fill the URL at the current Enter URL from email: , read the licence and accept if it is all fine to you.

In a second separate terminal on your local machine, commit the changes to a new image by running:

$ docker commit myAiDeckContainer aideck-with-autotiler

This will save the state of the container with the installed autotiler to a new image called aideck-with-autotiler that you will use later.

You can now go back to the first terminal and close the container

$ exit

Remember that this needs to be done every time you pull a new image of the bitcraze/aideck docker image

Next steps

Your AI deck  is now prepared and ready for either your own applications or to try out some of our examples. To continue the tutorial, clone the aideck-gap-example repository with

$ git clone https://github.com/bitcraze/aideck-gap8-examples.git

and jump over to one of the following examples:

Further reading