Translating an esp32+esp-idf bluetooth example to rust

A rusty heart beats away inside this thinking rock

TLDR

Research

Bluetooth

Recent efforts in communicating with the esp32 over bluetooth have demonstrated the core functionality of implementing bluetooth:

Demo Project

During investigation the Apache MyNewt project came up. It's an IoT specific operating system with a focus on wireless protocols. It provides the NimBLE Bluetooth Low Energy stack which was much simpler to configure than the previously used bluedroid stack.

The blehr project is a demo of the bare minimum required to present a readable characteristic to a bluetooth device,

Rust on ESP

There are several very useful repos demonstrating rust on the esp32. Specifically the esp32-hello project has been forked several times

lexxvir was one of the first
mattcaron made some excellent cargo.toml improvements to update it to ESP-IDF 4.0
reitermarkus is using it as a testbed to implement rust std for esp-idf
sdobz my fork of mattcarons tweaked to work on nix

The esp32-bluetooth started as a copy of my esp32-hello fork

Tooling Improvements

Rust Analyzer on nix

Several steps were required to get rust-analyzer working for a custom fork of rust.

First the nix expression for building rust-analyzer was overridden with the previously developed rustc-xtensa

rust-analyzer.nix

{ pkgs, rustPlatform }:

rec {
  rust-analyzer-unwrapped = pkgs.callPackage (pkgs.path + /pkgs/development/tools/rust/rust-analyzer/generic.nix) rec {
    inherit rustPlatform;
    rev = "2020-05-11";
    version = "unstable-${rev}";
    sha256 = "07sm3kqqva2jw41hb3smv3h3czf8f5m3rsrmb633psb1rgbsvmii";
    cargoSha256 = "1x1nkaf10lfa9xhkvk2rsq7865d9waxw0i4bg5kwq8kz7n9bqm90";
  };

  rust-analyzer = pkgs.callPackage (pkgs.path + /pkgs/development/tools/rust/rust-analyzer/wrapper.nix) {
    inherit rustPlatform;
  } {
    unwrapped = rust-analyzer-unwrapped;
  };
}

Next vscode has to be stood up with the custom rust-analyzer plugin. The nix vscode plugin essentially hardcodes the path to the rust-analyzer binary, here we just substitute the path to ours.

home.nix

  vscode.extensions = with pkgs.vscode-extensions; [
    (unstable.vscode-extensions.matklad.rust-analyzer.override {
      rust-analyzer = rust-esp.rust-analyzer;
    })
  ];

VSCode rust-analyzer plugin uses cargo check internally to produce the issues seen in the editor. Since I am using a fork of rust it was failing to find the core module.

./.vscode/settings.json

  "rust-analyzer.checkOnSave.overrideCommand": [
    "${workspaceFolder}/rust-analyzer-check.sh"
  ],

./rust-analyzer-check.sh

#!/usr/bin/env bash

DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"

export RUSTFLAGS=--sysroot=$DIR/target/sysroot
cargo check --message-format=json

The RUSTFLAGS environment variable forces cargo to use our custom sysroot, enabling it to find core.

Manually implementing macros

Bindgen does a very good job translating c function calls over to rust, but the macro support is lacking. There is work being done in rust-bindgen#753 but for now we have to manually implement the xTimerStart, xTimerStop, and xTimerReset macros as well as the uuid functions

Mechanical Work

Translating mostly consisted of pasting blocks of C into the rust source code and eliminating syntax errors.

The C -> Rust transformation is fairly straightforward, the most difficult aspects being determining how to write the type of function pointers. The effort was arduous but fairly straightforward. After a couple days of grinding on it the code finally compiled

An interesting macro was developed to construct null terminated c strings from rust strings at compile time:

#[macro_export]
macro_rules! cstr {
    ($src:expr) => {
        (concat!($src, "\0")).as_ptr() as *const _
    };
}

...

fn main() {
  printf(cstr!("int is: %d\n"), 15);
}

This probably only makes sense in static contexts or something, it'll bite me when it bites me.

Compiled but broken

Hunting down missing symbols

Many times I ended up pulling keywords from the air and hunting them down, the letters btdm kept appearing and repeated searching ended up pointing to the esp32-bt-lib repo.

build/bt/libbt.a(bt.o):(.literal.btdm_sleep_enter_phase2_wrapper+0x0): undefined reference to `btdm_controller_get_sleep_mode'
build/bt/libbt.a(bt.o):(.literal.btdm_sleep_exit_phase3_wrapper+0x4): undefined reference to `btdm_rf_bb_init_phase2'
build/bt/libbt.a(bt.o):(.literal.esp_vhci_host_check_send_available+0x0): undefined reference to `API_vhci_host_check_send_available'

This was because I had not enabled the NimBLE stack in menuconfig so the make process was not building the library files. Once sdkconfig was updated the files were hypothetically emitted, but I could not find them.

A command like the below used find to locate lib files, then the nm tool to list out symbols defined in the files, and finally grep to find the specific missing symbol:

$ find . -regex ".*\.[a]" | xargs nm -A | grep get_sleep_mode
./components/bt/controller/lib/libbtdm_app.a:arch_main.o:000010d4 T btdm_controller_get_sleep_mode

.cargo/config

  "-C", "link-arg=-L../esp-idf/components/bt/controller/lib", "-C", "link-arg=-lbtdm_app",

Fixing menuconfig

Menuconfig was giving me grief

$ make menuconfig
/nix/store/3b3ighb83nhifa1v4n7855hlbdl1mhf9-binutils-2.31.1/bin/ld: lxdialog/menubox.o: in function `print_arrows.constprop.0':
menubox.c:(.text+0x419): undefined reference to `wrefresh'
collect2: error: ld returned 1 exit status
make[1]: *** [Makefile:331: mconf-idf] Error 1
make[1]: Leaving directory '/home/vkhougaz/projects/hanging-plotter/esp32/esp-idf/tools/kconfig'
make: *** No rule to make target '/home/vkhougaz/projects/hanging-plotter/esp32/esp-idf/tools/kconfig/conf-idf', needed by '/home/vkhougaz/projects/hanging-plotter/esp32/esp-idf/tools/kconfig/mconf-idf'.  Stop.

Searching around revealed this might be related to missing ncurses which is solvable

default.nix

   shellHook = ''
     ...
     export NIX_CFLAGS_LINK=-lncurses
  '';

esp-idf.nix

  propagatedBuildInputs = [
    ...
    pkgs.ncurses
  ];

Parsing backtraces

The idf.py monitor tool included some smarts that took backtraces and showed files and line numbers.

console.sh

#!/usr/bin/env bash

DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"

ELF=$(find ./build -maxdepth 1 -type f -regex ".*\.elf")

: ${TTY:=/dev/ttyUSB0}

if [ ! -f $ELF ]; then
    echo "Could not find ELF"
    exit 1
fi

print_addrs() {
    local s=$1 regex=$2
    echo "===== $0 ====="
    while [[ $s =~ $regex ]]; do
        xtensa-esp32-elf-addr2line -pfiaC -e "$ELF" ${BASH_REMATCH[1]}
        s=${s#*"${BASH_REMATCH[1]}"}
    done
    echo "=============="
}

while read line
do
  echo $line;
  if [[ $line =~ "Backtrace:" ]] ; then
    print_addrs "$line" '(0x[a-f0-9]{8}):'
  fi
done < <(miniterm.py --raw $TTY 115200)

Static allocation

The initial attempts to rewrite the gatt_svr_svcs definition ended up taking the majority of the time on this effort.

It turns out convincing rust to allocate objects on the heap and then leave them alone is quite difficult! My first attempts looked similar to

#[macro_export]
macro_rules! mut_ptr {
    ($t:expr) => {
        $crate::UnsafeCellWrapper($crate::UnsafeCell::new($t))
    };
}

static GATT_SVR_SVCS: ble_gatt_svc_def_ptr = ble_gatt_svc_def_ptr::new(
    [
        ble_gatt_svc_def {
            type_: BLE_GATT_SVC_TYPE_PRIMARY as u8,
            uuid: ble_uuid16_declare!(GATT_HRS_UUID),
            includes: ptr::null_mut(),
            characteristics: [
                ble_gatt_chr_def {
                    uuid: ble_uuid16_declare!(GATT_HRS_MEASUREMENT_UUID),
                    access_cb: Some(gatt_svr_chr_access_heart_rate),
                    arg: mut_ptr!(ptr::null_mut()),
                    descriptors: mut_ptr!(ptr::null_mut()),
                    flags: BLE_GATT_CHR_F_NOTIFY as u16,
                    min_key_size: 0,
                    val_handle: mut_ptr!(unsafe { &mut HRS_HRM_HANDLE as *mut u16 }),
                },
                null_ble_gatt_chr_def(),
            ]
            .as_ptr(),
        },

Unfortunately no matter how I shuffled the code the val_handle pointer resulted in an error similar to:

cannot borrow a constant which may contain interior mutability, create a static instead

rust-lang#69908 seems to be blocking creating a static struct with a ref to a mutable static pointer

Some lessons learned:

Rust statics behave similarly to C but really hate being mutable
Rust consts are more similar to C macros and may not ever have a memory addressed assigned, the compiler might optimize them away

Rust ownership and debug code

The construct

static variable: *const struct_t = [
  struct_t {
    ... fields
  }
].as_ptr();

passes the type checker, but unfortunately doesn't make sense at runtime. It turns out that as_ptr does not retain ownership of the values and releases the memory to be reused.

Oodles of very unsafe custom debug printers assisted in discovering that any code similar to the above immediately got overwritten by fresh memory on the stack

The most interesting is this function:

pub unsafe fn print_ptr<T>(name: *const u8, p: *const T) {
    printf(cstr!("%p - %s:\n"), p, name);
    print_bytes(p as *const _, size_of::<T>());
    printf(cstr!("\n"));
}

pub unsafe fn print_bytes(bytes: *const u8, len: usize) {
    let u8p: &[u8];

    u8p = core::slice::from_raw_parts(bytes, len);

    for i in 0..len {
        if (i & 0b1111) == 0 && i > 0 {
            printf(cstr!("\n"));
        } else if (i & 0b1) == 0 {
            printf(cstr!(" "));
        }
        printf(cstr!("%02x"), u8p[i] as u32);
    }
}

which is used to print the name, location, and any memory associated with a raw pointer.

Doing it the easy (but complicated) way

After fighting static allocation I ended up allocating memory at runtime using the esp-idf-alloc crate from ctron and intentionally leaking it with the leaky box macro.

macro_rules! leaky_box {
    ($($items:expr),+) => (
        Box::into_raw(
            Box::new(
                [
                    $($items),*
                ]
            )
        ) as *const _
    )
}

here is the final construction of the service definition.

Stack Overflow!

There is a timer service that updates the pseudo heartrate every second.

***ERROR*** A stack overflow in task Tmr Svc has been detected.
abort() was called at PC 0x4008e784 on core 0

ELF file SHA256: 0000000000000000000000000000000000000000000000000000000000000000

Backtrace: 0x4008e3f5:0x3ffbcc30 0x4008e76d:0x3ffbcc50 0x4008e784:0x3ffbcc70 0x400911cc:0x3ffbcc90 0x400930d8:0x3ffbccb0 0x4009308e:0x00000003 |<-CORRUPTED
===== ./console.sh =====
0x4008e3f5: ?? ??:0
0x4008e76d: ?? ??:0
0x4008e784: ?? ??:0
0x400911cc: ?? ??:0
0x400930d8: ?? ??:0
0x4009308e: ?? ??:0
==============

I wasn't able to find any support in english, but a snippet from a content rehoster pushed me towards a useful direction

已经解决了，主要是代码默认时钟任务栈分配小了。

解决办法：Component config --->FreeRTOS ---> (2560) Timer stack size

默认是 2048, 2M 已经不小了，看来这个代码开销好大啊

Oh right, of course? Timer stack size was fairly easy to locate in menuconfig, bumping it to 4096 ended up adequate

Results

After a week of trials and tribulations we can finally witness embedded rust emitting radiation at a phone and causing pixels to shift around:

app displaying bluetooth heartrate service — You can't tell here, but the 100 bpm changes every second