test-pbio.sh compiler errors

[Novakasa]

Hi all,
I am struggling with running the pbio tests. I am on Windows 10 x64 and have the environment set up using cygwin. On top of the requirements listed in CONTRIBUTING.md, I also installed gcc-core (gcc version 11.2.0 (GCC)) and gdb, since otherwise I don't have a C compiler to compile the tests for PC.

When trying to run sh test-pbio.sh in powershell on current master, I get the following output:

PS E:\repos\pybricks-micropython> sh .\test-pbio.sh
CC ../../libfixmath/libfixmath/fix16_str.c
CC ../drv/bluetooth/bluetooth_btstack.c
CC ../drv/bluetooth/bluetooth_btstack_control_gpio.c
CC ../drv/bluetooth/bluetooth_btstack_run_loop_contiki.c
CC ../drv/bluetooth/bluetooth_btstack_uart_block_stm32_hal.c
In file included from ../../libfixmath/libfixmath/fix16_str.c:4:
../../libfixmath/libfixmath/fix16_str.c: In function 'fix16_from_str':
../../libfixmath/libfixmath/fix16_str.c:74:20: error: array subscript has type 'char' [-Werror=char-subscripts]
   74 |     while (isspace(*buf))
      |                    ^~~~
../../libfixmath/libfixmath/fix16_str.c:85:20: error: array subscript has type 'char' [-Werror=char-subscripts]
   85 |     while (isdigit(*buf))
      |                    ^~~~
../../libfixmath/libfixmath/fix16_str.c:105:24: error: array subscript has type 'char' [-Werror=char-subscripts]
  105 |         while (isdigit(*buf) && scale < 100000)
      |                        ^~~~
../../libfixmath/libfixmath/fix16_str.c:118:22: error: array subscript has type 'char' [-Werror=char-subscripts]
  118 |         if (!isdigit(*buf) && !isspace(*buf))
      |                      ^~~~
../../libfixmath/libfixmath/fix16_str.c:118:40: error: array subscript has type 'char' [-Werror=char-subscripts]
  118 |         if (!isdigit(*buf) && !isspace(*buf))
      |                                        ^~~~
CC ../drv/bluetooth/bluetooth_init_cc2564C_1.4.c
CC ../drv/bluetooth/bluetooth_stm32_bluenrg.c
cc1: all warnings being treated as errors
make: *** [Makefile:177: build/lib/pbio/test/../../libfixmath/libfixmath/fix16_str.o] Error 1
make: *** Waiting for unfinished jobs....
CC ../drv/bluetooth/bluetooth_stm32_cc2640.c

Just running make -C lib/pbio/test gives me the same errors.

Did I miss something when running the tests? I have noticed the file was most recently touched by @dlech specifically for replacing the isdigit and isspace functions, but I am assuming you are somehow running the tests without these errors appearing. Maybe in my environment the correct preprocessor flags aren't set, but I am not sure why.

[dlech]

We work primarily on Ubuntu 20.04, so probably have a different GCC version (9.3) and so don't see this error yet.

You could research the warning and fix it upstream (libfixmath) or we can disable that warning in the Makefile (something like CFLAGS += -no-warn-char-subscripts).

[Novakasa]

Also, I am seeing this message (sometimes?):

CC ../drv/bluetooth/bluetooth_btstack_run_loop_contiki.c
CC ../drv/bluetooth/bluetooth_btstack_uart_block_stm32_hal.c

[!] PyCryptodome required to calculate GATT Database Hash but not installed (using random value instead)
[!] Please install PyCryptodome, e.g. 'pip3 install pycryptodomex' or 'pip3 install pycryptodome'


BLE configuration generator for use with BTstack
Copyright 2018 BlueKitchen GmbH

Importing /cygdrive/e/repos/pybricks-micropython/lib/btstack/src/ble/gatt-service/nordic_spp_service.gatt
Created build/genhdr/pybricks_service.h
Compilation successful!

CC ../drv/bluetooth/bluetooth_init_cc2564C_1.4.c
CC ../drv/bluetooth/bluetooth_stm32_bluenrg.c
CC ../drv/bluetooth/bluetooth_stm32_cc2640.c

So it does seem to create the pybricks_service.h successfully here, but complains about the pycryptodome package.

Running pip3 install pycryptodomex I get:

 pip3 install pycryptodomex
Requirement already satisfied: pycryptodomex in e:\repos\pybricks-micropython\.venv\lib\site-packages (3.12.0)

I installed pycryptodome, but the message is still showing up and the linker errors are still here.

[Novakasa]

Here's the full log:

PS E:\repos\pybricks-micropython> sh .\test-pbio.sh
../drv/bluetooth/bluetooth_btstack.c:23:10: fatal error: genhdr/pybricks_service.h: No such file or directory
   23 | #include "genhdr/pybricks_service.h"
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
CC ../../../micropython/lib/tinytest/tinytest.c
CC ../../contiki-core/lib/list.c
CC ../../contiki-core/lib/memb.c
CC ../../contiki-core/sys/autostart.c
CC ../../contiki-core/sys/etimer.c
CC ../../contiki-core/sys/process.c
CC ../../contiki-core/sys/timer.c
CC ../../libfixmath/libfixmath/fix16.c
CC ../../libfixmath/libfixmath/fix16_exp.c
CC ../../libfixmath/libfixmath/fix16_sqrt.c
CC ../../libfixmath/libfixmath/fix16_trig.c
CC ../../libfixmath/libfixmath/fix16_str.c
CC ../../libfixmath/libfixmath/fract32.c
CC ../../libfixmath/libfixmath/uint32.c
CC ../../lwrb/src/lwrb/lwrb.c
CC ../../btstack/src/btstack_linked_list.c
CC ../../btstack/src/btstack_run_loop.c
CC ../../btstack/src/btstack_util.c
CC ../../btstack/src/ad_parser.c
CC ../../btstack/src/btstack_audio.c
CC ../../btstack/src/btstack_base64_decoder.c
CC ../../btstack/src/btstack_crypto.c
CC ../../btstack/src/btstack_hid_parser.c
CC ../../btstack/src/btstack_memory.c
CC ../../btstack/src/btstack_memory_pool.c
CC ../../btstack/src/btstack_ring_buffer.c
CC ../../btstack/src/btstack_slip.c
CC ../../btstack/src/btstack_tlv.c
CC ../../btstack/src/hci.c
CC ../../btstack/src/hci_cmd.c
CC ../../btstack/src/hci_dump.c
CC ../../btstack/src/hci_transport_em9304_spi.c
CC ../../btstack/src/hci_transport_h5.c
CC ../../btstack/src/hci_transport_h4.c
CC ../../btstack/src/l2cap.c
CC ../../btstack/src/l2cap_signaling.c
CC ../../btstack/src/ble/gatt-service/device_information_service_server.c
CC ../../btstack/src/ble/gatt-service/nordic_spp_service_server.c
CC ../../btstack/src/ble/att_db_util.c
CC ../../btstack/src/ble/att_db.c
CC ../../btstack/src/ble/att_dispatch.c
CC ../../btstack/src/ble/gatt_client.c
CC ../../btstack/src/ble/att_server.c
CC ../../btstack/src/ble/le_device_db_memory.c
CC ../../btstack/src/ble/sm.c
CC ../../btstack/chipset/cc256x/btstack_chipset_cc256x.c
CC ../../btstack/platform/posix/hci_dump_posix_stdout.c
CC ../drv/adc/adc_stm32f0.c
CC ../drv/adc/adc_stm32_hal.c
CC ../drv/battery/battery_adc.c
CC ../drv/battery/battery_linux_ev3.c
CC ../drv/battery/battery_nxt.c
CC ../drv/battery/battery_virtual.c
CC ../drv/bluetooth/bluetooth_btstack_control_gpio.c
CC ../drv/bluetooth/bluetooth_btstack_run_loop_contiki.c
CC ../drv/bluetooth/bluetooth_btstack_uart_block_stm32_hal.c

[!] PyCryptodome required to calculate GATT Database Hash but not installed (using random value instead)
[!] Please install PyCryptodome, e.g. 'pip3 install pycryptodomex' or 'pip3 install pycryptodome'


BLE configuration generator for use with BTstack
Copyright 2018 BlueKitchen GmbH

Importing /cygdrive/e/repos/pybricks-micropython/lib/btstack/src/ble/gatt-service/nordic_spp_service.gatt
Created build/genhdr/pybricks_service.h
Compilation successful!

CC ../drv/bluetooth/bluetooth_init_cc2564C_1.4.c
CC ../drv/bluetooth/bluetooth_stm32_bluenrg.c
CC ../drv/bluetooth/bluetooth_stm32_cc2640.c
CC ../drv/bluetooth/pybricks_service_server.c
CC ../drv/button/button_gpio.c
CC ../drv/button/button_linux_ev3.c
CC ../drv/button/button_nxt.c
CC ../drv/button/button_resistor_ladder.c
CC ../drv/button/button_virtual.c
CC ../drv/charger/charger_mp2639a.c
CC ../drv/city_hub/motor.c
CC ../drv/clock/clock_linux.c
CC ../drv/clock/clock_nxt.c
CC ../drv/clock/clock_stm32.c
CC ../drv/core.c
CC ../drv/counter/counter_core.c
CC ../drv/counter/counter_ev3dev_stretch_iio.c
CC ../drv/counter/counter_nxt.c
CC ../drv/counter/counter_stm32f0_gpio_quad_enc.c
CC ../drv/debug/motor.c
CC ../drv/essential_hub/motor.c
CC ../drv/gpio/gpio_stm32f0.c
CC ../drv/gpio/gpio_stm32f4.c
CC ../drv/gpio/gpio_stm32l4.c
CC ../drv/ioport/ioport_lpf2.c
CC ../drv/led/led_array_pwm.c
CC ../drv/led/led_array.c
CC ../drv/led/led_core.c
CC ../drv/led/led_pwm.c
CC ../drv/led/led_dual.c
CC ../drv/led/led_virtual.c
CC ../drv/move_hub/motor.c
CC ../drv/nxt/motor.c
CC ../drv/prime_hub/motor.c
CC ../drv/pwm/pwm_core.c
CC ../drv/pwm/pwm_lp50xx_stm32.c
CC ../drv/pwm/pwm_stm32_tim.c
CC ../drv/pwm/pwm_tlc5955_stm32.c
CC ../drv/reset/reset_nxt.c
CC ../drv/reset/reset_stm32.c
CC ../drv/resistor_ladder/resistor_ladder.c
CC ../drv/sound/sound_stm32_hal_dac.c
CC ../drv/technic_hub/motor.c
CC ../drv/uart/uart_stm32f0.c
CC ../drv/uart/uart_stm32f4_ll_irq.c
CC ../drv/uart/uart_stm32l4_ll_dma.c
CC ../drv/usb/usb_stm32.c
CC ../drv/usb/usb_stm32_serial.c
CC ../drv/virtual.c
CC ../src/battery.c
CC ../drv/watchdog/watchdog_stm32.c
CC ../src/color/conversion.c
CC ../src/config.c
CC ../src/control.c
CC ../src/dcmotor.c
CC ../src/drivebase.c
CC ../src/integrator.c
CC ../src/iodev.c
CC ../src/error.c
CC ../src/light/animation.c
CC ../src/light/color_light.c
CC ../src/light/light_matrix.c
CC ../src/logger.c
CC ../src/main.c
CC ../src/math.c
CC ../src/motor_process.c
CC ../src/observer.c
CC ../src/parent.c
CC ../src/protocol/lwp3.c
CC ../src/protocol/nus.c
CC ../src/protocol/pybricks.c
CC ../src/servo.c
CC ../src/tacho.c
CC ../src/task.c
CC ../src/trajectory.c
CC ../src/trajectory_ext.c
CC ../src/uartdev.c
CC ../src/util.c
CC ../sys/battery.c
CC ../sys/bluetooth.c
CC ../sys/command.c
CC ../sys/hmi.c
CC ../sys/io_ports.c
CC ../sys/light.c
CC ../sys/light_matrix.c
CC ../sys/main.c
CC ../sys/status.c
CC ../sys/user_program.c
CC ../sys/supervisor.c
CC ../platform/motors/settings.c
CC clock.c
CC drv/battery.c
CC drv/bluetooth.c
CC drv/button.c
CC drv/counter.c
CC drv/pwm.c
CC src/animation.c
CC src/color.c
CC src/color_light.c
CC src/light_matrix.c
CC src/math.c
CC src/servo.c
CC src/task.c
CC src/uartdev.c
CC src/util.c
CC sys/bluetooth.c
CC sys/status.c
CC test-pbio.c
CC ../drv/bluetooth/bluetooth_btstack.c
/usr/lib/gcc/x86_64-pc-cygwin/9.3.0/../../../../x86_64-pc-cygwin/bin/ld: build/lib/pbio/test/../drv/bluetooth/bluetooth_btstack_control_gpio.o:bluetooth_btstack_control_gpio.c:(.rdata$.refptr.pbdrv_bluetooth_btstack_control_gpio_platform_data[.refptr.pbdrv_bluetooth_btstack_control_gpio_platform_data]+0x0): undefined reference to `pbdrv_bluetooth_btstack_control_gpio_platform_data'
/usr/lib/gcc/x86_64-pc-cygwin/9.3.0/../../../../x86_64-pc-cygwin/bin/ld: build/lib/pbio/test/../sys/main.o: in function `pbsys_main':
/cygdrive/e/repos/pybricks-micropython/lib/pbio/test/../sys/main.c:53: undefined reference to `pbdrv_reset_power_off'
/usr/lib/gcc/x86_64-pc-cygwin/9.3.0/../../../../x86_64-pc-cygwin/bin/ld: build/lib/pbio/test/../sys/battery.o: in function `pbsys_battery_init':
/cygdrive/e/repos/pybricks-micropython/lib/pbio/test/../sys/battery.c:49: undefined reference to `pbdrv_battery_get_type'
collect2: error: ld returned 1 exit status
make: *** [Makefile:182: build/test-pbio] Error 1

[Novakasa]

Looking at the linker error one more time, it complains for example about missing reference to pbdrv_bluetooth_btstack_control_gpio_platform_data. But It seems to be defined in lib\pbio\drv\bluetooth\bluetooth_btstack_control_gpio.h. The corresponding .c file seems to be compiled according to the log.

[dlech]

I don't understand why the linker errors aren't showing up outside of Cygwin, but there were problems. I've pushed the fixes to https://github.com/pybricks/pybricks-micropython/tree/dlech and will merge once CI passes.

[Novakasa]

36 tests ok.  (0 skipped)

Thank you!

[dlech]

Back to the original issue of the libfixmath compiler error, https://stackoverflow.com/a/10186479/1976323 has a better explanation, so fixing upstream is the way to go.

[Novakasa]

submitted the pull request for it: PetteriAimonen/libfixmath#36 (comment)