Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make for ESP32 fails on nanopb build #105

Open
BartRijvers11 opened this issue Jan 24, 2022 · 22 comments
Open

Make for ESP32 fails on nanopb build #105

BartRijvers11 opened this issue Jan 24, 2022 · 22 comments

Comments

@BartRijvers11
Copy link

BartRijvers11 commented Jan 24, 2022

I came accross your github and became enthousiastic ! I want a dedicated embedded spotify player for my home office
I have a fresh install of the IDF toolchain on my Windows 10 machine and I updated the GIT repo including submodules. I do get stuck building the nanopb library though. I tried many things for the last couple of days, unfortunately unsuccessful but maybe it's a simple solution I have overlooked.
I have ordered a board from AliExpress which is in transit right now and I like to be able to compile the software before I actually receive the board.

Loggings:

C:\Users\bartr\Documents\Arduino\cspot>git submodule update --init --recursive

C:\Users\bartr\Documents\Arduino\cspot>cd targets/esp32

C:\Users\bartr\Documents\Arduino\cspot\targets\esp32>idf.py build
Executing action: all (aliases: build)
Running cmake in directory c:\users\bartr\documents\arduino\cspot\targets\esp32\build
Executing "cmake -G Ninja -DPYTHON_DEPS_CHECKED=1 -DESP_PLATFORM=1 -DIDF_TARGET=esp32 -DCCACHE_ENABLE=1 c:\users\bartr\documents\arduino\cspot\targets\esp32"...
-- Found Git: C:/Users/bartr/esp32/.espressif/tools/idf-git/2.30.1/cmd/git.exe (found version "2.30.1.windows.1")
-- ccache will be used for faster recompilation
-- The C compiler identification is GNU 8.4.0
-- The CXX compiler identification is GNU 8.4.0
-- The ASM compiler identification is GNU
-- Found assembler: C:/Users/bartr/esp32/.espressif/tools/xtensa-esp32-elf/esp-2020r3-8.4.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-gcc.exe
-- Check for working C compiler: C:/Users/bartr/esp32/.espressif/tools/xtensa-esp32-elf/esp-2020r3-8.4.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-gcc.exe
-- Check for working C compiler: C:/Users/bartr/esp32/.espressif/tools/xtensa-esp32-elf/esp-2020r3-8.4.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-gcc.exe -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: C:/Users/bartr/esp32/.espressif/tools/xtensa-esp32-elf/esp-2020r3-8.4.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-g++.exe
-- Check for working CXX compiler: C:/Users/bartr/esp32/.espressif/tools/xtensa-esp32-elf/esp-2020r3-8.4.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-g++.exe -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Project version: e9f1d4e-dirty
-- Building ESP-IDF components for target esp32
-- Project sdkconfig file C:/Users/bartr/Documents/Arduino/cspot/targets/esp32/sdkconfig
Loading defaults file C:/Users/bartr/Documents/Arduino/cspot/targets/esp32/sdkconfig.defaults...
-- Found PythonInterp: C:/Users/bartr/esp32/.espressif/python_env/idf4.1_py3.8_env/Scripts/python.exe (found version "3.8.7")
-- Could NOT find Perl (missing: PERL_EXECUTABLE)
-- Adding linker script C:/Users/bartr/Documents/Arduino/cspot/targets/esp32/build/esp-idf/esp32/esp32_out.ld
-- Adding linker script C:/Users/bartr/esp32/esp-idf/components/esp32/ld/esp32.extram.bss.ld
-- Adding linker script C:/Users/bartr/esp32/esp-idf/components/esp32/ld/esp32.project.ld.in
-- Adding linker script C:/Users/bartr/esp32/esp-idf/components/esp32/ld/esp32.peripherals.ld
-- Adding linker script C:/Users/bartr/esp32/esp-idf/components/esp_rom/esp32/ld/esp32.rom.ld
-- Adding linker script C:/Users/bartr/esp32/esp-idf/components/esp_rom/esp32/ld/esp32.rom.libgcc.ld
-- Adding linker script C:/Users/bartr/esp32/esp-idf/components/esp_rom/esp32/ld/esp32.rom.newlib-data.ld
-- Adding linker script C:/Users/bartr/esp32/esp-idf/components/esp_rom/esp32/ld/esp32.rom.syscalls.ld
-- Found Nanopb: C:/Users/bartr/Documents/Arduino/cspot/cspot/bell/nanopb
CMake Error at C:/Users/bartr/Documents/Arduino/cspot/cspot/bell/nanopb/extra/FindNanopb.cmake:263 (string):
  string sub-command REGEX, mode MATCH needs at least 5 arguments total to
  command.
Call Stack (most recent call first):
  C:/Users/bartr/Documents/Arduino/cspot/cspot/CMakeLists.txt:31 (nanopb_generate_cpp)


CMake Error at C:/Users/bartr/Documents/Arduino/cspot/cspot/bell/nanopb/extra/FindNanopb.cmake:269 (message):
  Your path includes a ':' character used as an option separator for nanopb.
  Upgrade to protoc version >= 3.6.0 or use a different path.
Call Stack (most recent call first):
  C:/Users/bartr/Documents/Arduino/cspot/cspot/CMakeLists.txt:31 (nanopb_generate_cpp)


-- Configuring incomplete, errors occurred!
See also "C:/Users/bartr/Documents/Arduino/cspot/targets/esp32/build/CMakeFiles/CMakeOutput.log".
cmake failed with exit code 1

C:\Users\bartr\Documents\Arduino\cspot\targets\esp32>cmake --version
cmake version 3.13.4

CMake suite maintained and supported by Kitware (kitware.com/cmake).

C:\Users\bartr\Documents\Arduino\cspot\targets\esp32>idf.py --version
ESP-IDF v4.1.2

C:\Users\bartr>python -V
Python 3.9.10```
@BartRijvers11 BartRijvers11 changed the title Make for ESP fails on nanopb build Make for ESP32 fails on nanopb build Jan 24, 2022
@BartRijvers11
Copy link
Author

I found a regex bug in the FindNanopb.cmake line 263: the last variable needs quotes to continue building (to the next error).
I installed Ubuntu in Virtualbox in the meantime.

@HeikoGr
Copy link
Contributor

HeikoGr commented Jan 25, 2022

you don't need to install a virtualbox machine. WSL2 could help you too.

https://github.com/HeikoGr/euphonium/wiki/using-Windows-Subsystem-for-Linux-(WSL)-for-compilation

this should work for cspot alone as well...

BUT: you can't flash via WSL2 without porblems. so maybe virtualbox is still a good alternative...

@BartRijvers11
Copy link
Author

Thanks ! Since I have worked with ubuntu in the past, it would be one less problem to worry about. I installed the last LTS version just tonight:

bart@bart-VirtualBox:~/cspot/targets/esp32$ uname -a
Linux bart-VirtualBox 5.13.0-27-generic #29~20.04.1-Ubuntu SMP Fri Jan 14 00:32:30 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
bart@bart-VirtualBox:~/cspot/targets/esp32$ protoc --version
libprotoc 3.6.1
bart@bart-VirtualBox:~/cspot/targets/esp32$ idf.py --version
ESP-IDF v5.0-dev-1340-g0027483e93
bart@bart-VirtualBox:~/cspot/targets/esp32$ python --version
Python 3.8.10

Patched locally:

bart@bart-VirtualBox:~/cspot/cspot/bell/nanopb$ git diff
diff --git a/extra/FindNanopb.cmake b/extra/FindNanopb.cmake
index 206f5e6..144326e 100644
--- a/extra/FindNanopb.cmake
+++ b/extra/FindNanopb.cmake
@@ -260,7 +260,7 @@ function(NANOPB_GENERATE_CPP SRCS HDRS)
     # - Or a newer one, using --nanopb_opt which requires a version of protoc >= 3.6
     # So we will determine which version of protoc we have available and choose accordingly.
     execute_process(COMMAND ${PROTOBUF_PROTOC_EXECUTABLE} --version OUTPUT_VARIABLE PROTOC_VERSION_STRING OUTPUT_STRIP_TRAILING_WHITESPACE)
-    string(REGEX MATCH "[(0-9)].*.[(0-9)].*.[(0-9)].*" PROTOC_VERSION ${PROTOC_VERSION_STRING})
+    string(REGEX MATCH "[(0-9)].*.[(0-9)].*.[(0-9)].*" PROTOC_VERSION "${PROTOC_VERSION_STRING}")
 
     if(PROTOC_VERSION VERSION_LESS "3.6.0")
         #try to use the older way

Now I get as far as building the bootloader but get stuck at the protoc version:

[93/95] Linking C executable bootloader.elf
[94/95] Generating binary image from built executable
esptool.py v3.3-dev
Creating esp32 image...
Merged 1 ELF section
Successfully created esp32 image.
Generated /home/bart/cspot/targets/esp32/build/bootloader/bootloader.bin
[95/95] cd /home/bart/cspot/targets/esp32/build/bootloader/esp-idf/esptool_py && /home/bart/.espressif/python_env/idf5.0_py3.8_env/bin/python /home/bart/esp/esp-idf/components/partition_table/check_sizes.py --offset 0x8000 bootloader 0x1000 /home/bart/cspot/targets/esp32/build/bootloader/bootloader.bin
Bootloader binary size 0x6a40 bytes. 0x5c0 bytes (5%) free.
[591/1292] Running C++ protocol buffer compiler using nanopb plugin on /home/bart/cspot/cspot/protobuf/authentication.proto
FAILED: esp-idf/main/cspot/protobuf/authentication.pb.c esp-idf/main/cspot/protobuf/authentication.pb.h 
cd /home/bart/cspot/targets/esp32/build/esp-idf/main/cspot && /usr/bin/protoc -I/home/bart/cspot/targets/esp32/build/esp-idf/main/cspot/nanopb/generator -I/home/bart/cspot/targets/esp32/build/esp-idf/main/cspot/nanopb/generator/proto -I/home/bart/cspot/targets/esp32/build/esp-idf/main/cspot -I/home/bart/cspot/cspot --plugin=protoc-gen-nanopb=/home/bart/cspot/targets/esp32/build/esp-idf/main/cspot/nanopb/generator/protoc-gen-nanopb "--nanopb_opt= -I/home/bart/cspot/cspot/protobuf -I/home/bart/cspot/cspot" --nanopb_out=/home/bart/cspot/targets/esp32/build/esp-idf/main/cspot /home/bart/cspot/cspot/protobuf/authentication.proto
[libprotobuf WARNING google/protobuf/compiler/parser.cc:562] No syntax specified for the proto file: protobuf/authentication.proto. Please use 'syntax = "proto2";' or 'syntax = "proto3";' to specify a syntax version. (Defaulted to proto2 syntax.)
Traceback (most recent call last):
  File "/home/bart/cspot/targets/esp32/build/esp-idf/main/cspot/nanopb/generator/nanopb_generator.py", line 51, in <module>
    from .proto import nanopb_pb2
ImportError: attempted relative import with no known parent package

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/bart/cspot/targets/esp32/build/esp-idf/main/cspot/nanopb/generator/nanopb_generator.py", line 72, in <module>
    import proto.nanopb_pb2 as nanopb_pb2
  File "/home/bart/cspot/targets/esp32/build/esp-idf/main/cspot/nanopb/generator/proto/nanopb_pb2.py", line 10, in <module>
    from google.protobuf import symbol_database as _symbol_database
ImportError: cannot import name 'symbol_database' from 'google.protobuf' (/home/bart/.espressif/python_env/idf5.0_py3.8_env/lib/python3.8/site-packages/google/protobuf/__init__.py)
--nanopb_out: protoc-gen-nanopb: Plugin failed with status code 1.
[592/1292] Linking C static library esp-idf/esp_http_server/libesp_http_server.a
ninja: build stopped: subcommand failed.
ninja failed with exit code 1

I wonder why it's so hard to just get started... but as long as I feel that Iḿ making progress I'm willing to play pitpull, bite until I figure it out. Google only provides answers of similar issues reported in 2016 and should be solved by now... Stuck again but that's it for me tonight. Hope someone has some ideas to get me going ?

@HeikoGr
Copy link
Contributor

HeikoGr commented Jan 25, 2022

Did you install the dependencies?

sudo pip3 install protobuf grpcio-tools

maybe you also need to install protobuf-compiler. I can’t test it now.

sudo apt install protobuf-compiler

@BartRijvers11
Copy link
Author

BartRijvers11 commented Jan 26, 2022

Thanks, Yes, I installed all dependencies mentioned in the main readme.md file:
Requirement already satisfied: protobuf in /usr/lib/python3/dist-packages (3.6.1)
Requirement already satisfied: grpcio-tools in /usr/local/lib/python3.8/dist-packages (1.43.0)
Requirement already satisfied: setuptools in /usr/lib/python3/dist-packages (from grpcio-tools) (45.2.0)
Requirement already satisfied: grpcio>=1.43.0 in /usr/local/lib/python3.8/dist-packages (from grpcio-tools) (1.43.0)
Requirement already satisfied: six>=1.5.2 in /usr/lib/python3/dist-packages (from grpcio>=1.43.0->grpcio-tools) (1.14.0)

protobuf-compiler is already the newest version (3.6.1.3-2ubuntu5).

If I put the following line at the top of authentication.proto file:

syntax = "proto2";

(I still don't know why it doesn't work out of the box, am I the only one having these kind of issues ?)
the build gives me another error:

[591/1292] Running C++ protocol buffer compiler using nanopb plugin on /home/bart/cspot/cspot/protobuf/authentication.proto
FAILED: esp-idf/main/cspot/protobuf/authentication.pb.c esp-idf/main/cspot/protobuf/authentication.pb.h 
cd /home/bart/cspot/targets/esp32/build/esp-idf/main/cspot && /usr/bin/protoc -I/home/bart/cspot/targets/esp32/build/esp-idf/main/cspot/nanopb/generator -I/home/bart/cspot/targets/esp32/build/esp-idf/main/cspot/nanopb/generator/proto -I/home/bart/cspot/targets/esp32/build/esp-idf/main/cspot -I/home/bart/cspot/cspot --plugin=protoc-gen-nanopb=/home/bart/cspot/targets/esp32/build/esp-idf/main/cspot/nanopb/generator/protoc-gen-nanopb "--nanopb_opt= -I/home/bart/cspot/cspot/protobuf -I/home/bart/cspot/cspot" --nanopb_out=/home/bart/cspot/targets/esp32/build/esp-idf/main/cspot /home/bart/cspot/cspot/protobuf/authentication.proto
Traceback (most recent call last):
  File "/home/bart/cspot/targets/esp32/build/esp-idf/main/cspot/nanopb/generator/nanopb_generator.py", line 51, in <module>
    from .proto import nanopb_pb2
ImportError: attempted relative import with no known parent package

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/bart/cspot/targets/esp32/build/esp-idf/main/cspot/nanopb/generator/nanopb_generator.py", line 72, in <module>
    import proto.nanopb_pb2 as nanopb_pb2
  File "/home/bart/cspot/targets/esp32/build/esp-idf/main/cspot/nanopb/generator/proto/nanopb_pb2.py", line 10, in <module>
    from google.protobuf import symbol_database as _symbol_database
ImportError: cannot import name 'symbol_database' from 'google.protobuf' (/home/bart/.espressif/python_env/idf5.0_py3.8_env/lib/python3.8/site-packages/google/protobuf/__init__.py)
--nanopb_out: protoc-gen-nanopb: Plugin failed with status code 1.
[592/1292] Linking C static library esp-idf/esp_http_server/libesp_http_server.a
ninja: build stopped: subcommand failed.
ninja failed with exit code 1

@HeikoGr
Copy link
Contributor

HeikoGr commented Jan 26, 2022

what version of esp-idf do you use?
the (working) github action https://github.com/feelfreelinux/cspot/blob/master/.github/workflows/esp-idf.yml uses version 4.4

@anabolyc
Copy link
Contributor

anabolyc commented Jan 26, 2022

Please make sure you've installed all the dependencies in the esp-idf virtual environment, not globally. Nornally it is located in the ~/.espressif/python_env/idf4.4_py_someting folder

@BartRijvers11
Copy link
Author

Hello people, I am up to date with https://github.com/espressif/esp-idf.git and have remove the build directory before building.
I have run the export shell command in de esp32 directory before installing stuff in the same terminal, I'm not sure how I can install the dependencies in the folder @anabolyc mentions or how I can check that, maybe this is sufficient info:

~/.espressif/python_env/idf5.0_py3.8_env/lib/python3.8/site-packages$ ls
alldefconfig.py                               contextlib2                            idna                         pygdbmi                                 semantic_version-2.8.5.dist-info
allmodconfig.py                               contextlib2-21.6.0.dist-info           idna-3.3.dist-info           pygdbmi-0.9.0.2.dist-info               serial
allnoconfig.py                                cryptography                           kconfiglib-13.7.1.dist-info  pyparsing                               setconfig.py
allyesconfig.py                               cryptography-36.0.1.dist-info          kconfiglib.py                pyparsing-3.0.7.dist-info               setuptools
bitstring-3.1.9.dist-info                     defconfig.py                           libfuturize                  pyserial-3.5.dist-info                  setuptools-60.5.0.dist-info
bitstring.py                                  _distutils_hack                        libpasteurize                python3_protobuf-2.5.0.dist-info        six-1.16.0.dist-info
certifi                                       distutils-precedence.pth               listnewconfig.py             python3_protobuf-2.5.0-py3.8-nspkg.pth  six.py
certifi-2021.10.8.dist-info                   ecdsa                                  menuconfig.py                PyYAML-6.0.dist-info                    tqdm
cffi                                          ecdsa-0.17.0.dist-info                 oldconfig.py                 reedsolo-1.5.4.dist-info                tqdm-4.62.3.dist-info
cffi-1.15.0.dist-info                         elftools                               olddefconfig.py              reedsolo.py                             urllib3
_cffi_backend.cpython-38-x86_64-linux-gnu.so  future                                 past                         requests                                urllib3-1.26.8.dist-info
cffi.libs                                     future-0.18.2.dist-info                pip                          requests-2.27.1.dist-info               _virtualenv.pth
charset_normalizer                            genconfig.py                           pip-21.3.1.dist-info         requests_toolbelt                       _virtualenv.py
charset_normalizer-2.0.10.dist-info           google                                 pkg_resources                requests_toolbelt-0.9.1.dist-info       wheel
click                                         guiconfig.py                           __pycache__                  savedefconfig.py                        wheel-0.37.1.dist-info
click-8.0.3.dist-info                         idf_component_manager                  pycparser                    schema-0.7.5.dist-info                  _yaml
construct                                     idf_component_manager-1.0.1.dist-info  pycparser-2.21.dist-info     schema.py                               yaml
construct-2.10.54.dist-info                   idf_component_tools                    pyelftools-0.27.dist-info    semantic_version

@HeikoGr
Copy link
Contributor

HeikoGr commented Jan 26, 2022

maybe this causes the problems.
can you try esp-idf in version 4.4? https://github.com/espressif/esp-idf/tree/v4.4

@anabolyc
Copy link
Contributor

Failed for me as well with IDF5, had to roll back to IDF4.4

@anabolyc
Copy link
Contributor

anabolyc commented Jan 26, 2022

To be sure that you've installed deps in the venv run commands from venv folder similar to this:

~/.espressif/python_env/idf4.4_py3.8_env$ ./bin/python --version
Python 3.8.10

and so on

@HeikoGr
Copy link
Contributor

HeikoGr commented Jan 26, 2022

Failed for me as well with IDF5, had to roll back to IDF4.4

i just tested idf5 on my cspot clone/fork with github action. the occuring errors don't relate to the problem(s) of @BartRijvers11

working idf4.4: https://github.com/HeikoGr/cspot/actions/runs/1750212376
failing idf5: https://github.com/HeikoGr/cspot/actions/runs/1750222894

@HeikoGr
Copy link
Contributor

HeikoGr commented Jan 26, 2022

@BartRijvers11 : maybe you need to undo your local patch

@HeikoGr
Copy link
Contributor

HeikoGr commented Jan 26, 2022

to be prepared when your order of the esp32 arrived, you could

  1. tell me, which chip you have ordered (to select the correct dac), and i'll set up my fork to copmpile it for you, or
  2. clone the repository yourself, activate github action (one extra click), modify the src, and let github compile cspot for you

how you receive the compiled artifacts:
https://github.com/HeikoGr/cspot/blob/HeikoGr-artifacts/.github/workflows/esp-idf.yml

(i will make a PR for this repo)

@BartRijvers11
Copy link
Author

BartRijvers11 commented Jan 26, 2022

To be sure that you've installed deps in the venv run commands from venv folder similar to this:

~/.espressif/python_env/idf4.4_py3.8_env$ ./bin/python --version
Python 3.8.10

and so on

I've rolled back to IDF 4.4 release branch, reverted local changes and I have the same version of python in that folder. If the venv file is important, can you share what it looks like on your platform and how I can fill missing entries (just add them textually or add using a command) ?

@BartRijvers11
Copy link
Author

Thanks a lot guys, success at last !
I first resorted to building the cli version. I get warnings about the proto version not being set, but it assumes version 2 and completes the build.
I then realized with @anabolyc 's remarks that I need to install the python extensions in the IDF toolchain python, not the global Ubuntu python. This did the trick for me to be able to build the ESP software, maybe some clarify can be provided in the readme.md file for newbees like me.

BTW. I ordered an AliExpress board with ES8388 as the AC101 is currently unavailable.

@BartRijvers11
Copy link
Author

BartRijvers11 commented Jan 26, 2022

(Unrelated to this issue)
Could you provide a template authBlob.json and a small description of how I can get that file on my ESP32 ?

@HeikoGr
Copy link
Contributor

HeikoGr commented Jan 26, 2022

as far as i understand authBlob.json stores the login credentials for the cli version.
you don't need the file for esp32.

@BartRijvers11
Copy link
Author

Ok, then how can I set op my Spotify credentials in the ESP32 ?

@HeikoGr
Copy link
Contributor

HeikoGr commented Jan 27, 2022

You don’t need to.

You need to compile cspot with your wifi credentials (this is the point, i remember why a precompiled firmware makes no sense, as long as there is no wifi manager included). The Spotify credentials come from the device you connect to the cspot.

@HeikoGr
Copy link
Contributor

HeikoGr commented Feb 3, 2022

@BartRijvers11 Did you succeed?

@BartRijvers11
Copy link
Author

Yes, I got the board 2 days ago and installed a pre-built version of euphonium (0.0.14). It works with webradio and spotify but the volume is very soft and I can't control it. My board has the ES8388 chip and Filip mentioned a new release coming soon. I haven't checked that as I did not yet have the time.
Cheers !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants