Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] ESP32: esptool below 4.8 silently produces broken firmware. #13824

Open
1 task done
cederom opened this issue Oct 4, 2024 · 5 comments
Open
1 task done

[BUG] ESP32: esptool below 4.8 silently produces broken firmware. #13824

cederom opened this issue Oct 4, 2024 · 5 comments
Labels
Arch: xtensa Issues related to the Xtensa architecture Area: Build system blocker Release Blocker OS: BSD Issues related to *BSD OSes (building system, etc)

Comments

@cederom
Copy link
Contributor

cederom commented Oct 4, 2024

Description / Steps to reproduce the issue

  1. Compiler.
esp32-elf-cc --version
xtensa-esp32-elf-cc (crosstool-NG esp-2021r2-patch5) 8.4.0
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
  1. With esptool 4.3 - cannot build firmware image - visible error:
% /usr/bin/time -h ./tools/configure.sh -B esp32-devkitc:nsh
        3,92s real              1,98s user              2,23s sys

% /usr/bin/time -h gmake -j8
 /usr/bin/time -h gmake -j8
Create version.h
Cloning Espressif HAL for 3rd Party Platforms
Clone: chip/esp-hal-3rdparty LN: platform/board to /XXX/nuttx-apps.git/platform/dummy
Register: nsh
Register: sh
Espressif HAL for 3rd Party Platforms: 20690e67695f0a8170a19ec99e2e9a13b620e94d
CPP:  /XXX/nuttx.git/arch/xtensa/src/chip/esp-hal-3rdparty/components/esp_rom/esp32/ld/esp32.rom.newlib-funcs.ld-> /XXX/nuttx.git/arch/xtensa/src/chip/esp-hal-3rdparty/components/esp_rom/esp32/ld/esp32.rom.newlib-funcs.ld.tmpLD: nuttx
CP: nuttx.hex
MKIMAGE: ESP32 binary
esptool.py -c esp32 elf2image --ram-only-header -fs 4MB -fm dio -ff 40m -o nuttx.bin nuttx
usage: esptool [-h] [--chip {auto,esp8266,esp32,esp32s2,esp32s3beta2,esp32s3,esp32c3,esp32c6beta,esp32h2beta1,esp32h2beta2,esp32c2,esp32c6}] [--port PORT] [--baud BAUD] [--before {default_reset,usb_reset,no_reset,no_reset_no_sync}] [--after {hard_reset,soft_reset,no_reset,no_reset_stub}] [--no-stub] [--trace]
               [--override-vddsdio [{1.8V,1.9V,OFF}]] [--connect-attempts CONNECT_ATTEMPTS]
               {load_ram,dump_mem,read_mem,write_mem,write_flash,run,image_info,make_image,elf2image,read_mac,chip_id,flash_id,read_flash_status,write_flash_status,read_flash,verify_flash,erase_flash,erase_region,merge_bin,get_security_info,version} ...
esptool: error: unrecognized arguments: --ram-only-header
gmake: *** [tools/Unix.mk:558: nuttx] Error 2
        40,96s real             1m32,23s user           1m1,82s sys

% pip show esptool
Name: esptool
Version: 4.3
Summary: A serial utility to communicate & flash code to Espressif chips.
Home-page: https://github.com/espressif/esptool/
Author: Fredrik Ahlberg (themadinventor) & Angus Gratton (projectgus) & Espressif Systems
Author-email:
License: GPLv2+
Location: /XXX/venv3.9embedded/lib/python3.9/site-packages
Requires: bitstring, cryptography, ecdsa, pyserial, reedsolo
Required-by: #N/A
  1. With latest esptool 4.8.1 - all works fine.
(venv3.9embedded) pip install -U esptool

(venv3.9embedded) /usr/bin/time -h gmake -j8
CPP:  /XXX/nuttx.git/arch/xtensa/src/chip/esp-hal-3rdparty/components/esp_rom/esp32/ld/esp32.rom.newlib-funcs.ld-> /XXX/nuttx.git/arch/xtensa/src/chip/esp-hal-3rdparty/components/esp_rom/esp32/ld/esp32.rom.newlib-funcs.ld.tmpLD: nuttx
CP: nuttx.hex
MKIMAGE: ESP32 binary
esptool.py -c esp32 elf2image --ram-only-header -fs 4MB -fm dio -ff 40m -o nuttx.bin nuttx
esptool.py v4.8.1
Creating esp32 image...
Image has only RAM segments visible. ROM segments are hidden and SHA256 digest is not appended.
Merged 1 ELF section
Successfully created esp32 image.
Generated: nuttx.bin
        4,19s real              2,96s user              3,44s sys

(venv3.9embedded) /usr/bin/time -h gmake flash
        15,26s real             3,03s user              3,24s sys

(venv3.9embedded) cu -l /dev/cuaU0 -s 115200
Stale lock on cuaU0 PID=4016... overriding.
Connected
ts Jul 29 2019 12:21:46

rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3ffb1f40,len:1388
ho 0 tail 12 room 4
load:0x40080000,len:20320
entry 0x40082650
*** Booting NuttX ***
dram: lma 0x00001020 vma 0x3ffb1f40 len 0x56c    (1388)
iram: lma 0x00001594 vma 0x40080000 len 0x4f60   (20320)
padd: lma 0x00006508 vma 0x00000000 len 0x9af0   (39664)
imap: lma 0x00010000 vma 0x400e0000 len 0x10d78  (68984)
padd: lma 0x00020d80 vma 0x00000000 len 0xf298   (62104)
dmap: lma 0x00030020 vma 0x3f400020 len 0x2848   (10312)
total segments stored 6

NuttShell (NSH) NuttX-12.7.0
nsh> uname -a
NuttX 12.7.0 10e44f8915-dirty Oct  4 2024 22:22:13 xtensa esp32-devkitc
nsh> free
                 total       used       free    maxused    maxfree  nused  nfree
      Umem:     323568       6424     317144       6800     186696     22      3
  1. With esptool 4.7.0 (default system package on FreeBSD) - all seems fine, no errors, but firmware is broken.
% gmake clean distclean

% /usr/bin/time -h ./tools/configure.sh -B esp32-devkitc:nsh
        3,89s real              1,77s user              2,41s sys

% /usr/bin/time -h gmake -j8
Create version.h
Cloning Espressif HAL for 3rd Party Platforms
Clone: chip/esp-hal-3rdparty LN: platform/board to /XXX/nuttx-apps.git/platform/dummy
Register: nsh
Register: sh
Espressif HAL for 3rd Party Platforms: 20690e67695f0a8170a19ec99e2e9a13b620e94d
CPP:  /XXX/nuttx.git/arch/xtensa/src/chip/esp-hal-3rdparty/components/esp_rom/esp32/ld/esp32.rom.newlib-funcs.ld-> /XXX/nuttx.git/arch/xtensa/src/chip/esp-hal-3rdparty/components/esp_rom/esp32/ld/esp32.rom.newlib-funcs.ld.tmp
LD: nuttx
CP: nuttx.hex
MKIMAGE: ESP32 binary
esptool.py -c esp32 elf2image --ram-only-header -fs 4MB -fm dio -ff 40m -o nuttx.bin nuttx
esptool.py v4.7.0
Creating esp32 image...
ROM segments hidden - only RAM segments are visible to the ROM loader!
Merged 1 ELF section
Successfully created esp32 image.
Generated: nuttx.bin
        39,74s real             1m31,54s user           1m2,00s sys

%  /usr/bin/time -h gmake flash
CPP:  /XXX/nuttx.git/arch/xtensa/src/chip/esp-hal-3rdparty/components/esp_rom/esp32/ld/esp32.rom.newlib-funcs.ld-> /XXX/nuttx.git/arch/xtensa/src/chip/esp-hal-3rdparty/components/esp_rom/esp32/ld/esp32.rom.newlib-funcs.ld.tmpLD: nuttx
CP: nuttx.hex
MKIMAGE: ESP32 binary
esptool.py -c esp32 elf2image --ram-only-header -fs 4MB -fm dio -ff 40m -o nuttx.bin nuttx
esptool.py v4.7.0
Creating esp32 image...
ROM segments hidden - only RAM segments are visible to the ROM loader!
Merged 1 ELF section
Successfully created esp32 image.
Generated: nuttx.bin
esptool.py -c esp32 -p /dev/cuaU0 -b 115200  write_flash -fs detect -fm dio -ff 40m 0x1000 nuttx.bin
esptool.py v4.7.0
Serial port /dev/cuaU0
Connecting.........
Chip is ESP32-D0WD-V3 (revision v3.0)
Features: WiFi, BT, Dual Core, 240MHz, VRef calibration in efuse, Coding Scheme None
Crystal is 40MHz
MAC: XXX
Uploading stub...
Running stub...
Stub running...
Configuring flash size...
Auto-detected Flash size: 4MB
Flash will be erased from 0x00001000 to 0x00033fff...
Compressed 206984 bytes to 72429...
Wrote 206984 bytes (72429 compressed) at 0x00001000 in 6.7 seconds (effective 247.0 kbit/s)...
Hash of data verified.

Leaving...
Hard resetting via RTS pin...
        15,25s real             3,07s user              3,17s sys


% cu -l /dev/cuaU0 -s 115200
Connected
ets Jul 29 2019 12:21:46

rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3ffb1f40,len:1388
ho 0 tail 12 room 4
load:0x40080000,len:20320
entry 0x40082650
*** Booting NuttX ***
dram: lma 0x00001020 vma 0x3ffb1f40 len 0x56c    (1388)
iram: lma 0x00001594 vma 0x40080000 len 0x4f60   (20320)
padd: lma 0x00006508 vma 0x00000000 len 0xaaf0   (43760)
imap: lma 0x00011000 vma 0x400e0000 len 0x10d78  (68984)
padd: lma 0x00021d80 vma 0x00000000 len 0xf298   (62104)
dmap: lma 0x00031020 vma 0x3f400020 len 0x2848   (10312)
total segments stored 6

>>> NO NSH HERE DEVICE HANGS <<<

Lets try with the CoreMark:

% gmake clean distclean

% /usr/bin/time -h ./tools/configure.sh -B esp32-devkitc:coremark
        3,93s real              1,73s user              2,46s sys

% /usr/bin/time -h gmake -j8
        42,46s real             1m35,59s user           1m0,58s sys

%  /usr/bin/time -h gmake flash
        13,78s real             2,69s user              2,63s sys

% cu -l /dev/cuaU0 -s 115200
Connected
ets Jul 29 2019 12:21:46

rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3ffb30a0,len:2016
load:0x40080000,len:30100
entry 0x40083940
*** Booting NuttX ***
dram: lma 0x00001020 vma 0x3ffb30a0 len 0x7e0    (2016)
iram: lma 0x00001808 vma 0x40080000 len 0x7594   (30100)
padd: lma 0x00008da8 vma 0x00000000 len 0x8250   (33360)
imap: lma 0x00011000 vma 0x400e0000 len 0xde14   (56852)
padd: lma 0x0001ee1c vma 0x00000000 len 0x21fc   (8700)
dmap: lma 0x00021020 vma 0x3f400020 len 0x14fc   (5372)
total segments stored 6

>>> COREMARK DOES NOT RUN <<<

I found this commit 457f9d3 to update the CI in order to build for ESP32 but manual builds should return error on older esptool versions or the older bootloader should be used? :-)

On which OS does this issue occur?

[OS: BSD]

What is the version of your OS?

13.3-RELEASE-p7 GENERIC amd64

NuttX Version

12.7.0-RC0

Issue Architecture

[Arch: xtensa]

Issue Area

[Area: Build System]

Verification

  • I have verified before submitting the report.
@github-actions github-actions bot added Arch: xtensa Issues related to the Xtensa architecture Area: Build system OS: BSD Issues related to *BSD OSes (building system, etc) labels Oct 4, 2024
@acassis
Copy link
Contributor

acassis commented Oct 6, 2024

@fdcavalcanti please take a look ^

@cederom
Copy link
Contributor Author

cederom commented Oct 6, 2024

Thanks @acassis, @fdcavalcanti this is quite important issue as it needs to be fixed in 12.7.0 release (it was found during 12.7.0-RC0 testing).

@xiaoxiang781216 xiaoxiang781216 added the blocker Release Blocker label Oct 7, 2024
@fdcavalcanti
Copy link
Contributor

I believe the proper way to proceed is using the most up-to-date esptool.
Let me check if there's any nuance we're missing.

@cederom
Copy link
Contributor Author

cederom commented Oct 7, 2024 •
edited
Loading

  • If version required is not present (i.e. 4.8 is required but 4.7 present) then error must be reported with build abort. So far no error is reported and silently the faulty image is built.
  • We should be able to build the firmware image also with older version of esptool (use older bootloader?).

@fdcavalcanti
Copy link
Contributor

@almir-okato

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Arch: xtensa Issues related to the Xtensa architecture Area: Build system blocker Release Blocker OS: BSD Issues related to *BSD OSes (building system, etc)
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@acassis @cederom @xiaoxiang781216 @fdcavalcanti