Can't run erase_flash and write_flash back-to-back from Python

[marcelstoer]

If you run esptool from within a Python script rather than triggering the individual operations from the commandline erase_flash and write_flash can't run back-to-back.

import esptool
from esptool import ESPROM
from argparse import Namespace

esp = ESPROM("/dev/cu.SLAB_USBtoUART")
esp.connect()
args = Namespace()
args.flash_size = "detect"
args.flash_mode = "dio"
args.flash_freq = "40m"
args.no_progress = False
args.verify = True
args.baud = 921600
args.addr_filename = [[int("0x00000", 0), open("/Users/marcelstoer/Downloads/nodemcu.bin", 'rb')]]
esptool.erase_flash(esp, args)
esptool.write_flash(esp, args)

The output is something along the lines of this:

Connecting...
Couldn't connect. [<class 'esptool.FatalError'>; Timed out waiting for packet header]. Retrying...
Couldn't connect. [<class 'esptool.FatalError'>; Timed out waiting for packet header]. Retrying...
Couldn't connect. [<class 'esptool.FatalError'>; Timed out waiting for packet header]. Retrying...
Couldn't connect. [<class 'esptool.FatalError'>; Timed out waiting for packet header]. Retrying...
Couldn't connect. [<class 'esptool.FatalError'>; Timed out waiting for packet header]. Retrying...
Running Cesanta flasher stub...
Erasing flash (this may take a while)...
Erase took 8.1 seconds
Traceback (most recent call last):
  File "run.py", line 16, in <module>
    esptool.write_flash(esp, args)
  File "build/bdist.macosx-10.11-x86_64/egg/esptool.py", line 876, in write_flash
  File "build/bdist.macosx-10.11-x86_64/egg/esptool.py", line 865, in detect_flash_size
  File "build/bdist.macosx-10.11-x86_64/egg/esptool.py", line 269, in flash_id
  File "build/bdist.macosx-10.11-x86_64/egg/esptool.py", line 219, in flash_begin
  File "build/bdist.macosx-10.11-x86_64/egg/esptool.py", line 116, in command
  File "build/bdist.macosx-10.11-x86_64/egg/esptool.py", line 85, in read
  File "build/bdist.macosx-10.11-x86_64/egg/esptool.py", line 715, in slip_reader
esptool.FatalError: Timed out waiting for packet header

Commenting either of the two esptool.erase_flash(esp, args) esptool.write_flash(esp, args) lines "fixes" this. So does executing esp.connect() between the two operations.

In case this can't/won't be fixed in esptool.py how could I properly work around it?

[sopi]

Have the same

[projectgus]

Sorry for the slow reply. Calling esp.connect() in between is probably the best workaround.

There's a half-realised intention that the "ESPROM" objects can be used as a generic Python API, however - as I'm sure you've noticed - at the moment too much functionality is in the module-level functions (which are intended for use from the command line.)

This is getting a bit better in esptool v2.0, as more functionality is being moved into the objects and less in the top-level functions. But there's probably still some work before it gets to what I'd like, which is something like:

esp = esptool.ESP8266ROM("/dev/serial")
esp.connect()
esp.erase_flash()
esp.write_flash(0, "image.bin")

[projectgus]

BTW, are you building a more complex script or do you just need to erase_flash and write_flash in the same pass? It would be feasible to add a --erase_flash option to the write_flash command, for the latter case.

[marcelstoer]

I'm working on a wxPython GUI for esptool: https://github.com/marcelstoer/nodemcu-pyflasher

The intention is for this to be packaged with PyInstaller into a self-contained binary for all the NodeMCU Windows users who work on a platform that a) doesn't come with Python and b) doesn't treat the CLI as a first-class citizen.

[projectgus]

Neat!

I'll let you know if/when I make any progress towards the kind of Python API mentioned in the previous comment. Until then, I think using esp.connect() twice is the best workaround.

[marcelstoer]

Neat!

Thanks, it's all give and take 😄

You provide a great flashing tool, I build a GUI for it, I run the NodeMCU cloud builder, you build firmware with it to improve esptool (c9cb4da), to be continued...

[marcelstoer]

If you want you may now close this. It works fine with chip == 'auto'.

ESPLoader.detect_chip calls connect() internally. Only if you manually select the chip you also need to manually connect.

initial_baud = min(ESPLoader.ESP_ROM_BAUD, my_baud)
esp = ESPLoader.detect_chip(self._config.port, initial_baud)
esp = esp.run_stub()
if my_baud > initial_baud:
    try:
        esp.change_baud(my_baud)
    except NotImplementedInROMError:
        print("WARNING: ROM doesn't support changing baud rate. Keeping initial baud rate %d" % initial_baud)
args = Namespace()
args.flash_size = "detect"
args.flash_mode = "dio"
args.flash_freq = "40m"
args.no_progress = False
args.no_stub = False
args.verify = False  # TRUE is deprecated
args.compress = True
args.addr_filename = [[int("0x00000", 0), open("/Users/marcelstoer/Downloads/nodemcu.bin", 'rb')]]
esptool.erase_flash(esp, args)
esptool.write_flash(esp, args)

[projectgus]

Thanks @marcelstoer . I'll close this, but I've opened another to keep track of work needed so esptool.py can be better used as Python module.

If you have additional suggestions for changes which would help, please feel free to comment there.

[marcelstoer]

That other issue is #208 for those who missed the issue reference above.

If you have additional suggestions

All is good for now. The latest snippet above is a slightly modified version of what went into my PyFlasher. Please have a look at https://github.com/marcelstoer/nodemcu-pyflasher/blob/master/Main.py#L46 to see how esptool.py is integrated in the flasher thread.

The only pain is redirecting the console output to the GUI component. I have a custom component for that at https://github.com/marcelstoer/nodemcu-pyflasher/blob/master/Main.py#L24 that I had to rewrite for the 2.0 upgrade to accommodate for the modified progress monitor (backspaces vs. carriage return). You addressed this in the description of #209.