uTracer and ESP32

(also check the discussion on the GoogleGroups if you have any questionsor just drop me an email)

I can also provide you with a set of properly flashed and working ESP32 (4MB flash), RS232 convertor and connector, LEDs, wires and 1.3″ OELD screen for the total cost of 7 euros plus the shipping costs (the solderless white bereadboard, as in the pictures below, is also available).

Starting from Jan. 2021, Dutch PostNL forbids sending goods in envelops. Only the packages will be allowed, and this will increase the shipping costs from the current 4,50 euros to 6,00 (for Europe) and 10,00 (for USA/Canada) euros, which makes it not interesting for you anymore, so it might be better for you to order directly from China.

As an alternative, I can also provide 16MB flash (so more space for tube curves) ESP32 boards with all the parts described above, for 12 euros plus shipping. Those ESP32 boards are also have newer ESP32 chips and more stable WiFi reception (I have boards with and without external antenna). This might be a more interesting option, because the 16MB ESP32 development boards are not available online, so one has to buy the ESP32 module and a PCB separately and then solder those SMD-like components together.

Contents

ESP32 Wifi acquisition and data analysis module for uTracer, with support for all uTracer versions, 3 (300V), 3+ (400V), LV (50V) and 6 (1000V).

Latest firmware (0.92 build 20210426) (4MB flash)

  • spiffs.bin
  • firmware.bin for ESP32 with 1.3″ SPI OLED screen (SH1106 chip)
  • firmware.bin for ESP32 with 1.3″ I2C OLED screen (SH1106 chip)
  • Change Log

    Note: Two files should be flashed “over-the-air” while updating the ESP32 module: spiffs.bin and firmware.bin. If you are not using 1.3″ OLED screen, then it does not matter which version of the firmware (SPI or I2C) you flash.

Latest firmware (0.92 build 20210426) for 16MB flash

BETA firmware (0.92 build 20210418) for 4MB flash

Firmware Installation (DIY way)

To install the software on ESP32 two files (‘firmware.bin’ and ‘spiffs.bin’) should be flushed. The process is done in a user-friendly fashion using over-the-air (OTA) update. Any new ESP32 board requires flashing a small initial script (which enables OTA functionality) using a USB cable and Arduino IDE. This step should be performed only once. After that, all the further updates are done using OTA updates. To proceed, the Arduino IDE should be downloaded and installed from here.

Enabling ESP32 support in Arduino IDE:

  • Open your Arduino IDE, go to File>Preferences
  • Put https://dl.espressif.com/dl/package_esp32_index.json into the “Additional Board Manager URLs” field as shown in the figure below. 
  • Check the boxes with “Show verbose output during” for both “compilation” and “upload”
  • Click the “OK” button
  • Go to the Boards Manager. Go to Tools > Board > Boards Manager
  • Go to the search bar and type ESP32 then install ESP32 by Espressif Systems
  • Press Install
  • Now, download and add two required libraries ESPAsyncWebServer-master.zip and AsyncTCP-master.zip
  • Install those two library by going to Sketch > Include Library > Add .ZIP Library

Uploading the initial script:

  1. Open Arduino IDE and plug the ESP32 Board to your computer
  2. Select your Board in Tools > Board menu (in my case “ESP32 Dev Module”)
  3. Select the Communication Port, go to Tools > Port > (in my case it’s COM4 on Windows and on a Mac it is /dev/cu.SLAB_USBtoUART)
  4. Check the ‘Tools” menu if it corresponds to the screenshot below. If you have ESP32 with 16MB flash size, and not more commonly available 4MB flash, then follow this extra steps to create a custom partition with a larger size and then follow step 5 to flash the initial script.

  1. Download and open the initial script ESP32uTracerFWUploader.ino
  2. Press Upload button and wait a few seconds (about 20-50 sec.)
    (Note: if the sketch cannot be uploaded due to timeout and you stuck with “Connecting…….._____….______” output string, try to Upload once again, but then when you see “Connecting…”, press both buttons on ESP32 (EN and BOOT) at the same time, then release EN and then (0.5-1sec. later) release BOOT). Another trick that might work is to press the button BOOT just before the “Connecting…” message while uploading, and then release BOOT when “Connecting…” appeared and flashing has started.
  3. Open the Arduino IDE Serial Monitor at a baud rate of 9600
  4. Press the ESP32 on-board EN (Enable/Reset) button and you will see the IP address that ESP32 got
  1. Connect to the newly created WiFi network with SSID “ESP32” (see the password in the terminal window above or on your screen)
  2. Open that IP address in your web browser, or just type http://utracer.local (some browsers do not like that URL when the ESP32 is acting as hotspot/access point with SSID: ESP32, then use the actual IP address, which by default should be http://192.168.4.1 The problems is due to lack of Bonjour services on Windows machines. In order to use the ‘utracer.local’ shortcut instead of the actual IP address, one can install that service separately from here).
  3. Upload files spiffs.bin and firmware.bin exactly in this order (the upload progress will be visible only in the serial monitor window and not in the browser, so just wait for the “Update DONE” message on the webpage (if you are not watching the serial monitor window)
  4. Reconnect to WiFi network with SSID “ESP32” (because during the flashing of ESP32 your computer might switch back to local home WLAN/WiFi spot) and start using the software
  5. (it is always possible to connect to Serial port of ESP32 using the Arduino IDE, as described above, and see the debug and warning messages between ESP32 and uTracer)

Erasing ESP32 Flash (“Factory Reset”)

If you want to erase ESP32 flash completely, because you think it might be corrupted, or experiencing some problems, try to use the official “Flash Download Tool” and press “Erase” button there. For 4MB flash ESP32, one would have to select 32Mbit flash size. You can also read a bit more here or see my screencast here. In this screencast erasing of 16MB flash (128Mbit) was done. To do the same with 4MB flash, select option Flash Size 32Mbit in the dialog.

For more experienced users, handy with command line tools, the simple command is:
esptool.py –port /dev/cu.SLAB_USBtoUART erase_flash
where you have to put your own “com” port number/location.

Firmware Updates (Over-The-Air)

  • open the page http://utracer.local/login in your browser (or use the actual IP address of EPS32, on Windows machines ‘utracer.local’ will not work by default)
  • select ‘Firmware update’ option in the dropdown menu
  • login with username ‘admin’ and password ‘admin’
  • choose the desired .bin file
  • wait until the update is done, ESP32 will restart itself

ESP32 as a WiFi client

To connect the ESP32 as a client within your existing WiFi network (with your own SSID and PASSWORD) follow this steps:

  • open the page http://utracer.local/login in your browser (or use the actual IP address of EPS32, on Windows machines ‘utracer.local’ will not work by default)
  • select ‘WiFi setup’ option in the dropdown menu
  • login with username ‘admin’ and password ‘admin’
  • in the fields SSID and Password enter the details of your WiFi router
  • select ‘ESP is a client for WLAN Router’ in the dropdown menu
  • press ‘Save to EEPROM’
  • reboot/reset the ESP32 module with EN button or by disconnecting and reconnecting the usb cable

Hardware connection to uTracer

There are two major types of developments boards on the market with ESP32 modules: 30- and 38-pins. 30-pin boards mostly contain ESP32-WROOM-32 module with 4MB flash, and 38-pin modules come in more variations, for example ESP32-WROOM-32U, ESP32-WROOM-32D, ESP32-WROOM-32E, with 4, 8 and 16MB flash.

.

The dashed line represents alternative powering (for example from uTracer) when USB cable is not connected (only one power source should be connected to ESP32 at a time). Be careful, on 38-pin boards (at least the ones which are shown in the image), there is a wrong labeling of pin 38 (next to pin 38 “5V”). It is printed “GND” but connected to “CMD” on the ESP32 module. Do not use it as a ground!
In the images above, “TX” and “RX” do not indicate the ‘serial’ communication with uTracer but show the measurement requests and responses during the communication between the webinterface and uTracer.

The “truth table” for three LEDs (R (Error) G (TX) and B (RX)) is as follows:

  • OFF ON OFF – transmitting the measurement command (10…)
  • OFF OFF ON – receiving the measurement from uTracer
  • ON ON OFF – error during the transmission (no echo from uTracer)
  • ON OFF ON – error during receiving (returned message is too short/long)
  • ON ON ON – error during receiving, nothing is transmitted by uTracer
  • ON OFF OFF – compliance error
  • OFF (On On Blinking) – ESP32 is connecting to local WiFi

Note 1: The external GREEN LED is the same as the built in BLUE LED on the 30 pin ESP32 development board.

Note 2: The pull-up resistor: “The module datasheet states that it’s UART TX pin is open drain and needs a 10K-15K ohm pull-up (to +3.3V).” More about the problem can be found here and here. So, the pull-up resistor should be present between the ESP32 and RS232-TTL converter, and can be placed either on ESP32 or on the converter. If you plan to use the ESP32 directly with TTL logic (e.g. connecting to TX/RX pins of PIC controller on uTracer), then remove the external pull-up resistor from ESP32. At the same time be sure that the input of the following device is pulled-up to HIGH. Some users report that RS22-TTL converters based on MAX3232 chip work fine without the pull-up resistor. In my tests with SP3232 chip I only noticed problems without the pull-up only in rare (reproducible) cases, so that resistor is a solution. I have not done such tests with MAX3232 chip.

Note 3: Do not touch ESP32 during the operation. By design, ESP32 has lots of pins that react on capacitive touch. Such touches typically start dumping garbage into the Serial, confusing ESP32 and uTracer.
After these kind of problems, sometimes my uTracer was showing that there still high voltage present (with the corresponding LED on the uTracer) even after switching uTracer off and on (for a short time). This kind of problematic behavior can be also almost surely observed when the “pull-up” resistor (see the figure above) is missing. Be sure you have installed it as well. What helps is to
– reconnect to uTracer with ESP32
– go to ‘Debug’ page in the webinterface
– send the command 300000000000000000
– wait until the high voltage LED will dim off

Experiments

How to connect other ESP boards

For all the experiments above, the ESP32 development board with 30 pins has been used. In order to use the 38-pin version EPS32 development board, one has to use the same GPIOs numbers (or labels with “D”) and not the pin numbers on the board (see the pinouts below). Note, do not use pin 37 on those boards (wrongly labeled as “GND”). The description of all ESSP32 modules (which are places on those development boards) can be found here. The tests have been done only with “ESP32-WROOM-32” with 4MB flash, which comes with both 30- and 38-pin development boards. There should be no problems with the boards that contain “ESP32-WROOM-32U” which has an external antenna connection and “ESP32-WROOM-32D” which has no functional or electrical differences with “ESP32-WROOM-32”. Both modules (“ESP32-WROOM-32D” and “ESP32-WROOM-32U”) integrate the ESP32-D0WD chip which has a smaller footprint than the chip ESP32-D0WDQ6 installed in ESP32-WROOM-32. If you are more adventurous, you can try to run the firmware on a development board with ESP32-WROVER-… modules. It works for those of them that do not have PSRAM integrated on the module (see the table on Espressif’s website), because this pseudo RAM uses GPIOs 16 and 17, which are right now also used serial communication with uTracer. In case you do have such boards, I can reconfigure pins 16 and 17 and recompile the firmware, so it will also work. Additionally, all the boards with 16MB of flash are also tested and fully supported.
Note: Similarly looking boards that use modules such as ESP-12E or ESP8266 are totally different (simpler) architectures which will not work with the described firmware.

The connection with uTracer via the RS232 converter uses 4 wires
– GPIO16 for RX
– GPIO17 for TX
– 3.3V
– GND

The connections for LEDs are pins with:
– GPIO5 for the (red) “ERROR” LED
– GPIO2 for the (green) “TX” LED
– GPIO4 for the (blue) “RX” LED
The colors of the LEDs can be arbitrary chosen.

Connecting the 1.3″ OLED 7-pin SPI screen:
– OLED GND — ESP32 GND
– OLED VDD — ESP32 3.3V
– OLED SCK — ESP32 GIPO18
– OLED SDA — ESP32 GIPO23
– OLED RES — ESP32 GIPO22
– OLED DC — ESP32 GIPO21
– OLED CS — ESP32 GIPO15

Connecting the 1.3″ OLED 4-pin I2C screen:
– OLED VDD — ESP32 3.3V
– OLED GND — ESP32 GND
– OLED SCK — ESP32 GIPO22
– OLED SDA — ESP32 GIPO21

All those 1.3″ and 0.96″ screens have a 3.3V voltage regulators on board, so can be also powered from +5V of ESP32 (in case you want to connect everything with hanging wires (without a breadboard) and you miss extra +3.3V connection point).

Monitoring ESP32 via Serial Port

To see the output of ESP32 a terminal window in order to monitor warning, error and output messages (if you do not use Arduino IDE already), one can do the following:

  • Download a serial terminal program, for example Tera Term
  • Connect ESP32 with a USB cable and find the COM port number in the Device Manager

  • Start the Tera Term and select the corresponding COM port

  • Press “Ok” and, for example, restart the ESP32 to see the booting messages

Online Version

Those who would like to test the webinterface and parts of the software (including modeling, load lines, THD analysis, basically everything except data acqusition) are welcome to try it at http://utracer.live or http://utracer.boffin.nl

Here are some screenshots:

FAQ (answers to user’s comments)

Q: Can ESP32 be connected directly to RX/TX pins of the PIC controller, avoiding two RS232 converters (one external and one from uTracer)?
A: It can definitely be done and it will work, but I have not tried it. The ESP32 connection in its current form was designed with the idea of having completely external “add-on” that can be externally connected to uTracer. So no messing up with your uTracer already placed in a nice case/box is required. Two notes here:
1) ESP32 module requires 5V power, which should be better supplied externally because it typically consumes from 60mA to about 220mA (while transmitting). This might be too much in terms of heat dissipation for the uTracer’s LM7805 5V regulator (depends on the size of the heatsink).
2) ESP32 has 3.3V logical levels and is not 5V tolerable. Making it work with uTracer’s PIC controller (5V logic) requires level shifters, or opamps, or in the simplest form a resistive voltage divider for ESP32’s RX pin. This already adds to complexity that you have to solve yourself. Another important note is that ESP32’s TX pin should “see” HIGH-pulled up input on uTracer’s or RS232TTL’s side. Right now I add a pull-up resistor either on the ESP32 or RS232TTL converter (in the TX line, between TX and 3.3V). With the PIC controller, I am not sure if there are internal pull-up resistors activated, or should be added externally. You should take care of that in your design.

Q: Can other (cheaper) RS232 to TTL adapters (based on MAX3232) be used?
A: Yes, but you have to be careful with fake “MAX3232” chips which are sold online. I recently bought the RS232 adapter with a built-in socket but it did not work from the very beginning (it was just getting very hot when powered). the schematics and the PCB were correct but the problem was with the fake (non-TI) Chinese MAX3232 chip, which they placed on a board designed for TI MAX3232 chip. The solution was to disconnect the C3 capacitor from the ground and connect to +3.3V (see the pictures). For real TI chips, C3 can be connected to GND or +3.3V, but not for fakes.

Wiring of the MAX3232 chip with ESP32 and uTracer:
DIN1 -> ESP32 TX (GPIO17)
DOUT1 -> uTracer RX pin (pin 3 on DB9 connector, or pin with “->” on PCB)
RIN1 -> uTracer TX pin (pin 2 on DB9 connector, or pin with “<-” on PCB)
ROUT1 -> ESP32 RX (GPIO16)

One of the users reported that the module below (and those are sold out everywhere) is also overheating all the time. It sounds like that typical problem with fake MAX3232 chips and a PCB which is designed for genuine TI chips. Try to avoid those chips and look for boards with SP3232, or be ready to re-solder the C3 capacitor.

NoteAvoid buying the following converters (with female connector, see below), because even though they do work, you will spend a lot of time figuring out the pinout in that converter and in the cable that will connect that converter and uTracer. Also avoid those male-to-female adapters. After quite some debugging one of the users figured out how it should work for his case, but it is not guaranteed that it will work in your case as well. Better stick with simple TTL to RS232 converters (described above) without connectors.

QHow do I see if at least the ESP32 works, when I connect EPS32, RS232toTTL converter to uTracer and nothing works?
A: to test if ESP32 does what it is supposed to do, one can shorten the TX and RX pins that go to uTracer (so GPIO16 and GPIO17) and then in the webinterface go to “Debug” and press “Ping”. One should get exactly this behavior and error in the end (pay attention only to the last 3 lines in the output window below):

To trace the problem further,
– connect the RS232toTTL converter
– shorten the TX and RX lines that go from RS232toTTL to uTracer
– go to “Debug” and press “Ping”, you should get the same output as above