[BenchMark]Web upload / Web Download on different FS and platform

[luc-github]

I think it would be very informative to bench the speed of upload / download a file using FTP / WebDav / Webserver
to LocalFS / SD on ESP32 and ESP8266
using MCVE to be sure the comparaison is correct

Target	Speed KB/S	Note
ESP32 + WEBSERVER + FAT	56.01
ESP32 + WEBSERVER + LITTLEFS	59.60
ESP8266 + WEBSERVER + LITTLEFS
ESP32 + WEBSERVER + SD native	75.06
ESP8266 + WEBSERVER + SD native
ESP32 + WEBSERVER + SD Fat	101.9 ~122.4	SDFAT 2.1.2
ESP8266 + WEBSERVER + SD Fat
ESP32 + WEBSERVER + SDIO
ESP8266 + WEBSERVER + SDIO
ESP32 + WEBDAV+ LITTLEFS
ESP8266 + WEBDAV+ LITTLEFS
ESP32 + WEBDAV+ SD native
ESP8266 + WEBDAV+ SD native
ESP32 + WEBDAV+ SD Fat
ESP8266 + WEBDAV+ SD Fat
ESP32 + WEBDAV+ SDIO
ESP8266 + WEBDAV+ SDIO
ESP32 + FTP+ LITTLEFS
ESP8266 + FTP+ LITTLEFS
ESP32 + FTP+ SD native
ESP8266 + FTP+ SD native
ESP32 + FTP+ SD Fat
ESP8266 + FTP+ SD Fat
ESP32 + FTP+ SDIO
ESP8266 + FTP+ SDIO

Others FW:

FW	Target	Speed KB/S	Note
grblHAL	ESP32 + FatFS + SDCard + WebDAV	300~350
grblHAL	iMXRT1062 (Teensy4) + FatFS + SDCard + WebDAV	1127	Ethernet
grblHAL	STM32F7xx + FatFS + SDCard + WebDAV	350	Ethernet

Adding with PSRAM could also be a thing to check on ESP32

Once done need to compare with speed in ESP3D / ESP3DLib and GRBL_ESP32 to see what must be improved and how
This is based on feedback ESP32 WebServer upload is very slow compare to ESP8266

Using any asyncwebserver / external webserver is not the current target so it is out of scope - the goal is to do a status of current situation to plan improvements / changes

[luc-github]

this PR may solve the speed issue espressif/arduino-esp32#4787 (not tested)

[luc-github]

Just listing files with Native ESP32 SD is slow in core 1.X, but it 10 times slower or more with core 2.X when using SPI, so it is a must to use the SDFAT2 on ESP32 with SPI
SD Native Core 1.x

SD Native Core 2.x

SDFat2

[luc-github]

I have added a simple bench check in FS file upload abd SD File Upload in ESP3DLib 3.0 - and tested ESP32

same test should be done on ESP3D to see if using core 0 instead of core 1 affect the perfomance

I will test other configurations when I have time

[liqijian101]

hi luc,I test esp32s2 with sdfat2.0 in spi mode .I find that the sector size of tf card can influence the speed of upload files.

[luc-github]

what size did you set? what speed did you get?

[liqijian101]

A low speed card, 8g capacity, 2048ksec, Max speed 270ksec
a high speed card, 16G capacity, 64ksec, Max speed 190KSEC,
and I found that SDFAT2 does not support 32G cards

[liqijian101]

I also found that using esp32s2sdfat2.0 is faster than using 1.0

[luc-github]

I did configuration with compatibility with 3Dprinter FW in mind, so 4G/8G is the target, 16G may be also supported, be sur you use https://www.sdcard.org/downloads/formatter/

and use Fat32 format
I did tested 32G/64G/128G in past but long time I did not recheck the support, and definitly not with SDFat 2.x

[terjeio]

Some grblHAL speeds with FatFS + SDCard + WebDAV, using WinSCP for testing:

• ESP32: 300-350KB/s up- and download speeds, this with my heavily modified lwIP http daemon (I have ditched the ESP32 http daemon).
• iMXRT1062 (Teensy4) 1.1MB/s upload, 3-3.5MB/s download (with ethernet).
• STM32F7xx: 300-350KB/s up- and download speeds. This is with a broken and slow low-level ethernet driver, there is a new available from ST and I will switch to this soon.

RP2040 (Pi Pico W) support for WebUI is on my todo list, will be interesting to see how this fares...

[luc-github]

@terjeio thank you I have added your figures on top

[luc-github]

For information @pipi61 reported #491 (comment) that esp32 + webdav + SDIO can reach 500KB/s

Additionnaly Reprap porting team, that added esp32 support to duetwifi, claim that they reach 1.2MB/s, but in case of Duet wifi, esp32 is only used as data bridge, the webserver is hosted by the main MCU, and I think the SD card is SDIO one, but communitation between ESP32 and MCU are SPI only - TBC as I never took time to dig in

But it shows that there is area of improvement 🐱