How to connect native OTA stack with LocalHost Listener [esp_modem] #820

bcebesoy · 2025-06-11T00:08:11Z

bcebesoy
Jun 11, 2025

I have been using esp_modem in our customer project. I have started it from modem_tcp_client example and so it seems like near to the finish line.

There is a problem, I can't get fixed about native OTA. http OTA couldn't be connected to the localhost and to the server. I have already done the mqtt connection between broker and local client using localhost but it didn't worked for OTA.

Here is the URL's for that configuration;

#define OTA_HTTP_URL "http://127.0.0.1/ota/download" // using in http OTA cfg

#define OTA_HTTP_GSM_URL "sub.domainname.com" // using GSM connection
#define OTA_HTTP_GSM_PORT 80 // using GSM connection

Is there any general problem or obligation etc. that I can't figure it out?

Thanks in advance!

Answered by david-cermak

Jun 12, 2025

I probably know what's wrong, didn't occur to me earlier, but seems like the host field is added to the http headers and with your configuration it uses 127.0.0.1. One workaround is to use the actual hostname and make the internal DNS resolve it to localhost.

#define OTA_HTTP_URL "http://"sub.domainname.com/ota/download"
#define OTA_HTTP_GSM_URL "sub.domainname.com" // using GSM connection

For the DNS part, you can use an lwip extern-resolve hook, set it to "weak" and implement it in your main component.

View full answer

david-cermak · 2025-06-11T06:05:24Z

david-cermak
Jun 11, 2025
Maintainer

I have already done the mqtt connection between broker and local client using localhost but it didn't worked for OTA.

Are you trying to run OTA after closing the MQTT connection, or simply from startup? I'm asking since this is a very simple example demonstrating only one connection at a time, so maybe the close functionality didn't deinitialize everything properly? Could you try to setup the OTA only application to see if it could actually connect and run?
Other than that, I think it should work. Also, you need to update the port number of the localhost listener, since the OTA would connect on port 80 (to localhost, the OTA_HTTP_GSM_PORT port is where the actual outgoing socket would go)

0 replies

bcebesoy · 2025-06-11T07:49:38Z

bcebesoy
Jun 11, 2025
Author

Hi David,

First of all, I apologize for not providing enough details earlier — I wrote the post in a rush, sorry about that.

In the project, I’m using esp_modem v1.3.0 and IDF v5.4.0.

As you mentioned, I tried to run the OTA process after shutting down the MQTT client. I have a recovery function that is triggered when perform_sock fails in the DCE component. This function first closes the existing listening socket by stopping listening, performs the necessary cleanup, and then resets the DCE pointer using its destructor before re-establishing the connection. This approach usually helps in recovering the connection. However, when I call the destructor without any prior error (i.e., in a normal case), the program ends up hanging for a long time and becomes unresponsive.

When this method didn’t work, I decided to perform a full soft restart of the ESP32. I also switched from an HTTPS (SSL/TLS) connection to a plain HTTP connection to avoid TLS-layer errors. That’s why I attempted to use port 80 instead of 443, and I used the #define values I shared in my previous post to configure the connection. Unfortunately, the OTA client that is supposed to connect to localhost returns an HTTP 404 error.

To avoid initial timeout issues, I increased the HTTP request timeout from 2 seconds to 10 seconds, but it had no effect. Later, I tried to apply the precautions you suggested with the test_ota example. For instance, I placed the UART ISR functions into IRAM as recommended — but that didn’t help either.

Thank you very much for your help and support. Wishing you all the best in your work.

4 replies

david-cermak Jun 11, 2025
Maintainer

returns an HTTP 404 error.

So maybe the http path is not propagated correctly? Could you please share the (debug-level) logs? Or alternatively use http client configuration, where you set scheme, host, path and port separately (instead one URI string)

Also, if you're using HTTP and MQTT only, maybe you can use the EXAMPLE_CUSTOM_TCP_TRANSPORT option?

david-cermak Jun 12, 2025
Maintainer

I probably know what's wrong, didn't occur to me earlier, but seems like the host field is added to the http headers and with your configuration it uses 127.0.0.1. One workaround is to use the actual hostname and make the internal DNS resolve it to localhost.

#define OTA_HTTP_URL "http://"sub.domainname.com/ota/download"
#define OTA_HTTP_GSM_URL "sub.domainname.com" // using GSM connection

For the DNS part, you can use an lwip extern-resolve hook, set it to "weak" and implement it in your main component.

Answer selected by bcebesoy

bcebesoy Sep 30, 2025
Author

Hi David,

First of all, sorry for the very late follow‑up — I’ve been swamped lately and had to pause development for a while. Thanks again for the previous guidance.

Quick status update:

I switched to using an external DNS resolver as you suggested; with that, the connection comes up correctly.

The HTTP client first performs a HEAD request to obtain metadata. The server responds with 200 OK and a valid Content-Length.

Immediately after the HEAD completes, I observe a stall and then a “peer closed” from the modem/AT socket bridge. The next write (for the GET with Range) often fails (AT responds with ERROR), which makes it look like lwIP or the AT socket layer closed the connection and didn’t reopen cleanly in time.

From the debug logs, the resolver returns 127.0.0.1 for the hostname in this setup. I’m not fully sure whether that’s intended in this topology or if I should resolve to the external IP for the OTA flow.

A few questions to validate my approach:

Is there any known caveat with doing a HEAD followed by a GET (with Range) when using esp-modem in CIPRXGET mode?
Would you recommend forcing “Connection: close” on the HEAD (so the client always reopens a fresh socket for the GET), or disabling the HEAD step in the HTTP client config?
Alternatively, would adding an explicit close/reopen (AT+CIPCLOSE / AT+CIPSTART) after the HEAD, or introducing a small delay, be advisable?
Any other knobs I should try (e.g., keep-alive settings, read polling cadence, ensuring all URCs are drained before the next write)?

Below is a sanitized snippet of the logs with identifiable details replaced by placeholders. I’d really appreciate any suggestions or insights you might have on what I’m missing or how to stabilize the transition from HEAD to GET. Thanks in advance!

V (9134) sock_commands: setAutoTimeRef
V (9134) command_lib: generic_command
D (9134) command_lib: generic_command command AT+CLTS=0

D (9144) command_lib: Response:
OK


I (9144) sock_dce: TIME REF SUCCESS CANCELLED
V (9154) command_lib: get_operator_name
V (9154) command_lib: generic_get_string
} (9164) command_lib: Token: {

V (9164) command_lib: Token: {+COPS: 0,0,"<OPERATOR>"}

} (9174) command_lib: Token: {

V (9174) command_lib: Token: {OK}

I (9174) sock_dce: Operator name: <OPERATOR>
I (9184) sock_dce: Socket created
I (9184) sock_dce: Socket bound, port 80
E (9194) modem_client: SIM_CD2: 0
V (9194) sock_commands: tcp_close
V (9194) command_lib: generic_command
D (9204) command_lib: generic_command command AT+CIPCLOSE=1

D (9214) command_lib: Response:
CLOSE OK


V (9224) sock_dce: 0x3ffd4f00   0d 0a 4f 4b 0d 0a                                 |..OK..|
D (9224) sock_commands: C_A_R RESPONDER!
OK

V (10424) sock_dce: 0x3ffd4f00   0d 0a 43 4f 4e 4e 45 43  54 20 4f 4b 0d 0a        |..CONNECT OK..|
D (10424) sock_commands: C_A_R RESPONDER!
CONNECT OK

W (10424) sock_commands: Connected!
W (11224) modem_client: OTA Starting!
I (11224) HTTPS_OTA: Advanced OTA task created successfully
I (11224) HTTPS_OTA: Starting Advanced OTA
E (11224) sock_dce: No active socket, accepting...
E (11274) modem_client: accelTask
W (11284) modem_client: Free heap size: 145336 bytes
W (11284) modem_client: Minimum ever free heap size: 145336 bytes
I (11294) main_task: Returned from app_main()
E (11744) sock_dce: No active socket, accepting...
E (12254) sock_dce: No active socket, accepting...
E (12764) sock_dce: No active socket, accepting...
W (13224) CUST_CONN: Free heap size before OTA: 150612
W (13224) CUST_CONN: Minimum ever free heap size before OTA: 145336
W (13224) HTTPS_OTA: Attempting to start OTA, attempt 1
W (13234) esp_https_ota: Continuing with insecure option because CONFIG_ESP_HTTPS_OTA_ALLOW_HTTP is set.
D (13234) event: running post ESP_HTTPS_OTA_EVENT:0 with handler 0x400def28 and context 0x3ffd2398 on loop 0x3ffd142c
--- 0x400def28: event_handler(void*, char const*, long, void*) at <WORKSPACE_PATH>

D (13244) HTTP_CLIENT: Begin connect to: http://<HOST_1>:80
I (13254) HTTPS_OTA: OTA started
D (13254) esp-tls: host:<HOST_1>: strlen 19
D (13264) esp-tls: [sock=55] Resolved IPv4 address: <IP_LOCAL>
E (13274) sock_dce: No active socket, accepting...
D (13274) esp-tls: Enable TCP keep alive. idle: 5, interval: 5, count: 3
D (13284) esp-tls: [sock=55] Connecting to server. HOST: <HOST_1>, Port: 80
W (13294) sock_dce: Socket accepted!
D (13294) HTTP_CLIENT: Write header[2]: HEAD /ota/download HTTP/1.1
User-Agent: ESP32 HTTP Client/1.0
Host: <HOST_1>


D (13294) event: no handlers have been registered for event ESP_HTTP_CLIENT_EVENT:1 posted to loop 0x3ffd142c
D (13314) sock_commands: start_sending!!!
V (13314) sock_dce: 0x3ffd4f00   0d 0a 3e 20                                       |..> |
D (13324) event: no handlers have been registered for event ESP_HTTP_CLIENT_EVENT:2 posted to loop 0x3ffd142c
D (13334) sock_commands: C_A_R RESPONDER!
>
I (13344) sock_commands: Data send IN PROGRESS!
V (14014) sock_dce: 0x3ffd4f00   0d 0a 53 45 4e 44 20 4f  4b 0d 0a                 |..SEND OK..|
D (14024) sock_commands: C_A_R RESPONDER! 
SEND OK

V (14224) sock_dce: 0x3ffd4f00   0d 0a 2b 43 49 50 52 58  47 45 54 3a 20 31 0d 0a  |..+CIPRXGET: 1..|
D (14224) sock_commands: C_A_R RESPONDER!
+CIPRXGET: 1

D (14224) sock_commands: start_receiving 1024
V (14244) sock_dce: 0x3ffd4f00   0d 0a 2b 43 49 50 52 58  47 45 54 3a 20 32 2c 34  |..+CIPRXGET: 2,4|
V (14244) sock_dce: 0x3ffd4f10   30 37 2c 30 0d 0a 48 54  54 50 2f 31 2e 31 20 32  |07,<HOST_2>/1.1 2|
V (14254) sock_dce: 0x3ffd4f20   30 30 20 4f 4b 0d 0a 53  65 72 76 65 72 3a 20 6e  |00 <HOST_3>: n|
V (14264) sock_dce: 0x3ffd4f30   67 69 6e 78 0d 0a 44 61  74 65 3a 20 54 75 65 2c  |<HOST_4>: Tue,|
V (14274) sock_dce: 0x3ffd4f00   20 33 30 20 53 65 70 20  32 30 32 35 20 31 36 3a  | 30 Sep 2025 16:|
V (14284) sock_dce: 0x3ffd4f10   35 36 3a 34 33 20 47 4d  54 0d 0a 43 6f 6e 74 65  |56:43 <HOST_5>|
D (14284) HTTP_CLIENT: on_message_begin
V (14294) sock_dce: 0x3ffd4f20   6e 74 2d 54 79 70 65 3a  20 61 70 70 6c 69 63 61  |nt-Type: applica|
D (14294) HTTP_CLIENT: HEADER=Server:nginx
V (14304) sock_dce: 0x3ffd4f30   74 69 6f 6e 2f 6f 63 74  65 74 2d 73 74 72 65 61  |tion/octet-strea|
V (14314) sock_dce: 0x3ffd4f40   6d 0d 0a 43 6f 6e 74 65  6e 74 2d 4c 65 6e 67 74  |m..Content-Lengt|
V (14324) sock_dce: 0x3ffd4f50   68 3a 20 31 33 33 36 30  33 32 0d 0a 43 6f 6e 6e  |h: <HOST_6>|
V (14334) sock_dce: 0x3ffd4f60   65 63 74 69 6f 6e 3a 20  6b 65 65 70 2d 61 6c 69  |ection: keep-ali|
V (14344) sock_dce: 0x3ffd4f70   76 65 0d 0a 58 2d 50 6f  77 65 72 65 64 2d 42 79  |ve..X-Powered-By|
V (14354) sock_dce: 0x3ffd4f80   3a 20 45 78 70 72 65 73  73 0d 0a 56 61 72 79 3a  |: <HOST_7>:|
V (14364) sock_dce: 0x3ffd4f90   20 4f 72 69 67 69 6e 0d  0a 41 63 63 65 73 73 2d  | Origin..Access-|
V (14374) sock_dce: 0x3ffd4fa0   43 6f 6e 74 72 6f 6c 2d  41 6c 6c 6f 77 2d 43 72  |Control-Allow-Cr|
V (14384) sock_dce: 0x3ffd4fb0   65 64 65 6e 74 69 61 6c  73 3a 20 74 72 75 65 0d  |edentials: true.|
V (14394) sock_dce: 0x3ffd4fc0   0a 58 2d 52 61 74 65 4c  69 6d 69 74 2d 4c 69 6d  |.X-RateLimit-Lim|
V (14404) sock_dce: 0x3ffd4fd0   69 74 3a 20 31 30 30 0d  0a 58 2d 52 61 74 65 4c  |it: 100..X-RateL|
V (14404) sock_dce: 0x3ffd4fe0   69 6d 69 74 2d 52 65 6d  61 69 6e 69 6e 67 3a 20  |imit-Remaining: |
V (14414) sock_dce: 0x3ffd4ff0   39 39 0d 0a 58 2d 52 61  74 65 4c 69 6d 69 74 2d  |99..X-RateLimit-|
V (14424) sock_dce: 0x3ffd5000   52 65 73 65 74 3a 20 31  37 35 39 32 35 31 34 36  |Reset: 175925146|
V (14434) sock_dce: 0x3ffd5010   34 0d 0a 43 6f 6e 74 65  6e 74 2d 4c 61 6e 67 75  |4..Content-Langu|
V (14444) sock_dce: 0x3ffd5020   61 67 65 3a 20 74 72 0d  0a 43 6f 6e 74 65 6e 74  |age: <HOST_8>|
V (14454) sock_dce: 0x3ffd5030   2d 44 69 73 70 6f 73 69  74 69 6f 6e 3a 20 61 74  |-Disposition: at|
D (14464) event: no handlers have been registered for event ESP_HTTP_CLIENT_EVENT:3 posted to loop 0x3ffd142c
V (14464) sock_dce: 0x3ffd4f00   74 61 63 68 6d 65 6e 74  3b 20 66 69 6c 65 6e 61  |tachment; filena|
D (14464) HTTP_CLIENT: HEADER=Date:Tue, 30 Sep 2025 16:56:43 GMT
V (14484) sock_dce: 0x3ffd4f10   6d 65 3d 4e 4f 4b 4b 4f  4e 5f 41 54 53 5f 76 31  |me=FIRMWARE_FILE|
D (14494) HTTP_CLIENT: HEADER=Content-Type:application/octet-stream
V (14494) sock_dce: 0x3ffd4f20   5f 32 5f 32 31 2e 62 69  6e 0d 0a 0d 0a 0d 0a 4f  |_vX_Y_Z.bin....O|
D (14504) HTTP_CLIENT: HEADER=Content-Length:1336032
V (14514) sock_dce: 0x3ffd4f30   4b 0d 0a                                          |K..|
D (14524) HTTP_CLIENT: HEADER=Connection:keep-alive
I (14524) sock_commands: Data received by func recv!
D (14534) HTTP_CLIENT: HEADER=X-Powered-By:Express
D (14534) sock_commands: C_A_R RESPONDER! tachment; filename=<FIRMWARE_FILE_vX_Y_Z>.bin


OK

D (14544) HTTP_CLIENT: HEADER=Vary:Origin
D (14504) event: no handlers have been registered for event ESP_HTTP_CLIENT_EVENT:3 posted to loop 0x3ffd142c
D (14554) HTTP_CLIENT: HEADER=Access-Control-Allow-Credentials:true
D (14564) event: no handlers have been registered for event ESP_HTTP_CLIENT_EVENT:3 posted to loop 0x3ffd142c
D (14574) HTTP_CLIENT: HEADER=X-RateLimit-Limit:100
D (14584) event: no handlers have been registered for event ESP_HTTP_CLIENT_EVENT:3 posted to loop 0x3ffd142c
D (14584) HTTP_CLIENT: HEADER=X-RateLimit-Remaining:99
D (14594) event: no handlers have been registered for event ESP_HTTP_CLIENT_EVENT:3 posted to loop 0x3ffd142c
D (14604) HTTP_CLIENT: HEADER=X-RateLimit-Reset:1759251464
D (14614) event: no handlers have been registered for event ESP_HTTP_CLIENT_EVENT:3 posted to loop 0x3ffd142c
D (14614) HTTP_CLIENT: HEADER=Content-Language:tr
D (14624) event: no handlers have been registered for event ESP_HTTP_CLIENT_EVENT:3 posted to loop 0x3ffd142c
D (14634) HTTP_CLIENT: HEADER=Content-Disposition:attachment; filename=<FIRMWARE_FILE_vX_Y_Z>.bin
D (14644) event: no handlers have been registered for event ESP_HTTP_CLIENT_EVENT:3 posted to loop 0x3ffd142c
D (14654) HTTP_CLIENT: http_on_headers_complete, status=200, offset=409, nread=409
D (14664) event: no handlers have been registered for event ESP_HTTP_CLIENT_EVENT:3 posted to loop 0x3ffd142c
D (14674) HTTP_CLIENT: http_on_message_complete, parser=0x3ffb7aec
D (14684) event: no handlers have been registered for event ESP_HTTP_CLIENT_EVENT:3 posted to loop 0x3ffd142c
D (14684) HTTP_CLIENT: content_length = 1336032
D (14694) event: no handlers have been registered for event ESP_HTTP_CLIENT_EVENT:3 posted to loop 0x3ffd142c
D (14704) HTTP_CLIENT: Read finish or server requests close
D (14714) event: no handlers have been registered for event ESP_HTTP_CLIENT_EVENT:3 posted to loop 0x3ffd142c
D (14714) esp-tls: host:<HOST_1>: strlen 19
W (14724) sock_dce: peer closed
D (14734) esp-tls: [sock=55] Resolved IPv4 address: <IP_LOCAL>
D (14734) sock_commands: start_sending!!!
V (14744) sock_dce: 0x3ffd4f00   0d 0a 45 52 52 4f 52 0d  0a                       |..ERROR..|
D (14744) esp-tls: Enable TCP keep alive. idle: 5, interval: 5, count: 3
D (14724) event: no handlers have been registered for event ESP_HTTP_CLIENT_EVENT:3 posted to loop 0x3ffd142c
D (14764) esp-tls: [sock=55] Connecting to server. HOST: <HOST_1>, Port: 80
D (14774) event: no handlers have been registered for event ESP_HTTP_CLIENT_EVENT:5 posted to loop 0x3ffd142c
D (14784) HTTP_CLIENT: Write header[3]: GET /ota/download HTTP/1.1
User-Agent: ESP32 HTTP Client/1.0
Host: <HOST_1>
Range: bytes=0-511


D (14784) event: no handlers have been registered for event ESP_HTTP_CLIENT_EVENT:6 posted to loop 0x3ffd142c
D (14814) event: no handlers have been registered for event ESP_HTTP_CLIENT_EVENT:1 posted to loop 0x3ffd142c
D (14824) event: no handlers have been registered for event ESP_HTTP_CLIENT_EVENT:2 posted to loop 0x3ffd142c
I (19744) modem_client: State=5 fail=0 backoff=5```

bcebesoy Oct 13, 2025
Author

Hi David,

Here I need to say, there is no more need to apply native OTA to my application with esp_modem library. I had applied a custom OTA stack to my application that communicates with http and partial http download. Also external lwip dns hook is a must to solve dns the http url to loopback address in my situation. It works for me.

If it's needed for anyone I will share my experience.

Thanks a lot for your support.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

How to connect native OTA stack with LocalHost Listener [esp_modem] #820

Uh oh!

{{title}}

Uh oh!

Replies: 2 comments 4 replies

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

How to connect native OTA stack with LocalHost Listener [esp_modem] #820

Uh oh!

bcebesoy Jun 11, 2025

Replies: 2 comments · 4 replies

Uh oh!

david-cermak Jun 11, 2025 Maintainer

Uh oh!

bcebesoy Jun 11, 2025 Author

Uh oh!

david-cermak Jun 11, 2025 Maintainer

Uh oh!

david-cermak Jun 12, 2025 Maintainer

Uh oh!

bcebesoy Sep 30, 2025 Author

Uh oh!

bcebesoy Oct 13, 2025 Author

bcebesoy
Jun 11, 2025

Replies: 2 comments 4 replies

david-cermak
Jun 11, 2025
Maintainer

bcebesoy
Jun 11, 2025
Author

david-cermak Jun 11, 2025
Maintainer

david-cermak Jun 12, 2025
Maintainer

bcebesoy Sep 30, 2025
Author

bcebesoy Oct 13, 2025
Author