2nd time now v3.8.2 simply stopped working

[m0wlheld]

DESCRIPTION

I have a Gateway E32 V2 connected to a Buderus WLW176i heatpump. It's in read-only mode, as I only want to monitor/record values using Home Assistant.

The Gateway E32 is connected via service cable to the EMS socket, no additional power supply connected.

With 3.8.1 the setup was working flawlessly for month, yesterday the 2nd time after upgrading to 3.8.2 the system simply stopped:

• no messages are sent to MQTT and consequently, Home Assistant reports all entities as unavailable
• the device is still connected to WIFI, but does not accept request
• the LED on the device is solid green

Simply reconnecting the service bus cable does bring everything back to life.

REQUESTED INFORMATION

Make sure your have performed every step and checked the applicable boxes before submitting your issue. Thank you!

• Searched the issue in issues
• Searched the issue in discussions
• Searched the issue in the docs
• Searched the issue in the chat
• Provide the System information in the area below, taken from http://<IP>/api/system

{
    "system": {
        "version": "3.8.2",
        "uptime": "000+00:02:49.453",
        "uptimeSec": 169,
        "resetReason": "Power on reset / APP CPU reset by PRO CPU",
        "platform": "ESP32",
        "cpuType": "ESP32-D0WD-V3",
        "arduino": "ESP32 Arduino v2.0.17",
        "sdk": "v4.4.7-dirty",
        "freeMem": 150,
        "maxAlloc": 107,
        "freeCaps": 4156,
        "usedApp": 2378,
        "freeApp": 2294,
        "partition": "app1",
        "flash_chip_size": 16384,
        "psram": true,
        "psramSize": 4093,
        "freePsram": 4039,
        "model": "BBQKees Electronics E32V2.2 rev.2.2/2025112",
        "txpause": false,
        "gpios_allowed": "2, 4, 5, 14, 32, 34, 36, 39",
        "gpios_in_use": "4, 5, 14, 32, 34, 36, 39",
        "gpios_available": "2"
    },
    "network": {
        "network": "WiFi",
        "hostname": "ems-esp",
        "RSSI": -53,
        "WIFIReconnects": 0,
        "BSSID": "set",
        "TxPowerSetting": 0,
        "staticIP": false,
        "lowBandwidth": false,
        "disableSleep": false,
        "enableMDNS": true,
        "enableCORS": false
    },
    "ntp": {
        "NTPstatus": "connected",
        "enabled": true,
        "server": "de.pool.ntp.org",
        "tzLabel": "Europe/Berlin",
        "timestamp": 1780138347,
        "NTPStatus": "connected"
    },
    "ap": {
        "provisionMode": "disconnected",
        "ssid": "ems-esp",
        "security": "wpa2",
        "channel": 1,
        "ssidHidden": false,
        "maxClients": 4,
        "localIP": "192.168.4.1",
        "gatewayIP": "192.168.4.1",
        "subnetMask": "255.255.255.0"
    },
    "mqtt": {
        "MQTTStatus": "connected",
        "MQTTPublishes": 343,
        "MQTTQueued": 0,
        "MQTTPublishFails": 0,
        "MQTTReconnects": 0,
        "enabled": true,
        "clientID": "esp32-c1215788",
        "keepAlive": 60,
        "cleanSession": false,
        "entityFormat": 3,
        "base": "ems-esp",
        "discoveryPrefix": "homeassistant",
        "discoveryType": 0,
        "nestedFormat": 1,
        "haEnabled": true,
        "mqttQos": 0,
        "mqttRetain": false,
        "publishTimeHeartbeat": 60,
        "publishTimeBoiler": 10,
        "publishTimeThermostat": 10,
        "publishTimeSolar": 10,
        "publishTimeMixer": 10,
        "publishTimeWater": 10,
        "publishTimeOther": 60,
        "publishTimeSensor": 10,
        "publishSingle": false,
        "publish2command": false,
        "sendResponse": false
    },
    "syslog": {
        "enabled": false
    },
    "modbus": {
        "enabled": false
    },
    "sensor": {
        "temperatureSensors": 1,
        "temperatureSensorReads": 33,
        "temperatureSensorFails": 0
    },
    "analog": {
        "enabled": false
    },
    "api": {
        "APICalls": 24,
        "APIFails": 2
    },
    "bus": {
        "busStatus": "connected",
        "busProtocol": "Buderus",
        "busTelegramsReceived": 937,
        "busReads": 242,
        "busWrites": 0,
        "busIncompleteTelegrams": 0,
        "busReadsFailed": 0,
        "busWritesFailed": 0,
        "busRxLineQuality": 100,
        "busTxLineQuality": 100
    },
    "settings": {
        "boardProfile": "E32V2_2",
        "locale": "de",
        "txMode": 1,
        "emsBusID": 11,
        "showerTimer": false,
        "showerMinDuration": 180,
        "showerAlert": false,
        "hideLed": false,
        "noTokenApi": false,
        "readonlyMode": true,
        "fahrenheit": false,
        "dallasParasite": false,
        "boolFormat": 1,
        "boolDashboard": 1,
        "enumFormat": 1,
        "analogEnabled": false,
        "telnetEnabled": false,
        "maxWebLogBuffer": 50,
        "modbusEnabled": false,
        "forceHeatingOff": false,
        "developerMode": false
    },
    "devices": [
        {
            "type": "boiler",
            "name": "XCU_THH/CS5800i, CS6800i, WLW176i, WLW186i",
            "deviceID": "0x08",
            "productID": 8,
            "brand": "Buderus",
            "version": "00.00",
            "entities": 185,
            "handlersReceived": "0xBF 0xC6 0x14 0x15 0xD1 0xE3 0xE4 0xE5 0xE9 0x0494 0x0495 0x048F",
            "handlersFetched": "0xE6 0xEA 0xEB 0x02CC 0x048D 0x048A 0x04A2 0x0485 0x0486 0x0492 0x0488 0x0484 0x048B 0x0491 0x0499 0x049C 0x049D 0x04AE 0x04AF 0x04A5 0x04AA 0x02D6",
            "handlersPending": "0x10 0x11 0xC2 0x1C 0x18 0x19 0x1A 0x35 0x34 0x2A 0x02E0",
            "handlersIgnored": "0xE7 0x02EA 0x061E 0x023E 0x0240 0x0291 0x0292 0x0293 0x0294 0x029B 0x029C 0x029D 0x029E 0x02A5 0x02F5 0x0490 0x04A8 0x04A9 0x04AB 0x04AC 0x04CA 0x04CB 0x04D8"
        },
        {
            "type": "thermostat",
            "name": "BC400-HP/Rego 3000, UI800, Logamatic BC400",
            "deviceID": "0x10",
            "productID": 253,
            "brand": "",
            "version": "47.12",
            "entities": 80,
            "handlersReceived": "0x06 0x02BA 0x02BB 0x02BC 0x031D 0x0267 0xC0",
            "handlersFetched": "0x02A5 0x02B9 0x02AF 0x029B 0x0467 0x0291 0x0292 0x0293 0x0294 0x02F5 0x023A 0x0240 0x023E 0x016E 0x0470",
            "handlersPending": "0xA3 0xA2 0x12 0x13 0x02A6 0x02B0 0x029C 0x0472 0x02A7 0x02B1 0x029D 0x0473 0x02A8 0x02B2 0x029E 0x0474 0x02A9 0x02BD 0x02B3 0x029F 0x0475 0x02AA 0x02BE 0x02B4 0x02A0 0x0476 0x02AB 0x02BF 0x02B5 0x02A1 0x0477 0x02AC 0x02C0 0x02B6 0x02A2 0x0478 0x0422 0x0468 0x0423 0x0469 0x0424 0x046A 0x031E 0xBF",
            "handlersIgnored": "0x0506 0x0507 0x0508 0x0509 0x050A 0x059D"
        },
        {
            "type": "gateway",
            "name": "MX400/MX400",
            "deviceID": "0x48",
            "productID": 17,
            "brand": "Buderus",
            "version": "14.00",
            "entities": 0,
            "handlersIgnored": "0xF9"
        },
        {
            "type": "gateway",
            "name": "MX400/MX400",
            "deviceID": "0x4B",
            "productID": 17,
            "brand": "",
            "version": "14.00",
            "entities": 0
        },
        {
            "type": "temperaturesensor",
            "name": "temperaturesensor",
            "entities": 1
        },
        {
            "type": "scheduler",
            "name": "scheduler",
            "entities": 1
        }
    ]
}

TO REPRODUCE

Nothing. It just stopps for no reason after working for approx. 1 week

EXPECTED BEHAVIOUR

Continue working, as it did with 3.8.1

SCREENSHOTS

If applicable, add screenshots to help explain your issue.

ADDITIONAL CONTEXT

Add any other context about the issue here.

(Please remember to close the issue when it has been addressed)

[proddy]

Sorry to hear that. Could you confirm a few things for us?

• are you positive EMS-ESP is not restarting? The uptime in the log you sent showed 2 minutes since last reboot
• when it becomes inactive, does it respond to a ping command?

You will need to help us pinpoint the root cause of this problem. Unfortunately, with the information provided, it's hard to determine the root cause. The first step I would take is to rule out memory-related issues by disabling MQTT and monitoring, for example.

read-only mode is not common and sometimes misunderstood. If you're not using EMS-ESP or HA to set values I would turn this off and see if it makes a difference.

[m0wlheld]

I had to restart it for getting the requested system info. When it's unresponsive, it's no longer accessible by Web UI.

My (UniFi) network reports the device still connected to the designated WiFi access point.

Will try to ping it, next time this happens again.

[MichaelDvP]

You can try to access via serial console when it happens again. With show log you can see the buffered log messages, show system reports the connection status.
As far as i see, only a firmware upload or onlineupdate stops the webUI and mqtt service and wait for upload complete.
Have you started an upload and this failed or was stoppt?

[m0wlheld]

No, I did not upload / update firmware during that (or the previous) event.

I have enabled console now, so next time I may be able to provide more info.

[proddy]

The WebUI should also be reachable during a firmware download or upload. There's an edge case if an unsupported GPIO is used but I don't believe you're using any analog sensors.

How long after a cold boot does this start happening?

And when you notice it, is there anything active (like a live Web connection in a browser) or a scheduler event is running?

[MichaelDvP]

@proddy I mean if SYSTEM_STATUS_ERROR_UPLOAD is set, it blocks mqtt and other services, but i can't find where it is set back to normal or restart.

[proddy]

@proddy I mean if SYSTEM_STATUS_ERROR_UPLOAD is set, it blocks mqtt and other services, but i can't find where it is set back to normal or restart.

ah yes, you're right. We should change this so the loop always runs, and if there is SYSTEM_STATUS_INVALID_GPIO or SYSTEM_STATUS_ERROR_UPLOAD report it and reset it on the next pass. Shall I do this in core3?

[MichaelDvP]

Yes, please do this. I don't understand this state, all errors are already reported if this is set, imo we can set in emsesp.cpp direkt to normal again.

[m0wlheld]

The WebUI should also be reachable during a firmware download or upload. There's an edge case if an unsupported GPIO is used but I don't believe you're using any analog sensors.

Nothing connected to the appliance but the EMS Bus Service cable.

How long after a cold boot does this start happening?

No fixed time. If I remember correctly, after installing 3.8.2 it took quite a while to the first crash. The 2nd one was faster (couple of days)

And when you notice it, is there anything active (like a live Web connection in a browser) or a scheduler event is running?

The only purpose of my installation is to read and forward via MQTT to Home Assistant for dashboards. I barely open the WebUI nor any other active connection. I do however regulary post data to a public site using a plan:

Looking at the Home Assistant graph of "EMS ESP Uptime", I see that the 2nd crash was on May 29th, 20:26:28 CEST. There was no noteable increase/decrease in resources (Free Memory, Max Alloc) or RX/TX failures. It just stopped working. I also checked, there was not MQTT service interruption during that timeframe.

[proddy]

Yes, please do this. I don't understand this state, all errors are already reported if this is set, imo we can set in emsesp.cpp direkt to normal again.

The SYSTEM_STATUS_ERROR_UPLOAD and all other system status are used in SystemMonitor.tsx, to notify the WebUI. I will look into a way to reset it.

[proddy]

I see that the 2nd crash was on May 29th, 20:26:28 CEST

@m0wlheld It's not crashing and restarting is it, it's just freezing all comms (web, mqtt). Could you disabled that Scheduler command and see if it runs for a few days without interuption? It will help us pin point the issue. The discussion above around the system status etc I don't think is related to your issue.

[MichaelDvP]

@proddy I don't think the system status causes this issue, but as we discussed it here: I've made a change in MichaelDvP@7349bc9

• reset after send in webStatusService
• if webStatus isalreadyclosed in browser, reset it 5 minutes after set.
• only stop services for firmware upload.

Do you think this is ok?

[proddy]

@proddy I don't think the system status causes this issue, but as we discussed it here: I've made a change in MichaelDvP@7349bc9

• reset after send in webStatusService
• if webStatus isalreadyclosed in browser, reset it 5 minutes after set.
• only stop services for firmware upload.

Do you think this is ok?

yes, nice. That'll work.

[m0wlheld]

Okay, it happened again, EMS ESP was "offline":

• it stopped yesterday evening, June 9th 19:26:57 CEST, almost exactly 3 weeks since last time.
• MQTT (Mosquitto) reported: "1781027787: Client esp32-c1215788 has exceeded timeout, disconnecting"
• Home Assistant reported all EMS ESP related entities as "unavailable" at the same time (due to the MQTT disconnect)
• WebUI is currently not accessible
• UniFi Controller reports device still be connected to WIFI, but no traffic
• the device's LED is green

I missed the change to login via telnet console, as my client device was on the wrong VLAN and console therefore seemed to be inaccesible. Next time I know better.

Finally, I did reset the device by unplugging / replugging the EMS service cable.

To my surprise, according to the UniFi Activity Log, the device executed the scheduled plan even after it became inaccessible, but with a delay of 7mins:

Will keep the Schedule active in favour of hopefully getting logs via console next time.

[MichaelDvP]

I missed the change to login via telnet console, as my client device was on the wrong VLAN and console therefore seemed to be inaccesible. Next time I know better.

I think telnet via IP will also not work if ping/web is not working. You have to use serial line, connect via USB to a laptop with a terminal at 115200 baud serial.