perf: warm-boot fast path + atomic cloud request ids (#29)

* fix(protocol): allocate cloud request ids atomically

Concurrent send() calls (MQTT thread + polling + entity commands) read
self._id outside the lock and incremented it after the call, so two
in-flight requests could share the same id and the cloud rejects the
duplicate with an empty result (silent, DEBUG-only).

Also documents why the first-refresh property fetch stays sequential
with 50-key batches: the robot rejects larger get_properties payloads
(100/200 fail) and answers the cloud bridge serially, so parallel
batches bring zero wall-clock gain (measured on device, 2026-07-02).

* perf: warm-boot fast path via persisted property inventory

First refresh previously blocked on 4 serial robot round-trips
(~5-8 s depending on Dreame cloud latency). The coordinator now
persists the device's answered-property inventory (model/firmware
keyed, homeassistant.helpers.storage.Store). On warm boots only the
priority batch (every property DreameVacuumDeviceCapability.load()
reads — some flags are absence-based — plus the primary state) is
fetched synchronously; the remaining present properties load from a
background thread. Deferred entities are created identically
(existence = inventory) but stay unavailable until their first value
arrives, i.e. exactly the timeline they had before.

Cold boots (first setup, firmware change) keep the full synchronous
load and then publish the fresh inventory.

Measured on HA dev (same degraded-cloud window): cold 10.74 s ->
warm 5.84 s entry setup; priority batch 1.8-2.0 s vs 4.7-8.4 s full.
Proof of no loss: entity registry diff empty (258/258, unique_id and
disabled state included) and zero entities downgraded from valued to
unavailable/unknown across the change (146 recorder states compared).

* docs: changelog for warm-boot fast path & request-id fix

Author: foXaCe

CHANGELOG.md

@@ -5,6 +5,23 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [Unreleased]
+
+### Changed
+- Faster startup on warm boots: the integration now persists the device's
+  answered-property inventory (model/firmware keyed) and only loads the
+  priority batch (capability inputs + primary state) synchronously during
+  the first refresh; the remaining properties load in the background while
+  their entities stay unavailable until the first value arrives — the exact
+  timeline they had before. Measured on the same network window: entry
+  setup 10.7 s → 5.8 s (≈3.5-4 s under normal cloud latency). First setup
+  and firmware changes keep the full synchronous load.
+
+### Fixed
+- Cloud request ids are now allocated atomically: concurrent requests
+  (MQTT push handling, polling, entity commands) could share the same id
+  and the Dreame cloud silently rejected the duplicate (empty result).
+
 ## [6.6.0] - 2026-07-02
 
 ### Changed

custom_components/dreame_vacuum/coordinator.py

@@ -31,6 +31,7 @@
 from homeassistant.helpers.dispatcher import async_dispatcher_connect
 from homeassistant.helpers.entity import generate_entity_id
 from homeassistant.helpers.issue_registry import IssueSeverity, async_create_issue, async_delete_issue
+from homeassistant.helpers.storage import Store
 from homeassistant.helpers.update_coordinator import DataUpdateCoordinator, UpdateFailed
 
 from .const import (
@@ -571,6 +572,28 @@ def _fire_event(self, event_id: str, data: Any) -> None:
             event_data.update(data)
         self.hass.loop.call_soon_threadsafe(self.hass.bus.async_fire, f"{DOMAIN}_{event_id}", event_data)
 
+    async def _async_setup(self) -> None:
+        """Load the persisted property inventory before the first refresh.
+
+        The inventory (model/firmware keyed) lets the device fetch only the
+        priority property batch synchronously on warm boots; the rest loads
+        in the background while the affected entities stay unavailable.
+        """
+        self._inventory_store: Store[dict[str, Any]] = Store(
+            self.hass, 1, f"{DOMAIN}.{self._entry.entry_id}.property_inventory"
+        )
+        inventory = await self._inventory_store.async_load()
+        if self._device:
+            self._device.set_property_inventory(inventory, self._save_property_inventory)
+
+    def _save_property_inventory(self, inventory: dict[str, Any]) -> None:
+        """Persist the inventory (called from device worker threads)."""
+
+        def _schedule() -> None:
+            self.hass.async_create_task(self._inventory_store.async_save(inventory))
+
+        self.hass.loop.call_soon_threadsafe(_schedule)
+
     async def _async_update_data(self) -> DreameVacuumDevice:
         """Update Dreame Vacuum."""
         if self._device is None:

custom_components/dreame_vacuum/dreame/device.py

@@ -7,9 +7,9 @@
 from functools import cmp_to_key
 import json
 import logging
-from threading import Lock, RLock, Timer
+from threading import Lock, RLock, Thread, Timer
 import time
-from typing import TYPE_CHECKING, Any, cast
+from typing import TYPE_CHECKING, Any, Final, cast
 import zlib
 
 if TYPE_CHECKING:
@@ -93,6 +93,53 @@ def _get_map_module() -> Any:
 
 _LOGGER = logging.getLogger(__name__)
 
+# Properties that MUST be loaded synchronously on the first refresh:
+# everything DreameVacuumDeviceCapability.load() reads (some flags are
+# absence-based, e.g. MAP_SAVING -> lidar_navigation) plus the primary
+# user-facing state. Everything else can arrive from the background tail
+# of the warm-boot load (entities stay unavailable until their value lands).
+_PRIORITY_BOOT_PROPERTIES: Final = (
+    DreameVacuumProperty.STATE,
+    DreameVacuumProperty.STATUS,
+    DreameVacuumProperty.TASK_STATUS,
+    DreameVacuumProperty.ERROR,
+    DreameVacuumProperty.BATTERY_LEVEL,
+    DreameVacuumProperty.CHARGING_STATUS,
+    DreameVacuumProperty.CLEANING_MODE,
+    DreameVacuumProperty.SUCTION_LEVEL,
+    DreameVacuumProperty.WATER_VOLUME,
+    DreameVacuumProperty.CLEANING_TIME,
+    DreameVacuumProperty.CLEANED_AREA,
+    DreameVacuumProperty.AI_DETECTION,
+    DreameVacuumProperty.AUTO_MOUNT_MOP,
+    DreameVacuumProperty.AUTO_SWITCH_SETTINGS,
+    DreameVacuumProperty.CAMERA_LIGHT_BRIGHTNESS,
+    DreameVacuumProperty.CARPET_CLEANING,
+    DreameVacuumProperty.CARPET_RECOGNITION,
+    DreameVacuumProperty.CRUISE_SCHEDULE,
+    DreameVacuumProperty.CUSTOMIZED_CLEANING,
+    DreameVacuumProperty.DETERGENT_LEFT,
+    DreameVacuumProperty.DND,
+    DreameVacuumProperty.DND_TASK,
+    DreameVacuumProperty.DRAINAGE_STATUS,
+    DreameVacuumProperty.DUST_COLLECTION,
+    DreameVacuumProperty.MAP_BACKUP_STATUS,
+    DreameVacuumProperty.MAP_SAVING,
+    DreameVacuumProperty.MULTI_FLOOR_MAP,
+    DreameVacuumProperty.OBSTACLE_AVOIDANCE,
+    DreameVacuumProperty.OFF_PEAK_CHARGING,
+    DreameVacuumProperty.PET_DETECTIVE,
+    DreameVacuumProperty.SELF_WASH_BASE_STATUS,
+    DreameVacuumProperty.SENSOR_DIRTY_LEFT,
+    DreameVacuumProperty.SHORTCUTS,
+    DreameVacuumProperty.TASK_TYPE,
+    DreameVacuumProperty.TIGHT_MOPPING,
+    DreameVacuumProperty.VOICE_ASSISTANT,
+    DreameVacuumProperty.WETNESS_LEVEL,
+    DreameVacuumProperty.WIFI_MAP,
+)
+_PRIORITY_BOOT_DIDS: Final = frozenset(prop.value for prop in _PRIORITY_BOOT_PROPERTIES)
+
 
 class DreameVacuumDevice(
     DreameVacuumDeviceSettersMixin,
@@ -185,6 +232,12 @@ def __init__(
 
         # Startup flags: two-phase loading for faster startup
         self._full_properties_loaded = False
+        # Warm-boot fast path: persisted property inventory (model/firmware
+        # keyed, provided by the coordinator). Deferred properties keep their
+        # entities unavailable until the first value arrives.
+        self._property_inventory: dict[str, Any] | None = None
+        self._pending_properties: set[int] = set()
+        self._inventory_callback: Callable[[dict[str, Any]], None] | None = None
         self._map_initialized = False
         self._deferred_cloud_loaded = False
 
@@ -666,9 +719,15 @@ def _request_properties(
                 if "aiid" not in mapping and (force_all or not self._ready or prop.value in self.data):
                     property_list.append({"did": str(prop.value), **mapping})
 
+        # Batch size is a device-side limit: the robot rejects get_properties
+        # beyond ~50 keys (100/200 fail), and it answers the cloud bridge
+        # serially, so parallel batches do not speed anything up either
+        # (measured 2026-07-02). The first refresh is therefore bound by
+        # 4 robot round-trips (~1.2 s each) — do not "optimize" this again
+        # without re-measuring on a real device.
         props = property_list.copy()
         results = []
-        batch_size = 50  # Increased from 25 for faster startup (fewer network calls)
+        batch_size = 50
         while props:
             result = self._protocol.get_properties(props[:batch_size])
             if result is None:
@@ -681,6 +740,108 @@ def _request_properties(
 
         return self._handle_properties(results)
 
+    def set_property_inventory(
+        self,
+        inventory: dict[str, Any] | None,
+        callback: Callable[[dict[str, Any]], None] | None = None,
+    ) -> None:
+        """Provide the persisted property inventory and its save callback."""
+        self._property_inventory = inventory
+        self._inventory_callback = callback
+
+    def property_pending(self, prop: DreameVacuumProperty) -> bool:
+        """Return True while a property's first value has not arrived yet."""
+        return prop.value in self._pending_properties
+
+    @property
+    def pending_properties(self) -> set[int]:
+        """Property ids whose first value has not arrived yet."""
+        return self._pending_properties
+
+    def _request_initial_properties(self) -> None:
+        """Load properties for the first refresh.
+
+        With a persisted inventory matching the current model/firmware, only
+        the priority batch (capability inputs + primary state) is fetched
+        synchronously; the remaining present properties load from a
+        background thread while their entities stay unavailable. Without a
+        usable inventory (first setup, firmware change) the full load stays
+        synchronous and the fresh inventory is published for persistence.
+        """
+        inventory = self._property_inventory
+        usable = bool(
+            inventory
+            and self.info is not None
+            and inventory.get("model") == self.info.model
+            and inventory.get("firmware") == self.info.firmware_version
+            and inventory.get("present")
+        )
+        if not usable:
+            self._request_properties(force_all=True)
+            self._full_properties_loaded = True
+            self._publish_inventory()
+            return
+
+        present = set(inventory["present"])
+        priority: list[DreameVacuumProperty] = []
+        deferred: list[DreameVacuumProperty] = []
+        for prop in self._default_properties:
+            mapping = self.property_mapping.get(prop)
+            if mapping is None or "aiid" in mapping:
+                continue
+            if prop.value in _PRIORITY_BOOT_DIDS:
+                priority.append(prop)
+            elif prop.value in present:
+                deferred.append(prop)
+        self._pending_properties = {prop.value for prop in deferred}
+        try:
+            self._request_properties(priority, force_all=True)
+        except Exception:
+            self._pending_properties = set()
+            raise
+        Thread(target=self._load_deferred_properties, args=(deferred,), daemon=True).start()
+
+    def _load_deferred_properties(self, deferred: list[DreameVacuumProperty]) -> None:
+        """Background tail of the warm-boot load (everything after batch 1)."""
+        try:
+            batch_size = 50
+            for i in range(0, len(deferred), batch_size):
+                if self.disconnected:
+                    return
+                batch = deferred[i : i + batch_size]
+                try:
+                    self._request_properties(batch, force_all=True)
+                finally:
+                    self._pending_properties -= {prop.value for prop in batch}
+            self._full_properties_loaded = True
+            self._publish_inventory()
+        except Exception:
+            _LOGGER.warning("Deferred property load failed", exc_info=True)
+        finally:
+            if self._pending_properties:
+                # Never leave entities gated on properties nobody will load;
+                # the regular update cycle takes over from here.
+                self._pending_properties = set()
+                self._property_changed(False)
+
+    def _publish_inventory(self) -> None:
+        """Report the answered-property inventory for persistence."""
+        if self._inventory_callback is None or self.info is None:
+            return
+        mapped = [
+            prop.value
+            for prop in self._default_properties
+            if prop in self.property_mapping and "aiid" not in self.property_mapping[prop]
+        ]
+        self._inventory_callback(
+            {
+                "model": self.info.model,
+                "firmware": self.info.firmware_version,
+                "present": [did for did in mapped if did in self.data],
+                "absent": [did for did in mapped if did not in self.data],
+            }
+        )
+
     def _update_status(self, task_status: DreameVacuumTaskStatus, status: DreameVacuumStatus) -> None:
         """Update status properties on memory for map renderer to update the image before action is sent to the device."""
         if task_status is not DreameVacuumTaskStatus.COMPLETED:
@@ -1718,11 +1879,9 @@ def connect_device(self) -> None:
             self._dirty_auto_switch_data = {}
             self._dirty_ai_data = {}
 
-            # Load ALL properties in a single pass (eliminates separate capability batch overhead)
             t_props = time.time()
-            self._request_properties(force_all=True)
-            self._full_properties_loaded = True
-            _LOGGER.debug("connect_device: all properties loaded in %.2fs", time.time() - t_props)
+            self._request_initial_properties()
+            _LOGGER.debug("connect_device: initial properties loaded in %.2fs", time.time() - t_props)
             self._last_update_failed = None
 
             # Set up map manager (capabilities are now available from loaded properties)

custom_components/dreame_vacuum/dreame/protocol.py

@@ -569,6 +569,7 @@ def send_async(self, callback: Any, method: Any, parameters: Any, retry_count: i
 
         with self._id_lock:
             self._id = self._id + 1
+            request_id = self._id
         self._api_call_async(
             lambda api_response: callback(
                 None
@@ -581,10 +582,10 @@ def send_async(self, callback: Any, method: Any, parameters: Any, retry_count: i
             f"{self._strings[37]}{host}/{self._strings[27]}/{self._strings[38]}",
             {
                 "did": str(self._did),
-                "id": self._id,
+                "id": request_id,
                 "data": {
                     "did": str(self._did),
-                    "id": self._id,
+                    "id": request_id,
                     "method": method,
                     "params": parameters,
                 },
@@ -597,22 +598,26 @@ def send(self, method: Any, parameters: Any, retry_count: int = 2) -> Any:
         if self._host and len(self._host):
             host = f"-{self._host.split('.')[0]}"
 
+        # Allocate the request id atomically BEFORE building the payload:
+        # concurrent send() calls previously shared the same id (read outside
+        # the lock, incremented afterwards), and the cloud rejects duplicates.
+        with self._id_lock:
+            self._id = self._id + 1
+            request_id = self._id
         api_response = self._api_call(
             f"{self._strings[37]}{host}/{self._strings[27]}/{self._strings[38]}",
             {
                 "did": str(self._did),
-                "id": self._id,
+                "id": request_id,
                 "data": {
                     "did": str(self._did),
-                    "id": self._id,
+                    "id": request_id,
                     "method": method,
                     "params": parameters,
                 },
             },
             retry_count,
         )
-        with self._id_lock:
-            self._id = self._id + 1
         if (
             api_response is None
             or "data" not in api_response

custom_components/dreame_vacuum/entity.py

@@ -53,7 +53,11 @@ def default_exists_fn(description: Any, device: Any) -> bool:
         (description.action_key is not None and description.action_key in device.action_mapping)
         or description.property_key is None
         or (
-            isinstance(description.property_key, DreameVacuumProperty) and description.property_key.value in device.data
+            isinstance(description.property_key, DreameVacuumProperty)
+            and (
+                description.property_key.value in device.data
+                or description.property_key.value in device.pending_properties
+            )
         )
         or (
             isinstance(description.property_key, DreameVacuumAutoSwitchProperty)
@@ -404,6 +408,12 @@ def available(self) -> bool:
         if self.device is None or not self.device.device_connected:
             return False
 
+        prop = getattr(self.entity_description, "property_key", None)
+        if isinstance(prop, DreameVacuumProperty) and prop.value in self.device.pending_properties:
+            # Warm-boot: the entity exists (persisted inventory) but its first
+            # value is still loading in the background.
+            return False
+
         if self._computed_available_fn is not None:
             return self._computed_available_fn(self.device)
         return self._attr_available