Mirror the Proxmox auto-sync pattern for the Zigbee2MQTT and Z-Wave JS
UI mesh imports. Connection config + MQTT credentials live in .env only,
are never persisted to scan_config.json, and are never returned by any
API or shown in the UI (single source of truth).
- config: ZIGBEE_* / ZWAVE_* env settings; only sync_enabled+interval
are persisted, connection/credentials stay env-only
- routes: GET/POST /config, POST /sync-now; auto-sync reuses the exact
manual _background_*_import + _persist_pending_import path (fresh
import when empty, update-in-place when nodes exist, ScanRun trace)
- scheduler: zigbee_sync / zwave_sync jobs with live enable + reschedule
- frontend: reusable MeshAutoSync section in Settings (Zigbee, Z-Wave)
- .env.example: documented both blocks
- tests: scheduler jobs, router config/sync-now/auth, credential-never-
persisted, SettingsModal sections
Manual Zigbee/Z-Wave import behaviour is unchanged.
ha-relevant: no
A device already on a canvas (Node exists) but with no pending_devices row
never showed in the discovery inventory — the inventory lists pending_devices,
not nodes. This stranded legacy auto-placed coordinators: on a canvas yet
invisible in the inventory list.
Both mesh imports now, in the already-approved-node branch, ensure an
inventory row exists: create one as status="approved" when missing, or
refresh its metadata (preserving status) when present. New/unplaced devices
still land as status="pending"; hidden rows stay hidden.
Tests updated: approved-node path now backfills an approved inventory row
(zigbee + zwave), plus a regression that a hidden inventory row is not
revived.
ha-relevant: yes
The coordinator was special-cased in both mesh imports to auto-create a
canvas Node, so it never appeared in the pending inventory and users could
not approve/hide/type it like every other device.
Remove the auto-placement: the coordinator now flows through the shared
pending path (upsert into pending_devices with suggested_type
zigbee_coordinator / zwave_coordinator). An already-approved coordinator
Node still gets its properties refreshed on re-import via the shared
approved-node path. The response's coordinator/coordinator_already_existed
fields are retained (now always unset) for backward-compatible shape; the
frontend already ignored them.
Tests updated: coordinator lands in pending (counts include it), no Node
auto-created, pending metadata carried, approved coordinator refresh + no
pending re-list.
ha-relevant: yes
Zigbee and Z-Wave imports looked up canvas nodes by ieee_address with
scalar_one_or_none(), assuming one node per IEEE globally. A device placed
on two designs (one Node per canvas — a supported feature) made re-import
crash with MultipleResultsFound.
- Both mesh imports now refresh properties on every matching node instead
of a single row (loop over .scalars().all()).
- approve_device guards against a true duplicate: same IEEE already on the
SAME design reuses that node instead of inserting a second one.
- New node_dedupe service: loss-free repair keyed on (ieee, design_id).
Collapses only genuine same-canvas duplicates (merges properties/services/
missing fields, re-points edges + parent_id, drops self-loops/parallel
edges). Cross-design placements preserved. Runs at start of both imports
and bulk-approve. No-op on healthy DBs.
- IP correlation path already handled multiple nodes; left unchanged.
Tests: dedupe unit tests (collapse, cross-design preservation, edge/parent
re-point, idempotent), zigbee + zwave multi-canvas regression, approve
no-dupe guard.
ha-relevant: yes
When a Zigbee device was approved (PendingDevice.status=approved + Node
created) and the user later deleted the canvas Node, the PendingDevice row
was left orphaned as approved. On re-import _persist_pending_import found
the existing row, refreshed its fields but left status=approved, so it
never reappeared in the Pending list (which filters status==pending) —
the import reported devices found but Pending stayed empty.
Reset such orphaned approved rows back to pending on re-import. Hidden
devices stay hidden (explicit user intent). Adds regression tests for
both revive-approved and keep-hidden paths.
ha-relevant: yes
- New zigbee props (approve + first-time re-import) ship with visible=false
so the canvas card stays clean. User opts in from the right panel.
- On re-import of an already-approved node, merge instead of overwrite:
keys already present keep their visible flag (and any user-edited value
is replaced with the freshly imported one), brand-new keys are appended
hidden. Non-zigbee custom properties are preserved untouched.
- Approval (single + bulk) of zigbee pending devices now writes IEEE, Vendor,
Model, LQI into Node.properties so they show in the right panel.
- Zigbee re-import refreshes properties on existing canvas Nodes and skips
creating a pending row when the device was already approved — keeps
approved devices out of pending/hidden modals.
- Coordinator Node also receives the same properties on first creation and
on re-import.
- Remove the Parent Container selector from the add/edit node modal.
Zigbee nodes have no IP-based check — hide Check Method and Check Target
fields in NodeModal for all three zigbee types. Default and force
check_method='none' so the scheduler marks them always online.
Backend approve and zigbee import routes also set status='online' and
check_method='none' for zigbee node types.
Mirrors IP scan flow: POST /zigbee/import-pending now creates a
ScanRun(kind=zigbee, status=running) and returns immediately.
Networkmap fetch + pending upsert run in the background, status
transitions to done/error when finished.
Frontend: import modal closes on submit, scan history shows the
run with a ZIG/IP kind chip and toasts on completion. Pending
modal auto-refreshes when run finishes.
scan_runs.kind column added (default 'ip', idempotent migration).
Existing zigbee tests refactored to exercise _persist_pending_import
directly (background tasks don't see the test session); route test
verifies the run is created with kind=zigbee.
Coordinator auto-approves to a canvas Node; routers/end devices land in
pending_devices keyed by IEEE. Discovered parent->child edges are stored
in pending_device_links so that approving a pending device later
auto-creates the Edge once both endpoints exist as canvas Nodes.
- new POST /api/v1/zigbee/import-pending (default mode in modal)
- new pending_device_links table; ieee_address on nodes + pending_devices
- pending_devices.ip migrated to nullable (table rebuild on existing DBs)
- approve / bulk-approve return auto-created edges; sidebar pushes them
into the canvas store with bottom -> top-t handles
- ZigbeeImportModal: radio toggle pending vs canvas; reset on close
- PendingDeviceModal: zigbee badge, IEEE/LQI/vendor/model rows, services
hidden for zigbee
- Sidebar pending row: ZIG source badge, LQI badge, friendly_name fallback
- SearchBar: null-safe IP, also searches friendly_name and ieee_address
- Tooltip trigger uses asChild to avoid nested-button hydration error
The previous parser read `data.routes` which is just an echo of the
`routes` request flag (a boolean). On real brokers this caused
`TypeError: 'bool' object is not iterable` and 500s during /import.
- Rewrite parse_networkmap to read data.value.nodes + data.value.links
with fallback to data.{nodes,links} for legacy variants
- Defensive: drop links to unknown nodes, propagate lqi from link to
target node, extract model/vendor from definition block
- Bump networkmap timeout 10s -> 180s (large meshes are slow)
- Tests: rewrite fixture builders + sample payload to real Z2M shape;
add cases for legacy shape, routes:false echo (regression), malformed
list, link to unknown node, lqi propagation, definition extraction
- Update docs to mention 60s+ wait window
53 backend tests pass, mypy + ruff clean.
aiomqtt/paho exception strings can include the broker URI with embedded
credentials (mqtt://user:pass@host) or auth detail. The 502 response
from /import and the message field on /test-connection echoed these
verbatim via str(exc).
- Add _sanitize_mqtt_error() that maps known patterns (auth, refused,
DNS, TLS, timeout) to coarse, credential-free categories
- Original exception still logged at WARNING level for operator debug
- Drop hostname:port from TimeoutError messages
- /test-connection unexpected-error path no longer interpolates exc
Tests: 6 new (auth/refused/DNS/TLS/unknown sanitization, end-to-end
fetch_networkmap leak check).
- Schema: mqtt_tls + mqtt_tls_insecure flags on import + test-connection
requests; model_validator enforces insecure requires tls
- Service: _build_tls_context() using ssl.create_default_context();
logger.warning when verification disabled; tls_context plumbed into
aiomqtt.Client for both fetch_networkmap and test_mqtt_connection
- Route: passes tls flags through to service
- Frontend: TLS checkbox auto-toggles port 1883<->8883 unless user
edited; insecure checkbox disabled until TLS on, red-tinted; password
field marked autocomplete=new-password
- Tests: 7 new backend tests (TLS context build, client kwargs assertion,
router happy path, insecure-without-tls 422)
Pouzor