Reconcile the same physical device discovered by both the nmap IP scan and
the Proxmox importer into a single inventory row, keyed on MAC. Previously
each path only deduped by IP, and the importer captured no MAC (and no IP for
stopped guests), so most guests double-listed.
Backend:
- mac_utils.normalize_mac: canonical MAC (lowercase, ':'-separated), the
cross-source join key. Normalized on write and on compare.
- proxmox_service: capture the guest NIC MAC agent-free from the net0 config
(qemu virtio=<MAC>, lxc hwaddr=<MAC>); works for stopped guests. Resolver
now returns (ip, mac).
- proxmox persist: match existing Node/PendingDevice by ieee OR ip OR MAC;
fill mac, keep the vm/lxc type, union sources.
- scanner persist: match PendingDevice by ip OR MAC; fill the IP a Proxmox
import lacked, keep a pve row's type, union the scan source. Stamp query
matches raw + normalized MAC (legacy-safe).
- Multi-source tags: new PendingDevice.discovery_sources JSON column so a
merged device shows under both the IP and Proxmox filters. Idempotent
migration backfills from discovery_source (legacy NULL-scalar rows with an
IP become ["arp"]). _sources_after_merge preserves a scanned row's IP origin
through the merge without tagging a pure Proxmox guest.
- Import now broadcasts a scan update on completion so an open inventory
reloads without a manual refresh.
Frontend:
- pendingSources: sourceBuckets/orderedSources map discovery_sources to filter
buckets; a device with ["arp","proxmox"] matches both filters and renders
both badges. PendingDevicesModal filter + badges use them.
Tests: MAC normalization, config MAC capture, cross-source merge both
directions, legacy-row IP-tag preservation, no-false-IP-tag guard, refresh
broadcast, and the frontend bucket mapping.
ha-relevant: maybe
Cluster edges created via the pending -> approve path rendered on the top
handle instead of left/right, because the edge and its endpoints lost their
handle information on the way to the canvas.
- Approve resolver (scan.py) now returns each edge's type + source/target
handle. Handle IDs are the bare slot-0 side names ('right'/'left'), the
canonical stored form React Flow resolves to the correct side; a '-t' target
id fails to resolve and falls back to the top handle.
- Frontend injectAutoEdges no longer hardcodes iot/bottom/top-t. It injects
each edge with its real type + handles and bumps the referenced nodes'
left/right handle counts (which default to 0) so the cluster endpoints exist.
Logic extracted to a pure, tested util (applyAutoEdges).
- clusterEdges direct-import path uses the bare 'left' target to match.
Tests: new autoEdges unit tests; updated backend handle assertions.
ha-relevant: maybe
Scan History:
- Proxmox is now a first-class scan kind (badge, filter chip, Server icon,
completion toast) instead of being mislabeled as an IP scan.
- A done run carrying a non-fatal advisory renders amber (info) with a
warning toast, distinct from red failures.
Import diagnostics:
- test-connection probes /access/permissions and warns when the API token
has no ACL (VMs/LXC would be invisible) — points at the PVEAuditor grant.
- import surfaces an advisory when hosts import but no guests are visible
(privilege-separated token whose rights are the intersection with the user),
rather than a silent "done".
Node style:
- Proxmox container mode is now opt-in (container_mode === true), matching the
rest of the codebase (App.tsx nesting logic). Imported proxmox nodes leave
the flag unset and render like a manually-created node instead of an empty
group container.
Cluster edges:
- Hosts from one import are chained with 'cluster' edges via left/right handles,
distinct from the vertical host->guest 'virtual' edges. Wired for both the
direct "Add to Canvas" path and the pending -> approve path (host<->host
proxmox_cluster links, resolved to cluster edges on approve; cluster hosts
get left/right handles).
Tests added on both sides.
ha-relevant: maybe
Add a Proxmox VE importer that reads the /api2/json REST API with a read-only
API token and drops hosts (proxmox), VMs (vm) and LXC containers (lxc) onto the
canvas as typed nodes with run state and hardware specs (vCPU/RAM/disk).
- Backend: proxmox_service (httpx) + proxmox routes (test-connection, import,
import-pending, config). Two-tier dedupe — merge onto an existing scanned node
by IP, else synthetic pve-{host}-{vmid} identity. Update-in-place, never
deletes. Host->guest rendered as a 'virtual' edge via the pending-link flow.
- Security: token is env-only (PROXMOX_TOKEN_*), never written to disk by the
app, never returned by any endpoint; errors are credential-sanitized.
- Auto-sync: optional scheduled re-import into pending (APScheduler job).
- PendingDevice.properties carries specs through approve (+ migration).
- Frontend: ProxmoxImportModal, sidebar entry, pending inventory source filter,
Settings auto-sync section, proxmoxApi client.
- Docs: docs/proxmox-import.md, README + FEATURES sections, .env.example keys.
- Tests: backend service/router/scheduler, frontend modal/client/pending.
ha-relevant: maybe
A device already on a canvas (Node exists) but with no pending_devices row
never showed in the discovery inventory — the inventory lists pending_devices,
not nodes. This stranded legacy auto-placed coordinators: on a canvas yet
invisible in the inventory list.
Both mesh imports now, in the already-approved-node branch, ensure an
inventory row exists: create one as status="approved" when missing, or
refresh its metadata (preserving status) when present. New/unplaced devices
still land as status="pending"; hidden rows stay hidden.
Tests updated: approved-node path now backfills an approved inventory row
(zigbee + zwave), plus a regression that a hidden inventory row is not
revived.
ha-relevant: yes
The coordinator was special-cased in both mesh imports to auto-create a
canvas Node, so it never appeared in the pending inventory and users could
not approve/hide/type it like every other device.
Remove the auto-placement: the coordinator now flows through the shared
pending path (upsert into pending_devices with suggested_type
zigbee_coordinator / zwave_coordinator). An already-approved coordinator
Node still gets its properties refreshed on re-import via the shared
approved-node path. The response's coordinator/coordinator_already_existed
fields are retained (now always unset) for backward-compatible shape; the
frontend already ignored them.
Tests updated: coordinator lands in pending (counts include it), no Node
auto-created, pending metadata carried, approved coordinator refresh + no
pending re-list.
ha-relevant: yes
Zigbee and Z-Wave imports looked up canvas nodes by ieee_address with
scalar_one_or_none(), assuming one node per IEEE globally. A device placed
on two designs (one Node per canvas — a supported feature) made re-import
crash with MultipleResultsFound.
- Both mesh imports now refresh properties on every matching node instead
of a single row (loop over .scalars().all()).
- approve_device guards against a true duplicate: same IEEE already on the
SAME design reuses that node instead of inserting a second one.
- New node_dedupe service: loss-free repair keyed on (ieee, design_id).
Collapses only genuine same-canvas duplicates (merges properties/services/
missing fields, re-points edges + parent_id, drops self-loops/parallel
edges). Cross-design placements preserved. Runs at start of both imports
and bulk-approve. No-op on healthy DBs.
- IP correlation path already handled multiple nodes; left unchanged.
Tests: dedupe unit tests (collapse, cross-design preservation, edge/parent
re-point, idempotent), zigbee + zwave multi-canvas regression, approve
no-dupe guard.
ha-relevant: yes
- Render floor plan inside React Flow ViewportPortal so it pans/zooms with
nodes (was screen-fixed, desynced on pan/zoom); zoom-stable resize handles.
- Move floor plan config from the left panel into the canvas (design) edit
modal; attach per-design and fix cross-design bleed on load.
- Store images via a new generic backend media endpoint (POST/GET/DELETE
/api/v1/media) on disk under <data_dir>/uploads, not base64 in the canvas.
- Disable floor plans in standalone mode (no backend to upload/serve); drop
base64 localStorage persistence. See ADR-001 in CLAUDE.md.
- Tests: backend media route, DesignModal floor plan + upload, store floorMap.
ha-relevant: maybe
create_node/create_edge persisted rows with design_id=null when the
client omitted it (the MCP write tools), so they existed in the DB but
never rendered on the canvas until a container restart reconciled them.
Both routes now fall back to the first design, matching bulk-approve.
Also fix MCP resource reads (homelable://canvas, homelable://edges):
the framework passes a pydantic AnyUrl, not a str, which raised
"'AnyUrl' object has no attribute 'startswith'". Coerce to str.
EdgeResponse now exposes design_id for symmetry with NodeResponse.
ha-relevant: no
Stop button had no effect: run_scan only checked the cancel flag between
CIDR ranges and between hosts, never inside the per-range nmap call. For a
single /24 the whole scan is one blocking call, so cancel was ignored for
minutes and the run status stayed 'running'.
- thread run_id into _nmap_scan/_ping_sweep/_nmap_port_scan; check cancel
before each phase and skip queued hosts once cancelled
- flip ScanRun status to 'cancelled' eagerly in the stop endpoint so the UI
reacts immediately instead of waiting for a checkpoint
Fixes#218
ha-relevant: yes
Bulk-approve filtered status=='pending', so a device already approved onto
another canvas (status is global, canvas membership is per-design) — or whose
node was later deleted — was silently skipped. Selecting 64 devices on an
empty canvas produced only the ~28 still pending.
Approve now places a node on the target design for any selected, non-hidden
device that isn't already on that design (deduped by ip/ieee_address, including
within the batch). Returned device_ids/node_ids stay index-aligned so the
client places them all.
ha-relevant: yes
Surface the same lifecycle timestamps on the Device Inventory cards as on the
detail panel. Tiles for devices placed on a canvas show their linked node's
created / last scan / last modified / last seen (correlated by ip or
ieee_address, aggregated across matches: created = oldest, others = newest).
Devices not yet on any canvas fall back to their discovered_at.
Rendered as compact relative times ("2d ago") with the full date on hover, in
a tight two-column footer so the tile keeps its original footprint.
Backend: PendingDeviceResponse gains node_created_at / node_last_scan /
node_last_modified / node_last_seen; the canvas correlation now also pulls node
timestamps in the same single query. Frontend: shared timeFormat util
(absolute + relative), reused by the detail panel.
ha-relevant: maybe
Approve (single + bulk) created the canvas Node under the first design
instead of the design the user is viewing, so approved devices were
invisible on the active canvas and got wiped on the next save — and a
re-approve returned "0 approved" because the rows were already approved.
- bulk-approve now accepts design_id; the UI sends the active design.
- single approve already honoured design_id; the UI now sends it too.
- generalize the wireless branch (status=online, mesh props, no ICMP
check) to Z-Wave as well as Zigbee, using build_zwave_properties.
ha-relevant: yes
Import a Z-Wave JS UI (zwavejs2mqtt) network over the MQTT gateway API,
mirroring the existing Zigbee pipeline:
- New Z-Wave Import modal + sidebar entry (broker, prefix, gateway name)
- coordinator/router/end-device typing with mesh tree from node neighbors
- import to Pending section or straight to canvas
- Pending Devices gains a Z-Wave source filter
- shared mqtt_common helpers extracted from the zigbee service
ha-relevant: yes
Reworks the "Pending Devices" panel into a "Device Inventory": scanned
devices already placed on a canvas are no longer suppressed — they stay
listed and badged with how many canvases they appear on.
- scanner: stop deleting/skipping on-canvas IPs (hidden still suppressed)
- scan API: /pending returns all non-hidden devices; compute canvas_count
by correlating ip/ieee_address against nodes grouped by design
- frontend: rename to "Device Inventory", top-right canvas-count corner,
toggle to show/hide on-canvas devices (default show)
ha-relevant: maybe
Adds scanner_http_ranges / scanner_http_probe_enabled / scanner_http_verify_tls
to Settings (persisted in scan_config.json, Options page defaults). /scan/trigger
accepts an optional body to override these per-scan; /scan/config GET/POST read
and persist the defaults. Port ranges validated at the API boundary.
ha-relevant: yes
The status WebSocket pool removed connections with list.remove(), which
raises ValueError on a double-remove (broadcast already dropped a dead
socket, then disconnect tries again), and only released a slot on
WebSocketDisconnect — any other error leaked the socket into the
broadcast pool. Centralise removal in an idempotent _drop() called from
a finally block and from _broadcast.
ha-relevant: yes
Adds optional live status checking per service (not just per node),
requested as a follow-up to issue #196.
Backend:
- New check_service / check_services: HTTP(S) GET for web services, TCP
connect otherwise; UDP and port-less non-web services stay 'unknown'.
- New scheduler job 'service_checks', independent interval (default 300s),
added/removed live via set_service_checks_enabled.
- Settings gain service_check_enabled + service_check_interval (>=30s),
persisted to scan_config.json. New WS message type 'service_status'.
Frontend:
- Live per-service status overlay in canvasStore (not persisted, so it
never round-trips through canvas save), fed by the WS message.
- DetailPanel + canvas node service rows: offline service turns red
(#f85149), otherwise keeps its category colour.
- SettingsModal: toggle + interval input (default 300s / 5 min).
Off by default — no behaviour change until enabled.
ha-relevant: yes
Add an authenticated GET /api/v1/liveview/config endpoint exposing the
configured LIVEVIEW_KEY to logged-in admins, so the UI builds a
ready-to-use share link. The header View button opens
/view?key=...&design=<activeDesignId>; LiveView forwards ?design=<id>
to the public endpoint, which renders that design's canvas.
ha-relevant: maybe
Make designs (canvases) fully user-manageable: create with a chosen name
and icon, rename, change icon, and delete. Replaces the hardcoded
"New Electrical Design" button with a generic "New Canvas" flow.
- Add Design.icon column + migration that backfills legacy rows
(electrical -> zap, others -> dashboard)
- DesignModal: name input + curated lucide icon picker (create + edit)
- Sidebar switcher gains per-canvas edit/delete; delete guards the last
canvas and confirms
- designStore: addDesign/updateDesign/removeDesign with active reassignment
- Fix data loss on design switch: abort load when the save fails and keep
unsaved edits; skip the save-old step when the previous canvas was deleted
- designsApi create/update carry icon; design_type kept for back-compat
Tests: backend design CRUD (icon + cascade + last-canvas guard), designStore
actions, designIcons resolver, DesignModal create/edit/validation.
ha-relevant: yes
The network scan captures each device's MAC, but approving a pending
device dropped it: the new Node had mac=NULL and no MAC in its
properties pane. Now both approve paths (single + bulk) copy the
pending device's MAC to Node.mac and add a MAC row to the node
properties so it shows in the standard properties pane with toggleable
visibility (hidden by default, consistent with zigbee props).
- build_mac_property: MAC NodeProperty row (hidden by default)
- merge_mac_property: append MAC to user-supplied props, idempotent,
preserves existing visibility
- single approve prefers the pending device MAC, falls back to payload
Adds unit tests for the helpers and API tests for both approve paths.
ha-relevant: yes
When a Zigbee device was approved (PendingDevice.status=approved + Node
created) and the user later deleted the canvas Node, the PendingDevice row
was left orphaned as approved. On re-import _persist_pending_import found
the existing row, refreshed its fields but left status=approved, so it
never reappeared in the Pending list (which filters status==pending) —
the import reported devices found but Pending stayed empty.
Reset such orphaned approved rows back to pending on re-import. Hidden
devices stay hidden (explicit user intent). Adds regression tests for
both revive-approved and keep-hidden paths.
ha-relevant: yes
- Live view rebuilt parent map from proxmox/group only, so docker_container
(and any vm/lxc with container_mode=true) lost parentId and rendered
outside its host. Match App.tsx logic: include any node with
container_mode=true.
- Live view ignored viewport.theme_id and custom_style on load, so the
shared canvas always rendered in default theme. Apply both like the
authenticated canvas does.
- Backend /liveview response now includes custom_style from CanvasState.
Adds regression tests on both ends.
Adds GET /api/v1/stats/summary, gated by HOMEPAGE_API_KEY env var and
X-API-Key header (hmac.compare_digest, matches the liveview pattern).
Payload:
nodes / online / offline / unknown
pending_devices (status='pending')
zigbee_devices (Node.ieee_address IS NOT NULL)
last_scan_at (max ScanRun.finished_at)
Disabled by default — endpoint returns 403 unless HOMEPAGE_API_KEY is set.
README documents activation and ships a ready-to-paste gethomepage
`customapi` widget snippet.
Tests cover: disabled-by-default, missing header, wrong key, empty DB,
and full aggregation across nodes/pending/zigbee/scan-runs.
- New zigbee props (approve + first-time re-import) ship with visible=false
so the canvas card stays clean. User opts in from the right panel.
- On re-import of an already-approved node, merge instead of overwrite:
keys already present keep their visible flag (and any user-edited value
is replaced with the freshly imported one), brand-new keys are appended
hidden. Non-zigbee custom properties are preserved untouched.
- Approval (single + bulk) of zigbee pending devices now writes IEEE, Vendor,
Model, LQI into Node.properties so they show in the right panel.
- Zigbee re-import refreshes properties on existing canvas Nodes and skips
creating a pending row when the device was already approved — keeps
approved devices out of pending/hidden modals.
- Coordinator Node also receives the same properties on first creation and
on re-import.
- Remove the Parent Container selector from the add/edit node modal.
Zigbee nodes have no IP-based check — hide Check Method and Check Target
fields in NodeModal for all three zigbee types. Default and force
check_method='none' so the scheduler marks them always online.
Backend approve and zigbee import routes also set status='online' and
check_method='none' for zigbee node types.
Approving a pending device created a Node with check_method=NULL, so
the status scheduler silently skipped it (scheduler.py:64 filters
falsy check_method) and the bubble stayed grey forever.
Default to 'ping' when the device has an IP, in both single and bulk
approve paths. Also persist caller-supplied check_method/check_target
in the single-approve path.
Add regression tests asserting default check_method='ping' after
approval for both endpoints.
Mirrors IP scan flow: POST /zigbee/import-pending now creates a
ScanRun(kind=zigbee, status=running) and returns immediately.
Networkmap fetch + pending upsert run in the background, status
transitions to done/error when finished.
Frontend: import modal closes on submit, scan history shows the
run with a ZIG/IP kind chip and toasts on completion. Pending
modal auto-refreshes when run finishes.
scan_runs.kind column added (default 'ip', idempotent migration).
Existing zigbee tests refactored to exercise _persist_pending_import
directly (background tasks don't see the test session); route test
verifies the run is created with kind=zigbee.
Replaces sidebar pending/hidden panels with a wide modal showing
devices as cards. Adds search, segmented source/status filters,
type filter, select mode for bulk approve/hide/restore, and
keyboard shortcuts. Hidden cards click-to-restore (no approval
detour); approval no longer pops the edit modal.
Backend: new restore + bulk-restore endpoints (hidden -> pending).
Coordinator auto-approves to a canvas Node; routers/end devices land in
pending_devices keyed by IEEE. Discovered parent->child edges are stored
in pending_device_links so that approving a pending device later
auto-creates the Edge once both endpoints exist as canvas Nodes.
- new POST /api/v1/zigbee/import-pending (default mode in modal)
- new pending_device_links table; ieee_address on nodes + pending_devices
- pending_devices.ip migrated to nullable (table rebuild on existing DBs)
- approve / bulk-approve return auto-created edges; sidebar pushes them
into the canvas store with bottom -> top-t handles
- ZigbeeImportModal: radio toggle pending vs canvas; reset on close
- PendingDeviceModal: zigbee badge, IEEE/LQI/vendor/model rows, services
hidden for zigbee
- Sidebar pending row: ZIG source badge, LQI badge, friendly_name fallback
- SearchBar: null-safe IP, also searches friendly_name and ieee_address
- Tooltip trigger uses asChild to avoid nested-button hydration error
The previous parser read `data.routes` which is just an echo of the
`routes` request flag (a boolean). On real brokers this caused
`TypeError: 'bool' object is not iterable` and 500s during /import.
- Rewrite parse_networkmap to read data.value.nodes + data.value.links
with fallback to data.{nodes,links} for legacy variants
- Defensive: drop links to unknown nodes, propagate lqi from link to
target node, extract model/vendor from definition block
- Bump networkmap timeout 10s -> 180s (large meshes are slow)
- Tests: rewrite fixture builders + sample payload to real Z2M shape;
add cases for legacy shape, routes:false echo (regression), malformed
list, link to unknown node, lqi propagation, definition extraction
- Update docs to mention 60s+ wait window
53 backend tests pass, mypy + ruff clean.
aiomqtt/paho exception strings can include the broker URI with embedded
credentials (mqtt://user:pass@host) or auth detail. The 502 response
from /import and the message field on /test-connection echoed these
verbatim via str(exc).
- Add _sanitize_mqtt_error() that maps known patterns (auth, refused,
DNS, TLS, timeout) to coarse, credential-free categories
- Original exception still logged at WARNING level for operator debug
- Drop hostname:port from TimeoutError messages
- /test-connection unexpected-error path no longer interpolates exc
Tests: 6 new (auth/refused/DNS/TLS/unknown sanitization, end-to-end
fetch_networkmap leak check).
- Schema: mqtt_tls + mqtt_tls_insecure flags on import + test-connection
requests; model_validator enforces insecure requires tls
- Service: _build_tls_context() using ssl.create_default_context();
logger.warning when verification disabled; tls_context plumbed into
aiomqtt.Client for both fetch_networkmap and test_mqtt_connection
- Route: passes tls flags through to service
- Frontend: TLS checkbox auto-toggles port 1883<->8883 unless user
edited; insecure checkbox disabled until TLS on, red-tinted; password
field marked autocomplete=new-password
- Tests: 7 new backend tests (TLS context build, client kwargs assertion,
router happy path, insecure-without-tls 422)
- db.flush() ensures node.id is populated before reading — fixes bulk approve
where node_ids were null, causing frontend to skip addNode for every device
- approve_device raises 404 on missing device, 409 on already-processed device
- _background_scan rollbacks dirty session then marks run as "failed"
- Explicit Node() field mapping instead of **model_dump() to prevent injection
- update_scan_config rolls back in-memory change if save_overrides() fails
- clear_pending uses bulk DELETE instead of N individual row deletes
- Backend: POST /scan/pending/bulk-approve and /scan/pending/bulk-hide endpoints (registered before dynamic routes to avoid conflict); bulk-approve response includes device_ids for frontend mapping
- Frontend: PendingDevicesPanel gains per-row checkboxes, select-all, and a bulk action bar (Approve N / Hide N) that appears when ≥1 device is selected
- Tests: 6 new backend API tests + 7 new frontend UI tests for bulk selection flows
- POST /scan/{run_id}/stop endpoint signals running scan to cancel
- Scanner checks cancellation flag between CIDR ranges and hosts, exits early
- Cancelled scans get status 'cancelled' instead of 'done'
- Stop button (red StopCircle) shown in Scan History panel for running scans
- 6 new backend tests, 5 new frontend tests
- New GET/POST /api/v1/settings for status check interval
- Scan /api/v1/scan/config now handles ranges only
- Frontend: settingsApi client, SettingsPanel uses settingsApi
- ScanConfigModal no longer reads/writes interval
- 4 new backend tests for settings endpoint
Pouzor