feat: merge IP-scanned and Proxmox-imported devices by MAC

Reconcile the same physical device discovered by both the nmap IP scan and
the Proxmox importer into a single inventory row, keyed on MAC. Previously
each path only deduped by IP, and the importer captured no MAC (and no IP for
stopped guests), so most guests double-listed.

Backend:
- mac_utils.normalize_mac: canonical MAC (lowercase, ':'-separated), the
  cross-source join key. Normalized on write and on compare.
- proxmox_service: capture the guest NIC MAC agent-free from the net0 config
  (qemu virtio=<MAC>, lxc hwaddr=<MAC>); works for stopped guests. Resolver
  now returns (ip, mac).
- proxmox persist: match existing Node/PendingDevice by ieee OR ip OR MAC;
  fill mac, keep the vm/lxc type, union sources.
- scanner persist: match PendingDevice by ip OR MAC; fill the IP a Proxmox
  import lacked, keep a pve row's type, union the scan source. Stamp query
  matches raw + normalized MAC (legacy-safe).
- Multi-source tags: new PendingDevice.discovery_sources JSON column so a
  merged device shows under both the IP and Proxmox filters. Idempotent
  migration backfills from discovery_source (legacy NULL-scalar rows with an
  IP become ["arp"]). _sources_after_merge preserves a scanned row's IP origin
  through the merge without tagging a pure Proxmox guest.
- Import now broadcasts a scan update on completion so an open inventory
  reloads without a manual refresh.

Frontend:
- pendingSources: sourceBuckets/orderedSources map discovery_sources to filter
  buckets; a device with ["arp","proxmox"] matches both filters and renders
  both badges. PendingDevicesModal filter + badges use them.

Tests: MAC normalization, config MAC capture, cross-source merge both
directions, legacy-row IP-tag preservation, no-false-IP-tag guard, refresh
broadcast, and the frontend bucket mapping.

ha-relevant: maybe
This commit is contained in:
Pouzor
2026-07-06 20:11:03 +02:00
parent 05ef746f22
commit adf82f8f01
16 changed files with 572 additions and 76 deletions
+31
View File
@@ -321,6 +321,37 @@ async def init_db() -> None:
with suppress(OperationalError):
sql = "UPDATE edges SET animated = 'none' WHERE animated = '0' OR animated = 0 OR animated IS NULL"
await conn.exec_driver_sql(sql)
# Multi-source discovery tags: a device found by both an IP scan and a
# Proxmox import carries every source. Backfill from the legacy single
# discovery_source so existing rows show under their filter.
with suppress(OperationalError):
await conn.exec_driver_sql("ALTER TABLE pending_devices ADD COLUMN discovery_sources JSON")
with suppress(OperationalError):
await conn.exec_driver_sql(
"UPDATE pending_devices SET discovery_sources = json_array(discovery_source) "
"WHERE discovery_sources IS NULL AND discovery_source IS NOT NULL"
)
# Legacy IP-scanned rows predating discovery_source have a NULL scalar
# but a real IP — treat them as an ARP scan so they keep the IP tag.
with suppress(OperationalError):
await conn.exec_driver_sql(
"UPDATE pending_devices SET discovery_sources = json_array('arp') "
"WHERE discovery_sources IS NULL AND discovery_source IS NULL AND ip IS NOT NULL"
)
with suppress(OperationalError):
await conn.exec_driver_sql(
"UPDATE pending_devices SET discovery_sources = '[]' WHERE discovery_sources IS NULL"
)
# Canonicalize stored MACs (lowercase, ':' separators) so cross-source
# dedup can match a Proxmox NIC MAC against an ARP-scanned one by equality.
with suppress(OperationalError):
await conn.exec_driver_sql(
"UPDATE pending_devices SET mac = lower(replace(mac, '-', ':')) WHERE mac IS NOT NULL"
)
with suppress(OperationalError):
await conn.exec_driver_sql(
"UPDATE nodes SET mac = lower(replace(mac, '-', ':')) WHERE mac IS NOT NULL"
)
async def get_db() -> AsyncGenerator[AsyncSession, None]:
+6
View File
@@ -119,7 +119,13 @@ class PendingDevice(Base):
services: Mapped[list[Any]] = mapped_column(JSON, default=list)
suggested_type: Mapped[str | None] = mapped_column(String)
status: Mapped[str] = mapped_column(String, default="pending")
# Origin/primary source (first discovery): "arp"/"mdns"/"zigbee"/"zwave"/
# "proxmox". Kept for back-compat; `discovery_sources` is the full set.
discovery_source: Mapped[str | None] = mapped_column(String)
# All sources that have observed this device. A device found by both an IP
# scan and a Proxmox import carries e.g. ["arp", "proxmox"] and shows under
# both inventory filters. Source of truth for the frontend source badges.
discovery_sources: Mapped[list[Any]] = mapped_column(JSON, default=list)
ieee_address: Mapped[str | None] = mapped_column(String, index=True, nullable=True, unique=True)
friendly_name: Mapped[str | None] = mapped_column(String, nullable=True)
device_subtype: Mapped[str | None] = mapped_column(String, nullable=True)