Commit Graph

28 Commits

Author SHA1 Message Date
Pouzor b99450db2f feat: scheduled auto-sync for Zigbee & Z-Wave imports
Mirror the Proxmox auto-sync pattern for the Zigbee2MQTT and Z-Wave JS
UI mesh imports. Connection config + MQTT credentials live in .env only,
are never persisted to scan_config.json, and are never returned by any
API or shown in the UI (single source of truth).

- config: ZIGBEE_* / ZWAVE_* env settings; only sync_enabled+interval
  are persisted, connection/credentials stay env-only
- routes: GET/POST /config, POST /sync-now; auto-sync reuses the exact
  manual _background_*_import + _persist_pending_import path (fresh
  import when empty, update-in-place when nodes exist, ScanRun trace)
- scheduler: zigbee_sync / zwave_sync jobs with live enable + reschedule
- frontend: reusable MeshAutoSync section in Settings (Zigbee, Z-Wave)
- .env.example: documented both blocks
- tests: scheduler jobs, router config/sync-now/auth, credential-never-
  persisted, SettingsModal sections

Manual Zigbee/Z-Wave import behaviour is unchanged.

ha-relevant: no
2026-07-10 14:51:01 +02:00
Pouzor 0792105b96 feat: prompt on duplicate device instead of silently blocking/merging approve
Single-device approve now guards duplicates per-design the same way bulk
approve does, and asks the user instead of failing or silently merging:

- create_node and approve_device reject a same-design duplicate (ieee, ip
  or mac) with 409 + the existing node; a force flag creates it anyway.
- Frontend shows a confirm dialog: go to existing node, add duplicate
  anyway, or cancel.
- approve_device no longer rejects a device already on another canvas
  (status is global, canvas membership is per-design) — it can be placed
  on a new design, matching bulk approve.
- IEEE (Zigbee/Z-Wave) devices now use the same prompt as ip/mac instead
  of auto-merging into the existing node.
- bulk approve reports which devices it skipped as duplicates.

Closes #260

ha-relevant: maybe
2026-07-09 13:05:19 +02:00
Pouzor 9437a74147 feat: manual Proxmox re-sync button + make connection config env-only
Add a 'Re-sync now' button to the Proxmox auto-sync settings section that
triggers an immediate inventory import (POST /proxmox/sync-now) using the
server env config — the manual counterpart to scheduled auto-sync.

Also fix a dual-source-of-truth bug: Proxmox connection config (host, port,
token, verify_tls) is now env-only and never persisted to scan_config.json.
Previously save_overrides() dumped host/port/verify alongside scan settings,
so saving an unrelated setting wrote an empty proxmox_host that load_overrides
then clobbered PROXMOX_HOST with on every boot. Only the auto-sync activation
(sync_enabled + sync_interval) stays user-editable and persisted.

ha-relevant: no
2026-07-07 21:20:13 +02:00
Pouzor 0b4bd5680d feat: create a new canvas by copying an existing one
Add a 'Copy from existing' option to the New Canvas modal. It lists every
canvas with node/group/text counts; picking one deep-copies its nodes, edges,
parent/child links and canvas state (viewport, custom style, floor plan) into
a fresh design.

Backend: POST /designs/{source_id}/copy remaps node ids, re-points edges and
parent links, and clones canvas state; GET /designs now returns per-design
counts for the picker. Standalone mode clones the localStorage canvas.

Closes #216

ha-relevant: maybe
2026-07-07 15:53:42 +02:00
Pouzor 05ef746f22 fix: cluster edges render on left/right handles from approve flow
Cluster edges created via the pending -> approve path rendered on the top
handle instead of left/right, because the edge and its endpoints lost their
handle information on the way to the canvas.

- Approve resolver (scan.py) now returns each edge's type + source/target
  handle. Handle IDs are the bare slot-0 side names ('right'/'left'), the
  canonical stored form React Flow resolves to the correct side; a '-t' target
  id fails to resolve and falls back to the top handle.
- Frontend injectAutoEdges no longer hardcodes iot/bottom/top-t. It injects
  each edge with its real type + handles and bumps the referenced nodes'
  left/right handle counts (which default to 0) so the cluster endpoints exist.
  Logic extracted to a pure, tested util (applyAutoEdges).
- clusterEdges direct-import path uses the bare 'left' target to match.

Tests: new autoEdges unit tests; updated backend handle assertions.

ha-relevant: maybe
2026-07-06 11:13:32 +02:00
Pouzor ab36ba6f81 feat: import hosts/VMs/LXC from Proxmox VE with optional auto-sync
Add a Proxmox VE importer that reads the /api2/json REST API with a read-only
API token and drops hosts (proxmox), VMs (vm) and LXC containers (lxc) onto the
canvas as typed nodes with run state and hardware specs (vCPU/RAM/disk).

- Backend: proxmox_service (httpx) + proxmox routes (test-connection, import,
  import-pending, config). Two-tier dedupe — merge onto an existing scanned node
  by IP, else synthetic pve-{host}-{vmid} identity. Update-in-place, never
  deletes. Host->guest rendered as a 'virtual' edge via the pending-link flow.
- Security: token is env-only (PROXMOX_TOKEN_*), never written to disk by the
  app, never returned by any endpoint; errors are credential-sanitized.
- Auto-sync: optional scheduled re-import into pending (APScheduler job).
- PendingDevice.properties carries specs through approve (+ migration).
- Frontend: ProxmoxImportModal, sidebar entry, pending inventory source filter,
  Settings auto-sync section, proxmoxApi client.
- Docs: docs/proxmox-import.md, README + FEATURES sections, .env.example keys.
- Tests: backend service/router/scheduler, frontend modal/client/pending.

ha-relevant: maybe
2026-07-05 18:58:12 +02:00
Pouzor 1ed013bde2 feat: floor plan viewport rendering, per-canvas config, server media upload
- Render floor plan inside React Flow ViewportPortal so it pans/zooms with
  nodes (was screen-fixed, desynced on pan/zoom); zoom-stable resize handles.
- Move floor plan config from the left panel into the canvas (design) edit
  modal; attach per-design and fix cross-design bleed on load.
- Store images via a new generic backend media endpoint (POST/GET/DELETE
  /api/v1/media) on disk under <data_dir>/uploads, not base64 in the canvas.
- Disable floor plans in standalone mode (no backend to upload/serve); drop
  base64 localStorage persistence. See ADR-001 in CLAUDE.md.
- Tests: backend media route, DesignModal floor plan + upload, store floorMap.

ha-relevant: maybe
2026-07-02 16:36:25 +02:00
Pouzor 13420bead8 fix: approve devices onto the active design + Z-Wave node fields
Approve (single + bulk) created the canvas Node under the first design
instead of the design the user is viewing, so approved devices were
invisible on the active canvas and got wiped on the next save — and a
re-approve returned "0 approved" because the rows were already approved.

- bulk-approve now accepts design_id; the UI sends the active design.
- single approve already honoured design_id; the UI now sends it too.
- generalize the wireless branch (status=online, mesh props, no ICMP
  check) to Z-Wave as well as Zigbee, using build_zwave_properties.

ha-relevant: yes
2026-06-26 21:58:31 +02:00
Pouzor 8faf5c1c79 feat: add Z-Wave network scan via MQTT gateway
Import a Z-Wave JS UI (zwavejs2mqtt) network over the MQTT gateway API,
mirroring the existing Zigbee pipeline:

- New Z-Wave Import modal + sidebar entry (broker, prefix, gateway name)
- coordinator/router/end-device typing with mesh tree from node neighbors
- import to Pending section or straight to canvas
- Pending Devices gains a Z-Wave source filter
- shared mqtt_common helpers extracted from the zigbee service

ha-relevant: yes
2026-06-26 11:19:37 +02:00
Pouzor d01630bf37 feat: add Deep Scan toggle to scan dialog
Collapsible Deep Scan section in ScanConfigModal exposes extra port ranges,
HTTP probe and TLS-verify switches. Pre-filled from saved defaults; edits are
passed to trigger() as a per-scan override and do not change the persisted
defaults (those live in the Options/scan config). scanApi.trigger now accepts
an optional deep-scan body.

ha-relevant: yes
2026-06-24 10:31:32 +02:00
Pouzor bcc488993d feat: per-service status checks with offline colouring
Adds optional live status checking per service (not just per node),
requested as a follow-up to issue #196.

Backend:
- New check_service / check_services: HTTP(S) GET for web services, TCP
  connect otherwise; UDP and port-less non-web services stay 'unknown'.
- New scheduler job 'service_checks', independent interval (default 300s),
  added/removed live via set_service_checks_enabled.
- Settings gain service_check_enabled + service_check_interval (>=30s),
  persisted to scan_config.json. New WS message type 'service_status'.

Frontend:
- Live per-service status overlay in canvasStore (not persisted, so it
  never round-trips through canvas save), fed by the WS message.
- DetailPanel + canvas node service rows: offline service turns red
  (#f85149), otherwise keeps its category colour.
- SettingsModal: toggle + interval input (default 300s / 5 min).

Off by default — no behaviour change until enabled.

ha-relevant: yes
2026-06-10 02:22:48 +02:00
Pouzor 05fb67e7f0 feat(liveview): header View link opens active design as read-only canvas
Add an authenticated GET /api/v1/liveview/config endpoint exposing the
configured LIVEVIEW_KEY to logged-in admins, so the UI builds a
ready-to-use share link. The header View button opens
/view?key=...&design=<activeDesignId>; LiveView forwards ?design=<id>
to the public endpoint, which renders that design's canvas.

ha-relevant: maybe
2026-06-04 14:10:29 +02:00
Pouzor 12d527aad6 feat: manage canvases with custom name and icon
Make designs (canvases) fully user-manageable: create with a chosen name
and icon, rename, change icon, and delete. Replaces the hardcoded
"New Electrical Design" button with a generic "New Canvas" flow.

- Add Design.icon column + migration that backfills legacy rows
  (electrical -> zap, others -> dashboard)
- DesignModal: name input + curated lucide icon picker (create + edit)
- Sidebar switcher gains per-canvas edit/delete; delete guards the last
  canvas and confirms
- designStore: addDesign/updateDesign/removeDesign with active reassignment
- Fix data loss on design switch: abort load when the save fails and keep
  unsaved edits; skip the save-old step when the previous canvas was deleted
- designsApi create/update carry icon; design_type kept for back-compat

Tests: backend design CRUD (icon + cascade + last-canvas guard), designStore
actions, designIcons resolver, DesignModal create/edit/validation.

ha-relevant: yes
2026-06-02 15:47:57 +02:00
Pranjal Joshi 46435605eb feat: multi-design canvas system with electrical nodes/edges
Backend: New Design model + designs table; design_id FK on nodes, edges, canvas_state; migration seeds default 'Network Topology' design; full CRUD API for designs; canvas load/save accept design_id.

Frontend: designStore (Zustand), design switcher in Sidebar, design-aware canvas load/save, auto-save on design switch.

Electrical node types (14): grid, ups, battery, generator, solar_panel, inverter, circuit_breaker, contactor, electrical_switch, socket, light, meter, transformer, load — icons, registrations, accent colors in all 6 themes.

Electrical edge type: registered in edgeTypes, BASE_STYLES, edgeColors, all theme edgeColors, EDGE_DEFAULT_COLORS.

Bug fixes: data corruption on design switch (stale closure), race condition on save-then-load, missing Zap import, missing Electrical group in NodeModal, missing electrical entries in custom theme edgeColors, inline imports hoisted.
2026-05-31 14:35:22 +05:30
Pouzor 0863c2db94 feat(zigbee): import as background scan run
Mirrors IP scan flow: POST /zigbee/import-pending now creates a
ScanRun(kind=zigbee, status=running) and returns immediately.
Networkmap fetch + pending upsert run in the background, status
transitions to done/error when finished.

Frontend: import modal closes on submit, scan history shows the
run with a ZIG/IP kind chip and toasts on completion. Pending
modal auto-refreshes when run finishes.

scan_runs.kind column added (default 'ip', idempotent migration).
Existing zigbee tests refactored to exercise _persist_pending_import
directly (background tasks don't see the test session); route test
verifies the run is created with kind=zigbee.
2026-05-10 00:45:21 +02:00
Pouzor 277879488f feat(pending): full-screen modal with grid cards, filters, bulk restore
Replaces sidebar pending/hidden panels with a wide modal showing
devices as cards. Adds search, segmented source/status filters,
type filter, select mode for bulk approve/hide/restore, and
keyboard shortcuts. Hidden cards click-to-restore (no approval
detour); approval no longer pops the edit modal.

Backend: new restore + bulk-restore endpoints (hidden -> pending).
2026-05-09 23:40:44 +02:00
Pouzor 3ae159d8d6 feat(zigbee): import to pending section with edge persistence
Coordinator auto-approves to a canvas Node; routers/end devices land in
pending_devices keyed by IEEE. Discovered parent->child edges are stored
in pending_device_links so that approving a pending device later
auto-creates the Edge once both endpoints exist as canvas Nodes.

- new POST /api/v1/zigbee/import-pending (default mode in modal)
- new pending_device_links table; ieee_address on nodes + pending_devices
- pending_devices.ip migrated to nullable (table rebuild on existing DBs)
- approve / bulk-approve return auto-created edges; sidebar pushes them
  into the canvas store with bottom -> top-t handles
- ZigbeeImportModal: radio toggle pending vs canvas; reset on close
- PendingDeviceModal: zigbee badge, IEEE/LQI/vendor/model rows, services
  hidden for zigbee
- Sidebar pending row: ZIG source badge, LQI badge, friendly_name fallback
- SearchBar: null-safe IP, also searches friendly_name and ieee_address
- Tooltip trigger uses asChild to avoid nested-button hydration error
2026-05-07 23:18:29 +02:00
Pouzor d49610163f feat(zigbee): add MQTT TLS support with optional cert verify skip
- Schema: mqtt_tls + mqtt_tls_insecure flags on import + test-connection
  requests; model_validator enforces insecure requires tls
- Service: _build_tls_context() using ssl.create_default_context();
  logger.warning when verification disabled; tls_context plumbed into
  aiomqtt.Client for both fetch_networkmap and test_mqtt_connection
- Route: passes tls flags through to service
- Frontend: TLS checkbox auto-toggles port 1883<->8883 unless user
  edited; insecure checkbox disabled until TLS on, red-tinted; password
  field marked autocomplete=new-password
- Tests: 7 new backend tests (TLS context build, client kwargs assertion,
  router happy path, insecure-without-tls 422)
2026-05-06 16:30:23 +02:00
pranjal-joshi 103e24e5fa feat: add Zigbee2MQTT network map importer
- Backend: async MQTT service (aiomqtt) to fetch Z2M networkmap via bridge API
- Backend: FastAPI router at /api/v1/zigbee with /import and /test-connection
- Backend: Pydantic v2 schemas for request/response validation
- Backend: coordinator → router → end-device parent_id hierarchy builder
- Frontend: ZigbeeImportModal with MQTT config form, Test Connection, Fetch Devices
- Frontend: device list grouped by type (coordinator/router/enddevice) with checkboxes
- Frontend: ZigbeeCoordinatorNode, ZigbeeRouterNode, ZigbeeEndDeviceNode canvas nodes
- Frontend: Zigbee Import button in sidebar alongside Scan Network
- Frontend: handleZigbeeAddToCanvas wires selected devices + edges onto canvas
- Tests: full unit test suite for parser, hierarchy builder, MQTT mocks
- Tests: API endpoint tests for /zigbee/import and /zigbee/test-connection
- Tests: Vitest component tests for ZigbeeImportModal
- Docs: docs/zigbee-import.md with full usage, MQTT config, troubleshooting guide
- Docs: README.md Zigbee2MQTT Import section

Co-authored-by: CyberKeys <noreply@openclaw.ai>
2026-05-04 13:58:58 +00:00
Pouzor babbcb1dc5 feat: add custom style theme with per-type node/edge style editor
- New 'custom' ThemeId with dedicated ThemeCard + pencil edit button
- CustomStyleModal: two-column editor for all node types (excl. groupRect/group) and edge types
  - Per node: border, background, icon color + opacity, default size (w/h)
  - Per edge: color + opacity, path style, animation
  - 'Apply to existing [Type]' per type, 'Apply All to Canvas' footer action
- canvasStore: applyTypeNodeStyle, applyTypeEdgeStyle, applyAllCustomStyles actions
- themeStore: customStyle state + setCustomStyle action
- Backend: custom_style JSON column on canvas_state (ALTER TABLE migration), saved/loaded with canvas
- App.tsx: custom_style included in save payload, restored on load (API + standalone)
- Tests: +8 frontend (themeStore + canvasStore), +3 backend (canvas API)
2026-04-24 11:42:31 +02:00
Pouzor 0193f933ce feat: bulk approve/hide pending devices (#70)
- Backend: POST /scan/pending/bulk-approve and /scan/pending/bulk-hide endpoints (registered before dynamic routes to avoid conflict); bulk-approve response includes device_ids for frontend mapping
- Frontend: PendingDevicesPanel gains per-row checkboxes, select-all, and a bulk action bar (Approve N / Hide N) that appears when ≥1 device is selected
- Tests: 6 new backend API tests + 7 new frontend UI tests for bulk selection flows
2026-04-19 22:13:40 +02:00
Pouzor 38c5bcb606 feat: configurable bottom handles, scanner rewrite, UI polish
## New features
- Configurable bottom connection points per node (1–4 handles)
- Fit view on load
- LiveView improvements
- Node modal: inline Type/Icon picker, default icon in trigger
- Remove redundant Save button from ScanConfigModal

## Scanner fixes
- Phase 1: replace nmap ARP sweep with concurrent asyncio ping sweep
  (50 parallel pings, 1s timeout). Zero false positives, works in any
  Docker network mode. Supplements with /proc/net/arp for ICMP-blocked devices.
- Phase 2: explicit -sS (root) / -sT (non-root) scan type; bump
  host-timeout to 60s; gather(return_exceptions=True) so one failing
  host doesn't abort the batch
- Fix 404 on missing device in hide/ignore
- Validate CIDR ranges to prevent nmap injection
- Thread-safe cancel set, pre-fetch canvas/hidden IPs (no N+1 queries)
- Logging: attach StreamHandler to root logger so app.* logs are visible

## Tests
- 21 backend scanner tests (ping sweep, ARP cache, Phase 2 tolerance)
- Full NodeModal coverage (53 tests)
- LiveView, store, edge label tests
2026-04-04 23:15:47 +02:00
Pouzor 057891f7d5 feat: add stop scan button in UI with backend cancellation support
- POST /scan/{run_id}/stop endpoint signals running scan to cancel
- Scanner checks cancellation flag between CIDR ranges and hosts, exits early
- Cancelled scans get status 'cancelled' instead of 'done'
- Stop button (red StopCircle) shown in Scan History panel for running scans
- 6 new backend tests, 5 new frontend tests
2026-04-01 14:18:44 +02:00
Pouzor 68c7672cea feat: split scan config and app settings into separate endpoints
- New GET/POST /api/v1/settings for status check interval
- Scan /api/v1/scan/config now handles ranges only
- Frontend: settingsApi client, SettingsPanel uses settingsApi
- ScanConfigModal no longer reads/writes interval
- 4 new backend tests for settings endpoint
2026-03-30 23:06:51 +02:00
Pouzor 210304394e feat: read-only live view at /view?key=<LIVEVIEW_KEY>
Implements issue #5. Off by default; set LIVEVIEW_KEY in .env to enable.
No JWT required — key-based auth via ?key= query param.
Returns 403 when disabled or key is wrong.
Read-only ReactFlow canvas (pan/zoom, no editing).
Standalone mode loads from localStorage without a key.
Includes 8 backend tests and 9 frontend tests.
2026-03-28 15:27:54 +01:00
Pouzor 9eba62c5b5 fix: save button now fully persists canvas to DB
Root cause: handleAddNode/handleEdgeConfirm only updated Zustand store,
never creating records in the DB. canvasApi.save() only updated positions
of existing nodes, so nothing survived a page refresh.

Fix: save now sends the full canvas state (all nodes + edges + data) and
the backend does a full sync — upsert incoming, delete anything removed.
This means Save is the single source of truth: no need to call individual
create/update/delete APIs for every drag or edit.
2026-03-07 01:54:59 +01:00
Pouzor 0bd714a68b feat: Phase 3 & 4 — monitoring, discovery, polish, deployment
Phase 3 — Discovery & Monitoring:
- Network scanner: nmap wrapper + mock fallback, fingerprint service (35 signatures)
- Status checker: ping/http/https/tcp/ssh/prometheus/health per-node checks
- APScheduler: status checks every 60s, WebSocket broadcast
- WebSocket /ws/status: live node status updates to frontend
- Sidebar panels: Pending Devices, Hidden Devices, Scan History
- Auth token persisted to localStorage (survive page refresh)
- 24 new backend tests (scan flow + status_checker)

Phase 4 — Polish & Deployment:
- Auto-layout: Dagre hierarchical TB via Toolbar button
- Export PNG: html-to-image download via Toolbar button
- Scan config modal: CIDR ranges + check interval, GET/POST /api/v1/scan/config
- Dockerfile.backend (Python 3.13 slim + nmap), Dockerfile.frontend (nginx)
- docker-compose.yml with data volume and NET_RAW cap for ping
- scripts/lxc-install.sh: Proxmox VE systemd bootstrap
- README.md: quick-start, config reference, stack overview
2026-03-07 00:45:50 +01:00
Pouzor 150302c3f7 feat: Phase 2 — auth, node/edge modals, API wiring
- Login page: full-screen dark, grid bg, JWT flow
- Auth store (Zustand) + API client (axios + interceptors)
- NodeModal: add/edit form (type, label, hostname, IP, check method)
- EdgeModal: link type picker + VLAN ID + label on connect
- App: auth gate, canvas load from API (fallback to demo), Ctrl+S save
- CanvasContainer: expose onConnect prop for edge modal flow
2026-03-06 23:33:55 +01:00