illusory-mapp/README.md

Illusory MAPP

Internal automation platform for aggregating device data into one backend, then running coordinated automations on top of that data.

Current Goal (Server-Side Stage 1)

Build the server-side ingestion and admin visibility for:

1. Device registration
2. Device heartbeat/ping tracking
3. SMS ingestion (ongoing)
4. Media metadata ingestion (initial sync)
5. Admin UI to inspect devices, SMS, and media

Mobile app implementation is out of scope for now.

Scope and Non-Goals

In scope now

• Processor endpoints that receive mobile data
• Domain logic + persistence for devices, SMS, media sync metadata
• Admin UI read views with polling for SMS (every 5 seconds)
• Observability for ingestion and read flows (logs + traces)

Not in scope now

• Mobile app UX/screens or client implementation
• Automation execution workflows
• Advanced media processing pipelines

Delivery Plan (Actionable Tasks)

Use this as the step-by-step implementation checklist.

Phase 0: Align Critical Decisions (before coding)

• Confirm device identity strategy (deviceId from mobile vs server-generated key).
• Confirm admin ownership rule for newly registered devices (single admin user mapping).
• Confirm SMS dedup strategy (client message id, hash, or (deviceId + timestamp + sender + body)).
• Confirm media sync contract: metadata-only now vs metadata + upload references.
• Confirm minimal auth for processor endpoints (shared token or signed header) for Stage 1.

Decisions captured:

• Keep two identifiers per device:
  • id (server-generated primary key)
  • externalDeviceId (sent by mobile)
• All devices are owned by the single admin user in Stage 1.
• SMS dedup uses:
  • optional client message id when available
  • fallback deterministic hash from (deviceId + timestamp + sender + body).
• Media sync is metadata plus upload reference (fileId) to the file domain/R2 object.
• Stage 1 endpoint auth:
  • register is open in trusted network
  • ping/sync endpoints must include device id header; request is allowed only if device exists.

Phase 1: Data Model and DB Migrations

Target: packages/db/schema/* + new migration files.

• Add mobile_device table:
  • Fields: id, externalDeviceId, name, manufacturer, model, androidVersion, ownerUserId, lastPingAt, createdAt, updatedAt.
• Add mobile_sms table:
  • Fields: id, deviceId, externalMessageId?, sender, recipient?, body, sentAt, receivedAt?, rawPayload?, createdAt.
• Add mobile_media_asset table:
  • Fields: id, deviceId, externalMediaId?, mimeType, filename?, capturedAt?, sizeBytes?, hash?, metadata, createdAt.
• Add indexes for common reads:
  • mobile_device.lastPingAt
  • mobile_sms.deviceId + sentAt desc
  • mobile_media_asset.deviceId + createdAt desc
• Export schema from packages/db/schema/index.ts.

Definition of done:

• Tables and indexes exist in schema + migration files.
• Naming matches existing conventions.

Phase 2: Logic Domain (@pkg/logic)

Target: packages/logic/domains/mobile/* (new domain).

• Create data.ts with Valibot schemas and inferred types for:
  • register device payload
  • ping payload
  • sms sync payload
  • media sync payload
  • admin query filters/pagination
• Create errors.ts with domain error constructors using getError.
• Create repository.ts with ResultAsync operations for:
  • upsert device
  • update last ping
  • bulk insert sms (idempotent)
  • bulk insert media metadata (idempotent)
  • list devices
  • get device detail
  • list sms by device (paginated, latest first)
  • list media by device (paginated, latest first)
  • delete single media asset
  • delete device (removed all child sms, and media assets, and by extensionn the associated files in r2+db as well)
• Create controller.ts as use-case layer.
• Wrap repository/controller operations with traceResultAsync and include fctx.

Definition of done:

• Processor and main app can consume this domain without direct DB usage.
• No ad-hoc thrown errors in domain flow; Result pattern is preserved.

Phase 3: Processor Ingestion API (apps/processor)

Target: apps/processor/src/domains/mobile/router.ts.

• Replace stub endpoints with full handlers:
  • POST /register
  • PUT /ping
  • PUT /sms/sync
  • PUT /media/sync
• Validate request body using schemas from @pkg/logic/domains/mobile/data.
• Build FlowExecCtx per request (flow id always; user/session when available).
• Call mobile controller methods; return { data, error } response shape.
• Add basic endpoint protection agreed in Phase 0.
• Add request-level structured logging for success/failure.

Definition of done:

• Endpoints persist data into new tables.
• Endpoint failures return normalized domain errors.

Phase 4: Admin UI in apps/main

Target:

• apps/main/src/lib/domains/mobile/* (new)

• apps/main/src/routes/(main)/devices/* (new)

• Add remote functions:

  • getDevicesSQ
  • getDeviceDetailSQ
  • getDeviceSmsSQ
  • getDeviceMediaSQ

• Add VM (devices.vm.svelte.ts) that:

  • fetches devices list
  • fetches selected device detail
  • polls SMS every 5 seconds while device view is open
  • handles loading/error states

• Add UI pages/components:

  • /devices list with device identity + last ping
  • /devices/[deviceId] detail with tabs:
    • Device info
    • SMS feed
    • Media assets list

• Add sidebar/navigation entry for Devices.

Definition of done:

• Admin can browse devices and open each device detail.
• SMS view refreshes every 5 seconds and shows new data.

Phase 5: Observability Stage 1

Targets:

• packages/logic/core/observability.ts (use existing helpers)

• Processor/mobile domain handlers and repository/controller paths

• Add span names for key flows:

  • mobile.register
  • mobile.ping
  • mobile.sms.sync
  • mobile.media.sync
  • mobile.devices.list
  • mobile.device.detail

• Add structured domain events with device id, counts, durations.

• Ensure errors include flowId consistently.

Definition of done:

• Can trace one request from processor endpoint to DB operation via shared flowId.

Phase 6: Handoff Readiness (Team Test Phase)

• Prepare endpoint payload examples for mobile team.
• Document pagination/sorting contract for admin UI queries.
• Document dedup behavior for SMS/media ingestion.
• Provide a short operator checklist:
  • register device
  • verify ping updates
  • verify sms appears in admin
  • verify media appears in admin

Suggested Build Order

1. Phase 0
2. Phase 1
3. Phase 2
4. Phase 3
5. Phase 4
6. Phase 5
7. Phase 6