198 lines
6.7 KiB
Markdown
198 lines
6.7 KiB
Markdown
# Illusory MAPP
|
|
|
|
Internal automation platform for aggregating device data into one backend, then running coordinated automations on top of that data.
|
|
|
|
## Current Goal (Server-Side Stage 1)
|
|
|
|
Build the server-side ingestion and admin visibility for:
|
|
|
|
1. Device registration
|
|
2. Device heartbeat/ping tracking
|
|
3. SMS ingestion (ongoing)
|
|
4. Media metadata ingestion (initial sync)
|
|
5. Admin UI to inspect devices, SMS, and media
|
|
|
|
Mobile app implementation is out of scope for now.
|
|
|
|
## Scope and Non-Goals
|
|
|
|
### In scope now
|
|
|
|
- Processor endpoints that receive mobile data
|
|
- Domain logic + persistence for devices, SMS, media sync metadata
|
|
- Admin UI read views with polling for SMS (every 5 seconds)
|
|
- Observability for ingestion and read flows (logs + traces)
|
|
|
|
### Not in scope now
|
|
|
|
- Mobile app UX/screens or client implementation
|
|
- Automation execution workflows
|
|
- Advanced media processing pipelines
|
|
|
|
## Delivery Plan (Actionable Tasks)
|
|
|
|
Use this as the step-by-step implementation checklist.
|
|
|
|
### Phase 0: Align Critical Decisions (before coding)
|
|
|
|
- [x] Confirm device identity strategy (`deviceId` from mobile vs server-generated key).
|
|
- [x] Confirm admin ownership rule for newly registered devices (single admin user mapping).
|
|
- [x] Confirm SMS dedup strategy (client message id, hash, or `(deviceId + timestamp + sender + body)`).
|
|
- [x] Confirm media sync contract: metadata-only now vs metadata + upload references.
|
|
- [x] Confirm minimal auth for processor endpoints (shared token or signed header) for Stage 1.
|
|
|
|
Decisions captured:
|
|
|
|
- Keep two identifiers per device:
|
|
- `id` (server-generated primary key)
|
|
- `externalDeviceId` (sent by mobile)
|
|
- All devices are owned by the single admin user in Stage 1.
|
|
- SMS dedup uses:
|
|
- optional client message id when available
|
|
- fallback deterministic hash from `(deviceId + timestamp + sender + body)`.
|
|
- Media sync is metadata plus upload reference (`fileId`) to the file domain/R2 object.
|
|
- Stage 1 endpoint auth:
|
|
- register is open in trusted network
|
|
- ping/sync endpoints must include device id header; request is allowed only if device exists.
|
|
|
|
### Phase 1: Data Model and DB Migrations
|
|
|
|
Target: `packages/db/schema/*` + new migration files.
|
|
|
|
- [x] Add `mobile_device` table:
|
|
- Fields: `id`, `externalDeviceId`, `name`, `manufacturer`, `model`, `androidVersion`, `ownerUserId`, `lastPingAt`, `createdAt`, `updatedAt`.
|
|
- [x] Add `mobile_sms` table:
|
|
- Fields: `id`, `deviceId`, `externalMessageId?`, `sender`, `recipient?`, `body`, `sentAt`, `receivedAt?`, `rawPayload?`, `createdAt`.
|
|
- [x] Add `mobile_media_asset` table:
|
|
- Fields: `id`, `deviceId`, `externalMediaId?`, `mimeType`, `filename?`, `capturedAt?`, `sizeBytes?`, `hash?`, `metadata`, `createdAt`.
|
|
- [x] Add indexes for common reads:
|
|
- `mobile_device.lastPingAt`
|
|
- `mobile_sms.deviceId + sentAt desc`
|
|
- `mobile_media_asset.deviceId + createdAt desc`
|
|
- [x] Export schema from `packages/db/schema/index.ts`.
|
|
|
|
Definition of done:
|
|
|
|
- [x] Tables and indexes exist in schema + migration files.
|
|
- [x] Naming matches existing conventions.
|
|
|
|
### Phase 2: Logic Domain (`@pkg/logic`)
|
|
|
|
Target: `packages/logic/domains/mobile/*` (new domain).
|
|
|
|
- [x] Create `data.ts` with Valibot schemas and inferred types for:
|
|
- register device payload
|
|
- ping payload
|
|
- sms sync payload
|
|
- media sync payload
|
|
- admin query filters/pagination
|
|
- [x] Create `errors.ts` with domain error constructors using `getError`.
|
|
- [x] Create `repository.ts` with ResultAsync operations for:
|
|
- upsert device
|
|
- update last ping
|
|
- bulk insert sms (idempotent)
|
|
- bulk insert media metadata (idempotent)
|
|
- list devices
|
|
- get device detail
|
|
- list sms by device (paginated, latest first)
|
|
- list media by device (paginated, latest first)
|
|
- delete single media asset
|
|
- delete device (removed all child sms, and media assets, and by extensionn the associated files in r2+db as well)
|
|
- [x] Create `controller.ts` as use-case layer.
|
|
- [x] Wrap repository/controller operations with `traceResultAsync` and include `fctx`.
|
|
|
|
Definition of done:
|
|
|
|
- [x] Processor and main app can consume this domain without direct DB usage.
|
|
- [x] No ad-hoc thrown errors in domain flow; Result pattern is preserved.
|
|
|
|
### Phase 3: Processor Ingestion API (`apps/processor`)
|
|
|
|
Target: `apps/processor/src/domains/mobile/router.ts`.
|
|
|
|
- [x] Replace stub endpoints with full handlers:
|
|
- `POST /register`
|
|
- `PUT /ping`
|
|
- `PUT /sms/sync`
|
|
- `PUT /media/sync`
|
|
- [x] Validate request body using schemas from `@pkg/logic/domains/mobile/data`.
|
|
- [x] Build `FlowExecCtx` per request (flow id always; user/session when available).
|
|
- [x] Call mobile controller methods; return `{ data, error }` response shape.
|
|
- [x] Add basic endpoint protection agreed in Phase 0.
|
|
- [x] Add request-level structured logging for success/failure.
|
|
|
|
Definition of done:
|
|
|
|
- [x] Endpoints persist data into new tables.
|
|
- [x] Endpoint failures return normalized domain errors.
|
|
|
|
### Phase 4: Admin UI in `apps/main`
|
|
|
|
Target:
|
|
|
|
- `apps/main/src/lib/domains/mobile/*` (new)
|
|
- `apps/main/src/routes/(main)/devices/[deviceId]` (new)
|
|
|
|
- [x] Add remote functions:
|
|
- `getDevicesSQ`
|
|
- `getDeviceDetailSQ`
|
|
- `getDeviceSmsSQ`
|
|
- `getDeviceMediaSQ`
|
|
- [x] Add VM (`devices.vm.svelte.ts`) that:
|
|
- fetches devices list
|
|
- fetches selected device detail
|
|
- polls SMS every 5 seconds while device view is open
|
|
- handles loading/error states
|
|
- [x] Add UI pages/components:
|
|
- `/dashboard` list with device identity + last ping
|
|
- `/devices/[deviceId]` detail with tabs:
|
|
- Device info
|
|
- SMS feed
|
|
- Media assets list
|
|
- [x] Add sidebar/navigation entry for Devices.
|
|
|
|
Definition of done:
|
|
|
|
- [x] Admin can browse devices and open each device detail.
|
|
- [x] SMS view refreshes every 5 seconds and shows new data.
|
|
|
|
### Phase 5: Observability Stage 1
|
|
|
|
Targets:
|
|
|
|
- `packages/logic/core/observability.ts` (use existing helpers)
|
|
- Processor/mobile domain handlers and repository/controller paths
|
|
|
|
- [x] Add span names for key flows:
|
|
- `mobile.register`
|
|
- `mobile.ping`
|
|
- `mobile.sms.sync`
|
|
- `mobile.media.sync`
|
|
- `mobile.devices.list`
|
|
- `mobile.device.detail`
|
|
- [x] Add structured domain events with device id, counts, durations.
|
|
- [x] Ensure errors include `flowId` consistently.
|
|
|
|
Definition of done:
|
|
|
|
- [x] Can trace one request from processor endpoint to DB operation via shared `flowId`.
|
|
|
|
### Phase 6: Handoff Readiness (Team Test Phase)
|
|
|
|
- [x] Prepare endpoint payload examples for mobile team (in a `spec.mobile.md` file).
|
|
- [x] Provide a short operator checklist:
|
|
- register device
|
|
- verify ping updates
|
|
- verify sms appears in admin
|
|
- verify media appears in admin
|
|
|
|
## Suggested Build Order
|
|
|
|
1. Phase 0
|
|
2. Phase 1
|
|
3. Phase 2
|
|
4. Phase 3
|
|
5. Phase 4
|
|
6. Phase 5
|
|
7. Phase 6
|