TemaDeveloper / TemaDeveloper/personal_planner
feat(onboarding): Attach images in onboarding chat for vision-based understanding
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 2
- Forks
- 4
- PR merge metrics
- No merged PRs in 30d
Description
Size: L — headline onboarding improvement.
Why
Onboarding is a pure text interview (ONBOARDING_SYSTEM_PROMPT, conversational-onboarding.tsx — the composer is a single text input, no attach affordance). People can describe their life far faster by showing it: a photo of their paper planner, a screenshot of a budgeting spreadsheet, a gym program, a class timetable, a meds list, a whiteboard of goals. Letting them attach images makes the derived life-profile richer and the generated sections fit better on the first try.
Scope
- Client (
src/components/onboarding/conversational-onboarding.tsx): add an attach button + thumbnail previews + remove; allow paste/drag-drop; send image refs alongside the text turn. - Upload: reuse the existing private-Blob path (
/api/notes/upload: image-only, 10MB, ownership-scoped serving via/api/notes/blob) — or a parallelonboarding/<userId>/...namespace. Don't send raw base64 through the chat body. - API (
src/app/api/profile/chat/route.ts): acceptimageson the turn; pass them to the vision-capable extraction + reply calls (depends on multimodalcallAI— see the multimodal-callAI issue). - Extraction (
src/lib/profile/facet-extract.ts): feed images so facets are derived from what's shown, not just typed. - Update
ONBOARDING_SYSTEM_PROMPTto invite "type or show me" and to reason over attached images.
Notes
- Vision inference on multiple images is slow — respect/raise
maxDuration(currently 60s) and stream or show progress. - Privacy: onboarding images can be sensitive (finances, health) — keep them private-access, scoped to the user, and consider deleting them after profile extraction.
Depends on: multimodal callAI. Pairs with: seed-from-artifact, confirm-understanding.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with src/components/onboarding/conversational-onboarding.tsx, /api/notes/upload, /api/notes/blob, src/app/api/profile/chat/route.ts, and src/lib/profile/facet-extract.ts. First verify the multimodal callAI dependency and existing private Blob flow. Done means image selection, previews, paste and drop work through extraction and replies while access remains user-scoped and the onboarding prompt handles images.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- nextjs, react, typescript
- Domain
- ai, api, backend, frontend, full-stack
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100