feat: Native platform support — Android components for smart glasses & mobile
- Dominant language
- TypeScript
- Stars
- 13.1k
- Forks
- 1.1k
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 669
Description
## Summary
XDS currently targets web (React + StyleX). This proposal extends XDS to **native platforms** — starting with Android (Kotlin/Compose) for Meta smart glasses (Ray-Ban Meta) and the companion mobile app.
The key insight: XDS already has a clean separation between **tokens**, **theme definitions**, and **component implementations**. The `defineTheme()` → `xds theme build` pipeline resolves all tokens and component style overrides into a structured intermediate format. Today that format generates CSS. It can also generate **Kotlin**, **Swift**, or any other platform's style definitions — from the same theme source.
## Motivation
Meta smart glasses run a native Android UI on a **600×600 square display** rendered through a see-through prism. The current SystemUI uses a Panels system with Bloks-based rendering — native Android views inflated from serialized payloads sent by the companion phone.
There's no web rendering surface on the glasses today. But there *is* a need for:
1. A consistent design language across glasses experiences
2. A theming system that works on constrained displays
3. Components purpose-built for the interaction model (head gestures, voice, no touch)
XDS can provide all three — if it speaks native.
## Two Distinct Surfaces
Smart glasses and the companion mobile app are **fundamentally different surfaces** and should be treated as separate targets:
### Glasses
| Constraint | Value |
|---|---|
| Display | 600×600px square, see-through prism |
| Color mode | Always dark (transparent background, light-on-dark text) |
| Interaction | Head gestures, voice commands, touchpad on temple — no touchscreen |
| Typography | Larger base sizes for readability at arm's length through optics |
| Motion | Minimal — battery constraints + perceptual limits of see-through display |
| Component set | Reduced: cards, text, icons, lists, status indicators, buttons. No dialogs, popovers, tables, forms |
| Layout | Card-based panels, single-focus UI, no scrolling |
### Mobile Companion
| Constraint | Value |
|---|---|
| Display | Standard phone screen, full color, light + dark mode |
| Interaction | Standard touch, gestures, keyboard |
| Typography | Standard mobile sizes |
| Motion | Full motion system |
| Component set | Broader — closer to web, but adapted for mobile patterns |
| Layout | Standard mobile layouts, navigation patterns |
These might share a base Android component library but diverge significantly in theme defaults, component subsets, and interaction patterns.
## Architecture
### Theme Build Targets
`xds theme build` gains `--target` flag:
```bash
xds theme build src/glassesTheme.ts # CSS (default)
xds theme build src/glassesTheme.ts --target kotlin # → XDSGlassesTheme.kt
xds theme build src/glassesTheme.ts --target swift # → XDSGlassesTheme.swift (future)
```
### What Gets Generated
Not just tokens — **full component style definitions**. The resolved theme already contains:
- `theme.tokens` → all resolved values (type scale, motion, radii expanded)
- `theme.components[name].base` → base component styles
- `theme.components[name]['variant:primary']` → variant styles
- `theme.components[name]['variant:secondary+size:sm']` → compound variant styles
The Kotlin generator maps CSS properties to native equivalents:
| CSS Property | Kotlin/Compose |
|---|---|
| `backgroundColor` | `Color(0xFF...)` |
| `borderRadius` | `RoundedCornerShape(X.dp)` |
| `fontSize` | `X.sp` |
| `padding*` | `PaddingValues(...)` |
| `fontWeight` | `FontWeight.Medium` |
| `color` | `Color(0xFF...)` |
| `gap` | `Arrangement.spacedBy(X.dp)` |
Generated output example:
```kotlin
// Generated by xds theme build --target kotlin
object XDSGlassesTheme : XDSTheme {
// Tokens
override val colorAccent = Color(0xFF4FC3F7)
override val colorTextPrimary = Color(0xFFFFFFFF)
override val spacingOuter = 12.dp
override val radiusContainer = 16.dp
// Component styles
override val button = ButtonStyles(
base = ButtonStyle(
cornerRadius = 8.dp,
paddingHorizontal = 12.dp,
paddingVertical = 8.dp,
fontSize = 14.sp,
fontWeight = FontWeight.Medium,
),
primary = ButtonStyle(
backgroundColor = Color(0xFF0064E0),
contentColor = Color(0xFFFFFFFF),
),
secondary = ButtonStyle(
backgroundColor = Color(0x1A053659),
),
)
}
```
### Package Structure
```
packages/
core/ ← React + StyleX (web) — existing
android/ ← Kotlin/Compose component library
src/
components/ ← XDSButton, XDSCard, XDSText, XDSStack...
theme/ ← XDSTheme interface, CompositionLocal provider
themes/
default/ ← builds to CSS + Kotlin + Swift
glasses/ ← NEW: always-dark, high-contrast, large targets
```
### Native Components
The native packages ship **behavior + layout + accessibility**. Visual styling comes entirely from the generated theme:
```kotlin
@Composable
fun XDSButton(
label: String,
variant: ButtonVariant = ButtonVariant.Primary,
size: ButtonSize = ButtonSize.Medium,
onClick: () -> Unit,
isDisabled: Boolean = false,
isLoading: Boolean = false,
icon: @Composable (() -> Unit)? = null,
) {
val theme = LocalXDSTheme.current
val style = theme.button.resolve(variant, size)
// Component handles layout, interaction, a11y
// Style object handles all visual properties
}
```
Same API conventions as `@xds/core` — same prop names, same variants, same mental model.
## Glasses-Specific Theme
```typescript
const glassesTheme = defineTheme({
name: 'glasses',
typography: { scale: { base: 18, ratio: 1.15 } },
motion: { fast: 100, medium: 250, ratio: 0.8 },
tokens: {
'--color-background-page': '#000000',
'--color-background-surface': 'rgba(20,20,20,0.85)',
'--color-text-primary': '#FFFFFF',
'--color-accent': '#4FC3F7',
'--radius-container': '16px',
},
components: {
button: {
base: { minHeight: '48px' },
},
card: {
base: {
backgroundColor: 'rgba(30,30,30,0.9)',
},
},
},
});
```
## Open Questions
1. **Package naming**: `@xds/android` vs `@xds/android-glasses` vs `@xds/glasses`? Should glasses be a subset of a broader Android package, or its own thing?
2. **Bloks integration**: The glasses Panels system uses Bloks for rendering. Should XDS generate Bloks payloads directly, or target Compose and let the platform bridge handle it?
3. **Component subset**: What's the minimum viable component set for glasses? Likely: Text, Button, Card, Icon, Stack, List, StatusDot, Spinner, Badge. What else?
4. **Interaction model**: How do head gestures and voice map to component interactions? Does Button need a `gazeTarget` prop? Does List need gesture-based navigation?
5. **iOS/SwiftUI**: The companion app exists on iOS too. When does `@xds/ios` enter the picture?
6. **Display fidelity**: The see-through prism has different color reproduction than a standard screen. Do we need glasses-specific color calibration in the theme?
## Prior Art
- The Panels system (`com.meta.wearable.panels`) already uses a widget-based architecture with data providers, which maps naturally to component + theme
- The existing SystemUI at `fbandroid/java/com/meta/smartglass/app/systemui/launcher/` uses a `PanelsManager` with `BloksRenderer` for glanceable widgets
- The emulator is available via `maui nova e -t greatwhite` (Hypernova) — 600×600 display, testable without physical hardware
## Phases
**Phase 1: Token + Style Generation**
- Add `--target kotlin` to `xds theme build`
- Generate complete token objects + component style definitions from the resolved theme
- Create `@xds/theme-glasses` with glasses-specific defaults
**Phase 2: Android Component Library**
- `@xds/android` package with Compose components
- XDSTheme provider consuming generated theme files
- Core components: Text, Button, Card, Icon, Stack
**Phase 3: Glasses Integration**
- Glasses-specific component adaptations (gesture targets, voice triggers)
- Bloks payload generation or Compose-to-Bloks bridge
- Emulator testing pipeline
**Phase 4: Multi-Platform Parity**
- Swift/SwiftUI target for iOS companion
- Cross-platform component API alignment
- Shared storybook/catalog for all platforms
---
## Exploration Notes (from prototyping session)
We built a working prototype: XDS components running on the Hypernova smart glasses emulator (600×600 display). This section captures ideas and findings from that session.
### Additive Light Display
Black pixels = transparent on the real glasses (the display is a projector onto a see-through prism). This fundamentally changes color theory:
- **You can only add light, never darken.** No shadows, no overlays that dim.
- **Bright colors (red, green, yellow) project strongly** through the prism
- **Blues are perceptually weaker** — shorter wavelength, less visible
- **White is the most "opaque" thing you can render**
- **Dark grays are nearly invisible** against bright environments
This means `light-dark()` doesn't apply. There's no light mode when black = transparent. But the existing `[a, b]` tuple format in `defineTheme` maps perfectly to **`[bright environment, dark environment]`** — same data shape, different semantic. The ambient light sensor drives which side of the tuple to use instead of `prefers-color-scheme`.
### Ambient-Adaptive Theming
The glasses have an ambient light sensor (`OPT3001`, 13-step lux-to-nits curve from 0 to 10,000 lux). We prototyped adaptive theming where the full visual treatment shifts based on environment:
| Environment | Surfaces | Text | Accent |
|---|---|---|---|
| Dark room | Subtle `rgba(40,40,45,0.85)` | Soft white `#E8E8E8` | Cool blue `#4FC3F7` |
| Indoor | Medium brightness | Crisp white | Standard blue |
| Outdoor | Bright `rgba(80,80,85,0.9)` | Full white `#FFFFFF` | **Warm orange `#FFA726`** |
Orange/amber accent in bright conditions because warm colors have higher additive visibility through the prism than cool blues.
The `AdaptiveColor(bright, dark)` type interpolates smoothly between the two values based on a normalized ambient level (0.0 = dark room, 1.0 = bright sun), using logarithmic scaling of lux readings.
### Camera-Aware Environment Adaptation (Future)
Beyond brightness — the outward-facing cameras could inform the theme:
- **Complementary colors** — in a green park, shift accent to colors that pop against green
- **Clash avoidance** — if looking at a red wall, don't use red for errors (it'll blend in)
- **Background-aware contrast** — boost surface opacity where the background behind the UI is bright, keep subtle where it's dark
- **Time-of-day warmth** — golden hour → warm amber UI, overcast → cooler tones
The token system supports this naturally. `AdaptiveColor` could become a function of a richer environment signal:
```ts
interface GlassesEnvironment {
brightness: number; // 0-1 from light sensor
dominantHue: number; // 0-360 from camera
dominantSaturation: number; // 0-1
backgroundLuminance: number; // what's behind the UI region
}
```
This is only possible on glasses — no other display has a camera showing what's *behind* the screen.
### Architecture: Compile-Time vs Runtime
We explored two approaches to getting XDS on glasses:
#### Approach A: JSX → Kotlin Compiler
The developer writes standard XDS React. A compiler parses the JSX and emits native Kotlin:
```
GlassesApp.tsx (36 lines) → compile.mjs → MainActivity.kt (244 lines) → APK
```
**Built a working prototype.** The compiler uses Babel to parse JSX, walks the component tree, and emits Android Views with theme tokens baked in. We deployed it to the glasses emulator successfully.
Pros: No runtime overhead, tree-shaking, static analysis, tiny APK
Cons: Can't update UI without rebuilding, no personalization
#### Approach B: RSC-over-Bluetooth Runtime
React Server Components already serialize UI as a JSON payload. The companion phone renders React, sends the RSC payload over Bluetooth, and a native XDS runtime on the glasses inflates it:
```
Phone (React server) → RSC payload (Bluetooth) → Glasses (native XDS renderer)
```
This is architecturally identical to how the existing Bloks/Panels system works — just a different wire format.
**RSC vs Bloks comparison** (same UI, our demo screen):
| Dimension | RSC + XDS | Bloks |
|---|---|---|
| Payload size | ~1.1 KB | ~4.2 KB (JSON) / ~2.5 KB (protobuf) |
| Node count | 17 (semantic) | 47 (primitive) |
| Abstraction | `XDSCard` | `LinearLayout > RoundedRect > Text` |
| Theme resolution | On device (adaptive) | On phone (static) |
| Ambient adaptation | Zero BT traffic | Full panel re-render over BT |
| Update granularity | Single component | Full panel |
**The killer advantage: adaptive theming.** Bloks bakes every color into the payload on the phone. If ambient light changes, the phone re-renders and re-sends the entire panel. RSC sends `type: "XDSCard"` and the glasses resolve colors locally. Zero Bluetooth traffic for theme changes.
### Component Availability: Style-Only vs Behavioral
We analyzed all ~72 XDS components by checking for `useState`, form elements, interactive ARIA roles, and keyboard handling:
- **~47 style-only** (pure presentation) — Card, Badge, Text, Icon, Stack, Grid, List, Spinner, ProgressBar, etc. These compile mechanically from props to native styles.
- **~25 behavioral** (have state, keyboard handling, ARIA) — Button, Calendar, Selector, TextInput, Slider, TabList, etc. These need native implementations.
For glasses specifically, the behavioral set that matters is small: Button (gaze/voice target), List (gesture navigation), and possibly SegmentedControl.
### Input-Aware Component Tiers
The available input method determines which components are viable. With the neural wristband (bracelet), many "unsupported" components become usable:
| Input | Components Enabled |
|---|---|
| Voice + head gestures only | Card, Text, Badge, Button, simple List, StatusDot |
| + Bracelet (EMG/pinch) | Table, CodeBlock, 2D ScrollArea, Selector, long Lists, code review |
| + Keyboard projection | TextInput, TextArea, PowerSearch, Typeahead |
A code review experience on glasses sounds impossible with just voice — but with the bracelet providing pinch-to-scroll and tap-to-select, it becomes natural. The component manifest could be input-aware:
```ts
export const glassesTarget = {
core: ['XDSCard', 'XDSText', 'XDSBadge', 'XDSButton', 'XDSList'],
withBracelet: ['XDSTable', 'XDSCodeBlock', 'XDSScrollArea', 'XDSSelector'],
withKeyboard: ['XDSTextInput', 'XDSTextArea', 'XDSPowerSearch'],
};
```
### Working Prototype
The prototype lives at `~/xds/glasses-demo/` and `~/xds/glasses-compiler/`:
- `glasses-compiler/GlassesApp.tsx` — XDS React source (36 lines)
- `glasses-compiler/compile.mjs` — JSX → Kotlin compiler
- `glasses-demo/glassesTheme.ts` — XDS theme with `[bright, dark]` tuples
- `glasses-demo/` — Android project, builds and runs on Hypernova emulator
Setup: `maui nova e -t greatwhite` → `maui skoobe` → `adb install`
### Revised Phases
**Phase 1: Theme Build Targets** (unchanged)
- `--target kotlin` / `--target swift` for `xds theme build`
- `@xds/theme-glasses` with adaptive `[bright, dark]` tokens
**Phase 2: XDS → Native Compiler**
- Compile XDS JSX to native Kotlin (Android Views or Compose)
- Style-only components → generated code
- Behavioral components → imported from `@xds/android-glasses`
- Input-aware manifest for component availability
**Phase 3: RSC Runtime**
- Native XDS renderer on glasses (inflate RSC payloads)
- RSC-over-Bluetooth from companion phone
- Device-local theme resolution (ambient-adaptive, zero BT traffic)
- Compare performance vs Bloks on real hardware
**Phase 4: Environment-Aware Theming**
- Camera-informed color adaptation
- Background-luminance-aware contrast
- Complementary/clash-avoidance color selection
- Bracelet input integration for component tier unlocking
Contributor guide
Research direction
This proposal spans the existing packages/core, the xds theme build entry point, and the Android SystemUI entry point at fbandroid/java/com/meta/smartglass/app/systemui/launcher/. Start by narrowing the work to one phase and reading the relevant theme-generation or emulator flow; done requires a defined scope, implementation target, and validation command, none of which is selected here.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- android, kotlin, react, typescript
- Domain
- design, mobile, tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100