# 01 — Current Architecture and Where FlatBuffers Fits ## What exists today ### API (Rust + axum) - **Entrypoint**: `api/src/main.rs` — binds `API_BIND_ADDR` (default `0.0.0.0:8080`), wires `tracing`, serves the axum app with graceful shutdown. - **Routes** (`api/src/routes/`): - `GET /health` → static `{"status":"ok"}` - `GET /api/v1/catalogs` → hardcoded list of 2 catalogs (`edr`, `dr1`) - `GET /api/v1/objects?catalog=&limit=` → **stubbed**, returns `[]` - **Models** (`api/src/models.rs`): ```rust pub struct Catalog { pub name: String, pub release: String, pub description: &'static str, pub object_count: Option, } pub struct CatalogObject { pub id: String, pub catalog: String, pub object_type: String, // galaxy / quasar / star pub ra: f64, // degrees pub dec: f64, // degrees pub redshift: f64, // dimensionless } ``` - **Serialization**: serde `Serialize` derives + `axum::Json`. `serde_json` is a dev-dependency only (used by integration tests). - **CORS**: `tower-http` is compiled with `cors` + `trace` features, but no CORS layer is currently added to `routes::app()`. This matters for the WASM GUI. ### GUI (Odin + raylib) - **Entrypoint**: `gui/src/main.odin` — `update()` → `draw()` loop, orbital `Camera3D`, 4,000 procedurally generated points via `make_universe()`. - **Placeholder data**: `Galaxy { position: rl.Vector3, color: rl.Color }` generated in a 500-unit-radius sphere. Colors are a distance-based redshift stand-in. - **Data layer**: `gui/src/data.odin` defines hand-written Odin structs that mirror the Rust API JSON contract (including pointer types `^string`, `^u64` to mirror Rust `Option`), plus stubbed procedures: ```odin get_catalogs :: proc(url: string) -> ([dynamic]Catalog, ^APIError) // nil, nil get_catalog :: proc(url: string, name: string) -> (^Catalog, ^APIError) // nil, nil get_catalog_objects :: proc(url: string, catalog_name: string) // nil, nil ``` - **No HTTP client exists** on the Odin side — no `core:net`, no curl bindings, nothing wired into the render loop. The structs are a *planned* interface. - **Web build**: `gui/www/` — WASM shell (Emscripten) loading `index.wasm` + `odin.js`, no API URL wiring yet. ## Data flow gap ```mermaid flowchart LR A["DESI catalog store"] B["Rust / axum API
serde JSON models"] C["Odin + raylib GUI
hand-mirrored structs
stubs, never used
"] A -. "future ingestion" .-> B B --x|"nothing today"| C ``` There is **no live data flow**. The API currently returns JSON placeholders; the GUI renders procedural points. The intended shape (per code comments): - The /api/v1/objects endpoint "will page through the centralized DESI catalog store." - Real survey data replaces the procedural cloud "once ingestion lands." ## Where FlatBuffers fits There are three natural insertion points: ### 1. Wire format for the object-stream endpoint (primary fit) Replace the JSON array response of `GET /api/v1/objects` (or add an `application/flatbuffer` variant / new `/api/v1/stream` endpoint) with a framed stream of FlatBuffer messages: - **Rust side**: build FlatBuffers with `FlatBufferBuilder` per batch (e.g. 10k-50k objects per batch), send as `application/octet-stream`. - **Odin side**: receive the byte blob, verify once, and access `ra`/`dec`/ `redshift` **directly from the network buffer** — no per-field copy. This is the killer use case for the render loop, where every point is projected and colored per frame. ### 2. Catalog metadata (secondary fit) `GET /api/v1/catalogs` returns tiny JSON today. The FlatBuffers win here is marginal (5-10 records), but using the *same* serialization everywhere keeps the codebase uniform — one schema, one reader path. Recommend serving both a JSON response (for curl/debugging) and a FlatBuffers variant (for the client). ### 3. Bidirectional client → server channel (future fit) If the GUI ever sends selection/filter requests (pan region, redshift range, object type filters), a WebSocket carrying FlatBuffers both ways gives consistent framing. The `get_catalog_objects()` stub signature suggests the GUI is expected to query **on demand** as the camera moves — a WebSocket request/ response protocol with a cheap binary payload would fit this well. ## Integration points (concrete anchors) | Layer | Today | FlatBuffers insertion point | |---|---|---| | Rust models | `Catalog`, `CatalogObject` (serde) | Generated `catalog_generated.rs` alongside or replacing | | Rust routes | `routes/catalogs.rs` | New route (e.g. `/ws/catalogs`) or response-format negotiation | | Rust Cargo | `serde`, `axum::Json` | `flatbuffers` crate + `flatbuffers-build` for `.fbs → .rs` | | Odin structs | `data.odin` mirrors JSON | Generated/FFI FlatBuffers reader; or hand-rolled reader | | Odin HTTP | none | `core:net` HTTP, WebSocket, or curl FFI | | Build | `make build`, `make test` per project | Schema shared & compiled by both sides | ## High-level target architecture ```mermaid flowchart TB S["catalog.fbs
single source of truth
checked into repo
"] R["flatc --rust"] C["flatcc --c
or hand-rolled Odin reader"] API["api/ · Rust"] GUI["gui/ · Odin + raylib"] S --> R --> API S --> C --> GUI API <-->|"HTTP / WebSocket
framed FlatBuffer binary stream"| GUI ``` - One schema file. Two generators. Byte-for-byte identical wire format. - A shared conformance test suite (see `08-testing-strategies.md`) ensures both sides stay in sync. ## Key architectural decisions to make later 1. **Transport**: HTTP chunked/range requests vs. WebSocket vs. both (see `05-streaming-protocols.md`). 2. **Framing**: size-prefixed FlatBuffers (`size_prefixed_root_*`) vs. a custom length prefix (e.g. flatstream's 4-byte LE length). Needed because a TCP/ WebSocket stream is byte-ambiguous — you must know where one message ends and the next begins. 3. **Batching**: per-object tables vs. packed tables of arrays (columnar-style). For millions of DESI objects, a `table { ra: [double]; dec: [double]; ... }` layout is dramatically smaller and faster than 1 table per object. 4. **Schema ownership**: options include a top-level `schema/` directory shared by both `api/` and `gui/`, checked-in generated code, or generated at build time. Checked-in generated code is the simplest CI-safe option for both `cargo build` and `odin build` isolation. ## What to read next - `02-flatbuffers-overview.md` — the format's internals and guarantees - `03-rust-integration.md` — how the Rust side consumes `.fbs` - `04-odin-client-integration.md` — how the Odin side consumes the same format - `05-streaming-protocols.md` — transport and framing choices - `06-schema-design.md` — proposed DESI schema