127 lines
6.2 KiB
Markdown
127 lines
6.2 KiB
Markdown
# DESI Explorer
|
|
|
|
An interactive 3D map of the universe, built from data gathered by the [Dark Energy Spectroscopic Instrument (DESI)](https://www.desi.lbl.gov).
|
|
|
|
Renders millions of galaxies / quasars in real time as a navigable 3D point cloud, combining [Odin](https://odin-lang.org) for the renderer with [raylib](https://www.raylib.com) for windowing, input, and 3D drawing.
|
|
|
|
## What is this?
|
|
|
|
A fun personal project with three interlocking goals:
|
|
|
|
- **Turn DESI's survey into something you can fly through** — the Dark Energy Spectroscopic Instrument maps a huge volume of the observable universe (galaxies, quasars, and the Lyman-alpha forest) in three dimensions. This project renders that data as an interactive 3D experience rather than a static plot.
|
|
- **Serve the data, not the download** — the datasets are large, so the experience pulls from a centralized repository / API instead of requiring you to keep the raw catalogs locally (see [Goals](#goals)).
|
|
- **Ship to native _and_ the web** — one Odin codebase compiled to a native desktop executable and to WebGL/WASM, so the map runs anywhere.
|
|
|
|
## Goals
|
|
|
|
- [x] Scaffold the project: Odin + raylib 3D renderer, Go/Pulumi infra, Gitea CI
|
|
- [ ] Pull data from a centralized repository to limit storing all the DESI data locally
|
|
- [ ] Be both a native executable and a WebGL-based interactive experience
|
|
- [ ] Build utilizing the Odin language
|
|
- [ ] (Future) Integrate with other space data to be able to incorporate both dark energy information and more
|
|
|
|
## Current State
|
|
|
|
Early scaffolding. The application currently:
|
|
|
|
- Opens a resizable 3D raylib window with an orbital camera (zoom + rotate + pan).
|
|
- Renders a procedurally generated point cloud standing in for the galaxy catalog (real DESI data ingestion is the next milestone).
|
|
- Compiles natively, to WebAssembly, and is deployed to an on-prem Kubernetes cluster as a placeholder web service.
|
|
|
|
## Roadmap
|
|
|
|
- [ ] Ingest real DESI data (EDR/DR1 catalogs) and map survey coordinates (RA / Dec / redshift) into 3D space
|
|
- [ ] Central data repository / API so large catalogs aren't stored locally
|
|
- [ ] Efficient rendering of large point clouds (instancing / vertex buffers rather than per-point draws)
|
|
- [ ] Camera flight / goto-object controls and a minimap
|
|
- [ ] Color-coding by redshift, survey, or object type
|
|
- [ ] Object selection & metadata inspection
|
|
- [ ] Web (WASM/WebGL) parity with the native build
|
|
- [ ] (Future) Fold in other datasets for dark-energy context
|
|
|
|
## Repository Layout
|
|
|
|
```
|
|
gui/ Odin renderer — the interactive 3D experience (entrypoint: gui/src/main.odin)
|
|
gui/lib/ Third-party Odin dependencies (submodules / vendored libs)
|
|
gui/www/ Web GUI — WASM shell / static web assets
|
|
gui/scripts/ Build scripts (WASM build helper)
|
|
api/ Rust backend API (axum) for serving DESI catalog data
|
|
infra/ Go + Pulumi infrastructure-as-code (deploys the experience to k8s)
|
|
scripts/ Repo-level build helpers (Odin install)
|
|
.gitea/ Gitea Actions CI/CD workflows
|
|
```
|
|
|
|
The renderer (`gui/`), the API (`api/`), and the infrastructure (`infra/`) are
|
|
kept in separate directories so their logic stays cleanly separated. Each
|
|
project carries its own `Makefile`; the root `Makefile` delegates the base
|
|
commands (`run`, `build`, `test`, ...) into them.
|
|
|
|
## Building & Running
|
|
|
|
Requires [Odin](https://odin-lang.org/docs/install) with the bundled raylib
|
|
vendor bindings. The web build additionally requires [Emscripten](https://emscripten.org)
|
|
(`emcc`). The API requires [Rust](https://www.rust-lang.org) (stable toolchain,
|
|
pinned via `api/rust-toolchain.toml`). The infrastructure requires
|
|
[Go](https://go.dev) and [Pulumi](https://www.pulumi.com).
|
|
|
|
The root `Makefile` delegates everything to the per-project Makefiles
|
|
(`make help` lists the base targets):
|
|
|
|
```sh
|
|
make setup # pull submodules + tidy Go deps
|
|
make run # run the native app (gui/)
|
|
make build # -> bin/desi_explorer (gui/)
|
|
make build-debug # debug native build (gui/)
|
|
make build-web # -> build/web (wasm + html, gui/)
|
|
make test # odin test + go test + cargo test
|
|
make clean # remove build artifacts from all projects
|
|
make fmt # format all projects
|
|
```
|
|
|
|
`make run` / `make run-web` configure the API and renderer from the example
|
|
assets under `resources/dev/` (see the dotenv lib in `gui/lib/local/dotenv`).
|
|
Override any of them on the command line:
|
|
|
|
```sh
|
|
make run \
|
|
GUI_ENV_FILE=/path/to/gui.env \
|
|
API_ENV_FILE=/path/to/api.env \
|
|
API_DESI_DATA=/path/to/desi_data.json
|
|
```
|
|
|
|
The defaults point at `resources/dev/gui.env.example` (renderer's `API_URL`),
|
|
`resources/dev/api.env.example` (API `API_BIND_ADDR`), and
|
|
`resources/dev/desi_subset.json` (a small JSON catalog subset served by the
|
|
API's `/api/v1/catalogs` and `/api/v1/objects` endpoints).
|
|
|
|
Project-specific targets live in their own `Makefile` and are reached with
|
|
`make -C <dir> <target>`:
|
|
|
|
```sh
|
|
make -C gui help # Odin renderer (run, build, build-debug, build-web, test, fmt)
|
|
make -C api help # Rust API (run, build, test, check, fmt)
|
|
make -C infra help # Infra (preview, up, down, refresh, static, test)
|
|
```
|
|
|
|
The root also provides thin passthroughs: `make api-<target>` and
|
|
`make infra-<target>` forward to the corresponding sub-project (e.g.
|
|
`make api-check` -> `make -C api check`).
|
|
|
|
## Dependency Updates (Renovate)
|
|
|
|
[Renovate](https://github.com/renovatebot/renovate) keeps dependencies up to
|
|
date by opening PRs. It runs **centrally on the homelab Gitea** — a dedicated
|
|
`renovate` Gitea Actions runner in the homelab k3s cluster runs `renovate` every
|
|
hour (`renovate` workflow in the `homelab` repo) with autodiscovery filtered to
|
|
`sam_oneal/*`. Because this repo carries a `renovate.json` on `main`, it is
|
|
picked up automatically. Config for this repo lives in `renovate.json` and is
|
|
validated with `make renovate-validate`.
|
|
|
|
It manages four categories:
|
|
|
|
- **Infra dependencies** — `infra/go.mod` (`gomod` manager); minor/patch updates grouped into a single PR.
|
|
- **API dependencies** — `api/Cargo.toml` (`cargo` manager); minor/patch updates grouped into a single PR.
|
|
- **Submodules** — `gui/lib/` submodules via `.gitmodules` (`git-submodules` manager).
|
|
- **Workflow actions** — action versions in `.gitea/workflows/*.yml` (`github-actions` manager, which understands Gitea's `.gitea` layout).
|