How STORMTRACE Works

01 · What this app is

STORMTRACE (codename SupercellSentinel) is a weather tracker and alert app. Its distinctive idea: take professional/public weather data and compare it, live, against a physical weather station sitting at home. When the two disagree — say the public forecast says calm but your own backyard anemometer is reading 60 km/h gusts — the app surfaces that disagreement in plain language as a Verdict.

It does five things:

• Reconciles public weather (Open-Meteo + MeteoAlarm warnings, Europe) with the home station (a Webber WS603, read over the Tuya IoT cloud).
• Maps & radar — an interactive radar/map view with rain, wind, and temperature overlays and webcams.
• History & charts — stores readings over time and plots them.
• Alerts — official severe-weather warnings from many feeds worldwide, collected by a separate background service.
• A research pipeline — it quietly records what every forecast model predicted and what actually happened, so forecast accuracy ("skill") can be scored in Jupyter notebooks.

The tech stack at a glance

02 · A 5-minute web-dev primer

If these words are already familiar, skip ahead. Otherwise, here are the concepts that the rest of the doc leans on.

03 · The big picture

STORMTRACE is really four programs sharing one database. Keeping them separate means the website staying responsive even while heavy data-collection grinds away in the background.

flowchart TB
    subgraph ext["☁️  External data sources"]
      direction LR
      OM["Open-Meteo
(forecast · archive/ERA5 · current)"]
      TUYA["Tuya Cloud
(Webber WS603 station)"]
      MA["MeteoAlarm EDR"]
      FEEDS["NWS · SPC · NHC
GDACS · USGS"]
      RV["RainViewer · MapTiler · Windy"]
    end

    subgraph app["🖥️  Next.js app (one server process)"]
      direction TB
      PAGES["Pages (frontend)
radar · dashboard · charts · auth"]
      APIR["API routes (backend)
/api/station · /api/wind · /api/cron/*"]
      LIB["lib/ — data clients + Verdict logic"]
      PAGES --> LIB
      APIR --> LIB
    end

    COLL["🐍 Python collector
(standalone daemon)"]
    DB[("🗄️  PostgreSQL + TimescaleDB")]
    NB["📓 Jupyter notebooks
(forecast-skill analysis)"]
    USER(["👤 Browser"])

    USER -->|HTTPS| PAGES
    OM --> LIB
    TUYA --> LIB
    RV --> PAGES
    MA --> COLL
    FEEDS --> COLL
    LIB <-->|read/write| DB
    COLL -->|write alerts| DB
    NB -->|read-only| DB

    classDef db fill:#0c3,stroke:#093,color:#031;
    class DB db;
    

The four programs

04 · The frontend — what the user sees

The frontend is built with Next.js's "App Router". The rule there is simple: the folder structure under app/ is the URL structure. A folder called dashboard becomes the page at /dashboard; a file named page.tsx is the page itself.

Internationalisation (i18n) & routing

Every page lives under app/[lang]/. The [lang] in brackets is a dynamic segment — a placeholder that matches en or pl. So the dashboard is really at /en/dashboard or /pl/dashboard. A small redirect layer (proxy.ts) catches any URL without a language prefix and bounces you to one based on your browser's Accept-Language header.

app/[lang]/
  page.tsx          → the radar/map home page   (/en)
  dashboard/        → station-vs-public + Verdict (/en/dashboard)
  charts/           → history time-series        (/en/charts)
  login/ register/  → auth screens
  dictionaries/     → en.json · pl.json (all UI text, both languages)

Server Components vs. Client Components

Next.js pages are Server Components by default: their code runs on the server, can talk to the database directly, and ships zero JavaScript to the browser. When a piece needs interactivity (clicking, dragging the map), it's marked "use client" and becomes a Client Component that runs in the browser.

The home page app/[lang]/page.tsx is a great example — it's a server component that fetches everything up front, then hands the data to an interactive client component:

// runs on the SERVER — fetches in parallel before rendering
const [user, location] = await Promise.all([getCurrentUser(), getLocation()]);
const [warnings, station, forecast] = await Promise.all([
  getAlertsForLocation(location),
  authed ? fetchStation() : null,   // station is auth-gated
  fetchForecast(location).catch(() => null),
]);
return <RadarConsole … station={station} forecast={forecast} />;  // → client

The big interactive piece, RadarConsole (a 550-line client component), owns the map, the zoom controls, the overlays, and the live polling. Notice it lazy-loads the actual map with next/dynamic and ssr:false — Leaflet needs a real browser window, so it must not run on the server.

The design system

The whole app wears a green phosphor-terminal skin — the "Greenscreen / STORMTRACE" design system in Greenscreen/. Fonts (VT323, Share Tech Mono), the flicker animation, and the colour tokens (--phosphor, --watch amber, --warn red) all come from there. The severity colours are meaningful, not decorative: green = agree, amber = watch, red = warn.

05 · A page load, step by step

Let's trace exactly what happens when someone opens the home radar page. This ties the frontend, the data clients, and the database together.

sequenceDiagram
    autonumber
    participant B as Browser
    participant P as proxy.ts
    participant S as page.tsx (server)
    participant L as lib/ data clients
    participant DB as Database
    participant OM as Open-Meteo / Tuya

    B->>P: GET /
    P-->>B: redirect → /en  (locale picked)
    B->>S: GET /en
    Note over S: Server Component runs
    S->>L: getCurrentUser() (session cookie)
    L->>DB: look up session
    DB-->>L: user or null
    par fetch in parallel
        S->>L: getAlertsForLocation()
        L->>DB: active warnings
    and
        S->>L: fetchStation()  (only if logged in)
        L->>OM: Tuya signed request
    and
        S->>L: fetchForecast()
        L->>OM: Open-Meteo current
    end
    OM-->>L: readings
    L-->>S: station · forecast · warnings
    S-->>B: finished HTML (already filled)
    Note over B: RadarConsole "hydrates"
→ map becomes interactive
    B->>B: polls /api/wind, /api/temp,
RainViewer frames on a timer
    

Two things worth calling out:

• Parallel fetching — Promise.all([...]) fires the station, forecast, and warnings requests at the same time instead of waiting for each in turn. Faster first paint.
• Auth-gating — fetchStation() only runs if you're logged in. The home station readings and the Verdict are private; the public Open-Meteo data is not.
• Graceful fallback — fetchForecast(location).catch(() => null) means if Open-Meteo is down, the page still renders (just without that reading) instead of erroring.

06 · API routes — the backend

Anything under app/api/ is a backend endpoint, not a page. A file app/api/station/route.ts answers requests to the URL /api/station. These are what the browser's JavaScript calls after the page has loaded (e.g. when you drag the map and need fresh wind data), and what the scheduled timers hit.

They fall into four families:

The read-through cache pattern (wind & temp)

The radar overlays are a nice example of defensive design. Open-Meteo has a daily request limit, and the map can fire many overlay requests. So /api/wind uses a read-through cache backed by the database:

flowchart LR
    A["Request
lat·lon·km"] --> B{"Fresh entry
in DB cache?
(< 15 min)"}
    B -- yes --> C["✅ serve cached"]
    B -- no --> D["Fetch from
Open-Meteo"]
    D -- ok --> E["Store in DB,
then serve"]
    D -- fail --> F{"Stale entry
exists?
(< 2 h)"}
    F -- yes --> G["⚠️ serve stale"]
    F -- no --> H["🟡 synthetic
field (SIM)"]
    

The center coordinates are bucketed to ~0.25° (≈28 km) so a small map nudge reuses the same cached grid instead of triggering a fresh fetch. The cache lives in the database, and each response notes whether it was served fresh, from cache, stale, or as a synthetic fallback.

07 · Data clients & the Verdict

The lib/ folder is the brains: it holds the data clients (code that talks to each external service) plus the comparison logic. Pages and API routes never call the outside world directly — they go through lib/, which keeps secrets server-side and gives every caller the same fallback behaviour.

The three Open-Meteo clients

Open-Meteo is a free, keyless weather API. The app uses three different "flavours" of it, each in its own file:

One source of truth for weather variables

A subtle but important file: lib/weather-vars.ts. It defines the canonical list of weather variables once — and records, per variable, what each Open-Meteo endpoint calls it and which station metric it maps to. The database columns, the pollers, and the station mapping all derive from this one list, so adding a variable is a one-line change. (Example: ERA5 has no UV index, so uv_index simply omits its archive name.)

The Tuya / station client

The home station is a Webber WS603 read through the Tuya IoT Cloud. lib/tuya.ts is a hand-rolled client: it signs each request with HMAC-SHA256 (a cryptographic signature proving the request is authentic), caches the access token, and reads the device's "data points" (DPs). The raw DP codes vary per device, so lib/normalize.ts maps them onto clean metric names (and applies scale factors — many DPs are integers ×10).

The orchestrator: lib/data.ts

The dashboard's data is assembled in one place — lib/data.ts → getDashboard(). It fetches the station + forecast + warnings, runs the comparison for each metric, computes the Verdict, and gathers the history series. One call, everything the dashboard needs.

The Verdict engine

This is the conceptual heart of the app, and it's refreshingly small (lib/compare.ts). For each compared metric it computes delta = station − public, then grades the absolute gap against per-metric thresholds into one of three severities. The overall Verdict is just the worst metric's severity.

flowchart TB
    subgraph perMetric["For each metric (temp, wind, gust, …)"]
      M1["delta = station − public"] --> M2["abs = |delta|"]
      M2 --> M3{"abs ≥ warn
threshold?"}
      M3 -- yes --> W["severity = WARN 🔴"]
      M3 -- no --> M4{"abs ≥ watch
threshold?"}
      M4 -- yes --> WT["severity = WATCH 🟠"]
      M4 -- no --> OK["severity = OK 🟢"]
    end
    W --> AGG["Verdict = worst
severity across
all metrics"]
    WT --> AGG
    OK --> AGG
    AGG --> MSG["message:
warn→'diverge'
watch→'minor'
ok→'agree'"]
    

The thresholds (what counts as "watch" vs "warn" for each metric) live in METRIC_META in lib/metrics.ts. The result carries a worst field naming the most-divergent metric, so the UI can point right at it. The message keys (agree/minor/diverge) are looked up in the language dictionary, keeping the logic language-agnostic.

08 · Scheduled jobs — pulling the data in

The live pages show "now". But the history, charts, and research all need data recorded over time. That's the job of the three cron endpoints under /api/cron/. They aren't pages — nobody visits them in a browser. Instead, a timer on the server calls them on a schedule (the README recommends systemd timers; see §13).

flowchart LR
    subgraph timers["⏱️ systemd timers"]
      T1["every 10 min"]
      T2["hourly"]
      T3["daily"]
    end
    T1 --> P1["/api/cron/poll-station"]
    T2 --> P2["/api/cron/poll-forecast"]
    T3 --> P3["/api/cron/archive-catchup"]

    P1 -->|"station reading"| O1["observations
source=station"]
    P1 -->|"live best-estimate
(all locations)"| O2["observations
source=model_analysis"]
    P2 -->|"5-day hourly forecast"| F["forecasts
model · lead_hours"]
    P3 -->|"ERA5 reanalysis
(12-day window)"| O3["observations
source=era5_archive"]

    O1 --> DB[("Database")]
    O2 --> DB
    F --> DB
    O3 --> DB
    classDef db fill:#0c3,stroke:#093,color:#031; class DB db;
    

Securing the cron endpoints

The data-capture endpoints aren't meant for the public — they're guarded so only the scheduler can trigger them. Unauthorised requests are rejected.

09 · The Python collector — severe-weather alerts

Severe-weather warnings (tornado warnings, hurricanes, earthquakes…) come from a separate, always-on Python daemon in collector/. Why separate? These feeds are numerous, flaky, and global — exactly the kind of noisy work you want isolated from the website. It shares only the database.

It uses APScheduler to run three "tiers" at different rates (collector/main.py, collector/engine.py):

flowchart TB
    subgraph sched["APScheduler (one async loop)"]
      L["LIVE · every 3 min"]
      S["SEMI-LIVE · every 15 min"]
      R["ROUTINE · hourly"]
    end
    L --> LF["NWS · GDACS · USGS"]
    S --> SF["NHC · MeteoAlarm · SPC"]
    R --> RF["prune old
raw snapshots"]

    LF --> CB{"Circuit breaker
per source"}
    SF --> CB
    CB -- open --> SKIP["skip + log
circuit_open"]
    CB -- closed --> FETCH["fetch feed"]
    FETCH --> UP["normalise →
upsert weather_alerts"]
    UP --> EXP["mark vanished
alerts inactive"]
    EXP --> LOG["log poll metrics
(poll_log)"]
    FETCH -. error .-> FAIL["record failure
→ may open breaker"]
    classDef w fill:#fff3,stroke:#999;
    

Each feed has its own fetcher in collector/fetchers/ (one file per source) that normalises wildly different formats into a single weather_alerts row shape. Two resilience patterns stand out:

• Circuit breaker (circuit_breaker.py) — if a source keeps failing, the breaker "opens" and stops hammering it for a cooldown, then tries again. Protects both the collector and the upstream service.
• Self-instrumentation — every poll cycle writes a row to poll_log (duration, items fetched/inserted/updated, status), so the collector's own health is queryable. Sample payloads go to raw_snapshots for debugging.

It ships as a systemd service (ss-collector.service) so the OS keeps it running and restarts it if it dies.

10 · The database

One PostgreSQL database is the shared memory for everything. The schema is defined in TypeScript with Drizzle ORM in db/schema.ts — that single file is the source of truth, and migrations are generated from it.

The tables

erDiagram
    locations ||--o{ observations : "has many"
    locations ||--o{ forecasts : "has many"
    users ||--o{ sessions : "has"
    users ||--o{ email_verification_tokens : "has"

    locations {
      serial id PK
      text slug UK
      text name
      float lat
      float lon
      text kind "station|city|reference"
    }
    observations {
      timestamp time PK "hypertable partition"
      int location_id FK
      text source "station|model_analysis|era5_archive"
      float temperature
      float wind_speed
      float wind_gust
      float precip
      string etc "~18 weather columns"
    }
    forecasts {
      timestamp valid_at PK "hypertable partition"
      timestamp issued_at
      int lead_hours
      int location_id FK
      text model "ecmwf|icon|gfs"
      string etc "same weather columns"
    }
    weather_alerts {
      text id PK "source:eventId"
      text source
      text event_type
      text severity
      bool is_active
      jsonb geometry
    }
    users {
      serial id PK
      text email UK
      text password_hash
      bool email_verified
    }
    grid_snapshots {
      serial id PK
      text kind "wind|temp"
      jsonb data
      timestamp fetched_at
    }
    

Grouped by who owns them:

Why TimescaleDB?

TimescaleDB is a Postgres extension built for time-series data. observations and forecasts become hypertables — automatically partitioned by time, so years of minute-by-minute data stay fast. The setup in db/timescale.sql also adds:

• A continuous aggregate observations_daily — a daily rollup (min/max/avg temp, totals…) that refreshes itself hourly. Instant "what was last month like?" queries without scanning raw rows.
• Compression policies — chunks older than 30 days get compressed, keeping multi-year history tiny.

Migrations

You don't edit the live database by hand. You change schema.ts, run npm run db:generate (Drizzle writes a numbered SQL file into db/migrations/), then npm run db:migrate applies it. The numbered files (0000_… → 0004_…) are the version history of the schema.

11 · The teaching loop — the research payoff

Here's where the pieces click together into something genuinely clever. Because the app stores both what was predicted (forecasts) and what actually happened (observations), it can score forecast accuracy automatically.

The join lives in a SQL view, forecast_errors (db/analytics.sql). For every forecast row it finds the matching observation at that exact location and valid-time, preferring the station as truth and falling back to ERA5, then computes the per-variable error:

flowchart LR
    F["forecasts
(what was predicted,
by model + lead_hours)"]
    O["observations
(what happened:
station ▶ else ERA5)"]
    F -->|"join on
location + valid_at"| V["forecast_errors view
err = forecast − truth"]
    O --> V
    V --> NB["📓 notebooks score
MAE / RMSE
by model & lead time"]
    NB --> INSIGHT["💡 'ICON's temp error
grows ~0.3°C per day of lead'"]
    

This is the loop that justifies all the capture machinery: predict → wait → observe → measure the gap → learn. It's also why the project nickname is Sentinel — it's quietly keeping score on the forecasters.

12 · Analysis notebooks

The offline analysis lives in analysis/ as Jupyter notebooks — interactive Python documents mixing code, charts, and prose. They connect to the database read-only (through a restricted Postgres role, see §13) so exploration can never corrupt production data.

13 · Deployment — where it all runs

In production everything runs on a single Linux server. The pieces are wired together by the operating system's standard tools — nginx (a web server / reverse proxy) out front, and systemd (the Linux service manager) keeping the long-running pieces alive and firing the timers.

flowchart TB
    NET(["🌍 Internet"]) -->|HTTPS :443| NGINX["nginx
(TLS, reverse proxy)"]
    NGINX -->|"reverse proxy"| APP["Next.js app
(systemd service)"]
    subgraph os["systemd"]
      APP
      COLL["collector daemon
(ss-collector.service)"]
      TM1["⏱ poll-station.timer → curl /api/cron/poll-station"]
      TM2["⏱ poll-forecast.timer"]
      TM3["⏱ archive-catchup.timer"]
    end
    TM1 --> APP
    TM2 --> APP
    TM3 --> APP
    APP --> PG[("PostgreSQL
+ TimescaleDB")]
    COLL --> PG
    classDef db fill:#0c3,stroke:#093,color:#031; class PG db;
    

How the schedule actually fires: a systemd .timer wakes on its calendar (e.g. every 10 minutes), which starts a one-shot .service that triggers the matching data-capture endpoint. Clean separation: the schedule is OS config, the work is app code.

The read-only analyst role

The notebooks log in through a separate, locked-down database user — it can SELECT but not change anything. Principle of least privilege: analysis can't break production.

14 · Config, secrets & mock mode

All server configuration funnels through one validated file, lib/config.ts, which parses environment variables with Zod (a library that checks shapes and supplies defaults). Secrets are read here and never reach the browser.

The neat part is how it decides whether to use real or fake data, per source. The public forecast (Open-Meteo) needs no key, so it runs live by default; the home station only goes live when its credentials are present, and otherwise returns believable synthetic data.

So with no configuration at all, the public forecast is real while the station is mocked — and the app honestly flags which is which with a MOCK/LIVE badge. Provide the station credentials and it flips to live with no code change. This is why the project can promise "just start it".

Auth, briefly

Accounts use server-side sessions: logging in sets an opaque, http-only cookie, and passwords are stored only as salted one-way hashes using an industry-standard algorithm. No password or usable session token is ever exposed to the client.

15 · Glossary