Skip to content
Demostatics Docs

How It Works

Demostatics is four tiers. Data is gathered and refined by a worker tier, stored and served by a display tier, and consumed by two clients with deliberately different jobs.

The four tiers

Section titled “The four tiers”
        ┌─────────────────────────────────────────────────────────┐
        │  WORKER TIER                              [ NOT BUILT ] │
        │  C/C++ task processor coordinating ML / LLM worker nodes│
        │  Ingestion → Processing → Structuring → Storage         │
        └────────────────────────┬────────────────────────────────┘
                                 │ writes rows        (no producer today)
                                 ▼
        ┌─────────────────────────────────────────────────────────┐
        │  DISPLAY TIER — demostatics-web (Laravel)     [SHIPPED] │
        │  Server of record · sole identity authority             │
        │  Blade UI  ·  JSON API /api/v1  ·  forum · polls · roles│
        └──────┬─────────────────────────────────┬────────────────┘
               │ Bearer (Sanctum)                │ reads
               ▼                                 ▼
        ┌──────────────────┐            ┌──────────────────────┐
        │ MOBILE CLIENT    │            │  world geo store     │
        │ Expo / RN        │            │  read-only, external │
        │      [SHIPPED]   │            │        [SHIPPED]     │
        └──────────────────┘            └──────────────────────┘


        ┌─────────────────────────────────────────────────────────┐
        │  DESKTOP CLIENT — Rust, thick client          [SHIPPED] │
        │  All statistics computed locally                        │
        └────────────────────────┬────────────────────────────────┘
                                 │ WSS /v1/stream
                                 ▼
                    examples/mock-stream-server
                    (in its own repo — NOT the Laravel server)

The disconnected edge at the bottom is not a drawing error. See The Two Contracts.

The pipeline

Section titled “The pipeline”

Four stages, all Planned.

StageWhat it does
IngestionAcquire unstructured and semi-structured data in real time
ProcessingClean, classify and contextualize using ML models and LLMs
StructuringOrganize into meta-descriptive dimensions
StorageStore refined data in accessible, query-ready formats

The declared sources are satellite observation, mapping and traffic APIs (Google Maps, Yandex, Baidu), exchange rates, ISO datasets, journal articles, news networks, social media and open web data. None is integrated.

→ Data Pipeline

Which tier computes what

Section titled “Which tier computes what”

This is the platform’s most important architectural rule, because getting it wrong quietly turns the display tier into a compute tier.

Kind of workWhere it belongsWhy
Request-scoped arithmetic — one user asks, the server answers from current state: a quote, pagination, a vote tally, an exportLaravelIt must be server-authoritative and auditable. A client that computes a fee from a cached rate shows a number you did not charge
Continuous processing over the dataset — evaluating threshold rules across every series on a schedule, computing rollupsWorker tierIt is pipeline work. Putting it in Laravel splits “when did this cross a line” from the forecasting that answers “what happens next”
Heavy statistics, forecasting, simulationDesktop client, locallyThat is what “thick client” means — it is the product’s differentiator, not a server cost
NothingMobile clientThe phone deliberately attempts none of it

So Laravel stores alert rules, serves them, and sends the notification; the worker decides when one fires.

Classification

Section titled “Classification”

There are three competing classification models in the project today, which is two too many.

ModelWhereShape
Seven fixed dimension columnsdemostatics-webregion_id, subregion_id, country_id, state_id, city_id, category_id, subcategory_id — all nullable
Server-defined taxonomyDesktop clientA flat list of {id, label, parent, facet} nodes forming one or more trees, delivered in the Welcome frame
Five meta-descriptive dimensionsFounding spectime · capital · workforce · material–energy · information

The taxonomy explicitly supersedes the five-dimension model, and the roadmap’s advice is to pick the taxonomy and model geography as a facet of it — because building both is how the two clients end up unable to render the same catalogue.

→ Glossary · The Two Contracts

The journey of one value

Section titled “The journey of one value”

What is supposed to happen, end to end, with the status of each step.

  1. A satellite pass or a provider API yields a raw observation. Not built.
  2. A worker node ingests it and an ML or LLM model cleans, classifies and contextualizes it. Not built.
  3. The result is structured against the category taxonomy and written to storage as a numeric observation with a timestamp and a unit. Not built.
  4. demostatics-web reads it and serves it — as HTML at /database, and as JSON at GET /api/v1/database-items. Shipped (against seeded rows).
  5. The phone renders it in the Data tab, filterable by geography and category. Shipped.
  6. The desktop streams it, computes statistics over it locally, and draws it. Shipped against a mock; Not built against real data.

Steps 4 through 6 work today. Steps 1 through 3 are the whole product.

Authentication

Section titled “Authentication”

One authority, two very different token models — one of which is only a proposal.

Implemented (/api/v1)Proposed (/v1)
TokenOpaque Sanctum personal access token, 3|xxxxSigned JWT carrying sub, tier, features, exp
RefreshNonePOST /v1/auth/refresh
Expiry30 days, configurable, pruned dailyConfigurable, dev TTL 1 hour
ScopeNamed abilities, declarative only todaytier + features[] + allowed_topics[]
Used byMobile clientDesktop client, against a mock

The roadmap’s recommendation is that Laravel remains the single identity authority and mints a short-lived signed ticket the stream backend verifies offline — because anything else means a revoked device keeps its data stream.

→ JSON API v1 · Stream Contract v0.2

Graceful degradation

Section titled “Graceful degradation”

Two behaviours worth internalizing, because both are deliberate and both are easy to misread as bugs.

The geo store degrades to empty, never to an error. Every read of the world store is wrapped so a query failure returns an empty collection. With no store connected, pages render and dropdowns are simply empty — nothing returns a 500. The cost is that “no regions exist” and “the geo store is down” look identical, which is exactly why GET /api/v1/health reports world_store.

The desktop may drop, never stall. The UI thread reads the latest snapshot lock-free once per frame and draws. If frames arrive faster than they can be shown, the data plane folds a burst into a single update rather than queueing. A sequence gap triggers a resync that pulls a fresh baseline.

Next

Section titled “Next”