5 Commits

Author SHA1 Message Date
Matt Isenhower
a1e934d0bd Improve readability of test setup and assertions 2026-09-07 17:52:50 -07:00
Matt Isenhower
abb11e972d Revert "Return a run summary from each updater"
This reverts commit 49ee02731c.
2026-09-06 18:04:42 -07:00
Matt Isenhower
49ee02731c Return a run summary from each updater
Updater.update() now returns what happened: per-phase timings (fetch,
localize, process, publish, images), which languages had to be fetched,
which locale documents changed, and how many images were downloaded.
Subclasses add their own details (stages seeded and new stages, festival
rankings fetched). updateAll() carries each summary in its results, so the
Worker's "Updaters finished" log entry contains the whole picture per
updater. FilesystemStorage.writeJson now skips unchanged content like
BucketStorage does, so "locale documents written" means the same locally.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-05 18:04:55 -07:00
Matt Isenhower
be6c23788f Keep the S3 sync and filesystem storage out of the updater import graph
updateAll() only runs the updaters now. The local (Node) flavour, which
also uploads to S3 when configured, lives in src/app/local.js next to the
filesystem storage wiring, and the entry points call that. The storage
barrel no longer re-exports FilesystemStorage. With both gone, the
updater tree bundles for a non-Node target with no unresolved imports.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-05 10:31:09 -07:00
Matt Isenhower
19d1fb5d3d Move updaters onto public and private storage
Updaters take a storage object instead of touching the filesystem: a
publicStorage (what the site serves: data/, assets/) and a privateStorage
(updater state). Two implementations share one small interface
(exists / readJson / writeJson / readBytes / writeText / writeBytes):

- FilesystemStorage over a directory, used by npm run splatnet, so dist/
  and storage/ end up exactly as they always have and the S3 sync is
  untouched;
- BucketStorage over an R2 bucket binding (MemoryBucket in tests), with a
  thin per-run cache: exists() lists a directory once and answers from
  memory, readJson() parses a document once, and writeJson() skips the put
  when the content is unchanged.

The updater base class and localization processor keep their original
structure with awaits added; the processor reads its document lazily and
writes once per update, as splatoon3.ink's does. Objects written to a
bucket carry the content type and cache-control the S3 sync used to apply.

Tests cover the storage contract (run against both implementations), the
BucketStorage cache behaviour, MemoryBucket's R2-style listing, the
localization processor, the updater base, and each concrete updater.
scripts/compare-data.mjs compares two data directories ignoring SplatNet's
random ordering and ICS timestamps.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-05 09:50:14 -07:00