6 Commits

Author SHA1 Message Date
Matt Isenhower
aa71bf2eda Hold a lane run while an earlier one may still be going; log data changes
Some checks failed
Build frontend / build (24.x) (push) Has been cancelled
Lint / lint (push) Has been cancelled
Tests / test (24.x) (push) Has been cancelled
Cloudflare replaced the fast lane's Durable Object during the 00:00 UTC run.
The alarm restarted on a new instance while the old one carried on posting,
and the old run's results were lost because it could no longer write to its
own storage. Like the old cron, a run now takes a lock in the private R2
bucket (which a replaced instance can still write), refreshes it every 10
seconds and deletes it at the end; a run that finds a fresh lock waits and
checks again in 15 seconds. A scheduled run that never saved its results is
recorded as cut off.

Each updater now reports whether its published file changed, in its log
line and in the run's results, to see when fresh data really arrives at the
top of the hour.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-15 16:23:40 -08:00
Matt Isenhower
8806239511 Remove the server's scheduler, cache warming and archive uploader
The Worker's lanes, RegionAuth renewals and direct archive writes replace
app/cron.mjs, warmCaches and DataArchiver, which only spectre used. Local
runs keep every updater, social, screenshot and archive tool; there is no
local scheduler, and local snapshots stay in storage/archive/.

ARCHIVE_DATA now has to be "true": any value, including the "false" in
.env.example, used to turn archiving on.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-15 09:51:51 -08:00
Matt Isenhower
008db9b459 Hold deferred archive snapshots in a map keyed by archive path
Each region's FestivalUpdater saves the combined festivals.json, so a run saves it several
times. A newer copy now replaces the held one instead of being added alongside it, so each
snapshot is held and uploaded once.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 09:39:08 -08:00
Matt Isenhower
e7388fce6a Archive data snapshots from the Worker to DigitalOcean Spaces
With ARCHIVE_DATA set, the Worker writes each published data file to the
archive Space, at the same YYYY/MM/DD/<date>.<time>.<name>.json keys, ACL,
and content type as the Node uploader (DataArchiver):

- S3ArchiveStorage signs PUTs with aws4fetch, tries three times with
  backoff, and stops early on errors that will not improve.
- A snapshot that cannot be written is a warning and a Sentry event, never
  a failed update.
- The fast lane holds its snapshots until after social posts (even failed
  ones) and then uploads the newest copy of each key, so a slow archive
  bucket never delays a post. The slow lane uploads as it goes, to avoid
  holding large X-Rank files in memory.
- Dev writes to its own Space (splatoon3ink-archive-test); production has
  no archive settings until cutover. Daily compression is still to come.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 22:47:38 -08:00
Matt Isenhower
21a4f5d85c Route backend file access through storage and auth seams
Prepare the shared updater and social code to run against R2 in a Worker
while keeping the local Node runner working:

- Add app/common/storage with public, private, and archive areas keyed like
  the R2 objects. FilesystemStorage maps them onto dist/ and storage/ for
  Node; BucketStorage wraps R2 bindings. Storage is configured once per
  process; only image existence is cached, so JSON reads are always fresh.
- Move ValueCache, DataUpdater (data, calendars, archive snapshots),
  ImageProcessor, LocalizationProcessor, festival updaters, status
  generators, social translations, ImageWriter, and Buffer/Threads image
  hosting onto storage. Query hashes are a JSON import.
- Add app/splatnet/auth.mjs: the existing bullet-token code behind auth(),
  so the Worker can route token requests to per-region Durable Objects.
  warmCaches uses auth().refresh().
- Remove the S3/R2 sync, vfs, the dual upload, the sync commands, and
  s3-sync-client. Local runs no longer read from or write to remote buckets.
- Give every test fresh in-memory storage via app/test/setup.mjs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 16:03:33 -08:00
Matt Isenhower
5dd2bbf172 Resolve npm audit vulnerabilities 2026-08-30 16:43:01 -07:00