Commit Graph

17 Commits

Author SHA1 Message Date
Matt Isenhower
db2e68b847 Add the Cloudflare updater Worker with fast and slow lanes
Run the shared app/ updaters and social posters from workers/updater:

- Scheduler Durable Object lanes: "fast" keeps the Node cron cadence
  (quick at :00-:04, default at :05/:10/:15/:20/:30/:45) and posts to
  social after checking SITE_URL serves the new data; "slow" runs X-Rank,
  Splatfest rankings, and avatar updates hourly. Bounded retries, queued
  manual runs, pause, and persisted automatic-scheduling state.
- RegionAuth Durable Object per region: the single owner of each
  Nintendo login, serving bullet tokens to both lanes and renewing them
  hourly at :35-:38.
- A five-minute cron watchdog that only re-arms alarms, bearer-token
  control endpoints, and dev/production environments. DO migrations keep
  Scheduler at v1 to match the earlier dev deployment.
- Per-run limits (RunLimits) and per-account NsoClient locks instead of
  module-level limiters, since Durable Objects sharing an isolate cannot
  share in-flight requests. Updaters without localization rules no longer
  rewrite locale files.
- 20 workerd tests (npm run updater:test, included in npm test) with all
  network access blocked.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-09 22:13:30 -08:00
Matt Isenhower
21a4f5d85c Route backend file access through storage and auth seams
Prepare the shared updater and social code to run against R2 in a Worker
while keeping the local Node runner working:

- Add app/common/storage with public, private, and archive areas keyed like
  the R2 objects. FilesystemStorage maps them onto dist/ and storage/ for
  Node; BucketStorage wraps R2 bindings. Storage is configured once per
  process; only image existence is cached, so JSON reads are always fresh.
- Move ValueCache, DataUpdater (data, calendars, archive snapshots),
  ImageProcessor, LocalizationProcessor, festival updaters, status
  generators, social translations, ImageWriter, and Buffer/Threads image
  hosting onto storage. Query hashes are a JSON import.
- Add app/splatnet/auth.mjs: the existing bullet-token code behind auth(),
  so the Worker can route token requests to per-region Durable Objects.
  warmCaches uses auth().refresh().
- Remove the S3/R2 sync, vfs, the dual upload, the sync commands, and
  s3-sync-client. Local runs no longer read from or write to remote buckets.
- Give every test fresh in-memory storage via app/test/setup.mjs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 16:03:33 -08:00
Matt Isenhower
6df93806c2 Prepare the backend for Cloudflare Workers
Groundwork for running the existing updaters and social posters in a Worker:

- Derive IDs with @noble/hashes shake256; the Workers runtime's node:crypto
  lacks shake256. A test pins an ID produced by node:crypto.
- Split updaters into fast (schedules, gear, co-op, festivals, stages; run
  concurrently) and slow (X-Rank, Splatfest rankings; run one at a time).
  The cron posts after the fast updaters, before the slow ones.
- Fetch X-Rank seasons and detail types sequentially, cutting peak heap per
  division from ~85 MB to ~38 MB.
- Move Splatfest rankings into FestivalRankingsUpdater, driven by the
  published festivals data, so results never wait on ranking pagination.
- Coalesce concurrent downloads of the same SplatNet image.
- Bring over the Browser-binding screenshot work from develop (f7293b6):
  Cloudflare captures use the BROWSER binding with retries, Browserless is
  removed, and workers/screenshots offers capture-only local testing. The
  local driver keeps puppeteer-core with an installed Chrome, always serves
  dist/, and fails on page-reported screenshot errors.
- Remove the Docker image, compose files, and GHCR deploy workflow.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 15:40:51 -08:00
Matt Isenhower
70e81bbc38 Generate canonical asset URLs upstream
Some checks failed
Deploy / deploy-frontend (push) Has been cancelled
Deploy / deploy-backend (push) Has been cancelled
2026-08-30 10:20:12 -07:00
Matt Isenhower
7adb9c313e Add mkdirp utility to avoid verbose fs.mkdir calls
Added mkdirp() helper to app/common/fs.mjs that wraps
fs.mkdir(dir, { recursive: true }). Updated all call sites to use it.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 11:09:02 -08:00
Matt Isenhower
8780463c66 Replace deprecated and unmaintained packages
- Removed mkdirp in favor of native fs.mkdir({ recursive: true })
- Replaced ecstatic (unmaintained since 2021) with sirv
- Replaced jsonpath (security vulnerability) with jsonpath-plus
  - Added jsonpathQuery/jsonpathApply helpers in app/common/util.mjs
- Updated sharp: 0.32.6 → 0.34.5
- Updated puppeteer-core: 23.8.0 → 24.37.3

Vulnerabilities reduced from 40 to 1 (only remaining: axios in
threads-api transitive dependency).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 11:05:49 -08:00
Matt Isenhower
e6004154b9 Migrate ESLint to v9 with flat config
- eslint: 8.57.0 → 9.39.2
- eslint-plugin-vue: 9.23.0 → 10.8.0
- eslint-plugin-jsdoc: 48.2.1 → 62.5.5
- Replaced .eslintrc.cjs with eslint.config.mjs (flat config)
- Removed @vue/eslint-config-airbnb and eslint-plugin-import
- Added @eslint/js, globals, vue-eslint-parser, typescript as deps
- Removed stale eslint-disable comments for import/no-unresolved
- Simplified lint script (ESLint 9 handles extensions/ignores via config)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 11:02:09 -08:00
Matt Isenhower
330a299e3e Ignore a weird issue with ESLint for now 2024-11-22 11:50:56 -08:00
Matt Isenhower
279671896e Formatting 2024-11-20 21:16:29 -08:00
Matt Isenhower
0fe794007e Prevent storing invalid images 2024-11-18 20:33:33 -08:00
Matt Isenhower
34c2d65b15 Download images concurrently 2024-11-17 21:16:24 -08:00
Matt Isenhower
37ee0c4dc9 Prefer single quotes and enforce semicolons 2024-03-17 10:20:17 -07:00
Matt Isenhower
cda854de0b Add a utility function to check whether a file exists 2023-04-22 13:33:04 -07:00
Samuel Elliott
110314ee17 Add Splatfest calendar feeds 2022-11-07 18:45:07 +00:00
Samuel Elliott
0e4812079f Support running data updaters without all region tokens 2022-11-07 18:36:08 +00:00
Matt Isenhower
f689f13d9c Tweak derived ID logic to ensure consistency
This now strips out the domain and query string from the URL before hashing it. This way, we can help ensure the hashes will stay the same when those parameters change.
2022-10-20 16:16:46 -07:00
Matt Isenhower
67387f9775 Download images from SplatNet 2022-09-10 14:32:23 -07:00