The Worker's lanes, RegionAuth renewals and direct archive writes replace
app/cron.mjs, warmCaches and DataArchiver, which only spectre used. Local
runs keep every updater, social, screenshot and archive tool; there is no
local scheduler, and local snapshots stay in storage/archive/.
ARCHIVE_DATA now has to be "true": any value, including the "false" in
.env.example, used to turn archiving on.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Both Workers and their Durable Objects are wrapped with @sentry/cloudflare, so the shared
code's @sentry/core calls (failed updaters, social posts, archive snapshots) reach Sentry.
A lane run that fails outright, a failed credential renewal and a failed archive compression
are reported too. Durable Object RPC methods use instrumentPrototypeMethods; without it they
run with no Sentry client and their events are dropped.
The Node runner and the compressor's container no longer use Sentry; @sentry/node is gone.
POST /sentry-test sends one test event from the updater Worker and one from a Durable Object.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With ARCHIVE_DATA set, the Worker writes each published data file to the
archive Space, at the same YYYY/MM/DD/<date>.<time>.<name>.json keys, ACL,
and content type as the Node uploader (DataArchiver):
- S3ArchiveStorage signs PUTs with aws4fetch, tries three times with
backoff, and stops early on errors that will not improve.
- A snapshot that cannot be written is a warning and a Sentry event, never
a failed update.
- The fast lane holds its snapshots until after social posts (even failed
ones) and then uploads the newest copy of each key, so a slow archive
bucket never delays a post. The slow lane uploads as it goes, to avoid
holding large X-Rank files in memory.
- Dev writes to its own Space (splatoon3ink-archive-test); production has
no archive settings until cutover. Daily compression is still to come.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Replace the remaining Node-only dependencies so the updaters and social
posters bundle and run in workerd, while the Node runner keeps working:
- Replace nxapi with app/splatnet/coral.mjs (fetch + WebCrypto), aligned
with nxapi 1.6.1-next.257: znca compatibility 3.5.0, the current client
version, and the slimmer Login parameter. Use one znca-api client per
Nintendo account, since its token binds to the first account it serves.
Cache the remote config in private storage instead of restarting daily.
Sessions cached by the nxapi package trigger a fresh login.
- Convert images through #image-converter: sharp in Node, the Images
binding in Workers. Read Bluesky aspect ratios from the PNG header.
- Replace twitter-api-v2 with OAuth 1.0a signed fetch; TwitterClient now
only updates the seasonal X profile (posts go through Buffer).
- Load avatars through #avatars (bundled as Data modules in Workers) and
keep the local Chrome driver out of Worker bundles via
#local-screenshot-driver.
- prefixedConsole forwards to the global console; shared code reports to
Sentry through @sentry/core.
Verified live: full credential refresh for all four regions in Node, and
the complete Coral → bullet token → GraphQL chain for EU inside workerd.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Prepare the shared updater and social code to run against R2 in a Worker
while keeping the local Node runner working:
- Add app/common/storage with public, private, and archive areas keyed like
the R2 objects. FilesystemStorage maps them onto dist/ and storage/ for
Node; BucketStorage wraps R2 bindings. Storage is configured once per
process; only image existence is cached, so JSON reads are always fresh.
- Move ValueCache, DataUpdater (data, calendars, archive snapshots),
ImageProcessor, LocalizationProcessor, festival updaters, status
generators, social translations, ImageWriter, and Buffer/Threads image
hosting onto storage. Query hashes are a JSON import.
- Add app/splatnet/auth.mjs: the existing bullet-token code behind auth(),
so the Worker can route token requests to per-region Durable Objects.
warmCaches uses auth().refresh().
- Remove the S3/R2 sync, vfs, the dual upload, the sync commands, and
s3-sync-client. Local runs no longer read from or write to remote buckets.
- Give every test fresh in-memory storage via app/test/setup.mjs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Groundwork for running the existing updaters and social posters in a Worker:
- Derive IDs with @noble/hashes shake256; the Workers runtime's node:crypto
lacks shake256. A test pins an ID produced by node:crypto.
- Split updaters into fast (schedules, gear, co-op, festivals, stages; run
concurrently) and slow (X-Rank, Splatfest rankings; run one at a time).
The cron posts after the fast updaters, before the slow ones.
- Fetch X-Rank seasons and detail types sequentially, cutting peak heap per
division from ~85 MB to ~38 MB.
- Move Splatfest rankings into FestivalRankingsUpdater, driven by the
published festivals data, so results never wait on ranking pagination.
- Coalesce concurrent downloads of the same SplatNet image.
- Bring over the Browser-binding screenshot work from develop (f7293b6):
Cloudflare captures use the BROWSER binding with retries, Browserless is
removed, and workers/screenshots offers capture-only local testing. The
local driver keeps puppeteer-core with an installed Chrome, always serves
dist/, and fails on page-reported screenshot errors.
- Remove the Docker image, compose files, and GHCR deploy workflow.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
VFS now loads on first has()/getMtime() call rather than requiring
explicit initialization in sync/index.mjs. This ensures it works
regardless of entry point (sync:download, splatnet:quick, etc.).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
exists() and olderThan() now check the local filesystem first and only
consult the VFS as a fallback. This means freshly written files are
found naturally without callers needing to register them.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Uses S3 ListObjectsV2 to build an in-memory file listing at startup,
allowing exists() and olderThan() to resolve from the listing instead
of requiring files on disk. sync:download now skips assets/splatnet/,
data/xrank/, data/festivals.ranking.*, and status-screenshots/.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
jsonpath v1.2.1 (published Feb 2025) fixed the security vulnerability
that prompted the switch to jsonpath-plus. Reverting to the original
package simplifies the code by removing the jsonpathQuery/jsonpathApply
wrapper functions.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Added mkdirp() helper to app/common/fs.mjs that wraps
fs.mkdir(dir, { recursive: true }). Updated all call sites to use it.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Deduplicate identical _calculateCacheExpiry methods from
SplatNet3Client and NsoClient into a shared calculateCacheExpiry
function in util.mjs.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This now strips out the domain and query string from the URL before hashing it. This way, we can help ensure the hashes will stay the same when those parameters change.