Leahnaya/sendou.ink
1
0
Fork 0
mirror of https://github.com/Sendouc/sendou.ink.git synced 2026-09-14 23:26:15 -05:00
Code Issues Actions 3 Packages Projects Releases Wiki Activity

Migrate Minio to SeaweedFS

Browse Source
This commit is contained in:
Kalle
2026-09-14 21:21:02 +03:00
parent 70216080b4
commit 89eada4cc1
12 changed files with 109 additions and 89 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
.claude/skills/e2e/SKILL.md
View File

@@ -13,7 +13,7 @@ description: Run, debug, and manage Playwright e2e tests. Use when running e2e t
- Port calculation: `E2E_BASE_PORT = PORT (from .env) + 500`. Worker N uses port base+N, except ports on the WHATWG fetch bad port list (e.g. 6679) are skipped — see `e2eWorkerPort` in `e2e/helpers/playwright.ts`
- Worker count: `E2E_WORKERS` env, defaulting to `min(8, max(4, cores - 2))`
- Worker databases: `db-test-e2e-<N>.sqlite3` in the project root; every test starts from a wiped database holding only the admin (Sendou) and N-ZAP users, and builds its own data with the `factories` fixture
- MinIO (S3-compatible storage) is started via Docker Compose if not already running
- SeaweedFS (S3-compatible storage) is started via Docker Compose if not already running
## Pre-flight checks (run before every test execution)
@@ -25,7 +25,7 @@ Before running tests, check for these common issues:
```
If ports are occupied by leftover e2e servers, kill them. If occupied by something else, warn the user.
2. **Docker running** — MinIO requires Docker. Check with `docker info` if there are storage-related failures.
2. **Docker running** — SeaweedFS requires Docker. Check with `docker info` if there are storage-related failures.
Stale worker databases (`db-test-e2e-*.sqlite3`) are handled automatically: global setup applies pending migrations and rebuilds databases whose migration history has drifted.
@@ -65,7 +65,7 @@ Follow this funnel when tests fail:
Common infrastructure errors and fixes:
- **"table already exists"** → Should not happen anymore (global setup rebuilds drifted worker DBs); if it does, `rm -f db-test-e2e-*.sqlite3` and investigate `scripts/ensure-test-db.ts`
- **"Server on port X did not start within timeout"** → Port conflict or app build error. Check ports with `lsof -i :<port>` and check for build errors
- **"MinIO failed to start"** → Docker not running or compose issue. Check `docker info`
- **"SeaweedFS failed to start"** → Docker not running or compose issue. Check `docker info`
- **"Test ended with database writes the server never saw"** → A factory call was not followed by a helper that talks to the server; add a `navigate`/`impersonate` after the writes
### Step 3: Reduce to single debug worker

4
.env.example
View File

@@ -13,8 +13,8 @@ PATREON_ACCESS_TOKEN=
// Image upload
STORAGE_END_POINT=http://127.0.0.1:9000
STORAGE_ACCESS_KEY=minio-user
STORAGE_SECRET=minio-password
STORAGE_ACCESS_KEY=seaweedfs-user
STORAGE_SECRET=seaweedfs-password
STORAGE_REGION=us-east-1
STORAGE_BUCKET=sendou

22
.github/workflows/e2e-tests.yml vendored
View File

@@ -20,23 +20,23 @@ jobs:
steps:
- uses: actions/checkout@v4
- name: Start MinIO
run: docker compose up -d minio
- name: Start SeaweedFS
run: docker compose up -d seaweedfs
# the bucket, not the health endpoint: MinIO answers health/live while it is still
# bootstrapping, so a run whose bucket never got created would only surface much later
# as a 500 in the one test that uploads an image
- name: Wait for MinIO to be ready
# the marker object, not the port: SeaweedFS answers on its S3 port, and even lists the
# bucket, while its volume server is still registering, so a premature run would only
# surface much later as a 500 in the one test that uploads an image
- name: Wait for SeaweedFS to be ready
run: |
for i in {1..30}; do
if curl -sf -o /dev/null http://127.0.0.1:9000/sendou/; then
echo "MinIO is ready"
if curl -sf -o /dev/null http://127.0.0.1:9000/sendou/.ready; then
echo "SeaweedFS is ready"
exit 0
fi
echo "Waiting for MinIO... ($i/30)"
echo "Waiting for SeaweedFS... ($i/30)"
sleep 2
done
echo "MinIO failed to start"
echo "SeaweedFS failed to start"
exit 1
- uses: pnpm/action-setup@v4
@@ -55,7 +55,7 @@ jobs:
- name: Run E2E tests
run: pnpm run test:e2e
- name: Stop MinIO
- name: Stop SeaweedFS
if: always()
run: docker compose down

2
.gitignore vendored
View File

@@ -30,7 +30,7 @@ dump
# Vitest auto-captured failure screenshots (numbered, without browser info)
# Real baselines have pattern: *-chromium-darwin.png
**/__screenshots__/**/*-[0-9].png
.e2e-minio-started
.e2e-seaweedfs-started
.e2e-build-marker
notepad.txt

4
README.md
View File

@@ -106,13 +106,13 @@ Static assets (weapon, gear and stage images, badges, sounds, etc.) are not stor
#### Docker
Optionally, if you want to develop image upload, you can use Docker to spin up Minio for image hosting. You will need [Docker](https://www.docker.com/) up and running and then run the following command:
Optionally, if you want to develop image upload, you can use Docker to spin up [SeaweedFS](https://github.com/seaweedfs/seaweedfs) for image hosting. You will need [Docker](https://www.docker.com/) up and running and then run the following command:
```
docker compose up -d
```
Minio admin UI to manage uploaded photos should be up and running at http://localhost:9001
It serves an S3 compatible API at http://localhost:9000 and creates the `sendou` bucket on startup. Uploaded photos can be browsed in the SeaweedFS filer UI at http://localhost:8888/buckets/sendou/
#### Windows performance tips

4
app/config.server.ts
View File

@@ -26,8 +26,8 @@ const schema = v.pipe(
DISCORD_CLIENT_SECRET: requiredInProd(isProd, ""),
STORAGE_END_POINT: requiredInProd(isProd, "http://127.0.0.1:9000"),
STORAGE_ACCESS_KEY: requiredInProd(isProd, "minio-user"),
STORAGE_SECRET: requiredInProd(isProd, "minio-password"),
STORAGE_ACCESS_KEY: requiredInProd(isProd, "seaweedfs-user"),
STORAGE_SECRET: requiredInProd(isProd, "seaweedfs-password"),
STORAGE_REGION: requiredInProd(isProd, "us-east-1"),
STORAGE_BUCKET: requiredInProd(isProd, "sendou"),

35
compose.yaml
View File

@@ -1,24 +1,25 @@
services:
minio:
image: "minio/minio:latest"
seaweedfs:
image: "chrislusf/seaweedfs:latest"
ports:
- "9000:9000"
- "9001:9001"
environment:
MINIO_ROOT_USER: minio-user
MINIO_ROOT_PASSWORD: minio-password
- "8888:8888"
entrypoint: >
/bin/sh -c '
isAlive() { curl -sf http://127.0.0.1:9000/minio/health/live; } # check if Minio is alive
minio $$0 "$$@" --quiet & echo $$! > /tmp/minio.pid # start Minio in the background
while ! isAlive; do sleep 0.1; done # wait until Minio is alive
until mc alias set minio http://127.0.0.1:9000 minio-user minio-password; do sleep 0.5; done # setup Minio client (health/live can answer before credentials are accepted)
until mc mb --ignore-existing minio/sendou; do sleep 0.5; done # create a test bucket
until mc anonymous set public minio/sendou; do sleep 0.5; done # make the test bucket public
kill -s INT $$(cat /tmp/minio.pid) && rm /tmp/minio.pid # stop Minio
while isAlive; do sleep 0.1; done # wait until Minio is stopped
exec minio $$0 "$$@" # start Minio in the foreground
isReady() { curl -sf -o /dev/null http://127.0.0.1:9000/sendou/.ready; } # reading the marker back over S3 proves the whole chain is up
(
until isReady; do
echo "s3.bucket.create -name sendou" | weed shell -master=127.0.0.1:9333 >/dev/null 2>&1 # create the test bucket
curl -sf -o /dev/null -T /etc/hostname http://127.0.0.1:8888/buckets/sendou/.ready # write the marker through the filer
sleep 1
done
) &
exec weed $$0 "$$@" # start SeaweedFS in the foreground
'
volumes:
- ~/minio/data:/data
command: server /data --console-address ":9001"
- ./docker/seaweedfs-s3.json:/etc/seaweedfs/s3.json:ro
- seaweedfs-data:/data
command: server -dir=/data -filer -s3 -s3.port=9000 -s3.config=/etc/seaweedfs/s3.json
volumes:
seaweedfs-data:

18
docker/seaweedfs-s3.json Normal file
View File

@@ -0,0 +1,18 @@
{
"identities": [
{
"name": "sendou",
"credentials": [
{
"accessKey": "seaweedfs-user",
"secretKey": "seaweedfs-password"
}
],
"actions": ["Admin", "Read", "List", "Tagging", "Write"]
},
{
"name": "anonymous",
"actions": ["Read", "List"]
}
]
}

47
e2e/global-setup.ts
View File

@@ -10,10 +10,10 @@ import {
const DEBUG = process.env.E2E_DEBUG === "true";
const SERVER_PROCESSES: ChildProcess[] = [];
const MINIO_MARKER_FILE = ".e2e-minio-started";
const SEAWEEDFS_MARKER_FILE = ".e2e-seaweedfs-started";
const STORAGE_BUCKET = "sendou";
/** Anonymously listable only once the bucket exists and its public policy is set. */
const MINIO_BUCKET_URL = `http://127.0.0.1:9000/${STORAGE_BUCKET}/`;
/** Marker object the container writes once the bucket exists; reading it back proves object reads work. */
const SEAWEEDFS_READY_URL = `http://127.0.0.1:9000/${STORAGE_BUCKET}/.ready`;
const BUILD_MARKER_FILE = ".e2e-build-marker";
const BUILD_INPUTS = [
"app",
@@ -28,24 +28,25 @@ declare global {
}
/**
* Whether the image storage is usable, which takes more than MinIO answering its health check:
* the container bootstraps the bucket only after startup, and a run whose bucket never got created
* would otherwise fail deep inside the one test that uploads an image (`art.spec.ts`) with an
* opaque 500.
* Whether the image storage is usable, which takes more than SeaweedFS answering on its port:
* the bucket is bootstrapped only after startup, and the volume server registers with the master
* later still, so listing the bucket succeeds while object reads are answering 500. Fetching the
* marker object exercises the whole chain — otherwise a run would fail deep inside the one test
* that uploads an image (`art.spec.ts`) with an opaque 500.
*/
async function isMinioBucketReady(): Promise<boolean> {
async function isSeaweedfsReady(): Promise<boolean> {
try {
const response = await fetch(MINIO_BUCKET_URL);
const response = await fetch(SEAWEEDFS_READY_URL);
return response.ok;
} catch {
return false;
}
}
async function waitForMinio(timeout = 60000): Promise<boolean> {
async function waitForSeaweedfs(timeout = 60000): Promise<boolean> {
const start = Date.now();
while (Date.now() - start < timeout) {
if (await isMinioBucketReady()) {
if (await isSeaweedfsReady()) {
return true;
}
await new Promise((resolve) => setTimeout(resolve, 1000));
@@ -53,28 +54,28 @@ async function waitForMinio(timeout = 60000): Promise<boolean> {
return false;
}
async function ensureMinioRunning(): Promise<boolean> {
if (await isMinioBucketReady()) {
async function ensureSeaweedfsRunning(): Promise<boolean> {
if (await isSeaweedfsReady()) {
// biome-ignore lint/suspicious/noConsole: CLI script output
console.log("MinIO is already running");
console.log("SeaweedFS is already running");
return false;
}
// biome-ignore lint/suspicious/noConsole: CLI script output
console.log("Starting MinIO...");
execSync("docker compose up -d minio", { stdio: "inherit" });
console.log("Starting SeaweedFS...");
execSync("docker compose up -d seaweedfs", { stdio: "inherit" });
const isReady = await waitForMinio();
const isReady = await waitForSeaweedfs();
if (!isReady) {
throw new Error(
`MinIO did not become usable within timeout (${MINIO_BUCKET_URL} never answered OK). If MinIO is running, its "${STORAGE_BUCKET}" bucket is missing or not public — recreate the container with "docker compose up -d --force-recreate minio".`,
`SeaweedFS did not become usable within timeout (${SEAWEEDFS_READY_URL} never answered OK). If SeaweedFS is running, its "${STORAGE_BUCKET}" bucket never finished bootstrapping — recreate the container with "docker compose up -d --force-recreate seaweedfs".`,
);
}
// biome-ignore lint/suspicious/noConsole: CLI script output
console.log("MinIO is ready");
console.log("SeaweedFS is ready");
fs.writeFileSync(MINIO_MARKER_FILE, "");
fs.writeFileSync(SEAWEEDFS_MARKER_FILE, "");
return true;
}
@@ -150,7 +151,7 @@ async function globalSetup(config: FullConfig) {
// biome-ignore lint/suspicious/noConsole: CLI script output
console.log(`\nStarting e2e test setup with ${workerCount} workers...`);
await ensureMinioRunning();
await ensureSeaweedfsRunning();
if (isBuildFresh()) {
// biome-ignore lint/suspicious/noConsole: CLI script output
@@ -217,8 +218,8 @@ async function globalSetup(config: FullConfig) {
VITE_SITE_DOMAIN: `http://localhost:${port}`,
VITE_E2E_TEST_RUN: "true",
STORAGE_END_POINT: "http://127.0.0.1:9000",
STORAGE_ACCESS_KEY: "minio-user",
STORAGE_SECRET: "minio-password",
STORAGE_ACCESS_KEY: "seaweedfs-user",
STORAGE_SECRET: "seaweedfs-password",
STORAGE_REGION: "us-east-1",
STORAGE_BUCKET,
// creds from .env must not reach test servers (SyncLiveStreams would

14
e2e/global-teardown.ts
View File

@@ -2,7 +2,7 @@ import { execSync } from "node:child_process";
import fs from "node:fs";
import type { FullConfig } from "@playwright/test";
const MINIO_MARKER_FILE = ".e2e-minio-started";
const SEAWEEDFS_MARKER_FILE = ".e2e-seaweedfs-started";
declare global {
var __E2E_SERVERS__: import("node:child_process").ChildProcess[];
@@ -39,16 +39,16 @@ async function globalTeardown(_config: FullConfig) {
new Promise((resolve) => setTimeout(resolve, 2000)),
]);
// only stop MinIO if we started it
if (fs.existsSync(MINIO_MARKER_FILE)) {
// only stop SeaweedFS if we started it
if (fs.existsSync(SEAWEEDFS_MARKER_FILE)) {
// biome-ignore lint/suspicious/noConsole: CLI script output
console.log("Stopping MinIO...");
console.log("Stopping SeaweedFS...");
try {
execSync("docker compose stop minio", { stdio: "inherit" });
execSync("docker compose stop seaweedfs", { stdio: "inherit" });
} catch {
// Ignore errors - MinIO might already be stopped
// Ignore errors - SeaweedFS might already be stopped
}
fs.unlinkSync(MINIO_MARKER_FILE);
fs.unlinkSync(SEAWEEDFS_MARKER_FILE);
}
// biome-ignore lint/suspicious/noConsole: CLI script output

40
scripts/seed-images.ts
View File

@@ -12,7 +12,7 @@ import {
SEED_NUMBERED_LOGOS,
} from "./seed-art-urls";
async function checkMinioConnection(): Promise<boolean> {
async function checkStorageConnection(): Promise<boolean> {
try {
const {
STORAGE_END_POINT,
@@ -66,7 +66,7 @@ async function readLocalImage(filename: string): Promise<Buffer> {
return await readFile(imagePath);
}
async function uploadToMinio(
async function uploadToStorage(
imageBuffer: Buffer,
filename: string,
): Promise<string> {
@@ -105,7 +105,7 @@ async function uploadToMinio(
return filename;
}
async function fileExistsInMinio(filename: string): Promise<boolean> {
async function fileExistsInStorage(filename: string): Promise<boolean> {
try {
const {
STORAGE_END_POINT,
@@ -137,11 +137,11 @@ async function fileExistsInMinio(filename: string): Promise<boolean> {
}
export async function seedImages(): Promise<void> {
const minioAvailable = await checkMinioConnection();
const storageAvailable = await checkStorageConnection();
if (!minioAvailable) {
if (!storageAvailable) {
logger.warn(
"⚠️ Minio is not available. Skipping image seeding. Make sure Docker is running if you want to seed images.",
"⚠️ SeaweedFS is not available. Skipping image seeding. Make sure Docker is running if you want to seed images.",
);
return;
}
@@ -158,25 +158,25 @@ export async function seedImages(): Promise<void> {
const smallFilename = filename.replace(/\.(\w+)$/, "-small.$1");
try {
const regularExists = await fileExistsInMinio(filename);
const smallExists = await fileExistsInMinio(smallFilename);
const regularExists = await fileExistsInStorage(filename);
const smallExists = await fileExistsInStorage(smallFilename);
if (regularExists && smallExists) {
skippedCount++;
logger.info(
` ↷ Files ${filename} and ${smallFilename} already exist in Minio`,
` ↷ Files ${filename} and ${smallFilename} already exist in storage`,
);
} else {
const imageBuffer = await downloadImage(url);
if (!regularExists) {
logger.info(` Uploading ${filename} to Minio...`);
await uploadToMinio(imageBuffer, filename);
logger.info(` Uploading ${filename} to storage...`);
await uploadToStorage(imageBuffer, filename);
}
if (!smallExists) {
logger.info(` Uploading ${smallFilename} to Minio...`);
await uploadToMinio(imageBuffer, smallFilename);
logger.info(` Uploading ${smallFilename} to storage...`);
await uploadToStorage(imageBuffer, smallFilename);
}
successCount++;
@@ -212,25 +212,25 @@ export async function seedImages(): Promise<void> {
const smallFilename = filename.replace(/\.(\w+)$/, "-small.$1");
try {
const regularExists = await fileExistsInMinio(filename);
const smallExists = await fileExistsInMinio(smallFilename);
const regularExists = await fileExistsInStorage(filename);
const smallExists = await fileExistsInStorage(smallFilename);
if (regularExists && smallExists) {
localSkippedCount++;
logger.info(
` ↷ Files ${filename} and ${smallFilename} already exist in Minio`,
` ↷ Files ${filename} and ${smallFilename} already exist in storage`,
);
} else {
const imageBuffer = await readLocalImage(sourceFilename);
if (!regularExists) {
logger.info(` Uploading ${filename} to Minio...`);
await uploadToMinio(imageBuffer, filename);
logger.info(` Uploading ${filename} to storage...`);
await uploadToStorage(imageBuffer, filename);
}
if (!smallExists) {
logger.info(` Uploading ${smallFilename} to Minio...`);
await uploadToMinio(imageBuffer, smallFilename);
logger.info(` Uploading ${smallFilename} to storage...`);
await uploadToStorage(imageBuffer, smallFilename);
}
localSuccessCount++;

2
scripts/setup.ts
View File

@@ -21,7 +21,7 @@ async function main() {
}
}
logger.info("🖼️ Seeding images to Minio...");
logger.info("🖼️ Seeding images to storage...");
try {
await seedImages();
} catch (err) {