Better README & getting started (#2284)

* initial * wip * wip * wip * pr comment * remove todo * add a few placeholders * todos done
2026-10-01 07:37:55 -05:00 · 2025-05-26 17:57:41 +03:00
parent dd2cdacfe7
commit 6ba4b9d6ff
19 changed files with 1394 additions and 254 deletions
--- a/docs/dev/api.md
+++ b/docs/dev/api.md
@@ -0,0 +1,18 @@
+# API
+
+API for external projects to access sendou.ink data for projects such as streams is available. This API is for reading data, writing is not supported. You will need a token to access the API. Currently access is limited but you can request a token from Sendou.
+
+## Endpoints
+
+Check out `sendou.ink/app/features/api-public/schema.ts`
+
+## Curl example
+
+```bash
+sendou@macbook ~ % curl -H "Authorization: Bearer mytoken" https://sendou.ink/api/tournament/1 
+{"name":"PICNIC mini","startTime":"2023-05-18T18:00:00.000Z","url":"https://sendou.ink/to/1/brackets","logoUrl":"https://sendou.ink/static-assets/img/tournament-logos/pn.png","teams":{"checkedInCount":25,"registeredCount":31},"brackets":[{"name":"Main bracket","type":"double_elimination"}],"organizationId":1,"isFinalized":true}% 
+```
+
+## Clients (unofficial)
+
+- [sendou.py (Python)](https://github.com/IPLSplatoon/sendou.py)
--- a/docs/dev/architecture.md
+++ b/docs/dev/architecture.md
@@ -0,0 +1,201 @@
+# Architecture
+
+Note: some code in the project is older and some newer. Not everything follows the concepts and structure as explained here. PR's welcome to improve the situation.
+
+## Diagram
+
+Here is how the application architecture looks like in production.
+
+```mermaid
+graph TD
+    subgraph Render
+        A[sendou.ink Server] -->|Reads/Writes| B[SQLite3 Database]
+        A -->|HTTP Requests| E[Skalop WebSocket Server]
+        D[Lohi Discord Bot] -->|HTTP Requests| A
+    end
+
+    subgraph DigitalOcean
+        C[S3-Compatible Image Hosting]
+    end
+
+    F[User] -->|HTTP & WS| G[Cloudflare]
+    G -->|HTTP| A
+    G -->|WebSocket| E
+
+    A -->|S3 Upload| C
+    F -->|Views images| C
+
+```
+
+List of the dependencies in production:
+
+- [Skalop](https://github.com/Sendouc/skalop) - WebSocket server
+- [Lohi](https://github.com/Sendouc/lohi) - Discord bot for profile updates, log-in links etc.
+- [Leanny/splat3](https://github.com/Leanny/splat3) - In-game data (manual update) 
+- [splatoon3.ink](https://github.com/misenhower/splatoon3.ink) - X Rank placement data (manual update)
+- Discord - Auth  
+- Twitch - Streams  
+- Bluesky - Front page changelog
+
+## Folder structure
+
+```
+sendou.ink/
+├── app/
+│   ├── components/ -- React components used by many features
+│   │   └── elements/ -- Wrappers providing styling etc. around React Aria Components
+│   ├── db/ -- Database seeds, types & connection
+│   ├── features/ -- See "feature folders" below
+│   ├── hooks/ -- React hooks used by many features
+│   ├── modules/ -- "node_modules but part of the app"
+│   ├── styles/ -- Global .css files
+│   ├── utils/ -- Helper functions grouped by domain used by many features
+│   ├── entry.client.tsx -- Client entry point (Remix concept)
+│   ├── entry.server.tsx -- Server entry point (Remix concept)
+│   ├── root.tsx -- Basic HTML structure, React context providers & root data loader
+│   └── routes.ts -- Route manifest
+├── content/ -- Markdown files containing articles
+├── docs/ -- Documentation to developers and users
+├── e2e/ -- Playwright tests
+├── locales/ -- Translation files
+├── migrations/ -- Database migrations
+├── public/ -- Images, built assets etc. static files to be served as is
+├── scripts/ -- Stand-alone scripts to be run outside of the app (i.e. not imported)
+└── types/ -- "global" type overwrites
+```
+
+## Feature folders
+
+Feature folders collect together all the code needed to make that particular feature happen: database, backend, frontend, core logic etc. Feature can mean an user facing feature like "map-planner" but also something of a more cross-cutting concern like "chat".
+
+You should aim to colocate code that "changes together" as much as possible. Features can depend (import) on other features. 
+
+### Feature folder files & folders
+
+- **actions/**: Remix actions per route
+- **components/**: React components
+- **core/**: "Core logic" meaning modules (see below) or other logic that is not typically rendering components or calling database
+- **queries/**: (deprecated) Database queries, should use repository instead
+- **loaders/**: Remix loaders per route
+- **routes/**: Remix actions per route
+- **FeatureRepository.server.ts**: Database queries & mappers
+- **feature-constants.ts**: Constant values
+- **feature-hooks**: React hooks
+- **feature-schemas.ts**: Zod schemas for validating form values, params, payloads
+- **feature-types.ts**: Typescript types
+- **feature-utils.ts**: Utilities too small to make up for their own modules
+- **feature.css**: (deprecated) CSS, should use CSS modules instead
+
+Note: we are not using file-based routing. To add a new route `routes.ts` needs to be updated  
+Note: a route file needs to re-export the action/loader of that route
+
+### Feature modules
+
+Define in a core folder:
+
+```ts
+// app/features/cool-feature/core/Module.ts
+
+/** Descriptive JSDoc goes here */
+export function doTheThing() {
+
+}
+function implementationDetail() {
+
+}
+```
+
+You should document any functions exported by the module well.
+
+Usage:
+
+```ts
+// anywhere else in the codebase, particularly inside that feature
+import * as Module from "../core/Module.ts"
+
+Module.doTheThing()
+```
+
+## Concepts
+
+### Testing
+
+Testing is important part of every feature work. The approach the project takes is pragmatic not super focused on writing test for every single thing but especially more mission critical features should have a better test coverage. E.g. if a tournament is canceled due to a bug that can mean a lot of lost confidence from users and waste of time but if some "edge of the system" type of feature has small graphical bugs we can just fix that on user feedback.
+
+Unit testing "core logic" (i.e. no React, no DB calls) with Vitest is highly encouraged whenever feasible. Most tests are like this.
+
+Vitest can also be used to write "integration tests" that call mocked actions/loaders (see `admin.test.ts` for example). This uses in-memory SQLite3. In practice this is best sparingly as they are typically slower than pure unit tests with more dependencies but also don't test the true end to end flow.
+
+Which brings us to E2E tests. For new features at least testing the happy path is encouraged. For more critical features (mainly tournament related stuff) it makes sense to test a bit more rigorously.
+
+See: [Playwright best practices](https://playwright.dev/docs/best-practices)
+
+### Authentication
+
+Accessing logged in user in React components:
+
+```tsx
+const user = useUser();
+```
+
+Accessing logged in user in loaders/actions:
+
+```ts
+const user = await requireUser(request); // get user or throw HTTP 401 if not logged in
+const user = await getUser(request); // get user (undefined if not logged in)
+```
+
+### Permissions
+
+1) Add a permission object in a `Repository` code.
+2) Read in React code via the `useHasPermission` hook.
+3) Read in server code via the `requirePermission` guard.
+
+User can also have global roles such as "staff" or "tournament adder". Set in the root loader and `getUser`/`requireUser` code.
+
+### Anatomy of an action
+
+TODO (after React server actions in use)
+
+### Performance
+
+Keeping server performance in mind is always necessary. Due to the monolithic nature of the server one badly optimized endpoint impacts all other routes.
+
+Use a load testing tool like `autocannon` to ensure new features scale.
+
+### Database
+
+Sendou.ink uses SQLite3 for its database solution. See for example ["Consider SQLite"](https://blog.wesleyac.com/posts/consider-sqlite) for motivation why to pick SQLite for a web project over something like PostgreSQL. Tldr; for a project of this scale it gets you far, low latency when accessing data store & simplifies testing when your database is just a file on the filesystem. When writing code it should be kept in mind that writes to the database are not concurrent so abusing the database can lead to the whole web server process freezing essentially.
+
+Check `database-relations.md` for more information about the database relations. See `tables.ts` for documentation on tables and columns.
+
+### React guidelines
+
+- Write modern React code as described by the documentation e.g. [seldom using useEffect](https://react.dev/learn/you-might-not-need-an-effect)
+- We use React Compiler so writing memos manually (useMemo, useCallback, React.memo) should normally not be needed
+- Structuring longer components to sub-components located in the same file is encouraged
+
+### State management
+
+We are not using a state management library such as Redux. Instead use React Context for the few global state needed and Remix's data loading hooks to share the state loaded from server. See also "Search params" section below.
+
+### Search params
+
+Often it's convenient to store state in search params. This allows for nice features like users to deep link to the view they are seeing. You have two options to achieve this:
+
+1) Use Remix's built-in solution. Use this if data loaders should rerun once search params are changed.
+2) `useSearchParamState` hook. Use this if it is not needed.
+
+### Routines
+
+Cron jobs to perform actions on the server at certain intervals. To add a new one, add a new file exporting an instance of the `Routine` class then add it to the appropriate array in the `app/routines/list.server.ts` file.
+
+### Real time
+
+Webhooks via Skalop service (see logic in the Chat module).
+
+Old way: server-sent events still in use for tournament bracket & match pages.
+
+### Notifications
+
+Both in-app and browser notifications. See `/app/features/notifications`. Good for notifying user about actions that they are interested in that might have happened when they are offline.
--- a/docs/dev/database-relations.md
+++ b/docs/dev/database-relations.md
@@ -0,0 +1,200 @@
+Note: some simple features omitted with only a few relations and no special notes
+
+See `tables.ts` for some more documentation on column-level.
+
+## Art
+
+```mermaid
+erDiagram
+    Art ||--o{ ArtUserMetadata : has
+    User ||--o{ ArtUserMetadata : has
+
+    Art ||--o{ TaggedArt : tagged_with
+    ArtTag ||--o{ TaggedArt : tags
+
+    ArtTag ||--o{ User : created_by
+    Art ||--o{ User : created_by
+```
+
+## Badges
+
+```mermaid
+erDiagram
+    Badge ||--o{ BadgeManager : managed_by
+    User ||--o{ BadgeManager : manages
+
+    Badge ||--o{ TournamentBadgeOwner : owned_by
+    User ||--o{ TournamentBadgeOwner : owns
+
+    Badge }o--|| User : author
+```
+
+- **BadgeOwner** - Tournament badges with supporter badges included from user's supporter status
+
+## Builds
+
+```mermaid
+erDiagram
+    BuildAbility }|--|| Build : belongs_to
+    BuildWeapon }|--|| Build : belongs_to
+
+    Build }o--|| User : owned_by
+```
+
+## Calendar Events
+
+```mermaid
+erDiagram
+    CalendarEvent ||--o{ CalendarEventBadge : has
+    CalendarEventBadge }o--|| Badge : badge
+    CalendarEvent ||--|{ CalendarEventDate : has
+    CalendarEvent ||--o{ CalendarEventResultTeam : has
+    CalendarEventResultTeam ||--o{ CalendarEventResultPlayer : has
+    CalendarEvent }o--|| User : author
+    CalendarEvent }o--o| Organization : organized_by
+    CalendarEvent ||--o{ Tournament : related_to
+```
+
+### Notes
+
+- "Calendar event result" concept is only for tournaments not hosted on sendou.ink
+- Regular calendar event can have many dates, tournaments only one
+
+## Groups (SendouQ)
+
+```mermaid
+erDiagram
+    Group ||--o{ GroupMember : has
+    User ||--o{ GroupMember : member_of
+
+    Group ||--o{ GroupLike : likes
+    Group ||--o{ GroupLike : liked_by
+
+    Group ||--o| GroupMatch : alpha_in
+    Group ||--o| GroupMatch : bravo_in
+    User ||--o{ GroupMatch : reported_by
+
+    GroupMatch ||--|{ GroupMatchMap : has
+    Group ||--o| Team : team_id
+```
+
+### Notes
+
+- Even if a group rejoins the queue with the same players after the match, a new "Group" is created in the DB
+
+## LFG Posts
+
+```mermaid
+erDiagram
+    LFGPost }o--|| User : author
+    LFGPost }o--o| Team : team
+```
+
+## Map Pools
+
+```mermaid
+erDiagram
+    MapPoolMap }o--|| CalendarEvent : calendar_event
+    MapPoolMap }o--|| CalendarEvent : tie_breaker_calendar_event
+    MapPoolMap }o--|| TournamentTeam : tournament_team
+```
+
+### Notes
+
+Can be one of the following:
+1) Regular calendar events map pool
+2) Tournament's tiebreaker maps (teams' pick mode, AUTO_ALL)
+3) Tournament's map pool (TO's map picking mode)
+4) Tournament teams map picks (teams' pick mode, AUTO_ALL, AUTO_SZ etc.)
+
+## Plus Server Suggestions
+
+```mermaid
+erDiagram
+    PlusSuggestion }o--|| User : author
+    PlusSuggestion }o--|| User : suggested
+```
+
+### Notes
+
+- Comments to suggestions are also just suggestions same as new suggestions
+
+## Plus Server Tiers
+
+```mermaid
+erDiagram
+    PlusTier |o--|| User : userId
+```
+
+### Views
+- **FreshPlusTier** - Calculates Plus Server Tiers based on the latest voting results
+
+### Notes
+- PlusTier is just FreshPlusTier materialized for performance reasons with players from the leaderboard added
+
+## Results (maps/head-to-head)
+
+```mermaid
+erDiagram
+    User ||--o{ MapResult : has
+    PlayerResult }o--|| User : owner
+    PlayerResult }o--|| User : other 
+```
+
+### Notes
+
+- Denormalized tables to make fetching these efficient
+
+## Scrims
+
+```mermaid
+erDiagram
+    ScrimPost ||--|{ ScrimPostUser : has
+    ScrimPost ||--o{ ScrimPostRequest : has
+    ScrimPostRequest ||--|{ ScrimPostRequestUser : has
+
+    User ||--o{ ScrimPostUser : participates
+    User ||--o{ ScrimPostRequestUser : participates
+```
+
+## Teams
+
+```mermaid
+erDiagram
+    AllTeam ||--o{ AllTeamMember : has
+    User ||--o{ AllTeamMember : member_of
+```
+
+### Views
+
+- **Team** - Teams excluding disbanded
+- **TeamMember** - `AllTeamMember` excluding members who already left their team & secondary teams
+- **TeamMemberWithSecondary** - `AllTeamMember` excluding members who already left their team but including secondary teams
+
+## Tournaments
+
+The database structure is mimicking the `brackets-manager.js` library. See this issue for a schema: [https://github.com/Drarig29/brackets-manager.js/issues/111#issuecomment-997417423](https://github.com/Drarig29/brackets-manager.js/issues/111#issuecomment-997417423)
+
+## Tournament organizations
+
+```mermaid
+erDiagram
+    TournamentOrganization ||--|{ TournamentOrganizationMember : has_member
+    User ||--o{ TournamentOrganizationMember : member_of
+
+    TournamentOrganization ||--o{ TournamentOrganizationBadge : has_badge
+    Badge ||--o{ TournamentOrganizationBadge : badge_of
+
+    TournamentOrganization ||--o{ TournamentOrganizationSeries : has_series
+```
+
+## Videos
+```mermaid
+erDiagram
+    UnvalidatedVideo ||--|{ VideoMatch : has
+    VideoMatch ||--o{ VideoMatchPlayer : has
+```
+
+### Notes
+
+- `Video` - Same as `UnvalidatedVideo` (redundant)
--- a/docs/dev/how-to.md
+++ b/docs/dev/how-to.md
@@ -0,0 +1,54 @@
+# How to...
+
+Guides on how to do different things when developing sendou.ink
+
+## Fix style/lint errors (Biome)
+
+Run the `npm run biome:fix` command. Also you might want to set up Biome as an extension to your IDE and run automatically when you save a file.
+
+## Add a new database migration
+
+1) Add a new file to the migrations folder incrementing the last used number by one e.g. `011-my-cool-feature.js`. Note: there is no script to do this.
+2) Take this file as a base and fill it out with your migration:
+
+```js
+export function up(db) {
+	db.transaction(() => {
+		// your migrations go here
+	})();
+}
+```
+
+Note: No need to implement the "down" migration
+
+3) Update the typings in `app/db/tables.ts`
+4) Run `npm run migrate up` to apply your migration
+4) Set env var `DB_PATH=db-test.sqlite3` in `.env` file & run the `npm run migrate up` command again to update the database used in unit tests
+
+## Add a new translation string
+
+1) Decide on where the translation should go. Either `common.json` which is available in every route by default or a feature specific one such as `builds.json`
+2) Add the translation string to the json with some descriptive key
+3) Access in code via the `useTranslation` hook
+
+```json
+// common.json
+{
+  ...
+  "my-cool.translation": "Translated"
+  ...
+}
+```
+
+```tsx
+// CoolComponent.tsx
+export function CoolComponent() {
+  const { t } = useTranslation(["common"]);
+
+  return (
+    <div>{t("common:my-cool.translation")}</div>
+  )
+}
+```
+
+When utilizing feature specific translations ensure the json is loaded. This is handled via the `handle` Remix function.
--- a/docs/dev/scripts.md
+++ b/docs/dev/scripts.md
@@ -0,0 +1,75 @@
+# Scripts
+
+Note: These are mostly useful if you are running the site in production as an admin, not typically for development.
+
+---
+
+## Add new badge to the database
+
+```bash
+npx tsx scripts/add-badge.ts fire_green "Octofin Eliteboard"
+```
+
+## Rename display name of a badge
+
+```bash
+npx tsx scripts/rename-badge.ts 10 "New 4v4 Sundaes"
+```
+
+## Add many badge owners
+
+```bash
+npx tsx scripts/add-badge-winners.ts 10 "750705955909664791,79237403620945920"
+```
+
+## Converting gifs (badges) to thumbnail (.png)
+
+```bash
+sips -s format png ./sundae.gif --out .
+```
+
+## Convert many .png files to .avif
+
+While in the folder with the images:
+
+```bash
+for i in *.png; do npx @squoosh/cli --avif '{"cqLevel":33,"cqAlphaLevel":-1,"denoiseLevel":0,"tileColsLog2":0,"tileRowsLog2":0,"speed":6,"subsample":1,"chromaDeltaQ":false,"sharpness":0,"tune":0}' $i; done
+```
+
+Note: it only works with Node 16.
+
+## Doing monthly update
+
+1. Fill /scripts/dicts with new data from leanny repository:
+   - weapon = contents of `weapon` folder
+   - langs = contents of `language` folder
+   - Couple of others at the root: `GearInfoClothes.json`, `GearInfoHead.json`, `GearInfoShoes.json`, `spl__DamageRateInfoConfig.pp__CombinationDataTableData.json`, `SplPlayer.game__GameParameterTable.json`, `WeaponInfoMain.json`, `WeaponInfoSpecial.json` and `WeaponInfoSub.json`
+1. Update all `CURRENT_SEASON` constants
+1. Update `CURRENT_PATCH` constants
+1. Update `PATCHES` constant with the late patch + remove the oldest
+1. Update the stage list in `stage-ids.ts` and `create-misc-json.ts`. Add images from Lean's repository and avify them.
+1. `npx tsx scripts/create-misc-json.ts`
+1. `npx tsx scripts/create-gear-json.ts`
+1. `npx tsx scripts/create-analyzer-json.ts`
+   8a. Double check that no hard-coded special damages changed
+1. `npx tsx scripts/create-object-dmg-json.ts`
+1. Fill new weapon IDs by category to `weapon-ids.ts` (easy to take from the diff of English weapons.json)
+1. Get gear IDs for each slot from /output folder and update `gear-ids.ts`.
+1. Replace `object-dmg.json` with the `object-dmg.json` in /output folder
+1. Replace `weapon-params.ts` with the `params.json` in /output folder
+1. Delete all images inside `main-weapons`, `main-weapons-outlined`, `main-weapons-outlined-2` and `gear` folders.
+1. Replace with images from Lean's repository.
+1. Run the `npx tsx scripts/replace-img-names.ts` command
+1. Run the `npx tsx scripts/replace-weapon-names.ts` command
+1. Run the .avif generating command in each image folder.
+2. Update manually any languages that use English `gear.json` and `weapons.json` files
+
+## Download the production database from Render.com
+
+Note: This is only useful if you have access to a production running on Render.com
+
+1. Access the "Shell" tab
+2. `cd /var/data`
+3. `cp db.sqlite3 db-copy.sqlite3`
+4. `wormhole send db-copy.sqlite3`
+5. On the receiving computer use the command shown.