mirror of
https://github.com/MatthewL246/pretendo-docker.git
synced 2026-08-24 03:44:29 -05:00
Create server backup and restore scripts
This commit is contained in:
Matthew Lopez
@@ -1,52 +1,41 @@
|
||||
# Scripts
|
||||
|
||||
These are important scripts that are used for container setup and
|
||||
administration. Here is a list of each one and what it does.
|
||||
These are important scripts that are used for container setup and administration. Here is a list of each one and what it
|
||||
does.
|
||||
|
||||
- `run-in-container/`: These are scripts that, as the name suggests, are run
|
||||
inside containers. They should not be run directly, but rather through the
|
||||
other scripts.
|
||||
- `compile-custom-inkay.sh [--reset]`: This compiles a custom version of the
|
||||
Inkay patches that whitelists the mitmproxy certificate for the Miiverse
|
||||
applet. It must be run if you want to use Juxt. Run with `--reset` to reset
|
||||
the certificate to Pretendo's official certificate.
|
||||
- `run-in-container/`: These are scripts that, as the name suggests, are run inside containers. They should not be run
|
||||
directly, but rather through the other scripts.
|
||||
- `backup.sh [backup_name]`: This backs up MongoDB, PostgreSQL, MinIO, Redis, and mitmproxy data to the `backups`
|
||||
directory. Run this before doing something risky with the datbases to prevent data loss.
|
||||
- `compile-custom-inkay.sh [--reset]`: This compiles a custom version of the Inkay patches that whitelists the mitmproxy
|
||||
certificate for the Miiverse applet. It must be run if you want to use Juxt. Run with `--reset` to reset the
|
||||
certificate to Pretendo's official certificate.
|
||||
- `create-juxt-community.sh <name> <description> <comma-separated title IDs> [icon image path] [banner image path]`:
|
||||
This creates a community in Juxtaposition, which is required before posting
|
||||
anything. Specify a name, description, and linked title IDs, and optionally
|
||||
set a custom icon and banner image. It should be run once for each community
|
||||
you want to create.
|
||||
- `firstrun-*.sh`: These are scripts that are run on the first run of certain
|
||||
containers. They are run automatically by the main setup script and shouldn't
|
||||
be used after that.
|
||||
- `get-boss-keys.sh [--write]`: This validates the BOSS keys from the dumped key
|
||||
files in `console-files`. Run with `--write` to validate the keys and write
|
||||
them to the BOSS server environment variables file.
|
||||
- `make-pnid-dev.sh <PNID to give developer access>`: This sets the access level
|
||||
of a PNID to developer, which gives it administrative permissions across the
|
||||
servers. For example, this makes the PNID's posts "verified" on Juxt. It
|
||||
should be run after creating a new PNID that should have administrative
|
||||
permissions.
|
||||
- `setup-environment.sh <server IP address> [Wii U IP address] [3DS IP address]`:
|
||||
This sets up local environment variables in `*.local.env` files, including
|
||||
randomly-generated secrets. Important external secrets are logged to
|
||||
`secrets.txt` in the root of the repository. Specify the server's IP address
|
||||
(must be accessible to the console) and, optionally, IP addresses for the
|
||||
consoles for automatic FTP uploads. It should be run whenever this repository
|
||||
is updated and new servers are added or if the server or Wii U IP addresses
|
||||
change.
|
||||
- `setup-submodule-patches.sh`: This sets up the various submodules in`repos`
|
||||
and applies patches to them. It should also be run whenever this repository is
|
||||
updated and new servers are added.
|
||||
- `update-account-servers-database.sh`: This updates the database of account
|
||||
servers. It should be run whenever a new account server is added to this
|
||||
repository or the server IP address changes, and it is automatically run by
|
||||
This creates a community in Juxtaposition, which is required before posting anything. Specify a name, description, and
|
||||
linked title IDs, and optionally set a custom icon and banner image. It should be run once for each community you want
|
||||
to create.
|
||||
- `firstrun-*.sh`: These are scripts that are run on the first run of certain containers. They are run automatically by
|
||||
the main setup script and shouldn't be used after that.
|
||||
- `get-boss-keys.sh [--write]`: This validates the BOSS keys from the dumped key files in `console-files`. Run with
|
||||
`--write` to validate the keys and write them to the BOSS server environment variables file.
|
||||
- `make-pnid-dev.sh <PNID to give developer access>`: This sets the access level of a PNID to developer, which gives it
|
||||
administrative permissions across the servers. For example, this makes the PNID's posts "verified" on Juxt. It should
|
||||
be run after creating a new PNID that should have administrative permissions.
|
||||
- `restore.sh <backup_name>`: This restores MongoDB, PostgreSQL, MinIO, Redis, and mitmproxy data from a specific backup
|
||||
in the `backups` directory. Note that it also re-runs `setup-environment.sh` to ensure the environment is correct.
|
||||
- `setup-environment.sh <server IP address> [Wii U IP address] [3DS IP address]`: This sets up local environment
|
||||
variables in `*.local.env` files, including randomly-generated secrets. Important external secrets are logged to
|
||||
`secrets.txt` in the root of the repository. Specify the server's IP address (must be accessible to the console) and,
|
||||
optionally, IP addresses for the consoles for automatic FTP uploads. It should be run whenever this repository is
|
||||
updated and new servers are added or if the server or Wii U IP addresses change.
|
||||
- `setup-submodule-patches.sh`: This sets up the various submodules in`repos` and applies patches to them. It should
|
||||
also be run whenever this repository is updated and new servers are added.
|
||||
- `update-account-servers-database.sh`: This updates the database of account servers. It should be run whenever a new
|
||||
account server is added to this repository or the server IP address changes, and it is automatically run by
|
||||
`setup-environment.sh`.
|
||||
- `update-miiverse-endpoints.sh`: This updates the Miiverse endpoints for the
|
||||
discovery API server. It currently uses hard-coded values, so it should not be
|
||||
neccesary to run it again.
|
||||
- `update-postgres-password.sh`: This updates the password for the PostgreSQL
|
||||
database. It should be run whenever the password is regenerated, and it is
|
||||
automatically run by `setup-environment.sh`.
|
||||
- `upload-3ds-files.sh [--reset]`: This uploads required files from
|
||||
`/console-files` to your 3DS to help with connection setup. Run with `--reset`
|
||||
to reset the Juxt certificate to Pretendo's official certificate.
|
||||
- `update-miiverse-endpoints.sh`: This updates the Miiverse endpoints for the discovery API server. It currently uses
|
||||
hard-coded values, so it should not be neccesary to run it again.
|
||||
- `update-postgres-password.sh`: This updates the password for the PostgreSQL database. It should be run whenever the
|
||||
password is regenerated, and it is automatically run by `setup-environment.sh`.
|
||||
- `upload-3ds-files.sh [--reset]`: This uploads required files from `/console-files` to your 3DS to help with connection
|
||||
setup. Run with `--reset` to reset the Juxt certificate to Pretendo's official certificate.
|
||||
|
||||
45
scripts/backup.sh
Executable file
45
scripts/backup.sh
Executable file
@@ -0,0 +1,45 @@
|
||||
#! /bin/sh
|
||||
|
||||
set -eu
|
||||
|
||||
git_base=$(git rev-parse --show-toplevel)
|
||||
. "$git_base/scripts/.function-lib.sh"
|
||||
|
||||
backup_name="${1-}"
|
||||
if [ -z "$backup_name" ]; then
|
||||
backup_name="backup_$(date +%Y-%m-%dT%H.%M.%S)"
|
||||
fi
|
||||
|
||||
backup_dir="$git_base/backups/$backup_name"
|
||||
if [ ! -d "$backup_dir" ]; then
|
||||
mkdir -p "$backup_dir"
|
||||
fi
|
||||
info "Backing up to $backup_dir"
|
||||
|
||||
docker compose up -d mitmproxy-pretendo mongodb postgres minio redis
|
||||
|
||||
info "Backing up MongoDB..."
|
||||
docker compose exec mongodb rm -rf /tmp/backup
|
||||
docker compose exec mongodb mongodump -o /tmp/backup --quiet
|
||||
docker compose cp mongodb:/tmp/backup "$backup_dir/mongodb"
|
||||
docker compose exec mongodb rm -rf /tmp/backup
|
||||
|
||||
info "Backing up Postgres..."
|
||||
docker compose exec postgres sh -c 'pg_dumpall -U "$POSTGRES_USER" --clean' >"$backup_dir/postgres.sql"
|
||||
|
||||
info "Backing up MinIO..."
|
||||
docker compose exec minio sh -c 'mc alias set minio http://minio.pretendo.cc "$MINIO_ROOT_USER" "$MINIO_ROOT_PASSWORD"'
|
||||
docker compose exec minio rm -rf /tmp/backup
|
||||
docker compose exec minio mkdir -p /tmp/backup
|
||||
docker compose exec minio mc mirror minio/ /tmp/backup
|
||||
docker compose cp minio:/tmp/backup "$backup_dir/minio"
|
||||
docker compose exec minio rm -rf /tmp/backup
|
||||
|
||||
info "Backing up Redis..."
|
||||
docker compose exec redis redis-cli save
|
||||
docker compose cp redis:/data/dump.rdb "$backup_dir/redis.rdb"
|
||||
|
||||
info "Backing up Mitmproxy..."
|
||||
docker compose cp mitmproxy-pretendo:/home/mitmproxy/.mitmproxy "$backup_dir/mitmproxy"
|
||||
|
||||
success "Backup completed successfully."
|
||||
66
scripts/restore.sh
Executable file
66
scripts/restore.sh
Executable file
@@ -0,0 +1,66 @@
|
||||
#! /bin/sh
|
||||
|
||||
set -eu
|
||||
|
||||
git_base=$(git rev-parse --show-toplevel)
|
||||
. "$git_base/scripts/.function-lib.sh"
|
||||
|
||||
if [ -z "${1-}" ]; then
|
||||
echo "Usage: $0 <backup_name>"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
backup_name="$1"
|
||||
backup_dir="$git_base/backups/$backup_name"
|
||||
if [ ! -d "$backup_dir" ]; then
|
||||
error "Backup directory $backup_dir does not exist."
|
||||
exit 1
|
||||
fi
|
||||
info "Restoring from $backup_dir"
|
||||
|
||||
if [ "${2-}" != "--force" ]; then
|
||||
warning "Restoring a backup will overwrite your current Pretendo server data. Backing up your data first is strongly recommended."
|
||||
printf "Continue? [y/N] "
|
||||
read -r continue
|
||||
if [ "$continue" != "Y" ] && [ "$continue" != "y" ]; then
|
||||
echo "Aborting."
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
info "Stopping unnecessary services..."
|
||||
docker compose down
|
||||
docker compose up -d mitmproxy-pretendo mongodb postgres minio redis
|
||||
|
||||
info "Restoring MongoDB..."
|
||||
docker compose exec mongodb rm -rf /tmp/backup
|
||||
docker compose cp "$backup_dir/mongodb" mongodb:/tmp/backup
|
||||
docker compose exec mongodb mongorestore /tmp/backup --drop --quiet
|
||||
docker compose exec mongodb rm -rf /tmp/backup
|
||||
|
||||
info "Restoring Postgres..."
|
||||
# According to the pg_dumpall documentation, dropping and creating the superuser role is expected to cause an error
|
||||
docker compose exec -T postgres sh -c 'psql -U "$POSTGRES_USER" -d postgres' <"$backup_dir/postgres.sql" >/dev/null
|
||||
|
||||
info "Restoring MinIO..."
|
||||
docker compose exec minio /bin/sh -c 'mc alias set minio http://minio.pretendo.cc "$MINIO_ROOT_USER" "$MINIO_ROOT_PASSWORD"'
|
||||
docker compose exec minio rm -rf /tmp/backup
|
||||
docker compose cp "$backup_dir/minio" minio:/tmp/backup
|
||||
docker compose exec minio mc mirror /tmp/backup minio/ --overwrite --remove
|
||||
docker compose exec minio rm -rf /tmp/backup
|
||||
|
||||
info "Restoring Redis..."
|
||||
# Redis cannot be running when restoring a dump or it will overwrite the restored dump when it exits
|
||||
docker compose stop redis
|
||||
docker compose cp "$backup_dir/redis.rdb" redis:/data/dump.rdb
|
||||
docker compose start redis
|
||||
|
||||
info "Restoring Mitmproxy..."
|
||||
docker compose cp "$backup_dir/mitmproxy" mitmproxy-pretendo:/home/mitmproxy/.mitmproxy
|
||||
|
||||
# The restored backup might be using a different password than what is currently in the .env files
|
||||
info "Now running the environment setup script to regenerate database passwords."
|
||||
. "$git_base/environment/system.local.env"
|
||||
"$git_base/scripts/setup-environment.sh" "$SERVER_IP" "${WIIU_IP-}" "${DS_IP-}"
|
||||
|
||||
success "Restore completed successfully."
|
||||
Reference in New Issue
Block a user