Rewrite scripts and improve user-friendliness

- Update all of the scripts to use Bash instead of sh so I can use arrays and other bashisms
- Create a complete Bash option parsing framework to replace function-lib.sh
- Rewrite all of the scripts to use this framework
- General script user-friendliness improvements, including loading existing environment variables by default instead of always requiring them to be specified
This commit is contained in:
Matthew Lopez
2024-05-10 17:22:25 -04:00
parent 76c38ca5d7
commit 05a67989cd
22 changed files with 681 additions and 442 deletions

View File

@@ -25,7 +25,7 @@ jobs:
run: yq eval "del(.services.*.ports)" -i compose.yml
- name: Run the setup script
run: ./setup.sh
run: ./setup.sh --server-ip 127.0.0.1 --force
# Testing whether any containers exit requires disabling restarts
- name: Remove container restart configuration

View File

@@ -1,33 +1,4 @@
# Scripts
These are important scripts that are used for container setup and administration. Here is a list of each one and what it
does.
- `internal/`: These scripts are consider internal and are run by other scripts.
- `run-in-container/`: These are scripts that, as the name suggests, are run inside containers. They should not be run
directly, but rather through the other scripts.
- `backup.sh [backup_name]`: This backs up MongoDB, PostgreSQL, MinIO, Redis, and mitmproxy data to the `backups`
directory. Run this before doing something risky with the datbases to prevent data loss.
- `compile-custom-inkay.sh [--reset]`: This compiles a custom version of the Inkay patches that whitelists the mitmproxy
certificate for the Miiverse applet. It must be run if you want to use Juxt. Run with `--reset` to reset the
certificate to Pretendo's official certificate.
- `create-juxt-community.sh <name> <description> <comma-separated title IDs> [icon image path] [banner image path]`:
This creates a community in Juxtaposition, which is required before posting anything. Specify a name, description, and
linked title IDs, and optionally set a custom icon and banner image. It should be run once for each community you want
to create.
- `get-boss-keys.sh [--write]`: This validates the BOSS keys from the dumped key files in `console-files`. Run with
`--write` to validate the keys and write them to the BOSS server environment variables file.
- `make-pnid-dev.sh <PNID to give developer access>`: This sets the access level of a PNID to developer, which gives it
administrative permissions across the servers. For example, this makes the PNID's posts "verified" on Juxt. It should
be run after creating a new PNID that should have administrative permissions.
- `restore.sh <backup_name>`: This restores MongoDB, PostgreSQL, MinIO, Redis, and mitmproxy data from a specific backup
in the `backups` directory. Note that it also re-runs `setup-environment.sh` to ensure the environment is correct.
- `setup-environment.sh <server IP address> [Wii U IP address] [3DS IP address]`: This sets up local environment
variables in `*.local.env` files, including randomly-generated secrets. Important external secrets are logged to
`secrets.txt` in the root of the repository. Specify the server's IP address (must be accessible to the console) and,
optionally, IP addresses for the consoles for automatic FTP uploads. It should be run whenever this repository is
updated and new servers are added or if the server or Wii U IP addresses change.
- `setup-submodule-patches.sh`: This sets up the various submodules in`repos` and applies patches to them. It should
also be run whenever this repository is updated and new servers are added.
- `upload-3ds-files.sh [--reset]`: This uploads required files from `/console-files` to your 3DS to help with connection
setup. Run with `--reset` to reset the Juxt certificate to Pretendo's official certificate.
These are important scripts that are used for container setup and administration. Run each script with `-h` or `--help`
to see a description and usage information.

View File

@@ -1,34 +1,36 @@
#! /bin/sh
#!/usr/bin/env bash
set -eu
# shellcheck source=./internal/framework.sh
source "$(dirname "$(realpath "$0")")/internal/framework.sh"
set_description "This backs up MongoDB, PostgreSQL, MinIO, Redis, and mitmproxy data to the backups directory. Run \
this before doing something risky with the datbases to prevent data loss."
add_positional_argument "backup-name" "backup_name" "Name of the backup directory, defaults to the current date and time" false
parse_arguments "$@"
git_base=$(git rev-parse --show-toplevel)
. "$git_base/scripts/internal/function-lib.sh"
backup_name="${1-}"
if [ -z "$backup_name" ]; then
if [[ -z "$backup_name" ]]; then
backup_name="backup_$(date +%Y-%m-%dT%H.%M.%S)"
fi
backup_dir="$git_base/backups/$backup_name"
if [ -d "$backup_dir" ]; then
error "Backup directory $backup_dir already exists."
backup_dir="$git_base_dir/backups/$backup_name"
if [[ -d "$backup_dir" ]]; then
print_error "Backup directory $backup_dir already exists."
exit 1
fi
mkdir -p "$backup_dir"
info "Backing up to $backup_dir"
print_info "Backing up to $backup_dir"
docker compose up -d mitmproxy-pretendo mongodb postgres minio redis
info "Backing up MongoDB..."
print_info "Backing up MongoDB..."
docker compose exec mongodb rm -rf /tmp/backup
docker compose exec mongodb mongodump -o /tmp/backup --quiet
docker compose cp mongodb:/tmp/backup "$backup_dir/mongodb"
docker compose exec mongodb rm -rf /tmp/backup
info "Backing up Postgres..."
print_info "Backing up Postgres..."
docker compose exec postgres sh -c 'pg_dumpall -U "$POSTGRES_USER" --clean' >"$backup_dir/postgres.sql"
info "Backing up MinIO..."
print_info "Backing up MinIO..."
docker compose exec minio sh -c 'mc alias set minio http://minio.pretendo.cc "$MINIO_ROOT_USER" "$MINIO_ROOT_PASSWORD"'
docker compose exec minio rm -rf /tmp/backup
docker compose exec minio mkdir -p /tmp/backup
@@ -36,11 +38,11 @@ docker compose exec minio mc mirror minio/ /tmp/backup
docker compose cp minio:/tmp/backup "$backup_dir/minio"
docker compose exec minio rm -rf /tmp/backup
info "Backing up Redis..."
print_info "Backing up Redis..."
docker compose exec redis redis-cli save
docker compose cp redis:/data/dump.rdb "$backup_dir/redis.rdb"
info "Backing up Mitmproxy..."
print_info "Backing up Mitmproxy..."
docker compose cp mitmproxy-pretendo:/home/mitmproxy/.mitmproxy "$backup_dir/mitmproxy"
success "Backup completed successfully."
print_success "Backup completed successfully."

View File

@@ -1,31 +1,28 @@
#! /bin/sh
#!/usr/bin/env bash
set -eu
# shellcheck source=./internal/framework.sh
source "$(dirname "$(realpath "$0")")/internal/framework.sh"
set_description "This compiles a custom version of the Inkay patches that whitelists the mitmproxy certificate for the \
Miiverse applet. It must be run if you want to use Juxt."
add_option "-r --reset" "should_reset" "Resets the Inkay CA certificate to Pretendo's instead of using the mitmproxy certificate"
parse_arguments "$@"
should_reset=false
if [ "${1-}" = "--reset" ]; then
should_reset=true
fi
git_base=$(git rev-parse --show-toplevel)
. "$git_base/scripts/internal/function-lib.sh"
if [ ! -f "$git_base/environment/system.local.env" ]; then
error "Missing environment file system.local.env. Did you run setup-environment.sh?"
if [[ ! -f "$git_base_dir/environment/server.local.env" ]]; then
print_error "Missing environment file server.local.env. Did you run setup-environment.sh?"
exit 1
fi
. "$git_base/environment/system.local.env"
if [ -z "${WIIU_IP+x}" ]; then
warning "Missing environment variable WIIU_IP. Did you specify a Wii U IP address when you ran setup-environment.sh?"
info "Continuing without automatic FTP upload."
source "$git_base_dir/environment/server.local.env"
if [[ -z "${WIIU_IP:-}" ]]; then
print_warning "Missing environment variable WIIU_IP. Did you specify a Wii U IP address when you ran setup-environment.sh?"
print_info "Continuing without automatic FTP upload."
fi
cd "$git_base/repos/Inkay"
cd "$git_base_dir/repos/Inkay"
if [ "$should_reset" = false ]; then
if [[ -z "$should_reset" ]]; then
docker compose up -d mitmproxy-pretendo
while ! docker compose exec mitmproxy-pretendo ls /home/mitmproxy/.mitmproxy/mitmproxy-ca-cert.pem >/dev/null 2>&1; do
info "Waiting for mitmproxy to generate a certificate..."
print_info "Waiting for mitmproxy to generate a certificate..."
sleep 1
done
@@ -33,10 +30,10 @@ if [ "$should_reset" = false ]; then
docker compose cp mitmproxy-pretendo:/home/mitmproxy/.mitmproxy/mitmproxy-ca-cert.pem ./data/ca.pem
else
git restore ./data/ca.pem
info "Reset Inkay CA certificate."
print_info "Reset Inkay CA certificate."
fi
rm ./*.elf ./*.wps || true
rm -f ./*.elf ./*.wps
# Set up the Inkay build environment and then build the patches
docker build . -t inkay-build
@@ -44,7 +41,9 @@ docker run -it --rm -v .:/app -w /app inkay-build
success "Inkay patches built successfully at $(pwd)/Inkay-pretendo.wps"
# Upload the new Inkay patches to the Wii U
if [ -n "${WIIU_IP+x}" ]; then
ftp -u "ftp://user:pass@$WIIU_IP/fs/vol/external01/wiiu/environments/aroma/plugins/Inkay-pretendo.wps" ./Inkay-pretendo.wps
info "Reboot your Wii U now to apply the new patches."
if [[ -n "${WIIU_IP:-}" ]]; then
tnftp -u "ftp://user:pass@$WIIU_IP/fs/vol/external01/wiiu/environments/aroma/plugins/Inkay-pretendo.wps" ./Inkay-pretendo.wps
print_info "Reboot your Wii U now to apply the new patches."
else
print_warning "The modified patches were not uploaded to your Wii U because you did not set an IP address."
fi

View File

@@ -1,30 +1,28 @@
#! /bin/sh
#!/usr/bin/env bash
set -eu
# shellcheck source=./internal/framework.sh
source "$(dirname "$(realpath "$0")")/internal/framework.sh"
set_description "This creates a community in Juxtaposition, which is required to post anything. It should be run once \
for each community you want to create."
add_positional_argument "name" "name" "The new community's name" true
add_positional_argument "description" "description" "The new community's description" false
add_option_with_value "t --title-ids" "title_ids" "comma-separated-title-IDs" "Comma-separated list of title IDs to include in the community, like \"1,2,3\"" false
add_option_with_value "-i --icon-path" "icon_path" "image-path" "Path to an icon image for the community" false
add_option_with_value "-b --banner-path" "banner_path" "image-path" "Path to a banner image for the community" false
parse_arguments "$@"
git_base=$(git rev-parse --show-toplevel)
. "$git_base/scripts/internal/function-lib.sh"
create_community_script=$(cat "$git_base/scripts/run-in-container/create-juxt-community.js")
if [ "$#" -lt 3 ]; then
echo "Usage: $0 <name> <description> <comma-separated title IDs> [icon image path] [banner image path]"
exit 1
fi
create_community_script=$(cat "$git_base_dir/scripts/run-in-container/create-juxt-community.js")
# Clean up title IDs by removing non-alphanumeric characters, but keep commas
title_ids=$(echo "$3" | tr -dc "a-zA-Z0-9,")
title_ids=$(echo "$title_ids" | tr -dc "a-zA-Z0-9,")
docker compose up -d juxtaposition-ui
icon_path=
if [ "$#" -ge 4 ]; then
docker compose cp "$4" juxtaposition-ui:/tmp/icon
icon_path="/tmp/icon"
if [[ -n "$icon_path" ]]; then
docker compose cp "$icon_path" juxtaposition-ui:/tmp/icon
fi
banner_path=
if [ "$#" -eq 5 ]; then
docker compose cp "$5" juxtaposition-ui:/tmp/banner
banner_path="/tmp/banner"
if [[ -n "$banner_path" ]]; then
docker compose cp "$banner_path" juxtaposition-ui:/tmp/banner
fi
docker compose exec juxtaposition-ui node -e "$create_community_script" "$1" "$2" "$title_ids" "$icon_path" "$banner_path"
docker compose exec juxtaposition-ui node -e "$create_community_script" "$name" "$description" "$title_ids" "${icon_path:+/tmp/icon}" "${banner_path:+/tmp/banner}"

View File

@@ -1,63 +1,61 @@
#! /bin/sh
#!/usr/bin/env bash
set -eu
write_keys=false
if [ "${1-}" = "--write" ]; then
write_keys=true
fi
# shellcheck source=./internal/framework.sh
source "$(dirname "$(realpath "$0")")/internal/framework.sh"
set_description "This validates the BOSS keys from the dumped key files in console-files by comparing them to expected \
MD5 hashes."
add_option "-w --write" "write_keys" "Write the encryption keys to the BOSS environment file after validating"
parse_arguments "$@"
expected_ds_aes_key_hash="86fbc2bb4cb703b2a4c6cc9961319926"
expected_wiiu_aes_key_hash="5202ce5099232c3d365e28379790a919"
expected_wiiu_hmac_key_hash="b4482fef177b0100090ce0dbeb8ce977"
git_base=$(git rev-parse --show-toplevel)
. "$git_base/scripts/internal/function-lib.sh"
cd "$git_base/console-files"
cd "$git_base_dir/console-files"
if ls ./boss_keys.bin 1>/dev/null 2>&1; then
if [[ -f "./boss_keys.bin" ]]; then
wiiu_aes_key=$(head -c 16 <./boss_keys.bin)
wiiu_hmac_key=$(tail -c +33 ./boss_keys.bin | head -c 64)
wiiu_aes_key_hash=$(printf "%s" "$wiiu_aes_key" | md5sum | cut -d ' ' -f 1)
wiiu_hmac_key_hash=$(printf "%s" "$wiiu_hmac_key" | md5sum | cut -d ' ' -f 1)
wiiu_aes_key_hash=$(echo -n "$wiiu_aes_key" | md5sum | cut -d ' ' -f 1)
wiiu_hmac_key_hash=$(echo -n "$wiiu_hmac_key" | md5sum | cut -d ' ' -f 1)
if [ "$wiiu_aes_key_hash" = "$expected_wiiu_aes_key_hash" ]; then
success "Found valid Wii U BOSS AES key."
if [ "$write_keys" = true ]; then
echo "PN_BOSS_CONFIG_BOSS_WIIU_AES_KEY=$wiiu_aes_key" >>"$git_base/environment/boss.local.env"
if [[ "$wiiu_aes_key_hash" = "$expected_wiiu_aes_key_hash" ]]; then
print_success "Found valid Wii U BOSS AES key."
if [[ -n "$write_keys" ]]; then
echo "PN_BOSS_CONFIG_BOSS_WIIU_AES_KEY=$wiiu_aes_key" >>"$git_base_dir/environment/boss.local.env"
fi
else
error "Wii U BOSS AES key has the wrong hash! Try re-running full_keys_dumper."
print_error "Wii U BOSS AES key has the wrong hash! Try re-running full_keys_dumper."
exit 1
fi
if [ "$wiiu_hmac_key_hash" = "$expected_wiiu_hmac_key_hash" ]; then
success "Found valid Wii U HMAC key."
if [ "$write_keys" = true ]; then
echo "PN_BOSS_CONFIG_BOSS_WIIU_HMAC_KEY=$wiiu_hmac_key" >>"$git_base/environment/boss.local.env"
if [[ "$wiiu_hmac_key_hash" = "$expected_wiiu_hmac_key_hash" ]]; then
print_success "Found valid Wii U HMAC key."
if [[ -n "$write_keys" ]]; then
echo "PN_BOSS_CONFIG_BOSS_WIIU_HMAC_KEY=$wiiu_hmac_key" >>"$git_base_dir/environment/boss.local.env"
fi
else
error "Wii U BOSS HMAC key has the wrong hash! Try re-running full_keys_dumper."
print_error "Wii U BOSS HMAC key has the wrong hash! Try re-running full_keys_dumper."
exit 1
fi
else
warning "console-files/boss_keys.bin (Wii U) not found! Please read the README for instructions on dumping your BOSS keys."
print_warning "console-files/boss_keys.bin (Wii U) not found! Please read the README for instructions on dumping your BOSS keys."
fi
if ls ./aes_keys.txt 1>/dev/null 2>&1; then
if [[ -f "./aes_keys.txt" ]]; then
ds_aes_key=$(grep -oP 'slot0x38KeyN=\K.*' ./aes_keys.txt)
ds_aes_key_bin=$(printf "%s" "$ds_aes_key" | xxd -r -p)
ds_aes_key_hash=$(printf "%s" "$ds_aes_key_bin" | md5sum | cut -d ' ' -f 1)
ds_aes_key_bin=$(echo -n "$ds_aes_key" | xxd -r -p)
ds_aes_key_hash=$(echo -n "$ds_aes_key_bin" | md5sum | cut -d ' ' -f 1)
if [ "$ds_aes_key_hash" = "$expected_ds_aes_key_hash" ]; then
success "Found valid 3DS BOSS AES key."
if [ "$write_keys" = true ]; then
echo "PN_BOSS_CONFIG_BOSS_3DS_AES_KEY=$ds_aes_key" >>"$git_base/environment/boss.local.env"
if [[ "$ds_aes_key_hash" = "$expected_ds_aes_key_hash" ]]; then
print_success "Found valid 3DS BOSS AES key."
if [[ -n "$write_keys" ]]; then
echo "PN_BOSS_CONFIG_BOSS_3DS_AES_KEY=$ds_aes_key" >>"$git_base_dir/environment/boss.local.env"
fi
else
error "3DS BOSS AES key has the wrong hash! Try re-running DumpKeys.gm9."
print_error "3DS BOSS AES key has the wrong hash! Try re-running DumpKeys.gm9."
exit 1
fi
else
warning "console-files/aes_keys.txt (3DS) not found! Please read the README for instructions on dumping your BOSS keys."
print_warning "console-files/aes_keys.txt (3DS) not found! Please read the README for instructions on dumping your BOSS keys."
fi

View File

@@ -1,10 +1,10 @@
#! /bin/sh
#!/usr/bin/env bash
set -eu
# shellcheck source=./framework.sh
source "$(dirname "$(realpath "$0")")/framework.sh"
parse_arguments "$@"
git_base=$(git rev-parse --show-toplevel)
. "$git_base/scripts/internal/function-lib.sh"
minio_init_script=$(cat "$git_base/scripts/run-in-container/minio-init.sh")
minio_init_script=$(cat "$git_base_dir/scripts/run-in-container/minio-init.sh")
docker compose up -d minio

View File

@@ -1,10 +1,10 @@
#! /bin/sh
#!/usr/bin/env bash
set -eu
# shellcheck source=./framework.sh
source "$(dirname "$(realpath "$0")")/framework.sh"
parse_arguments "$@"
git_base=$(git rev-parse --show-toplevel)
. "$git_base/scripts/internal/function-lib.sh"
mongodb_init_script=$(cat "$git_base/scripts/run-in-container/mongodb-init.js")
mongodb_init_script=$(cat "$git_base_dir/scripts/run-in-container/mongodb-init.js")
docker compose up -d mongodb
@@ -14,7 +14,7 @@ docker compose up -d mongodb
# https://github.com/docker-library/mongo/issues/339
while ! docker compose exec mongodb mongosh --eval "db.adminCommand('ping')" >/dev/null 2>&1; do
info "Waiting for mongodb to be ready..."
print_info "Waiting for mongodb to be ready..."
sleep 2
done

View File

@@ -0,0 +1,314 @@
#!/usr/bin/env bash
# Usage:
# # shellcheck source=./internal/framework.sh
# source "$(dirname "$(realpath "$0")")/internal/framework.sh"
# # Add options and arguments here
# parse_arguments "$@"
# Update the script path to be the correct relative path to framework.sh
# Basic setup
set -euo pipefail
if [[ "${BASH_SOURCE[0]}" = "${0}" ]]; then
echo "The framework script needs to be sourced, not executed."
exit 1
fi
# Provide the Git base directory
framework_dir="$(dirname "$(realpath "${BASH_SOURCE[0]}")")"
git_base_dir="$(cd "$framework_dir" && git rev-parse --show-toplevel)"
# Terminal styling codes
term_reset=$(tput sgr0)
term_bold=$(tput bold)
term_underline=$(tput smul)
term_nounderline=$(tput rmul)
term_reverse=$(tput rev)
term_standout=$(tput smso)
term_nostandout=$(tput rmso)
term_dim=$(tput dim)
term_red=$(tput setaf 1)
term_green=$(tput setaf 2)
term_yellow=$(tput setaf 3)
term_blue=$(tput setaf 4)
term_magenta=$(tput setaf 5)
term_cyan=$(tput setaf 6)
term_white=$(tput setaf 7)
term_bgred=$(tput setab 1)
term_bggreen=$(tput setab 2)
term_bgyellow=$(tput setab 3)
term_bgblue=$(tput setab 4)
term_bgmagenta=$(tput setab 5)
term_bgcyan=$(tput setab 6)
term_bgwhite=$(tput setab 7)
# Set up the stage counter
stage_count=1
# Useful output functions
print_title() {
echo "${term_bold}${term_white}${term_bgmagenta}==================== ${*} ====================${term_reset}"
# Set the terminal emulator title
printf "\033]0;%s\a" "${*}"
}
print_header() {
echo "${term_bold}${term_cyan}---------- ${*} ----------${term_reset}"
}
print_stage() {
print_header "Stage ${stage_count}: ${*}"
stage_count=$((stage_count + 1))
}
print_error() {
echo "${term_bold}${term_red}Error: ${*}${term_reset}" >&2
}
print_warning() {
echo "${term_bold}${term_yellow}Warning: ${*}${term_reset}" >&2
}
print_info() {
echo "${term_cyan}${*}${term_reset}"
}
print_success() {
echo "${term_bold}${term_green}${*}${term_reset}"
}
# Argument parsing framework
# Uses a lot of Bash parameter expansion tricks: https://www.gnu.org/software/bash/manual/bash.html#Shell-Parameter-Expansion
argument_variables=()
required_argument_variable_indices=()
positional_argument_variables=()
option_cases=""
help_text_description=""
help_text_usage_arguments=()
help_text_arguments=()
help_text_argument_descriptions=()
# Sets the description of the overall script in the help text.
#
# Usage: set_description description
# Example: set_description "This script sets up the configuration."
set_description() {
help_text_description="$*"
}
# Adds a boolean option to the script. The variable is set to true if the option is provided or an empty string if it
# is not. The description is shown in the help text.
#
# Usage: add_option options variable_name description
# Example: add_option "-o --option" "option_enabled" "Enables the option"
add_option() {
local options="${1:?${FUNCNAME[0]}: Option arguments are required}"
local variable="${2:?${FUNCNAME[0]}: Option variable is required}"
local description="${3:?${FUNCNAME[0]}: Option description is required}"
argument_variables+=("$variable")
option_cases+="
${options// /|})
$variable=true
shift
;;"
help_text_usage_arguments+=("[${options// / | }]")
help_text_arguments+=("${options// /, }")
help_text_argument_descriptions+=("$description")
}
# Adds an option that allows specifying a value to the script. The variable is set to the value provided or an empty
# string if the option is not provided. The value name and description are shown in the help text. If the option is
# required, the script will exit with an error if the option is not provided. If a default value is set, the variable
# will be set to the default value if the option is provided without a value.
#
# Usage: add_option_with_value options variable_name value_name description required default_value
# Example: add_option_with_value "-o --option" "option_value" "option-value" "Sets the option value" false "default value"
add_option_with_value() {
local options="${1:?${FUNCNAME[0]}: Option arguments are required}"
local variable="${2:?${FUNCNAME[0]}: Option variable is required}"
local value_name="${3:?${FUNCNAME[0]}: Option value name is required}"
local description="${4:?${FUNCNAME[0]}: Option description is required}"
local required="${5:?${FUNCNAME[0]}: Option requirement boolean is required}"
local default_value="${6:-}"
argument_variables+=("$variable")
if [[ -n "$default_value" ]]; then
# If the argument after this is another option, it should not be set as this option's value and the
# arguments should only be shifted once. If this is the last argument, shift errors can be safely ignored.
option_cases+="
${options// /|})
shift
$variable=\"\${1:-}\"
if [[ \"\$$variable\" == -* ]]; then
$variable=
else
shift || true
fi
if [[ -z \"\$$variable\" ]]; then
$variable=\"$default_value\"
fi
;;"
else
option_cases+="
${options// /|})
shift
$variable=\"\${1:-}\"
if [[ \"\$$variable\" == -* ]]; then
$variable=
else
shift || true
fi
if [[ -z \"\$$variable\" ]]; then
print_error \"Option ${options// /, } requires a value for <$value_name>\"
options_error=true
fi
;;"
fi
if [[ "$required" = true ]]; then
required_argument_variable_indices+=($((${#argument_variables[@]} - 1)))
help_text_usage_arguments+=("${options// / | } <$value_name>")
help_text_arguments+=("${options// /, } <$value_name>")
help_text_argument_descriptions+=("$description (required)")
else
help_text_usage_arguments+=("[${options// / | } <$value_name>]")
help_text_arguments+=("${options// /, } <$value_name>")
help_text_argument_descriptions+=("$description (optional${default_value:+, default: $default_value})")
fi
}
# Adds a positional argument to the script. The variable is set to the value provided. If the argument is required, the
# script will exit with an error if the argument is not provided.
#
# Usage: add_positional_argument name variable_name description required
# Example: add_positional_argument "file-name" "file_name" "The file to process" true
add_positional_argument() {
local name="${1:?${FUNCNAME[0]}: Argument name is required}"
local variable="${2:?${FUNCNAME[0]}: Argument variable is required}"
local description="${3:?${FUNCNAME[0]}: Argument description is required}"
local required="${4:?${FUNCNAME[0]}: Argument requirement boolean is required}"
argument_variables+=("$variable")
positional_argument_variables+=("$variable")
if [[ "$required" = true ]]; then
required_argument_variable_indices+=($((${#argument_variables[@]} - 1)))
help_text_usage_arguments+=("<$name>")
help_text_arguments+=("<$name>")
help_text_argument_descriptions+=("$description (required)")
else
help_text_usage_arguments+=("[<$name>]")
help_text_arguments+=("<$name>")
help_text_argument_descriptions+=("$description (optional)")
fi
}
# Parses the provided arguments and sets the variables based on the configured options and positional arguments. Check
# if options are enabled or arguments are provided by using `if [[ -n "$option_variable" ]]`.
#
# Usage: parse_arguments "$@"
parse_arguments() {
local positional_arguments=()
local variable
for variable in "${argument_variables[@]}"; do
# Variables should start out global and empty strings
declare -g "$variable="
done
# Split combined short options into individual options
local split_args=()
local arg
for arg in "$@"; do
# Regex matches a single dash followed by any characters are not other dashes
if [[ "$arg" =~ ^-[^-]+$ ]]; then
for ((i = 1; i < ${#arg}; i++)); do
split_args+=("-${arg:$i:1}")
done
else
split_args+=("$arg")
fi
done
# Replace the original arguments with the split ones
set -- "${split_args[@]}"
while [[ $# -gt 0 ]]; do
# It is very important to use \$1 instead of $1 here to prevent command injection
eval "case \"\$1\" in
$option_cases
-*)
print_error \"Unknown option: \$1\"
options_error=true
shift
;;
*)
positional_arguments+=(\"\$1\")
shift
;;
esac"
done
# Show help before validating arguments
if [[ -n "$show_help" ]]; then
show_help
exit 0
fi
# Validate the provided arguments
if [[ -n "${options_error:-}" ]]; then
# Errors were already displayed in the case statement
exit 1
fi
# Validate the number of positional arguments before setting variables to avoid going out of range
if [[ "${#positional_arguments[@]}" -gt "${#positional_argument_variables[@]}" ]]; then
print_error "Too many positional arguments provided"
exit 1
fi
local i
for i in "${!positional_arguments[@]}"; do
declare -g "${positional_argument_variables[i]}=${positional_arguments[$i]}"
done
for i in "${required_argument_variable_indices[@]}"; do
if [[ -z "${!argument_variables[i]}" ]]; then
print_error "Required argument not provided: ${help_text_arguments[i]}"
exit 1
fi
done
}
# Shows the auto-generated help text based on the configured options and positional arguments.
show_help() {
echo "Usage: $0 ${help_text_usage_arguments[*]}"
if [[ -n "$help_text_description" ]]; then
echo "Description: $help_text_description"
fi
if [[ "${#help_text_arguments[@]}" -ne 0 ]]; then
local max_argument_length=0
local argument
for argument in "${help_text_arguments[@]}"; do
if [[ "${#argument}" -gt "$max_argument_length" ]]; then
max_argument_length="${#argument}"
fi
done
echo
echo "Arguments:"
local i
for i in "${!help_text_arguments[@]}"; do
printf " %-${max_argument_length}s %s\n" "${help_text_arguments[$i]}" "${help_text_argument_descriptions[$i]}"
done
fi
}
add_option "-h --help" "show_help" "Displays this help message"

View File

@@ -1,62 +0,0 @@
#! /bin/sh
# Terminal styling codes
reset=$(tput sgr0)
bold=$(tput bold)
underline=$(tput smul)
nounderline=$(tput rmul)
reverse=$(tput rev)
standout=$(tput smso)
nostandout=$(tput rmso)
dim=$(tput dim)
red=$(tput setaf 1)
green=$(tput setaf 2)
yellow=$(tput setaf 3)
blue=$(tput setaf 4)
magenta=$(tput setaf 5)
cyan=$(tput setaf 6)
white=$(tput setaf 7)
bgred=$(tput setab 1)
bggreen=$(tput setab 2)
bgyellow=$(tput setab 3)
bgblue=$(tput setab 4)
bgmagenta=$(tput setab 5)
bgcyan=$(tput setab 6)
bgwhite=$(tput setab 7)
# Set up the stage counter
stage=1
# Useful functions
title() {
echo "${bold}${white}${bgmagenta}==================== ${*} ====================${reset}"
# Set the terminal emulator title
printf "\033]0;%s\a" "${*}"
}
header() {
echo "${bold}${cyan}---------- ${*} ----------${reset}"
}
stage() {
header "Stage ${stage}: ${*}"
stage=$((stage + 1))
}
error() {
echo "${bold}${red}Error: ${*}${reset}" >&2
}
warning() {
echo "${bold}${yellow}Warning: ${*}${reset}" >&2
}
info() {
echo "${cyan}${*}${reset}"
}
success() {
echo "${bold}${green}${*}${reset}"
}

View File

@@ -1,25 +1,25 @@
#! /bin/sh
#!/usr/bin/env bash
set -eu
# shellcheck source=./framework.sh
source "$(dirname "$(realpath "$0")")/framework.sh"
parse_arguments "$@"
git_base=$(git rev-parse --show-toplevel)
. "$git_base/scripts/internal/function-lib.sh"
create_server_script=$(cat "$git_base/scripts/run-in-container/update-account-servers-database.js")
create_server_script=$(cat "$git_base_dir/scripts/run-in-container/update-account-servers-database.js")
if [ ! -f "$git_base/environment/system.local.env" ]; then
error "Missing environment file system.local.env. Did you run setup-environment.sh?"
if [[ ! -f "$git_base_dir/environment/server.local.env" ]]; then
print_error "Missing environment file server.local.env. Did you run setup-environment.sh?"
exit 1
fi
. "$git_base/environment/system.local.env"
source "$git_base_dir/environment/server.local.env"
necessary_environment_files="friends miiverse-api wiiu-chat super-mario-maker"
for environment in $necessary_environment_files; do
if [ ! -f "$git_base/environment/$environment.local.env" ]; then
error "Missing environment file $environment.local.env. Did you run setup-environment.sh?"
necessary_environment_files=("friends" "miiverse-api" "wiiu-chat" "super-mario-maker")
for environment in "${necessary_environment_files[@]}"; do
if [[ ! -f "$git_base_dir/environment/$environment.local.env" ]]; then
print_error "Missing environment file $environment.local.env. Did you run setup-environment.sh?"
exit 1
fi
. "$git_base/environment/$environment.env"
. "$git_base/environment/$environment.local.env"
source "$git_base_dir/environment/$environment.env"
source "$git_base_dir/environment/$environment.local.env"
done
docker compose up -d account

View File

@@ -1,10 +1,10 @@
#! /bin/sh
#!/usr/bin/env bash
set -eu
# shellcheck source=./framework.sh
source "$(dirname "$(realpath "$0")")/framework.sh"
parse_arguments "$@"
git_base=$(git rev-parse --show-toplevel)
. "$git_base/scripts/internal/function-lib.sh"
create_endpoint_script=$(cat "$git_base/scripts/run-in-container/update-miiverse-endpoints.js")
create_endpoint_script=$(cat "$git_base_dir/scripts/run-in-container/update-miiverse-endpoints.js")
docker compose up -d miiverse-api

View File

@@ -1,26 +1,26 @@
#! /bin/sh
#!/usr/bin/env bash
set -eu
# shellcheck source=./framework.sh
source "$(dirname "$(realpath "$0")")/framework.sh"
parse_arguments "$@"
git_base=$(git rev-parse --show-toplevel)
. "$git_base/scripts/internal/function-lib.sh"
if [ ! -f "$git_base/environment/postgres.local.env" ]; then
error "Missing environment file postgres.local.env. Did you run setup-environment.sh?"
if [[ ! -f "$git_base_dir/environment/postgres.local.env" ]]; then
print_error "Missing environment file postgres.local.env. Did you run setup-environment.sh?"
exit 1
fi
. "$git_base/environment/postgres.env"
. "$git_base/environment/postgres.local.env"
source "$git_base_dir/environment/postgres.env"
source "$git_base_dir/environment/postgres.local.env"
docker compose up -d postgres
while ! docker compose exec postgres psql -v ON_ERROR_STOP=1 -U "$POSTGRES_USER" -c "\l" >/dev/null 2>&1; do
info "Waiting for PostgreSQL to start..."
print_info "Waiting for PostgreSQL to start..."
sleep 2
done
# During the first run, this sometimes fails because the entrypoint script
# restarts the server after running the initdb scripts
while ! docker compose exec postgres psql -U "$POSTGRES_USER" -c "ALTER USER $POSTGRES_USER PASSWORD '$POSTGRES_PASSWORD';"; do
warning "Failed to change Postgres password, retrying..."
print_warning "Failed to change Postgres password, retrying..."
sleep 2
done

View File

@@ -1,14 +1,13 @@
#! /bin/sh
#!/usr/bin/env bash
set -eu
# shellcheck source=./internal/framework.sh
source "$(dirname "$(realpath "$0")")/internal/framework.sh"
set_description "This sets the access level of a PNID to developer, which gives it administrative permissions across the \
servers. For example, this makes the PNID's posts \"verified\" on Juxt. It should be run after creating a new PNID that \
should have administrative permissions."
add_positional_argument "dev-pnid" "dev_pnid" "The PNID to give developer access" true
parse_arguments "$@"
git_base=$(git rev-parse --show-toplevel)
. "$git_base/scripts/internal/function-lib.sh"
update_pnid_access_level_script=$(cat "$git_base/scripts/run-in-container/make-pnid-dev.js")
update_pnid_access_level_script=$(cat "$git_base_dir/scripts/run-in-container/make-pnid-dev.js")
if [ "$#" -lt 1 ]; then
echo "Usage: $0 <PNID to give developer access>"
exit 1
fi
docker compose exec account node -e "$update_pnid_access_level_script" "$1"
docker compose exec account node -e "$update_pnid_access_level_script" "$dev_pnid"

View File

@@ -1,65 +1,63 @@
#! /bin/sh
#!/usr/bin/env bash
set -eu
git_base=$(git rev-parse --show-toplevel)
. "$git_base/scripts/internal/function-lib.sh"
# shellcheck source=./internal/framework.sh
source "$(dirname "$(realpath "$0")")/internal/framework.sh"
set_description "This restores MongoDB, PostgreSQL, MinIO, Redis, and mitmproxy data from a specific backup directory. \
Note that it also re-runs setup-environment.sh to ensure the environment is consistent."
add_positional_argument "backup-directory" "backup_dir" "The backup directory to restore from" true
add_option "-f --force" "force" "Skips the restore confirmation prompt"
parse_arguments "$@"
if [ -z "${1-}" ]; then
echo "Usage: $0 <backup_name>"
if [[ ! -d "$backup_dir" ]]; then
print_error "Backup directory $backup_dir does not exist."
exit 1
fi
print_info "Restoring from $backup_dir"
backup_name="$1"
backup_dir="$git_base/backups/$backup_name"
if [ ! -d "$backup_dir" ]; then
error "Backup directory $backup_dir does not exist."
exit 1
fi
info "Restoring from $backup_dir"
if [ "${2-}" != "--force" ]; then
warning "Restoring a backup will overwrite your current Pretendo server data. Backing up your data first is strongly recommended."
print_warning "Restoring a backup will overwrite your current Pretendo server data. Backing up your data first is strongly recommended."
if [[ -z "$force" ]]; then
printf "Continue? [y/N] "
read -r continue
if [ "$continue" != "Y" ] && [ "$continue" != "y" ]; then
if [[ "$continue" != "Y" && "$continue" != "y" ]]; then
echo "Aborting."
exit 1
fi
fi
info "Stopping unnecessary services..."
print_info "Stopping unnecessary services..."
docker compose down
docker compose up -d mitmproxy-pretendo mongodb postgres minio redis
info "Restoring MongoDB..."
print_info "Restoring MongoDB..."
docker compose exec mongodb rm -rf /tmp/backup
docker compose cp "$backup_dir/mongodb" mongodb:/tmp/backup
docker compose exec mongodb mongorestore /tmp/backup --drop --quiet
docker compose exec mongodb rm -rf /tmp/backup
info "Restoring Postgres..."
print_info "Restoring Postgres..."
# According to the pg_dumpall documentation, dropping and creating the superuser role is expected to cause an error
print_info "Note: the errors \"current user cannot be dropped\" and \"role ... already exists\" are expected and can be safely ignored."
docker compose exec -T postgres sh -c 'psql -U "$POSTGRES_USER" -d postgres' <"$backup_dir/postgres.sql" >/dev/null
info "Restoring MinIO..."
print_info "Restoring MinIO..."
docker compose exec minio /bin/sh -c 'mc alias set minio http://minio.pretendo.cc "$MINIO_ROOT_USER" "$MINIO_ROOT_PASSWORD"'
docker compose exec minio rm -rf /tmp/backup
docker compose cp "$backup_dir/minio" minio:/tmp/backup
docker compose exec minio mc mirror /tmp/backup minio/ --overwrite --remove
docker compose exec minio rm -rf /tmp/backup
info "Restoring Redis..."
print_info "Restoring Redis..."
# Redis cannot be running when restoring a dump or it will overwrite the restored dump when it exits
docker compose stop redis
docker compose cp "$backup_dir/redis.rdb" redis:/data/dump.rdb
docker compose start redis
info "Restoring Mitmproxy..."
print_info "Restoring Mitmproxy..."
docker compose cp "$backup_dir/mitmproxy" mitmproxy-pretendo:/home/mitmproxy/.mitmproxy
# The restored Postgres backup might be using a different password than what is currently in the .env files
info "Updating the Postgres password from the environment..."
"$git_base/scripts/internal/update-postgres-password.sh"
# The restored backup might be using different secrets than what are currently in the .env files
"$git_base_dir/scripts/setup-environment.sh" --force
success "Restore completed successfully."
print_success "Restore completed successfully."

View File

@@ -14,13 +14,6 @@ const s3 = new S3({
});
async function runAsync() {
if (process.argv.length < 4) {
console.log(
"Usage: <name> <description> <comma-separated title IDs> [icon image path] [banner image path]"
);
process.exit(1);
}
await connect();
await createMainCommunity(
@@ -39,14 +32,7 @@ runAsync().then(() => {
process.exit(0);
});
async function createMainCommunity(
platform_id,
name,
description,
title_ids,
iconPath,
bannerPath
) {
async function createMainCommunity(platform_id, name, description, title_ids, iconPath, bannerPath) {
const communityId = Math.floor(Math.random() * 1000000) + 1;
const newCommunity = new COMMUNITY({
@@ -95,6 +81,7 @@ async function uploadAssets(community_id, iconPath, bannerPath) {
};
await s3.putObject(uploadParams).promise();
}
await fs.rm(iconPath);
}
if (bannerPath) {
@@ -118,5 +105,6 @@ async function uploadAssets(community_id, iconPath, bannerPath) {
};
await s3.putObject(uploadParams).promise();
}
await fs.rm(bannerPath);
}
}

View File

@@ -1,4 +1,4 @@
#! /bin/sh
#!/bin/sh
set -eu

View File

@@ -1,4 +1,4 @@
#! /bin/sh
#!/bin/sh
set -eu

View File

@@ -1,6 +1,16 @@
#! /bin/sh
#!/usr/bin/env bash
set -eu
# shellcheck source=./internal/framework.sh
source "$(dirname "$(realpath "$0")")/internal/framework.sh"
set_description "This sets up local environment variables in the *.local.env files, including randomly-generated \
secrets. Important external secrets are written to secrets.txt in the root of the repository. By default, the script \
re-uses existing configuration values from server.local.env unless at least one is passed as an argument."
add_option_with_value "-s --server-ip" "server_ip" "ip-address" "The IP address of the Pretendo Network server (must be accessible to the clients)" false
add_option_with_value "-w --wiiu-ip" "wiiu_ip" "ip-address" "The IP address of the Wii U for FTP uploads" false
add_option_with_value "-3 --3ds-ip" "ds_ip" "ip-address" "The IP address of the 3DS for FTP uploads" false
add_option "-n --no-environment" "no_environment" "Disables reading existing configuration values from the environment"
add_option "-f --force" "force" "Skip the confirmation prompt and always overwrite existing local environment files"
parse_arguments "$@"
generate_password() {
length=$1
@@ -12,34 +22,34 @@ generate_hex() {
head /dev/urandom | tr -dc "A-F0-9" | head -c "$length"
}
# Validate arguments
if [ "$#" -lt 1 ]; then
echo "Usage: $0 <server IP address> [Wii U IP address] [3DS IP address]"
cd "$git_base_dir/environment"
print_info "Setting up local environment variables..."
if [[ -z "$no_environment" ]]; then
if [[ -f "./server.local.env" ]]; then
source ./server.local.env
fi
# Copy the configuration from server.local.env if necessary
: "${server_ip:=${SERVER_IP:-}}"
: "${wiiu_ip:=${WIIU_IP:-}}"
: "${ds_ip:=${DS_IP:-}}"
fi
if [[ -z "$server_ip" ]]; then
print_error "A server IP address was neither passed as an argument nor found in the environment. Please provide one."
exit 1
fi
server_ip=$1
wiiu_ip=
if [ "$#" -ge 2 ]; then
wiiu_ip=$2
fi
ds_ip=
if [ "$#" -ge 3 ]; then
ds_ip=$3
fi
git_base=$(git rev-parse --show-toplevel)
. "$git_base/scripts/internal/function-lib.sh"
cd "$git_base/environment"
info "Setting up local environment variables..."
if ls ./*.local.env 1>/dev/null 2>&1; then
warning "Local environment files already exist. They will be overwritten if you continue."
printf "Continue? [y/N] "
read -r continue
if [ "$continue" != "Y" ] && [ "$continue" != "y" ]; then
echo "Aborting."
exit 1
if [[ -n "$(echo ./*.local.env)" ]]; then
print_warning "Local environment files already exist. They will be overwritten if you continue."
if [[ -z "$force" ]]; then
printf "Continue? [y/N] "
read -r continue
if [[ "$continue" != "Y" && "$continue" != "y" ]]; then
echo "Aborting."
exit 1
fi
fi
docker compose down
@@ -112,33 +122,33 @@ boss_api_key=$(generate_password 32)
echo "PN_BOSS_CONFIG_GRPC_BOSS_SERVER_API_KEY=$boss_api_key" >>./boss.local.env
# Set up the server IP address
info "Using server IP address $server_ip."
echo "SERVER_IP=$server_ip" >>./system.local.env
print_info "Using server IP address $server_ip."
echo "SERVER_IP=$server_ip" >>./server.local.env
echo "PN_FRIENDS_SECURE_SERVER_HOST=$server_ip" >>./friends.local.env
echo "PN_WIIU_CHAT_SECURE_SERVER_LOCATION=$server_ip" >>./wiiu-chat.local.env
echo "PN_SMM_SECURE_SERVER_HOST=$server_ip" >>./super-mario-maker.local.env
# Get the Wii U IP address
if [ -n "$wiiu_ip" ]; then
info "Using Wii U IP address $wiiu_ip."
echo "WIIU_IP=$wiiu_ip" >>./system.local.env
if [[ -n "$wiiu_ip" ]]; then
print_info "Using Wii U IP address $wiiu_ip."
echo "WIIU_IP=$wiiu_ip" >>./server.local.env
else
info "Skipping Wii U IP address."
print_info "Skipping Wii U IP address."
fi
# Get the 3DS IP address
if [ -n "$ds_ip" ]; then
info "Using 3DS IP address $ds_ip."
echo "DS_IP=$ds_ip" >>./system.local.env
if [[ -n "$ds_ip" ]]; then
print_info "Using 3DS IP address $ds_ip."
echo "DS_IP=$ds_ip" >>./server.local.env
else
info "Skipping 3DS IP address."
print_info "Skipping 3DS IP address."
fi
# Get the BOSS keys
"$git_base"/scripts/get-boss-keys.sh --write
"$git_base_dir/scripts/get-boss-keys.sh" --write
# Create a list of important secrets
cat >"$git_base/secrets.txt" <<EOF
cat >"$git_base_dir/secrets.txt" <<EOF
Pretendo Network server secrets
===============================
@@ -151,15 +161,15 @@ Wii U IP address: ${wiiu_ip:-(not set)}
3DS IP address: ${ds_ip:-(not set)}
EOF
success "Successfully set up environment."
print_success "Successfully set up environment."
# Some things need to be updated with the new environment variables and secrets,
# but only if the setup script isn't in progress. The MongoDB container replica
# set won't be configured during initial setup, and the scripts will fail.
if [ -z "${PRETENDO_SETUP_IN_PROGRESS+x}" ]; then
info "Running necessary container update scripts..."
"$git_base"/scripts/internal/update-postgres-password.sh
"$git_base"/scripts/internal/update-account-servers-database.sh
if [[ -z "${PRETENDO_SETUP_IN_PROGRESS:-}" ]]; then
print_info "Running necessary container update scripts..."
"$git_base_dir/scripts/internal/update-postgres-password.sh"
"$git_base_dir/scripts/internal/update-account-servers-database.sh"
docker compose down
success "Successfully updated containers with new environment variables."
print_success "Successfully updated the containers with new environment variables."
fi

View File

@@ -1,30 +1,59 @@
#! /bin/sh
#!/usr/bin/env bash
set -eu
# shellcheck source=./internal/framework.sh
source "$(dirname "$(realpath "$0")")/internal/framework.sh"
set_description "This configures, resets the contents of, and applies patches to the submodules in the repos directory."
add_option "-u --update-remote" "update_remote" "Updates the submodules from their remotes before applying patches. Only \
use this if you're trying to update the submodules to a newer version than is supported by this project. Patches will \
be applied first with --reject, and if that fails, a 3-way merge will be attempted."
parse_arguments "$@"
git_base=$(git rev-parse --show-toplevel)
. "$git_base/scripts/internal/function-lib.sh"
cd "$git_base_dir"
info "Resetting all submodules..."
git submodule sync --recursive >/dev/null
git submodule foreach --recursive "git reset --hard" >/dev/null
git submodule foreach --recursive "git clean -fd" >/dev/null
git submodule update --init --recursive --checkout --force >/dev/null
print_info "Resetting all submodules..."
git submodule sync >/dev/null
git submodule foreach "git reset --hard" >/dev/null
git submodule foreach "git clean -fd" >/dev/null
git submodule update --init --checkout >/dev/null
if [[ -n "$update_remote" ]]; then
print_info "Updating submodules from their remotes..."
git submodule update --remote
fi
info "Applying patches to submodules..."
num_patches=0
for dir in "$git_base/patches/"*; do
if [ -d "$dir" ]; then
print_info "Applying patches to submodules..."
patch_count=0
error_count=0
for dir in "$git_base_dir/patches/"*; do
if [[ -d "$dir" ]]; then
subdir=$(basename "$dir")
cd "$git_base/repos/$subdir"
cd "$git_base_dir/repos/$subdir"
for patch in "$git_base/patches/$subdir"/*; do
git apply "$patch"
num_patches=$((num_patches + 1))
for patch in "$git_base_dir/patches/$subdir"/*; do
if [[ -n "$update_remote" ]]; then
print_info "Applying patch $patch..."
if ! git apply --reject "$patch"; then
print_error "Failed to apply patch $patch! Attempting 3-way merge..."
git add .
if ! git apply --3way "$patch"; then
print_error "There are merge conflicts with the patch that need to be resolved manually."
else
print_success "Successfully applied the patch with a 3-way merge. Make sure to re-generate the patch."
fi
error_count=$((error_count + 1))
fi
else
git apply "$patch"
fi
patch_count=$((patch_count + 1))
done
git add .
fi
done
success "Successfully applied $num_patches patches."
if [[ "$error_count" -gt 0 ]]; then
print_error "Failed to apply $error_count patches out of $patch_count."
exit 1
fi
print_success "Successfully applied $patch_count patches."

View File

@@ -1,48 +1,47 @@
#! /bin/sh
#!/usr/bin/env bash
set -eu
# shellcheck source=./internal/framework.sh
source "$(dirname "$(realpath "$0")")/internal/framework.sh"
set_description "This uploads required files from console-files to your 3DS to connect to your Pretendo server."
add_option "-r --reset" "should_reset" "Reset the Juxt certificate to Pretendo's instead of using the mitmproxy certificate"
parse_arguments "$@"
should_reset=false
if [ "${1-}" = "--reset" ]; then
should_reset=true
fi
git_base=$(git rev-parse --show-toplevel)
. "$git_base/scripts/internal/function-lib.sh"
cd "$git_base"
if [ ! -f "$git_base/environment/system.local.env" ]; then
error "Missing environment file system.local.env. Did you run setup-environment.sh?"
if [[ ! -f "$git_base_dir/environment/server.local.env" ]]; then
print_error "Missing environment file server.local.env. Did you run setup-environment.sh?"
exit 1
fi
. "$git_base/environment/system.local.env"
if [ -z "${DS_IP+x}" ]; then
warning "Missing environment variable DS_IP. Did you specify a 3DS IP address when you ran setup-environment.sh?"
info "Continuing without automatic FTP upload."
source "$git_base_dir/environment/server.local.env"
if [[ -z "${DS_IP:-}" ]]; then
print_warning "Missing environment variable DS_IP. Did you specify a 3DS IP address when you ran setup-environment.sh?"
print_info "Continuing without automatic FTP upload."
fi
if [ "$should_reset" = false ]; then
cd "$git_base_dir/console-files"
if [[ -z "$should_reset" ]]; then
docker compose up -d mitmproxy-pretendo
while ! docker compose exec mitmproxy-pretendo ls /home/mitmproxy/.mitmproxy/mitmproxy-ca-cert.pem >/dev/null 2>&1; do
info "Waiting for mitmproxy to generate a certificate..."
print_info "Waiting for mitmproxy to generate a certificate..."
sleep 1
done
# Get the current certificate
docker compose cp mitmproxy-pretendo:/home/mitmproxy/.mitmproxy/mitmproxy-ca-cert.pem ./console-files/mitmproxy-ca-cert.pem
docker compose cp mitmproxy-pretendo:/home/mitmproxy/.mitmproxy/mitmproxy-ca-cert.pem ./mitmproxy-ca-cert.pem
else
info "Reset the Juxt certificate."
print_info "Reset the Juxt certificate."
fi
# Upload the required files
if [ -n "${DS_IP+x}" ]; then
if [ "$should_reset" = false ]; then
ftp -u "ftp://user:pass@$DS_IP:5000/3ds/juxt-prod.pem" ./console-files/mitmproxy-ca-cert.pem
ftp -u "ftp://user:pass@$DS_IP:5000/gm9/scripts/FriendsAccountSwitcher.gm9" ./console-files/FriendsAccountSwitcher.gm9
ftp -u "ftp://user:pass@$DS_IP:5000/3ds/ResetFriendsTestAccount.3dsx" ./console-files/ResetFriendsTestAccount.3dsx
if [[ -n "${DS_IP:-}" ]]; then
if [[ -z "$should_reset" ]]; then
tnftp -u "ftp://user:pass@$DS_IP:5000/3ds/juxt-prod.pem" ./mitmproxy-ca-cert.pem
tnftp -u "ftp://user:pass@$DS_IP:5000/gm9/scripts/FriendsAccountSwitcher.gm9" ./FriendsAccountSwitcher.gm9
tnftp -u "ftp://user:pass@$DS_IP:5000/3ds/ResetFriendsTestAccount.3dsx" ./ResetFriendsTestAccount.3dsx
else
ftp -u "ftp://user:pass@$DS_IP:5000/3ds/juxt-prod.pem" ./console-files/juxt-prod.pem
tnftp -u "ftp://user:pass@$DS_IP:5000/3ds/juxt-prod.pem" ./juxt-prod.pem
fi
success "Successfully uploaded the required files to your 3DS."
print_success "Successfully uploaded the required files to your 3DS."
else
print_warning "The required files were not uploaded to your 3DS because you did not set an IP address."
fi

134
setup.sh
View File

@@ -1,131 +1,127 @@
#! /bin/sh
#!/usr/bin/env bash
set -eu
check_git_repository() {
# Temporary function because function-lib isn't sourced yet
error() {
echo "$(tput bold)$(tput setaf 1)Error: ${*}$(tput sgr0)" >&2
}
if ! git --version >/dev/null; then
error "Git is not installed. Please install it: https://git-scm.com/downloads/"
exit 1
fi
if ! git rev-parse --is-inside-work-tree >/dev/null 2>&1; then
error "This script must be run from inside the pretendo-docker Git repository."
echo "Make sure that you cloned the repository with Git, not downloaded it as a ZIP file from GitHub."
exit 1
fi
if [ "$(basename -s .git "$(git config --get remote.origin.url)")" != "pretendo-docker" ]; then
error "This script must be run from inside the pretendo-docker Git repository."
echo "It looks like you are running it from a different Git repository at $(pwd) from $(git config --get remote.origin.url)."
exit 1
fi
git config --local submodule.recurse true
}
set -euo pipefail
check_prerequisites() {
prerequisites_failed=false
prerequisites_warning=false
prerequisites_failed=
prerequisites_warning=
if ! docker version >/dev/null; then
error "Docker is not installed. Please install it: https://docs.docker.com/get-docker/"
info "If you see a \"Permission denied while trying to connect to the Docker daemon\" error, you need to run this script with sudo."
print_error "Docker is not installed. Please install it: https://docs.docker.com/get-docker/"
print_info "If you see a \"Permission denied while trying to connect to the Docker daemon\" error, you need to \
add your user to the docker group: https://docs.docker.com/engine/install/linux-postinstall/#manage-docker-as-a-non-root-user."
prerequisites_failed=true
fi
if ! docker compose version >/dev/null; then
error "Docker Compose is not installed. Please install it: https://docs.docker.com/compose/install/"
print_error "Docker Compose is not installed. Please install it: https://docs.docker.com/compose/install/"
prerequisites_failed=true
fi
# The tnftp "enhanced ftp client" has the -u option for direct uploads,
# unlike the netkit-ftp "classical ftp client"
if ! command -v tnftp >/dev/null; then
warning "tnftp is not installed. You will not be able to upload files to your consoles automatically."
print_warning "tnftp is not installed. You will not be able to upload files to your consoles automatically."
prerequisites_warning=true
fi
if [ "$prerequisites_failed" = true ]; then
error "Prerequisites check failed. Please install the missing prerequisites and try again."
if [[ "$prerequisites_failed" = true ]]; then
print_error "Prerequisites check failed. Please install the missing prerequisites and try again."
exit 1
elif [ "$prerequisites_warning" = true ]; then
warning "Prerequisites check completed with warnings."
elif [[ "$prerequisites_warning" = true ]]; then
print_warning "Prerequisites check completed with warnings."
if [ -z "${CI+x}" ]; then
# Optional prerequisites can be ignored in CI
if [[ -z "$force" ]]; then
printf "Do you want to continue anyway (y/N)? "
read -r continue_anyway
if [ "$continue_anyway" != "Y" ] && [ "$continue_anyway" != "y" ]; then
if [[ "$continue_anyway" != "Y" && "$continue_anyway" != "y" ]]; then
echo "Aborting."
exit 1
fi
fi
else
success "All prerequisites are installed."
print_success "All prerequisites are installed."
fi
}
setup_environment_variables() {
if [ -n "${CI+x}" ]; then
warning "It looks like the script is running in CI. Using test IP addresses."
server_ip="0.0.0.0"
wiiu_ip=""
ds_ip=""
else
if [[ -n "$reconfigure" || (-z "$server_ip" && ! -f "$git_base_dir/environment/server.local.env") ]]; then
echo "Enter the IP address of your Pretendo Network server. It must be accessible to your console."
read -r server_ip
echo "Enter the IP address of your Wii U (optional). It is only used for automatic FTP uploads of modified Inkay patches."
echo "Enter the IP address of your Wii U (optional). It is used for automatic FTP uploads."
read -r wiiu_ip
echo "Enter the IP address of your 3DS (optional). It is only used for automatic FTP uploads."
echo "Enter the IP address of your 3DS (optional). It is used for automatic FTP uploads."
read -r ds_ip
fi
./scripts/setup-environment.sh "$server_ip" "$wiiu_ip" "$ds_ip"
./scripts/setup-environment.sh "$server_ip" "$wiiu_ip" "$ds_ip" ${force:+--force} ${reconfigure:+--no-environment}
}
setup_containers() {
info "Setting up MongoDB container..."
print_info "Setting up MongoDB container..."
./scripts/internal/firstrun-mongodb-container.sh
info "Setting up MinIO container..."
print_info "Setting up MinIO container..."
./scripts/internal/firstrun-minio-container.sh
info "Setting up Pretendo account servers database..."
print_info "Setting up Pretendo account servers database..."
./scripts/internal/update-account-servers-database.sh
info "Setting up Pretendo Miiverse endpoints database..."
print_info "Setting up Pretendo Miiverse endpoints database..."
./scripts/internal/update-miiverse-endpoints.sh
info "Updating Postgres password..."
print_info "Updating Postgres password..."
./scripts/internal/update-postgres-password.sh
info "Stopping containers after initial setup..."
print_info "Stopping containers after initial setup..."
docker compose down
}
export PRETENDO_SETUP_IN_PROGRESS=true
check_git_repository
# Temporary function because the framework script isn't sourced yet and we don't know if tput is available
print_error() {
echo -e "\e[1;31mError: ${*}\e[0m" >&2
}
git_base=$(git rev-parse --show-toplevel)
cd "$git_base"
. "$git_base/scripts/internal/function-lib.sh"
# The framework script requires git and tput, so check for them first
if ! tput setaf 0 >/dev/null; then
print_error "Either the tput command is not installed, or your \$TERM environment variable is not set correctly. \
Please install your distribution's ncurses package (such as ncurses-bin) and/or configure your terminal to set \$TERM."
exit 1
fi
if ! git --version >/dev/null; then
print_error "Git is not installed. Please install it: https://git-scm.com/downloads/"
exit 1
fi
title "Unofficial Pretendo Network setup script"
header "Pretendo setup script started at $(date)."
git config --local submodule.recurse true
stage "Checking prerequisites."
# shellcheck source=./scripts/internal/framework.sh
source "$(dirname "$(realpath "$0")")/scripts/internal/framework.sh"
set_description "This is the main setup script for your self-hosted Pretendo Network server. By default, it will prompt \
for configuration values the first run and re-use those values for future runs."
add_option "-r --reconfigure" "reconfigure" "Always shows configuration prompts, even if the values are already set. \
Also disables reading configuration values from the environment."
add_option "-f --force" "force" "Ignores warnings and confirmation prompts during the setup process."
add_option_with_value "-s --server-ip" "server_ip" "IP-address" "The IP address of your Pretendo Network server. It \
must be accessible to your console. Disables interactive prompts by default, unless --force-interactive is specified." false
add_option_with_value "-w --wiiu-ip" "wiiu_ip" "IP-address" "The IP address of your Wii U. It is used for automatic FTP uploads." false
add_option_with_value "-3 --3ds-ip" "ds_ip" "IP-address" "The IP address of your 3DS. It is used for automatic FTP uploads." false
parse_arguments "$@"
print_title "Unofficial Pretendo Network server setup script started"
print_stage "Checking prerequisites."
check_prerequisites
stage "Setting up submodules and applying patches."
print_stage "Setting up submodules and applying patches."
./scripts/setup-submodule-patches.sh
stage "Setting up environment variables."
print_stage "Setting up environment variables."
setup_environment_variables
stage "Pulling Docker images."
print_stage "Pulling Docker images."
docker compose pull
stage "Building Docker images."
print_stage "Building Docker images."
docker compose build
stage "Setting up containers with first-run scripts."
print_stage "Setting up containers with first-run scripts."
setup_containers
success "Setup completed! You can now start your Pretendo Network server with \"docker compose up -d\"."
header "Pretendo setup script finished at $(date)."
print_title "Pretendo Network server setup script finished"
print_success "Setup completed! You can now start your Pretendo server with \"docker compose up -d\"."