feat: Webdav support for remote storage

- Created common interface that GDrive and Webdav implement (rfs::IRemoteFS)
- Moved shared functionality into shared interface/implementation
- drive.h/.cpp was replaced by remote.h/.cpp
- fld.cpp now gets a copy of RfsItem (former gdIems), because the implementation is not required to retain these (e.g. Webdav does not)
- UI presentation changed from [GD] to [R] (Remote)
This commit is contained in:
Martin Riedel 2023-07-07 23:52:34 -04:00
parent a893d3fa56
commit 938c52b603
35 changed files with 805 additions and 278 deletions

View File

@ -12,12 +12,13 @@ set(SOURCE_FILES
src/gd.cpp
src/gfx.cpp
src/main.cpp
src/rfs.cpp
src/type.cpp
src/ui.cpp
src/util.cpp
src/webdav.cpp
src/fs/dir.cpp
src/fs/drive.cpp
src/fs/remote.cpp
src/fs/file.cpp
src/fs/fsfile.c
src/fs/zip.cpp

View File

@ -57,7 +57,7 @@ CXXFLAGS:= $(CFLAGS) -fno-rtti -fno-exceptions -std=gnu++17
ASFLAGS := -g $(ARCH)
LDFLAGS = -specs=$(DEVKITPRO)/libnx/switch.specs -g $(ARCH) -Wl,-Map,$(notdir $*.map)
LIBS := `sdl2-config --libs` `freetype-config --libs` `curl-config --libs` -lSDL2_image -lwebp -lpng -ljpeg -lz -lminizip -ljson-c -lnx
LIBS := `sdl2-config --libs` `freetype-config --libs` `curl-config --libs` -lSDL2_image -lwebp -lpng -ljpeg -lz -lminizip -ljson-c -ltinyxml2 -lnx
#---------------------------------------------------------------------------------
# list of directories containing libraries, this must be the top level containing

View File

@ -12,26 +12,27 @@ This started as a simple, straight port of my 3DS save manager I publicly releas
4. Dump and restore cache Saves.
5. Dump system save data.
* Dumping this data is always enabled, but writing back needs to be enabled from the options menu. Writing to this can be very dangerous.
* Processes can be terminated from the Extras menu allowing you to open even more of these and explore more.
* Processes can be terminated from the Extras menu allowing you to open even more of these and explore more.
6. Export save data to folders like the orignal or compressed zip files to save space.
7. Upload and download save backups to [Google Drive](https://github.com/J-D-K/JKSV/blob/master/GD_INSTRUCTIONS.MD) if it is configured.
8. Create save data so the user no longer needs to boot games to import saves.
* Titles can be rescanned from the Extras menu. For example, if you insert a game card while JKSV is open, rescanning will load and add it to the save creation menu(s).
9. Export and use SVI files to create save data for titles not installed on your system. For games that detect other game saves to unlock content.
* SVI files are simply the title ID, NACP, and icon packed into a file. Placing them in `JKSV/svi` will load them as if they are any other game on your switch. They will appear in the save creation menus with the rest.
10. Extend save data containers to any size the user wants or automatically if the save cannot fit into the current one.
11. Delete save data from the system.
12. Reset save data as if the game was never launched.
13. Display stats and information about the game/save: Play time, launch count, title ID (TID), save ID(SID)/name of save file on user nand partition, etc.
14. Open and explore bis storage partitions via the Extras menu
7. Upload and download save backups to [Google Drive](./REMOTE_INSTRUCTIONS.MD#gdrive) if it is configured.
8. Upload and download save backups to [WebDav](./REMOTE_INSTRUCTIONS.MD#webdav) if it is configured.
9. Create save data so the user no longer needs to boot games to import saves.
* Titles can be rescanned from the Extras menu. For example, if you insert a game card while JKSV is open, rescanning will load and add it to the save creation menu(s).
10. Export and use SVI files to create save data for titles not installed on your system. For games that detect other game saves to unlock content.
* SVI files are simply the title ID, NACP, and icon packed into a file. Placing them in `JKSV/svi` will load them as if they are any other game on your switch. They will appear in the save creation menus with the rest.
11. Extend save data containers to any size the user wants or automatically if the save cannot fit into the current one.
12. Delete save data from the system.
13. Reset save data as if the game was never launched.
14. Display stats and information about the game/save: Play time, launch count, title ID (TID), save ID(SID)/name of save file on user nand partition, etc.
15. Open and explore bis storage partitions via the Extras menu
* BIS Storage is opened inside a basic filebrowser. The partition's listing is on the left. Your SD is on the right.
* Only copying to SD and file properties work on BIS partitions. Writing to and deleting are disabled unless enabled like system save data.
15. Misc Extras:
16. Misc Extras:
* Ability to remove downloaded firmware updates from NAND. This is located under Extras.
* Terminating processes by [ID](https://switchbrew.org/wiki/Title_list#System_Modules). Allowing you to dump normally unopenable system archives.
* Mount by System Save [ID](https://switchbrew.org/wiki/Flash_Filesystem#System_Savegames). Normally used when the terminated process makes JKSV unable to rescan titles without the Switch crashing.
* Mount and open RomFS of process the homebrew menu takes over (if launched as NRO).
* Hold R while opening a game or applet with Atmosphere so the homebrew menu loads. Open JKSV and press minus and select **Mount Process RomFS**. The romfs of the app should appear in the browser along with your SD on the right.
* Mount and open RomFS of process the homebrew menu takes over (if launched as NRO).
* Hold R while opening a game or applet with Atmosphere so the homebrew menu loads. Open JKSV and press minus and select **Mount Process RomFS**. The romfs of the app should appear in the browser along with your SD on the right.
**NOTE: Some features may require building JKSV from source. I am extremely picky and only release when I am satisfied and sure things work 100% as expected.**

View File

@ -1,4 +1,5 @@
# <center> How to use Google Drive with JKSV </center>
# Remote Storage
## <a name="gdrive"></a><center> How to use Google Drive with JKSV </center>
**USING GOOGLE DRIVE WITH JKSV CURRENTLY REQUIRES BUILDING IT YOURSELF. I AM VERY BUSY LATELY AND THINGS WILL ONLY GET FINISHED WHEN I HAVE TIME. Thanks, sorry for yelling.**
**NOTE: As of Feb 2023, JKSV now uses the JSON downloaded from Google directly instead of editing JKSV's configuration file.**
@ -19,7 +20,7 @@
9. Click on it and click Enable on the next screen.
10. On the next screen, Google should be informing you that you need to create credentials in order to use Drive. Click Create Credentials.<br><center><img src="https://i.imgur.com/CRhFXQ4.png" /></center>
11. Under **Which API are you using?**, find **Cloud Storage API**. Under **What data will you be accessing?**, select **User data**. Click **Next**. <br><center><img src="https://i.imgur.com/fiulRpn.png" /></center>
12. Fill out the following the following screen. **Save and continue**.
12. Fill out the following screen. **Save and continue**.
13. Click **Add or Remove Scopes**.
14. Find **.../auth/drive** in the API list, select it, and click update. **Save and Continue**.
15. At this point you should be at a section named **OAuth Client ID**. Select **Desktop app**, name it **JKSV** and click **Create**.
@ -27,4 +28,24 @@
17. Next, open the navigation menu in the top left again. Go down to **APIs and Services** and click on **OAuth consent screen**.<br><center><img src="https://i.imgur.com/OrMtG1x.png" /></center>
18. Scroll down to the section named **Test users**. Add yourself as a test user. This concludes the Google side of things.<br><center><img src="https://i.imgur.com/RTV2LMZ.png" /></center>
19. Next, find the JSON file you downloaded earlier. Copy it or send it via FTP to the following folder on your SD Card: `SD:/config/JKSV/`
20. The next time you start JKSV on your Switch, you should be prompted to login to Google via the Switch's web browser. Ta-da!
20. The next time you start JKSV on your Switch, you should be prompted to login to Google via the Switch's web browser. Ta-da!
## <a name="webdav"></a><center> How to use WebDav with JKSV </center>
**NOTE: If you have [GDrive](#gdrive) configured (via JSON), it takes preference over WebDav**
1. Create a file `webdav.json` with the following content:
```json
{
"origin": "https://your-webdav-server",
"basepath": "optional-base-path",
"username": "testuser",
"password": "testpassword"
}
```
- `origin` (mandatory): protocol + serveraddress + (optional port), e.g. `https://your-webdav-server` or `http://localhost:8080` - **No trailing slash**
- `basepath` (optional): e.g. `dir`, `dir/subdir` must exist beforehand - **No leading AND trailing slash**
- `username` (optional): username, if server uses credentials
- `password` (optional): username, if server uses credentials
2. Copy file to following folder on your card `SD:/config/JKSV/`
3. The next time you start JKSV on your Switch, you should get a popup about the Webdav status
4. If problems arise, check the log at `SD:/JKSV/log.txt`

View File

@ -35,4 +35,5 @@ namespace cfg
extern std::vector<uint64_t> favorites;
extern uint8_t sortType;
extern std::string driveClientID, driveClientSecret, driveRefreshToken;
extern std::string webdavOrigin, webdavBasePath, webdavUser, webdavPassword;
}

View File

@ -8,7 +8,7 @@
#include "fs/dir.h"
#include "fs/zip.h"
#include "fs/fsfile.h"
#include "fs/drive.h"
#include "fs/remote.h"
#include "ui/miscui.h"
#define BUFF_SIZE 0x4000

View File

@ -1,15 +0,0 @@
#pragma once
#include "../gd.h"
#define JKSV_DRIVE_FOLDER "JKSV"
namespace fs
{
extern drive::gd *gDrive;
extern std::string jksvDriveID;
void driveInit();
void driveExit();
std::string driveSignInGetAuthCode();
}

21
inc/fs/remote.h Normal file
View File

@ -0,0 +1,21 @@
#pragma once
#include "../rfs.h"
#define JKSV_DRIVE_FOLDER "JKSV"
namespace fs
{
extern rfs::IRemoteFS *rfs;
extern std::string rfsRootID;
void remoteInit();
void remoteExit();
// Google Drive
void driveInit();
std::string driveSignInGetAuthCode();
// Webdav
void webDavInit();
}

View File

@ -8,6 +8,7 @@
#include <unordered_map>
#include "curlfuncs.h"
#include "rfs.h"
#define HEADER_CONTENT_TYPE_APP_JSON "Content-Type: application/json; charset=UTF-8"
#define HEADER_AUTHORIZATION "Authorization: Bearer "
@ -16,14 +17,7 @@
namespace drive
{
typedef struct
{
std::string name, id, parent;
bool isDir = false;
unsigned int size;
} gdItem;
class gd
class gd : public rfs::IRemoteFS
{
public:
void setClientID(const std::string& _clientID) { clientID = _clientID; }
@ -36,16 +30,17 @@ namespace drive
bool tokenIsValid();
void clearDriveList() { driveList.clear(); }
// TODO: This also gets files that do not belong to JKSV
void driveListInit(const std::string& _q);
void driveListAppend(const std::string& _q);
void getListWithParent(const std::string& _parent, std::vector<drive::gdItem *>& _out);
std::vector<rfs::RfsItem> getListWithParent(const std::string& _parent);
void debugWriteList();
bool createDir(const std::string& _dirName, const std::string& _parent);
// TODO: This is problematic, because multiple files could share the same name without a parent.
bool dirExists(const std::string& _dirName);
bool dirExists(const std::string& _dirName, const std::string& _parent);
bool fileExists(const std::string& _filename);
bool fileExists(const std::string& _filename, const std::string& _parent);
void uploadFile(const std::string& _filename, const std::string& _parent, curlFuncs::curlUpArgs *_upload);
void updateFile(const std::string& _fileID, curlFuncs::curlUpArgs *_upload);
@ -55,16 +50,17 @@ namespace drive
std::string getClientID() const { return clientID; }
std::string getClientSecret() const { return secretID; }
std::string getRefreshToken() const { return rToken; }
std::string getFileID(const std::string& _name);
std::string getFileID(const std::string& _name, const std::string& _parent);
// TODO: This is problematic, because multiple files could share the same name without a parent.
std::string getDirID(const std::string& _name);
std::string getDirID(const std::string& _name, const std::string& _parent);
size_t getDriveListCount() const { return driveList.size(); }
drive::gdItem *getItemAt(unsigned int _ind) { return &driveList[_ind]; }
rfs::RfsItem *getItemAt(unsigned int _ind) { return &driveList[_ind]; }
private:
std::vector<gdItem> driveList;
std::vector<rfs::RfsItem> driveList;
std::string clientID, secretID, token, rToken;
};
}

60
inc/rfs.h Normal file
View File

@ -0,0 +1,60 @@
//
// Created by Martin Riedel on 7/7/23.
//
#include <string>
#include "curlfuncs.h"
#include <mutex>
#ifndef JKSV_RFS_H
#define JKSV_RFS_H
#define UPLOAD_BUFFER_SIZE 0x8000
#define DOWNLOAD_BUFFER_SIZE 0xC00000
#define USER_AGENT "JKSV"
namespace rfs {
typedef struct
{
std::string name, id, parent;
bool isDir = false;
unsigned int size;
} RfsItem;
class IRemoteFS
{
public:
virtual ~IRemoteFS() {} // Virtual destructor to allow correct deletion through the base class pointer
virtual bool createDir(const std::string& _dirName, const std::string& _parent) = 0;
virtual bool dirExists(const std::string& _dirName, const std::string& _parent) = 0;
virtual bool fileExists(const std::string& _filename, const std::string& _parent) = 0;
virtual void uploadFile(const std::string& _filename, const std::string& _parent, curlFuncs::curlUpArgs *_upload) = 0;
virtual void updateFile(const std::string& _fileID, curlFuncs::curlUpArgs *_upload) = 0;
virtual void downloadFile(const std::string& _fileID, curlFuncs::curlDlArgs *_download) = 0;
virtual void deleteFile(const std::string& _fileID) = 0;
virtual std::string getFileID(const std::string& _name, const std::string& _parent) = 0;
virtual std::string getDirID(const std::string& _name, const std::string& _parent) = 0;
virtual std::vector<RfsItem> getListWithParent(const std::string& _parent) = 0;
};
// Shared multi-threading definitions
typedef struct
{
curlFuncs::curlDlArgs *cfa;
std::mutex dataLock;
std::condition_variable cond;
std::vector<uint8_t> sharedBuffer;
bool bufferFull = false;
unsigned int downloaded = 0;
} dlWriteThreadStruct;
extern std::vector<uint8_t> downloadBuffer;
void writeThread_t(void *a);
size_t writeDataBufferThreaded(uint8_t *buff, size_t sz, size_t cnt, void *u);
}
#endif //JKSV_RFS_H

59
inc/webdav.h Normal file
View File

@ -0,0 +1,59 @@
//
// Created by Martin Riedel on 7/7/23.
//
#include <curl/curl.h>
#include <string>
#include <tinyxml2.h>
#include "rfs.h"
#ifndef JKSV_WEBDAV_H
#define JKSV_WEBDAV_H
namespace rfs {
// Note: Everything declared an "id" is the full path component from the origin to the resource starting with a "/".
// Note: Directories ALWAYS have a trailing / while files NEVER have a trailing /
// e.g. /<basePath>/JKSV/
// e.g. /<basePath>/JKSV/<title-id>/
// e.g. /<basePath>/JKSV/<title-id>/<file>
// e.g. /
// other string arguments never have any leading or trailing "/"
class WebDav : public IRemoteFS {
private:
CURL* curl;
std::string origin;
std::string basePath;
std::string username;
std::string password;
std::vector<RfsItem> parseXMLResponse(const std::string& xml);
bool resourceExists(const std::string& id);
std::string appendResourceToParentId(const std::string& resourceName, const std::string& parentId, bool isDir);
std::string getNamespacePrefix(tinyxml2::XMLElement* root, const std::string& nsURI);
public:
WebDav(const std::string& origin,
const std::string& username,
const std::string& password);
~WebDav();
bool createDir(const std::string& dirName, const std::string& parentId);
bool dirExists(const std::string& dirName, const std::string& parentId);
bool fileExists(const std::string& filename, const std::string& parentId);
void uploadFile(const std::string& filename, const std::string& parentId, curlFuncs::curlUpArgs *_upload);
void updateFile(const std::string& fileID, curlFuncs::curlUpArgs *_upload);
void downloadFile(const std::string& fileID, curlFuncs::curlDlArgs *_download);
void deleteFile(const std::string& fileID);
std::string getFileID(const std::string& name, const std::string& parentId);
std::string getDirID(const std::string& dirName, const std::string& parentId);
std::vector<RfsItem> getListWithParent(const std::string& _parent);
};
}
#endif //JKSV_WEBDAV_H

View File

@ -69,6 +69,8 @@ popChangeOutputFolder = 0, "#%s# changed to #%s#"
popDriveFailed = 0, "Failed to start Google Drive."
popDriveNotActive = 0, "Google Drive is not available"
popDriveStarted = 0, "Google Drive started successfully."
popWebdavStarted = 0, "Webdav started successfully."
popWebdavFailed =, 0, "Failed to start Webdav."
popErrorCommittingFile = 0, "Error committing file to save!"
popFolderIsEmpty = 0, "Folder is empty!"
popProcessShutdown = 0, "#%s# successfully shutdown."

View File

@ -69,6 +69,8 @@ popChangeOutputFolder = 0, "#%s# changed to #%s#"
popDriveFailed = 0, "Failed to start Google Drive."
popDriveNotActive = 0, "Google Drive is not available"
popDriveStarted = 0, "Google Drive started successfully."
popWebdavStarted = 0, "Webdav started successfully."
popWebdavFailed =, 0, "Failed to start Webdav."
popErrorCommittingFile = 0, "Error committing file to save!"
popFolderIsEmpty = 0, "Folder is empty!"
popProcessShutdown = 0, "#%s# successfully shutdown."

View File

@ -69,6 +69,8 @@ popChangeOutputFolder = 0, "#%s# changed to #%s#"
popDriveFailed = 0, "Failed to start Google Drive."
popDriveNotActive = 0, "Google Drive is not available"
popDriveStarted = 0, "Google Drive started successfully."
popWebdavStarted = 0, "Webdav started successfully."
popWebdavFailed =, 0, "Failed to start Webdav."
popErrorCommittingFile = 0, "Error committing file to save!"
popFolderIsEmpty = 0, "Folder is empty!"
popProcessShutdown = 0, "#%s# successfully shutdown."

View File

@ -86,6 +86,8 @@ popChangeOutputFolder = 0, "Se cambia #%s# por #%s#"
popDriveFailed = 0, "Failed to start Google Drive."
popDriveNotActive = 0, "Google Drive is not available"
popDriveStarted = 0, "Google Drive started successfully."
popWebdavStarted = 0, "Webdav started successfully."
popWebdavFailed =, 0, "Failed to start Webdav."
#<====================================================
popErrorCommittingFile = 0, "¡Error guardando archivo!"

View File

@ -69,6 +69,8 @@ popChangeOutputFolder = 0, "#%s# changed to #%s#"
popDriveFailed = 0, "Failed to start Google Drive."
popDriveNotActive = 0, "Google Drive is not available"
popDriveStarted = 0, "Google Drive started successfully."
popWebdavStarted = 0, "Webdav started successfully."
popWebdavFailed =, 0, "Failed to start Webdav."
popErrorCommittingFile = 0, "Error committing file to save!"
popFolderIsEmpty = 0, "Folder is empty!"
popProcessShutdown = 0, "#%s# successfully shutdown."

View File

@ -69,6 +69,8 @@ popChangeOutputFolder = 0, "#%s# changed to #%s#"
popDriveFailed = 0, "Failed to start Google Drive."
popDriveNotActive = 0, "Google Drive is not available"
popDriveStarted = 0, "Google Drive started successfully."
popWebdavStarted = 0, "Webdav started successfully."
popWebdavFailed =, 0, "Failed to start Webdav."
popErrorCommittingFile = 0, "Error committing file to save!"
popFolderIsEmpty = 0, "Folder is empty!"
popProcessShutdown = 0, "#%s# successfully shutdown."

View File

@ -68,6 +68,8 @@ popChangeOutputFolder = 0, "#%s# modifié en #%s#"
popDriveFailed = 0, "Echec du démarrage de Google Drive."
popDriveNotActive = 0, "Google Drive n'est pas utilisable"
popDriveStarted = 0, "Google Drive lancé avec succès."
popWebdavStarted = 0, "Webdav started successfully."
popWebdavFailed =, 0, "Failed to start Webdav."
popErrorCommittingFile = 0, "Erreur d'ajout du fichier à la sauvegarde!"
popFolderIsEmpty = 0, "Le dossier est vide!"
popProcessShutdown = 0, "#%s# terminé avec succès."

View File

@ -68,6 +68,8 @@ popChangeOutputFolder = 0, "#%s# cambiato in #%s#"
popDriveFailed = 0, "Impossibile avviare Google Drive."
popDriveNotActive = 0, "Google Drive non è Disponibile"
popDriveStarted = 0, "Google Drive avviato con successo."
popWebdavStarted = 0, "Webdav started successfully."
popWebdavFailed =, 0, "Failed to start Webdav."
popErrorCommittingFile = 0, "Errore durante il commiting del file da salvare!"
popFolderIsEmpty = 0, "La cartella è vuota!"
popProcessShutdown = 0, "#%s# spento con successo."

View File

@ -86,6 +86,8 @@ popChangeOutputFolder = 0, "#%s# から #%s# に変更"
popDriveFailed = 0, "Google Driveの起動に失敗しました。"
popDriveNotActive = 0, "Googleドライブは使用できません"
popDriveStarted = 0, "Google Driveが正常に起動しました。"
popWebdavStarted = 0, "Webdav started successfully."
popWebdavFailed =, 0, "Failed to start Webdav."
#<====================================================
popErrorCommittingFile = 0, "保存するファイルのコミット中にエラーが発生しました!"

View File

@ -69,6 +69,8 @@ popChangeOutputFolder = 0, "#%s#이(가) #%s#로 변경되었습니다."
popDriveFailed = 0, "구글 드라이브 시작에 실패했습니다."
popDriveNotActive = 0, "구글 드라이브를 사용할 수 없습니다."
popDriveStarted = 0, "구글 드라이브가 성공적으로 시작되었습니다."
popWebdavStarted = 0, "Webdav started successfully."
popWebdavFailed =, 0, "Failed to start Webdav."
popErrorCommittingFile = 0, "저장할 파일을 커밋하는 동안 오류가 발생했습니다!"
popFolderIsEmpty = 0, "폴더가 비어 있습니다!"
popProcessShutdown = 0, "#%s#이(가) 성공적으로 종료되었습니다."

View File

@ -69,6 +69,8 @@ popChangeOutputFolder = 0, "#%s# changed to #%s#"
popDriveFailed = 0, "Failed to start Google Drive."
popDriveNotActive = 0, "Google Drive is not available"
popDriveStarted = 0, "Google Drive started successfully."
popWebdavStarted = 0, "Webdav started successfully."
popWebdavFailed =, 0, "Failed to start Webdav."
popErrorCommittingFile = 0, "Error committing file to save!"
popFolderIsEmpty = 0, "Folder is empty!"
popProcessShutdown = 0, "#%s# successfully shutdown."

View File

@ -69,6 +69,8 @@ popChangeOutputFolder = 0, "#%s# changed to #%s#"
popDriveFailed = 0, "Failed to start Google Drive."
popDriveNotActive = 0, "Google Drive is not available"
popDriveStarted = 0, "Google Drive started successfully."
popWebdavStarted = 0, "Webdav started successfully."
popWebdavFailed =, 0, "Failed to start Webdav."
popErrorCommittingFile = 0, "Error committing file to save!"
popFolderIsEmpty = 0, "Folder is empty!"
popProcessShutdown = 0, "#%s# successfully shutdown."

View File

@ -69,6 +69,8 @@ popChangeOutputFolder = 0, "#%s# changed to #%s#"
popDriveFailed = 0, "Failed to start Google Drive."
popDriveNotActive = 0, "Google Drive is not available"
popDriveStarted = 0, "Google Drive started successfully."
popWebdavStarted = 0, "Webdav started successfully."
popWebdavFailed =, 0, "Failed to start Webdav."
popErrorCommittingFile = 0, "Error committing file to save!"
popFolderIsEmpty = 0, "Folder is empty!"
popProcessShutdown = 0, "#%s# successfully shutdown."

View File

@ -68,6 +68,8 @@ popChangeOutputFolder = 0, "#%s#更改到#%s#"
popDriveFailed = 0, "Google Drive启动失败。"
popDriveNotActive = 0, "Google Drive不可用。"
popDriveStarted = 0, "Google Drive启动成功。"
popWebdavStarted = 0, "Webdav started successfully."
popWebdavFailed =, 0, "Failed to start Webdav."
popErrorCommittingFile = 0, "提交要保存的文件时出错!"
popFolderIsEmpty = 0, "文件夹为空!"
popProcessShutdown = 0, "#%s#已成功关闭。"

View File

@ -68,6 +68,8 @@ popChangeOutputFolder = 0, "#%s# 更改到 #%s#"
popDriveFailed = 0, "無法啟用Google雲端硬碟."
popDriveNotActive = 0, "沒有可用的Google雲端硬碟."
popDriveStarted = 0, "Google雲端硬碟已正確啟用."
popWebdavStarted = 0, "Webdav started successfully."
popWebdavFailed =, 0, "Failed to start Webdav."
popErrorCommittingFile = 0, "提交檔案進行儲存時發生錯誤!"
popFolderIsEmpty = 0, "資料夾內沒有檔案!"
popProcessShutdown = 0, "#%s# 程序已關閉."

View File

@ -16,6 +16,8 @@ std::vector<uint64_t> cfg::favorites;
static std::unordered_map<uint64_t, std::string> pathDefs;
uint8_t cfg::sortType;
std::string cfg::driveClientID, cfg::driveClientSecret, cfg::driveRefreshToken;
std::string cfg::webdavOrigin, cfg::webdavBasePath, cfg::webdavUser, cfg::webdavPassword;
const char *cfgPath = "sdmc:/config/JKSV/JKSV.cfg", *titleDefPath = "sdmc:/config/JKSV/titleDefs.txt", *workDirLegacy = "sdmc:/switch/jksv_dir.txt";
static std::unordered_map<std::string, unsigned> cfgStrings =
@ -294,8 +296,8 @@ static void loadTitleDefs()
static void loadDriveConfig()
{
// Start Google Drive
fs::dirList cfgList("/config/JKSV/", true);
std::string clientSecretPath;
for(unsigned i = 0; i < cfgList.getCount(); i++)
{
@ -324,6 +326,26 @@ static void loadDriveConfig()
json_object_put(driveJSON);
}
}
// End Google Drive
// Webdav
json_object *webdavJSON = json_object_from_file("/config/JKSV/webdav.json");
json_object *origin, *basepath, *username, *password;
if (webdavJSON)
{
if (json_object_object_get_ex(webdavJSON, "origin", &origin)) {
cfg::webdavOrigin = json_object_get_string(origin);
}
if (json_object_object_get_ex(webdavJSON, "basepath", &basepath)) {
cfg::webdavBasePath = json_object_get_string(basepath);
}
if (json_object_object_get_ex(webdavJSON, "username", &username)) {
cfg::webdavUser = json_object_get_string(username);
}
if (json_object_object_get_ex(webdavJSON, "password", &password)) {
cfg::webdavPassword = json_object_get_string(password);
}
}
}
void cfg::loadConfig()

View File

@ -1,90 +0,0 @@
#include "fs.h"
#include "drive.h"
#include "cfg.h"
#include "ui.h"
drive::gd *fs::gDrive = NULL;
std::string fs::jksvDriveID;
void fs::driveInit()
{
if(cfg::driveClientID.empty() || cfg::driveClientSecret.empty())
return;
bool refreshed = false, exchanged = false;
fs::gDrive = new drive::gd;
fs::gDrive->setClientID(cfg::driveClientID);
fs::gDrive->setClientSecret(cfg::driveClientSecret);
if(!cfg::driveRefreshToken.empty())
{
fs::gDrive->setRefreshToken(cfg::driveRefreshToken);
refreshed = fs::gDrive->refreshToken();
}
if(!refreshed)
{
std::string authCode = driveSignInGetAuthCode();
exchanged = fs::gDrive->exhangeAuthCode(authCode);
}
if(fs::gDrive->hasToken())
{
if(exchanged)
{
cfg::driveRefreshToken = fs::gDrive->getRefreshToken();
cfg::saveConfig();
}
fs::gDrive->driveListInit("");
if(!fs::gDrive->dirExists(JKSV_DRIVE_FOLDER))
fs::gDrive->createDir(JKSV_DRIVE_FOLDER, "");
jksvDriveID = fs::gDrive->getDirID(JKSV_DRIVE_FOLDER);
ui::showPopMessage(POP_FRAME_DEFAULT, ui::getUICString("popDriveStarted", 0));
}
else
{
delete fs::gDrive;
fs::gDrive = NULL;
ui::showPopMessage(POP_FRAME_DEFAULT, ui::getUICString("popDriveFailed", 0));
}
}
void fs::driveExit()
{
if(fs::gDrive)
delete gDrive;
}
std::string fs::driveSignInGetAuthCode()
{
std::string url = "https://accounts.google.com/o/oauth2/v2/auth?client_id=" + cfg::driveClientID + "&redirect_uri=urn:ietf:wg:oauth:2.0:oob:auto&response_type=code&scope=https://www.googleapis.com/auth/drive";
std::string replyURL;
WebCommonConfig webCfg;
WebCommonReply webReply;
webPageCreate(&webCfg, url.c_str());
webConfigSetCallbackUrl(&webCfg, "https://accounts.google.com/o/oauth2/approval/");
webConfigShow(&webCfg, &webReply);
size_t rLength = 0;
char replyURLCstr[0x1000];
webReplyGetLastUrl(&webReply, replyURLCstr, 0x1000, &rLength);
//Prevent crash if empty.
if(strlen(replyURLCstr) == 0)
return "";
replyURL.assign(replyURLCstr);
int unescLength = 0;
size_t codeBegin = replyURL.find("approvalCode") + 13, codeEnd = replyURL.find_last_of('#');
size_t codeLength = codeEnd - codeBegin;
replyURL = replyURL.substr(codeBegin, codeLength);
char *urlUnesc = curl_easy_unescape(NULL, replyURL.c_str(), replyURL.length(), &unescLength);
replyURL = urlUnesc;
curl_free(urlUnesc);
//Finally
return replyURL;
}

136
src/fs/remote.cpp Normal file
View File

@ -0,0 +1,136 @@
#include "fs.h"
#include "rfs.h"
#include "gd.h"
#include "webdav.h"
#include "cfg.h"
#include "ui.h"
rfs::IRemoteFS *fs::rfs = NULL;
std::string fs::rfsRootID;
void fs::remoteInit()
{
// Google Drive has priority
driveInit();
webDavInit();
}
void fs::remoteExit()
{
if(rfs) {
delete rfs;
rfs = NULL;
}
}
void fs::driveInit()
{
// Already initialized?
if (rfs)
return;
if(cfg::driveClientID.empty() || cfg::driveClientSecret.empty())
return;
bool refreshed = false, exchanged = false;
drive::gd *gDrive = new drive::gd;
gDrive->setClientID(cfg::driveClientID);
gDrive->setClientSecret(cfg::driveClientSecret);
if(!cfg::driveRefreshToken.empty())
{
gDrive->setRefreshToken(cfg::driveRefreshToken);
refreshed = gDrive->refreshToken();
}
if(!refreshed)
{
std::string authCode = driveSignInGetAuthCode();
exchanged = gDrive->exhangeAuthCode(authCode);
}
if(gDrive->hasToken())
{
if(exchanged)
{
cfg::driveRefreshToken = gDrive->getRefreshToken();
cfg::saveConfig();
}
gDrive->driveListInit("");
if(!gDrive->dirExists(JKSV_DRIVE_FOLDER))
gDrive->createDir(JKSV_DRIVE_FOLDER, "");
rfsRootID = gDrive->getDirID(JKSV_DRIVE_FOLDER);
rfs = gDrive;
ui::showPopMessage(POP_FRAME_DEFAULT, ui::getUICString("popDriveStarted", 0));
}
else
{
delete gDrive;
ui::showPopMessage(POP_FRAME_DEFAULT, ui::getUICString("popDriveFailed", 0));
}
}
std::string fs::driveSignInGetAuthCode()
{
std::string url = "https://accounts.google.com/o/oauth2/v2/auth?client_id=" + cfg::driveClientID + "&redirect_uri=urn:ietf:wg:oauth:2.0:oob:auto&response_type=code&scope=https://www.googleapis.com/auth/drive";
std::string replyURL;
WebCommonConfig webCfg;
WebCommonReply webReply;
webPageCreate(&webCfg, url.c_str());
webConfigSetCallbackUrl(&webCfg, "https://accounts.google.com/o/oauth2/approval/");
webConfigShow(&webCfg, &webReply);
size_t rLength = 0;
char replyURLCstr[0x1000];
webReplyGetLastUrl(&webReply, replyURLCstr, 0x1000, &rLength);
//Prevent crash if empty.
if(strlen(replyURLCstr) == 0)
return "";
replyURL.assign(replyURLCstr);
int unescLength = 0;
size_t codeBegin = replyURL.find("approvalCode") + 13, codeEnd = replyURL.find_last_of('#');
size_t codeLength = codeEnd - codeBegin;
replyURL = replyURL.substr(codeBegin, codeLength);
char *urlUnesc = curl_easy_unescape(NULL, replyURL.c_str(), replyURL.length(), &unescLength);
replyURL = urlUnesc;
curl_free(urlUnesc);
//Finally
return replyURL;
}
void fs::webDavInit() {
// Already initialized?
if (rfs)
return;
if (cfg::webdavOrigin.empty())
return;
rfs::WebDav *webdav = new rfs::WebDav(cfg::webdavOrigin,
cfg::webdavUser,
cfg::webdavPassword);
std::string baseId = "/" + cfg::webdavBasePath + (cfg::webdavBasePath.empty() ? "" : "/");
rfsRootID = webdav->getDirID(JKSV_DRIVE_FOLDER, baseId);
// check access
if (!webdav->dirExists(JKSV_DRIVE_FOLDER, baseId)) // this could return false on auth/config related errors
{
if (!webdav->createDir(JKSV_DRIVE_FOLDER, baseId))
{
delete webdav;
ui::showPopMessage(POP_FRAME_DEFAULT, ui::getUICString("popWebdavFailed", 0));
return;
}
}
// webdav->listInit(rfsRootID);
rfs = webdav;
ui::showPopMessage(POP_FRAME_DEFAULT, ui::getUICString("popWebdavStarted", 0));
}

View File

@ -16,27 +16,12 @@ Google Drive code for JKSV.
Still major WIP
*/
#define DRIVE_UPLOAD_BUFFER_SIZE 0x8000
#define DRIVE_DOWNLOAD_BUFFER_SIZE 0xC00000
#define DRIVE_DEFAULT_PARAMS_AND_QUERY "?fields=files(name,id,mimeType,size,parents)&pageSize=1000&q=trashed=false\%20and\%20\%27me\%27\%20in\%20owners"
#define tokenURL "https://oauth2.googleapis.com/token"
#define tokenCheckURL "https://oauth2.googleapis.com/tokeninfo"
#define driveURL "https://www.googleapis.com/drive/v3/files"
#define driveUploadURL "https://www.googleapis.com/upload/drive/v3/files"
#define userAgent "JKSV"
std::vector<uint8_t> downloadBuffer;
typedef struct
{
curlFuncs::curlDlArgs *cfa;
std::mutex dataLock;
std::condition_variable cond;
std::vector<uint8_t> sharedBuffer;
bool bufferFull = false;
unsigned int downloaded = 0;
} dlWriteThreadStruct;
static inline void writeDriveError(const std::string& _function, const std::string& _message)
{
@ -48,54 +33,6 @@ static inline void writeCurlError(const std::string& _function, int _cerror)
fs::logWrite("Drive/%s: CURL returned error %i\n", _function.c_str(), _cerror);
}
static void writeThread_t(void *a)
{
dlWriteThreadStruct *in = (dlWriteThreadStruct *)a;
std::vector<uint8_t> localBuff;
unsigned written = 0;
FILE *out = fopen(in->cfa->path.c_str(), "wb");
while(written < in->cfa->size)
{
std::unique_lock<std::mutex> dataLock(in->dataLock);
in->cond.wait(dataLock, [in]{ return in->bufferFull; });
localBuff.clear();
localBuff.assign(in->sharedBuffer.begin(), in->sharedBuffer.end());
in->sharedBuffer.clear();
in->bufferFull = false;
dataLock.unlock();
in->cond.notify_one();
written += fwrite(localBuff.data(), 1, localBuff.size(), out);
}
fclose(out);
downloadBuffer.clear();
}
static size_t writeDataBufferThreaded(uint8_t *buff, size_t sz, size_t cnt, void *u)
{
dlWriteThreadStruct *in = (dlWriteThreadStruct *)u;
downloadBuffer.insert(downloadBuffer.end(), buff, buff + (sz * cnt));
in->downloaded += sz * cnt;
if(in->downloaded == in->cfa->size || downloadBuffer.size() == DRIVE_DOWNLOAD_BUFFER_SIZE)
{
std::unique_lock<std::mutex> dataLock(in->dataLock);
in->cond.wait(dataLock, [in]{ return in->bufferFull == false; });
in->sharedBuffer.assign(downloadBuffer.begin(), downloadBuffer.end());
downloadBuffer.clear();
in->bufferFull = true;
dataLock.unlock();
in->cond.notify_one();
}
if(in->cfa->o)
*in->cfa->o = in->downloaded;
return sz * cnt;
}
bool drive::gd::exhangeAuthCode(const std::string& _authCode)
{
// Header
@ -119,7 +56,7 @@ bool drive::gd::exhangeAuthCode(const std::string& _authCode)
std::string *jsonResp = new std::string;
CURL *curl = curl_easy_init();
curl_easy_setopt(curl, CURLOPT_HTTPPOST, 1);
curl_easy_setopt(curl, CURLOPT_USERAGENT, userAgent);
curl_easy_setopt(curl, CURLOPT_USERAGENT, USER_AGENT);
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, postHeader);
curl_easy_setopt(curl, CURLOPT_URL, tokenURL);
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, curlFuncs::writeDataString);
@ -177,7 +114,7 @@ bool drive::gd::refreshToken()
std::string *jsonResp = new std::string;
CURL *curl = curl_easy_init();
curl_easy_setopt(curl, CURLOPT_HTTPPOST, 1);
curl_easy_setopt(curl, CURLOPT_USERAGENT, userAgent);
curl_easy_setopt(curl, CURLOPT_USERAGENT, USER_AGENT);
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, header);
curl_easy_setopt(curl, CURLOPT_URL, tokenURL);
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, curlFuncs::writeDataString);
@ -220,7 +157,7 @@ bool drive::gd::tokenIsValid()
CURL *curl = curl_easy_init();
std::string *jsonResp = new std::string;
curl_easy_setopt(curl, CURLOPT_HTTPGET, 1);
curl_easy_setopt(curl, CURLOPT_USERAGENT, userAgent);
curl_easy_setopt(curl, CURLOPT_USERAGENT, USER_AGENT);
curl_easy_setopt(curl, CURLOPT_URL, url.c_str());
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, curlFuncs::writeDataString);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, jsonResp);
@ -252,7 +189,7 @@ static int requestList(const std::string& _url, const std::string& _token, std::
CURL *curl = curl_easy_init();
curl_easy_setopt(curl, CURLOPT_HTTPGET, 1);
curl_easy_setopt(curl, CURLOPT_ACCEPT_ENCODING, "");
curl_easy_setopt(curl, CURLOPT_USERAGENT, userAgent);
curl_easy_setopt(curl, CURLOPT_USERAGENT, USER_AGENT);
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, postHeaders);
curl_easy_setopt(curl, CURLOPT_URL, _url.c_str());
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, curlFuncs::writeDataString);
@ -266,7 +203,7 @@ static int requestList(const std::string& _url, const std::string& _token, std::
return ret;
}
static void processList(const std::string& _json, std::vector<drive::gdItem>& _drvl, bool _clear)
static void processList(const std::string& _json, std::vector<rfs::RfsItem>& _drvl, bool _clear)
{
if(_clear)
_drvl.clear();
@ -287,7 +224,7 @@ static void processList(const std::string& _json, std::vector<drive::gdItem>& _d
json_object_object_get_ex(curFile, "size", &size);
json_object_object_get_ex(curFile, "parents", &parentArray);
drive::gdItem newDirItem;
rfs::RfsItem newDirItem;
newDirItem.name = json_object_get_string(nameString);
newDirItem.id = json_object_get_string(idString);
newDirItem.size = json_object_get_int(size);
@ -353,14 +290,14 @@ void drive::gd::driveListAppend(const std::string& _q)
writeCurlError("driveListAppend", error);
}
void drive::gd::getListWithParent(const std::string& _parent, std::vector<drive::gdItem *>& _out)
{
_out.clear();
std::vector<rfs::RfsItem> drive::gd::getListWithParent(const std::string& _parent) {
std::vector<rfs::RfsItem> filtered;
for(unsigned i = 0; i < driveList.size(); i++)
{
if(driveList[i].parent == _parent)
_out.push_back(&driveList[i]);
filtered.push_back(driveList[i]);
}
return filtered;
}
void drive::gd::debugWriteList()
@ -403,7 +340,7 @@ bool drive::gd::createDir(const std::string& _dirName, const std::string& _paren
std::string *jsonResp = new std::string;
CURL *curl = curl_easy_init();
curl_easy_setopt(curl, CURLOPT_HTTPPOST, 1);
curl_easy_setopt(curl, CURLOPT_USERAGENT, userAgent);
curl_easy_setopt(curl, CURLOPT_USERAGENT, USER_AGENT);
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, postHeaders);
curl_easy_setopt(curl, CURLOPT_URL, driveURL);
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, curlFuncs::writeDataString);
@ -419,7 +356,7 @@ bool drive::gd::createDir(const std::string& _dirName, const std::string& _paren
json_object *id;
json_object_object_get_ex(respParse, "id", &id);
drive::gdItem newDir;
rfs::RfsItem newDir;
newDir.name = _dirName;
newDir.id = json_object_get_string(id);
newDir.isDir = true;
@ -458,16 +395,6 @@ bool drive::gd::dirExists(const std::string& _dirName, const std::string& _paren
return false;
}
bool drive::gd::fileExists(const std::string& _filename)
{
for(unsigned i = 0; i < driveList.size(); i++)
{
if(!driveList[i].isDir && driveList[i].name == _filename)
return true;
}
return false;
}
bool drive::gd::fileExists(const std::string& _filename, const std::string& _parent)
{
for(unsigned i = 0; i < driveList.size(); i++)
@ -508,7 +435,7 @@ void drive::gd::uploadFile(const std::string& _filename, const std::string& _par
std::vector<std::string> *headers = new std::vector<std::string>;
CURL *curl = curl_easy_init();
curl_easy_setopt(curl, CURLOPT_HTTPPOST, 1);
curl_easy_setopt(curl, CURLOPT_USERAGENT, userAgent);
curl_easy_setopt(curl, CURLOPT_USERAGENT, USER_AGENT);
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, postHeaders);
curl_easy_setopt(curl, CURLOPT_URL, url.c_str());
curl_easy_setopt(curl, CURLOPT_HEADERFUNCTION, curlFuncs::writeHeaders);
@ -526,7 +453,7 @@ void drive::gd::uploadFile(const std::string& _filename, const std::string& _par
curl_easy_setopt(curlUp, CURLOPT_WRITEDATA, jsonResp);
curl_easy_setopt(curlUp, CURLOPT_READFUNCTION, curlFuncs::readDataFile);
curl_easy_setopt(curlUp, CURLOPT_READDATA, _upload);
curl_easy_setopt(curlUp, CURLOPT_UPLOAD_BUFFERSIZE, DRIVE_UPLOAD_BUFFER_SIZE);
curl_easy_setopt(curlUp, CURLOPT_UPLOAD_BUFFERSIZE, UPLOAD_BUFFER_SIZE);
curl_easy_setopt(curlUp, CURLOPT_UPLOAD, 1);
curl_easy_perform(curlUp);
curl_easy_cleanup(curlUp);
@ -538,7 +465,7 @@ void drive::gd::uploadFile(const std::string& _filename, const std::string& _par
if(name && id && mimeType)
{
drive::gdItem uploadData;
rfs::RfsItem uploadData;
uploadData.id = json_object_get_string(id);
uploadData.name = json_object_get_string(name);
uploadData.isDir = false;
@ -577,7 +504,7 @@ void drive::gd::updateFile(const std::string& _fileID, curlFuncs::curlUpArgs *_u
CURL *curl = curl_easy_init();
curl_easy_setopt(curl, CURLOPT_CUSTOMREQUEST, "PATCH");
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, patchHeader);
curl_easy_setopt(curl, CURLOPT_USERAGENT, userAgent);
curl_easy_setopt(curl, CURLOPT_USERAGENT, USER_AGENT);
curl_easy_setopt(curl, CURLOPT_URL, url.c_str());
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, curlFuncs::writeDataString);
curl_easy_setopt(curl, CURLOPT_HEADERFUNCTION, curlFuncs::writeHeaders);
@ -593,7 +520,7 @@ void drive::gd::updateFile(const std::string& _fileID, curlFuncs::curlUpArgs *_u
curl_easy_setopt(curlPatch, CURLOPT_URL, location.c_str());
curl_easy_setopt(curlPatch, CURLOPT_READFUNCTION, curlFuncs::readDataFile);
curl_easy_setopt(curlPatch, CURLOPT_READDATA, _upload);
curl_easy_setopt(curlPatch, CURLOPT_UPLOAD_BUFFERSIZE, DRIVE_UPLOAD_BUFFER_SIZE);
curl_easy_setopt(curlPatch, CURLOPT_UPLOAD_BUFFERSIZE, UPLOAD_BUFFER_SIZE);
curl_easy_setopt(curlPatch, CURLOPT_UPLOAD, 1);
curl_easy_perform(curlPatch);
curl_easy_cleanup(curlPatch);
@ -629,19 +556,19 @@ void drive::gd::downloadFile(const std::string& _fileID, curlFuncs::curlDlArgs *
getHeaders = curl_slist_append(getHeaders, std::string(HEADER_AUTHORIZATION + token).c_str());
//Downloading is threaded because it's too slow otherwise
dlWriteThreadStruct dlWrite;
rfs::dlWriteThreadStruct dlWrite;
dlWrite.cfa = _download;
Thread writeThread;
threadCreate(&writeThread, writeThread_t, &dlWrite, NULL, 0x8000, 0x2B, 2);
threadCreate(&writeThread, rfs::writeThread_t, &dlWrite, NULL, 0x8000, 0x2B, 2);
//Curl
CURL *curl = curl_easy_init();
curl_easy_setopt(curl, CURLOPT_HTTPGET, 1);
curl_easy_setopt(curl, CURLOPT_USERAGENT, userAgent);
curl_easy_setopt(curl, CURLOPT_USERAGENT, USER_AGENT);
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, getHeaders);
curl_easy_setopt(curl, CURLOPT_URL, url.c_str());
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, writeDataBufferThreaded);
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, rfs::writeDataBufferThreaded);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, &dlWrite);
threadStart(&writeThread);
@ -670,7 +597,7 @@ void drive::gd::deleteFile(const std::string& _fileID)
//Curl
CURL *curl = curl_easy_init();
curl_easy_setopt(curl, CURLOPT_CUSTOMREQUEST, "DELETE");
curl_easy_setopt(curl, CURLOPT_USERAGENT, userAgent);
curl_easy_setopt(curl, CURLOPT_USERAGENT, USER_AGENT);
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, delHeaders);
curl_easy_setopt(curl, CURLOPT_URL, url.c_str());
curl_easy_perform(curl);
@ -688,16 +615,6 @@ void drive::gd::deleteFile(const std::string& _fileID)
curl_easy_cleanup(curl);
}
std::string drive::gd::getFileID(const std::string& _name)
{
for(unsigned i = 0; i < driveList.size(); i++)
{
if(!driveList[i].isDir && driveList[i].name == _name)
return driveList[i].id;
}
return "";
}
std::string drive::gd::getFileID(const std::string& _name, const std::string& _parent)
{
for(unsigned i = 0; i < driveList.size(); i++)

View File

@ -53,13 +53,13 @@ int main(int argc, const char *argv[])
curl_global_init(CURL_GLOBAL_ALL);
//Drive needs config read
if(!util::isApplet())
fs::driveInit();
fs::remoteInit();
else
ui::showMessage(ui::getUICString("appletModeWarning", 0));
while(ui::runApp()){ }
fs::driveExit();
fs::remoteExit();
curl_global_cleanup();
cfg::saveConfig();
ui::exit();

51
src/rfs.cpp Normal file
View File

@ -0,0 +1,51 @@
#include "rfs.h"
std::vector<uint8_t> rfs::downloadBuffer;
void rfs::writeThread_t(void *a)
{
rfs::dlWriteThreadStruct *in = (rfs::dlWriteThreadStruct *)a;
std::vector<uint8_t> localBuff;
unsigned written = 0;
FILE *out = fopen(in->cfa->path.c_str(), "wb");
while(written < in->cfa->size)
{
std::unique_lock<std::mutex> dataLock(in->dataLock);
in->cond.wait(dataLock, [in]{ return in->bufferFull; });
localBuff.clear();
localBuff.assign(in->sharedBuffer.begin(), in->sharedBuffer.end());
in->sharedBuffer.clear();
in->bufferFull = false;
dataLock.unlock();
in->cond.notify_one();
written += fwrite(localBuff.data(), 1, localBuff.size(), out);
}
fclose(out);
rfs::downloadBuffer.clear();
}
size_t rfs::writeDataBufferThreaded(uint8_t *buff, size_t sz, size_t cnt, void *u)
{
rfs::dlWriteThreadStruct *in = (rfs::dlWriteThreadStruct *)u;
rfs::downloadBuffer.insert(rfs::downloadBuffer.end(), buff, buff + (sz * cnt));
in->downloaded += sz * cnt;
if(in->downloaded == in->cfa->size || rfs::downloadBuffer.size() == DOWNLOAD_BUFFER_SIZE)
{
std::unique_lock<std::mutex> dataLock(in->dataLock);
in->cond.wait(dataLock, [in]{ return in->bufferFull == false; });
in->sharedBuffer.assign(rfs::downloadBuffer.begin(), rfs::downloadBuffer.end());
rfs::downloadBuffer.clear();
in->bufferFull = true;
dataLock.unlock();
in->cond.notify_one();
}
if(in->cfa->o)
*in->cfa->o = in->downloaded;
return sz * cnt;
}

View File

@ -12,7 +12,7 @@ static SDL_Texture *fldBuffer;
static unsigned int fldGuideWidth = 0;
static Mutex fldLock = 0;
static std::string driveParent;
static std::vector<drive::gdItem *> driveFldList;
static std::vector<rfs::RfsItem> driveFldList;
static void fldMenuCallback(void *a)
{
@ -126,13 +126,13 @@ static void fldFuncUpload_t(void *a)
upload.f = fopen(path.c_str(), "rb");
upload.o = &cpyArgs->offset;
if(fs::gDrive->fileExists(filename))
if(fs::rfs->fileExists(filename, driveParent))
{
std::string id = fs::gDrive->getFileID(filename);
fs::gDrive->updateFile(id, &upload);
std::string id = fs::rfs->getFileID(filename, driveParent);
fs::rfs->updateFile(id, &upload);
}
else
fs::gDrive->uploadFile(filename, driveParent, &upload);
fs::rfs->uploadFile(filename, driveParent, &upload);
fclose(upload.f);
@ -152,7 +152,7 @@ static void fldFuncUpload_t(void *a)
static void fldFuncUpload(void *a)
{
if(fs::gDrive)
if(fs::rfs)
ui::newThread(fldFuncUpload_t, a, NULL);
else
ui::showPopMessage(POP_FRAME_DEFAULT, ui::getUICString("popDriveNotActive", 0));
@ -161,7 +161,7 @@ static void fldFuncUpload(void *a)
static void fldFuncDownload_t(void *a)
{
threadInfo *t = (threadInfo *)a;
drive::gdItem *in = (drive::gdItem *)t->argPtr;
rfs::RfsItem *in = (rfs::RfsItem *)t->argPtr;
data::userTitleInfo *utinfo = data::getCurrentUserTitleInfo();
std::string targetPath = util::generatePathByTID(utinfo->tid) + in->name;
t->status->setStatus(ui::getUICString("threadStatusDownloadingFile", 0), in->name.c_str());
@ -185,7 +185,7 @@ static void fldFuncDownload_t(void *a)
dlFile.size = in->size;
dlFile.o = &cpy->offset;
fs::gDrive->downloadFile(in->id, &dlFile);
fs::rfs->downloadFile(in->id, &dlFile);
//fclose(dlFile.f);
@ -202,7 +202,7 @@ static void fldFuncDownload_t(void *a)
static void fldFuncDownload(void *a)
{
drive::gdItem *in = (drive::gdItem *)a;
rfs::RfsItem *in = (rfs::RfsItem *)a;
data::userTitleInfo *utinfo = data::getCurrentUserTitleInfo();
std::string testPath = util::generatePathByTID(utinfo->tid) + in->name;
if(fs::fileExists(testPath))
@ -218,16 +218,16 @@ static void fldFuncDownload(void *a)
static void fldFuncDriveDelete_t(void *a)
{
threadInfo *t = (threadInfo *)a;
drive::gdItem *gdi = (drive::gdItem *)t->argPtr;
rfs::RfsItem *gdi = (rfs::RfsItem *)t->argPtr;
t->status->setStatus(ui::getUICString("threadStatusDeletingFile", 0));
fs::gDrive->deleteFile(gdi->id);
fs::rfs->deleteFile(gdi->id);
ui::fldRefreshMenu();
t->finished = true;
}
static void fldFuncDriveDelete(void *a)
{
drive::gdItem *in = (drive::gdItem *)a;
rfs::RfsItem *in = (rfs::RfsItem *)a;
ui::confirmArgs *conf = ui::confirmArgsCreate(cfg::config["holdDel"], fldFuncDriveDelete_t, NULL, a, ui::getUICString("confirmDelete", 0), in->name.c_str());
ui::confirm(conf);
}
@ -235,7 +235,7 @@ static void fldFuncDriveDelete(void *a)
static void fldFuncDriveRestore_t(void *a)
{
threadInfo *t = (threadInfo *)a;
drive::gdItem *gdi = (drive::gdItem *)t->argPtr;
rfs::RfsItem *gdi = (rfs::RfsItem *)t->argPtr;
t->status->setStatus(ui::getUICString("threadStatusDownloadingFile", 0), gdi->name.c_str());
fs::copyArgs *cpy = fs::copyArgsCreate("", "", "", NULL, NULL, false, false, 0);
@ -249,7 +249,7 @@ static void fldFuncDriveRestore_t(void *a)
dlFile.size = gdi->size;
dlFile.o = &cpy->offset;
fs::gDrive->downloadFile(gdi->id, &dlFile);
fs::rfs->downloadFile(gdi->id, &dlFile);
unzFile tmp = unzOpen64("sdmc:/tmp.zip");
fs::copyZipToDir(tmp, "sv:/", "sv", t);
@ -265,7 +265,7 @@ static void fldFuncDriveRestore_t(void *a)
static void fldFuncDriveRestore(void *a)
{
drive::gdItem *in = (drive::gdItem *)a;
rfs::RfsItem *in = (rfs::RfsItem *)a;
ui::confirmArgs *conf = ui::confirmArgsCreate(cfg::config["holdOver"], fldFuncDriveRestore_t, NULL, a, ui::getUICString("confirmRestore", 0), in->name.c_str());
ui::confirm(conf);
}
@ -315,21 +315,21 @@ void ui::fldPopulateMenu()
fldMenu->optAddButtonEvent(0, HidNpadButton_A, fs::createNewBackup, NULL);
unsigned fldInd = 1;
if(fs::gDrive)
if(fs::rfs)
{
if(!fs::gDrive->dirExists(t->title, fs::jksvDriveID))
fs::gDrive->createDir(t->title, fs::jksvDriveID);
if(!fs::rfs->dirExists(t->title, fs::rfsRootID))
fs::rfs->createDir(t->title, fs::rfsRootID);
driveParent = fs::gDrive->getDirID(t->title, fs::jksvDriveID);
driveParent = fs::rfs->getDirID(t->title, fs::rfsRootID);
driveFldList = fs::rfs->getListWithParent(driveParent);
fs::gDrive->getListWithParent(driveParent, driveFldList);
for(unsigned i = 0; i < driveFldList.size(); i++, fldInd++)
{
fldMenu->addOpt(NULL, "[GD] " + driveFldList[i]->name);
fldMenu->addOpt(NULL, "[R] " + driveFldList[i].name);
fldMenu->optAddButtonEvent(fldInd, HidNpadButton_A, fldFuncDownload, driveFldList[i]);
fldMenu->optAddButtonEvent(fldInd, HidNpadButton_X, fldFuncDriveDelete, driveFldList[i]);
fldMenu->optAddButtonEvent(fldInd, HidNpadButton_Y, fldFuncDriveRestore, driveFldList[i]);
fldMenu->optAddButtonEvent(fldInd, HidNpadButton_A, fldFuncDownload, &driveFldList[i]);
fldMenu->optAddButtonEvent(fldInd, HidNpadButton_X, fldFuncDriveDelete, &driveFldList[i]);
fldMenu->optAddButtonEvent(fldInd, HidNpadButton_Y, fldFuncDriveRestore, &driveFldList[i]);
}
}
@ -362,17 +362,17 @@ void ui::fldRefreshMenu()
fldMenu->optAddButtonEvent(0, HidNpadButton_A, fs::createNewBackup, NULL);
unsigned fldInd = 1;
if(fs::gDrive)
if(fs::rfs)
{
fs::gDrive->getListWithParent(driveParent, driveFldList);
driveFldList = fs::rfs->getListWithParent(driveParent);
for(unsigned i = 0; i < driveFldList.size(); i++, fldInd++)
{
fldMenu->addOpt(NULL, "[GD] " + driveFldList[i]->name);
fldMenu->addOpt(NULL, "[R] " + driveFldList[i].name);
fldMenu->optAddButtonEvent(fldInd, HidNpadButton_A, fldFuncDownload, driveFldList[i]);
fldMenu->optAddButtonEvent(fldInd, HidNpadButton_X, fldFuncDriveDelete, driveFldList[i]);
fldMenu->optAddButtonEvent(fldInd, HidNpadButton_Y, fldFuncDriveRestore, driveFldList[i]);
fldMenu->optAddButtonEvent(fldInd, HidNpadButton_A, fldFuncDownload, &driveFldList[i]);
fldMenu->optAddButtonEvent(fldInd, HidNpadButton_X, fldFuncDriveDelete, &driveFldList[i]);
fldMenu->optAddButtonEvent(fldInd, HidNpadButton_Y, fldFuncDriveRestore, &driveFldList[i]);
}
}

View File

@ -203,7 +203,7 @@ void ui::initStrings()
addUIString("settingsMenu", 18, "Enable Trash Bin: ");
addUIString("settingsMenu", 19, "Title Sorting Type: ");
addUIString("settingsMenu", 20, "Animation Scale: ");
addUIString("settingsMenu", 21, "Auto-upload to Drive: ");
addUIString("settingsMenu", 21, "Auto-upload to Drive/Webdav: ");
//Main menu
addUIString("mainMenuSettings", 0, "Settings");
@ -288,6 +288,8 @@ void ui::initStrings()
addUIString("popDriveStarted", 0, "Google Drive started successfully.");
addUIString("popDriveFailed", 0, "Failed to start Google Drive.");
addUIString("popDriveNotActive", 0, "Google Drive is not available");
addUIString("popWebdavStarted", 0, "Webdav started successfully.");
addUIString("popWebdavFailed", 0, "Failed to start Webdav.");
//Keyboard hints
addUIString("swkbdEnterName", 0, "Enter a new name");

314
src/webdav.cpp Normal file
View File

@ -0,0 +1,314 @@
#include "webdav.h"
#include "fs.h"
#include "tinyxml2.h"
rfs::WebDav::WebDav(const std::string& origin, const std::string& username, const std::string& password)
: origin(origin), username(username), password(password)
{
curl = curl_easy_init();
if (curl) {
if (!username.empty())
curl_easy_setopt(curl, CURLOPT_USERNAME, username.c_str());
if (!password.empty())
curl_easy_setopt(curl, CURLOPT_PASSWORD, password.c_str());
}
}
rfs::WebDav::~WebDav() {
if (curl) {
curl_easy_cleanup(curl);
}
}
bool rfs::WebDav::resourceExists(const std::string& id) {
CURL* local_curl = curl_easy_duphandle(curl);
// we expect id to be properly escaped and starting with a /
std::string fullUrl = origin + id;
struct curl_slist *headers = NULL;
headers = curl_slist_append(headers, "Depth: 0");
curl_easy_setopt(local_curl, CURLOPT_URL, fullUrl.c_str());
curl_easy_setopt(local_curl, CURLOPT_CUSTOMREQUEST, "PROPFIND");
curl_easy_setopt(local_curl, CURLOPT_HTTPHEADER, headers);
curl_easy_setopt(local_curl, CURLOPT_NOBODY, 1L); // do not include the response body
CURLcode res = curl_easy_perform(local_curl);
curl_slist_free_all(headers); // free the custom headers
bool ret = false;
if(res == CURLE_OK) {
long response_code;
curl_easy_getinfo(local_curl, CURLINFO_RESPONSE_CODE, &response_code);
if(response_code == 207) { // 207 Multi-Status is a successful response for PROPFIND
ret = true;
}
} else {
fs::logWrite("WebDav: directory exists failed: %s\n", curl_easy_strerror(res));
}
curl_easy_cleanup(local_curl);
return ret;
}
// parent ID can never be empty
std::string rfs::WebDav::appendResourceToParentId(const std::string& resourceName, const std::string& parentId, bool isDir) {
char *escaped = curl_easy_escape(curl, resourceName.c_str(), 0);
// we always expect parent to be properly URL encoded.
std::string ret = parentId + std::string(escaped) + (isDir ? "/" : "");
curl_free(escaped);
return ret;
}
bool rfs::WebDav::createDir(const std::string& dirName, const std::string& parentId) {
CURL* local_curl = curl_easy_duphandle(curl);
std::string urlPath = appendResourceToParentId(dirName, parentId, true);
std::string fullUrl = origin + urlPath;
fs::logWrite("WebDav: Create directory at %s\n", fullUrl.c_str());
curl_easy_setopt(local_curl, CURLOPT_URL, fullUrl.c_str());
curl_easy_setopt(local_curl, CURLOPT_CUSTOMREQUEST, "MKCOL");
CURLcode res = curl_easy_perform(local_curl);
if(res != CURLE_OK) {
fs::logWrite("WebDav: directory creation failed: %s\n", curl_easy_strerror(res));
}
bool ret = res == CURLE_OK;
curl_easy_cleanup(local_curl);
return ret;
}
// we always expect parent to be properly URL encoded.
void rfs::WebDav::uploadFile(const std::string& filename, const std::string& parentId, curlFuncs::curlUpArgs *_upload) {
std::string fileId = appendResourceToParentId(filename, parentId, false);
updateFile(fileId, _upload);
}
void rfs::WebDav::updateFile(const std::string& _fileID, curlFuncs::curlUpArgs *_upload) {
// for webdav, same as upload
CURL* local_curl = curl_easy_duphandle(curl);
std::string fullUrl = origin + _fileID;
curl_easy_setopt(local_curl, CURLOPT_URL, fullUrl.c_str());
curl_easy_setopt(local_curl, CURLOPT_UPLOAD, 1L); // implicit PUT
curl_easy_setopt(local_curl, CURLOPT_READFUNCTION, curlFuncs::readDataFile);
curl_easy_setopt(local_curl, CURLOPT_READDATA, _upload);
curl_easy_setopt(local_curl, CURLOPT_UPLOAD_BUFFERSIZE, UPLOAD_BUFFER_SIZE);
curl_easy_setopt(local_curl, CURLOPT_UPLOAD, 1);
CURLcode res = curl_easy_perform(local_curl);
if(res != CURLE_OK) {
fs::logWrite("WebDav: file upload failed: %s\n", curl_easy_strerror(res));
}
curl_easy_cleanup(local_curl); // Clean up the CURL handle
}
void rfs::WebDav::downloadFile(const std::string& _fileID, curlFuncs::curlDlArgs *_download) {
//Downloading is threaded because it's too slow otherwise
dlWriteThreadStruct dlWrite;
dlWrite.cfa = _download;
Thread writeThread;
threadCreate(&writeThread, writeThread_t, &dlWrite, NULL, 0x8000, 0x2B, 2);
CURL* local_curl = curl_easy_duphandle(curl);
std::string fullUrl = origin + _fileID;
curl_easy_setopt(local_curl, CURLOPT_URL, fullUrl.c_str());
curl_easy_setopt(local_curl, CURLOPT_WRITEFUNCTION, writeDataBufferThreaded);
curl_easy_setopt(local_curl, CURLOPT_WRITEDATA, &dlWrite);
threadStart(&writeThread);
CURLcode res = curl_easy_perform(local_curl);
// Copied from gd.cpp implementation.
// TODO: Not sure how a thread helps if this parent waits here.
threadWaitForExit(&writeThread);
threadClose(&writeThread);
if(res != CURLE_OK) {
fs::logWrite("WebDav: file download failed: %s\n", curl_easy_strerror(res));
}
curl_easy_cleanup(local_curl);
}
void rfs::WebDav::deleteFile(const std::string& _fileID) {
CURL* local_curl = curl_easy_duphandle(curl);
std::string fullUrl = origin + _fileID;
curl_easy_setopt(local_curl, CURLOPT_URL, fullUrl.c_str());
curl_easy_setopt(local_curl, CURLOPT_CUSTOMREQUEST, "DELETE");
CURLcode res = curl_easy_perform(local_curl);
if(res != CURLE_OK) {
fs::logWrite("WebDav: file deletion failed: %s\n", curl_easy_strerror(res));
}
curl_easy_cleanup(local_curl);
}
bool rfs::WebDav::dirExists(const std::string& dirName, const std::string& parentId) {
std::string urlPath = getDirID(dirName, parentId);
return resourceExists(urlPath);
}
bool rfs::WebDav::fileExists(const std::string& filename, const std::string& parentId) {
std::string urlPath = appendResourceToParentId(filename, parentId, false);
return resourceExists(urlPath);
}
std::string rfs::WebDav::getFileID(const std::string& filename, const std::string& parentId) {
return appendResourceToParentId(filename, parentId, false);
}
std::string rfs::WebDav::getDirID(const std::string& dirName, const std::string& parentId) {
return appendResourceToParentId(dirName, parentId, true);
}
std::vector<rfs::RfsItem> rfs::WebDav::getListWithParent(const std::string& _parentId) {
std::vector<rfs::RfsItem> list;
CURL* local_curl = curl_easy_duphandle(curl);
// we expect _resource to be properly escaped
std::string fullUrl = origin + _parentId;
struct curl_slist *headers = NULL;
headers = curl_slist_append(headers, "Depth: 1");
std::string responseString;
curl_easy_setopt(local_curl, CURLOPT_URL, fullUrl.c_str());
curl_easy_setopt(local_curl, CURLOPT_CUSTOMREQUEST, "PROPFIND");
curl_easy_setopt(local_curl, CURLOPT_HTTPHEADER, headers);
curl_easy_setopt(local_curl, CURLOPT_WRITEFUNCTION, curlFuncs::writeDataString);
curl_easy_setopt(local_curl, CURLOPT_WRITEDATA, &responseString);
CURLcode res = curl_easy_perform(local_curl);
if(res == CURLE_OK) {
long response_code;
curl_easy_getinfo(local_curl, CURLINFO_RESPONSE_CODE, &response_code);
if(response_code == 207) { // 207 Multi-Status is a successful response for PROPFIND
fs::logWrite("WebDav: Response from WebDav. Parsing.\n");
std::vector<rfs::RfsItem> items = parseXMLResponse(responseString);
// insert into array
// TODO: Filter for zip?
list.insert(list.end(), items.begin(), items.end());
}
} else {
fs::logWrite("WebDav: directory listing failed: %s\n", curl_easy_strerror(res));
}
curl_slist_free_all(headers); // free the custom headers
curl_easy_cleanup(local_curl);
return list;
}
// Helper
std::string rfs::WebDav::getNamespacePrefix(tinyxml2::XMLElement* root, const std::string& nsURI) {
for(const tinyxml2::XMLAttribute* attr = root->FirstAttribute(); attr; attr = attr->Next()) {
std::string name = attr->Name();
std::string value = attr->Value();
if(value == nsURI) {
auto pos = name.find(':');
if(pos != std::string::npos) {
return name.substr(pos + 1);
} else {
return ""; // Default namespace (no prefix)
}
}
}
return ""; // No namespace found
}
std::vector<rfs::RfsItem> rfs::WebDav::parseXMLResponse(const std::string& xml) {
std::vector<RfsItem> items;
tinyxml2::XMLDocument doc;
if(doc.Parse(xml.c_str()) != tinyxml2::XML_SUCCESS) {
fs::logWrite("WebDav: Failed to parse XML from Server\n");
return items;
}
// Get the root element
tinyxml2::XMLElement *root = doc.RootElement();
std::string nsPrefix = getNamespacePrefix(root, "DAV:");
nsPrefix = !nsPrefix.empty() ? nsPrefix + ":" : nsPrefix; // Append colon if non-empty
fs::logWrite("WebDav: Parsing response, using prefix: %s\n", nsPrefix.c_str());
// Loop through the responses
tinyxml2::XMLElement* responseElem = root->FirstChildElement((nsPrefix + "response").c_str());
std::string parentId;
while (responseElem) {
RfsItem item;
item.size = 0;
tinyxml2::XMLElement* hrefElem = responseElem->FirstChildElement((nsPrefix + "href").c_str());
if (hrefElem) {
item.id = hrefElem->GetText();
item.parent = parentId;
}
tinyxml2::XMLElement* propstatElem = responseElem->FirstChildElement((nsPrefix + "propstat").c_str());
if (propstatElem) {
tinyxml2::XMLElement* propElem = propstatElem->FirstChildElement((nsPrefix + "prop").c_str());
if (propElem) {
tinyxml2::XMLElement* displaynameElem = propElem->FirstChildElement((nsPrefix + "displayname").c_str());
if (displaynameElem) {
item.name = displaynameElem->GetText();
}
tinyxml2::XMLElement* resourcetypeElem = propElem->FirstChildElement((nsPrefix + "resourcetype").c_str());
if (resourcetypeElem) {
item.isDir = resourcetypeElem->FirstChildElement((nsPrefix + "collection").c_str()) != nullptr;
}
tinyxml2::XMLElement* contentLengthElem = propElem->FirstChildElement((nsPrefix + "getcontentlength").c_str());
if (contentLengthElem) {
const char* sizeStr = contentLengthElem->GetText();
if (sizeStr) {
item.size = std::stoi(sizeStr);
}
}
}
}
responseElem = responseElem->NextSiblingElement((nsPrefix + "response").c_str());
// first Item is always the parent.
if (parentId.empty()) {
parentId = hrefElem->GetText();
continue; // do not push parent to list (we are only interested in the children)
}
// fs::logWrite("WebDav: Adding %s with\n", item.name.c_str());
// fs::logWrite("WebDav: ID: %s\n", item.id.c_str());
// fs::logWrite("WebDav: ParentID: %s\n", item.parent.c_str());
// fs::logWrite("WebDav: Directory?: %s\n", item.isDir ? "yes" : "no");
// fs::logWrite("WebDav: Size: %u\n", item.size);
items.push_back(item);
}
return items;
}