Compare commits

...

16 Commits

Author SHA1 Message Date
11c1029e26 refactor
Some checks reported errors
BlockStorage/repertory/pipeline/pr-master Something is wrong with the build of this commit
BlockStorage/repertory/pipeline/head Build started...
2025-02-11 11:27:23 -06:00
ab8b74eec3 refactor 2025-02-11 11:26:00 -06:00
5397d3d780 refactor 2025-02-11 11:24:52 -06:00
3079ad977e updated README.md 2025-02-11 10:51:08 -06:00
92d914a2d5 updated README.md 2025-02-11 10:49:36 -06:00
6fd9d15c81 updated README.md 2025-02-11 09:09:08 -06:00
9fdc0c172d updated README.md 2025-02-11 08:53:29 -06:00
051e9e1130 updated README.md 2025-02-11 08:47:39 -06:00
8d2c6b77b8 updated README.md 2025-02-11 08:46:49 -06:00
db46aa43ac updated README.md 2025-02-11 08:44:16 -06:00
aac056237d refactor 2025-02-11 08:43:29 -06:00
2a7013331f updated README.md 2025-02-11 08:37:42 -06:00
cf74c43432 updated README.md 2025-02-11 08:36:53 -06:00
759d464c48 updated README.md 2025-02-11 08:35:11 -06:00
7a4b08d69e updated CHANGELOG.md 2025-02-11 07:59:34 -06:00
23f2f35170 updated README.md 2025-02-11 07:48:51 -06:00
6 changed files with 82 additions and 67 deletions

View File

@ -128,6 +128,7 @@ libcurl
libdsm
libevent
libexample
libexpat
libfuse3
libgmock
libgtest

View File

@ -4,10 +4,6 @@
### Issues
* ~~\#12 \[Unit Test\] Complete all providers unit tests~~
* ~~\#20 Add support to evict open files~~
* ~~\#21 \[Unit Test \] Complete WinFSP unit tests~~
* ~~\#22 \[Unit Test\] Complete FUSE unit tests~~
* \#28 \[bug\] Address slow directory responses in S3 mounts for deeply nested directories
* \#29 \[bug\] S3 error responses are not being logged
* \#30 \[bug\] Sia provider error responses are not logged

129
README.md
View File

@ -1,23 +1,22 @@
# Repertory
Repertory allows you to mount AWS S3 and Sia via FUSE on Linux ~~/OS X~~ or via WinFSP
Repertory allows you to mount S3 and Sia via FUSE on Linux or via WinFSP
on Windows.
## Details and Features
* Optimized for [Plex Media Server](https://www.plex.tv/)
* Single application to mount AWS S3 and/or Sia
* Remote mounting of Repertory instances on Linux ~~, OS X~~ and Windows
* Securely share your mounts over TCP/IP (`XChaCha20-Poly1305` stream cipher) with other systems on your network or the internet.
* Cross-platform support (Linux 64-bit, Linux arm64/aarch64, ~~OS X,~~ Windows 64-bit)
* Single application to mount S3 and/or Sia
* Remote mounting of Repertory instances on Linux and Windows
* Securely share your mounts over TCP/IP via `XChaCha20-Poly1305` with other systems on your network or over the internet.
* Cross-platform support (Linux 64-bit, Linux arm64/aarch64, Windows 64-bit)
* Optionally encrypt file names and file data via `XChaCha20-Poly1305` in S3 mounts
## Minimum Requirements
* [Sia renterd](https://github.com/SiaFoundation/renterd/releases) v0.4.0+ for Sia support
* Only 64-bit operating systems are supported
* By default, Linux requires `fusermount3`; otherwise, `repertory` must be manually compiled with `libfuse2` support
* ~~OS X requires the following dependency to be installed:~~
* ~~[FUSE for macOS v4.5.0](https://github.com/osxfuse/osxfuse/releases/download/macfuse-4.5.0/macfuse-4.5.0.dmg)~~
* Windows requires the following dependencies to be installed:
* [WinFSP 2023](https://github.com/winfsp/winfsp/releases/download/v2.0/winfsp-2.0.23075.msi)
@ -25,40 +24,10 @@ on Windows.
* Linux `arm64/aarch64`
* Linux `amd64`
* ~~OS X Mojave and above~~
* Windows 64-bit 10, 11
## Usage
### Notable Options
* `-dc`
* Display mount configuration.
* For Sia, `--name` is optional
* For S3, the `-s3` option is required along with `--name`
* `--help`
* Display all mount utility options.
* `--name, -na [name]`
* The `--name` option can be set to any valid value allowed as a file name for your filesystem.
* For Sia, the bucket name will be set to the same value if it is empty in the configuration file.
* If the `--name` option is not specified, `default` will be used.
* For S3, the `--name` option is required and does not affect the bucket name.
* `-set SiaConfig.Bucket`
* Sets the Sia bucket name to mount.
* Can be used in combination with `--name` to target a unique configuration.
* `-set S3Config.Bucket`
* Sets the S3 bucket name to mount.
* Must be used in combination with `--name` to target a unique configuration.
* Must be used in combination with `-s3`.
### Data Directories
* Linux
* `~/.local/repertory2`
* Windows
* `%LOCALAPPDATA%\repertory2`
* i.e. `C:\\Users\\Tom\\AppData\\Local\\repertory2`
### Sia
* Initial Configuration
@ -72,45 +41,82 @@ on Windows.
* For all other configurations:
* `repertory --name '<my config name>' -set HostConfig.ApiPassword '<my password>'`
* `repertory --name '<my config name>' -set SiaConfig.Bucket '<my bucket name>'`
* To verify all configuration options:
* `repertory -dc` or `repertory --name '<my config name>' -dc`
* To verify/view all configuration options:
* `repertory -dc`
* `repertory --name '<my config nam>' -dc`
* Mounting on Linux:
* `repertory /mnt/location`
* `repertory --name default /mnt/location`
* `repertory --name '<my config nam>' /mnt/location`
* Example:
* `repertory --name default /mnt/location`
* Mounting on Windows:
* `repertory t:`
* `repertory --name default t:`
* `repertory --name '<my config nam>' t:`
* Example:
* `repertory --name default t:`
### S3
* Initial Configuration
* S3 steps:
* Set the appropriate bucket name:
* `repertory --name '<my config name>' -set S3Config.Bucket '<my bucket name>'`
* Set the appropriate base URL:
* `repertory --name '<my config name>' -set S3Config.URL '<my url>'`
* i.e. `repertory --name minio -set S3Config.URL 'http://localhost:9000'`
* `repertory -s3 --name '<my config name>' -set S3Config.URL '<my url>'`
* Example:
* `repertory -s3 --name minio -set S3Config.URL 'http://localhost:9000'`
* Set the appropriate bucket name:
* `repertory -s3 --name '<my config name>' -set S3Config.Bucket '<my bucket name>'`
* Set the appropriate access key:
* `repertory --name '<my config name>' -set S3Config.AccessKey '<my access key>'`
* `repertory -s3 --name '<my config name>' -set S3Config.AccessKey '<my access key>'`
* Set the appropriate secret key:
* `repertory --name '<my config name>' -set S3Config.SecretKey '<my secret key>'`
* `repertory -s3 --name '<my config name>' -set S3Config.SecretKey '<my secret key>'`
* For Sia and most local S3 gateway instances, enable path style URL's:
* `repertory --name '<my config name>' -set S3Config.UsePathStyle true`
* Optionally, set the appropriate region. Default is set to `any`:
* `repertory --name '<my config name>' -set S3Config.Region '<my region>'`
* Optionally, to enable encrypted file names and file data, set a strong, random encryption token:
* `repertory --name '<my config name>' -set S3Config.EncryptionToken '<my strong password>'`
* To verify all configuration options:
* `repertory -dc` or `repertory --name '<my config nam>' -dc`
* `repertory -s3 --name '<my config name>' -set S3Config.UsePathStyle true`
* Optional steps:
* Set the appropriate region. Default is set to `any`:
* `repertory -s3 --name '<my config name>' -set S3Config.Region '<my region>'`
* Enable encrypted file names and file data. Set a strong, random encryption token and be sure to store it in a secure backup location:
* `repertory -s3 --name '<my config name>' -set S3Config.EncryptionToken '<my strong password>'`
* To verify/view all configuration options:
* `repertory -s3 --name '<my config name>' -dc`
* Mounting on Linux:
* `repertory -s3 --name '<my config name>' /mnt/location`
* Example:
* `repertory -s3 --name minio /mnt/location`
* Mounting on Windows:
* `repertory -s3 --name '<my config name>' t:`
* Example:
* `repertory -s3 --name minio t:`
### Notable Options
* `--help`
* Display all mount utility options
* `--name, -na [name]`
* The `--name` option can be set to any valid value allowed as a file name for your filesystem.
* For Sia, the bucket name will be set to the same value if it is empty in the configuration file.
* If the `--name` option is not specified, `default` will be used.
* For S3, the `--name` option is required and does not affect the bucket name.
* `-dc`
* Display mount configuration
* For Sia, `--name` is optional
* For S3, the `-s3` option is required along with `--name`
### Data Directories
* Linux
* `~/.local/repertory2`
* Windows
* `%LOCALAPPDATA%\repertory2`
* Example:
* `C:\Users\Tom\AppData\Local\repertory2`
* IMPORTANT:
* It is highly recommended to exclude this folder from any anti-virus/anti-malware applications as severe performance issues may arise.
* Excluding the mounted drive letter is also highly recommended.
## Remote Mounting
`Repertory` allows local mounts to be shared with other computers on your network. This option is referred to as remote mounting.
Instructions TBD.
`Repertory` allows local mounts to be shared with other computers on your network.
This option is referred to as remote mounting. Instructions TBD.
## Compiling
@ -139,17 +145,26 @@ Instructions TBD.
## Credits
* [binutils](https://www.gnu.org/software/binutils/)
* [boost c++ libraries](https://www.boost.org/)
* [cpp-httplib](https://github.com/yhirose/cpp-httplib)
* [curl](https://curl.haxx.se/)
* ~~[FUSE for macOS](https://osxfuse.github.io/)~~
* [docker](https://www.docker.com/)
* [Google Test](https://github.com/google/googletest)
* [ICU](https://icu.unicode.org/)
* [JSON for Modern C++](https://github.com/nlohmann/json)
* [libexpat](https://github.com/libexpat/libexpat)
* [libfuse](https://github.com/libfuse/libfuse)
* [libsodium](https://doc.libsodium.org/)
* [mingw-w64](https://www.mingw-w64.org/)
* [MSYS2](https://www.msys2.org)
* [OpenSSL](https://www.openssl.org/)
* [pkg-config](https://www.freedesktop.org/wiki/Software/pkg-config/)
* [pugixml](https://pugixml.org/)
* [RocksDB](https://rocksdb.org)
* [ScPrime](https://scpri.me/)
* [Sia Decentralized Cloud Storage](https://sia.tech/)
* [spdlog](https://github.com/gabime/spdlog)
* [SQLite](https://www.sqlite.org)
* [stduuid](https://github.com/mariusbancila/stduuid)
* [Storj](https://storj.io/)

View File

@ -43,7 +43,8 @@ constexpr const auto max_orphaned_file_retention_days{std::uint16_t(31U)};
constexpr const auto max_ring_buffer_file_size{std::uint16_t(1024U)};
constexpr const auto max_s3_object_name_length{1024U};
constexpr const auto min_cache_size_bytes{
std::uint64_t(100UL * 1024UL * 1024UL)};
std::uint64_t(100UL * 1024UL * 1024UL),
};
constexpr const auto min_download_timeout_secs{std::uint8_t(5U)};
constexpr const auto min_online_check_retry_secs{std::uint16_t(15U)};
constexpr const auto min_orphaned_file_retention_days{std::uint16_t(1U)};

View File

@ -27,7 +27,7 @@
namespace repertory::cli::actions {
template <typename drive> inline void version(std::vector<const char *> args) {
std::cout << "Repertory core version: " << project_get_version() << std::endl;
std::cout << "Repertory Git revision: " << project_get_git_rev() << std::endl;
std::cout << "Repertory git revision: " << project_get_git_rev() << std::endl;
drive::display_version_information(args);
}
} // namespace repertory::cli::actions

View File

@ -41,7 +41,7 @@ using repertory_drive = repertory::winfsp_drive;
using remote_client = repertory::remote_winfsp::remote_client;
using remote_drive = repertory::remote_winfsp::remote_winfsp_drive;
using remote_instance = repertory::remote_winfsp::i_remote_instance;
#else
#else // !defined(_WIN32)
#include "drives/fuse/fuse_drive.hpp"
#include "drives/fuse/remotefuse/remote_client.hpp"
#include "drives/fuse/remotefuse/remote_fuse_drive.hpp"
@ -50,7 +50,7 @@ using repertory_drive = repertory::fuse_drive;
using remote_client = repertory::remote_fuse::remote_client;
using remote_drive = repertory::remote_fuse::remote_fuse_drive;
using remote_instance = repertory::remote_fuse::i_remote_instance;
#endif
#endif // defined(_WIN32)
namespace repertory::cli::actions {
[[nodiscard]] inline auto
@ -77,7 +77,8 @@ mount(std::vector<const char *> args, std::string data_directory,
config.set_remote_config(cfg);
} else if (prov == provider_type::sia &&
config.get_sia_config().bucket.empty()) {
config.set_value_by_name("SiaConfig.Bucket", unique_id);
[[maybe_unused]] auto bucket =
config.set_value_by_name("SiaConfig.Bucket", unique_id);
}
std::cout << "Generated " << app_config::get_provider_display_name(prov)
@ -157,7 +158,8 @@ mount(std::vector<const char *> args, std::string data_directory,
} else {
if (prov == provider_type::sia &&
config.get_sia_config().bucket.empty()) {
config.set_value_by_name("SiaConfig.Bucket", unique_id);
[[maybe_unused]] auto bucket =
config.set_value_by_name("SiaConfig.Bucket", unique_id);
}
try {