v2.0.3-rc (#32)

# Changelog

## v2.0.3-rc

### Issues

* \#28 \[bug\] Address slow directory responses in S3 mounts for deeply nested directories
* \#29 \[bug\] S3 error responses are not being logged
* \#30 \[bug\] Sia provider error responses are not logged
* \#31 \[bug\] S3 provider should limit max key size to 1024

### Changes from v2.0.2-rc

* Always use direct for read-only providers
* Fixed externally removed files not being processed during cleanup
* Fixed http headers not being added for requests
* Fixed incorrect `stat` values for remote mounts
* Fixed invalid directory nullptr error on remote mounts
* Fixed memory leak in event system
* Refactored application shutdown
* Refactored event system
* Updated build system to Alpine 3.21.0
* Updated build system to MinGW-w64 12.0.0
* Updated copyright to 2018-2025

Reviewed-on: #32

.cspell/words.txt

@@ -128,6 +128,7 @@ libcurl
 libdsm
 libevent
 libexample
+libexpat
 libfuse3
 libgmock
 libgtest
@@ -140,10 +141,12 @@ libuuid
 libuuid_include_dirs
 libvlc
 linkflags
+localappdata
 lptr
 lpwstr
 markdownlint
 mbig
+minio
 msvc
 msvcr120
 msys2
@@ -200,6 +203,7 @@ stduuid_project
 strequal
 ularge_integer
 uring
+url
 userenv
 utimens_impl
 utimensat

CHANGELOG.md

@@ -4,10 +4,6 @@
 
 ### Issues
 
-* ~~\#12 \[Unit Test\] Complete all providers unit tests~~
-* ~~\#20 Add support to evict open files~~
-* ~~\#21 \[Unit Test \] Complete WinFSP unit tests~~
-* ~~\#22 \[Unit Test\] Complete FUSE unit tests~~
 * \#28 \[bug\] Address slow directory responses in S3 mounts for deeply nested directories
 * \#29 \[bug\] S3 error responses are not being logged
 * \#30 \[bug\] Sia provider error responses are not logged
@@ -16,6 +12,7 @@
 ### Changes from v2.0.2-rc
 
 * Always use direct for read-only providers
+* Fixed externally removed files not being processed during cleanup
 * Fixed http headers not being added for requests
 * Fixed incorrect `stat` values for remote mounts
 * Fixed invalid directory nullptr error on remote mounts

CMakeLists.txt

@@ -3,6 +3,10 @@ cmake_minimum_required(VERSION 3.27)
 cmake_policy(SET CMP0135 NEW)
 cmake_policy(SET CMP0144 NEW)
 
+if (NOT PROJECT_INTERFACE) 
+  message(FATAL_ERROR "Project must be compiled via 'make_win32.cmd'/'make_win32.sh' or 'make_unix.sh' build scripts. Invoking 'cmake' directly is not supported.")
+endif()
+
 project(${PROJECT_NAME}
   DESCRIPTION ${PROJECT_DESC}
   HOMEPAGE_URL ${PROJECT_URL}
@@ -158,6 +162,7 @@ endif()
       -DPROJECT_FUSE=${PROJECT_FUSE}
       -DPROJECT_FUSE_INCLUDE_DIRS=${PROJECT_FUSE_INCLUDE_DIRS}
       -DPROJECT_GIT_REV=${PROJECT_GIT_REV}
+      -DPROJECT_INTERFACE=1
       -DPROJECT_IS_ALPINE=${PROJECT_IS_ALPINE}
       -DPROJECT_IS_ARM64=${PROJECT_IS_ARM64}
       -DPROJECT_IS_MINGW=${PROJECT_IS_MINGW}

README.md

@@ -1,23 +1,22 @@
 # Repertory
 
-Repertory allows you to mount AWS S3 and Sia via FUSE on Linux ~~/OS X~~ or via WinFSP
+Repertory allows you to mount S3 and Sia via FUSE on Linux or via WinFSP
 on Windows.
 
 ## Details and Features
 
 * Optimized for [Plex Media Server](https://www.plex.tv/)
-* Single application to mount AWS S3 and/or Sia
+* Single application to mount S3 and/or Sia
-* Remote mounting of Repertory instances on Linux ~~, OS X~~ and Windows
+* Remote mounting of Repertory instances on Linux and Windows
-  * Securely share your mounts over TCP/IP (`XChaCha20-Poly1305` stream cipher)
+  * Securely share your mounts over TCP/IP via `XChaCha20-Poly1305` with other systems on your network or over the internet.
-* Cross-platform support (Linux 64-bit, Linux arm64/aarch64, ~~OS X,~~ Windows 64-bit)
+* Cross-platform support (Linux 64-bit, Linux arm64/aarch64, Windows 64-bit)
+* Optionally encrypt file names and file data via `XChaCha20-Poly1305` in S3 mounts
 
 ## Minimum Requirements
 
 * [Sia renterd](https://github.com/SiaFoundation/renterd/releases) v0.4.0+ for Sia support
 * Only 64-bit operating systems are supported
   * By default, Linux requires `fusermount3`; otherwise, `repertory` must be manually compiled with `libfuse2` support
-  * ~~OS X requires the following dependency to be installed:~~
-    * ~~[FUSE for macOS v4.5.0](https://github.com/osxfuse/osxfuse/releases/download/macfuse-4.5.0/macfuse-4.5.0.dmg)~~
   * Windows requires the following dependencies to be installed:
     * [WinFSP 2023](https://github.com/winfsp/winfsp/releases/download/v2.0/winfsp-2.0.23075.msi)
 
@@ -25,47 +24,103 @@ on Windows.
 
 * Linux `arm64/aarch64`
 * Linux `amd64`
-* ~~OS X Mojave and above~~
 * Windows 64-bit 10, 11
 
 ## Usage
 
+### Sia
+
+* Initial Configuration
+  * Sia steps:
+    * Set the appropriate bucket name and `renterd` API password in `repertory` configuration:
+      * To use `default` as the bucket name and configuration name:
+        * `repertory -set HostConfig.ApiPassword '<my password>'`
+      * To use a different bucket name with `default` as the configuration name:
+        * `repertory -set HostConfig.ApiPassword '<my password>'`
+        * `repertory -set SiaConfig.Bucket '<my bucket>'`
+      * For all other configurations:
+        * `repertory --name '<my config name>' -set HostConfig.ApiPassword '<my password>'`
+        * `repertory --name '<my config name>' -set SiaConfig.Bucket '<my bucket name>'`
+  * To verify/view all configuration options:
+    * `repertory -dc`
+    * `repertory --name '<my config name>' -dc`
+      * Example:
+        * `repertory --name default -dc`
+* Mounting on Linux:
+  * `repertory /mnt/location`
+  * `repertory --name '<my config name>' /mnt/location`
+    * Example:
+      * `repertory --name default /mnt/location`
+* Mounting on Windows:
+  * `repertory t:`
+  * `repertory --name '<my config name>' t:`
+    * Example:
+      * `repertory --name default t:`
+
+### S3
+
+* Initial Configuration
+  * S3 steps:
+    * Set the appropriate base URL:
+      * `repertory -s3 --name '<my config name>' -set S3Config.URL '<my url>'`
+        * Example:
+          * `repertory -s3 --name minio -set S3Config.URL 'http://localhost:9000'`
+    * Set the appropriate bucket name:
+      * `repertory -s3 --name '<my config name>' -set S3Config.Bucket '<my bucket name>'`
+    * Set the appropriate access key:
+      * `repertory -s3 --name '<my config name>' -set S3Config.AccessKey '<my access key>'`
+    * Set the appropriate secret key:
+      * `repertory -s3 --name '<my config name>' -set S3Config.SecretKey '<my secret key>'`
+    * For Sia and most local S3 gateway instances, enable path style URL's:
+      * `repertory -s3 --name '<my config name>' -set S3Config.UsePathStyle true`
+    * Optional steps:
+      * Set an appropriate region. Default is set to `any`:
+        * `repertory -s3 --name '<my config name>' -set S3Config.Region '<my region>'`
+      * Enable encrypted file names and file data. Set a strong, random encryption token and be sure to store it in a secure backup location:
+        * `repertory -s3 --name '<my config name>' -set S3Config.EncryptionToken '<my strong password>'`
+  * To verify/view all configuration options:
+    * `repertory -s3 --name '<my config name>' -dc`
+      * Example:
+        * `repertory -s3 --name minio -dc`
+* Mounting on Linux:
+  * `repertory -s3 --name '<my config name>' /mnt/location`
+    * Example:
+      * `repertory -s3 --name minio /mnt/location`
+* Mounting on Windows:
+  * `repertory -s3 --name '<my config name>' t:`
+    * Example:
+      * `repertory -s3 --name minio t:`
+
 ### Notable Options
 
-* `-dc`
-  * Display mount configuration.
-  * For Sia, `--name` is optional
-  * For S3, the `-s3` option is required along with `--name`
 * `--help`
-  * Display all mount utility options.
+  * Display all mount utility options
 * `--name, -na [name]`
   * The `--name` option can be set to any valid value allowed as a file name for your filesystem.
   * For Sia, the bucket name will be set to the same value if it is empty in the configuration file.
     * If the `--name` option is not specified, `default` will be used.
   * For S3, the `--name` option is required and does not affect the bucket name.
-* `-set SiaConfig.Bucket`
+* `-dc`
-  * Set Sia bucket name for the mount.
+  * Display mount configuration
-  * Can be used in combination with `--name` to target a unique configuration.
+  * For Sia, `--name` is optional
-* `-set S3Config.Bucket`
+  * For S3, the `-s3` option is required along with `--name`
-  * S3 bucket name for the mount.
-  * Must be used in combination with `--name` to target a unique configuration.
-  * Must be used in combination with `-s3`.
 
-### Sia
+### Data Directories
 
 * Linux
-  * `repertory /mnt/location`
+  * `~/.local/repertory2`
-  * `repertory --name default /mnt/location`
 * Windows
-  * `repertory.exe t:`
+  * `%LOCALAPPDATA%\repertory2`
-  * `repertory.exe --name default t:`
+    * Example:
+      * `C:\Users\Tom\AppData\Local\repertory2`
+    * IMPORTANT:
+      * It is highly recommended to exclude this folder from any anti-virus/anti-malware applications as severe performance issues may arise.
+      * Excluding the mounted drive letter is also highly recommended.
 
-### S3
+## Remote Mounting
 
-* Linux 
+`Repertory` allows local mounts to be shared with other computers on your network.
-  * `repertory --name storj -s3 /mnt/location`
+This option is referred to as remote mounting. Instructions TBD.
-* Windows
-  * `repertory.exe --name storj -s3 t:`
 
 ## Compiling
 
@@ -94,17 +149,26 @@ on Windows.
 
 ## Credits
 
+* [binutils](https://www.gnu.org/software/binutils/)
 * [boost c++ libraries](https://www.boost.org/)
 * [cpp-httplib](https://github.com/yhirose/cpp-httplib)
 * [curl](https://curl.haxx.se/)
-* ~~[FUSE for macOS](https://osxfuse.github.io/)~~
+* [docker](https://www.docker.com/)
 * [Google Test](https://github.com/google/googletest)
+* [ICU](https://icu.unicode.org/)
 * [JSON for Modern C++](https://github.com/nlohmann/json)
+* [libexpat](https://github.com/libexpat/libexpat)
 * [libfuse](https://github.com/libfuse/libfuse)
 * [libsodium](https://doc.libsodium.org/)
+* [mingw-w64](https://www.mingw-w64.org/)
+* [MSYS2](https://www.msys2.org)
 * [OpenSSL](https://www.openssl.org/)
+* [pkg-config](https://www.freedesktop.org/wiki/Software/pkg-config/)
+* [pugixml](https://pugixml.org/)
+* [RocksDB](https://rocksdb.org)
 * [ScPrime](https://scpri.me/)
 * [Sia Decentralized Cloud Storage](https://sia.tech/)
+* [spdlog](https://github.com/gabime/spdlog)
 * [SQLite](https://www.sqlite.org)
 * [stduuid](https://github.com/mariusbancila/stduuid)
 * [Storj](https://storj.io/)

cmake/versions.cmake

@@ -1,15 +1,15 @@
 set(BINUTILS_VERSION 2.43)
-set(BOOST2_MAJOR_VERSION 1)
-set(BOOST2_MINOR_VERSION 76)
-set(BOOST2_PATCH_VERSION 0)
 set(BOOST_MAJOR_VERSION 1)
 set(BOOST_MINOR_VERSION 87)
 set(BOOST_PATCH_VERSION 0)
+set(BOOST2_MAJOR_VERSION 1)
+set(BOOST2_MINOR_VERSION 76)
+set(BOOST2_PATCH_VERSION 0)
 set(CPP_HTTPLIB_VERSION 0.18.1)
-set(CURL2_VERSION 8_11_0)
 set(CURL_VERSION 8.11.0)
-set(EXPAT2_VERSION 2_6_4)
+set(CURL2_VERSION 8_11_0)
 set(EXPAT_VERSION 2.6.4)
+set(EXPAT2_VERSION 2_6_4)
 set(GCC_VERSION 14.2.0)
 set(GTEST_VERSION 1.15.2)
 set(ICU_VERSION 76-1)
@@ -22,7 +22,7 @@ set(PKG_CONFIG_VERSION 0.29.2)
 set(PUGIXML_VERSION 1.14)
 set(ROCKSDB_VERSION 9.7.4)
 set(SPDLOG_VERSION 1.15.0)
-set(SQLITE2_VERSION 3.46.1)
 set(SQLITE_VERSION 3460100)
+set(SQLITE2_VERSION 3.46.1)
 set(STDUUID_VERSION 1.2.3)
 set(ZLIB_VERSION 1.3.1)

repertory/librepertory/include/types/repertory.hpp

@@ -43,7 +43,8 @@ constexpr const auto max_orphaned_file_retention_days{std::uint16_t(31U)};
 constexpr const auto max_ring_buffer_file_size{std::uint16_t(1024U)};
 constexpr const auto max_s3_object_name_length{1024U};
 constexpr const auto min_cache_size_bytes{
-    std::uint64_t(100UL * 1024UL * 1024UL)};
+    std::uint64_t(100UL * 1024UL * 1024UL),
+};
 constexpr const auto min_download_timeout_secs{std::uint8_t(5U)};
 constexpr const auto min_online_check_retry_secs{std::uint16_t(15U)};
 constexpr const auto min_orphaned_file_retention_days{std::uint16_t(1U)};

repertory/librepertory/src/db/impl/rdb_file_db.cpp

@@ -191,6 +191,10 @@ void rdb_file_db::enumerate_item_list(
       list.clear();
     }
   }
+
+  if (not list.empty()) {
+    callback(list);
+  }
 }
 
 auto rdb_file_db::get_api_path(const std::string &source_path,
@@ -203,9 +207,8 @@ auto rdb_file_db::get_api_path(const std::string &source_path,
   });
 }
 
-auto rdb_file_db::get_directory_api_path(const std::string &source_path,
+auto rdb_file_db::get_directory_api_path(
-                                         std::string &api_path) const
+    const std::string &source_path, std::string &api_path) const -> api_error {
-    -> api_error {
   REPERTORY_USES_FUNCTION_NAME();
 
   auto result = perform_action(function_name, [&]() -> rocksdb::Status {
@@ -228,9 +231,8 @@ auto rdb_file_db::get_directory_api_path(const std::string &source_path,
                                              : result;
 }
 
-auto rdb_file_db::get_directory_source_path(const std::string &api_path,
+auto rdb_file_db::get_directory_source_path(
-                                            std::string &source_path) const
+    const std::string &api_path, std::string &source_path) const -> api_error {
-    -> api_error {
   REPERTORY_USES_FUNCTION_NAME();
 
   auto result = perform_action(function_name, [&]() -> rocksdb::Status {
@@ -291,9 +293,8 @@ auto rdb_file_db::get_file_data(const std::string &api_path,
   return result;
 }
 
-auto rdb_file_db::get_file_source_path(const std::string &api_path,
+auto rdb_file_db::get_file_source_path(
-                                       std::string &source_path) const
+    const std::string &api_path, std::string &source_path) const -> api_error {
-    -> api_error {
   REPERTORY_USES_FUNCTION_NAME();
 
   auto result = perform_action(function_name, [&]() -> rocksdb::Status {

repertory/librepertory/src/db/impl/rdb_meta_db.cpp

@@ -81,6 +81,10 @@ void rdb_meta_db::enumerate_api_path_list(
     callback(list);
     list.clear();
   }
+
+  if (not list.empty()) {
+    callback(list);
+  }
 }
 
 auto rdb_meta_db::get_api_path(const std::string &source_path,
@@ -314,10 +318,9 @@ void rdb_meta_db::remove_api_path(const std::string &api_path) {
   }
 }
 
-auto rdb_meta_db::remove_api_path(const std::string &api_path,
+auto rdb_meta_db::remove_api_path(
-                                  const std::string &source_path,
+    const std::string &api_path, const std::string &source_path,
-                                  rocksdb::Transaction *txn)
+    rocksdb::Transaction *txn) -> rocksdb::Status {
-    -> rocksdb::Status {
   auto txn_res = txn->Delete(pinned_family_, api_path);
   if (not txn_res.ok()) {
     return txn_res;

repertory/librepertory/src/db/impl/sqlite_file_db.cpp

@@ -65,9 +65,8 @@ sqlite_file_db::sqlite_file_db(const app_config &cfg) {
 
 sqlite_file_db::~sqlite_file_db() { db_.reset(); }
 
-auto sqlite_file_db::add_directory(const std::string &api_path,
+auto sqlite_file_db::add_directory(
-                                   const std::string &source_path)
+    const std::string &api_path, const std::string &source_path) -> api_error {
-    -> api_error {
   REPERTORY_USES_FUNCTION_NAME();
 
   auto result = utils::db::sqlite::db_insert{*db_, file_table}
@@ -158,6 +157,10 @@ void sqlite_file_db::enumerate_item_list(
       list.clear();
     }
   }
+
+  if (not list.empty()) {
+    callback(list);
+  }
 }
 
 auto sqlite_file_db::get_api_path(const std::string &source_path,
@@ -179,9 +182,8 @@ auto sqlite_file_db::get_api_path(const std::string &source_path,
   return api_error::item_not_found;
 }
 
-auto sqlite_file_db::get_directory_api_path(const std::string &source_path,
+auto sqlite_file_db::get_directory_api_path(
-                                            std::string &api_path) const
+    const std::string &source_path, std::string &api_path) const -> api_error {
-    -> api_error {
   auto result = utils::db::sqlite::db_select{*db_, file_table}
                     .column("api_path")
                     .where("source_path")
@@ -202,9 +204,8 @@ auto sqlite_file_db::get_directory_api_path(const std::string &source_path,
   return api_error::directory_not_found;
 }
 
-auto sqlite_file_db::get_directory_source_path(const std::string &api_path,
+auto sqlite_file_db::get_directory_source_path(
-                                               std::string &source_path) const
+    const std::string &api_path, std::string &source_path) const -> api_error {
-    -> api_error {
   auto result = utils::db::sqlite::db_select{*db_, file_table}
                     .column("source_path")
                     .where("api_path")
@@ -225,9 +226,8 @@ auto sqlite_file_db::get_directory_source_path(const std::string &api_path,
   return api_error::directory_not_found;
 }
 
-auto sqlite_file_db::get_file_api_path(const std::string &source_path,
+auto sqlite_file_db::get_file_api_path(
-                                       std::string &api_path) const
+    const std::string &source_path, std::string &api_path) const -> api_error {
-    -> api_error {
   auto result = utils::db::sqlite::db_select{*db_, file_table}
                     .column("api_path")
                     .where("source_path")
@@ -286,9 +286,8 @@ auto sqlite_file_db::get_file_data(const std::string &api_path,
   return api_error::item_not_found;
 }
 
-auto sqlite_file_db::get_file_source_path(const std::string &api_path,
+auto sqlite_file_db::get_file_source_path(
-                                          std::string &source_path) const
+    const std::string &api_path, std::string &source_path) const -> api_error {
-    -> api_error {
   auto result = utils::db::sqlite::db_select{*db_, file_table}
                     .column("source_path")
                     .where("api_path")
@@ -330,9 +329,8 @@ auto sqlite_file_db::get_item_list(stop_type_callback stop_requested_cb) const
   return ret;
 }
 
-auto sqlite_file_db::get_source_path(const std::string &api_path,
+auto sqlite_file_db::get_source_path(
-                                     std::string &source_path) const
+    const std::string &api_path, std::string &source_path) const -> api_error {
-    -> api_error {
   auto result = utils::db::sqlite::db_select{*db_, file_table}
                     .column("source_path")
                     .where("api_path")

repertory/librepertory/src/db/impl/sqlite_meta_db.cpp

@@ -95,6 +95,10 @@ void sqlite_meta_db::enumerate_api_path_list(
       list.clear();
     }
   }
+
+  if (not list.empty()) {
+    callback(list);
+  }
 }
 
 auto sqlite_meta_db::get_api_path(const std::string &source_path,

repertory/librepertory/src/drives/fuse/fuse_drive.cpp

@@ -85,8 +85,8 @@ auto fuse_drive::chown_impl(std::string api_path, uid_t uid, gid_t gid,
                             struct fuse_file_info * /*file_info*/)
     -> api_error {
 #else
-auto fuse_drive::chown_impl(std::string api_path, uid_t uid, gid_t gid)
+auto fuse_drive::chown_impl(std::string api_path, uid_t uid,
-    -> api_error {
+                            gid_t gid) -> api_error {
 #endif
   return check_and_perform(
       api_path, X_OK, [&](api_meta_map &meta) -> api_error {
@@ -494,8 +494,8 @@ auto fuse_drive::getattr_impl(std::string api_path, struct stat *unix_st,
                               struct fuse_file_info * /*file_info*/)
     -> api_error {
 #else
-auto fuse_drive::getattr_impl(std::string api_path, struct stat *unix_st)
+auto fuse_drive::getattr_impl(std::string api_path,
-    -> api_error {
+                              struct stat *unix_st) -> api_error {
 #endif
   auto parent = utils::path::get_parent_api_path(api_path);
 
@@ -561,8 +561,8 @@ auto fuse_drive::getxtimes_impl(std::string api_path, struct timespec *bkuptime,
 #endif // __APPLE__
 
 #if FUSE_USE_VERSION >= 30
-auto fuse_drive::init_impl(struct fuse_conn_info *conn, struct fuse_config *cfg)
+auto fuse_drive::init_impl(struct fuse_conn_info *conn,
-    -> void * {
+                           struct fuse_config *cfg) -> void * {
 #else
 void *fuse_drive::init_impl(struct fuse_conn_info *conn) {
 #endif
@@ -666,13 +666,17 @@ auto fuse_drive::mkdir_impl(std::string api_path, mode_t mode) -> api_error {
 }
 
 void fuse_drive::notify_fuse_main_exit(int &ret) {
-  if (was_mounted_) {
+  REPERTORY_USES_FUNCTION_NAME();
-    event_system::instance().raise<drive_mount_result>(
+
-        function_name, get_mount_location(), std::to_string(ret));
+  if (not was_mounted_) {
-    event_system::instance().stop();
+    return;
-    logging_consumer_.reset();
-    console_consumer_.reset();
   }
+
+  event_system::instance().raise<drive_mount_result>(
+      function_name, get_mount_location(), std::to_string(ret));
+  event_system::instance().stop();
+  logging_consumer_.reset();
+  console_consumer_.reset();
 }
 
 auto fuse_drive::open_impl(std::string api_path,
@@ -800,9 +804,8 @@ auto fuse_drive::release_impl(std::string /*api_path*/,
   return api_error::success;
 }
 
-auto fuse_drive::releasedir_impl(std::string /*api_path*/,
+auto fuse_drive::releasedir_impl(
-                                 struct fuse_file_info *file_info)
+    std::string /*api_path*/, struct fuse_file_info *file_info) -> api_error {
-    -> api_error {
   auto iter = directory_cache_->get_directory(file_info->fh);
   if (iter == nullptr) {
     return api_error::invalid_handle;
@@ -820,8 +823,8 @@ auto fuse_drive::rename_directory(const std::string &from_api_path,
 }
 
 auto fuse_drive::rename_file(const std::string &from_api_path,
-                             const std::string &to_api_path, bool overwrite)
+                             const std::string &to_api_path,
-    -> int {
+                             bool overwrite) -> int {
   auto res = fm_->rename_file(from_api_path, to_api_path, overwrite);
   errno = std::abs(utils::from_api_error(res));
   return (res == api_error::success) ? 0 : -1;
@@ -831,8 +834,8 @@ auto fuse_drive::rename_file(const std::string &from_api_path,
 auto fuse_drive::rename_impl(std::string from_api_path, std::string to_api_path,
                              unsigned int /*flags*/) -> api_error {
 #else
-auto fuse_drive::rename_impl(std::string from_api_path, std::string to_api_path)
+auto fuse_drive::rename_impl(std::string from_api_path,
-    -> api_error {
+                             std::string to_api_path) -> api_error {
 #endif
   auto res = check_parent_access(to_api_path, W_OK | X_OK);
   if (res != api_error::success) {
@@ -934,15 +937,15 @@ auto fuse_drive::getxattr_impl(std::string api_path, const char *name,
 }
 #else  // __APPLE__
 auto fuse_drive::getxattr_impl(std::string api_path, const char *name,
-                               char *value, size_t size, int &attribute_size)
+                               char *value, size_t size,
-    -> api_error {
+                               int &attribute_size) -> api_error {
   return getxattr_common(api_path, name, value, size, attribute_size, nullptr);
 }
 #endif // __APPLE__
 
 auto fuse_drive::listxattr_impl(std::string api_path, char *buffer, size_t size,
-                                int &required_size, bool &return_size)
+                                int &required_size,
-    -> api_error {
+                                bool &return_size) -> api_error {
   auto check_size = (size == 0);
 
   auto res = check_parent_access(api_path, X_OK);
@@ -982,8 +985,8 @@ auto fuse_drive::listxattr_impl(std::string api_path, char *buffer, size_t size,
   return res;
 }
 
-auto fuse_drive::removexattr_impl(std::string api_path, const char *name)
+auto fuse_drive::removexattr_impl(std::string api_path,
-    -> api_error {
+                                  const char *name) -> api_error {
   std::string attribute_name;
 #if defined(__APPLE__)
   auto res = parse_xattr_parameters(name, 0, attribute_name, api_path);
@@ -1011,8 +1014,8 @@ auto fuse_drive::setxattr_impl(std::string api_path, const char *name,
                                uint32_t position) -> api_error {
 #else // __APPLE__
 auto fuse_drive::setxattr_impl(std::string api_path, const char *name,
-                               const char *value, size_t size, int flags)
+                               const char *value, size_t size,
-    -> api_error {
+                               int flags) -> api_error {
 #endif
   std::string attribute_name;
 #if defined(__APPLE__)
@@ -1090,8 +1093,8 @@ void fuse_drive::set_item_meta(const std::string &api_path,
 }
 
 #if defined(__APPLE__)
-auto fuse_drive::setattr_x_impl(std::string api_path, struct setattr_x *attr)
+auto fuse_drive::setattr_x_impl(std::string api_path,
-    -> api_error {
+                                struct setattr_x *attr) -> api_error {
   bool exists{};
   auto res = provider_.is_file(api_path, exists);
   if (res != api_error::success) {
@@ -1145,7 +1148,7 @@ auto fuse_drive::setattr_x_impl(std::string api_path, struct setattr_x *attr)
       ts[0].tv_sec = attr->acctime.tv_sec;
       ts[0].tv_nsec = attr->acctime.tv_nsec;
     } else {
-      struct timeval tv{};
+      struct timeval tv {};
       gettimeofday(&tv, NULL);
       ts[0].tv_sec = tv.tv_sec;
       ts[0].tv_nsec = tv.tv_usec * 1000;
@@ -1190,9 +1193,8 @@ auto fuse_drive::setattr_x_impl(std::string api_path, struct setattr_x *attr)
   return api_error::success;
 }
 
-auto fuse_drive::setbkuptime_impl(std::string api_path,
+auto fuse_drive::setbkuptime_impl(
-                                  const struct timespec *bkuptime)
+    std::string api_path, const struct timespec *bkuptime) -> api_error {
-    -> api_error {
   return check_and_perform(
       api_path, X_OK, [&](api_meta_map &meta) -> api_error {
         auto nanos = bkuptime->tv_nsec +
@@ -1228,8 +1230,8 @@ auto fuse_drive::setvolname_impl(const char * /*volname*/) -> api_error {
   return api_error::success;
 }
 
-auto fuse_drive::statfs_x_impl(std::string /*api_path*/, struct statfs *stbuf)
+auto fuse_drive::statfs_x_impl(std::string /*api_path*/,
-    -> api_error {
+                               struct statfs *stbuf) -> api_error {
   if (statfs(&config_.get_cache_directory()[0], stbuf) != 0) {
     return api_error::os_error;
   }
@@ -1254,8 +1256,8 @@ auto fuse_drive::statfs_x_impl(std::string /*api_path*/, struct statfs *stbuf)
   return api_error::success;
 }
 #else  // __APPLE__
-auto fuse_drive::statfs_impl(std::string /*api_path*/, struct statvfs *stbuf)
+auto fuse_drive::statfs_impl(std::string /*api_path*/,
-    -> api_error {
+                             struct statvfs *stbuf) -> api_error {
   if (statvfs(config_.get_cache_directory().data(), stbuf) != 0) {
     return api_error::os_error;
   }
@@ -1339,8 +1341,8 @@ auto fuse_drive::utimens_impl(std::string api_path, const struct timespec tv[2],
                               struct fuse_file_info * /*file_info*/)
     -> api_error {
 #else
-auto fuse_drive::utimens_impl(std::string api_path, const struct timespec tv[2])
+auto fuse_drive::utimens_impl(std::string api_path,
-    -> api_error {
+                              const struct timespec tv[2]) -> api_error {
 #endif
   api_meta_map meta;
   auto res = provider_.get_item_meta(api_path, meta);

repertory/librepertory/src/drives/fuse/fuse_drive_base.cpp

@@ -26,6 +26,8 @@
 #include "platform/platform.hpp"
 #include "providers/i_provider.hpp"
 #include "utils/common.hpp"
+#include "utils/path.hpp"
+#include "utils/string.hpp"
 #include "utils/time.hpp"
 
 namespace repertory {
@@ -33,8 +35,8 @@ auto fuse_drive_base::access_impl(std::string api_path, int mask) -> api_error {
   return check_access(api_path, mask);
 }
 
-auto fuse_drive_base::check_access(const std::string &api_path, int mask) const
+auto fuse_drive_base::check_access(const std::string &api_path,
-    -> api_error {
+                                   int mask) const -> api_error {
   REPERTORY_USES_FUNCTION_NAME();
 
   api_meta_map meta;
@@ -135,9 +137,8 @@ auto fuse_drive_base::check_and_perform(
   return action(meta);
 }
 
-auto fuse_drive_base::check_open_flags(int flags, int mask,
+auto fuse_drive_base::check_open_flags(
-                                       const api_error &fail_error)
+    int flags, int mask, const api_error &fail_error) -> api_error {
-    -> api_error {
   return (((flags & mask) == 0) ? api_error::success : fail_error);
 }
 
@@ -194,8 +195,8 @@ auto fuse_drive_base::check_parent_access(const std::string &api_path,
   return ret;
 }
 
-auto fuse_drive_base::check_readable(int flags, const api_error &fail_error)
+auto fuse_drive_base::check_readable(int flags,
-    -> api_error {
+                                     const api_error &fail_error) -> api_error {
   auto mode = (flags & O_ACCMODE);
   return ((mode == O_WRONLY) ? fail_error : api_error::success);
 }
@@ -253,16 +254,13 @@ auto fuse_drive_base::get_uid_from_meta(const api_meta_map &meta) -> uid_t {
 }
 
 #if defined(__APPLE__)
-auto fuse_drive_base::parse_xattr_parameters(const char *name,
+auto fuse_drive_base::parse_xattr_parameters(
-                                             const uint32_t &position,
+    const char *name, const uint32_t &position, std::string &attribute_name,
-                                             std::string &attribute_name,
+    const std::string &api_path) -> api_error {
-                                             const std::string &api_path)
-    -> api_error {
 #else  // !defined(__APPLE__)
-auto fuse_drive_base::parse_xattr_parameters(const char *name,
+auto fuse_drive_base::parse_xattr_parameters(
-                                             std::string &attribute_name,
+    const char *name, std::string &attribute_name,
-                                             const std::string &api_path)
+    const std::string &api_path) -> api_error {
-    -> api_error {
 #endif // defined(__APPLE__)
   auto res = api_path.empty() ? api_error::bad_address : api_error::success;
   if (res != api_error::success) {
@@ -295,11 +293,9 @@ auto fuse_drive_base::parse_xattr_parameters(
     std::string &attribute_name, const std::string &api_path) -> api_error {
   auto res = parse_xattr_parameters(name, position, attribute_name, api_path);
 #else  // !defined(__APPLE__)
-auto fuse_drive_base::parse_xattr_parameters(const char *name,
+auto fuse_drive_base::parse_xattr_parameters(
-                                             const char *value, size_t size,
+    const char *name, const char *value, size_t size,
-                                             std::string &attribute_name,
+    std::string &attribute_name, const std::string &api_path) -> api_error {
-                                             const std::string &api_path)
-    -> api_error {
   auto res = parse_xattr_parameters(name, attribute_name, api_path);
 #endif // defined(__APPLE__)
   if (res != api_error::success) {

repertory/librepertory/src/file_manager/open_file.cpp

@@ -21,6 +21,7 @@
 */
 #include "file_manager/open_file.hpp"
 
+#include "app_config.hpp"
 #include "events/event_system.hpp"
 #include "events/types/download_begin.hpp"
 #include "events/types/download_end.hpp"

repertory/librepertory/src/file_manager/ring_buffer_base.cpp

@@ -21,6 +21,7 @@
 */
 #include "file_manager/ring_buffer_base.hpp"
 
+#include "app_config.hpp"
 #include "events/event_system.hpp"
 #include "events/types/download_begin.hpp"
 #include "events/types/download_end.hpp"

repertory/librepertory/src/providers/base_provider.cpp

@@ -41,13 +41,13 @@
 #include "file_manager/cache_size_mgr.hpp"
 #include "file_manager/i_file_manager.hpp"
 #include "platform/platform.hpp"
+#include "utils/config.hpp"
 #include "utils/error_utils.hpp"
 #include "utils/file_utils.hpp"
 #include "utils/path.hpp"
 #include "utils/polling.hpp"
 #include "utils/tasks.hpp"
 #include "utils/time.hpp"
-#include <utils/config.hpp>
 
 namespace repertory {
 void base_provider::add_all_items(stop_type &stop_requested) {
@@ -69,8 +69,8 @@ void base_provider::add_all_items(stop_type &stop_requested) {
 }
 
 auto base_provider::create_api_file(std::string path, std::string key,
-                                    std::uint64_t size, std::uint64_t file_time)
+                                    std::uint64_t size,
-    -> api_file {
+                                    std::uint64_t file_time) -> api_file {
   api_file file{};
   file.api_path = utils::path::create_api_path(path);
   file.api_parent = utils::path::get_parent_api_path(file.api_path);
@@ -102,8 +102,8 @@ auto base_provider::create_api_file(std::string path, std::uint64_t size,
 }
 
 auto base_provider::create_directory_clone_source_meta(
-    const std::string &source_api_path, const std::string &api_path)
+    const std::string &source_api_path,
-    -> api_error {
+    const std::string &api_path) -> api_error {
   REPERTORY_USES_FUNCTION_NAME();
 
   bool exists{};
@@ -201,8 +201,8 @@ auto base_provider::create_directory(const std::string &api_path,
   return api_error::error;
 }
 
-auto base_provider::create_file(const std::string &api_path, api_meta_map &meta)
+auto base_provider::create_file(const std::string &api_path,
-    -> api_error {
+                                api_meta_map &meta) -> api_error {
   REPERTORY_USES_FUNCTION_NAME();
 
   try {
@@ -259,9 +259,8 @@ auto base_provider::create_file(const std::string &api_path, api_meta_map &meta)
   return api_error::error;
 }
 
-auto base_provider::get_api_path_from_source(const std::string &source_path,
+auto base_provider::get_api_path_from_source(
-                                             std::string &api_path) const
+    const std::string &source_path, std::string &api_path) const -> api_error {
-    -> api_error {
   REPERTORY_USES_FUNCTION_NAME();
 
   if (source_path.empty()) {
@@ -274,9 +273,8 @@ auto base_provider::get_api_path_from_source(const std::string &source_path,
   return db3_->get_api_path(source_path, api_path);
 }
 
-auto base_provider::get_directory_items(const std::string &api_path,
+auto base_provider::get_directory_items(
-                                        directory_item_list &list) const
+    const std::string &api_path, directory_item_list &list) const -> api_error {
-    -> api_error {
   REPERTORY_USES_FUNCTION_NAME();
 
   try {
@@ -344,10 +342,9 @@ auto base_provider::get_file_size(const std::string &api_path,
   return api_error::success;
 }
 
-auto base_provider::get_filesystem_item(const std::string &api_path,
+auto base_provider::get_filesystem_item(
-                                        bool directory,
+    const std::string &api_path, bool directory,
-                                        filesystem_item &fsi) const
+    filesystem_item &fsi) const -> api_error {
-    -> api_error {
   bool exists{};
   auto res = is_directory(api_path, exists);
   if (res != api_error::success) {
@@ -380,10 +377,9 @@ auto base_provider::get_filesystem_item(const std::string &api_path,
   return api_error::success;
 }
 
-auto base_provider::get_filesystem_item_and_file(const std::string &api_path,
+auto base_provider::get_filesystem_item_and_file(
-                                                 api_file &file,
+    const std::string &api_path, api_file &file,
-                                                 filesystem_item &fsi) const
+    filesystem_item &fsi) const -> api_error {
-    -> api_error {
   auto res = get_file(api_path, file);
   if (res != api_error::success) {
     return res;
@@ -502,6 +498,9 @@ void base_provider::process_removed_files(std::deque<removed_item> removed_list,
     }
 
     if (not utils::file::file{item.source_path}.exists()) {
+      db3_->remove_api_path(item.api_path);
+      event_system::instance().raise<file_removed_externally>(
+          item.api_path, function_name, item.source_path);
       continue;
     }
 
@@ -550,6 +549,8 @@ void base_provider::process_removed_files(std::deque<removed_item> removed_list,
 }
 
 void base_provider::process_removed_items(stop_type &stop_requested) {
+  REPERTORY_USES_FUNCTION_NAME();
+
   const auto get_stop_requested = [&stop_requested]() -> bool {
     return stop_requested || app_config::get_stop_requested();
   };
@@ -565,7 +566,8 @@ void base_provider::process_removed_items(stop_type &stop_requested) {
               tasks::instance().schedule({
                   [this, api_path](auto &&task_stopped) {
                     api_meta_map meta{};
-                    if (get_item_meta(api_path, meta) != api_error::success) {
+                    auto result = get_item_meta(api_path, meta);
+                    if (result != api_error::success) {
                       return;
                     }
 
@@ -613,6 +615,8 @@ void base_provider::process_removed_items(stop_type &stop_requested) {
 }
 
 void base_provider::remove_deleted_items(stop_type &stop_requested) {
+  REPERTORY_USES_FUNCTION_NAME();
+
   const auto get_stop_requested = [&stop_requested]() -> bool {
     return stop_requested || app_config::get_stop_requested();
   };

repertory/librepertory/src/utils/tasks.cpp

@@ -84,7 +84,7 @@ void tasks::start(app_config *config) {
   stop_requested_ = false;
   tasks_.clear();
 
-  for (std::uint32_t idx = 0U; idx < std::thread::hardware_concurrency();
+  for (std::uint32_t idx = 0U; idx < std::thread::hardware_concurrency() * 2U;
        ++idx) {
     task_threads_.emplace_back(
         std::make_unique<std::jthread>([this]() { task_thread(); }));

repertory/repertory/include/cli/get_version.hpp

@@ -27,7 +27,7 @@
 namespace repertory::cli::actions {
 template <typename drive> inline void version(std::vector<const char *> args) {
   std::cout << "Repertory core version: " << project_get_version() << std::endl;
-  std::cout << "Repertory Git revision: " << project_get_git_rev() << std::endl;
+  std::cout << "Repertory git revision: " << project_get_git_rev() << std::endl;
   drive::display_version_information(args);
 }
 } // namespace repertory::cli::actions

repertory/repertory/include/cli/mount.hpp

@@ -41,7 +41,7 @@ using repertory_drive = repertory::winfsp_drive;
 using remote_client = repertory::remote_winfsp::remote_client;
 using remote_drive = repertory::remote_winfsp::remote_winfsp_drive;
 using remote_instance = repertory::remote_winfsp::i_remote_instance;
-#else
+#else // !defined(_WIN32)
 #include "drives/fuse/fuse_drive.hpp"
 #include "drives/fuse/remotefuse/remote_client.hpp"
 #include "drives/fuse/remotefuse/remote_fuse_drive.hpp"
@@ -50,7 +50,7 @@ using repertory_drive = repertory::fuse_drive;
 using remote_client = repertory::remote_fuse::remote_client;
 using remote_drive = repertory::remote_fuse::remote_fuse_drive;
 using remote_instance = repertory::remote_fuse::i_remote_instance;
-#endif
+#endif // defined(_WIN32)
 
 namespace repertory::cli::actions {
 [[nodiscard]] inline auto
@@ -77,7 +77,8 @@ mount(std::vector<const char *> args, std::string data_directory,
         config.set_remote_config(cfg);
       } else if (prov == provider_type::sia &&
                  config.get_sia_config().bucket.empty()) {
-        config.set_value_by_name("SiaConfig.Bucket", unique_id);
+        [[maybe_unused]] auto bucket =
+            config.set_value_by_name("SiaConfig.Bucket", unique_id);
       }
 
       std::cout << "Generated " << app_config::get_provider_display_name(prov)
@@ -157,7 +158,8 @@ mount(std::vector<const char *> args, std::string data_directory,
       } else {
         if (prov == provider_type::sia &&
             config.get_sia_config().bucket.empty()) {
-          config.set_value_by_name("SiaConfig.Bucket", unique_id);
+          [[maybe_unused]] auto bucket =
+              config.set_value_by_name("SiaConfig.Bucket", unique_id);
         }
 
         try {

repertory/repertory_test/src/file_db_test.cpp

@@ -339,4 +339,30 @@ TYPED_TEST(file_db_test, item_not_found_is_returned_for_non_existing_api_path) {
             this->file_db->get_source_path("/file", source_path));
   EXPECT_TRUE(source_path.empty());
 }
+
+TYPED_TEST(file_db_test, can_enumerate_item_list) {
+  this->file_db->clear();
+
+  EXPECT_EQ(api_error::success,
+            this->file_db->add_directory("/", std::filesystem::current_path().string()));
+  EXPECT_EQ(api_error::success, this->file_db->add_or_update_file({
+                                    "/file",
+                                    0U,
+                                    {},
+                                    "c:\\test\\file.txt",
+                                }));
+
+  auto call_count{0U};
+  const auto get_stop_requested = []() -> bool { return false; };
+
+  this->file_db->enumerate_item_list(
+      [&call_count](auto &&list) {
+        EXPECT_EQ(std::size_t(2U), list.size());
+        ++call_count;
+      },
+      get_stop_requested);
+
+  EXPECT_EQ(std::size_t(1U), call_count);
+}
+
 } // namespace repertory

repertory/repertory_test/src/meta_db_test.cpp

@@ -615,4 +615,35 @@ TYPED_TEST(meta_db_test, check_set_item_meta_file_defaults) {
   EXPECT_EQ(0U, utils::string::to_uint64(meta[META_SIZE]));
   EXPECT_TRUE(meta[META_SOURCE].empty());
 }
+
+TYPED_TEST(meta_db_test, can_enumerate_api_path_list) {
+  this->meta_db->clear();
+
+  auto test_dir = create_test_file();
+  EXPECT_EQ(api_error::success,
+            this->meta_db->set_item_meta(
+                test_dir, {
+                              {META_DIRECTORY, utils::string::from_bool(true)},
+                          }));
+
+  auto test_file = create_test_file();
+  EXPECT_EQ(
+      api_error::success,
+      this->meta_db->set_item_meta(
+          test_file, {
+                         {META_DIRECTORY, utils::string::from_bool(false)},
+                     }));
+
+  auto call_count{0U};
+  const auto get_stop_requested = []() -> bool { return false; };
+
+  this->meta_db->enumerate_api_path_list(
+      [&call_count](auto &&list) {
+        EXPECT_EQ(std::size_t(2U), list.size());
+        ++call_count;
+      },
+      get_stop_requested);
+
+  EXPECT_EQ(std::size_t(1U), call_count);
+}
 } // namespace repertory

scripts/make_common.sh

@@ -23,6 +23,7 @@ cmake "${PROJECT_SOURCE_DIR}" \
   -DPROJECT_COMPANY_NAME="${PROJECT_COMPANY_NAME}" \
   -DPROJECT_COPYRIGHT="${PROJECT_COPYRIGHT}" \
   -DPROJECT_DESC="${PROJECT_DESC}" \
+  -DPROJECT_INTERFACE=1 \
   -DPROJECT_URL="${PROJECT_URL}" \
   ${PROJECT_CMAKE_OPTS} || exit 1