v2.0.3-rc (#32)

# Changelog

## v2.0.3-rc

### Issues

* \#28 \[bug\] Address slow directory responses in S3 mounts for deeply nested directories
* \#29 \[bug\] S3 error responses are not being logged
* \#30 \[bug\] Sia provider error responses are not logged
* \#31 \[bug\] S3 provider should limit max key size to 1024

### Changes from v2.0.2-rc

* Always use direct for read-only providers
* Fixed externally removed files not being processed during cleanup
* Fixed http headers not being added for requests
* Fixed incorrect `stat` values for remote mounts
* Fixed invalid directory nullptr error on remote mounts
* Fixed memory leak in event system
* Refactored application shutdown
* Refactored event system
* Updated build system to Alpine 3.21.0
* Updated build system to MinGW-w64 12.0.0
* Updated copyright to 2018-2025

Reviewed-on: #32

support/include/utils/all.hpp

@@ -1,5 +1,5 @@
 /*
-  Copyright <2018-2024> <scott.e.graves@protonmail.com>
+  Copyright <2018-2025> <scott.e.graves@protonmail.com>
 
   Permission is hereby granted, free of charge, to any person obtaining a copy
   of this software and associated documentation files (the "Software"), to deal

support/include/utils/collection.hpp

@@ -1,5 +1,5 @@
 /*
-  Copyright <2018-2024> <scott.e.graves@protonmail.com>
+  Copyright <2018-2025> <scott.e.graves@protonmail.com>
 
   Permission is hereby granted, free of charge, to any person obtaining a copy
   of this software and associated documentation files (the "Software"), to deal

support/include/utils/com_init_wrapper.hpp

@@ -1,5 +1,5 @@
 /*
-  Copyright <2018-2024> <scott.e.graves@protonmail.com>
+  Copyright <2018-2025> <scott.e.graves@protonmail.com>
 
   Permission is hereby granted, free of charge, to any person obtaining a copy
   of this software and associated documentation files (the "Software"), to deal

support/include/utils/common.hpp

@@ -1,5 +1,5 @@
 /*
-  Copyright <2018-2024> <scott.e.graves@protonmail.com>
+  Copyright <2018-2025> <scott.e.graves@protonmail.com>
 
   Permission is hereby granted, free of charge, to any person obtaining a copy
   of this software and associated documentation files (the "Software"), to deal

support/include/utils/config.hpp

@@ -1,5 +1,5 @@
 /*
-  Copyright <2018-2024> <scott.e.graves@protonmail.com>
+  Copyright <2018-2025> <scott.e.graves@protonmail.com>
 
   Permission is hereby granted, free of charge, to any person obtaining a copy
   of this software and associated documentation files (the "Software"), to deal
@@ -416,6 +416,7 @@ using data_buffer = std::vector<unsigned char>;
 using mutex_lock = std::lock_guard<std::mutex>;
 using recur_mutex_lock = std::lock_guard<std::recursive_mutex>;
 using stop_type = std::atomic_bool;
+using stop_type_callback = std::function<std::atomic_bool()>;
 using unique_mutex_lock = std::unique_lock<std::mutex>;
 using unique_recur_mutex_lock = std::unique_lock<std::recursive_mutex>;
 
@@ -462,16 +463,14 @@ struct file_deleter final {
 };
 using file_t = std::unique_ptr<FILE, file_deleter>;
 
-#if defined(PROJECT_ENABLE_CURL)
 struct http_range final {
   std::uint64_t begin{};
   std::uint64_t end{};
 };
 
-using http_headers = std::unordered_map<std::string, std::string>;
+using http_headers = std::map<std::string, std::string>;
 using http_query_parameters = std::map<std::string, std::string>;
 using http_ranges = std::vector<http_range>;
-#endif // defined(PROJECT_ENABLE_CURL)
 } // namespace repertory
 #endif // defined(__cplusplus)
 

support/include/utils/db/sqlite/db_common.hpp

@@ -1,5 +1,5 @@
 /*
-  Copyright <2018-2024> <scott.e.graves@protonmail.com>
+  Copyright <2018-2025> <scott.e.graves@protonmail.com>
 
   Permission is hereby granted, free of charge, to any person obtaining a copy
   of this software and associated documentation files (the "Software"), to deal

support/include/utils/db/sqlite/db_delete.hpp

@@ -1,5 +1,5 @@
 /*
-  Copyright <2018-2024> <scott.e.graves@protonmail.com>
+  Copyright <2018-2025> <scott.e.graves@protonmail.com>
 
   Permission is hereby granted, free of charge, to any person obtaining a copy
   of this software and associated documentation files (the "Software"), to deal

support/include/utils/db/sqlite/db_insert.hpp

@@ -1,5 +1,5 @@
 /*
-  Copyright <2018-2024> <scott.e.graves@protonmail.com>
+  Copyright <2018-2025> <scott.e.graves@protonmail.com>
 
   Permission is hereby granted, free of charge, to any person obtaining a copy
   of this software and associated documentation files (the "Software"), to deal

support/include/utils/db/sqlite/db_select.hpp

@@ -1,5 +1,5 @@
 /*
-  Copyright <2018-2024> <scott.e.graves@protonmail.com>
+  Copyright <2018-2025> <scott.e.graves@protonmail.com>
 
   Permission is hereby granted, free of charge, to any person obtaining a copy
   of this software and associated documentation files (the "Software"), to deal

support/include/utils/db/sqlite/db_update.hpp

@@ -1,5 +1,5 @@
 /*
-  Copyright <2018-2024> <scott.e.graves@protonmail.com>
+  Copyright <2018-2025> <scott.e.graves@protonmail.com>
 
   Permission is hereby granted, free of charge, to any person obtaining a copy
   of this software and associated documentation files (the "Software"), to deal

support/include/utils/db/sqlite/db_where_t.hpp

@@ -1,5 +1,5 @@
 /*
-  Copyright <2018-2024> <scott.e.graves@protonmail.com>
+  Copyright <2018-2025> <scott.e.graves@protonmail.com>
 
   Permission is hereby granted, free of charge, to any person obtaining a copy
   of this software and associated documentation files (the "Software"), to deal

support/include/utils/encrypting_reader.hpp

@@ -1,5 +1,5 @@
 /*
-  Copyright <2018-2024> <scott.e.graves@protonmail.com>
+  Copyright <2018-2025> <scott.e.graves@protonmail.com>
 
   Permission is hereby granted, free of charge, to any person obtaining a copy
   of this software and associated documentation files (the "Software"), to deal
@@ -32,17 +32,19 @@ namespace repertory::utils::encryption {
 class encrypting_reader final {
 public:
   encrypting_reader(std::string_view file_name, std::string_view source_path,
-                    stop_type &stop_requested, std::string_view token,
+                    stop_type_callback stop_requested_cb,
+                    std::string_view token,
                     std::optional<std::string> relative_parent_path,
                     std::size_t error_return = 0U);
 
   encrypting_reader(std::string_view encrypted_file_path,
-                    std::string_view source_path, stop_type &stop_requested,
+                    std::string_view source_path,
+                    stop_type_callback stop_requested_cb,
                     std::string_view token, std::size_t error_return = 0U);
 
   encrypting_reader(
       std::string_view encrypted_file_path, std::string_view source_path,
-      stop_type &stop_requested, std::string_view token,
+      stop_type_callback stop_requested_cb, std::string_view token,
       std::vector<std::array<unsigned char,
                              crypto_aead_xchacha20poly1305_IETF_NPUBBYTES>>
           iv_list,
@@ -62,7 +64,7 @@ public:
 
 private:
   utils::encryption::hash_256_t key_;
-  stop_type &stop_requested_;
+  stop_type_callback stop_requested_cb_;
   size_t error_return_;
   std::unique_ptr<utils::file::i_file> source_file_;
 
@@ -87,16 +89,16 @@ private:
   auto reader_function(char *buffer, size_t size, size_t nitems) -> size_t;
 
 public:
-  [[nodiscard]] static auto
-  calculate_decrypted_size(std::uint64_t total_size) -> std::uint64_t;
+  [[nodiscard]] static auto calculate_decrypted_size(std::uint64_t total_size)
+      -> std::uint64_t;
 
   [[nodiscard]] static auto
   calculate_encrypted_size(std::string_view source_path) -> std::uint64_t;
 
   [[nodiscard]] auto create_iostream() const -> std::shared_ptr<iostream>;
 
-  [[nodiscard]] static constexpr auto
-  get_encrypted_chunk_size() -> std::size_t {
+  [[nodiscard]] static constexpr auto get_encrypted_chunk_size()
+      -> std::size_t {
     return encrypted_chunk_size_;
   }
 
@@ -116,14 +118,17 @@ public:
     return error_return_;
   }
 
-  [[nodiscard]] auto get_iv_list()
-      -> std::vector<std::array<unsigned char,
-                                crypto_aead_xchacha20poly1305_IETF_NPUBBYTES>> {
+  [[nodiscard]] static constexpr auto get_header_size() -> std::size_t {
+    return header_size_;
+  }
+
+  [[nodiscard]] auto get_iv_list() -> std::vector<
+      std::array<unsigned char, crypto_aead_xchacha20poly1305_IETF_NPUBBYTES>> {
     return iv_list_;
   }
 
   [[nodiscard]] auto get_stop_requested() const -> bool {
-    return stop_requested_;
+    return stop_requested_cb_();
   }
 
   [[nodiscard]] auto get_total_size() const -> std::uint64_t {
@@ -131,8 +136,8 @@ public:
   }
 
   [[nodiscard]] static auto reader_function(char *buffer, size_t size,
-                                            size_t nitems,
-                                            void *instream) -> size_t {
+                                            size_t nitems, void *instream)
+      -> size_t {
     return reinterpret_cast<encrypting_reader *>(instream)->reader_function(
         buffer, size, nitems);
   }

support/include/utils/encryption.hpp

@@ -1,5 +1,5 @@
 /*
-  Copyright <2018-2024> <scott.e.graves@protonmail.com>
+  Copyright <2018-2025> <scott.e.graves@protonmail.com>
 
   Permission is hereby granted, free of charge, to any person obtaining a copy
   of this software and associated documentation files (the "Software"), to deal
@@ -185,7 +185,6 @@ encrypt_data(const std::array<unsigned char,
                          buf.size(), res);
 }
 
-#if defined(PROJECT_ENABLE_CURL)
 using reader_func_t =
     std::function<bool(data_buffer &cypher_text, std::uint64_t start_offset,
                        std::uint64_t end_offset)>;
@@ -195,7 +194,11 @@ read_encrypted_range(const http_range &range,
                      const utils::encryption::hash_256_t &key,
                      reader_func_t reader_func, std::uint64_t total_size,
                      data_buffer &data) -> bool;
-#endif // defined(PROJECT_ENABLE_CURL)
+
+[[nodiscard]] auto read_encrypted_range(
+    const http_range &range, const utils::encryption::hash_256_t &key,
+    reader_func_t reader_func, std::uint64_t total_size, unsigned char *data,
+    std::size_t size, std::size_t &bytes_read) -> bool;
 #endif // defined(PROJECT_ENABLE_BOOST)
 
 template <typename hash_t>

support/include/utils/error.hpp

@@ -1,5 +1,5 @@
 /*
-  Copyright <2018-2024> <scott.e.graves@protonmail.com>
+  Copyright <2018-2025> <scott.e.graves@protonmail.com>
 
   Permission is hereby granted, free of charge, to any person obtaining a copy
   of this software and associated documentation files (the "Software"), to deal

support/include/utils/file.hpp

@@ -1,5 +1,5 @@
 /*
-  Copyright <2018-2024> <scott.e.graves@protonmail.com>
+  Copyright <2018-2025> <scott.e.graves@protonmail.com>
 
   Permission is hereby granted, free of charge, to any person obtaining a copy
   of this software and associated documentation files (the "Software"), to deal

support/include/utils/file_directory.hpp

@@ -1,5 +1,5 @@
 /*
-  Copyright <2018-2024> <scott.e.graves@protonmail.com>
+  Copyright <2018-2025> <scott.e.graves@protonmail.com>
 
   Permission is hereby granted, free of charge, to any person obtaining a copy
   of this software and associated documentation files (the "Software"), to deal

support/include/utils/file_enc_file.hpp

@@ -1,5 +1,5 @@
 /*
-  Copyright <2018-2024> <scott.e.graves@protonmail.com>
+  Copyright <2018-2025> <scott.e.graves@protonmail.com>
 
   Permission is hereby granted, free of charge, to any person obtaining a copy
   of this software and associated documentation files (the "Software"), to deal
@@ -45,12 +45,13 @@ public:
 
 private:
   fs_file_t file_;
+  std::string encryption_token_;
 
 public:
   void close() override;
 
-  [[nodiscard]] auto copy_to(std::string_view new_path,
-                             bool overwrite) const -> bool override;
+  [[nodiscard]] auto copy_to(std::string_view new_path, bool overwrite) const
+      -> bool override;
 
   [[nodiscard]] auto exists() const -> bool override { return file_->exists(); }
 
@@ -68,8 +69,8 @@ public:
     return file_->get_read_buffer_size();
   }
 
-  [[nodiscard]] auto
-  get_time(time_type type) const -> std::optional<std::uint64_t> override {
+  [[nodiscard]] auto get_time(time_type type) const
+      -> std::optional<std::uint64_t> override {
     return file_->get_time(type);
   }
 
@@ -97,9 +98,10 @@ public:
 
   [[nodiscard]] auto truncate(std::size_t size) -> bool override;
 
-  [[nodiscard]] auto
-  write(const unsigned char *data, std::size_t to_write, std::size_t offset,
-        std::size_t *total_written = nullptr) -> bool override;
+  [[nodiscard]] auto write(const unsigned char *data, std::size_t to_write,
+                           std::size_t offset,
+                           std::size_t *total_written = nullptr)
+      -> bool override;
 
 public:
   [[nodiscard]] operator bool() const override {

support/include/utils/file_file.hpp

@@ -1,5 +1,5 @@
 /*
-  Copyright <2018-2024> <scott.e.graves@protonmail.com>
+  Copyright <2018-2025> <scott.e.graves@protonmail.com>
 
   Permission is hereby granted, free of charge, to any person obtaining a copy
   of this software and associated documentation files (the "Software"), to deal

support/include/utils/file_smb_directory.hpp

@@ -1,5 +1,5 @@
 /*
-  Copyright <2018-2024> <scott.e.graves@protonmail.com>
+  Copyright <2018-2025> <scott.e.graves@protonmail.com>
 
   Permission is hereby granted, free of charge, to any person obtaining a copy
   of this software and associated documentation files (the "Software"), to deal

support/include/utils/file_smb_file.hpp

@@ -1,5 +1,5 @@
 /*
-  Copyright <2018-2024> <scott.e.graves@protonmail.com>
+  Copyright <2018-2025> <scott.e.graves@protonmail.com>
 
   Permission is hereby granted, free of charge, to any person obtaining a copy
   of this software and associated documentation files (the "Software"), to deal

support/include/utils/file_thread_file.hpp

@@ -1,5 +1,5 @@
 /*
-  Copyright <2018-2024> <scott.e.graves@protonmail.com>
+  Copyright <2018-2025> <scott.e.graves@protonmail.com>
 
   Permission is hereby granted, free of charge, to any person obtaining a copy
   of this software and associated documentation files (the "Software"), to deal
@@ -67,11 +67,44 @@ public:
   thread_file(thread_file &&move_file) noexcept
       : file_(std::move(move_file.file_)) {}
 
-  ~thread_file() override { close(); }
+  ~thread_file() override;
+
+private:
+  using action_t = std::function<bool()>;
+
+  struct io_item final {
+    action_t action;
+    bool complete{false};
+    std::unique_ptr<std::mutex> mtx{
+        std::make_unique<std::mutex>(),
+    };
+    mutable std::unique_ptr<std::condition_variable> notify{
+        std::make_unique<std::condition_variable>(),
+    };
+    bool success{false};
+
+    void done(bool result);
+
+    void wait() const;
+  };
 
 private:
   fs_file_t file_;
 
+private:
+  mutable std::vector<std::shared_ptr<io_item>> actions_;
+  mutable std::unique_ptr<std::thread> io_thread_;
+  mutable std::unique_ptr<std::mutex> mtx_{std::make_unique<std::mutex>()};
+  mutable std::unique_ptr<std::condition_variable> notify_{
+      std::make_unique<std::condition_variable>(),
+  };
+  stop_type stop_requested_{false};
+
+private:
+  auto do_io(action_t action) const -> bool;
+
+  void thread_func() const;
+
 public:
   void close() override;
 

support/include/utils/hash.hpp

@@ -1,5 +1,5 @@
 /*
-  Copyright <2018-2024> <scott.e.graves@protonmail.com>
+  Copyright <2018-2025> <scott.e.graves@protonmail.com>
 
   Permission is hereby granted, free of charge, to any person obtaining a copy
   of this software and associated documentation files (the "Software"), to deal

support/include/utils/path.hpp

@@ -1,5 +1,5 @@
 /*
-  Copyright <2018-2024> <scott.e.graves@protonmail.com>
+  Copyright <2018-2025> <scott.e.graves@protonmail.com>
 
   Permission is hereby granted, free of charge, to any person obtaining a copy
   of this software and associated documentation files (the "Software"), to deal

support/include/utils/string.hpp

@@ -1,5 +1,5 @@
 /*
-  Copyright <2018-2024> <scott.e.graves@protonmail.com>
+  Copyright <2018-2025> <scott.e.graves@protonmail.com>
 
   Permission is hereby granted, free of charge, to any person obtaining a copy
   of this software and associated documentation files (the "Software"), to deal

support/include/utils/time.hpp

@@ -1,5 +1,5 @@
 /*
-  Copyright <2018-2024> <scott.e.graves@protonmail.com>
+  Copyright <2018-2025> <scott.e.graves@protonmail.com>
 
   Permission is hereby granted, free of charge, to any person obtaining a copy
   of this software and associated documentation files (the "Software"), to deal

support/include/utils/types/file/i_directory.hpp

@@ -1,5 +1,5 @@
 /*
-  Copyright <2018-2024> <scott.e.graves@protonmail.com>
+  Copyright <2018-2025> <scott.e.graves@protonmail.com>
 
   Permission is hereby granted, free of charge, to any person obtaining a copy
   of this software and associated documentation files (the "Software"), to deal

support/include/utils/types/file/i_file.hpp

@@ -1,5 +1,5 @@
 /*
-  Copyright <2018-2024> <scott.e.graves@protonmail.com>
+  Copyright <2018-2025> <scott.e.graves@protonmail.com>
 
   Permission is hereby granted, free of charge, to any person obtaining a copy
   of this software and associated documentation files (the "Software"), to deal

support/include/utils/types/file/i_fs_item.hpp

@@ -1,5 +1,5 @@
 /*
-  Copyright <2018-2024> <scott.e.graves@protonmail.com>
+  Copyright <2018-2025> <scott.e.graves@protonmail.com>
 
   Permission is hereby granted, free of charge, to any person obtaining a copy
   of this software and associated documentation files (the "Software"), to deal

support/include/utils/unix.hpp

@@ -1,5 +1,5 @@
 /*
-  Copyright <2018-2024> <scott.e.graves@protonmail.com>
+  Copyright <2018-2025> <scott.e.graves@protonmail.com>
 
   Permission is hereby granted, free of charge, to any person obtaining a copy
   of this software and associated documentation files (the "Software"), to deal

support/include/utils/windows.hpp

@@ -1,5 +1,5 @@
 /*
-  Copyright <2018-2024> <scott.e.graves@protonmail.com>
+  Copyright <2018-2025> <scott.e.graves@protonmail.com>
 
   Permission is hereby granted, free of charge, to any person obtaining a copy
   of this software and associated documentation files (the "Software"), to deal