From 8623e56cf2779286101f87306744b4018df2b7af Mon Sep 17 00:00:00 2001 From: "Scott E. Graves" Date: Thu, 14 Aug 2025 19:50:36 -0500 Subject: [PATCH] Update Download Types --- Download-Types.md | 78 +++++++++++++++++++++++++++++++---------------- 1 file changed, 51 insertions(+), 27 deletions(-) diff --git a/Download-Types.md b/Download-Types.md index a65b161..cbfc687 100644 --- a/Download-Types.md +++ b/Download-Types.md @@ -1,38 +1,65 @@ # 📥 Repertory Download Types -Repertory supports **three modes** for reading and writing files: -`default`, `direct`, and `ring_buffer`. +Repertory supports three file access modes: **`default`**, **`direct`**, and **`ring_buffer`**. -You can set your preferred mode using the `PreferredDownloadType` option in `config.json`. -> **Note:** The actual mode may switch automatically depending on the file operation. -> Example: If a file is opened read-only and a write occurs, Repertory will switch to **default** mode. +Set your preference with the `PreferredDownloadType` option in `config.json`. + +> ⚠️ **Important clarification** +> As of the current implementation, **`default`** and **`ring_buffer`** behave the same for normal file opens: +> - Repertory attempts **ring-buffer mode** first +> - Falls back to **direct** if ring buffer isn’t viable +> - Promotes to **cached read/write** on a write +> In the future, `default` may switch to a different selection mechanism. Choose `ring_buffer` if you want to lock in today’s behavior. --- ## 🔹 Mode Overview -| Mode | Description | Access Type | -|-----------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------| -| **default** | Starts in **direct mode** (read-only, minimal caching). If a write occurs, it switches to **read/write mode**, creating a local cache file matching the remote file size and reading/writing in 8 MiB chunks. Uploads occur after full download. If the mount is unmounted prematurely, uploads resume on the next mount. Currently behaves the same as `ring_buffer` but may change in the future to a different selection mechanism. | Read/Write | -| **direct** | Reads directly from the provider without creating a local cache file. Uses an **in-memory ring buffer** (up to five 8 MiB chunks). | Read-only | -| **ring_buffer** | Allocates a **fixed-size buffer file** (default 512 MiB) inside the `buffer` directory. Used for reading large files that exceed the in-memory buffer size. Currently behaves the same as `default`. | Read-only | +| Mode | Description | Access Type | +|--------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------| +| **default** | Currently equivalent to `ring_buffer` for normal files: tries ring-buffer mode first; if not viable, falls back to direct. If a write occurs, the file is promoted to a fully cached read/write file.
**Note:** This may change in the future. | Read-only until promotion → Read/Write | +| **ring_buffer** | Uses a fixed-size buffer file (default **512 MiB**) in the `buffer/` directory. Attempted first (for normal files) and falls back to direct if not viable. Promotes to cached read/write on write. | Read-only until promotion → Read/Write | +| **direct** | Streams directly from the provider without a disk cache file. Does not attempt ring buffer. Promotes to cached read/write on write. Lowest disk usage for read-only workloads. | Read-only until promotion → Read/Write | --- -## 🛠 Choosing the Right Mode +## 🧠 How selection works (current behavior) -- **Use `default`** when: - - You need both read and write access. - - You want to ensure files can upload after partial downloads or disconnections. - - You are okay with it currently acting like `ring_buffer` but potentially changing in the future. -- **Use `direct`** when: - - You only need read access. - - You want the lowest disk usage. - - You’re streaming smaller files and have sufficient RAM for the buffer. -- **Use `ring_buffer`** when: - - You are reading very large files repeatedly. - - You want to limit disk space usage by using a fixed-size disk buffer. - - You want behavior currently identical to `default`. +When a path is opened, Repertory chooses a mode with these rules: + +1. **Early overrides → cached immediately** + - Opening a directory + - Opening a zero-byte file + - Opening an in-progress item + - A complete local cache already exists for the file + +2. **Provider is read-only → `direct`** + +3. **Preference for normal files** + - `PreferredDownloadType = direct` → `direct` (never tries ring buffer) + - `PreferredDownloadType = default` → try `ring_buffer`, else `direct` + - `PreferredDownloadType = ring_buffer` → try `ring_buffer`, else `direct` + +4. **Ring-buffer viability checks** + - File must be handleable under `RingBufferFileSize` (chunk size ≈ 8 MiB; ring size = `RingBufferFileSize / chunk`) + - `buffer/` directory must exist (created if needed) + - The buffer drive must have at least `RingBufferFileSize` free + - If any check fails → `direct` + +5. **Write promotion** + - If a file opened read-only (`direct` / `ring_buffer`) is later written to, Repertory replaces the open handle with a **cached read/write** file. + - Writes are uploaded only after the file is fully downloaded. + - If interrupted, uploads resume after remount. + +--- + +## 🛠 Choosing the right mode + +- **Pick `ring_buffer`** if you want to lock in today’s adaptive behavior (bounded disk usage with automatic fallback to direct and promotion to cached on write). +- **Pick `default`** if you’re okay with the current `ring_buffer`-first behavior and want to follow any future improvements to the selection strategy automatically. +- **Pick `direct`** if you: + - Want minimal disk usage and never want ring buffer used automatically. + - Are doing read-only streaming and are fine with promotion to cached only if/when a write happens. --- @@ -41,7 +68,4 @@ You can set your preferred mode using the `PreferredDownloadType` option in `con ```json { "PreferredDownloadType": "default" -} -``` - -Replace `"default"` with `"direct"` or `"ring_buffer"` to change the mode. \ No newline at end of file +} \ No newline at end of file