BlockStorage/repertory
1
0
Fork 0
Code Issues 3 Pull Requests Releases 9 Wiki Activity

Update Download Types

sgraves76
2025-08-14 19:50:36 -05:00
parent 873ea89145
commit 8623e56cf2
1 changed files with 51 additions and 27 deletions
Download Patch File Download Diff File Expand all files Collapse all files

78
Download-Types.md

@@ -1,38 +1,65 @@
# 📥 Repertory Download Types # 📥 Repertory Download Types
Repertory supports **three modes** for reading and writing files: Repertory supports three file access modes: **`default`**, **`direct`**, and **`ring_buffer`**.
`default`, `direct`, and `ring_buffer`.
You can set your preferred mode using the `PreferredDownloadType` option in `config.json`. Set your preference with the `PreferredDownloadType` option in `config.json`.
> **Note:** The actual mode may switch automatically depending on the file operation.
> Example: If a file is opened read-only and a write occurs, Repertory will switch to **default** mode. > ⚠️ **Important clarification**
> As of the current implementation, **`default`** and **`ring_buffer`** behave the same for normal file opens:
> - Repertory attempts **ring-buffer mode** first
> - Falls back to **direct** if ring buffer isn’t viable
> - Promotes to **cached read/write** on a write
> In the future, `default` may switch to a different selection mechanism. Choose `ring_buffer` if you want to lock in today’s behavior.
--- ---
## 🔹 Mode Overview ## 🔹 Mode Overview
| Mode | Description | Access Type | | Mode | Description | Access Type |
|-----------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------| |--------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------|
| **default** | Starts in **direct mode** (read-only, minimal caching). If a write occurs, it switches to **read/write mode**, creating a local cache file matching the remote file size and reading/writing in 8 MiB chunks. Uploads occur after full download. If the mount is unmounted prematurely, uploads resume on the next mount. Currently behaves the same as `ring_buffer` but may change in the future to a different selection mechanism. | Read/Write | | **default** | Currently equivalent to `ring_buffer` for normal files: tries ring-buffer mode first; if not viable, falls back to direct. If a write occurs, the file is promoted to a fully cached read/write file.<br>**Note:** This may change in the future. | Read-only until promotion → Read/Write |
| **direct** | Reads directly from the provider without creating a local cache file. Uses an **in-memory ring buffer** (up to five 8 MiB chunks). | Read-only | | **ring_buffer** | Uses a fixed-size buffer file (default **512 MiB**) in the `buffer/` directory. Attempted first (for normal files) and falls back to direct if not viable. Promotes to cached read/write on write. | Read-only until promotion → Read/Write |
| **ring_buffer** | Allocates a **fixed-size buffer file** (default 512 MiB) inside the `buffer` directory. Used for reading large files that exceed the in-memory buffer size. Currently behaves the same as `default`. | Read-only | | **direct** | Streams directly from the provider without a disk cache file. Does not attempt ring buffer. Promotes to cached read/write on write. Lowest disk usage for read-only workloads. | Read-only until promotion → Read/Write |
--- ---
## 🛠 Choosing the Right Mode ## 🧠 How selection works (current behavior)
- **Use `default`** when: When a path is opened, Repertory chooses a mode with these rules:
- You need both read and write access.
- You want to ensure files can upload after partial downloads or disconnections. 1. **Early overrides → cached immediately**
- You are okay with it currently acting like `ring_buffer` but potentially changing in the future. - Opening a directory
- **Use `direct`** when: - Opening a zero-byte file
- You only need read access. - Opening an in-progress item
- You want the lowest disk usage. - A complete local cache already exists for the file
- You’re streaming smaller files and have sufficient RAM for the buffer.
- **Use `ring_buffer`** when: 2. **Provider is read-only → `direct`**
- You are reading very large files repeatedly.
- You want to limit disk space usage by using a fixed-size disk buffer. 3. **Preference for normal files**
- You want behavior currently identical to `default`. - `PreferredDownloadType = direct` → `direct` (never tries ring buffer)
- `PreferredDownloadType = default` → try `ring_buffer`, else `direct`
- `PreferredDownloadType = ring_buffer` → try `ring_buffer`, else `direct`
4. **Ring-buffer viability checks**
- File must be handleable under `RingBufferFileSize` (chunk size ≈ 8 MiB; ring size = `RingBufferFileSize / chunk`)
- `buffer/` directory must exist (created if needed)
- The buffer drive must have at least `RingBufferFileSize` free
- If any check fails → `direct`
5. **Write promotion**
- If a file opened read-only (`direct` / `ring_buffer`) is later written to, Repertory replaces the open handle with a **cached read/write** file.
- Writes are uploaded only after the file is fully downloaded.
- If interrupted, uploads resume after remount.
---
## 🛠 Choosing the right mode
- **Pick `ring_buffer`** if you want to lock in today’s adaptive behavior (bounded disk usage with automatic fallback to direct and promotion to cached on write).
- **Pick `default`** if you’re okay with the current `ring_buffer`-first behavior and want to follow any future improvements to the selection strategy automatically.
- **Pick `direct`** if you:
- Want minimal disk usage and never want ring buffer used automatically.
- Are doing read-only streaming and are fine with promotion to cached only if/when a write happens.
--- ---
@@ -41,7 +68,4 @@ You can set your preferred mode using the `PreferredDownloadType` option in `con
```json ```json
{ {
"PreferredDownloadType": "default" "PreferredDownloadType": "default"
} }
```
Replace `"default"` with `"direct"` or `"ring_buffer"` to change the mode.