Added siad for unit testing
This commit is contained in:
Scott E. Graves
289
3rd-party/Sia-v1.1.0-windows-amd64/doc/File Contract Negotiation.md
vendored
Normal file
289
3rd-party/Sia-v1.1.0-windows-amd64/doc/File Contract Negotiation.md
vendored
Normal file
@@ -0,0 +1,289 @@
|
||||
File Contract Negotiation
|
||||
=========================
|
||||
|
||||
Securing data on Sia requires creating and revising file contracts in an
|
||||
untrusted environment. Managing data on Sia happens through several protocols:
|
||||
|
||||
+ Settings Request - the host sends the renter its settings.
|
||||
|
||||
+ Revision Request - the renter will send the host a file contract id, and the
|
||||
host will send the most recent file contract revision that it knows of for
|
||||
that file contract, with the signatures. A challenge and response is also
|
||||
performed to verify that the renter is able to create the signatures to
|
||||
modify the file contract revision.
|
||||
|
||||
+ File Contract Creation - no data is uploaded during the initial creation of a
|
||||
file contract, but funds are allocated so that the file contract can be
|
||||
iteratively revised in the future.
|
||||
|
||||
+ File Contract Revision - an existing file contract is revised so that data
|
||||
can be added to an arbitrary place, or removed from an arbitrary place.
|
||||
|
||||
+ File Contract Renewal - an existing file contract is renewed, meaning that a
|
||||
new file contract with a different id is created, but that has the same data.
|
||||
New funds are added to this file contract, and it can now be modified
|
||||
separately from the previous contract.
|
||||
|
||||
+ Data Request - data is requested from the host by hash.
|
||||
|
||||
+ (planned for later) Storage Proof Request - the renter requests that the host
|
||||
perform an out-of-band storage proof.
|
||||
|
||||
+ (planned for later) Metadata Request - the renter requests some metadata
|
||||
about the file contract from the host, namely the list of hashes that compose
|
||||
the file. This list of hashes is provided along with a cryptographic proof
|
||||
that the hashes are valid. The proof is only needed if only a subset of
|
||||
hashes are being sent.
|
||||
|
||||
A frequently seen construction is 'acceptance'. The renter or host may have the
|
||||
opportunity to accept or reject a communication, which takes the form of a
|
||||
string. The acceptance string is always the same, and any string that is not
|
||||
the acceptance string is a rejection. The rejection string can include reasons
|
||||
why the rejection occurred, but most not exceed 255 bytes. After a rejection,
|
||||
the connection is always closed.
|
||||
|
||||
The protocols described below are numbered. The number indicates when the
|
||||
communicator is switching. Each pair of numbers is a full round trip of
|
||||
communications.
|
||||
|
||||
All communications attempt to support slow connections and Tor connections. Any
|
||||
connection with a throughput below 100kbps may struggle to perform the uploads
|
||||
and downloads, and any connection with a rountrip latency greater than 2
|
||||
minutes may struggle to complete the protocols.
|
||||
|
||||
Settings Request
|
||||
----------------
|
||||
|
||||
The host signs the settings request to prove that the connection has opened to
|
||||
the right party. Hosts announce on the blockchain and perform burn, therefore
|
||||
identity is important.
|
||||
|
||||
1. The renter makes an RPC to the host, opening a connection. The connection
|
||||
deadline should be at least 120 seconds.
|
||||
|
||||
2. The host sends the renter the most recent copy of its external settings,
|
||||
signed by the host public key. The connection is then closed.
|
||||
|
||||
Revision Request
|
||||
----------------
|
||||
|
||||
The renter requests a recent revision from the host. Often, this request
|
||||
precedes modifications. A file contract can only be open for revision with one
|
||||
party at a time. To prevent DoS attacks, the party must authenticate here by
|
||||
performing a challenge-response protocol during the revision request. Putting
|
||||
this challenge-response requirement in the revision-request can help improve
|
||||
privacy, though the host is under no cryptographic or incentive-based
|
||||
obligation to preserve the privacy of the revision.
|
||||
|
||||
1. The renter makes an RPC to the host, opening a connection. The connection
|
||||
deadline should be at least 120 seconds. The renter sends the file contract
|
||||
id for the revision being requested.
|
||||
|
||||
2. The host writes 32 bytes of random data that the renter must sign for the
|
||||
renter key in the corresponding file contract.
|
||||
|
||||
3. The renter returns the signed challenge.
|
||||
|
||||
4. The host verifies the signature from the renter and then sends the renter
|
||||
the most recent file contract revision, along with the transaction
|
||||
signatures from both the renter and the host. The connection is then closed.
|
||||
|
||||
File Contract Creation
|
||||
----------------------
|
||||
|
||||
A few decisions were made regarding the file contract protocol. The first is
|
||||
that the renter should not sign the file contract until the host has formally
|
||||
accepted the file contract. The second is that the host should be the last one
|
||||
to sign the file contract, as the renter is the party with the strong
|
||||
reputation system.
|
||||
|
||||
Instead of sending a whole transaction each time, the transaction is sent
|
||||
piecemeal, and only the new parts at each step are sent to the other party.
|
||||
This minimizes the surface area of data for a malicious party to manipulate,
|
||||
which means less verification code, which means less chances of having a bug in
|
||||
the verification code.
|
||||
|
||||
The renter pays for the siafund fee on the host's collateral and contract fee.
|
||||
If a renter opens a file contract and then never uses it, the host does not
|
||||
lose money. This does put the renter at risk, as they may open up a file
|
||||
contract and then watch the host leave, but the renter is spreading the risk
|
||||
over communications with many hosts, and has a reputation system that will help
|
||||
ensure that the renter is only dealing with upstanding hosts.
|
||||
|
||||
1. The renter makes an RPC to the host, opening a connection. The connection
|
||||
deadline should be at least 360 seconds.
|
||||
|
||||
2. The host sends the renter the most recent copy of its settings, signed. If
|
||||
the host is not accepting new file contracts, the connection is closed.
|
||||
|
||||
3. The renter sends a notice of acceptance or rejection. If the renter accepts,
|
||||
the renter then sends a funded file contract transaction without a
|
||||
signature, followed by the public key that will be used to create the
|
||||
renter's portion of the UnlockConditions for the file contract.
|
||||
|
||||
4. The host sends an acceptance or rejection of the file contract. If the host
|
||||
accepts, the host will add collateral to the file contract, and will send
|
||||
the renter the inputs + outputs for the collateral, followed by any new
|
||||
parent transactions. The length of any of these may be zero.
|
||||
|
||||
5. The renter indicates acceptance or rejection of the file contract. If the
|
||||
renter accepts, the renter will sign the file contract and send the
|
||||
transaction signatures to the host. The renter will also send a signature
|
||||
for a no-op file contract revision that follows the file contract.
|
||||
|
||||
6. The host may only reject the file contract in the event that the renter has
|
||||
sent invalid signatures, so the acceptance step is skipped. The host signs
|
||||
the file contract and sends the transaction signatures to the renter, and
|
||||
the host creates and sends a signature for the no-op revision that follows
|
||||
the file contract. The connection is closed.
|
||||
|
||||
File Contract Revision
|
||||
----------------------
|
||||
|
||||
1. The renter makes an RPC to the host, opening a connection. The minimum
|
||||
deadline for the connection is 600 seconds. The renter then sends a file
|
||||
contract ID, indicating the file contract that is getting revised during the
|
||||
RPC.
|
||||
|
||||
2. The host will respond with a 32 byte challenge - a random 32 bytes that the
|
||||
renter will need to sign.
|
||||
|
||||
3. The renter will sign the challenge with the renter key that protects the
|
||||
file contract. This is to prove that the renter has access to the file
|
||||
contract.
|
||||
|
||||
4. The host will verify the challenge signature, then send an acceptance or
|
||||
rejection. If accetped, the host will send the most recent file contract
|
||||
revision for the file contract along with the transaction signagtures that
|
||||
validate the revision. The host will lock the file contract, meaning no
|
||||
other changes can be made to the revision file contract until this
|
||||
connection has closed.
|
||||
|
||||
A loop begins. The host sends the most recent revision of the host settings
|
||||
to the renter, signed. The settings are sent after each iteration of the
|
||||
loop to enable high resolution dynamic pricing for the host, especially for
|
||||
bandwidth.
|
||||
|
||||
6. The renter may reject or accept the settings + revision. A specific
|
||||
rejection message will gracefully terminate the loop here. The renter will
|
||||
send an unsigned file contract revision followed by a batch of modification
|
||||
actions which the revision pays for. Batching allows the renter to send a
|
||||
lot of data in a single, one-way connection, improving throughput. The
|
||||
renter will send a number indicating how many modifications will be made in
|
||||
a batch, and then sends each modification in order.
|
||||
|
||||
A single modification can either be an insert, a modify, or a delete. An
|
||||
insert is an index, indicating the index where the data is going to be
|
||||
inserted. '0' indicates that the data is inserted at the very beginning, '1'
|
||||
indicates that the data will be inserted between the first and second
|
||||
existing sectors, etc. The index is followed by the 4MB of data. A modify is
|
||||
an index indicating which sector is being modified, followed by an offset
|
||||
indicating which data within the sector is being modified. Finally, some
|
||||
data is provided indicating what the data in the sector should be replaced
|
||||
with starting from that offset. The offset + len of the data should not
|
||||
exceed the sector size of 4MB. A delete is an index indicating the index of
|
||||
the sector that is being deleted. Each operation within a batch is atomic,
|
||||
meaning if you are inserting 3 sectors at the front of the file contract,
|
||||
the indexes of each should be '0', '1', '2'.
|
||||
|
||||
7. The host indicates either acceptance or rejection of the new revision.
|
||||
|
||||
8. The renter signs the revision and sends the signature to the host.
|
||||
|
||||
9. The host signs the revision and sends the siganture to the renter. Both
|
||||
parties submit the new revision to the transaction pool. The connection
|
||||
deadline is reset to 600 seconds (unless the maximum deadline has been
|
||||
reached), and the loop restarts.
|
||||
|
||||
File Contract Renewal
|
||||
---------------------
|
||||
|
||||
1. The renter makes an RPC to the host, opening a connection. The minimum
|
||||
deadline for the connection is 600 seconds. The renter then sends a file
|
||||
contract ID, indicating the file contract that is getting revised during the
|
||||
RPC.
|
||||
|
||||
2. The host will respond with a 32 byte challenge - a random 32 bytes that the
|
||||
renter will need to sign.
|
||||
|
||||
3. The renter will sign the challenge with the renter key that protects the
|
||||
file contract. This is to prove that the renter has access to the file
|
||||
contract.
|
||||
|
||||
4. The host will verify the challenge signature, then send an acceptance or
|
||||
rejection. If accetped, the host will send the most recent file contract
|
||||
revision for the file contract along with the transaction signagtures that
|
||||
validate the revision. The host will lock the file contract, meaning no
|
||||
other changes can be made to the revision file contract until this
|
||||
connection has closed. The host sends the most recent revision of the host
|
||||
settings to the renter, signed. If the host is not accepting new file
|
||||
contracts, the connection is closed.
|
||||
|
||||
5. The renter either accepts or rejects the settings. If accepted, the renter
|
||||
sends a funded, unsigned file contract to the host, containing the same
|
||||
Merkle root as the previous file contract, and also containing a renewed
|
||||
payout with conditional payments to the host to cover the host storing the
|
||||
data for the extended duration.
|
||||
|
||||
6. The host will accept or reject the renewed file contract. If accepted, the
|
||||
host will add collateral (and miner fees if desired) and send the inputs +
|
||||
outputs for the collateral, along with any new parent transactions. The
|
||||
length of any of these may be zero.
|
||||
|
||||
7. The renter will accept or reject the host's additions. If accepting, the
|
||||
renter will send signatures for the transaction to the host. The renter will
|
||||
also send a signature for a no-op file contract revision that follows the
|
||||
file contract.
|
||||
|
||||
8. The host may only reject the file contract in the event that the renter has
|
||||
sent invalid signatures, so the acceptance step is skipped. The host signs
|
||||
the file contract and sends the transaction signatures to the renter, and
|
||||
the host creates and sends a signature for the no-op revision that follows
|
||||
the file contract. The connection is closed.
|
||||
|
||||
Data Request
|
||||
------------
|
||||
|
||||
1. The renter makes an RPC to the host, opening a connection. The connection
|
||||
deadline is at least 600 seconds. The renter will send a file contract id
|
||||
corresponding to the file contract that will be used to pay for the
|
||||
download.
|
||||
|
||||
2. The host will respond with a 32 byte challenge - a random 32 bytes that the
|
||||
renter will need to sign.
|
||||
|
||||
3. The renter will sign the challenge with the public key that protects the
|
||||
file contract being used to pay for the download. This proves that the
|
||||
renter has access to the payment.
|
||||
|
||||
4. The host will verify the challenge signature, and then send an acceptance or
|
||||
rejection. If accepted, the host will send the most recent file contract
|
||||
revision followed by the signautres that validate the revision. The host
|
||||
will lock the file contract, preventing other connections from making
|
||||
changes to the underlying storage obligation.
|
||||
|
||||
A loop begins. The host sends the most recent external settings to the
|
||||
renter, signed. The settings are sent each iteration to provide high
|
||||
resolution dynamic bandwidth pricing.
|
||||
|
||||
5. The host will send the renter the most recent file contract revision, along
|
||||
with the signatures that validate the revision.
|
||||
|
||||
A loop begins, which will allow the renter to download multiple batches of
|
||||
data from the same connection. The host will send the host settings, and the
|
||||
most recent file contract revision transaction. If there is no revision yet,
|
||||
the host will send a blank transaction. The host is expected to always have
|
||||
the most recent revision (the host signs last), the renter may not have the
|
||||
most recent revision.
|
||||
|
||||
6. The renter will accept or reject the host's settings. If accepting, the
|
||||
renter will send a file contract revision, unsigned, to pay for the download
|
||||
request. The renter will then send the download request itself.
|
||||
|
||||
7. The host will either accept or reject the revision.
|
||||
|
||||
8. The renter will send a signature for the file contract revision.
|
||||
|
||||
9. The host sends a signature for the file contract revision, followed by the
|
||||
data that was requested by the download request. The loop starts over, and
|
||||
the connection deadline is reset to a minimum of 600 seconds.
|
||||
Reference in New Issue
Block a user