Store file contracts

[chris124567]

I did most of it but I was wondering how we wanted to handle valid/missed outputs before moving on. We can't just add them to siacoin_elements at the time of contract formation obviously. And if we just store the address/value then we'll break balance calculations etc.

We could do something like this:

// same for valid proof outputs
CREATE TABLE file_contract_missed_proof_outputs (
        contract_id INTEGER REFERENCES file_contract_elements(id) ON DELETE CASCADE NOT NULL,
        contract_order INTEGER NOT NULL,
        output_id INTEGER REFERENCES siacoin_elements(id) ON DELETE CASCADE NOT NULL,
        address BLOB NOT NULL,
        value BLOB NOT NULL,
        UNIQUE(contract_id, contract_order)
);

and set output_id to some invalid sentinel value by default but specify the address and value, then when the output is actually created we set output_id to the correct value and just refer to the data in siacoin_elements instead of the address and value columns.

[n8maninger]

The contract outputs will be added by ForEachSiacoinElement. Ignoring the output ID for the contract data should be fine.

[chris124567]

If I do:

func (s *Store) Contracts(ids []types.FileContractID) (result []explorer.FileContract, err error) {
    ...
    begin := time.Now()
    for i := 0; i < 10000; i++ {
        result = result[:0]
        err = s.transaction(func(tx txn) error {
            query := `SELECT fc1.id, fc1.contract_id, fc1.leaf_index, fc1.merkle_proof, fc1.resolved, fc1.valid, fc1.filesize, fc1.file_merkle_root, fc1.window_start, fc1.window_end, fc1.payout, fc1.unlock_hash, fc1.revision_number
            FROM file_contract_elements fc1
            INNER JOIN last_contract_revision rev ON (rev.contract_element_id = fc1.id)
            WHERE rev.contract_id IN (` + queryPlaceHolders(len(ids)) + `)`

            // query := `SELECT fc1.id, fc1.contract_id, fc1.leaf_index, fc1.merkle_proof, fc1.resolved, fc1.valid, fc1.filesize, fc1.file_merkle_root, fc1.window_start, fc1.window_end, fc1.payout, fc1.unlock_hash, fc1.revision_number
            // FROM file_contract_elements fc1
            // WHERE fc1.contract_id IN (` + queryPlaceHolders(len(ids)) + `)
            // AND fc1.revision_number = (SELECT max(revision_number) FROM file_contract_elements fc2 WHERE fc2.contract_id = fc1.contract_id)`
            rows, err := tx.Query(query, encodedIDs(ids)...)
            if err != nil {
                return err
            }
            defer rows.Close()

            var contractIDs []int64
            idContract := make(map[int64]explorer.FileContract)
            for rows.Next() {
                var contractID int64
                var fc explorer.FileContract
                if err := rows.Scan(&contractID, dbDecode(&fc.StateElement.ID), dbDecode(&fc.StateElement.LeafIndex), dbDecode(&fc.StateElement.MerkleProof), &fc.Resolved, &fc.Valid, &fc.Filesize, dbDecode(&fc.FileMerkleRoot), &fc.WindowStart, &fc.WindowEnd, dbDecode(&fc.Payout), dbDecode(&fc.UnlockHash), &fc.RevisionNumber); err != nil {
                    return fmt.Errorf("failed to scan transaction: %w", err)
                }

                idContract[contractID] = fc
                contractIDs = append(contractIDs, contractID)
            }

            proofOutputs, err := fileContractOutputs(tx, contractIDs)
            if err != nil {
                return fmt.Errorf("failed to get file contract outputs: %w", err)
            }
            for contractID, output := range proofOutputs {
                fc := idContract[contractID]
                fc.ValidProofOutputs = output.valid
                fc.MissedProofOutputs = output.missed
                result = append(result, fc)
            }

            return nil
        })
    }
    log.Println("Finished in:", time.Now().Sub(begin))
    ...
}

10000 iterations at block height of about 15000 with SELECT MAX on a random contract:

INFO    stdlib  Finished in: 2.639148135s
INFO    stdlib  Finished in: 2.661095619s
INFO    stdlib  Finished in: 2.610469021s

Average: 2.636s

10000 iterations at block height of about 15000 with lookup table on a random contract:

INFO    stdlib  Finished in: 1.508978577s
INFO    stdlib  Finished in: 1.46709396s
INFO    stdlib  Finished in: 1.472796309s

Average: 1.482s

So a lookup table is indeed faster 👍

[chris124567]

Should be OK to merge I think. Going to work on testing next.