[Client Tooling] Keybase (issue #455) (#459)

## Description

This is an implementation of a Keybase for the pocket V1 client.

It uses `BadgerDB` as the local database for the persistent storage of
keys. The current `Keybase` interface has the following methods


![image](https://user-images.githubusercontent.com/53987565/214436549-efcf0895-ddf1-477a-8624-225ab094c713.png)

These are covered with the following tests


![image](https://user-images.githubusercontent.com/53987565/214436651-c32da614-df53-416d-9753-25c8fcb261eb.png)

The tests use an in-memory database created with `NewKeybaseInMemory()`
whereas when exposed to the user the CLI should call `NewKeybase()`
instead with a default path/user-supplied path. The CLI integration has
not been implemented as part of this PR however.

### How it works

The keybase heavily relies upon `shared/crypto/ed25519.go` and the
`PublicKey` and `PrivateKey` interfaces it exposes.

The keys are created using the methods from this library. Keys are then
encrypted and armoured in the same fashion as in V0 and stored in the DB
as a `KeyPair` struct (encoded to a `[]byte`).


![image](https://user-images.githubusercontent.com/53987565/214108865-bdb8f184-a1ff-4524-b978-af6e2b8a41c9.png)

This struct contains the public key and the encrypted JSON encoded
private key string. This struct has the following methods:
- `GetAddressBytes() []byte` -> returns `[]byte` of the public key
address
- `GetAddressString() string` -> returns the hex `string` of the public
key address
- `Unarmour(passphrase string) (crypto.PrivateKey, error)` -> returns
the unencrypted unarmoured private key if passphrase is correct
- `ExportString(passphrase string) (string, error)` -> returns raw
private key string is passphrase is correct
- `ExportJSON(passphrase string) (string, error)` -> returns json
armoured private key string if the passphrase is correct

Keys are stored in the database in the following manner:
- The keys for the BadgerDB key-value storage are the `[]byte` value
returned by `KeyPair.GetAddressBytes()` aka the `[]byte` address of the
public key
- The values are the `[]byte` encoded `KeyPair` structs of the key

Key encryption/decryption works with or without a passphrase (when no
passphrase is given `""` must still be passed to the function as the
passphrase argument)

For ease of use the hex `string` returned by
`KeyPair.GetAddressString()` is used to access the keys in storage.

Importing and exporting keys in either JSON string format or the private
key hex string is fully interoperable between V1 and V0

Signing and Verification of messages works - and is covered with a Tx in
the tests. I would need to look into the use cases for this more (for
example multisig) as the current implementation is very rudimentary and
in theory should work for producing a signature on any `[]byte` message
but I would like to look into this more.

## Issue

Fixes #455 

## Type of change

Please mark the relevant option(s):

- [x] New feature, functionality or library
- [ ] Bug fix
- [ ] Code health or cleanup
- [ ] Major breaking change
- [x] Documentation
- [ ] Other <!-- add details here if it a different type of change -->

## List of changes

- Implement a keybase with BadgerDB local DB
- Add functionality to Import/Export keys, Get/List keys, Create new
keys, Delete Keys, Sign Messages, Verify Signatures
- Add `make test_app` entry point in Makefile
- Add unit tests to cover Keybase use cases
- Add documentation for Keybase

## Testing

- [x] `make develop_test`
- [x]
[LocalNet](https://github.com/pokt-network/pocket/blob/main/docs/development/README.md)
w/ all of the steps outlined in the `README`

## Required Checklist

- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have tested my changes using the available tooling
- [x] I have updated the corresponding CHANGELOG

### If Applicable Checklist
- [x] I have updated the corresponding README(s); local and/or global
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] I have added, or updated,
[mermaid.js](https://mermaid-js.github.io) diagrams in the corresponding
README(s)
- [x] I have added, or updated, documentation and
[mermaid.js](https://mermaid-js.github.io) diagrams in `shared/docs/*`
if I updated `shared/*`README(s)

---------

Co-authored-by: Daniel Olshansky <[email protected]>

Makefile

@@ -312,6 +312,10 @@ test_all_with_json_coverage: generate_rpc_openapi ## Run all go unit tests, outp
 test_race: ## Identify all unit tests that may result in race conditions
 	go test ${VERBOSE_TEST} -race ./...
 
+.PHONY: test_app
+test_app: ## Run all go app module unit tests
+	go test ${VERBOSE_TEST} -p=1 -count=1  ./app/...
+
 .PHONY: test_utility
 test_utility: ## Run all go utility module unit tests
 	go test ${VERBOSE_TEST} -p=1 -count=1  ./utility/...

app/client/cli/doc/CHANGELOG.md → app/client/doc/CHANGELOG.md

@@ -7,6 +7,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.0.0.5] - 2023-02-02
+
+### Added
+
+- Create `Keybase` interface to handle CRUD operations for `KeyPairs` with a `BadgerDB` backend
+- Add logic to create, import, export, list, delete and update (passphrase) key pairs
+- Add logic to sign and verify arbitrary messages
+- Add unit tests for the keybase
+
 ## [0.0.0.4] - 2023-01-10
 
 - The `client` (i.e. CLI) no longer instantiates a `P2P` module along with a bus of optional modules. Instead, it instantiates a `client-only` `P2P` module that is disconnected from consensus and persistence. Interactions with the persistence & consensus layer happen via RPC.

app/client/keybase/README.md

@@ -0,0 +1,81 @@
+# Keybase <!-- omit in toc -->
+
+This document is intended to outline the current Keybase implementation used by the V1 client, and is primarily focused on its design and implementation as well as testing.
+
+- [Backend Database](#backend-database)
+- [Keybase Interface](#keybase-interface)
+  - [V0\<-\>V1 Interoperability](#v0-v1-interoperability)
+  - [Keybase Code Structure](#keybase-code-structure)
+- [Makefile Testing Helper](#makefile-testing-helper)
+- [KeyPair Encryption \& Armouring](#keypair-encryption--armouring)
+- [TODO: Future Work](#todo-future-work)
+
+_TODO(#150): The current keybase has not been integrated with any CLI endpoints, and as such is only accessible through the [keybase interface](#keybase-interface)_
+
+## Backend Database
+
+The Keybase package uses a filesystem key-value database, `BadgerDB`, as its backend to persistently store keys locally on the client machine. The DB stores the local keys encoded as `[]byte` using `encoding/gob`.
+
+The `KeyPair` defined in [crypto package](../../../shared/core/crypto) is the data structure that's stored in the DB. Specifically:
+
+- **Key**: The `[]byte` returned by the `GetAddressBytes()` function is used as the key in the key-value store.
+- **Value**: The `gob` encoded struct of the entire `KeyPair`, containing both the `PublicKey` and `PrivKeyArmour` (JSON encoded, encrypted private key string), is the value.
+
+The Keybase DB layer exposes several functions, defined by the [Keybase interface](#keybase-interface), to fulfill CRUD operations on the DB itself and oeprate with the Keypairs.
+
+## Keybase Interface
+
+The [Keybase interface](./keybase.go) exposes the CRUD operations to operate on keys, and supports the following operations:
+
+- Create password protected private keys
+- Export/Import string/json keypairs
+- Retrieve public/private keys or keypairs
+- List all keys stored
+- Check keys exist in the keybase
+- Update passphrase on a private key
+- Message signing and verification
+
+### V0<->V1 Interoperability
+
+The `Keybase` interface supports full interoperability of key export & import between Pocket [V0](https://github.com/pokt-network/pocket-core)<->[V1](https://github.com/pokt-network/pocket).
+
+Any private key created in the V0 protocol can be imported into V1 via one of the following two ways:
+
+1. **JSON keyfile**: This method will take the JSON encoded, encrypted private key, and will import it into the V1 keybase. The `passphrase` supplied must be the same as the one use to encrypt the key in the first place or the key won't be importable.
+
+2. **Private Key Hex String**: This method will directly import the private key from the hex string provided and encrypt it with the passphrase provided. This enables the passphrase to be different from the original as the provided plaintext is already decrypted.
+
+Although key pairs are stored in the local DB using the serialized (`[]byte`) representation of the public key, the associated address can be used for accessing the record in the DB for simplicity.
+
+Keys can be created without a password by specifying an empty (`""`) passphrase. The private key will still be encrypted at rest but will use the empty string as the passphrase for decryption.
+
+### Keybase Code Structure
+
+```bash
+app
+└── client
+    └── keybase
+          ├── README.md
+          ├── keybase.go
+          ├── keybase_test.go
+          └── keystore.go
+```
+
+The interface is found in [keybase.go](./keybase.go) whereas its implementation can be found in [keystore.go](./keystore.go)
+
+## Makefile Testing Helper
+
+The unit tests for the keybase are defined in [keybase_test.go](./keybase_test.go) and can therefore be executed alongside other application specific tests by running `make test_app`.
+
+## KeyPair Encryption & Armouring
+
+The [documentation in the crypto library](../../../shared/crypto/README.md) covers all of the details related to the `KeyPair` interface, as well as `PrivateKey` encryption, armouring and unarmouring.
+
+The primitives and functions defined there are heavily used throughout this package.
+
+## TODO: Future Work
+
+- [ ] Improve error handling and error messages for importing keys with invalid strings/invalid JSON
+- [ ] Research and implement threshold signatures and threshold keys
+- [ ] Look into a fully feature signature implementation beyond trivial `[]byte` messages
+- [ ] Integrate the keybase with the CLI (#150)

app/client/keybase/keybase.go

@@ -0,0 +1,37 @@
+package keybase
+
+import "github.com/pokt-network/pocket/shared/crypto"
+
+// Keybase interface implements the CRUD operations for the keybase
+type Keybase interface {
+	// Close the DB connection
+	Stop() error
+
+	// Create new keypair entry in DB
+	Create(passphrase, hint string) error
+	// Insert a new keypair from the private key hex string provided into the DB
+	ImportFromString(privStr, passphrase, hint string) error
+	// Insert a new keypair from the JSON string of the encrypted private key into the DB
+	ImportFromJSON(jsonStr, passphrase string) error
+
+	// Accessors
+	Get(address string) (crypto.KeyPair, error)
+	GetPubKey(address string) (crypto.PublicKey, error)
+	GetPrivKey(address, passphrase string) (crypto.PrivateKey, error)
+	GetAll() (addresses []string, keyPairs []crypto.KeyPair, err error)
+	Exists(address string) (bool, error)
+
+	// Exporters
+	ExportPrivString(address, passphrase string) (string, error)
+	ExportPrivJSON(address, passphrase string) (string, error)
+
+	// Updator
+	UpdatePassphrase(address, oldPassphrase, newPassphrase, hint string) error
+
+	// Sign Messages
+	Sign(address, passphrase string, msg []byte) ([]byte, error)
+	Verify(address string, msg, sig []byte) (bool, error)
+
+	// Removals
+	Delete(address, passphrase string) error
+}