Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PebbleDB #281

Closed
wants to merge 82 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
82 commits
Select commit Hold shift + click to select a range
16f0d61
add pebble
catShaark Apr 13, 2022
5c9bfe8
fix: remove mutex in prefixdb
kjessec Mar 4, 2022
a0ff309
fmt
faddat Apr 25, 2022
636377b
fmt
faddat Apr 25, 2022
0155808
Delete .gitpod.yml
faddat Apr 25, 2022
973a74a
remove mutex entirely instead of commenting out
faddat Apr 25, 2022
34ca0e7
fmt
faddat Apr 25, 2022
d81b67b
remove .gitpod.yml
faddat Apr 25, 2022
4c9b066
update changelog
faddat Apr 25, 2022
0566540
Merge branch 'master' into remove-mutex
faddat Apr 25, 2022
51bdeca
Merge https://github.com/tendermint/tm-db
faddat Apr 29, 2022
f531a3f
prefixdb test
faddat May 6, 2022
cc11706
all test passed in prefixdb_test.go
vuong177 May 6, 2022
417a583
Merge branch 'master' into remove-mutex
catShaark May 6, 2022
3214560
simple taskKey in prefixdb_test.go
vuong177 May 6, 2022
fd0bb3d
Update prefixdb.go
faddat May 6, 2022
31c47b6
Merge branch 'master' into remove-mutex
faddat May 6, 2022
835d8fd
fmt db test so pr passes linter
faddat May 7, 2022
cb000ee
refactor func Run
vuong177 May 7, 2022
c6d827f
check nil err before create PrefixDB in TestWithGolevelDB
vuong177 May 7, 2022
b78aed6
add other db tests
catShaark May 8, 2022
e2c3c4e
Update prefixdb_test.go
faddat May 9, 2022
d56e038
Merge branch 'master' into remove-mutex
faddat May 9, 2022
84a67af
update prefixdb_test.go
catShaark May 10, 2022
72c47a4
minor defer
vuong177 May 11, 2022
0931c90
Merge branch 'master' into remove-mutex
faddat May 19, 2022
5c81bf8
fixup with build tags
faddat May 19, 2022
11fa71a
Create .gitpod.yml
faddat May 19, 2022
d57bb1f
fix linting
faddat May 19, 2022
8e4e401
itr.isInvalid would be unconditionally set anyhow, which is why the i…
faddat May 19, 2022
602ef89
Merge branch 'master' into remove-mutex
faddat May 20, 2022
d4bb034
Update cleveldb_test.go
faddat May 23, 2022
2640477
Update prefixdb_test.go
faddat May 23, 2022
d75f0c0
Update prefixdb_test.go
faddat May 23, 2022
4355c2a
Merge branch 'master' into remove-mutex
faddat Jun 4, 2022
f330226
Firing this into CI, let's see.
faddat Jun 4, 2022
230b7cc
add filepath lib to tests
faddat Jun 4, 2022
cdc2374
fix filepath in goleveldb
faddat Jun 4, 2022
e74373d
gofumpt formatting
faddat Jun 4, 2022
1ab7644
Merge branch 'master' into remove-mutex
faddat Jun 15, 2022
ea504e8
Merge branch 'master' into remove-mutex
faddat Jun 20, 2022
7d4b2c7
Merge branch 'master' into remove-mutex
faddat Jun 28, 2022
f39f7bb
Merge branch 'master' into remove-mutex
faddat Jul 15, 2022
d91a34e
regenerate certs
faddat Jul 26, 2022
3d7247e
pebble
faddat Jul 26, 2022
8e9d5b9
heavy iterator testing for pebble
faddat Jul 27, 2022
33bb0c6
separate pebble
faddat Jul 27, 2022
4c28d6d
Merge remote-tracking branch 'origin/fix-main' into pebble-again
faddat Jul 27, 2022
6a41b77
update pebble upstream lib
faddat Jul 27, 2022
e52e865
Update ci.yml
faddat Jul 27, 2022
809d1e6
Merge remote-tracking branch 'origin/remove-mutex' into pebble-again
faddat Jul 27, 2022
2224915
lower pebble to 1000
faddat Jul 27, 2022
3f944b0
Update ci.yml
faddat Jul 27, 2022
d975760
Update pebble_test.go
faddat Jul 27, 2022
ae54df1
deps for pebble test
faddat Jul 27, 2022
e706d11
more pebble iterator testing to catch the issue
faddat Jul 27, 2022
baa168f
Update ci.yml
faddat Jul 27, 2022
944db9c
Update ci.yml
faddat Jul 27, 2022
50279cd
100k iterator tests per commit
faddat Jul 27, 2022
cb6cbed
more pebble testing
faddat Jul 27, 2022
5d72869
Update pebble.go
faddat Jul 27, 2022
c6f54cf
more pebble testing
faddat Jul 27, 2022
7e9694a
use the pebble validity check before attempting to close
faddat Jul 27, 2022
c5063cc
revert changes to go code
faddat Jul 27, 2022
bc8ec5d
enable config
faddat Jul 27, 2022
e671259
fewer files
faddat Jul 27, 2022
f4b1fd6
Update pebble.go
faddat Jul 27, 2022
583ae5b
Update pebble.go
faddat Jul 27, 2022
b062571
pebble
faddat Jul 27, 2022
aee474a
change iterator style
faddat Jul 27, 2022
6148c8a
notes for participants
faddat Jul 27, 2022
bef0c1d
give clearer layout
faddat Jul 27, 2022
bf52cd1
pebble
faddat Jul 27, 2022
9b3e704
update docs and re-organize code
faddat Jul 27, 2022
137479e
default settings
faddat Jul 27, 2022
6c40221
default settings
faddat Jul 27, 2022
8e04354
default settings
faddat Jul 27, 2022
dce8ebe
go 1.18
faddat Jul 27, 2022
83c02f6
randos
faddat Jul 27, 2022
90a4aef
remove badger and bolt
faddat Jul 27, 2022
d2b550d
remove badger and bolt
faddat Jul 27, 2022
14b534a
fewer in each run
faddat Jul 28, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
424 changes: 422 additions & 2 deletions .github/workflows/ci.yml

Large diffs are not rendered by default.

7 changes: 7 additions & 0 deletions .gitpod.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# This configuration file was automatically generated by Gitpod.
# Please adjust to your needs (see https://www.gitpod.io/docs/config-gitpod-file)
# and commit this file to your remote git repository to share the goodness with others.

image: tendermintdev/docker-tm-db-testing

# this means that there's a one-click known good environment available to developers.
6 changes: 6 additions & 0 deletions .golangci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,12 @@ linters:
# - gocognit
- nolintlint

run:
build-tags:
- cleveldb
- rocksdb
- boltdb

issues:
exclude-rules:
- path: _test\.go
Expand Down
7 changes: 7 additions & 0 deletions .vscode/settings.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
{
"makefile.extensionOutputFolder": "./.vscode",
"go.buildTags": "pebbledb goleveldb cleveldb rocksdb",
"go.lintTool": "golangci-lint",
"go.lintOnSave": "workspace",
"go.inferGopath": false
}
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
# Changelog

## Unreleased

- remove mutex from prefixdb

## 0.6.7

**2022-2-21**
Expand Down
6 changes: 1 addition & 5 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Tendermint DB
# Tendermint DB

[![version](https://img.shields.io/github/tag/tendermint/tm-db.svg)](https://github.com/tendermint/tm-db/releases/latest)
[![license](https://img.shields.io/github/license/tendermint/tm-db.svg)](https://github.com/tendermint/tm-db/blob/master/LICENSE)
Expand All @@ -22,12 +22,8 @@ Go 1.13+

- **[LevelDB](https://github.com/google/leveldb) [experimental]:** A [Go wrapper](https://github.com/jmhodges/levigo) around [LevelDB](https://github.com/google/leveldb). Uses LSM-trees for on-disk storage, which have good performance for write-heavy workloads, particularly on spinning disks, but requires periodic compaction to maintain decent read performance and reclaim disk space. Does not support transactions.

- **[BoltDB](https://github.com/etcd-io/bbolt) [experimental]:** A [fork](https://github.com/etcd-io/bbolt) of [BoltDB](https://github.com/boltdb/bolt). Uses B+trees for on-disk storage, which have good performance for read-heavy workloads and range scans. Supports serializable ACID transactions.

- **[RocksDB](https://github.com/cosmos/gorocksdb) [experimental]:** A [Go wrapper](https://github.com/cosmos/gorocksdb) around [RocksDB](https://rocksdb.org). Similarly to LevelDB (above) it uses LSM-trees for on-disk storage, but is optimized for fast storage media such as SSDs and memory. Supports atomic transactions, but not full ACID transactions.

- **[BadgerDB](https://github.com/dgraph-io/badger) [experimental]:** A key-value database written as a pure-Go alternative to e.g. LevelDB and RocksDB, with LSM-tree storage. Makes use of multiple goroutines for performance, and includes advanced features such as serializable ACID transactions, write batches, compression, and more.

## Meta-databases

- **PrefixDB [stable]:** A database which wraps another database and uses a static prefix for all keys. This allows multiple logical databases to be stored in a common underlying databases by using different namespaces. Used by the Cosmos SDK to give different modules their own namespaced database in a single application database.
Expand Down
1 change: 0 additions & 1 deletion backend_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -315,7 +315,6 @@ func testDBIterator(t *testing.T, backend BackendType) {
ritr, err = db2.ReverseIterator(nil, nil)
require.NoError(t, err)
verifyIterator(t, ritr, nil, "reverse iterator with empty db")

}

func verifyIterator(t *testing.T, itr Iterator, expected []int64, msg string) {
Expand Down
295 changes: 0 additions & 295 deletions badger_db.go

This file was deleted.

Loading