Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feature: export leveldb parameter #204

Open
wants to merge 1 commit into
base: dev
Choose a base branch
from
Open

Feature: export leveldb parameter #204

wants to merge 1 commit into from

Conversation

0xcb9ff9
Copy link

Description

  • export CompactionTableSizeMultiplier, Control the file number of per-level
  • export Compression config (in btrfs/zfs system, transparent file compression better)

Changes include

  • Bugfix (non-breaking change that solves an issue)
  • Hotfix (change that solves an urgent issue, and requires immediate attention)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (change that is not backwards-compatible and/or changes current functionality)

Checklist

  • I have assigned this PR to myself
  • I have added at least 1 reviewer
  • I have added the relevant labels
  • I have updated the official documentation
  • I have added sufficient documentation in code

@0xcb9ff9 0xcb9ff9 added the feature New update to Dogechain label Sep 29, 2022
@0xcb9ff9 0xcb9ff9 self-assigned this Sep 29, 2022
DarianShawn
DarianShawn reviewed Sep 29, 2022
View reviewed changes
Copy link
Collaborator

@DarianShawn DarianShawn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure why we should expose more and more leveldb options here.
Shouldn't we provide higher level and more clear options to our users.
People don't care about leveldb or xxxdb exactly, they just need a well-inner-configured and stable storage.

@0xcb9ff9
Copy link
Author

leveldb optimization is complex, there is no silver bullet (especially the unmaintained leveldb library)

Using transparent file compression can save 60%+ storage space, but the default configuration of leveldb uses snappy compression, which will reduce the efficiency of transparent file compression, and the performance of the compression library used by goleveldb is not as good as the C library(used by the file system in the kernel)

Image

DarianShawn
DarianShawn reviewed Oct 8, 2022
View reviewed changes
server/config.go

CompactionTableSizeMultiplier float64

DisbableCompression bool
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After disable, all Compaction*** would not work?

Copy link
Author

@0xcb9ff9 0xcb9ff9 Oct 8, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no, this is leveldb snappy compression(storage data level), not related to Compaction*** (key-value level)

@DarianShawn
Copy link
Collaborator

Using transparent file compression can save 60%+ storage space, but the default configuration of leveldb uses snappy compression, which will reduce the efficiency of transparent file compression, and the performance of the compression library used by goleveldb is not as good as the C library(used by the file system in the kernel)

@abrahamcruise321 What's your opinion?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@DarianShawn DarianShawn DarianShawn left review comments

@abrahamcruise321 abrahamcruise321 Awaiting requested review from abrahamcruise321 abrahamcruise321 is a code owner

At least 1 approving review is required to merge this pull request.

Assignees

@0xcb9ff9 0xcb9ff9

Labels
feature New update to Dogechain
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@0xcb9ff9 @DarianShawn