Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add redundancy options #487

Merged
merged 2 commits into from
Apr 15, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
84 changes: 54 additions & 30 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@
"typescript": "^4.8.4"
},
"dependencies": {
"@ethersphere/bee-js": "^6.7.3",
"@ethersphere/bee-js": "^6.9.0",
"@fairdatasociety/bmt-js": "^2.1.0",
"bignumber.js": "^9.1.0",
"chalk": "^2.4.2",
Expand Down
61 changes: 60 additions & 1 deletion src/command/upload.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { Tag, Utils } from '@ethersphere/bee-js'
import { RedundancyLevel, Tag, Utils } from '@ethersphere/bee-js'
import { Presets, SingleBar } from 'cli-progress'
import * as FS from 'fs'
import { Argument, LeafCommand, Option } from 'furious-commander'
Expand Down Expand Up @@ -104,6 +104,12 @@
})
public contentType!: string

@Option({
key: 'redundancy',
description: 'Redundancy of the upload (MEDIUM, STRONG, INSANE, PARANOID)',
})
public redundancy!: string

// CLASS FIELDS

public hash!: string
Expand Down Expand Up @@ -140,6 +146,7 @@
}

await this.maybeRunSizeChecks()
await this.maybePrintRedundancyStats()

const tag = this.sync ? await this.bee.createTag() : undefined

Expand Down Expand Up @@ -211,6 +218,7 @@
encrypt: this.encrypt,
contentType,
deferred: this.deferred,
redundancyLevel: this.determineRedundancyLevel(),
})
this.hash = reference

Expand All @@ -219,6 +227,8 @@
const { reference } = await this.bee.uploadData(this.stamp, this.stdinData, {
tag: tag?.uid,
deferred: this.deferred,
encrypt: this.encrypt,
redundancyLevel: this.determineRedundancyLevel(),
})
this.hash = reference

Expand All @@ -239,6 +249,7 @@
pin: this.pin,
encrypt: this.encrypt,
deferred: this.deferred,
redundancyLevel: this.determineRedundancyLevel(),
})
this.hash = reference

Expand All @@ -260,6 +271,7 @@
encrypt: this.encrypt,
contentType,
deferred: this.deferred,
redundancyLevel: this.determineRedundancyLevel(),
})
this.hash = reference

Expand Down Expand Up @@ -340,6 +352,35 @@
}
}

private async maybePrintRedundancyStats(): Promise<void> {
if (!this.redundancy || this.quiet) {
return
}

const currentSetting = Utils.getRedundancyStat(this.redundancy)
const originalSize = await this.getUploadSize()
const originalChunks = Math.ceil(originalSize.getBytes() / 4e3)
const sizeMultiplier = Utils.approximateOverheadForRedundancyLevel(
originalChunks,
currentSetting.value,
this.encrypt,
)
const newSize = new Storage(originalChunks * 4e3 * (1 + sizeMultiplier))
const extraSize = new Storage(newSize.getBytes() - originalSize.getBytes())

this.console.log(createKeyValue('Redundancy setting', currentSetting.label))
this.console.log(`This setting will provide ${Math.round(currentSetting.errorTolerance * 100)}% error tolerance.`)
this.console.log(`An additional ${extraSize.toString()} of data will be uploaded approximately.`)
this.console.log(`${originalSize.toString()} → ${newSize.toString()} (+${extraSize.toString()})`)
if (!this.yes && !this.quiet) {

Check failure on line 375 in src/command/upload.ts

View workflow job for this annotation

GitHub Actions / check (18.x)

Expected blank line before this statement
const confirmation = await this.console.confirm('Do you want to proceed?')

if (!confirmation) {
exit(0)
}
}
}

private async getUploadSize(): Promise<Storage> {
let size = -1

Expand Down Expand Up @@ -424,4 +465,22 @@

return defaultName
}

private determineRedundancyLevel(): RedundancyLevel | undefined {
if (!this.redundancy) {
return undefined
}
switch (this.redundancy.toUpperCase()) {
case 'MEDIUM':
return RedundancyLevel.MEDIUM
case 'STRONG':
return RedundancyLevel.STRONG
case 'INSANE':
return RedundancyLevel.INSANE
case 'PARANOID':
return RedundancyLevel.PARANOID
default:
throw new CommandLineError(`Invalid redundancy level: ${this.redundancy}`)
}
}
}
Loading