Skip to content

Commit

Permalink
progress: Task(s) progress tracker (#39)
Browse files Browse the repository at this point in the history
  • Loading branch information
jedib0t authored May 20, 2018
1 parent 4c495c9 commit 3734d9f
Show file tree
Hide file tree
Showing 18 changed files with 1,650 additions and 9 deletions.
49 changes: 42 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,8 @@ Pretty-print tables into ASCII/Unicode strings.
+-----+------------+-----------+--------+-----------------------------+
```

A demonstration of all the capabilities can be found here: [cmd/demo-table](cmd/demo-table)
A demonstration of all the capabilities can be found here:
[cmd/demo-table](cmd/demo-table)

## List

Expand Down Expand Up @@ -75,7 +76,40 @@ Pretty-print lists with multiple levels/indents into ASCII/Unicode strings.
■ The Gunslinger
```

A demonstration of all the capabilities can be found here: [cmd/demo-list](cmd/demo-list)
A demonstration of all the capabilities can be found here:
[cmd/demo-list](cmd/demo-list)

# Progress

Track the Progress of one or more Tasks (like downloading multiple files in
parallel).

- Track one or more Tasks at the same time
- Dynamically add one or more Task Trackers while `Render()` is in progress
- Choose to have the Writer auto-stop the Render when no more Trackers are
in queue, or manually stop using `Stop()`
- Redirect output to an io.Writer object (like os.StdOut)
- Completely customizable styles
- Many ready-to-use styles: [progress/style.go](progress/style.go)
- Colorize various parts of the Tracker using `StyleColors`
- Customize how Trackers get rendered using `StyleOptions`

Sample Progress Tracking:
```
Calculating Total # 1 ... done! [3.25K in 100ms]
Calculating Total # 2 ... done! [6.50K in 100ms]
Downloading File # 3 ... done! [9.75KB in 100ms]
Transferring Amount # 4 ... done! [$26.00K in 200ms]
Transferring Amount # 5 ... done! [£32.50K in 201ms]
Downloading File # 6 ... done! [58.50KB in 300ms]
Calculating Total # 7 ... done! [91.00K in 400ms]
Transferring Amount # 8 ... 60.9% (●●●●●●●●●●●●●●◌◌◌◌◌◌◌◌◌) [$78.00K in 399.071ms]
Downloading File # 9 ... 32.1% (●●●●●●●○◌◌◌◌◌◌◌◌◌◌◌◌◌◌◌) [58.50KB in 298.947ms]
Transferring Amount # 10 ... 13.0% (●●○◌◌◌◌◌◌◌◌◌◌◌◌◌◌◌◌◌◌◌◌) [£32.50K in 198.84ms]
```

A demonstration of all the capabilities can be found here:
[cmd/demo-progress](cmd/demo-progress)

## Text

Expand All @@ -97,9 +131,10 @@ The unit-tests for each of the above show how these are to be used.

Partial output of `make bench`:
```
BenchmarkList_Render-8 1000000 1836 ns/op 808 B/op 22 allocs/op
BenchmarkTable_Render-8 100000 20736 ns/op 5426 B/op 191 allocs/op
BenchmarkTable_RenderCSV-8 300000 4394 ns/op 2336 B/op 45 allocs/op
BenchmarkTable_RenderHTML-8 200000 6563 ns/op 3793 B/op 44 allocs/op
BenchmarkTable_RenderMarkdown-8 300000 4666 ns/op 2272 B/op 43 allocs/op
BenchmarkList_Render-8 1000000 1848 ns/op 808 B/op 22 allocs/op
BenchmarkProgress_Render-8 2 800904500 ns/op 8832 B/op 373 allocs/op
BenchmarkTable_Render-8 100000 21025 ns/op 5538 B/op 188 allocs/op
BenchmarkTable_RenderCSV-8 300000 4507 ns/op 2464 B/op 45 allocs/op
BenchmarkTable_RenderHTML-8 200000 6471 ns/op 3921 B/op 44 allocs/op
BenchmarkTable_RenderMarkdown-8 300000 4720 ns/op 2400 B/op 43 allocs/op
```
26 changes: 26 additions & 0 deletions bench_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,11 @@ import (
"testing"

"github.com/jedib0t/go-pretty/list"
"github.com/jedib0t/go-pretty/progress"
"github.com/jedib0t/go-pretty/table"
"github.com/jedib0t/go-pretty/text"
"io/ioutil"
"time"
)

var (
Expand All @@ -21,6 +24,9 @@ var (
{20, "Jon", "Snow", 2000, "You know nothing, Jon Snow!"},
{300, "Tyrion", "Lannister", 5000},
}
tracker1 = progress.Tracker{Message: "Calculation Total # 1", Total: 1000, Units: progress.UnitsDefault}
tracker2 = progress.Tracker{Message: "Downloading File # 2", Total: 1000, Units: progress.UnitsBytes}
tracker3 = progress.Tracker{Message: "Transferring Amount # 3", Total: 1000, Units: progress.UnitsCurrencyDollar}
)

func generateBenchmarkTable() table.Writer {
Expand All @@ -45,6 +51,26 @@ func BenchmarkList_Render(b *testing.B) {
}
}

func BenchmarkProgress_Render(b *testing.B) {
trackSomething := func(pw progress.Writer, tracker *progress.Tracker) {
tracker.Reset()
pw.AppendTracker(tracker)
time.Sleep(time.Millisecond * 500)
tracker.Increment(tracker.Total)
}

for i := 0; i < b.N; i++ {
pw := progress.NewWriter()
pw.SetAutoStop(true)
pw.SetOutputWriter(ioutil.Discard)
go trackSomething(pw, &tracker1)
go trackSomething(pw, &tracker2)
go trackSomething(pw, &tracker3)
time.Sleep(time.Millisecond * 50)
pw.Render()
}
}

func BenchmarkTable_Render(b *testing.B) {
for i := 0; i < b.N; i++ {
generateBenchmarkTable().Render()
Expand Down
24 changes: 24 additions & 0 deletions cmd/demo-progress/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
Output of `go run cmd/demo-list/demo.go`:

```
Tracking Progress of 13 trackers ...
Calculating Total # 1 ... done! [250 in 101ms]
Calculating Total # 2 ... done! [2.00K in 101ms]
Downloading File # 3 ... done! [6.75KB in 101ms]
Transferring Amount # 4 ... done! [$16.00K in 200ms]
Transferring Amount # 5 ... done! [£31.25K in 201ms]
Downloading File # 6 ... done! [54.00KB in 300ms]
Calculating Total # 7 ... done! [85.75K in 400ms]
Transferring Amount # 8 ... done! [$128.00K in 500ms]
Downloading File # 9 ... done! [182.25KB in 700ms]
Transferring Amount # 10 ... done! [£250.00K in 801ms]
Calculating Total # 11 ... done! [332.75K in 1s]
Transferring Amount # 12 ... done! [$432.00K in 1.2s]
Calculating Total # 13 ... done! [549.25K in 1.301s]
All done!
```

Real-time playback of the output @ asciinema.org:
[![asciicast](https://asciinema.org/a/KcPw8aoBSsYCBOj60wluhu5z3.png)](https://asciinema.org/a/KcPw8aoBSsYCBOj60wluhu5z3)
102 changes: 102 additions & 0 deletions cmd/demo-progress/demo.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
package main

import (
"flag"
"fmt"
"time"

"github.com/jedib0t/go-pretty/progress"
)

var (
autoStop = flag.Bool("auto-stop", false, "Auto-stop rendering?")
numTrackers = flag.Int64("num-trackers", 13, "Number of Trackers")
)

func trackSomething(pw progress.Writer, idx int64) {
total := idx * idx * idx * 250
incrementPerCycle := idx * (*numTrackers) * 250

var units progress.Units
switch {
case idx%5 == 0:
units = progress.UnitsCurrencyPound
case idx%4 == 0:
units = progress.UnitsCurrencyDollar
case idx%3 == 0:
units = progress.UnitsBytes
default:
units = progress.UnitsDefault
}

var message string
switch units {
case progress.UnitsBytes:
message = fmt.Sprintf("Downloading File #%3d", idx)
case progress.UnitsCurrencyDollar, progress.UnitsCurrencyEuro, progress.UnitsCurrencyPound:
message = fmt.Sprintf("Transferring Amount #%3d", idx)
default:
message = fmt.Sprintf("Calculating Total #%3d", idx)
}
tracker := progress.Tracker{Message: message, Total: total, Units: units}

pw.AppendTracker(&tracker)

c := time.Tick(time.Millisecond * 100)
for !tracker.IsDone() {
select {
case <-c:
tracker.Increment(incrementPerCycle)
}
}
}

func main() {
flag.Parse()
fmt.Printf("Tracking Progress of %d trackers ...\n\n", *numTrackers)

// instantiate a Progress Writer and set up the options
pw := progress.NewWriter()
pw.SetAutoStop(*autoStop)
pw.SetTrackerLength(25)
pw.ShowTime(true)
pw.ShowTracker(true)
pw.ShowValue(true)
pw.SetSortBy(progress.SortByPercentDsc)
pw.SetStyle(progress.StyleCircle)
pw.SetTrackerPosition(progress.PositionRight)
pw.SetUpdateFrequency(time.Millisecond * 100)
pw.Style().Colors = progress.StyleColorsExample
pw.Style().Options.PercentFormat = "%4.1f%%"

// call Render() in async mode; yes we don't have any trackers at the moment
go pw.Render()

// add a bunch of trackers with random parameters to demo most of the
// features available; do this in async too like a client might do (for ex.
// when downloading a bunch of files in parallel)
for idx := int64(1); idx <= *numTrackers; idx++ {
go trackSomething(pw, idx)

// in auto-stop mode, the Render logic terminates the moment it detects
// zero active trackers; but in a manual-stop mode, it keeps waiting and
// is a good chance to demo trackers being added dynamically while other
// trackers are active or done
if !*autoStop {
time.Sleep(time.Millisecond * 100)
}
}

// wait for one or more trackers to become active (just blind-wait for a
// second) and then keep watching until Rendering is in progress
time.Sleep(time.Second)
for pw.IsRenderInProgress() {
// for manual-stop mode, stop when there are no more active trackers
if !*autoStop && pw.LengthActive() == 0 {
pw.Stop()
}
time.Sleep(time.Millisecond * 100)
}

fmt.Println("\nAll done!")
}
60 changes: 60 additions & 0 deletions cmd/profile-progress/profile.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
package main

import (
"fmt"
"io/ioutil"
"os"
"strconv"
"time"

"github.com/jedib0t/go-pretty/progress"
"github.com/pkg/profile"
)

var (
tracker1 = progress.Tracker{Message: "Calculation Total # 1", Total: 1000, Units: progress.UnitsDefault}
tracker2 = progress.Tracker{Message: "Downloading File # 2", Total: 1000, Units: progress.UnitsBytes}
tracker3 = progress.Tracker{Message: "Transferring Amount # 3", Total: 1000, Units: progress.UnitsCurrencyDollar}
profilers = []func(*profile.Profile){
profile.CPUProfile,
profile.MemProfileRate(512),
}
)

func profileRender(profiler func(profile2 *profile.Profile), n int) {
defer profile.Start(profiler, profile.ProfilePath("./")).Stop()

trackSomething := func(pw progress.Writer, tracker *progress.Tracker) {
tracker.Reset()
pw.AppendTracker(tracker)
time.Sleep(time.Millisecond * 500)
tracker.Increment(tracker.Total)
}

for i := 0; i < n; i++ {
pw := progress.NewWriter()
pw.SetAutoStop(true)
pw.SetOutputWriter(ioutil.Discard)
go trackSomething(pw, &tracker1)
go trackSomething(pw, &tracker2)
go trackSomething(pw, &tracker3)
time.Sleep(time.Millisecond * 50)
pw.Render()
}
}

func main() {
numRenders := 5
if len(os.Args) > 1 {
var err error
numRenders, err = strconv.Atoi(os.Args[2])
if err != nil {
fmt.Printf("Invalid Argument: '%s'\n", os.Args[2])
os.Exit(1)
}
}

for _, profiler := range profilers {
profileRender(profiler, numRenders)
}
}
3 changes: 2 additions & 1 deletion list/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,4 +26,5 @@ Pretty-print lists with multiple levels/indents into ASCII/Unicode strings.
■ The Gunslinger
```

A demonstration of all the capabilities can be found here: [../cmd/demo-list](../cmd/demo-list)
A demonstration of all the capabilities can be found here:
[../cmd/demo-list](../cmd/demo-list)
2 changes: 1 addition & 1 deletion profile.sh
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
rm -fr profile

# profile each supported package
for what in "list" "table"
for what in "list" "progress" "table"
do
echo "Profiling ${what} ..."
mkdir -p profile/${what}
Expand Down
39 changes: 39 additions & 0 deletions progress/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# Progress
[![GoDoc](https://godoc.org/github.com/jedib0t/go-pretty/progress?status.svg)](https://godoc.org/github.com/jedib0t/go-pretty/progress)

Track the Progress of one or more Tasks (like downloading multiple files in
parallel).

- Track one or more Tasks at the same time
- Dynamically add one or more Task Trackers while `Render()` is in progress
- Choose to have the Writer auto-stop the Render when no more Trackers are
in queue, or manually stop using `Stop()`
- Redirect output to an io.Writer object (like os.StdOut)
- Completely customizable styles
- Many ready-to-use styles: [style.go](style.go)
- Colorize various parts of the Tracker using `StyleColors`
- Customize how Trackers get rendered using `StyleOptions`

Sample Progress Tracking:
```
Calculating Total # 1 ... done! [3.25K in 100ms]
Calculating Total # 2 ... done! [6.50K in 100ms]
Downloading File # 3 ... done! [9.75KB in 100ms]
Transferring Amount # 4 ... done! [$26.00K in 200ms]
Transferring Amount # 5 ... done! [£32.50K in 201ms]
Downloading File # 6 ... done! [58.50KB in 300ms]
Calculating Total # 7 ... done! [91.00K in 400ms]
Transferring Amount # 8 ... 60.9% (●●●●●●●●●●●●●●◌◌◌◌◌◌◌◌◌) [$78.00K in 399.071ms]
Downloading File # 9 ... 32.1% (●●●●●●●○◌◌◌◌◌◌◌◌◌◌◌◌◌◌◌) [58.50KB in 298.947ms]
Transferring Amount # 10 ... 13.0% (●●○◌◌◌◌◌◌◌◌◌◌◌◌◌◌◌◌◌◌◌◌) [£32.50K in 198.84ms]
```

Real-time playback of the output @ asciinema.org:
[![asciicast](https://asciinema.org/a/KcPw8aoBSsYCBOj60wluhu5z3.png)](https://asciinema.org/a/KcPw8aoBSsYCBOj60wluhu5z3)

A demonstration of all the capabilities can be found here:
[../cmd/demo-progress](../cmd/demo-progress)

# TODO

- Optimize CPU and Memory Usage
Loading

0 comments on commit 3734d9f

Please sign in to comment.