Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🐛 address performance issue with perl #751

Merged
merged 3 commits into from
Dec 16, 2024
+191 −2
Merged

🐛 address performance issue with perl #751

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
93a7291
:bug: add benchmark test for os specific grep performance
pranavgaikwad Dec 13, 2024
0ca9e58
:bug: add benchmark workflow
pranavgaikwad Dec 13, 2024
e157ffe
:bug: fix performance issue with perl
pranavgaikwad Dec 13, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 37 additions & 0 deletions .github/workflows/benchmark.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
name: Run benchmark tests

on:
push:
branches:
- main
pull_request:
branches:
- main

jobs:
benchmark:
runs-on: ${{ matrix.os.id }}

strategy:
matrix:
os:
- id: macos-latest
name: mac

steps:
- name: Checkout repository
uses: actions/checkout@v3

- name: Set up Go
uses: actions/setup-go@v4
with:
go-version: '1.21'

- name: Run benchmark tests
run: |
go install golang.org/x/perf/cmd/benchstat@latest
go mod tidy
go test -run=^$ -bench=. -benchmem -count=6 ./... > benchmark_results

- name: Display Benchstat results
run: benchstat benchmark.${{ matrix.os.name }} benchmark_results
138 changes: 138 additions & 0 deletions benchmark.mac
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,138 @@
? github.com/konveyor/analyzer-lsp/cmd/analyzer [no test files]
? github.com/konveyor/analyzer-lsp/cmd/dep [no test files]
PASS
ok github.com/konveyor/analyzer-lsp/engine 0.401s
PASS
ok github.com/konveyor/analyzer-lsp/engine/internal 0.343s
PASS
ok github.com/konveyor/analyzer-lsp/engine/labels 0.303s
goos: darwin
goarch: arm64
pkg: github.com/konveyor/analyzer-lsp/event
cpu: Apple M2 Pro
Benchmark/Baseline-12 24095722 50.54 ns/op 0 B/op 0 allocs/op
Benchmark/Baseline-12 23726140 49.87 ns/op 0 B/op 0 allocs/op
Benchmark/Baseline-12 23899740 49.84 ns/op 0 B/op 0 allocs/op
Benchmark/Baseline-12 24208536 49.64 ns/op 0 B/op 0 allocs/op
Benchmark/Baseline-12 24224194 49.56 ns/op 0 B/op 0 allocs/op
Benchmark/Baseline-12 24192754 49.65 ns/op 0 B/op 0 allocs/op
Benchmark/StdLog-12 4844557 247.6 ns/op 152 B/op 12 allocs/op
Benchmark/StdLog-12 4867081 246.8 ns/op 152 B/op 12 allocs/op
Benchmark/StdLog-12 4861424 247.7 ns/op 152 B/op 12 allocs/op
Benchmark/StdLog-12 4803439 247.4 ns/op 152 B/op 12 allocs/op
Benchmark/StdLog-12 4855087 247.6 ns/op 152 B/op 12 allocs/op
Benchmark/StdLog-12 4815972 247.2 ns/op 152 B/op 12 allocs/op
Benchmark/LogNoExporter-12 3493294 344.3 ns/op 0 B/op 0 allocs/op
Benchmark/LogNoExporter-12 3489210 344.1 ns/op 0 B/op 0 allocs/op
Benchmark/LogNoExporter-12 3488331 343.6 ns/op 0 B/op 0 allocs/op
Benchmark/LogNoExporter-12 3485558 344.0 ns/op 0 B/op 0 allocs/op
Benchmark/LogNoExporter-12 3491460 343.8 ns/op 0 B/op 0 allocs/op
Benchmark/LogNoExporter-12 3493819 343.6 ns/op 0 B/op 0 allocs/op
Benchmark/TraceNoExporter-12 1000000 1060 ns/op 2816 B/op 16 allocs/op
Benchmark/TraceNoExporter-12 1000000 1057 ns/op 2816 B/op 16 allocs/op
Benchmark/TraceNoExporter-12 1000000 1063 ns/op 2816 B/op 16 allocs/op
Benchmark/TraceNoExporter-12 1000000 1059 ns/op 2816 B/op 16 allocs/op
Benchmark/TraceNoExporter-12 1000000 1056 ns/op 2816 B/op 16 allocs/op
Benchmark/TraceNoExporter-12 1000000 1058 ns/op 2816 B/op 16 allocs/op
Benchmark/StatsNoExporter-12 2286189 524.4 ns/op 0 B/op 0 allocs/op
Benchmark/StatsNoExporter-12 2283171 524.1 ns/op 0 B/op 0 allocs/op
Benchmark/StatsNoExporter-12 2288745 529.0 ns/op 0 B/op 0 allocs/op
Benchmark/StatsNoExporter-12 2288671 524.7 ns/op 0 B/op 0 allocs/op
Benchmark/StatsNoExporter-12 2289902 524.8 ns/op 0 B/op 0 allocs/op
Benchmark/StatsNoExporter-12 2287399 524.3 ns/op 0 B/op 0 allocs/op
Benchmark/LogNoop-12 800296 1476 ns/op 2816 B/op 16 allocs/op
Benchmark/LogNoop-12 809149 1474 ns/op 2816 B/op 16 allocs/op
Benchmark/LogNoop-12 814642 1473 ns/op 2816 B/op 16 allocs/op
Benchmark/LogNoop-12 804619 1476 ns/op 2816 B/op 16 allocs/op
Benchmark/LogNoop-12 813055 1475 ns/op 2816 B/op 16 allocs/op
Benchmark/LogNoop-12 801357 1476 ns/op 2816 B/op 16 allocs/op
Benchmark/TraceNoop-12 320142 3751 ns/op 9216 B/op 64 allocs/op
Benchmark/TraceNoop-12 318978 3765 ns/op 9216 B/op 64 allocs/op
Benchmark/TraceNoop-12 317174 3762 ns/op 9216 B/op 64 allocs/op
Benchmark/TraceNoop-12 321618 3751 ns/op 9216 B/op 64 allocs/op
Benchmark/TraceNoop-12 318663 3752 ns/op 9216 B/op 64 allocs/op
Benchmark/TraceNoop-12 316610 3737 ns/op 9216 B/op 64 allocs/op
Benchmark/StatsNoop-12 432556 2757 ns/op 5632 B/op 32 allocs/op
Benchmark/StatsNoop-12 431367 2755 ns/op 5632 B/op 32 allocs/op
Benchmark/StatsNoop-12 439297 2759 ns/op 5632 B/op 32 allocs/op
Benchmark/StatsNoop-12 438567 2756 ns/op 5632 B/op 32 allocs/op
Benchmark/StatsNoop-12 435147 2753 ns/op 5632 B/op 32 allocs/op
Benchmark/StatsNoop-12 434988 2815 ns/op 5632 B/op 32 allocs/op
Benchmark/Log-12 217500 5545 ns/op 2816 B/op 16 allocs/op
Benchmark/Log-12 213459 5557 ns/op 2816 B/op 16 allocs/op
Benchmark/Log-12 217459 5535 ns/op 2816 B/op 16 allocs/op
Benchmark/Log-12 215778 5536 ns/op 2816 B/op 16 allocs/op
Benchmark/Log-12 213157 5542 ns/op 2816 B/op 16 allocs/op
Benchmark/Log-12 215679 5541 ns/op 2816 B/op 16 allocs/op
Benchmark/Trace-12 57816 20692 ns/op 22222 B/op 384 allocs/op
Benchmark/Trace-12 57600 20707 ns/op 22222 B/op 384 allocs/op
Benchmark/Trace-12 58332 20819 ns/op 22223 B/op 384 allocs/op
Benchmark/Trace-12 57784 20719 ns/op 22223 B/op 384 allocs/op
Benchmark/Trace-12 58201 20829 ns/op 22223 B/op 384 allocs/op
Benchmark/Trace-12 58064 20801 ns/op 22223 B/op 384 allocs/op
Benchmark/Stats-12 247478 5032 ns/op 5632 B/op 32 allocs/op
Benchmark/Stats-12 240490 5104 ns/op 5632 B/op 32 allocs/op
Benchmark/Stats-12 235659 4923 ns/op 5632 B/op 32 allocs/op
Benchmark/Stats-12 234410 4951 ns/op 5632 B/op 32 allocs/op
Benchmark/Stats-12 242782 4903 ns/op 5632 B/op 32 allocs/op
Benchmark/Stats-12 234861 5083 ns/op 5632 B/op 32 allocs/op
PASS
ok github.com/konveyor/analyzer-lsp/event 89.042s
? github.com/konveyor/analyzer-lsp/event/core [no test files]
PASS
ok github.com/konveyor/analyzer-lsp/event/export 0.270s
? github.com/konveyor/analyzer-lsp/event/export/eventtest [no test files]
? github.com/konveyor/analyzer-lsp/event/export/metric [no test files]
PASS
ok github.com/konveyor/analyzer-lsp/event/export/ocagent 0.364s
PASS
ok github.com/konveyor/analyzer-lsp/event/export/ocagent/wire 0.278s
? github.com/konveyor/analyzer-lsp/event/export/prometheus [no test files]
? github.com/konveyor/analyzer-lsp/event/keys [no test files]
PASS
ok github.com/konveyor/analyzer-lsp/event/label 0.243s
? github.com/konveyor/analyzer-lsp/event/tag [no test files]
? github.com/konveyor/analyzer-lsp/jsonrpc2 [no test files]
PASS
ok github.com/konveyor/analyzer-lsp/jsonrpc2_v2 0.245s
? github.com/konveyor/analyzer-lsp/lsp/base_service_client [no test files]
? github.com/konveyor/analyzer-lsp/lsp/protocol [no test files]
PASS
ok github.com/konveyor/analyzer-lsp/lsp/protocol/generate 0.359s
? github.com/konveyor/analyzer-lsp/output/v1/konveyor [no test files]
PASS
ok github.com/konveyor/analyzer-lsp/parser 0.322s
goos: darwin
goarch: arm64
pkg: github.com/konveyor/analyzer-lsp/provider
cpu: Apple M2 Pro
BenchmarkMultilineGrepFileSizeSmall-12 18542 66409 ns/op 56058 B/op 322 allocs/op
BenchmarkMultilineGrepFileSizeSmall-12 18519 67844 ns/op 56106 B/op 322 allocs/op
BenchmarkMultilineGrepFileSizeSmall-12 18536 65137 ns/op 56021 B/op 322 allocs/op
BenchmarkMultilineGrepFileSizeSmall-12 18601 65112 ns/op 56100 B/op 322 allocs/op
BenchmarkMultilineGrepFileSizeSmall-12 17031 67742 ns/op 56140 B/op 322 allocs/op
BenchmarkMultilineGrepFileSizeSmall-12 18585 64159 ns/op 56015 B/op 322 allocs/op
BenchmarkMultilineGrepFileSizeBig-12 2812 419900 ns/op 201959 B/op 2224 allocs/op
BenchmarkMultilineGrepFileSizeBig-12 2863 419901 ns/op 202090 B/op 2224 allocs/op
BenchmarkMultilineGrepFileSizeBig-12 2820 420061 ns/op 202098 B/op 2224 allocs/op
BenchmarkMultilineGrepFileSizeBig-12 2852 419859 ns/op 201826 B/op 2224 allocs/op
BenchmarkMultilineGrepFileSizeBig-12 2814 420313 ns/op 201906 B/op 2224 allocs/op
BenchmarkMultilineGrepFileSizeBig-12 2844 421526 ns/op 202314 B/op 2224 allocs/op
PASS
ok github.com/konveyor/analyzer-lsp/provider 19.117s
? github.com/konveyor/analyzer-lsp/provider/grpc [no test files]
goos: darwin
goarch: arm64
pkg: github.com/konveyor/analyzer-lsp/provider/internal/builtin
cpu: Apple M2 Pro
BenchmarkRunOSSpecificGrepCommand-12 69 17053207 ns/op 43916 B/op 65 allocs/op
BenchmarkRunOSSpecificGrepCommand-12 72 17169915 ns/op 43725 B/op 64 allocs/op
BenchmarkRunOSSpecificGrepCommand-12 73 16839756 ns/op 43707 B/op 64 allocs/op
BenchmarkRunOSSpecificGrepCommand-12 69 17492351 ns/op 43743 B/op 64 allocs/op
BenchmarkRunOSSpecificGrepCommand-12 76 16651446 ns/op 43775 B/op 64 allocs/op
BenchmarkRunOSSpecificGrepCommand-12 70 17191546 ns/op 43852 B/op 65 allocs/op
PASS
ok github.com/konveyor/analyzer-lsp/provider/internal/builtin 8.223s
? github.com/konveyor/analyzer-lsp/provider/internal/grpc [no test files]
? github.com/konveyor/analyzer-lsp/provider/lib [no test files]
? github.com/konveyor/analyzer-lsp/tracing [no test files]
4 changes: 2 additions & 2 deletions provider/internal/builtin/service_client.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -624,8 +624,8 @@ func runOSSpecificGrepCommand(pattern string, location string, providerContext p
escapedPattern = strings.ReplaceAll(escapedPattern, "'", "'\\''")
escapedPattern = strings.ReplaceAll(escapedPattern, "$", "\\$")
cmd := fmt.Sprintf(
`find %v -type f | \
while read file; do perl -ne '/%v/ && print "$ARGV:$.:$1\n";' "$file"; done`,
`find %v -type f -print0 | \
xargs -0 perl -ne '/%v/ && print "$ARGV:$.:$1\n";'`,
location, escapedPattern,
)
findstr := exec.Command("/bin/sh", "-c", cmd)
Expand Down
14 changes: 14 additions & 0 deletions provider/internal/builtin/service_client_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,13 @@ package builtin

import (
"context"
"path/filepath"
"reflect"
"sync"
"testing"

"github.com/go-logr/logr/testr"
"github.com/konveyor/analyzer-lsp/engine"
"github.com/konveyor/analyzer-lsp/provider"
)

Expand Down Expand Up @@ -122,3 +124,15 @@ func Test_builtinServiceClient_filterByIncludedPaths(t *testing.T) {
})
}
}

func BenchmarkRunOSSpecificGrepCommand(b *testing.B) {
for i := 0; i < b.N; i++ {
path, err := filepath.Abs("../../../external-providers/java-external-provider/examples/customers-tomcat-legacy/")
if err != nil {
return
}
runOSSpecificGrepCommand("Apache License 1.1",
path,
provider.ProviderContext{Template: map[string]engine.ChainTemplate{}})
}
}
Loading