Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: SSAPI Checkpointing (BPS-277) #1969

Merged
merged 12 commits into from
Nov 19, 2024
+398 −41
Merged

feat: SSAPI Checkpointing (BPS-277) #1969

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
567c7f7
checkpoint methods
Caleb-Hurshman Nov 11, 2024
67ae37d
Merge branch 'feat/ssapi-receiver' into feat/ssapi-checkpoints
Caleb-Hurshman Nov 14, 2024
33d9b0f
WIP
Caleb-Hurshman Nov 14, 2024
61b1452
functional checkpoint
Caleb-Hurshman Nov 15, 2024
1a3407a
debug logs, rm print
Caleb-Hurshman Nov 15, 2024
f753e78
loadCheckpoint return error
Caleb-Hurshman Nov 15, 2024
e775560
splunk failure test
Caleb-Hurshman Nov 18, 2024
f6694a0
Merge branch 'feat/ssapi-receiver' into feat/ssapi-checkpoints
Caleb-Hurshman Nov 18, 2024
28927d9
storage config test
Caleb-Hurshman Nov 18, 2024
d4df0db
lint, tidy
Caleb-Hurshman Nov 18, 2024
2153d9e
return error on export fail
Caleb-Hurshman Nov 19, 2024
34907e1
tidy
Caleb-Hurshman Nov 19, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -308,6 +308,7 @@ require (
github.com/briandowns/spinner v1.23.0 // indirect
github.com/cihub/seelog v0.0.0-20170130134532-f561c5e57575 // indirect
github.com/containerd/cgroups/v3 v3.0.3 // indirect
github.com/coreos/etcd v3.3.27+incompatible // indirect
github.com/coreos/go-oidc/v3 v3.11.0 // indirect
github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f // indirect
github.com/distribution/reference v0.6.0 // indirect
Expand Down Expand Up @@ -398,6 +399,7 @@ require (
go.elastic.co/apm/module/apmzap/v2 v2.6.0 // indirect
go.elastic.co/apm/v2 v2.6.0 // indirect
go.elastic.co/fastjson v1.3.0 // indirect
go.etcd.io/etcd v3.3.27+incompatible // indirect
go.opencensus.io v0.24.0 // indirect
go.opentelemetry.io/collector v0.113.0 // indirect
go.opentelemetry.io/collector/client v1.19.0 // indirect
Expand Down
4 changes: 4 additions & 0 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1127,6 +1127,8 @@ github.com/coreos/bbolt v1.3.2/go.mod h1:iRUV2dpdMOn7Bo10OQBFzIJO9kkE559Wcmn+qkE
github.com/coreos/bbolt v1.3.3/go.mod h1:iRUV2dpdMOn7Bo10OQBFzIJO9kkE559Wcmn+qkEiiKk=
github.com/coreos/etcd v3.3.10+incompatible/go.mod h1:uF7uidLiAD3TWHmW31ZFd/JWoc32PjwdhPthX9715RE=
github.com/coreos/etcd v3.3.13+incompatible/go.mod h1:uF7uidLiAD3TWHmW31ZFd/JWoc32PjwdhPthX9715RE=
github.com/coreos/etcd v3.3.27+incompatible h1:QIudLb9KeBsE5zyYxd1mjzRSkzLg9Wf9QlRwFgd6oTA=
github.com/coreos/etcd v3.3.27+incompatible/go.mod h1:uF7uidLiAD3TWHmW31ZFd/JWoc32PjwdhPthX9715RE=
github.com/coreos/go-etcd v2.0.0+incompatible/go.mod h1:Jez6KQU2B/sWsbdaef3ED8NzMklzPG4d5KIOhIy30Tk=
github.com/coreos/go-oidc/v3 v3.11.0 h1:Ia3MxdwpSw702YW0xgfmP1GVCMA9aEFWu12XUZ3/OtI=
github.com/coreos/go-oidc/v3 v3.11.0/go.mod h1:gE3LgjOgFoHi9a4ce4/tJczr0Ai2/BoDhf0r5lltWI0=
Expand Down Expand Up @@ -2612,6 +2614,8 @@ go.elastic.co/fastjson v1.3.0/go.mod h1:K9vDh7O0ODsVKV2B5e2XYLY277QZaCbB3tS1SnAR
go.etcd.io/bbolt v1.3.2/go.mod h1:IbVyRI1SCnLcuJnV2u8VeU0CEYM7e686BmAb1XKL+uU=
go.etcd.io/bbolt v1.3.11 h1:yGEzV1wPz2yVCLsD8ZAiGHhHVlczyC9d1rP43/VCRJ0=
go.etcd.io/bbolt v1.3.11/go.mod h1:dksAq7YMXoljX0xu6VF5DMZGbhYYoLUalEiSySYAS4I=
go.etcd.io/etcd v3.3.27+incompatible h1:5hMrpf6REqTHV2LW2OclNpRtxI0k9ZplMemJsMSWju0=
go.etcd.io/etcd v3.3.27+incompatible/go.mod h1:yaeTdrJi5lOmYerz05bd8+V7KubZs8YSFZfzsF9A6aI=
go.etcd.io/etcd/api/v3 v3.5.4/go.mod h1:5GB2vv4A4AOn3yk7MftYGHkUfGtDHnEraIjym4dYz5A=
go.etcd.io/etcd/client/pkg/v3 v3.5.4/go.mod h1:IJHfcCEKxYu1Os13ZdwCwIUTUVGYTSAM3YSwc9/Ac1g=
go.etcd.io/etcd/client/v3 v3.5.4/go.mod h1:ZaRkVgBZC+L+dLCjTcF1hRXpgZXQPOvnA/Ak/gq3kiY=
Expand Down
6 changes: 6 additions & 0 deletions receiver/splunksearchapireceiver/config.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ import (
"strings"
"time"

"go.opentelemetry.io/collector/component"
"go.opentelemetry.io/collector/config/confighttp"
)

Expand All @@ -34,6 +35,7 @@ type Config struct {
Password string `mapstructure:"splunk_password"`
Searches []Search `mapstructure:"searches"`
JobPollInterval time.Duration `mapstructure:"job_poll_interval"`
StorageID *component.ID `mapstructure:"storage"`
}

// Search struct to represent a Splunk search
Expand All @@ -60,6 +62,10 @@ func (cfg *Config) Validate() error {
return errors.New("at least one search must be provided")
}

if cfg.StorageID == nil {
return errors.New("storage configuration must be provided")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
return errors.New("storage configuration must be provided")
return errors.New("storage configuration is required for this receiver")

}

for _, search := range cfg.Searches {
if search.Query == "" {
return errors.New("missing query in search")
Expand Down
10 changes: 6 additions & 4 deletions receiver/splunksearchapireceiver/factory.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,10 +42,12 @@ func createLogsReceiver(_ context.Context,
) (receiver.Logs, error) {
ssapirConfig := cfg.(*Config)
ssapir := &splunksearchapireceiver{
logger: params.Logger,
logsConsumer: consumer,
config: ssapirConfig,
settings: params.TelemetrySettings,
logger: params.Logger,
logsConsumer: consumer,
config: ssapirConfig,
id: params.ID,
settings: params.TelemetrySettings,
checkpointRecord: &EventRecord{},
}
return ssapir, nil
}
Expand Down
35 changes: 33 additions & 2 deletions receiver/splunksearchapireceiver/go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,34 +3,65 @@ module github.com/open-telemetry/opentelemtry-collector-contrib/receiver/splunks
go 1.22.5

require (
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza v0.113.0
github.com/stretchr/testify v1.9.0
go.etcd.io/etcd v3.3.27+incompatible
go.opentelemetry.io/collector/component v0.113.0
go.opentelemetry.io/collector/consumer v0.113.0
go.opentelemetry.io/collector/pdata v1.19.0
go.opentelemetry.io/collector/receiver v0.112.0
go.opentelemetry.io/collector/receiver v0.113.0
go.uber.org/zap v1.27.0
)

require (
github.com/cenkalti/backoff/v4 v4.3.0 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/coreos/etcd v3.3.27+incompatible // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/elastic/lunes v0.1.0 // indirect
github.com/expr-lang/expr v1.16.9 // indirect
github.com/felixge/httpsnoop v1.0.4 // indirect
github.com/fsnotify/fsnotify v1.8.0 // indirect
github.com/go-logr/logr v1.4.2 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/go-viper/mapstructure/v2 v2.2.1 // indirect
github.com/goccy/go-json v0.10.3 // indirect
github.com/golang/protobuf v1.5.0 // indirect
github.com/golang/snappy v0.0.4 // indirect
github.com/hashicorp/go-version v1.7.0 // indirect
github.com/klauspost/compress v1.17.11 // indirect
github.com/knadh/koanf/maps v0.1.1 // indirect
github.com/knadh/koanf/providers/confmap v0.1.0 // indirect
github.com/knadh/koanf/v2 v2.1.1 // indirect
github.com/leodido/go-syslog/v4 v4.2.0 // indirect
github.com/leodido/ragel-machinery v0.0.0-20190525184631-5f46317e436b // indirect
github.com/magefile/mage v1.15.0 // indirect
github.com/mitchellh/copystructure v1.2.0 // indirect
github.com/mitchellh/reflectwalk v1.0.2 // indirect
github.com/open-telemetry/opentelemetry-collector-contrib/internal/coreinternal v0.113.0 // indirect
github.com/pierrec/lz4/v4 v4.1.21 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/rs/cors v1.11.1 // indirect
github.com/valyala/fastjson v1.6.4 // indirect
go.opentelemetry.io/collector/client v1.19.0 // indirect
go.opentelemetry.io/collector/config/configauth v0.113.0 // indirect
go.opentelemetry.io/collector/config/configcompression v1.19.0 // indirect
go.opentelemetry.io/collector/config/configopaque v1.19.0 // indirect
go.opentelemetry.io/collector/config/configtls v1.19.0 // indirect
go.opentelemetry.io/collector/config/internal v0.113.0 // indirect
go.opentelemetry.io/collector/confmap v1.19.0 // indirect
go.opentelemetry.io/collector/consumer/consumererror v0.113.0 // indirect
go.opentelemetry.io/collector/consumer/consumerprofiles v0.113.0 // indirect
go.opentelemetry.io/collector/consumer/consumertest v0.113.0 // indirect
go.opentelemetry.io/collector/extension v0.113.0 // indirect
go.opentelemetry.io/collector/extension/auth v0.113.0 // indirect
go.opentelemetry.io/collector/extension/experimental/storage v0.113.0 // indirect
go.opentelemetry.io/collector/featuregate v1.19.0 // indirect
go.opentelemetry.io/collector/pdata/pprofile v0.113.0 // indirect
go.opentelemetry.io/collector/semconv v0.113.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.56.0 // indirect
gonum.org/v1/gonum v0.15.1 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20240814211410-ddb44dafa142 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
)

Expand All @@ -41,7 +72,7 @@ require (
github.com/modern-go/reflect2 v1.0.2 // indirect
go.opentelemetry.io/collector/config/confighttp v0.113.0
go.opentelemetry.io/collector/config/configtelemetry v0.113.0 // indirect
go.opentelemetry.io/collector/pipeline v0.112.0 // indirect
go.opentelemetry.io/collector/pipeline v0.113.0 // indirect
go.opentelemetry.io/otel v1.31.0 // indirect
go.opentelemetry.io/otel/metric v1.31.0 // indirect
go.opentelemetry.io/otel/trace v1.31.0 // indirect
Expand Down
Loading
Loading