Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stanza crashes when using k8s_metadata_decorator operator #288

Open
Mayur2021 opened this issue Apr 28, 2021 · 0 comments
Open

Stanza crashes when using k8s_metadata_decorator operator #288

Mayur2021 opened this issue Apr 28, 2021 · 0 comments
Assignees

Comments

@Mayur2021
Copy link

Mayur2021 commented Apr 28, 2021

Describe the bug
Stanza crashes when using k8s_metadata_decorator operator. Gets the following exception

github.com/observiq/stanza/operator/builtin/transformer/k8smetadata.(*K8sMetadataDecorator).decorateEntryWithNamespaceMetadata(0xc0001e8b40, 0x0, 0x0, 0xc0034753b0, 0x24, 0xc017915f235ea7e8, 0x8bdab987c4, 0x3207fa0, 0xc003844f90, 0xc003844fc0, ...)
/build/cmd/stanza/vendor/github.com/observiq/stanza/operator/builtin/transformer/k8smetadata/k8s_metadata_decorator.go:321 +0x2d0
github.com/observiq/stanza/operator/builtin/transformer/k8smetadata.(*K8sMetadataDecorator).Process(0xc0001e8b40, 0x2164540, 0xc0005d7dc0, 0xc000119400, 0x0, 0xc00348fa10)

To Reproduce
Steps to reproduce the behavior:

  1. Build stanza locally and deploy in k8s clustter
  2. Used the following configuration
kind: ConfigMap
metadata:
  name: stanza-config
  namespace: stanza-test
apiVersion: v1
data:
  config.yaml: |2-
    pipeline:
      - type: file_input
        include:
          - /var/log/containers/*.log
      - type: json_parser
        id: parser-docker
        output: extract_metadata_from_filepath
        timestamp:
          parse_from: time
          layout: '%Y-%m-%dT%H:%M:%S.%LZ'
      # Extract metadata from file path
      - type: regex_parser
        id: extract_metadata_from_filepath
      #  regex: '^.*\/(?P<namespace>[^_]+)_(?P<pod_name>[^_]+)_(?P<uid>[a-f0-9\-]{36})\/(?P<container_name>[^\._]+)\/(?P<run_id>\d+)\.log$'
        regex: '(?P<pod_name>^.+?)_(?P<namespace>.+?)_(?P<container>.*)-(?P<runid>.+?).log$'
        parse_from: $labels.file_name
        #parse_to: kubernetes
      - type: k8s_metadata_decorator
        namespace_field: namespace
        pod_name_field: pod_name
      - type: otlp_output
        endpoint: opentelemetrycollector.stanza-test.svc.cluster.local:55681
        insecure: true
      #- type: stdout
  1. Exception found
	/build/cmd/stanza/vendor/github.com/observiq/stanza/operator/builtin/transformer/k8smetadata/k8s_metadata_decorator.go:321 +0x2d0
github.com/observiq/stanza/operator/builtin/transformer/k8smetadata.(*K8sMetadataDecorator).Process(0xc0001e8b40, 0x2164540, 0xc0005d7dc0, 0xc000119400, 0x0, 0xc00348fa10)
	/build/cmd/stanza/vendor/github.com/observiq/stanza/operator/builtin/transformer/k8smetadata/k8s_metadata_decorator.go:179 +0x20d
github.com/observiq/stanza/operator/helper.(*WriterOperator).Write(0xc00015bce0, 0x2164540, 0xc0005d7dc0, 0xc000119400)
	/build/cmd/stanza/vendor/github.com/observiq/stanza/operator/helper/writer.go:56 +0x10c
github.com/observiq/stanza/operator/helper.(*ParserOperator).ProcessWith(0xc00015bce0, 0x2164540, 0xc0005d7dc0, 0xc000119400, 0xc00348fa98, 0xc003599680, 0x30)
	/build/cmd/stanza/vendor/github.com/observiq/stanza/operator/helper/parser.go:89 +0xfb
github.com/observiq/stanza/operator/builtin/parser/regex.(*RegexParser).Process(0xc00015bce0, 0x2164540, 0xc0005d7dc0, 0xc000119400, 0xc00027bb40, 0x413861)
	/build/cmd/stanza/vendor/github.com/observiq/stanza/operator/builtin/parser/regex/regex.go:77 +0x72
github.com/observiq/stanza/operator/helper.(*WriterOperator).Write(0xc000222300, 0x2164540, 0xc0005d7dc0, 0xc000119400)
	/build/cmd/stanza/vendor/github.com/observiq/stanza/operator/helper/writer.go:56 +0x10c
github.com/observiq/stanza/operator/helper.(*ParserOperator).ProcessWith(0xc000222300, 0x2164540, 0xc0005d7dc0, 0xc000119400, 0xc00027bba8, 0xc00027bbe0, 0x559603)
	/build/cmd/stanza/vendor/github.com/observiq/stanza/operator/helper/parser.go:89 +0xfb
github.com/observiq/stanza/operator/builtin/parser/json.(*JSONParser).Process(0xc000222300, 0x2164540, 0xc0005d7dc0, 0xc000119400, 0x55909a, 0xc000421aa0)
	/build/cmd/stanza/vendor/github.com/observiq/stanza/operator/builtin/parser/json/json.go:52 +0x72
github.com/observiq/stanza/operator/helper.(*WriterOperator).Write(0xc0002db090, 0x2164540, 0xc0005d7dc0, 0xc000119400)
	/build/cmd/stanza/vendor/github.com/observiq/stanza/operator/helper/writer.go:56 +0x10c
github.com/observiq/stanza/operator/builtin/input/file.(*Reader).emit(0xc0037c8d80, 0x2164540, 0xc0005d7dc0, 0xc003830000, 0x8d, 0x4000, 0x0, 0x0)
	/build/cmd/stanza/vendor/github.com/observiq/stanza/operator/builtin/input/file/reader.go:129 +0x399
github.com/observiq/stanza/operator/builtin/input/file.(*Reader).ReadToEnd(0xc0037c8d80, 0x2164540, 0xc0005d7dc0)
	/build/cmd/stanza/vendor/github.com/observiq/stanza/operator/builtin/input/file/reader.go:98 +0x40d
github.com/observiq/stanza/operator/builtin/input/file.(*InputOperator).poll.func1(0xc00057e140, 0x2164540, 0xc0005d7dc0, 0xc0037c8d80)
	/build/cmd/stanza/vendor/github.com/observiq/stanza/operator/builtin/input/file/file.go:149 +0x6b
created by github.com/observiq/stanza/operator/builtin/input/file.(*InputOperator).poll
	/build/cmd/stanza/vendor/github.com/observiq/stanza/operator/builtin/input/file/file.go:147 +0x3c7

Expected behavior
Stanza should not crash, atleast tell there is a config error.

Screenshots
If applicable, add screenshots to help explain your problem.

Environment:

  • OS
  • Stanza Version or Commit Hash

Additional context
Add any other context about the problem here.

@jsirianni jsirianni self-assigned this May 11, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants