Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problems loading a plugin - Google BigQuery #54

Open
SelComputas opened this issue Aug 29, 2019 · 0 comments
Open

Problems loading a plugin - Google BigQuery #54

SelComputas opened this issue Aug 29, 2019 · 0 comments

Comments

@SelComputas
Copy link

SelComputas commented Aug 29, 2019

Please post all product and debugging questions on our forum. Your questions will reach our wider community members there, and if we confirm that there is a bug, then we can open a new issue here.

For all general issues, please provide the following details for fast resolution:

  • Version: 7.3.1
  • Operating System: Windows 10
  • Config File (if you have sensitive info, please remove it):
    input {
    file {
    path => "full-path/logstash-tutorial-dataset"
    sincedb_path => "nul"
    start_position => "beginning"
    }
    }

filter {
grok {
match => {
"message" => "%{COMBINEDAPACHELOG}"
#'message' => '%{IP:ip} - - [%{GREEDYDATA:date}] %{GREEDYDATA:request}'
}
}
geoip {
source => "clientip"
}
}

output {
google_bigquery {
project_id => "akbm-infrastructur"
dataset => "test_logstash"
csv_schema => "clientip:STRING"
json_key_file => "full-path-to-key-file.json"
error_directory => "/tmp/bigquery-errors"
}
}

  • Sample Data:
    83.149.9.216 - - [04/Jan/2015:05:13:42 +0000] "GET /presentations/logstash-monitorama-2013/images/kibana-search.png HTTP/1.1" 200 203023 "http://semicomplete.com/presentations/logstash-monitorama-2013/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.77 Safari/537.36"

  • Steps to Reproduce:
    Running the config file with `bin/logstash -r -f “” with the purpose of sending local data to BigQuery, using Logstash, produces a log file consisting of mainly

[2019-08-29T09:47:18,692][ERROR][logstash.plugins.registry] Problems loading a plugin with {:type=>"output", :name=>"google_bigquery", :path=>"logstash/outputs/google_bigquery", :error_message=>"\n\n\tyou might need to reinstall the gem which depends on the missing jar or in case there is Jars.lock then resolve the jars with `lock_jars` command\n\nno such file to load -- com/google/cloud/google-cloud-bigquery/1.24.1/google-cloud-bigquery-1.24.1 (LoadError)", :error_class=>RuntimeError, :error_backtrace=>["uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/jar_dependencies.rb:356:in `do_require'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/jar_dependencies.rb:265:in `block in require_jar'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/jar_dependencies.rb:307:in `require_jar_with_block'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/jar_dependencies.rb:264:in `require_jar'", 

And

[2019-08-29T09:47:18,697][ERROR][logstash.agent           ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::PluginLoadingError", :message=>"Couldn't find any output plugin named 'google_bigquery'. Are you sure this is correct? Trying to load the google_bigquery output plugin resulted in this error: Problems loading the requested plugin named google_bigquery of type output. Error: RuntimeError \n\n\tyou might need to reinstall the gem which depends on the missing jar or in case there is Jars.lock then resolve the jars with `lock_jars` command\n\nno such file to load -- com/google/cloud/google-cloud-bigquery/1.24.1/google-cloud-bigquery-1.24.1 (LoadError)", :backtrace=>

Please note that when running bin/logstash-plugin list it outputs logstash-output-google_bigquery

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant