Fluent::Plugin::Triton

Welcome to your new gem! In this directory, you'll find the files you need to be able to package up your Ruby library into a gem. Put your Ruby code in the file lib/fluent/plugin/triton. To experiment with that code, run bin/console for an interactive prompt.

This gem contains Custom Fluentd plugins used at Triton One.

Installation

Add this line to your application's Gemfile:

gem 'fluent-plugin-triton'

And then execute:

$ bundle install

Or install it yourself as:

$ gem install fluent-plugin-triton

Usage

Nomad filter plugin

This filter plugin allow you to add tags nomad related records from logs emitted by nomad client.

Here is an example how to write a fluent.conf file to add useful nomad tag:

<source>
  @type tail
  path '/var/lib/nomad/alloc/**/**/alloc/logs/*.stdout.*'
  pos_file /var/log/td-agent/tmp/access.log.pos
  <parse>
    @type json
  </parse>
  path_key tailed_path
  tag nomad.log
</source>

<filter nomad.log>
  @type record_transformer
  enable_ruby
  <record>
    alloc_id ${record['tailed_path'].split('alloc/')[1].split('/')[0]}
  </record>
</filter>

<filter nomad.log>
  @type nomad
  alloc_id_field alloc_id
  nomad_addr <NOMAD_ADDR>
  nomad_token <NOMAD_TOKEN>
</filter>

The first nomad.log source tail all allocation folder generated by nomad-client. For each record we add the provenance "tailed_path" to it so we can extract it's allocation id.

The first nomad.log filter extract the alloc_id used to query allocation information such as job, node, task group, namespace, etc.

The second nomad.log filter descripe what field contains the alloc_id using alloc_id_field and you can provide nomad_addr and nomad_token as connection information.

The last two parameters are optional if fluentd's environment variable contains NOMAD_TOKEN and NOMAD_ADDR.

Development

After checking out the repo, run bin/setup to install dependencies. You can also run bin/console for an interactive prompt that will allow you to experiment.

To install this gem onto your local machine, run bundle exec rake install. To release a new version, update the version number in version.rb, and then run bundle exec rake release, which will create a git tag for the version, push git commits and the created tag, and push the .gem file to rubygems.org.

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/[USERNAME]/fluent-plugin-triton.