happo-agent
is yet another Nagios nrpe plugin. And improvement nrpe functions.
- More secure communication. Supports TLS 1.2.
- Less fork cost at bastion(proxy) mode. Proxy request handled by thread (not fork()).
- Metric collection. Compatible to Sensu plugin format.
- inventory collection.
- Red Hat Enterprise Linux (RHEL) 6.x, 7.x
- CentOS 6.x, 7.x
- Ubuntu 12.04 or later
/path/to/happo-agent daemon -A [Accept from IP/Subnet] -B [Public key file] -R [Private key file] -M [Metric config file (Accept empty file)]
Many configuration can be with environment variables.
See /etc/default/happo-agent.env
(example is in contrib/etc/default/happo-agent.env)
Call plugin from check_happo
, happo-agent
calls local nagios plugin program. Then, return code and value to check_happo
.
For more information, please see check_happo
README.
Every one minute, execute sensu metrics plugin defined by metrics.yaml
, and buffering results.
If you collect buffering results, you can use API /metric
method.
Get command based inventory data via API /inventory
method.
You create happo-agent
client management server if you want.
Use api client commands, happo-agent
calls endpoint url which is client management server.
/path/to/happo-agent add -e [ENDPOINT_URL] -g [GROUP_NAME[!SUB_GROUP_NAME]] -i [OWN_IP] -H [HOSTNAME] [-p BASTON_IP]
/path/to/happo-agent is_added -e [ENDPOINT_URL] -g [GROUP_NAME[!SUB_GROUP_NAME]] -i [OWN_IP]
/path/to/happo-agent remove -e [ENDPOINT_URL] -g [GROUP_NAME[!SUB_GROUP_NAME]] -i [OWN_IP]
$ sudo yum install epel-release
$ sudo yum install nagios-plugins-all
$ go get -dv github.com/heartbeatsjp/happo-agent
$ sudo install $GOHOME/src/bin/happo-agent /usr/local/bin/happo-agent
$ sudo install -d -m 755 /etc/happo
$ cd /etc/happo
$ sudo openssl genrsa -aes128 -out happo-agent.key 2048
$ sudo openssl req -new -key happo-agent.key -sha256 -out happo-agent.csr
$ sudo openssl x509 -in happo-agent.csr -days 3650 -req -signkey happo-agent.key -sha256 -out happo-agent.pub
$ sudo touch metrics.yaml
$ sudo chmod go-rwx happo-agent.key
$ sudo install contrib/etc/default/happo-agent.env /etc/default/happo-agent.env
$ sudo install contrib/etc/init/happo-agent.conf /etc/init/happo-agent.conf
$ sudo initctl reload-configuration
$ sudo initctl start happo-agent
You want to use sensu metrics plugins, should install /usr/local/bin
.
Pre build binary maybe useful. Releases · heartbeatsjp/happo-agent
metrics.yaml
metrics:
- hostname: [HOSTNAME]
plugins:
- plugin_name: [Sensu plugin name (Path not needed)]
plugin_option: [Sensu plugin name options]
- ...
- ...
Since the 2.0.0 release, AWS EC2 Auto Scaling is supported.
Bastion's agent stores Auto Scaling instance info like an instance id, private ip address in dbms. Also each instances are assigned alias, that too stored in dbms too.
e.g)
※ Be careful this is different of structure of actual stored in dbms.
Alias | Instance id | Private ip address |
---|---|---|
hb-autoscaling-web-01 | i-aaaaaa | 192.0.2.1 |
hb-autoscaling-web-02 | i-bbbbbb | 192.0.2.2 |
hb-autoscaling-web-03 | i-cccccc | 192.0.2.3 |
: | : | : |
Instance info assigned to alias is automatic change according to the actual change of Auto Scaling instances.
Bastion's agent transfers request to private ip address resolved from alias when received request of proxy to alias. This concrete behave is difference depend on request type(parameter of /proxy).
Monitoring
- When
request_type: monitor
, proxy to private ip address resolved from alias
Metric collection
- When
request_type: metric
, proxy to private ip address resolved from alias - When
request_type: metric/config/update
(proxy_hostport
need to be Auto Scaling Group Name instead of alias), proxy to active instances contained in Auto Scaling Group
Inventory collection
- When
request_type: inventory
(proxy_hostport
need to be Auto Scaling Group Name instead of alias), proxy to one of active instance contained in Auto Scaling Group
Subcommand for API client mode
/path/to/happo-agent add_ag -e [ENDPOINT_URL] -g [GROUP_NAME[!SUB_GROUP_NAME]] -n [AUTOSCALING_GROUP_NAME] -H [HOST_PREFIX] -c [AUTOSCALING_COUNT] [-p BASTON_IP]
/path/to/happo-agent resolve_alias -b [BASTION_ENDPOINT_URL] <alias>
/path/to/happo-agent leave -n [NODE_ENDPOINT_URL]
/path/to/happo-agent list_aliases -b [BASTION_ENDPOINT_URL] -n [AUTOSCALING_GROUP_NAME] -all
$ cd /etc/happo
$ sudo touch autoscaling.yaml
autoscaling.yaml
autoscalings:
- autoscaling_group_name: [AutoScaling Group Name]
autoscaling_count: [Number of AutoScaling Group Instances]
host_prefix: [HOSTNAME Prefix]
- ...
IMPORTANT NOTICE:
AutoScaling instances data is stored with DBMS, DB key prefix is composed of autoscaling group name and hostprefix (see also DBMS). You should take care about DB key confrict when update autoscaling configuration.
In this case, DB key confrict. because it will generate DB key in same prefix composed of autoscaling_group_name
and host_prefix
.
autoscalings:
- autoscaling_group_name: sysx-web-a
autoscaling_count: 4
host_prefix: ap
- autoscaling_group_name: sysx-web
autoscaling_count: 4
host_prefix: a-ap
Should be specify some parameters in /etc/default/happo-agent.env
.
HAPPO_AGENT_DAEMON_AUTOSCALING_NODE="true"
HAPPO_AGENT_DAEMON_AUTOSCALING_BASTION_ENDPOINT="https://192.0.2.100:6777"
HAPPO_AGENT_DAEMON_AUTOSCALING_JOIN_WAIT_SECONDS="60"
You can also specify HAPPO_AGENT_DAEMON_AUTOSCALING_BASTION_ENDPOINT
and HAPPO_AGENT_DAEMON_AUTOSCALING_JOIN_WAIT_SECONDS
in AWS SSM Parameter Store.
See also contrib/etc/default/happo-agent.env.
Enable Upstart job of leave at node shutdown
$ sudo install contrib/etc/init/happo-agent-autoscaling-leave.conf /etc/init/happo-agent-autoscaling-leave.conf
$ sed -ie "s/^#start on/start on/" /etc/init/happo-agent-autoscaling-leave.conf
$ sed -ie "s/^#pre-stop/pre-stop/" /etc/init/happo-agent.conf
$ sudo initctl reload-configuration
$ sudo initctl restart happo-agent
happo-agent uses AWS API when using feature of AWS EC2 Auto Scaling, you should specify a AWS credentials setting one of the following way.  
- EC2 IAM role
- Credentials file (
~/.aws/credentials
) - Configuration file (
~/.aws/config
)- If the
AWS_SDK_LOAD_CONFIG
is set
- If the
- Environment variables
- Listen port: 6777 (Default)
- HTTPS, TLS 1.2, CipherSuite:
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
Check available.
- Input format
- None
- Return format
- String "OK"
$ wget -q --no-check-certificate -O - https://127.0.0.1:6777/
OK
Use agent bastion(proxy) mode.
- Input format
- JSON
- Input variables
- proxy_hostport:
- (Array) bastion_ip:port. It can multiple define.
- request_type: request type (e.g.
monitor
) - request_json: Send JSON string to server.
- proxy_hostport:
- Return format
- JSON
- Return variables
- By
request_type
type.
- By
In case --proxy-timeout-seconds
reached, return 504 Gateway Timeout
.
If destination host is AutoScaling instance, it will behave as follows.
request_type: monitor
- In case it can be resolved alias, proxy request to Auto Scaling instance.
- In case it can't be resolved alias, return dummy response from bastion.
- dummy response:
{"return_value":0,"message":"<alias> has not been assigned Instance\n"}
- dummy response:
$ wget -q --no-check-certificate -O - https://192.0.2.1:6777/proxy --post-data='{"proxy_hostport": ["198.51.100.1:6777"], "request_type": "monitor", "request_json": "{\"apikey\": \"\", \"plugin_name\": \"check_procs\", \"plugin_option\": \"-w 100 -c 200\"}"}'
{"return_value":1,"message":"PROCS WARNING: 168 processes\n"}
Example calls wget host -> https://192.0.2.1:6777/proxy -> https://198.51.100.1:6777/monitor
.
Get inventory information from command.
- Input format
- JSON
- Input variables
- apikey: ""
- command: execute command
- command_option: command option
- Return format
- JSON
- Return variables
- return_code: commands return code
- return_value: commands return value (stdout, stderr)
$ wget -q --no-check-certificate -O - https://127.0.0.1:6777/inventory --post-data='{"apikey": "", "command": "uname", "command_option": "-a"}'
{"return_code":0,"return_value":"Linux saito-hb-vm101 2.6.32-573.3.1.el6.x86_64 #1 SMP Thu Aug 13 22:55:16 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux\n"}
Call monitor plugin. It likes nrpe.
- Input format
- JSON
- Input variables
- apikey: ""
- command: execute nagios plugin command
- command_option: command option
- Return format
- JSON
- Return variables
- return_code: commands return code
- return_value: commands return value (stdout, stderr)
In case --command-timeout
reached, return 500 Internal Server Error
.
$ wget -q --no-check-certificate -O - https://127.0.0.1:6777/monitor --post-data='{"apikey": "", "plugin_name": "check_procs", "plugin_option": "-w 100 -c 200"}'
{"return_value":1,"message":"PROCS WARNING: 168 processes\n"}
Get collected metric values.
- Input format
- JSON
- Input variables
- apikey: ""
- Return format
- JSON
- Return variables
- MetricData:
- (Array)
- hostname: Hostname
- timestamp: Unix time
- metrics: metric name - metric value (key-value)
- (Array)
- Message: message from agent (if error occurred)
- MetricData:
$ wget -q --no-check-certificate -O - https://127.0.0.1:6777/metric --post-data='{"apikey": ""}'
{"metric_data":[{"hostname":"saito-hb-vm101","timestamp":1444028730,"metrics":{"linux.context_switches.context_switches":32662,"linux.disk.elapsed.iotime_sda":52,"linux.disk.elapsed.iotime_weighted_sda":82,"linux.disk.rwtime.tsreading_sda":0,"linux.disk.rwtime.tswriting_sda":82,"linux.forks.forks":88,"linux.interrupts.interrupts":19642,"linux.ss.CLOSE-WAIT":0,"linux.ss.CLOSING":0,"linux.ss.ESTAB":9,"linux.ss.FIN-WAIT-1":0,"linux.ss.FIN-WAIT-2":0,"linux.ss.LAST-ACK":0,"linux.ss.LISTEN":31,"linux.ss.SYN-RECV":0,"linux.ss.SYN-SENT":0,"linux.ss.TIME-WAIT":7,"linux.ss.UNCONN":0,"linux.ss.UNKNOWN":0,"linux.swap.pswpin":0,"linux.swap.pswpout":0,"linux.users.users":1}},…(snip)…],"message":""}
Append metric values. (passive metrics collection)
- Input format
- JSON
- Input variables
- apikey: ""
- MetricData:
- (Array)
- hostname: Hostname
- timestamp: Unix time
- metrics: metric name - metric value (key-value)
- (Array)
- Return format
- JSON
- Return variables
- Message: message from agent (if error occurred)
$ wget -q --no-check-certificate -O - https://127.0.0.1:6777/metric/append --post-data='{"apikey": "", "metric_data":[{"hostname":"saito-hb-vm101","timestamp":1444028730,"metrics":{"linux.context_switches.context_switches":32662,"linux.disk.elapsed.iotime_sda":52,"linux.disk.elapsed.iotime_weighted_sda":82,"linux.disk.rwtime.tsreading_sda":0,"linux.disk.rwtime.tswriting_sda":82,"linux.forks.forks":88,"linux.interrupts.interrupts":19642,"linux.ss.CLOSE-WAIT":0,"linux.ss.CLOSING":0,"linux.ss.ESTAB":9,"linux.ss.FIN-WAIT-1":0,"linux.ss.FIN-WAIT-2":0,"linux.ss.LAST-ACK":0,"linux.ss.LISTEN":31,"linux.ss.SYN-RECV":0,"linux.ss.SYN-SENT":0,"linux.ss.TIME-WAIT":7,"linux.ss.UNCONN":0,"linux.ss.UNKNOWN":0,"linux.swap.pswpin":0,"linux.swap.pswpout":0,"linux.users.users":1}},...(snip)...]}'
{"status": "ok", "message": ""}
TODO
replaced to /status
List registered autoscaling instances
- Input format
- None
- Input variables
- None
- Return format
- JSON
- Return variables
- autoscaling: autoscaling list
- (Array)
- autoscaling_group_name: autoscaling group name
- instances: autoscaling group instances
- (Array)
- alias: alias of instance
- instance_data:
- ip: private ip address by Amazon EC2
- instance_id: instance id by Amazon EC2
- metric_plugins:
- (Array)
- plugin_name: metric plugin name
- plugin_option: metric plugin option
- (Array)
- (Array)
- (Array)
- autoscaling: autoscaling list
$ wget -q --no-check-certificate -O - https://127.0.0.1:6777/autoscaling
{"autoscaling":[{"autoscaling_group_name":"hb-autoscaling","instances":[{"alias":"hb-autoscaling-app-1","instance_data":{"ip":"192.0.2.11","instance_id":"i-aaaaaaaaaaaaaaaaa","metric_plugins":[{"plugin_name":"","plugin_option":""}]}},{"alias":"hb-autoscaling-app-2","instance_data":{"ip":"192.0.2.12","instance_id":"i-bbbbbbbbbbbbbbbbb","metric_plugins":[{"plugin_name":"","plugin_option":""}]}},{"alias":"hb-autoscaling-app-3","instance_data":{"ip":"192.0.2.13","instance_id":"i-ccccccccccccccccc","metric_plugins":[{"plugin_name":"","plugin_option":""}]}},{"alias":"hb-autoscaling-app-4","instance_data":{"ip":"192.0.2.14","instance_id":"i-ddddddddddddddddd","metric_plugins":[{"plugin_name":"","plugin_option":""}]}}]}]}
Resolve ip from alias
- Input format
- None
- Input variables
- None
- Return format
- JSON
- Return variables
- status: result status
- ip: private ip address by Amazon EC2
# wget -q --no-check-certificate -O - https://127.0.0.1:6777/autoscaling/resolve/hb-autoscaling-app-1
{"Status":"OK","ip":"192.0.2.11"}
Update autoscaling config
- Input format
- JSON
- Input variables
- apikey: ""
- config: configuration of autoscaling groups
- autoscalings:
- autoscaling_group_name: autoscaling group name
- autoscaling_count: num of autoscaling instances
- host_prefix: hostname(alias) prefix
- autoscalings:
- Return format
- JSON
- Return variables
- status: result status
- message: message from agent (if error occurred)
$ wget -q --no-check-certificate -O - https://127.0.0.1:6777/autoscaling/config/update --post-data="{\"apikey\":\"\",\"config\":{\"autoscalings\":[{\"autoscaling_group_name\":\"hb-autoscaling\",\"autoscaling_count\":"4",\"host_prefix\":\"app\"}]}}"
{"status":"OK","message":""}
Refresh autoscaling instances
- Input format
- JSON
- Input variables
- autoscaling_group_name: autoscaling group name
- Return format
- JSON
- Return variables
- status: result status
- message: message from agent (if error occurred)
$ wget -q --no-check-certificate -O - https://127.0.0.1:6777/autoscaling/refresh --post-data="{\"autoscaling_group_name\": \"hb-autoscaling\"}"
{"status":"OK","message":""}
Delete autoscaling instances data
- Input format
- JSON
- Input variables
- autoscaling_group_name: autoscaling group name
- Return format
- JSON
- Return variables
- status: result status
- message: message from agent (if error occurred)
$ wget -q --no-check-certificate -O - https://127.0.0.1:6777/autoscaling/delete --post-data="{\"autoscaling_group_name\": \"hb-autoscaling\"}"
{"status":"OK","message":""}
Register autoscaling instance
- Input format
- JSON
- Input variables
- apikey: ""
- autoscaling_group_name: autoscaling group name
- ip: private ip address by Amazon EC2
- instance_id: instance id by Amazon EC2
- Return format
- JSON
- Return variables
- status: result status
- message: message from agent (if error occurred)
- alias: assigned alias to instance
- instance_data:
- ip: private ip address by Amazon EC2
- instance_id: instance id by Amazon EC2
- metric_plugins:
- (Array)
- plugin_name: metric plugin name
- plugin_option: metric plugin option
- (Array)
$ wget -q --no-check-certificate -O - https://127.0.0.1:6777/autoscaling/instance/register --post-data="{\"apikey\":\"\",\"autoscaling_group_name\": \"hb-autoscaling\",\"ip\":\"192.0.2.1\",\"instance_id\":\"i-aaaaaa\"}"
{"status":"OK","message":"","alias":"hb-autoscaling-web-01","instance_data":{"ip":"192.0.2.1","instance_id":"i-aaaaaa","metric_config":{"Metrics":null}}}
Deregister autocaling instances
- Input format
- JSON
- Input variables
- apikey: ""
- instance_id: instance id by Amazon EC2
- Return format
- JSON
- Return variables
- status: result status
- message: message from agent (if error occurred)
$ wget -q --no-check-certificate -O - https://127.0.0.1:6777/autoscaling/instance/deregister --post-data="{\"apikey\":\"\",\"instance_id\":\"i-aaaaaa\"}"
{"status":"OK","message":""}
Deregister node from autoscaling bastion. This handler is available only in agent running with autoscaling node
- Input format
- JSON
- Input variables
- apikey: ""
- Return format
- JSON
- Return variables
- status: result status
- message: message from agent (if error occurred)
wget -q --no-check-certificate -O - https://127.0.0.1:6777/autoscaling/leave --post-data="{\"apikey\":\"\"}"
{"status":"OK","message":""}
- Input format
- None
- Input variables
- None
- Return format
- JSON
- Return variables
- status: result status
- message: message from agent (if error occurred)
- ip: private ip address by Amazon EC2
# wget -q --no-check-certificate -O - https://127.0.0.1:6777/autoscaling/health/hb-autoscaling-app-1
{"Status":"OK","message":"",ip:"192.0.2.1"}
Get happo-agent status
- Input format
- None
- Input variables
- None
- Return format
- JSON
- Return variables
- app_version: happo-agent version ( equivalent to
happo-agent -v
) - uptime_seconds: seconds from happo-agent started
- num_goroutine: number of goroutine
- metric_buffer_status
- oldest_timestamp: oldest Timestamp(int64) in metric_data_buffer
- newest_timestamp: newest Timestamp(int64) in metric_data_buffer
- callers:
filepath:linenum
of each goroutines
- app_version: happo-agent version ( equivalent to
$ wget -q --no-check-certificate -O - https://127.0.0.1:6777/status
{"app_version":"1.0.0","uptime_seconds":13,"num_goroutine":15,"metric_buffer_status":{"newest_timestamp":1505180794,"oldest_timestamp":1504852118},"callers":["/goroot/src/runtime/extern.go:219","/gopath/src/github.com/heartbeatsjp/happo-agent/model/status.go:28",...(snip)...]}
Get happo-agent memory usage status
- Input format
- None
- Input variables
- None
- Return format
- JSON
- Return variables
- runtime.MemStatus
$ wget -q --no-check-certificate -O - https://127.0.0.1:6777/status/memory
{"Alloc":7155296,"TotalAlloc":12148632,"Sys":14395640,"Lookups":34,"Mallocs":23456,"Frees":6565,...(snip)...}%
Get request status/count.
- Input format
- None
- Input variables
- None
- Return format
- JSON
- Return variables
- last1: Last 1 Minutes results
- url: url
- counts:
<status_code>
- count
- last5: Last 5 Minutes results
- same as last1
- last1: Last 1 Minutes results
$ wget -q --no-check-certificate -O - https://127.0.0.1:6777/status/request
{"keys":["s-1498112479","s-1498112819"]}
{"last1":[{"url":"/","counts":{"200":3,"403":1}},{"url":"/proxy","counts":{"200":1,"403":1}}],"last5":[{"url":"/","counts":{"200":3,"403":1}},{"url":"/proxy","counts":{"200":1,"403":1}}]}
Get autoscaling status
- Input format
- None
- Input variables
- None
- Return format
- JSON
- Return variables
- (Array)
- autoscaling_group_name: autoscaling group name
- status: result status
- message: message from agent (if error occurred)
- (Array)
$ wget -q --no-check-certificate -O - https://127.0.0.1:6777/status/autoscaling
[{"autoscaling_group_name":"hb-autoscaling","status":"ok","message":""}]
Get machine state key list.
- Input format
- None
- Input variables
- None
- Return format
- JSON
- Return variables
- keys: machine-state key list
$ wget -q --no-check-certificate -O - https://127.0.0.1:6777/machine-state
{"keys":["s-1498112479","s-1498112819"]}
Get machine state.
- Input format
- None
- Input variables
- key (can find from
/machine-state/
)
- key (can find from
- Return format
- JSON
- Return variables
- machineState: command results
$ wget -q --no-check-certificate -O - https://127.0.0.1:6777/machine-state/s-1498112479
{"machineState":"********** w (2017-06-22T15:21:19+09:00) cron 15:21:19 up 13 days, ..."}
- key
m-<timestamp>
are metrics(timestamp is unixtime).- value:
happo_agent.MetricsData
- value:
- key
s-<timestamp>
are saved machine state(timestamp is unixtime).- value:
string
- value:
- key
ag-<autoscaling group name>-<host prefix>-<serial number>
are saved autoscaling instance data.- value:
happo_agent.InstanceData
- value:
syndtr/goleveldb: LevelDB key/value database in Go.
Supported API
/
/proxy
/inventory
/monitor
/metric
/metric/append
/metric/config/update
/metric/status
/status
/status/memory
Use NSSM to handle happo-agent as Windows' Service
NSSM - the Non-Sucking Service Manager https://nssm.cc/
In below example, use NSSM.
- Download NSSM binary from https://nssm.cc/download
- Extract ZIP and put
nssm.exe
toC:\happo-agent
C:\happo-agent\nssm.exe
Use with sensu-plugins-windows is very convenient. https://github.com/sensu-plugins/sensu-plugins-windows
In below example, use sensu-plugins-windows(powershell)
- Clone or Download https://github.com/sensu-plugins/sensu-plugins-windows
- In below example, clone to
C:\happo-agent\sensu-plugins-windows
- In below example, clone to
- Do
Set-ExecutionPolicy RemoteSigned
in Administrative PowerShell- Find
Windows PowerShell
in Start Menu - Right Click => Run as Administrator
Set-ExecutionPolicy RemoteSigned
and Enter
- Find
- Download happo-agent release binary for Windows from GitHub
- https://github.com/heartbeatsjp/happo-agent/releases
- Save as
C:\happo-agent\happo-agent.exe
- Install NSSM and sensu-plugins-windows
- see above
- Put required files(Can generate on another host)
- Public Key : In this example, use
C:\happo-agent/etc/happo-agent.pub
- Private Key : In this example, use
C:\happo-agent/etc/happo-agent.key
- Metric Config : In this example, use
C:\happo-agent/etc/metrics.yaml
- Public Key : In this example, use
- Install happo-agent as Windows' Service on administrative
cmd.exe
-
Find
cmd.exe
in Start Menu -
Right Click => Run as Administrator
-
Install
cd c:\happo-agent nssm.exe install happo-agent C:\happo-agent\happo-agent.exe daemon nssm.exe set happo-agent AppEnvironmentExtra PATH="%PATH%;C:\happo-agent;C:\happo-agent\sensu-plugins-windows\bin\powershell" HAPPO_AGENT_ALLOWED_HOSTS=0.0.0.0/0 HAPPO_AGENT_PUBLIC_KEY=C:\happo-agent\etc\happo-agent.pub HAPPO_AGENT_PRIVATE_KEY=C:\happo-agent\etc\happo-agent.key HAPPO_AGENT_METRIC_CONFIG=C:\happo-agent\etc\metrics.yaml HAPPO_AGENT_NAGIOS_PLUGIN_PATHS=C:\happo-agent\sensu-plugins-windows\bin\powershell HAPPO_AGENT_SENSU_PLUGIN_PATHS=C:\happo-agent\sensu-plugins-windows\bin\powershell nssm.exe set happo-agent AppStdout C:\happo-agent\happo-agent.out nssm.exe set happo-agent AppStderr C:\happo-agent\happo-agent.out nssm.exe start happo-agent
-
When you use happo-agent.exe
in production, below environments are useful.
MARTINI_ENV="production"
HAPPO_AGENT_LOG_LEVEL="warn"
nssm.exe set happo-agent AppEnvironmentExtra ...
overwrites every time.
so if you want to ADD MARTINI_ENV="production"
and HAPPO_AGENT_LOG_LEVEL="warn"
to above installation,
do below.
nssm.exe set happo-agent AppEnvironmentExtra PATH="%PATH%;C:\happo-agent;C:\happo-agent\sensu-plugins-windows\bin\powershell" HAPPO_AGENT_ALLOWED_HOSTS=0.0.0.0/0 HAPPO_AGENT_PUBLIC_KEY=C:\happo-agent\etc\happo-agent.pub HAPPO_AGENT_PRIVATE_KEY=C:\happo-agent\etc\happo-agent.key HAPPO_AGENT_METRIC_CONFIG=C:\happo-agent\etc\metrics.yaml HAPPO_AGENT_NAGIOS_PLUGIN_PATHS=C:\happo-agent\sensu-plugins-windows\bin\powershell HAPPO_AGENT_SENSU_PLUGIN_PATHS=C:\happo-agent\sensu-plugins-windows\bin\powershell MARTINI_ENV="production" HAPPO_AGENT_LOG_LEVEL="warn"
nssm.exe restart happo-agent
- Fork (http://github.com/heartbeatsjp/happo-agent/fork)
- Create a feature branch
- Commit your changes
- Rebase your local changes against the master branch
- Run test suite with the
go test ./...
command and confirm that it passes - Run
gofmt -s
- Create a new Pull Request
You also can run test suite with docker on local PC.
Overview : Basically use go test
- Unit test :
go test
in CI - Endpoint behavior test :
go test
in CI - Regression test : automatically do in CI
daemontest
pipeline is the daemon running test.
if you run daemontest
on local with wercker-cli,
run below.
wercker build --pipeline daemontest
To confirm binary will suite to the criteria.
- Case:
- Test Duration: 2700sec(45min)
- Monitor Requests: 10kreq/45min => 3333/3min
- Metric Count: 200 => metrics data stored 200 metrics per minute
- Criteria:
- CPU Usage: up to 4%
- Monitoring agent's cpu usage shoud be small.
- Mem Usage: up to 500MB
- Monitoring agent's memory usage shoud be small. And more, we have to avoid memory leaking.
- Disk Usage: up to 250KB
- Disk Usage is almost related to the amount of storing metrics. We have to keep disk usage properly.
- CPU Usage: up to 4%
We know that long-long running test is good for daemon, but max is 59min, because of Wercker's restriction.
... note about implementation:
- daemontest configurations are in wercker.yml
daemontest > steps > script.name=="test daemon behavior" > code
- yq filter is
.daemontest.steps[] | select(.script.name=="test daemon behavior") | .script.code
- yq filter is
- Case:
- Test Duration:
TEST_DURATION_SEC
- to complete requests while test duration, maybe we have to change
MONITOR_REQUESTS
andMONITOR_REQUESTS_INTERVAL
- to complete requests while test duration, maybe we have to change
- Monitor Requests:
MONITOR_REQUESTS / TEST_DURATION_SEC
- Metric Count:
METRICS_COUNT
- Test Duration:
- Criteria:
- CPU Usage:
CPU_THRESHOLD_PERCENT
- Mem Usage:
MEM_THRESHOLD_KB
- Disk Usage:
DB_DISK_THRESHOLD_KB
- CPU Usage:
Copyright 2016 HEARTBEATS Corporation.
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.