Required config for ssl config for elk #164

ck-7 · 2021-01-14T14:37:17Z

We configured the ELK with ssl based. How to config Elastalert with SSL based authentication?
please share the SSL parameter to pass it config.json and elastalert.yaml ??

Kibana.yml

[root@elk-logging elastalert]# cat /etc/kibana/kibana.yml
server.host: "elk-logging"
server.port: 5601
elasticsearch.hosts: ["https://elk-logging.xxcxcx.net:9200"]
elasticsearch.password: XXXXXCXCX

Elasticsearch from/to Kibana

elasticsearch.ssl.certificateAuthorities: /etc/kibana/certs/ca/ca.crt
elasticsearch.ssl.certificate: /etc/kibana/certs/kibana.crt
elasticsearch.ssl.key: /etc/kibana/certs/kibana.key
elasticsearch.ssl.verificationMode: none

Browser from/to Kibana

server.ssl.enabled: true
server.ssl.certificate: /etc/kibana/certs/kibana.crt
server.ssl.key: /etc/kibana/certs/kibana.key

Elasticsearch authentication

xpack.security.enabled: true
elasticsearch.username: elastic
server.defaultRoute: /app/wazuh

#Elastalert Hosts
elastalert-kibana-plugin.serverHost: elk-logging
elastalert-kibana-plugin.serverPort: 3030

elastalert -- config
[root@elk-logging config]# cat config.json
{
"appName": "elastalert-server",
"port": 3030,
"wsport": 3333,
"elastalertPath": "/opt/elastalert",
"verbose": true,
"es_debug": true,
"debug": true,
"rulesPath": {
"relative": true,
"path": "/rules"
},
"templatesPath": {
"relative": true,
"path": "/rule_templates"
},
"es_host": "elk-logging",
"es_username": "elastic", // Option basic-auth username and password for Elasticsearch
"es_password": "XXXXXCXCX", // Option basic-auth username and password for Elasticsearch
"es_ssl": true, // Enable/Disable SSL
"es_ca_certs": "/etc/elasticsearch/certs/ca/ca.crt", // Path to ca for ElasticSearch (SSL must be enabled)
"es_client_cert": "/etc/elasticsearch/certs/elasticsearch.crt", // Path to cert for ElasticSearch (SSL must be enabled)
"es_client_key": "/etc/elasticsearch/certs/elasticsearch.key", // Path to key for ElasticSearch (SSL must be enabled)
"es_port": 9200,
"writeback_index": "elastalert_status"
}

Elastalert Elasticsearch.yaml

The elasticsearch hostname for metadata writeback

Note that every rule can have its own elasticsearch host

es_host: elk-logging

The elasticsearch port

es_port: 9200

This is the folder that contains the rule yaml files

Any .yaml file will be loaded as a rule

rules_folder: rules

How often ElastAlert will query elasticsearch

The unit can be anything from weeks to seconds

run_every:
seconds: 5

ElastAlert will buffer results from the most recent

period of time, in case some log sources are not in real time

buffer_time:
minutes: 1

Optional URL prefix for elasticsearch

#es_url_prefix: elasticsearch

Connect with TLS to elasticsearch

use_ssl: True

Verify TLS certificates

verify_certs: True
client_cert: "/etc/elasticsearch/certs/elasticsearch.crt"
client_key: "/etc/elasticsearch/certs/elasticsearch.key"
ca_certs: "/etc/elasticsearch/certs/ca/ca.crt"

GET request with body is the default option for Elasticsearch.

If it fails for some reason, you can pass 'GET', 'POST' or 'source'.

See http://elasticsearch-py.readthedocs.io/en/master/connection.html?highlight=send_get_body_as#transport

for details

#es_send_get_body_as: GET

Option basic-auth username and password for elasticsearch

es_username: elastic
es_password: XXXXXXCXXX

The index on es_host which is used for metadata storage

This can be a unmapped index, but it is recommended that you run

elastalert-create-index to set a mapping

writeback_index: elastalert_status

If an alert fails for some reason, ElastAlert will retry

sending the alert until this time period has elapsed

alert_time_limit:
days: 2

[root@elk-logging elastalert]# docker start --interactive elastalert

@bitsensor/[email protected] start /opt/elastalert-server
sh ./scripts/start.sh

14:31:12.693Z INFO elastalert-server: Config: No config.dev.json file was found in /opt/elastalert-server/config/config.dev.json.
14:31:12.696Z INFO elastalert-server: Config: Proceeding to look for normal config file.
14:31:12.697Z INFO elastalert-server: Config: A config file was found in /opt/elastalert-server/config/config.json. Using that config.
14:31:12.720Z INFO elastalert-server: Router: Listening for GET request on /.
14:31:12.720Z INFO elastalert-server: Router: Listening for GET request on /status.
14:31:12.721Z INFO elastalert-server: Router: Listening for GET request on /status/control/:action.
14:31:12.721Z INFO elastalert-server: Router: Listening for GET request on /status/errors.
14:31:12.721Z INFO elastalert-server: Router: Listening for GET request on /rules.
14:31:12.723Z INFO elastalert-server: Router: Listening for GET request on /rules/:id.
14:31:12.723Z INFO elastalert-server: Router: Listening for POST request on /rules/:id.
14:31:12.723Z INFO elastalert-server: Router: Listening for DELETE request on /rules/:id.
14:31:12.723Z INFO elastalert-server: Router: Listening for GET request on /templates.
14:31:12.723Z INFO elastalert-server: Router: Listening for GET request on /templates/:id.
14:31:12.724Z INFO elastalert-server: Router: Listening for POST request on /templates/:id.
14:31:12.724Z INFO elastalert-server: Router: Listening for DELETE request on /templates/:id.
14:31:12.724Z INFO elastalert-server: Router: Listening for POST request on /test.
14:31:12.724Z INFO elastalert-server: Router: Listening for GET request on /config.
14:31:12.724Z INFO elastalert-server: Router: Listening for POST request on /config.
14:31:12.724Z INFO elastalert-server: Router: Listening for POST request on /download.
14:31:12.724Z INFO elastalert-server: Router: Listening for GET request on /metadata/:type.
14:31:12.725Z INFO elastalert-server: Router: Listening for GET request on /mapping/:index.
14:31:12.725Z INFO elastalert-server: Router: Listening for POST request on /search/:index.
14:31:12.742Z ERROR elastalert-server:
Server: Starting server failed with error: TypeError: object must be passed
at module.exports (/opt/elastalert-server/node_modules/object-resolve-path/object-resolve-path.js:13:11)
at ServerConfig.get (/opt/elastalert-server/src/common/config/server_config.js:32:12)
at /opt/elastalert-server/src/elastalert_server.js:67:58
at /opt/elastalert-server/src/common/config/server_config.js:60:9
at Array.forEach ()
at /opt/elastalert-server/src/common/config/server_config.js:59:22
14:31:12.742Z INFO elastalert-server: Server: Stopping server
/opt/elastalert-server/src/common/websocket.js:34
wss.clients.forEach(function (ws) {
^

TypeError: Cannot read property 'clients' of null
at Timeout._onTimeout (/opt/elastalert-server/src/common/websocket.js:22:7)
at listOnTimeout (internal/timers.js:531:17)
at processTimers (internal/timer

nsano-rururu · 2021-01-15T02:16:48Z

kibana.yml configurations -

elastalert-kibana-plugin.serverHost: <HostName.Domain.com>
elastalert-kibana-plugin.serverPort: 443
elastalert-kibana-plugin.serverSsl: true

nsano-rururu · 2021-01-15T02:17:44Z

Is it docker? .. What is the docker image name of elastalert-server specified?

nsano-rururu · 2021-01-15T02:34:45Z

If you do not delete the comments after "//" and "//", an error should occur.
Also, are the es_ca_certs, es_client_cert, and es_client_key files mounted when the docker container is started so that they can be referenced from within the docker container?

"es_username": "elastic", // Option basic-auth username and password for Elasticsearch
"es_password": "XXXXXCXCX", // Option basic-auth username and password for Elasticsearch
"es_ssl": true, // Enable/Disable SSL
"es_ca_certs": "/etc/elasticsearch/certs/ca/ca.crt", // Path to ca for ElasticSearch (SSL must be enabled)
"es_client_cert": "/etc/elasticsearch/certs/elasticsearch.crt", // Path to cert for ElasticSearch (SSL must be enabled)
"es_client_key": "/etc/elasticsearch/certs/elasticsearch.key", // Path to key for ElasticSearch (SSL must be enabled)

after

"es_username": "elastic",
"es_password": "XXXXXCXCX",
"es_ssl": true,
"es_ca_certs": "/etc/elasticsearch/certs/ca/ca.crt",
"es_client_cert": "/etc/elasticsearch/certs/elasticsearch.crt",
"es_client_key": "/etc/elasticsearch/certs/elasticsearch.key",

nsano-rururu · 2021-01-15T02:45:54Z

As a precaution when debugging, note that the alert will not be skipped if debug is set to true in config.json of ElastAlert Server.

example

  "es_debug": false,
  "debug": false,

nsano-rururu · 2021-01-15T03:20:36Z

I think that elastalert-kibana-plugin has been discontinued, so I don't think it will be fixed even if it doesn't work due to a bug.

ck-7 · 2021-01-15T05:05:13Z

Is it docker? .. What is the docker image name of elastalert-server specified?

with latest image bitsensor/elastalert:3.0.0-beta.0

ck-7 · 2021-01-15T05:29:37Z

I think that elastalert-kibana-plugin has been discontinued, so I don't think it will be fixed even if it doesn't work due to a bug.

any alternative tool for GUI Based Kibana alert Plugin?

nsano-rururu · 2021-01-15T07:48:41Z

with latest image bitsensor/elastalert:3.0.0-beta.0

bitsensor / elastalert does not have the following settings.
This is a setting added to the fork johnsusek / elastalert-server (former repository ServerCentral / elastalert-server).

"es_ssl": true,
"es_ca_certs": "/etc/elasticsearch/certs/ca/ca.crt",
"es_client_cert": "/etc/elasticsearch/certs/elasticsearch.crt",
"es_client_key": "/etc/elasticsearch/certs/elasticsearch.key",

nsano-rururu · 2021-01-15T07:50:07Z

any alternative tool for GUI Based Kibana alert Plugin?

Praeco. By the way, I'm the co-maintainer of Praeco.
https://github.com/johnsusek/praeco

nsano-rururu · 2021-01-15T07:51:11Z

By the way, I'm also the co-maintainer of johnsusek / elastalert-server.
https://github.com/johnsusek/elastalert-server

nsano-rururu · 2021-01-15T07:52:54Z

It forks elastalert-kibana-plugin and supports up to Kibana 7.9.3. It needs to be remade to support Kibana 7.10 or later, and it has not been started yet.
https://github.com/nsano-rururu/elastalert-kibana-plugin

ck-7 · 2021-01-15T08:04:11Z

It forks elastalert-kibana-plugin and supports up to Kibana 7.9.3. It needs to be remade to support Kibana 7.10 or later, and it has not been started yet.
https://github.com/nsano-rururu/elastalert-kibana-plugin

by the way am using kibana 7.9.3 version for the elastalert kibana plugin..

ck-7 · 2021-01-15T08:09:59Z

It forks elastalert-kibana-plugin and supports up to Kibana 7.9.3. It needs to be remade to support Kibana 7.10 or later, and it has not been started yet.
https://github.com/nsano-rururu/elastalert-kibana-plugin

Looking for it ASAP.
Thank for the quick reply and response.

nsano-rururu · 2022-01-27T14:18:11Z

Ask a question in the repository you are maintaining
https://github.com/Karql/elastalert-kibana-plugin

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Required config for ssl config for elk #164

Required config for ssl config for elk #164

ck-7 commented Jan 14, 2021 •

edited

Loading

nsano-rururu commented Jan 15, 2021

nsano-rururu commented Jan 15, 2021

nsano-rururu commented Jan 15, 2021 •

edited

Loading

nsano-rururu commented Jan 15, 2021 •

edited

Loading

nsano-rururu commented Jan 15, 2021

ck-7 commented Jan 15, 2021

ck-7 commented Jan 15, 2021

nsano-rururu commented Jan 15, 2021

nsano-rururu commented Jan 15, 2021

nsano-rururu commented Jan 15, 2021

nsano-rururu commented Jan 15, 2021

ck-7 commented Jan 15, 2021 •

edited

Loading

ck-7 commented Jan 15, 2021

nsano-rururu commented Jan 27, 2022

Required config for ssl config for elk #164

Required config for ssl config for elk #164

Comments

ck-7 commented Jan 14, 2021 • edited Loading

Kibana.yml

Elasticsearch from/to Kibana

Browser from/to Kibana

Elasticsearch authentication

Elastalert Elasticsearch.yaml

The elasticsearch hostname for metadata writeback

Note that every rule can have its own elasticsearch host

The elasticsearch port

This is the folder that contains the rule yaml files

Any .yaml file will be loaded as a rule

How often ElastAlert will query elasticsearch

The unit can be anything from weeks to seconds

ElastAlert will buffer results from the most recent

period of time, in case some log sources are not in real time

Optional URL prefix for elasticsearch

Connect with TLS to elasticsearch

Verify TLS certificates

GET request with body is the default option for Elasticsearch.

If it fails for some reason, you can pass 'GET', 'POST' or 'source'.

See http://elasticsearch-py.readthedocs.io/en/master/connection.html?highlight=send_get_body_as#transport

for details

Option basic-auth username and password for elasticsearch

The index on es_host which is used for metadata storage

This can be a unmapped index, but it is recommended that you run

elastalert-create-index to set a mapping

If an alert fails for some reason, ElastAlert will retry

sending the alert until this time period has elapsed

nsano-rururu commented Jan 15, 2021

nsano-rururu commented Jan 15, 2021

nsano-rururu commented Jan 15, 2021 • edited Loading

nsano-rururu commented Jan 15, 2021 • edited Loading

nsano-rururu commented Jan 15, 2021

ck-7 commented Jan 15, 2021

ck-7 commented Jan 15, 2021

nsano-rururu commented Jan 15, 2021

nsano-rururu commented Jan 15, 2021

nsano-rururu commented Jan 15, 2021

nsano-rururu commented Jan 15, 2021

ck-7 commented Jan 15, 2021 • edited Loading

ck-7 commented Jan 15, 2021

nsano-rururu commented Jan 27, 2022

ck-7 commented Jan 14, 2021 •

edited

Loading

nsano-rururu commented Jan 15, 2021 •

edited

Loading

nsano-rururu commented Jan 15, 2021 •

edited

Loading

ck-7 commented Jan 15, 2021 •

edited

Loading