Create ECS compliant index templates #270

AlexRuiz7 · 2024-06-18T15:20:14Z

Description

As part of the Data Persistence Model Redesign, new indices will appear in the Wazuh Indexer. We want these indices to be created automatically after Wazuh Indexer boots up. For that, we'll build a plugin.

As owners of the Wazuh Indexer, we will define the indices settings and fields. These need to be ECS compliant.

Tasks

Implementation restrictions

agent.groups must be present on all indices but the commands index.
Inventory is split in 7 indices:

Agent's inventory module	Index
processes	wazuh-states-inventory-processes
hardware	wazuh-states-inventory-hardware
os	wazuh-states-inventory-system
ports	wazuh-states-inventory-networks
network	wazuh-states-inventory-networks
packages	wazuh-states-inventory-packages
hotfixes	wazuh-states-inventory-hotfixes

The agents index contains basic information about the host (IP, OS info) (see Enrich index templates with agents' information #525)

The text was updated successfully, but these errors were encountered:

AlexRuiz7 · 2024-06-18T16:25:04Z

`agents` index data model

Fields summary

The fields are based on wazuh/wazuh#23396 (comment)

Based on ECS Agent Fields.

Field	ECS field	Type	Description
uuid	`agent.id`	keyword	Agent's ID
name	`agent.name`	keyword	Agent's name
groups	*`agent.groups`	keyword	Agent's groups
internal_key	*`agent.key`	keyword	Agent's registration key
type	`agent.type`	keyword	Type of agent
version	`agent.version`	keyword	Agent's version
connection_status	*`agent.is_connected`	boolean	Agents' interpreted connection status depending on `agent.last_login`
last_keepalive	*`agent.last_login`	date	Agent's last login
ip	`host.ip`	ip	Host IP addresses. Note: this field should contain an array of values.
os_*	`host.os.full`	keyword	Operating system name, including the version or code name.

* Custom field

ECS mapping

---
name: agent
fields:
  base:
    fields:
      tags: []
  agent:
    fields:
      id: {}
      name: {}
      type: {}
      version: {}
      groups: {}
      key: {}
      last_login: {}
      is_connected: {}
  host:
    fields:
      ip: {}
      os:
        fields:
          full: {}

---
---
- name: agent
  title: Wazuh Agents
  short: Wazuh Inc. custom fields.
  type: group
  group: 2
  fields:
    - name: groups
      type: keyword
      level: custom
      description: >
        The groups the agent belongs to.
    - name: key
      type: keyword
      level: custom
      description: >
        The agent's registration key.
    - name: last_login
      type: date
      level: custom
      description: >
        The agent's last login.
    - name: is_connected
      type: boolean
      level: custom
      description: >
        Agents' interpreted connection status depending on `agent.last_login`.

Index settings

{
    "index_patterns": [".agents*"],
    "priority": 1,
    "template": {
        "settings": {
            "index": {
                "hidden": true,
                "number_of_shards": "1",
                "number_of_replicas": "0",
                "refresh_interval": "5s",
                "query.default_field": [
                    "agent.id",
                    "agent.groups",
                    "agent.name",
                    "agent.type",
                    "agent.version",
                    "agent.name",
                    "host.os.full",
                    "host.ip"
                ]
            }
        }
    }
}

AlexRuiz7 · 2024-06-18T17:27:48Z

`wazuh-alerts-5.x` time series index

Stateless index.

Fields summary

For this stage, we are using all the fields of the ECS. No custom fields are used. As a result, we are using the default mapping of the ECS.

ECS main mappings

The generated template must match this one.

ECS mapping

---
name: main
fields:
    base:
        fields: "*"
    agent:
        fields: "*"
    as:
        fields: "*"
    client:
        fields:
            address: {}
            as:
                fields: "*"
            bytes: {}
            domain: {}
            geo:
                fields: "*"
            ip: {}
            mac: {}
            nat:
                fields:
                    ip: {}
                    port: {}
            packets: {}
            port: {}
            subdomain: {}
            registered_domain: {}
            top_level_domain: {}
            user:
                fields:
                    domain: {}
                    email: {}
                    full_name: {}
                    group:
                        fields: "*"
                    hash: {}
                    id: {}
                    name: {}
                    roles: {}
    cloud:
        fields: "*"
    code_signature:
        fields: "*"
    container:
        fields: "*"
    data_stream:
        fields: "*"
    destination:
        fields:
            address: {}
            as:
                fields: "*"
            bytes: {}
            domain: {}
            geo:
                fields: "*"
            ip: {}
            mac: {}
            nat:
                fields:
                    ip: {}
                    port: {}
            packets: {}
            port: {}
            subdomain: {}
            registered_domain: {}
            top_level_domain: {}
            user:
                fields:
                    domain: {}
                    email: {}
                    full_name: {}
                    group:
                        fields: "*"
                    hash: {}
                    id: {}
                    name: {}
                    roles: {}
    device:
        fields: "*"
    dll:
        fields: "*"
    dns:
        fields: "*"
    ecs:
        fields: "*"
    elf:
        fields: "*"
    email:
        fields: "*"
    error:
        fields: "*"
    event:
        fields: "*"
    faas:
        fields: "*"
    file:
        fields: "*"
    geo:
        fields: "*"
    group:
        fields: "*"
    hash:
        fields: "*"
    host:
        fields: "*"
    http:
        fields: "*"
    interface:
        fields: "*"
    log:
        fields: "*"
    macho:
        fields: "*"
    network:
        fields: "*"
    observer:
        fields: "*"
    orchestrator:
        fields: "*"
    organization:
        fields: "*"
    os:
        fields: "*"
    package:
        fields: "*"
    pe:
        fields: "*"
    process:
        fields:
            args: {}
            args_count: {}
            code_signature:
                fields: "*"
            command_line: {}
            elf:
                fields: "*"
            end: {}
            entity_id: {}
            entry_leader:
                fields:
                    args: {}
                    args_count: {}
                    command_line: {}
                    entity_id: {}
                    entry_meta:
                        fields:
                            type: {}
                            source:
                                fields:
                                    ip: {}
                    executable: {}
                    interactive: {}
                    name: {}
                    parent:
                        fields:
                            entity_id: {}
                            pid: {}
                            vpid: {}
                            start: {}
                            session_leader:
                                fields:
                                    entity_id: {}
                                    pid: {}
                                    vpid: {}
                                    start: {}
                    pid: {}
                    vpid: {}
                    same_as_process: {}
                    start: {}
                    tty:
                        fields:
                            char_device:
                                fields:
                                    major: {}
                                    minor: {}
                    working_directory: {}
                    user:
                        fields:
                            id: {}
                            name: {}
                    real_user:
                        fields:
                            id: {}
                            name: {}
                    saved_user:
                        fields:
                            id: {}
                            name: {}
                    group:
                        fields:
                            id: {}
                            name: {}
                    real_group:
                        fields:
                            id: {}
                            name: {}
                    saved_group:
                        fields:
                            id: {}
                            name: {}
                    supplemental_groups:
                        fields:
                            id: {}
                            name: {}
                    attested_user:
                        fields:
                            id: {}
                            name: {}
                    attested_groups:
                        fields:
                            name: {}
            entry_meta:
                fields:
                    type:
                        docs_only: True
            env_vars: {}
            executable: {}
            exit_code: {}
            group_leader:
                fields:
                    args: {}
                    args_count: {}
                    command_line: {}
                    entity_id: {}
                    executable: {}
                    interactive: {}
                    name: {}
                    pid: {}
                    vpid: {}
                    same_as_process: {}
                    start: {}
                    tty:
                        fields:
                            char_device:
                                fields:
                                    major: {}
                                    minor: {}
                    working_directory: {}
                    user:
                        fields:
                            id: {}
                            name: {}
                    real_user:
                        fields:
                            id: {}
                            name: {}
                    saved_user:
                        fields:
                            id: {}
                            name: {}
                    group:
                        fields:
                            id: {}
                            name: {}
                    real_group:
                        fields:
                            id: {}
                            name: {}
                    saved_group:
                        fields:
                            id: {}
                            name: {}
                    supplemental_groups:
                        fields:
                            id: {}
                            name: {}
            hash:
                fields: "*"
            interactive: {}
            io:
                fields: "*"
            macho:
                fields: "*"
            name: {}
            parent:
                fields:
                    args: {}
                    args_count: {}
                    code_signature:
                        fields: "*"
                    command_line: {}
                    elf:
                        fields: "*"
                    end: {}
                    entity_id: {}
                    executable: {}
                    exit_code: {}
                    group_leader:
                        fields:
                            entity_id: {}
                            pid: {}
                            vpid: {}
                            start: {}
                    hash:
                        fields: "*"
                    interactive: {}
                    macho:
                        fields: "*"
                    name: {}
                    pe:
                        fields: "*"
                    pgid: {}
                    pid: {}
                    vpid: {}
                    start: {}
                    thread:
                        fields:
                            id: {}
                            name: {}
                            capabilities:
                                fields:
                                    effective: {}
                                    permitted: {}
                    title: {}
                    tty:
                        fields:
                            char_device:
                                fields:
                                    major: {}
                                    minor: {}
                    uptime: {}
                    working_directory: {}
                    user:
                        fields:
                            id: {}
                            name: {}
                    real_user:
                        fields:
                            id: {}
                            name: {}
                    saved_user:
                        fields:
                            id: {}
                            name: {}
                    group:
                        fields:
                            id: {}
                            name: {}
                    real_group:
                        fields:
                            id: {}
                            name: {}
                    saved_group:
                        fields:
                            id: {}
                            name: {}
                    supplemental_groups:
                        fields:
                            id: {}
                            name: {}
            pe:
                fields: "*"
            pgid: {}
            pid: {}
            vpid: {}
            previous:
                fields:
                    args: {}
                    args_count: {}
                    executable: {}
            real_group:
                fields:
                    id: {}
                    name: {}
            real_user:
                fields:
                    id: {}
                    name: {}
            same_as_process:
                docs_only: True
            saved_group:
                fields:
                    id: {}
                    name: {}
            saved_user:
                fields:
                    id: {}
                    name: {}
            start: {}
            supplemental_groups:
                fields:
                    id: {}
                    name: {}
            session_leader:
                fields:
                    args: {}
                    args_count: {}
                    command_line: {}
                    entity_id: {}
                    executable: {}
                    interactive: {}
                    name: {}
                    pid: {}
                    vpid: {}
                    same_as_process: {}
                    start: {}
                    tty:
                        fields:
                            char_device:
                                fields:
                                    major: {}
                                    minor: {}
                    working_directory: {}
                    parent:
                        fields:
                            entity_id: {}
                            pid: {}
                            vpid: {}
                            start: {}
                            session_leader:
                                fields:
                                    entity_id: {}
                                    pid: {}
                                    vpid: {}
                                    start: {}
                    user:
                        fields:
                            id: {}
                            name: {}
                    real_user:
                        fields:
                            id: {}
                            name: {}
                    saved_user:
                        fields:
                            id: {}
                            name: {}
                    group:
                        fields:
                            id: {}
                            name: {}
                    real_group:
                        fields:
                            id: {}
                            name: {}
                    saved_group:
                        fields:
                            id: {}
                            name: {}
                    supplemental_groups:
                        fields:
                            id: {}
                            name: {}
            thread:
                fields:
                    id: {}
                    name: {}
                    capabilities:
                        fields:
                            effective: {}
                            permitted: {}
            title: {}
            tty:
                fields: "*"
            uptime: {}
            user:
                fields:
                    id: {}
                    name: {}
            working_directory: {}
    registry:
        fields: "*"
    related:
        fields: "*"
    risk:
        fields: "*"
    rule:
        fields: "*"
    server:
        fields:
            address: {}
            as:
                fields: "*"
            bytes: {}
            domain: {}
            geo:
                fields: "*"
            ip: {}
            mac: {}
            nat:
                fields:
                    ip: {}
                    port: {}
            packets: {}
            port: {}
            subdomain: {}
            registered_domain: {}
            top_level_domain: {}
            user:
                fields:
                    domain: {}
                    email: {}
                    full_name: {}
                    group:
                        fields: "*"
                    hash: {}
                    id: {}
                    name: {}
                    roles: {}
    service:
        fields: "*"
    source:
        fields:
            address: {}
            as:
                fields: "*"
            bytes: {}
            domain: {}
            geo:
                fields: "*"
            ip: {}
            mac: {}
            nat:
                fields:
                    ip: {}
                    port: {}
            packets: {}
            port: {}
            subdomain: {}
            registered_domain: {}
            top_level_domain: {}
            user:
                fields:
                    domain: {}
                    email: {}
                    full_name: {}
                    group:
                        fields: "*"
                    hash: {}
                    id: {}
                    name: {}
                    roles: {}
    threat:
        fields: "*"
    tls:
        fields: "*"
    tracing:
        fields: "*"
    url:
        fields: "*"
    user_agent:
        fields: "*"
    user:
        fields:
            changes:
                fields:
                    domain: {}
                    email: {}
                    group:
                        fields: "*"
                    full_name: {}
                    hash: {}
                    id: {}
                    name: {}
                    roles: {}
            domain: {}
            effective:
                fields:
                    domain: {}
                    email: {}
                    group:
                        fields: "*"
                    full_name: {}
                    hash: {}
                    id: {}
                    name: {}
                    roles: {}
            email: {}
            group:
                fields: "*"
            full_name: {}
            hash: {}
            id: {}
            name: {}
            risk:
                fields: "*"
            roles: {}
            target:
                fields:
                    domain: {}
                    email: {}
                    group:
                        fields: "*"
                    full_name: {}
                    hash: {}
                    id: {}
                    name: {}
                    roles: {}
    vlan:
        fields: "*"
    vulnerability:
        fields: "*"
    x509:
        fields: "*"

{
  "index_patterns": [
    "wazuh-alerts-5.x-*"
  ],
  "priority": 1,
  "template": {
    "settings": {
      "index": {
        "mapping": {
          "total_fields": {
            "limit": 2500
          }
        },
        "refresh_interval": "5s"
      }
    }
  }
}

AlexRuiz7 · 2024-06-18T17:42:09Z

The index templates above are in draft. When reviewed and approved. I'll create the tooling to generate these index templates automatically.

See https://github.com/wazuh/wazuh-indexer/blob/4.9.0/ecs

AlexRuiz7 · 2024-06-26T10:03:31Z

`wazuh-states-fim` index data model

Fields summary

The fields are based on #282 (comment)

Based on ECS:

Field	ECS field	Type	Description
	`agent.id`	keyword	Agent's ID
	*`agent.groups`	keyword	Agent's groups
arch	* ?	keyword	Is arch a file property?
attributes	`file.attributes`	keyword	Array of file attributes.
file	`file.name`	keyword	Name of the file including the extension, without the directory.
full_path	`file.path`	keyword	Full path to the file, including the file name.
gid	`file.gid`	keyword	Primary group ID (GID) of the file.
gname	`file.group`	keyword	Primary group name of the file.
inode	`file.inode`	keyword	Inode representing the file in the filesystem.
md5	`file.hash.md5`	keyword	MD5 hash of the file.
mtime	`file.mtime`	date	Last time the file's metadata changed.
perm	`file.mode`	keyword	File permissions in octal mode.
sha1	`file.hash.sha1`	keyword	SHA1 hash of the file.
sha256	`file.hash.sha256`	keyword	SHA256 hash of the file.
size	`file.size`	long	File size in bytes.
symbolic_path	`file.target_path`	keyword	Target path for symlinks.
type	`file.type`	keyword	File type (file, dir, or symlink).
uid	`file.uid`	keyword	User ID (UID) of the file owner.
uname	`file.owner`	keyword	File owner’s username.
value_name	`registry.key`	keyword	Hive-relative path of keys.
value_type	`registry.value`	keyword	Name of the value written.

* Custom field

ECS mapping

---
name: fim
fields:
    agent:
        fields:
            id: {}
            groups: {}
    file:
        fields:
            attributes: {}
            name: {}
            path: {}
            gid: {}
            group: {}
            inode: {}
            hash:
                fields:
                    md5: {}
                    sha1: {}
                    sha256: {}
            mtime: {}
            mode: {}
            size: {}
            target_path: {}
            type: {}
            uid: {}
            owner: {}
    registry:
        fields:
            key: {}
            value: {}

Index settings

{
    "index_patterns": ["wazuh-states-fim*"],
    "priority": 1,
    "template": {
        "settings": {
            "index": {
                "number_of_shards": "1",
                "number_of_replicas": "0",
                "refresh_interval": "5s",
                "query.default_field": [
                    "agent.id",
                    "agent.groups",
                    "file.name",
                    "file.path",
                    "file.target_path",
                    "file.group",
                    "file.uid",
                    "file.gid"
                ]
            }
        }
    }
}

AlexRuiz7 · 2024-06-26T10:58:25Z

`wazuh-states-vulnerabilities` index data model

Fields summary

The fields are based on https://github.com/wazuh/wazuh-indexer/blob/4.9.0/ecs/vulnerability-detector

Based on ECS:

ECS field	Type	Description
`agent.id`	keyword	Unique identifier of this agent (if one exists).
*`agent.groups`	keyword	Agent's groups
`agent.name`	keyword	Custom name of the agent.
`agent.type`	keyword	Type of the agent.
`agent.version`	keyword	Version of the agent.
`host.os.full`	keyword	Operating system name, including the version or code name.
`host.os.kernel`	keyword	Operating system kernel version as a raw string.
`host.os.name`	keyword	Operating system name, without the version.
`host.os.platform`	keyword	Operating system platform (such centos, ubuntu, windows).
`host.os.type`	keyword	Use the os.type field to categorize the operating system into one of the broad commercial families.
`host.os.version`	keyword	Operating system version as a raw string.
`package.architecture`	keyword	Package architecture.
`package.build_version`	keyword	Additional information about the build version of the installed package.
`package.checksum`	keyword	Checksum of the installed package for verification.
`package.description`	keyword	Description of the package.
`package.install_scope`	keyword	Indicating how the package was installed, e.g. user-local, global.
`package.installed`	date	Time when package was installed.
`package.license`	keyword	License under which the package was released.
`package.name`	keyword	Package name
`package.path`	keyword	Path where the package is installed.
`package.reference`	keyword	Home page or reference URL of the software in this package, if available.
`package.size`	long	Package size in bytes.
`package.type`	keyword	Type of package.
`package.version`	keyword	Package version
`vulnerability.category`	keyword	The type of system or architecture that the vulnerability affects
`vulnerability.classification`	keyword	The classification of the vulnerability scoring system.
`vulnerability.description`	keyword	The description of the vulnerability that provides additional context of the vulnerability
*`vulnerability.detected_at`	date	Vulnerability's detection date.
`vulnerability.enumeration`	keyword	The type of identifier used for this vulnerability.
`vulnerability.id`	keyword	The identification (ID) is the number portion of a vulnerability entry.
*`vulnerability.published_at`	date	Vulnerability's publication date.
`vulnerability.reference`	keyword	A resource that provides additional information, context, and mitigations for the identified vulnerability.
`vulnerability.report_id`	keyword	The report or scan identification number.
*`vulnerability.scanner.source`	keyword	The origin of the decision of the scanner (AKA feed used to detect the vulnerability).
`vulnerability.scanner.vendor`	keyword	The name of the vulnerability scanner vendor.
`vulnerability.score.base`	float	Scores can range from 0.0 to 10.0, with 10.0 being the most severe.
`vulnerability.score.environmental`	float	Scores can range from 0.0 to 10.0, with 10.0 being the most severe.
`vulnerability.score.temporal`	float	Scores can range from 0.0 to 10.0, with 10.0 being the most severe.
`vulnerability.score.version`	keyword	The National Vulnerability Database (NVD) provides qualitative severity rankings of "Low", "Medium", and "High" for CVSS v2.0 base score ranges in addition to the severity ratings for CVSS v3.0 as they are defined in the CVSS v3.0 specification.
`vulnerability.severity`	keyword	The severity of the vulnerability can help with metrics and internal prioritization regarding remediation.
*`vulnerability.under_evaluation`	boolean	Indicates if the vulnerability is awaiting analysis by the NVD.
*`wazuh.cluster.name`	keyword	Name of the Wazuh cluster.
*`wazuh.cluster.node`	keyword	Name of the Wazuh cluster node.
*`wazuh.schema.version`	keyword	Version of the Wazuh schema.

* Custom field

ECS mapping

---
name: wazuh-states-vulnerabilities
fields:
    base:
        tags: []
    agent:
        fields: "*"
    package:
        fields: "*"
    host:
        fields:
            os:
                fields:
                    full: ""
                    kernel: ""
                    name: ""
                    platform: ""
                    type: ""
                    version: ""
    vulnerability:
        fields: "*"
    wazuh:
        fields: "*"

---
- name: vulnerability
  title: Vulnerability
  group: 2
  short: Fields to describe the vulnerability relevant to an event.
  description: >
      The vulnerability fields describe information about a vulnerability that is
      relevant to an event.
  type: group
  fields:
      - name: detected_at
        type: date
        level: custom
        description: >
            Vulnerability's detection date.
      - name: published_at
        type: date
        level: custom
        description: >
            Vulnerability's publication date.
      - name: under_evaluation
        type: boolean
        level: custom
        description: >
            Indicates if the vulnerability is awaiting analysis by the NVD.
      - name: scanner.source
        type: keyword
        level: custom
        description: >
            The origin of the decision of the scanner (AKA feed used to detect the vulnerability).

---
---
- name: wazuh
  title: Wazuh
  description: >
      Wazuh Inc. custom fields
  fields:
      - name: cluster.name
        type: keyword
        level: custom
        description: >
            Wazuh cluster name.
      - name: cluster.node
        type: keyword
        level: custom
        description: >
            Wazuh cluster node name.
      - name: schema.version
        type: keyword
        level: custom
        description: >
            Wazuh schema version.

Index settings

{
    "index_patterns": ["wazuh-states-vulnerabilities*"],
    "priority": 1,
    "template": {
        "settings": {
            "index": {
                "number_of_shards": "1",
                "number_of_replicas": "0",
                "refresh_interval": "5s",
                "query.default_field": [
                    "agent.id",
                    "agent.group",
                    "host.os.full",
                    "host.os.version",
                    "package.name",
                    "package.version",
                    "vulnerability.id",
                    "vulnerability.description",
                    "vulnerability.severity",
                    "wazuh.cluster.name"
                ]
            }
        }
    }
}

AlexRuiz7 · 2024-07-02T09:53:41Z

Index templates for wazuh-monitoring and wazuh-statistics will be done for the MVP phase.

AlexRuiz7 · 2024-07-02T09:55:02Z

Index template for stateful-sca will be done for the MVP phase. It contains many static data that should not be indexed as part of each document.

marcinliszka12 · 2024-09-18T11:46:22Z

Hello. Please add agent.labels in Vulnerability Module the same way like in "Threat Hunting" module. Labels are awesome and can easy filter data based on personal entries in agent configuration.

f-galland · 2024-11-07T10:21:38Z

`wazuh-states-inventory-networks` index data model

Fields summary

The fields are based on #282 (comment)

Based on ECS:

	Field name	ECS field name	Data type	Description
	adapter	observer.ingress.interface.alias	keyword	Adapter name of the network interface
	address	host.ip	ip	Network address
	iface	observer.ingress.interface.name	keyword	Name of the network interface
	inode	file.inode	keyword	The unix inode of the port
	item_id	device.id	keyword	Identifier of interface/protocol/address/port item
	local_ip	source.ip	ip	Local IP address
	local_port	source.port	long	Local port number
	mac	host.mac	keyword	MAC address of the network interface
	name	observer.ingress.interface.name	keyword	Name of the network interface
	pid	process.pid	long	Process ID
	process	process.name	keyword	Process name
	proto	network.protocol	keyword	Type of network protocol
	protocol	network.protocol	keyword	Protocol used
	protocol	network.protocol	keyword	Protocol used
	remote_ip	destination.ip	ip	Remote IP address
	remote_port	destination.port	long	Remote port number
	rx_bytes	host.network.ingress.bytes	long	Number of received bytes
	rx_packets	host.network.ingress.packets	long	Number of received packets
	scan_id	event.id	keyword	Reference to the scan information
	scan_time	@timestamp	date	Timestamp of the scan
	tx_bytes	host.network.egress.bytes	long	Number of transmitted bytes
	tx_packets	host.network.egress.packets	long	Number of transmitted packets
	type	network.type	keyword	IPv4 or IPv6 for protocols, interface type for interface records
*	broadcast	network.broadcast	ip	Broadcast address
*	dhcp	network.dhcp	keyword	DHCP status (enabled, disabled, unknown, BOOTP)
*	gateway	network.gateway	ip	Gateway address
*	metric	network.metric	long	Metric of the network protocol
*	mtu	interface.mtu	long	Maximum transmission unit size
*	netmask	network.netmask	ip	Network mask
*	rx_dropped	host.network.ingress.drops	long	Number of dropped received packets
*	rx_errors	host.network.ingress.errors	long	Number of reception errors
*	rx_queue	host.network.ingress.queue	long	Receive queue length
*	state	interface.state	keyword	State of the network interface
*	tx_dropped	host.network.egress.drops	long	Number of dropped transmitted packets
*	tx_errors	host.network.egress.errors	long	Number of transmission errors
*	tx_queue	host.network.egress.queue	long	Transmit queue length
*	type	interface.type	keyword	Interface type (eg. "wireless" or "ethernet")

* Custom fields

ECS mapping

---
name: wazuh-states-inventory-networks
fields:
  base:
    fields:
      tags: []
      "@timestamp": {}
  agent:
    fields:
      id: {}
      groups: {}
  destination:
    fields:
      ip: {}
      port: {}
  device:
    fields:
      id: {}
  file:
    fields:
      inode: {}
  host:
    fields:
      ip: {}
      mac: {}
      network:
        fields:
          egress:
            fields:
              bytes: {}
              packets: {}
          ingress:
            fields:
              bytes: {}
              packets: {}
  network:
    fields:
      protocol: {}
      type: {}
  observer:
    fields:
      ingress:
        fields:
          interface:
            fields:
              alias: {}
              name: {}
  process:
    fields:
      name: {}
      pid: {}
  source:
    fields:
      ip: {}
      port: {}

Index settings

{
  "index_patterns": [
    "wazuh-states-inventory-networks*"
  ],
  "priority": 1,
  "template": {
    "settings": {
      "index": {
        "number_of_replicas": "0",
        "number_of_shards": "1",
        "query.default_field": [
          "agent.id",
          "agent.groups",
          "device.id",
          "event.id",
          "host.ip",
          "observer.ingress.interface.name",
          "observer.ingress.interface.alias",
          "process.name"
        ],
        "refresh_interval": "5s"
      }
    },
    "mappings": {
      "date_detection": false,
      "dynamic": "strict",
      "properties": {
        "@timestamp": {
          "type": "date"
        },
        "agent": {
          "properties": {
            "groups": {
              "ignore_above": 1024,
              "type": "keyword"
            },
            "id": {
              "ignore_above": 1024,
              "type": "keyword"
            }
          }
        },
        "destination": {
          "properties": {
            "ip": {
              "type": "ip"
            },
            "port": {
              "type": "long"
            }
          }
        },
        "device": {
          "properties": {
            "id": {
              "ignore_above": 1024,
              "type": "keyword"
            }
          }
        },
        "file": {
          "properties": {
            "inode": {
              "ignore_above": 1024,
              "type": "keyword"
            }
          }
        },
        "host": {
          "properties": {
            "ip": {
              "type": "ip"
            },
            "mac": {
              "ignore_above": 1024,
              "type": "keyword"
            },
            "network": {
              "properties": {
                "egress": {
                  "properties": {
                    "bytes": {
                      "type": "long"
                    },
                    "packets": {
                      "type": "long"
                    }
                  }
                },
                "ingress": {
                  "properties": {
                    "bytes": {
                      "type": "long"
                    },
                    "packets": {
                      "type": "long"
                    }
                  }
                }
              }
            }
          }
        },
        "network": {
          "properties": {
            "protocol": {
              "ignore_above": 1024,
              "type": "keyword"
            },
            "type": {
              "ignore_above": 1024,
              "type": "keyword"
            }
          }
        },
        "observer": {
          "properties": {
            "ingress": {
              "properties": {
                "interface": {
                  "properties": {
                    "alias": {
                      "ignore_above": 1024,
                      "type": "keyword"
                    },
                    "name": {
                      "ignore_above": 1024,
                      "type": "keyword"
                    }
                  }
                }
              },
              "type": "object"
            }
          }
        },
        "process": {
          "properties": {
            "name": {
              "fields": {
                "text": {
                  "type": "match_only_text"
                }
              },
              "ignore_above": 1024,
              "type": "keyword"
            },
            "pid": {
              "type": "long"
            }
          }
        },
        "source": {
          "properties": {
            "ip": {
              "type": "ip"
            },
            "port": {
              "type": "long"
            }
          }
        }
      }
    }
  }
}

f-galland · 2024-11-07T12:06:58Z

`wazuh-states-inventory-hardware` index data model

Fields summary

The fields are based on #282 (comment)

Based on ECS:

	Field name	ECS field name	Data type	Description
	scan_time	@timestamp	date	Timestamp of the scan
	board_serial	observer.serial_number	keyword	Serial number of the motherboard
*	cpu_name	host.cpu.name	keyword	Name of the CPU
*	cpu_cores	host.cpu.cores	long	Number of CPU cores
*	cpu_mhz	host.cpu.speed	long	Speed of the CPU in MHz
*	ram_total	host.memory.total	long	Total RAM in the system
*	ram_free	host.memory.free	long	Free RAM in the system
*	ram_usage	host.memory.used.percentage	long	RAM usage as a percentage

* Custom fields

ECS mapping

---
name: wazuh-states-inventory-hardware
fields:
  base:
    fields:
      tags: []
      "@timestamp": {}
  agent:
    fields:
      id: {}
      groups: {}
  observer:
    fields:
      serial_number: {}
  host:
    fields:
      memory:
        fields:
          total: {}
          free: {}
          used:
            fields:
              percentage: {}
      cpu:
        fields:
          name: {}
          cores: {}
          speed: {}

Index settings

{
  "index_patterns": [
    "wazuh-states-inventory-hardware*"
  ],
  "priority": 1,
  "template": {
    "settings": {
      "index": {
        "number_of_replicas": "0",
        "number_of_shards": "1",
        "query.default_field": [
          "observer.board_serial"
        ],
        "refresh_interval": "5s"
      }
    },
    "mappings": {
      "date_detection": false,
      "dynamic": "strict",
      "properties": {
        "@timestamp": {
          "type": "date"
        },
        "agent": {
          "properties": {
            "groups": {
              "ignore_above": 1024,
              "type": "keyword"
            },
            "id": {
              "ignore_above": 1024,
              "type": "keyword"
            }
          }
        },
        "host": {
          "properties": {
            "cpu": {
              "properties": {
                "cores": {
                  "type": "long"
                },
                "name": {
                  "ignore_above": 1024,
                  "type": "keyword"
                },
                "speed": {
                  "type": "long"
                }
              },
              "type": "object"
            },
            "memory": {
              "properties": {
                "free": {
                  "type": "long"
                },
                "total": {
                  "type": "long"
                },
                "used": {
                  "properties": {
                    "percentage": {
                      "type": "long"
                    }
                  },
                  "type": "object"
                }
              },
              "type": "object"
            }
          }
        },
        "observer": {
          "properties": {
            "serial_number": {
              "ignore_above": 1024,
              "type": "keyword"
            }
          }
        }
      }
    }
  }
}

f-galland · 2024-11-07T12:40:59Z

`wazuh-states-inventory-hotfixes` index data model

Fields summary

The fields are based on #282 (comment)

Based on ECS:

Package Fields.

	Field name	ECS field name	Data type	Description
	scan_time	@timestamp	date	Timestamp of the scan
*	hotfix	package.hotfix.name	keyword	Name of the hotfix

* Custom fields

ECS mapping

---
name: wazuh-states-inventory-hotfixes
fields:
  base:
    fields:
      tags: []
      "@timestamp": {}
  agent:
    fields:
      id: {}
      groups: {}
  package:
    fields:
      hotfix:
        fields:
          name: {}

Index settings

{
  "index_patterns": [
    "wazuh-states-inventory-hotfixes*"
  ],
  "priority": 1,
  "template": {
    "settings": {
      "index": {
        "number_of_replicas": "0",
        "number_of_shards": "1",
        "query.default_field": [
          "package.hotfix.name"
        ],
        "refresh_interval": "5s"
      }
    },
    "mappings": {
      "date_detection": false,
      "dynamic": "strict",
      "properties": {
        "@timestamp": {
          "type": "date"
        },
        "agent": {
          "properties": {
            "groups": {
              "ignore_above": 1024,
              "type": "keyword"
            },
            "id": {
              "ignore_above": 1024,
              "type": "keyword"
            }
          }
        },
        "package": {
          "properties": {
            "hotfix": {
              "properties": {
                "name": {
                  "ignore_above": 1024,
                  "type": "keyword"
                }
              },
              "type": "object"
            }
          }
        }
      }
    }
  }
}

f-galland · 2024-11-11T10:07:15Z

`wazuh-states-inventory-ports` index data model

Fields summary

The fields are based on #282 (comment)

Based on ECS:

	Field name	ECS field name	Data type	Description
	inode	file.inode	keyword	The unix inode of the port
	item_id	device.id	keyword	Identifier of interface/protocol/address/port item
	local_ip	source.ip	ip	Local IP address
	local_port	source.port	long	Local port number
	pid	process.pid	long	Process ID
	process	process.name	keyword	Process name
	protocol	network.protocol	keyword	Protocol used
	remote_ip	destination.ip	ip	Remote IP address
	remote_port	destination.port	long	Remote port number
	scan_time	@timestamp	date	Timestamp of the scan
*	rx_queue	host.network.ingress.queue	long	Receive queue length
*	state	interface.state	keyword	State of the network interface
*	tx_queue	host.network.egress.queue	long	Transmit queue length

* Custom fields

ECS mapping

---
name: wazuh-states-inventory-ports
fields:
  base:
    fields:
      tags: []
      "@timestamp": {}
  agent:
    fields:
      id: {}
      groups: {}
  destination:
    fields:
      ip: {}
      port: {}
  device:
    fields:
      id: {}
  file:
    fields:
      inode: {}
  host:
    fields:
      network:
        fields:
          egress:
            fields:
              queue: {}
          ingress:
            fields:
              queue: {}
  network:
    fields:
      protocol: {}
  process:
    fields:
      name: {}
      pid: {}
  source:
    fields:
      ip: {}
      port: {}
  interface:
    fields:
      state: {}

Index settings

{
  "index_patterns": [
    "wazuh-states-inventory-ports*"
  ],
  "priority": 1,
  "template": {
    "settings": {
      "index": {
        "number_of_shards": "1",
        "number_of_replicas": "0",
        "refresh_interval": "5s",
        "query.default_field": [
          "agent.id",
          "agent.groups",
          "process.name",
          "source.ip",
          "destination.ip"
        ]
      }
    }
  }
}

AlexRuiz7 added level/task Task issue request/operational Operational requests type/research Research issue labels Jun 18, 2024

This was referenced Jun 18, 2024

Spike - Initial indexer initialization plugin design #256

Closed

Initialize wazuh-indexer-setup plugin wazuh/wazuh-indexer-plugins#3

Closed

wazuhci added this to Release 5.0.0 Jun 18, 2024

wazuhci moved this to Backlog in Release 5.0.0 Jun 18, 2024

wazuhci moved this from Backlog to Pending review in Release 5.0.0 Jun 18, 2024

AlexRuiz7 self-assigned this Jun 19, 2024

wazuhci moved this from Pending review to In progress in Release 5.0.0 Jun 25, 2024

vikman90 mentioned this issue Jun 25, 2024

Define fields for stateful modules data #282

Closed

This comment was marked as outdated.

Sign in to view

wazuhci moved this from In progress to On hold in Release 5.0.0 Jun 26, 2024

This comment was marked as outdated.

Sign in to view

wazuhci moved this from On hold to In progress in Release 5.0.0 Jun 27, 2024

wazuhci moved this from In progress to Pending final review in Release 5.0.0 Jul 2, 2024

wazuhci moved this from Pending final review to On hold in Release 5.0.0 Jul 9, 2024

This was referenced Jul 9, 2024

Indexer initialization plugin wazuh/wazuh-indexer-plugins#9

Closed

Create UML diagrams for the wazuh-indexer-setup plugin wazuh/wazuh-indexer-plugins#11

Closed

AlexRuiz7 mentioned this issue Aug 7, 2024

Data model definition #344

Closed

4 tasks

This was referenced Aug 7, 2024

POC data persistence UX - endpoint inventory wazuh/wazuh-dashboard#213

Closed

POC data persistence UX - agent commands wazuh/wazuh-dashboard#214

Closed

GGP1 mentioned this issue Aug 14, 2024

Comms API - POST /events/stateful endpoint implementation wazuh/wazuh#24647

Closed

This was referenced Sep 11, 2024

Add states-inventory-system index template definition to the ECS tooling #402

Closed

Add states-vulnerabilities index template definition to the ECS tooling #404

Closed

wazuhci moved this from In progress to On hold in Release 5.0.0 Sep 16, 2024

GGP1 mentioned this issue Sep 16, 2024

Update agent model wazuh/wazuh#25726

Merged

AlexRuiz7 mentioned this issue Oct 15, 2024

Add ECS docs #462

Closed

3 tasks

This was referenced Oct 28, 2024

Update stateful events models wazuh/wazuh#26575

Merged

Update stateful events data models wazuh/wazuh#26568

Closed

cborla mentioned this issue Nov 1, 2024

Events generated by agent must comply with the common schema wazuh/wazuh-agent#253

Closed

f-galland self-assigned this Nov 6, 2024

wazuhci moved this from On hold to In progress in Release 5.0.0 Nov 6, 2024

wazuhci moved this from In progress to Pending review in Release 5.0.0 Nov 7, 2024

wazuhci moved this from Pending review to In progress in Release 5.0.0 Nov 7, 2024

AlexRuiz7 mentioned this issue Nov 7, 2024

Events generator for 5.x indices #518

Closed

9 tasks

wazuhci moved this from In progress to Pending review in Release 5.0.0 Nov 7, 2024

wazuhci moved this from Pending review to In progress in Release 5.0.0 Nov 8, 2024

vikman90 mentioned this issue Nov 8, 2024

Refactor network interfaces and protocols inventory tables wazuh/wazuh-agent#284

Closed

wazuhci moved this from In progress to Pending review in Release 5.0.0 Nov 11, 2024

AlexRuiz7 mentioned this issue Nov 12, 2024

Add index templates definitions for Wazuh 5 #543

Merged

3 tasks

AlexRuiz7 closed this as completed in #543 Nov 12, 2024

wazuhci moved this from Pending review to Done in Release 5.0.0 Nov 12, 2024

fdalmaup mentioned this issue Nov 27, 2024

Comms API - Stateful endpoint with ECS compliant models wazuh/wazuh#27079

Closed

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Create ECS compliant index templates #270

Create ECS compliant index templates #270

AlexRuiz7 commented Jun 18, 2024 •

edited by f-galland

Loading

AlexRuiz7 commented Jun 18, 2024 •

edited

Loading

AlexRuiz7 commented Jun 18, 2024 •

edited

Loading

AlexRuiz7 commented Jun 18, 2024 •

edited

Loading

This comment was marked as outdated.

AlexRuiz7 commented Jun 26, 2024 •

edited

Loading

AlexRuiz7 commented Jun 26, 2024 •

edited

Loading

This comment was marked as outdated.

AlexRuiz7 commented Jul 2, 2024

AlexRuiz7 commented Jul 2, 2024

marcinliszka12 commented Sep 18, 2024 •

edited

Loading

f-galland commented Nov 7, 2024 •

edited

Loading

f-galland commented Nov 7, 2024 •

edited

Loading

f-galland commented Nov 7, 2024 •

edited

Loading

f-galland commented Nov 11, 2024

Create ECS compliant index templates #270

Create ECS compliant index templates #270

Comments

AlexRuiz7 commented Jun 18, 2024 • edited by f-galland Loading

Description

Tasks

Implementation restrictions

AlexRuiz7 commented Jun 18, 2024 • edited Loading

agents index data model

Fields summary

ECS mapping

Index settings

AlexRuiz7 commented Jun 18, 2024 • edited Loading

wazuh-alerts-5.x time series index

Fields summary

ECS mapping

AlexRuiz7 commented Jun 18, 2024 • edited Loading

This comment was marked as outdated.

AlexRuiz7 commented Jun 26, 2024 • edited Loading

wazuh-states-fim index data model

Fields summary

ECS mapping

Index settings

AlexRuiz7 commented Jun 26, 2024 • edited Loading

wazuh-states-vulnerabilities index data model

Fields summary

ECS mapping

Index settings

This comment was marked as outdated.

AlexRuiz7 commented Jul 2, 2024

AlexRuiz7 commented Jul 2, 2024

marcinliszka12 commented Sep 18, 2024 • edited Loading

f-galland commented Nov 7, 2024 • edited Loading

wazuh-states-inventory-networks index data model

Fields summary

ECS mapping

Index settings

f-galland commented Nov 7, 2024 • edited Loading

wazuh-states-inventory-hardware index data model

Fields summary

ECS mapping

Index settings

f-galland commented Nov 7, 2024 • edited Loading

wazuh-states-inventory-hotfixes index data model

Fields summary

ECS mapping

Index settings

f-galland commented Nov 11, 2024

wazuh-states-inventory-ports index data model

Fields summary

ECS mapping

Index settings

AlexRuiz7 commented Jun 18, 2024 •

edited by f-galland

Loading

AlexRuiz7 commented Jun 18, 2024 •

edited

Loading

`agents` index data model

AlexRuiz7 commented Jun 18, 2024 •

edited

Loading

`wazuh-alerts-5.x` time series index

AlexRuiz7 commented Jun 18, 2024 •

edited

Loading

AlexRuiz7 commented Jun 26, 2024 •

edited

Loading

`wazuh-states-fim` index data model

AlexRuiz7 commented Jun 26, 2024 •

edited

Loading

`wazuh-states-vulnerabilities` index data model

marcinliszka12 commented Sep 18, 2024 •

edited

Loading

f-galland commented Nov 7, 2024 •

edited

Loading

`wazuh-states-inventory-networks` index data model

f-galland commented Nov 7, 2024 •

edited

Loading

`wazuh-states-inventory-hardware` index data model

f-galland commented Nov 7, 2024 •

edited

Loading

`wazuh-states-inventory-hotfixes` index data model

`wazuh-states-inventory-ports` index data model