DOCS-72 - Cloud SIEM rule limits

cid-redirects.json

@@ -1636,6 +1636,7 @@
   "/cid/10146": "/docs/cse/sensors",
   "/cid/10147": "/docs/cse/integrations",
   "/cid/10148": "/docs/cse/rules",
+  "/cid/101481": "/docs/cse/rules/about-cse-rules",
   "/cid/10149": "/docs/cse/rules/write-match-rule",
   "/cid/1015": "/docs/send-data/reference-information/use-wildcards-paths",
   "/cid/10150": "/docs/cse/rules/write-threshold-rule",

docs/cse/rules/about-cse-rules.md

@@ -9,7 +9,7 @@ import useBaseUrl from '@docusaurus/useBaseUrl';
 
 A Cloud SIEM rule is logic that fires based on information in incoming records. When a rule fires, it creates a signal.
 
-[**Classic UI**](/docs/get-started/sumo-logic-ui-classic). To view rules, in the top menu select **Content > Rules**. 
+[**Classic UI**](/docs/get-started/sumo-logic-ui-classic). To view Rules, in the top Cloud SIEM menu select **Content > Rules**. 
 
 [**New UI**](/docs/get-started/sumo-logic-ui). To view rules, in the main Sumo Logic menu select **Cloud SIEM > Rules**. You can also click the **Go To...** menu at the top of the screen and select **Rules**. 
 
@@ -32,6 +32,100 @@ Watch this micro lesson to learn more about rules.
 
 import Iframe from 'react-iframe'; 
 
+## Rule types
+
+There are several kinds of rules. Each supports a different sort of firing behavior (For a complete list of out-of-the-box rules, see [Rules](https://github.com/SumoLogic/cloud-siem-content-catalog/blob/master/rules/README.md) in the [Cloud SIEM Content Catalog](https://github.com/SumoLogic/cloud-siem-content-catalog/blob/master/README.md).)
+
+* **Match rule**. Fires when an incoming record matches the rule expression. A match rule is stateless: it looks at a single record, and it either fires or it doesn’t. The expression in the previous section is an example of a match rule expression. If a record matches the expression, the rule fires. For more information about match rules, see [Write a Match Rule](/docs/cse/rules/write-match-rule).
+* **Chain rule**. You can use a chain rule to look for two or more types of events, and to fire, based on the frequency of each over a time window. For example, when a user has more than 10 failed login attempts and one successful login attempt in a one hour window. Like a threshold rule, a chain rule is stateful and counts multiple records—the difference is that a chain rule applies multiple expressions to a record. For more information about chain rules, see [Write a Chain Rule](/docs/cse/rules/write-chain-rule).
+* **Aggregation rule**. Fires when up to three aggregation conditions are met within a specified period of time. For example, when a large variety of different AWS CloudTrail event IDs from the same `device_ip` are observed within a 30 minute period. For more information about aggregation rules, see [Write an Aggregation Rule](/docs/cse/rules/write-aggregation-rule).
+* **Threshold rule**. Fires when the rule expression is matched at least a certain number times during a specified length of time. For example, if there are five or more failed login attempts for the same IP address within one hour. A threshold rule is stateful, a condition must be satisfied by multiple records over a period of time. For more information about threshold rules, see [Write a Threshold Rule](/docs/cse/rules/write-threshold-rule).
+* **First Seen rule**. Fires when behavior by an entity is encountered that hasn't been seen before. For example, the first time when a user logs in from a new location, or when a new admin account is created. For more information about first seen rules, see [Write a First Seen Rule](/docs/cse/rules/write-first-seen-rule).
+* **Outlier rule**. Fires when behavior by an entity is encountered that deviates from its baseline activity. For each outlier rule, Cloud SIEM automatically creates a baseline model of normal behavior. After the baseline learning period is completed, activity that deviates from the mean (normal baseline behavior) creates a signal. For more information about outlier rules, see [Write an Outlier Rule](/docs/cse/rules/write-outlier-rule).
+
+## Rule limits
+
+Limits are set on how often a rule fires so that the system is not overloaded. For example, if a rule fires too many signals in an hour, it can cause performance problems for all rule processes. If a rule exceeds a limit, its [rule status](#rule-status) changes from Active to Failed and the rule is disabled. 
+
+| Type | Limit |
+| :-- | :-- |
+| Total allowed custom rules of each [rule type](#rule-types) | 100 <!-- <br/>200 - Tier 2<br/>500 - Tier 3 --> |
+| Signals per hour | 50K<!-- - Tier 1<br/>100K - Tier 2<br/>150K - Tier 3 -->|
+| Signals per 24 hours | 1M<!-- Tier 1<br/>2M - Tier 2<br/>3M - Tier 3 --> |
+| Matched records per day* | 200K<!-- - Tier 1<br/>400K - Tier 2<br/>600K - Tier 3 --> |
+| Rule group cardinality per day** | 100K<!-- - Tier 1<br/>200K - Tier 2<br/>300K - Tier 3 --> |
+
+*Applies to all [rule types](#rule-types) except match rules. 
+<br/>**Group cardinality is the number of distinct key values in a grouping function of a complex rule type. For instance, if a rule is grouped by email address, the cardinality would be the total number of distinct email addresses.
+
+:::note
+Rule limits can be higher if you are in a higher tenant tier level. If you have questions about what your tenant tier level is, contact your Sumo Logic account representative or [contact Sumo Logic Support](https://support.sumologic.com/support/s/).
+:::
+
+## Rule status
+
+### View a rule's status
+
+You can see a rule's status while viewing the rule:
+* On the rule list page: <br/><img src={useBaseUrl('img/cse/rule-status-on-list-page.png')} alt="Rule status on list page" style={{border: '1px solid gray'}} width="800"/>
+* On the rule details page: <br/><img src={useBaseUrl('img/cse/rule-status-on-detail-page.png')} alt="Rule status on details page" style={{border: '1px solid gray'}} width="800"/>
+
+### Kinds of rule status
+
+Following are the different kinds of rule statuses. A rule's status can change depending on whether it exceeds [rule limits](#rule-limits).
+
+| Status | Description | Action required |
+| :-- | :-- | :-- |
+| **Active** | The rule is executing normally. | No action required. |
+| **Degraded** | The rule is approaching a rule limit and it is removed from execution for one hour to allow processing to catch up. At the end of the hour, the rule is allowed to execute again and its status changes back to Active. | Click the information button <img src={useBaseUrl('img/cse/rule-degraded-info-button.png')} alt="Rule degraded information button" width="20"/> on the **Degraded** label for details. Depending on the information provided, you may want to edit the rule to reduce the chance it will become degraded again later. |
+| **Disabled** | The rule was manually disabled using the toggle in the UI, or was disabled with the API. | Enable the rule with the toggle in the UI, or enable the rule with the [API](https://api.sumologic.com/docs/sec/#operation/UpdateRuleEnabled). |
+| **Failed** | The rule exceeded a rule limit and was automatically disabled. | Click the information button <img src={useBaseUrl('img/cse/rule-failed-info-button.png')} alt="Rule failed information button" width="20"/> on the **Failed** label for details about the failure.  Depending on the reasons provided in the details, you may need to edit the rule to prevent it from failing again in the future. After addressing the reasons for the failure, enable the rule with the toggle in the UI, or enable the rule with the API. |
+| **Warning** | The rule is approaching a rule limit and risks being disabled. | Click the information button <img src={useBaseUrl('img/cse/rule-warning-info-button.png')} alt="Rule warning information button" width="20"/> on the **Warning** label for details about the warning. Depending on the reasons provided in the details, you may need to edit the rule to prevent it from being disabled. |
+
+### Query for rule status changes
+
+You can query audit logs for rule status changes. (For more information about querying audit logs, see [Cloud SIEM Audit Logging](/docs/cse/administration/cse-audit-logging/).)
+
+#### Example query for rule status changes
+
+The following query queries for match rules whose status was changed automatically to `Warning` by the system:
+
+```sql
+_index=sumologic_system_events _sourceCategory=cseRule
+| json field=_raw "templatedMatchRule.status"
+| where eventname = "TemplatedMatchRuleUpdated" 
+| where templatedMatchRule.status = "Warning"
+```
+
+You can set up this query for one or multiple rules, with one or multiple statuses.
+* To query for other rule types, replace the `field` and `eventname` strings. 
+* To query for for other statuses, replace `Warning` in the example above with another status. For example:
+   * `"Active"`
+   * `"Degraded"`
+   * `"Disabled"`
+   * `"Failed"`
+   * `"Warning"`
+
+#### Example query for disabled rules
+
+If you want to query simply for match rules that are disabled, you could execute a query like this:
+
+```sql
+(_index=sumologic_audit_events OR _index=sumologic_system_events) _sourceCategory=cseRule
+| json field=_raw "templatedMatchRule.enabled"
+| where eventname = "TemplatedMatchRuleUpdated" 
+| where templatedMatchRule.enabled = "false"
+```
+
+This query looks for match rules that were manually disabled (`_index=sumologic_audit_events`) or automatically disabled by the system (`_index=sumologic_system_events`).
+
+### Create a monitor to alert on rule status changes
+
+You can [create a monitor](/docs/alerts/monitors/create-monitor/) to generate alerts when rules statuses change. This will alert you when you need to take action.
+
+For example, you could use the [example query for rule status changes](#example-query-for-rule-status-changes) above in your monitor. It will alert when the status of match rules change to `Warning`. 
+
+<img src={useBaseUrl('img/cse/example-monitor-for-rule-status-change.png')} alt="Example monitor for rule status change" style={{border: '1px solid gray'}} width="700"/>
 
 ## About rule expressions
 
@@ -77,17 +171,6 @@ The screenshot below shows a rule whose "On Entity" attributes are `srcDevice
 
 When an incoming record meets a rule's conditions, a signal is generated for each of the rule's On Entity attributes found in the record. When the example rule above fires, it generates two signals: one on the IP address held in the `srcDevice_ip` attribute, and  another on the IP address held in the `dstDevice_ip` attribute.
 
-## Rule types
-
-There are several kinds of rules. Each supports a different sort of firing behavior. (For a complete list of out-of-the-box rules, see [Rules](https://github.com/SumoLogic/cloud-siem-content-catalog/blob/master/rules/README.md) in the [Cloud SIEM Content Catalog](https://github.com/SumoLogic/cloud-siem-content-catalog/blob/master/README.md).)
-
-* **Match rule**. Fires when an incoming record matches the rule expression. A match rule is stateless: it looks at a single record, and it either fires or it doesn’t. The expression in the previous section is an example of a match rule expression. If a record matches the expression, the rule fires. For more information about match rules, see [Write a Match Rule](/docs/cse/rules/write-match-rule).
-* **Chain rule**. You can use a chain rule to look for two or more types of events, and to fire, based on the frequency of each over a time window. For example, when a user has more than 10 failed login attempts and one successful login attempt in a one hour window. Like a threshold rule, a chain rule is stateful and counts multiple records. The difference is that a chain rule applies multiple expressions to a record. For more information about chain rules, see [Write a Chain Rule](/docs/cse/rules/write-chain-rule).
-* **Aggregation rule**. Fires when up to three aggregation conditions are met within a specified period of time. For example, when a large variety of different AWS CloudTrail event IDs from the same `device_ip` are observed within a 30 minute period. For more information about aggregation rules, see [Write an Aggregation Rule](/docs/cse/rules/write-aggregation-rule).
-* **Threshold rule**. Fires when the rule expression is matched at least a certain number times during a specified length of time. For example, if there are five or more failed login attempts for the same IP address within one hour. A threshold rule is stateful, a condition must be satisfied by multiple records over a period of time. For more information about threshold rules, see [Write a Threshold Rule](/docs/cse/rules/write-threshold-rule).
-* **First seen rule**. Fires when behavior by an entity is encountered that hasn't been seen before. For example, the first time when a user logs in from a new location, or when a new admin account is created. For more information about first seen rules, see [Write a First Seen Rule](/docs/cse/rules/write-first-seen-rule).
-* **Outlier rule**. Fires when behavior by an entity is encountered that deviates from its baseline activity. For each outlier rule, Cloud SIEM automatically creates a baseline model of normal behavior. After the baseline learning period is completed, activity that deviates from the mean (normal baseline behavior) creates a signal. For more information about outlier rules, see [Write an Outlier Rule](/docs/cse/rules/write-outlier-rule).
-
 ## Product identification metadata fields
 
 During the record parsing process, Cloud SIEM adds metadata that identifies the product or service that generated the record. You use this metadata in a rule to specify what records the rule should be applied to. For example, the rule fragment below will match records generated by Trend Micro Deep Security devices with IDs in a specified range: 
@@ -114,9 +197,9 @@ The subsections below explain how match lists work, and how to leverage them in
 
 #### Match lists are used to enrich record data
 
-This section describes what [match lists](/docs/cse/match-lists-suppressed-lists) are, and how Cloud SIEM uses them to enrich record data. The short story is that when a record is ingested, Cloud SIEM uses match lists to add information to the record. So, your rule doesn’t directly refer to a match list, it checks the record for data that Cloud SIEM may have added to the record at the time of ingestion. 
+This section describes what [match lists](/docs/cse/match-lists-suppressed-lists) are, and how Cloud SIEM uses them to enrich record data. The short story is that when a record is ingested, Cloud SIEM uses match lists to add information to the record. So, your rule doesn’t directly refer to a match List, it checks the record for data that Cloud SIEM may have added to the record at the time of ingestion. 
 
-Match lists are lists of important indicators and identifiers, typically configured by a Cloud SIEM analyst. Match lists are often used to define allowlists of entities, like IP addresses, URLs, and hostnames, and so on, that you want to exempt from ordinary rule processing. For example, you might want to prevent a rule from firing for records that contain one of a certain set of IP addresses. 
+Match lists are lists of important indicators and identifiers, typically configured by a Cloud SIEM analyst. match lists are often used to define allowlists of entities, like IP addresses, URLs, and hostnames, and so on, that you want to exempt from ordinary rule processing. For example, you might want to prevent a rule from firing for records that contain one of a certain set of IP addresses. 
 
 Here’s an example of a match list in the Cloud SIEM UI. 
 
@@ -203,4 +286,4 @@ where 
 
 :::note
 If your `array_contains` statement refers to a threat intelligence source whose name contains embedded spaces, be sure to replace the spaces with underscores.
-::: 
+:::