next/251/20240119/v1

doc/userguide/capture-hardware/dpdk.rst

@@ -15,6 +15,57 @@ learn more about the basic setup for DPDK.
 The following sections contain examples of how to set up DPDK and Suricata for
 more obscure use-cases.
 
+Hugepage analysis
+-----------------
+
+Suricata can analyse utilized hugepages on the system. This can be particularly 
+beneficial when there's a potential overallocation of hugepages. 
+The hugepage analysis is designed to examine the hugepages in use and 
+provide recommendations on an adequate number of hugepages. This then ensures 
+Suricata operates optimally while leaving sufficient memory for other 
+applications on the system. The analysis works by comparing snapshots of the
+hugepages before and after Suricata is initialized. After the initialization,
+no more hugepages are allocated by Suricata.
+The hugepage analysis can be seen in the Perf log level and is printed out 
+during the Suricata start. It is only printed when Suricata detects some 
+disrepancies in the system related to hugepage allocation.
+
+It's recommended to perform this analysis from a "clean" state - 
+that is a state when all your hugepages are free. It is especially recommended 
+when no other hugepage-dependent applications are running on your system.
+This can be checked in one of two ways:
+
+.. code-block:: 
+
+  # global check
+  cat /proc/meminfo
+
+  HugePages_Total:    1024
+  HugePages_Free:     1024
+
+  # per-numa check depends on NUMA node ID, hugepage size, 
+  # and nr_hugepages/free_hugepages - e.g.:
+  cat /sys/devices/system/node/node0/hugepages/hugepages-2048kB/free_hugepages
+
+After the termination of Suricata and other hugepage-related applications, 
+if the count of free hugepages is not equal with the total number of hugepages, 
+it indicates some hugepages were not freed completely.
+This can be fixed by removing DPDK-related files from the hugepage-mounted 
+directory (filesystem). 
+It's important to exercise caution while removing hugepages, especially when 
+other hugepage-dependent applications are in operation, as this action will 
+disrupt their memory functionality.
+Removing the DPDK files from the hugepage directory can often be done as:
+
+.. code-block:: bash
+
+  sudo rm -rf /dev/hugepages/rtemap_*
+
+  # To check where hugepages are mounted:
+  dpdk-hugepages.py -s
+  # or 
+  mount | grep huge
+
 Bond interface
 --------------
 
@@ -95,3 +146,41 @@ management and worker CPU set.
         - worker-cpu-set:
             cpu: [ 2,4,6,8 ]
     ...
+
+Interrupt (power-saving) mode
+-----------------------------
+
+The DPDK is traditionally recognized for its polling mode operation. 
+In this mode, CPU cores are continuously querying for packets from 
+the Network Interface Card (NIC). While this approach offers benefits like 
+reduced latency and improved performance, it might not be the most efficient 
+in scenarios with sporadic or low traffic. 
+The constant polling can lead to unnecessary CPU consumption. 
+To address this, DPDK offers an `interrupt` mode.
+
+The obvious advantage that interrupt mode brings is power efficiency. 
+So far in our tests, we haven't observed a decrease in performance. Suricata's
+performance has actually seen a slight improvement.
+The (IPS runmode) users should be aware that interrupts can 
+introduce non-deterministic latency. However, the latency should never be 
+higher than in other (e.g. AF_PACKET/AF_XDP/...) capture methods. 
+
+Interrupt mode in DPDK can be configured on a per-interface basis. 
+This allows for a hybrid setup where some workers operate in polling mode, 
+while others utilize the interrupt mode. 
+The configuration for the interrupt mode can be found and modified in the 
+DPDK section of the suricata.yaml file.
+
+Below is a sample configuration that demonstrates how to enable the interrupt mode for a specific interface:
+
+::
+
+  ...
+  dpdk:
+      eal-params:
+        proc-type: primary
+
+      interfaces:
+        - interface: 0000:3b:00.0
+          interrupt-mode: true
+          threads: 4

doc/userguide/configuration/exception-policies.rst

@@ -46,7 +46,7 @@ Auto
 ''''
 
 **In IPS mode**, the default behavior for most of the exception policies is to
-fail close. This means droping the flow, or the packet, when the flow action is
+fail close. This means dropping the flow, or the packet, when the flow action is
 not supported. The default policy for the midstream exception will be ignore if
 midstream flows are accepted.
 
@@ -131,7 +131,7 @@ midstream pick-ups enabled or not and the various exception policy values:
      - Midstream pick-up sessions ENABLED (stream.midstream=true)
      - Midstream pick-up sessions DISABLED (stream.midstream=false)
    * - Ignore
-     - Session tracket and parsed.
+     - Session tracked and parsed, inspect and log app-layer traffic, do detection.
      - Session not tracked. No app-layer inspection or logging. No detection. No stream reassembly.
    * - Drop-flow
      - Not valid.*
@@ -169,7 +169,7 @@ whole flow.
      - Midstream pick-up sessions ENABLED (stream.midstream=true)
      - Midstream pick-up sessions DISABLED (stream.midstream=false)
    * - Ignore
-     - Session tracket and parsed.
+     - Session tracked and parsed, inspect and log app-layer traffic, do detection.
      - Session not tracked. No app-layer inspection or logging. No detection. No stream reassembly.
    * - Drop-flow
      - Not valid.*

doc/userguide/configuration/suricata-yaml.rst

@@ -643,9 +643,9 @@ For setting the option sgh-mpm-context, you can choose from auto, full
 or single. The default setting is 'auto', meaning Suricata selects
 full or single based on the algorithm you use. 'Full' means that every
 group has its own MPM-context, and 'single' that all groups share one
-MPM-context. The two algorithms ac and ac-gfbs are new in 1.03. These
-algorithms use a single MPM-context if the Sgh-MPM-context setting is
-'auto'. The rest of the algorithms use full in that case.
+MPM-context. The algorithm "ac" uses a single MPM-context if the 
+Sgh-MPM-context setting is 'auto'. The rest of the algorithms use full 
+in that case.
 
 The inspection-recursion-limit option has to mitigate that possible
 bugs in Suricata cause big problems. Often Suricata has to deal with
@@ -1287,7 +1287,7 @@ the default behavior).
 
 Each supported protocol has a dedicated subsection under ``protocols``.
 
-Asn1_max_frames (new in 1.0.3 and 1.1)
+Asn1_max_frames
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 Asn1 (`Abstract Syntax One
@@ -1860,14 +1860,15 @@ Default Log Format
 ~~~~~~~~~~~~~~~~~~
 
 A logging line exists of two parts. First it displays meta information
-(thread id, date etc.), and finally the actual log message. Example:
+(Log-level, Suricata module), and finally the actual log message. Example:
 
 ::
 
-  [27708] 15/10/2010 -- 11:40:07 - (suricata.c:425) <Info> (main) – This is Suricata version 1.0.2
+  i: suricata: This is Suricata version 7.0.2 RELEASE running in USER mode
 
-(Here the part until the – is the meta info, "This is Suricata 1.0.2"
-is the actual message.)
+(Here the part until the second `:` is the meta info, 
+"This is Suricata version 7.0.2 RELEASE running in USER mode" is the actual 
+message.)
 
 It is possible to determine which information will be displayed in
 this line and (the manner how it will be displayed) in which format it

doc/userguide/output/custom-http-logging.rst

@@ -1,8 +1,6 @@
 Custom http logging
 ===================
 
-As of Suricata 1.3.1 you can enable a custom http logging option.
-
 In your Suricata.yaml, find the http-log section and edit as follows:
 
 

doc/userguide/rules/fast-pattern-explained.rst

@@ -17,25 +17,23 @@ The fast_pattern selection criteria are as follows:
 
 #. Suricata first identifies all content matches that have the highest
    "priority" that are used in the signature.  The priority is based
-   off of the buffer being matched on and generally 'http_*' buffers
-   have a higher priority (lower number is higher priority).  See
-   :ref:`Appendix B <fast-pattern-explained-appendix-b>` for details
-   on which buffers have what priority.
+   off of the buffer being matched on and generally application layer buffers
+   have a higher priority (lower number is higher priority). The buffer
+   `http_method` is an exception and has lower priority than the general 
+   `content` buffer.
 #. Within the content matches identified in step 1 (the highest
    priority content matches), the longest (in terms of character/byte
    length) content match is used as the fast pattern match.
 #. If multiple content matches have the same highest priority and
    qualify for the longest length, the one with the highest
    character/byte diversity score ("Pattern Strength") is used as the
-   fast pattern match.  See :ref:`Appendix C
-   <fast-pattern-explained-appendix-c>` for details on the algorithm
+   fast pattern match.  See :ref:`Appendix A
+   <fast-pattern-explained-appendix-a>` for details on the algorithm
    used to determine Pattern Strength.
 #. If multiple content matches have the same highest priority, qualify
    for the longest length, and the same highest Pattern Strength, the
    buffer ("list_id") that was *registered last* is used as the fast
-   pattern match.  See :ref:`Appendix B
-   <fast-pattern-explained-appendix-b>` for the registration order of
-   the different buffers/lists.
+   pattern match.
 #. If multiple content matches have the same highest priority, qualify
    for the longest length, the same highest Pattern Strength, and have
    the same list_id (i.e. are looking in the same buffer), then the
@@ -52,63 +50,7 @@ Appendices
 
 .. _fast-pattern-explained-appendix-a:
 
-Appendix A - Buffers, list_id values, and Registration Order for Suricata 1.3.4
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-This should be pretty much the same for Suricata 1.1.x - 1.4.x.
-
-======= ============================== ======================== ==================
-list_id Content Modifier Keyword       Buffer Name              Registration Order
-======= ============================== ======================== ==================
-1       <none> (regular content match) DETECT_SM_LIST_PMATCH    1 (first)
-2       http_uri                       DETECT_SM_LIST_UMATCH    2
-6       http_client_body               DETECT_SM_LIST_HCBDMATCH 3
-7       http_server_body               DETECT_SM_LIST_HSBDMATCH 4
-8       http_header                    DETECT_SM_LIST_HHDMATCH  5
-9       http_raw_header                DETECT_SM_LIST_HRHDMATCH 6
-10      http_method                    DETECT_SM_LIST_HMDMATCH  7
-11      http_cookie                    DETECT_SM_LIST_HCDMATCH  8
-12      http_raw_uri                   DETECT_SM_LIST_HRUDMATCH 9
-13      http_stat_msg                  DETECT_SM_LIST_HSMDMATCH 10
-14      http_stat_code                 DETECT_SM_LIST_HSCDMATCH 11
-15      http_user_agent                DETECT_SM_LIST_HUADMATCH 12 (last)
-======= ============================== ======================== ==================
-
-Note: registration order doesn't matter when it comes to determining the fast pattern match for Suricata 1.3.4 but list_id value does.
-
-.. _fast-pattern-explained-appendix-b:
-
-Appendix B - Buffers, list_id values, Priorities, and Registration Order for Suricata 2.0.7
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-This should be pretty much the same for Suricata 2.0.x.
-
-========================================== ================== ============================== ============================= =======
-Priority (lower number is higher priority) Registration Order Content Modifier Keyword       Buffer Name                   list_id
-========================================== ================== ============================== ============================= =======
-3                                          11                 <none> (regular content match) DETECT_SM_LIST_PMATCH         1
-3                                          12                 http_method                    DETECT_SM_LIST_HMDMATCH       12
-3                                          13                 http_stat_code                 DETECT_SM_LIST_HSCDMATCH      9
-3                                          14                 http_stat_msg                  DETECT_SM_LIST_HSMDMATCH      8
-2                                          1 (first)          http_client_body               DETECT_SM_LIST_HCBDMATCH      4
-2                                          2                  http_server_body               DETECT_SM_LIST_HSBDMATCH      5
-2                                          3                  http_header                    DETECT_SM_LIST_HHDMATCH       6
-2                                          4                  http_raw_header                DETECT_SM_LIST_HRHDMATCH      7
-2                                          5                  http_uri                       DETECT_SM_LIST_UMATCH         2
-2                                          6                  http_raw_uri                   DETECT_SM_LIST_HRUDMATCH      3
-2                                          7                  http_host                      DETECT_SM_LIST_HHHDMATCH      10
-2                                          8                  http_raw_host                  DETECT_SM_LIST_HRHHDMATCH     11
-2                                          9                  http_cookie                    DETECT_SM_LIST_HCDMATCH       13
-2                                          10                 http_user_agent                DETECT_SM_LIST_HUADMATCH      14
-2                                          15 (last)          dns_query                      DETECT_SM_LIST_DNSQUERY_MATCH 20
-========================================== ================== ============================== ============================= =======
-
-Note: list_id value doesn't matter when it comes to determining the
-fast pattern match for Suricata 2.0.7 but registration order does.
-
-.. _fast-pattern-explained-appendix-c:
-
-Appendix C - Pattern Strength Algorithm
+Appendix A - Pattern Strength Algorithm
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 From detect-engine-mpm.c. Basically the Pattern Strength "score"

doc/userguide/rules/smb-keywords.rst

@@ -61,6 +61,57 @@ Examples::
 
 ``smb.ntlmssp_domain`` can be used as ``fast_pattern``.
 
+
+smb.version
+------------
+
+Keyword to match on the SMB version seen in an SMB transaction.
+
+Signature Example:
+
+.. container:: example-rule
+
+  alert smb $HOME_NET any -> any any (msg:"SMBv1 version rule"; \
+  :example-rule-options:`smb.version:1;` sid:1;)
+
+  alert smb $HOME_NET any -> any any (msg:"SMBv2 version rule"; \
+  :example-rule-options:`smb.version:2;` sid:2;)
+
+
+Matching in transition from SMBv1 to SMBv2
+******************************************
+
+In the initial protocol negotiation request, a client supporting SMBv1 and SMBv2 can send an initial SMBv1 request and receive a SMBv2 response from server, indicating that SMBv2 will be used.
+
+This first SMBv2 response made by the server will match as SMBv1, since the entire transaction will be considered a SMBv1 transaction.
+
+Will ``smb.version`` match SMBv3 traffic?
+*****************************************
+
+Yes, it will match SMBv3 messages using `smb.version:2;`, which will match SMBv2 and SMBv3, since they use the same version identifier in the SMB header.
+
+This keyword will use the Protocol ID specified in SMB header to determine the version. Here is a summary of the Protocol ID codes:
+
+- 0xffSMB is SMBv1 `header <https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-smb/3c0848a6-efe9-47c2-b57a-f7e8217150b9>`_
+- 0xfeSMB is SMBv2 `normal header <https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-smb2/5cd64522-60b3-4f3e-a157-fe66f1228052>`_ (can be `sync <https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-smb2/fb188936-5050-48d3-b350-dc43059638a4>`_ or `async <https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-smb2/ea4560b7-90da-4803-82b5-344754b92a79>`_)
+- 0xfdSMB is SMBv2 `transform header <https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-smb2/d6ce2327-a4c9-4793-be66-7b5bad2175fa>`_. This is only valid for the SMB 3.x dialect family.
+- 0xfcSMB is SMBv2 `transform compression header <https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-smb2/d6ce2327-a4c9-4793-be66-7b5bad2175fa>`_ (can be `chained <https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-smb2/aa880fe8-ebed-4409-a474-ec6e0ca0dbcb>`_ or `unchained <https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-smb2/793db6bb-25b4-4469-be49-a8d7045ba3a6>`_). These ones require the use of the 3.1.1 dialect.
+
+The Protocol ID in the header distinguishes only SMBv1 and SMBv2 since they are completely different protocols with entirely
+different message formats, types and implementations.
+
+On the other hand, SMBv3 is more like an extension of SMBv2. When using SMBv2 we can select one of the following dialects for the conversation between client and server:
+
+- 2.0.2
+- 2.1
+- 3.0
+- 3.0.2
+- 3.1.1
+
+We say we are using SMBv3 when we select a 3.x dialect for the conversation, so you can use SMB 3.0, SMB 3.0.2 or SMB 3.1.1. The higher you choose, the more capabilities you have, but the message syntax and message command number remains the same.
+
+SMB version and dialect are separate components. In the case of SMBv3 for instance, the SMB version will be 2 but the dialect will be 3.x. Dialect specification is not available currently via keyword.
+
 file.name
 ---------
 
@@ -74,4 +125,4 @@ Signature Example:
   :example-rule-options:`file.name; content:"file.txt";` \
   classtype:bad-unknown; sid:1; rev:1;)
 
-For additional information on the ``file.name`` keyword, see :doc:`file-keywords`.
+For additional information on the ``file.name`` keyword, see :doc:`file-keywords`.