Skip to content

Commit

Permalink
doc: hyperscan information updated
Browse files Browse the repository at this point in the history
Signed-off-by: jason taylor <[email protected]>
  • Loading branch information
jmtaylor90 authored and victorjulien committed Jul 13, 2023
1 parent c95fce3 commit 62170d2
Showing 1 changed file with 46 additions and 98 deletions.
144 changes: 46 additions & 98 deletions doc/userguide/performance/hyperscan.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,133 +4,81 @@ Hyperscan
Introduction
~~~~~~~~~~~~

"Hyperscan is a high-performance multiple regex matching library." https://www.hyperscan.io
"Hyperscan is a high performance regular expression matching library (...)" (https://www.intel.com/content/www/us/en/developer/articles/technical/introduction-to-hyperscan.html)

In Suricata it can be used to perform multi pattern matching (mpm). Support was implemented by Justin Viiret and Jim Xu from Intel: https://github.com/inliniac/suricata/pull/1965, https://redmine.openinfosecfoundation.org/issues/1704
In Suricata it can be used to perform multi pattern matching (mpm) or single pattern matching (spm).

Compilation
~~~~~~~~~~~
Support for hyperscan in Suricata was initially implemented by Justin Viiret and Jim Xu from Intel via https://github.com/OISF/suricata/pull/1965.

It's possible to pass --with-libhs-includes=/usr/local/include/hs/ --with-libhs-libraries=/usr/local/lib/, although by default this shouldn't be necessary. Suricata should pick up Hyperscan's pkg-config file automagically.
Hyperscan is only for Intel x86 based processor architectures at this time. For ARM processors, vectorscan is a drop in replacement for hyperscan, https://github.com/VectorCamp/vectorscan.

When Suricata's compilation succeeded, you should have:

::


suricata --build-info|grep Hyperscan
Hyperscan support: yes


Using Hyperscan
~~~~~~~~~~~~~~~

To use the hyperscan support edit your suricata.yaml. Change the mpm-algo and spm-algo values to 'hs'.

Alternatively, use this command-line option: --set mpm-algo=hs --set spm-algo=hs

Basic Installation (Package)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Some Linux distributions include hyperscan in their respective package collections.

Fedora 37+/Centos 8+: sudo dnf install hyperscan-devel
Ubuntu/Debian: sudo apt-get install libhyperscan-dev

Ubuntu Hyperscan Installation
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

To use Suricata with Hyperscan support, install dependencies:
Advanced Installation (Source)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Hyperscan has the following dependencies in order to build from
source:

::


apt-get install cmake ragel
* boost development libraries (minimum boost library version is 1.58)
* cmake
* C++ compiler (e.g. gcc-c++)
* libpcap development libraries
* pcre2 development libraries
* python3
* ragel
* sqlite development libraries

libboost headers
----------------

Hyperscan needs the libboost headers from 1.58+.

On Ubuntu 15.10 or 16.04+, simply do:
**Note:** git is an additional dependency if cloning the
hyperscan GitHub repository. Otherwise downloading the
hyperscan zip from the GitHub repository will work too.

The steps to build and install hyperscan are:

::


apt-get install libboost-dev


Trusty
------

Trusty has 1.57, so it's too old. We can grab a newer libboost version, but we *don't* install it system wide. It's only the headers we care about during compilation of Hyperscan.


::


sudo apt-get python-dev libbz2-dev
wget https://dl.bintray.com/boostorg/release/1.66.0/source/boost_1_66_0.tar.gz
tar xvzf boost_1_66_0.tar.gz
cd boost_1_66_0
./bootstrap.sh --prefix=~/tmp/boost-1.66
./b2 install

Hyperscan
---------

We'll install version 5.0.0.


::


git clone https://github.com/intel/hyperscan
cd hyperscan
mkdir build
cd build
cmake -DBUILD_STATIC_AND_SHARED=1 ../

If you have your own libboost headers, use this cmake line instead:
cmake -DBUILD_STATIC_AND_SHARED=1
cmake --build ./
sudo cmake --install ./

::
**Note:** Hyperscan can take a a long time to build/compile.

**Note:** It may be necessary to add /usr/local/lib or
/usr/local/lib64 to the `ld` search path. Typically this is
done by adding a file under /etc/ld.so.conf.d/ with the contents
of the directory location of libhs.so.5 (for hyperscan 5.x).

cmake -DBUILD_STATIC_AND_SHARED=1 -DBOOST_ROOT=~/tmp/boost-1.66 ../

Finally, make and make install:

::


make
sudo make install

Compilation can take a long time, but it should in the end look something like this:

Using Hyperscan
~~~~~~~~~~~~~~~

Confirm that the suricata version installed has hyperscan enabled.
::


Install the project...
-- Install configuration: "RELWITHDEBINFO"
-- Installing: /usr/local/lib/pkgconfig/libhs.pc
-- Up-to-date: /usr/local/include/hs/hs.h
-- Up-to-date: /usr/local/include/hs/hs_common.h
-- Up-to-date: /usr/local/include/hs/hs_compile.h
-- Up-to-date: /usr/local/include/hs/hs_runtime.h
-- Installing: /usr/local/lib/libhs_runtime.a
-- Installing: /usr/local/lib/libhs_runtime.so.4.2.0
-- Installing: /usr/local/lib/libhs_runtime.so.4.2
-- Installing: /usr/local/lib/libhs_runtime.so
-- Installing: /usr/local/lib/libhs.a
-- Installing: /usr/local/lib/libhs.so.4.2.0
-- Installing: /usr/local/lib/libhs.so.4.2
-- Installing: /usr/local/lib/libhs.so
suricata --build-info | grep Hyperscan
Hyperscan support: yes

Note that you may have to add /usr/local/lib to your ld search path

To use hyperscan support, edit the suricata.yaml.
Change the mpm-algo and spm-algo values to 'hs'.

::
Alternatively, use this command-line option: --set mpm-algo=hs --set spm-algo=hs

**Note**: The default suricata.yaml configuration settings for
mpm-algo and spm-algo are "auto". Suricata will use hyperscan
if it is present on the system in case of the "auto" setting.

echo "/usr/local/lib" | sudo tee --append /etc/ld.so.conf.d/usrlocal.conf
sudo ldconfig

If the current suricata installation does not have hyperscan
support, refer to :ref:`installation`

0 comments on commit 62170d2

Please sign in to comment.