Malcolm v2.4.1

Malcolm v2.4.1 contains the following changes:

v2.4.0...v2.4.1

• Zeek

  • added plugin to detect "bad neighbor" (CVE-2020-16898)

• Version bumps

  • supercronic (for Docker images) 0.1.11
  • nginx 1.19.3
  • bison (for Zeek compile) 3.7.2
  • cmake (for Zeek compile) 3.18.4
  • Zeek 3.0.11
  • Moloch 2.4.1
  • Linux Kernel (for ISOs) 5.8.0

Malcolm and Hedgehog Linux may be obtained by pulling or building the Docker images and/or building the ISO installer images as described in the documentation. Unofficial ISO installer images for Malcolm and Hedgehog Linux are not hosted on GitHub, but may be downloaded from https://malcolm.fyi/download/.

Malcolm v2.4.0.1

Malcolm v2.4.0.1 is a repack of the Malcolm v2.4.0 release with one minor fix for the ISO installers for Malcolm and Hedgehog Linux to fix idaholab#27. The rest of the code is identical. If you are deploying Malcolm with Docker rather than the ISO-installed version, you can ignore this release.

Malcolm v2.4.0

Malcolm v2.4.0 contains the following new features, improvements and bug fixes:

• Extracted file scanning
  • added Capa as an optional extracted file scanner
  • improvements to the way file scanners work when more than one are enabled
• Version updates
  • updated Moloch to 2.4.1
  • updated Zeek to 3.0.10
  • updated Linux Kernel for ISO installers to 5.7
• Zeek plugins
  • added Corelight's Zerologon plugin to detect CVE-2020-1472
• Tweaks and bug fixes
  • Don't allow docker to mess with firewall rules in Malcolm ISO
  • Fix idaholab#26, ISO installers result in blank screen when booting with BIOS
  • Fix idaholab#24, install.py won't prompt to change ownership of extracted directory correctly if run as root
  • Leave some development packages in place in Hedgehog ISO so that Spicy plugins can be compiled

v2.3.0...v2.4.0

Malcolm and Hedgehog Linux may be obtained by pulling or building the Docker images and/or building the ISO installer images as described in the documentation. Unofficial ISO installer images for Malcolm and Hedgehog Linux are not hosted on GitHub, but may be downloaded from https://malcolm.fyi/download/.

Malcolm v2.3.0

Malcolm v2.3.0 contains the following new features, improvements and bug fixes:

• Carved file scanning improvements

  • Multiple file scanners can now be enabled concurrently (previously only one at a time was allowed)
  • Yara added as carved file scanner feeding signatures.log with Florian Roth's Signature-Base Yara ruleset enabled by default and the ability to provide other yara signatures under yara/rules under the Malcolm directory (see #148 and #14)

• Bumped versions

  • Moloch v2.4.0

• Bug fixes

  • #150 docker-compose having issues with start and logs under macOS
  • Hedgehog was missing new environment variables for finer control of Zeek local policy behavior
  • miscellaneous tweaks to Docker and ISO images (mainly for file size)

idaholab/Malcolm@v2.2.1...v2.3.0

Malcolm and Hedgehog Linux may be obtained by pulling or building the Docker images and/or building the ISO installer images as described in the documentation. Unofficial ISO installer images for Malcolm and Hedgehog Linux are not hosted on Github, but may be downloaded from https://malcolm.fyi/download/.

Malcolm v2.2.1

Malcolm v2.2.1 is a very minor bugfix release, fixing the DNP3 dashboard in Kibana (see issue #146).

idaholab/Malcolm@v2.2.0...v2.2.1

Malcolm and Hedgehog Linux may be obtained by pulling or building the Docker images and/or building the ISO installer images as described in the documentation. Unofficial ISO installer images for Malcolm and Hedgehog Linux are not hosted on Github, but may be downloaded from https://malcolm.fyi/download/.

Malcolm v2.2.0

Malcolm v2.2.0 is a minor feature release.

• Zeek:
  • Update Zeek to 3.0.8
  • Include Spicy
  • Added ability to disable certain zeek features/parsers using environment variables
  • Added Wireguard parser
  • Added a few Corelight plugins:
    • Corelight's callstranger-detector plugin
    • Corelight's ripple20 plugin
    • Corelight's SIGred plugin
  • Logstash:
    • Added parsing for Zeek Wireguard (noise.log)
    • Initial work towards mapping Zeek log fields to Elastic Common Schema (see issue #79)
      • Disabled by default, can be enabled with LOGSTASH_TO_ECS : 'true' in x-logstash-variables in docker-compose.yml
      • not 100% complete. Good first effort, more will be done in the future
    • Some fixes to the JA3 signature mapping generation
  • ISOs
    • Updated Hedgehog and Malcolm ISOs to use 5.6 kernel
    • Get virtualbox guest VM debs from unofficial backport rather than building for VM installs
  • Documentation
    • Documentation, scripts, Vagrantfiles and sample configurations for using Beats to forward host logs to Malcolm

idaholab/Malcolm@v2.1.1...v2.2.0

Malcolm and Hedgehog Linux may be obtained by pulling or building the Docker images and/or building the ISO installer images as described in the documentation. Unofficial ISO installer images for Malcolm and Hedgehog Linux are not hosted on Github, but may be downloaded from https://malcolm.fyi/download/.

Malcolm v2.1.1

Malcolm v2.1.1 contains bug fixes and a component version update.

idaholab/Malcolm@v2.1.0...v2.1.1

• Bug Fixes

  • Fixed issue #137 (Many permission issues when run as uid:gid other than 1000:1000)

• Version updates

  • Moloch 2.3.2

Malcolm and Hedgehog Linux may be obtained by pulling or building the Docker images and/or building the ISO installer images as described in the documentation. Unofficial ISO installer images for Malcolm and Hedgehog Linux are not hosted on Github, but may be downloaded from https://malcolm.fyi/download/.

Malcolm v2.1.0

Malcolm v2.1.0 contains new features and bug fixes.

idaholab/Malcolm@v2.0.5...v2.1.0

• Incorporated some new Zeek scripts:

  • Cybera's Sniffpass plugin for detecting cleartext passwords in HTTP POST requests
  • Andrew Klaus's zeek-httpattacks plugin for detecting noncompliant HTTP requests
  • Johanna Amann's CVE-2020-0601 ECC certificate validation plugin

• Kibana

  • new "actions and results" dashboard
  • sankey diagram
  • network visualization
  • general improvements and cleanup
  • drilldown both directions between Kibana <-> Moloch (issue #133)
  • many more links to external URLs for RFCs, port numbers, IANA, etc.
  • load all known field mappings at startup

• Parsing/enrichment

  • added support for telnet/rsh/rlogin in Zeek and Logstash
  • better normalization of "zeek.action" field for many protocols (esp. SNMP and DNP3)
  • new normalization of success/result/status/error into "zeek.result"

• NGINX

  • fix how NGINX works with CA certificates for trusted LDAP servers (issue #130 and #128)
  • fix #126 (ldap authentication no longer handles group membership correctly)

• Misc bug fixes and improvements

  • fix #122 by installing bro-xor-exe-plugin correctly
  • fix #134 (wait until Elasticsearch has log data before starting ElastAlert)
  • improved the way Dockerfiles add external links
  • documentation improvements

• Version updates

  • Zeek 3.0.7
  • Moloch 2.3.0

Malcolm and Hedgehog Linux may be obtained by pulling or building the Docker images and/or building the ISO installer images as described in the documentation. Unofficial ISO installer images for Malcolm and Hedgehog Linux are not hosted on Github, but may be downloaded from https://malcolm.fyi/download/.

Malcolm v2.0.6

This release updates the version of the LLVM toolchain (used to build some Malcolm components from source code) from version 9 to version 10. As these are internal changes, Malcolm v2.0.6 is not functionally different from v2.0.5.

idaholab/Malcolm@v2.0.5...v2.0.6

• Update LLVM toolchain from version 9.x to version 10.x
• Minor tweak in malcolm_appliance_packager.sh
• Minor documentation updates

Malcolm and Hedgehog Linux may be obtained by pulling or building the Docker images and/or building the ISO installer images as described in the documentation. Unofficial ISO installer images for Malcolm and Hedgehog Linux are not hosted on Github, but may be downloaded from https://malcolm.fyi/download/.

Malcolm v2.0.5

This release includes the following minor fixes and improvements:

idaholab/Malcolm@v2.0.4...v2.0.5

• bump Curator from 5.7.6 to 5.8.1
• build Docker images in a way that should result in smaller images by downloading artifacts inside the build process of the container with RUN rather than with ADD or with git clone
• bump Moloch from 2.2.3 to 2.3.0
• bump nginx (main nginx-proxy container) from 1.17.9 to 1.19.0
• bump Zeek from 3.0.5 to 3.0.6
• build Zeek with clang/llvm (instead of gcc)
• build Spicy plugin with Zeek
• added zeek-sniffpass Zeek plugin
• added zeek-httpattacks Zeek plugin
• documentation fixes
• bump indices.query.bool.max_clause_count to 2048 for elasticsearch
• fix #134, wait until Elasticsearch has log data before starting ElastAlert
• fix some Kibana dashboards' "Notice" visualizations to include zeek_notice.msg
• fix some Kibana dashboards where a timezone was hard-coded in the dashboard JSON
• remove _dateparsefailure tag in finalization of Logstash enrichment filters
• merge in fixes from development branch dealing with logs from corelight/bro-xor-exe-plugin to make files.log entries searchable and notice.log entries more meaningful
• populate zeek.action from SNMP logs where possible
• various fixes/tweaks to WISE data source for Moloch
• reduce debug log verbosity when being fed by a Hedgehog
• minor tweaks to setting up template file for LDAP login information
• bump netsniff-ng from 0.6.6 to 0.6.7 in Hedgehog
• remove recommendation to install haveged, include random.trust_cpu=on CPU flag in ISO kernel boot parameters
• handle dhcp.log client_software and server_software fields
• preprocessing of Zeek log files prior to sending them to filebeat was affected: ordered-set broke compatibility with python 2.7 either with this commit (rspeer/ordered-set@a412f22) or earlier; rather than using the latest release, use 3.1.1 which is the last one that worked; see also rspeer/ordered-set#59
• cut verbosity of stuff from hedgehog (POST) requests

Malcolm and Hedgehog Linux may be obtained by pulling or building the Docker images and/or building the ISO installer images as described in the documentation. Unofficial ISO installer images for Malcolm and Hedgehog Linux are not hosted on Github, but may be downloaded from https://malcolm.fyi/download/.