Malcolm v3.0.0
Malcolm v3.0.0 is a major release with some big replacements in the project's underpinnings, including a few backwards compatibility-breaking changes. Please continue reading for more details.
List of changes in Malcolm v3.0.0:
- Change base for Elasticsearch and Kibana Docker images (version 7.6.2) from Elastic.co to Open Distro for Elastic (based on Elastic 7.10.0); see idaholab#15. This is a major change which breaks backwards compatibility for several features (listed below). If you are using these features, you will need to back up the data and/or configuration associated with them and migrate them manually to the new tools. No automatic migration or upgrade of these features is performed. It's recommended that you re-run
install.py --configure(see System configuration and tuning) prior to running Malcolm v3.0.0.- Kibana comments replaced with Notebooks
- Kibana elastalert plugin replaced with Alerting plugin
- Elasticsearch curator replaced with Index Management plugin
- The third-party Sankey visualization plugin has been temporarily removed due to compatibility issues, although it is planned to be reintegrated in a Malcolm point release in the near future (see uniberg/kbn_sankey_vis#15)
- The third-party Kibana drill-down plugin providing Kibana-to-Moloch pivoting has been temporarily removed due to compatibility issues, although it is planned to be reintegrated in a Malcolm point release in the near future (see goodlabs-studio/kibana-plugin-drilldownmenu#5)
- In addition to those replacements, the Real Time Anomaly Detection feature is now available:
- Real Time Anomaly Detection in Open Distro for Elasticsearch blog announcement
- Anomaly Detection documentation and source code for Elasticsearch and Kibana components
- Random Cut Forests writeup
- If you are not up-to-date on the recent developments in Elasticsearch's licensing, here are a few of the official statements from the various parties involved:
- Elastic.co's original announcement, clarification, Elastic License v2 announcement, "Why we had to change" post and FAQ on 2021 License Change
- Open Distro for Elasticsearch initial response post, Amazon AWS Open Source Blog post and fork updates post
- Malcolm startup time (especially the Logstash container) has been reduced drastically
- Improvements to Malcolm's prebuilt Kibana dashboards
- Improvements to build scripts
- Minor tweaks and bugfixes for ISO-installed environments for Malcolm and Hedgehog Linux
- Minor other bug fixes and performance improvements
- Version bump
- Yara v4.0.5
Malcolm and Hedgehog Linux may be obtained by pulling or building the Docker images and/or building the ISO installer images as described in the documentation. Unofficial ISO installer images for Malcolm and Hedgehog Linux are not hosted on GitHub, but may be downloaded from https://malcolm.fyi/download/.