Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #295 from cisagov/v24.01.0_merge_cisagov
Malcolm v24.01.0 contains new features, improvements, bug fixes and component version updates. v23.12.1...v24.0.1 * Features and enhancements + new Malcolm instance landing page (idaholab#252) + file carve download with password-protected .zip file (idaholab#288) + new "all files exept common plain text files" option for Malcolm's file carving to match Hedgehog capability (idaholab#290) + allow customizing indexes for logs written to OpenSearch/Elasticsearch (idaholab#313) + more consistently differentiate between uploaded and live-captured traffic (idaholab#321) + make download extracted file context item from Arkime smarter (idaholab#330) + improve netbox device type library import by using "official" import script (idaholab#384) * Component version updates + Alpine Linux to [v3.19](https://alpinelinux.org/posts/Alpine-3.19.0-released.html) as the base for some Docker images + Fluent Bit to [v2.2.2](https://github.com/fluent/fluent-bit/releases/tag/v2.2.2) + Beats to [v8.11.4](https://www.elastic.co/guide/en/beats/libbeat/8.11/release-notes-8.11.4.html) + LogStash to [v8.11.4](https://www.elastic.co/guide/en/logstash/current/logstash-8-11-4.html) * Bug fixes + Suricata Alerts dashboard "Alerts - Tags" visualization is useless (idaholab#314) + third party logs are not parsed correctly from fluentbit -> fluentd aggregator -> Malcolm (idaholab#318) + update document lookup APIs to search either network or host data (idaholab#322) + suricata rule update is broken (idaholab#323) + time sync from hedgehog to Malcolm opensearch instance not working (idaholab#324) + fix issue specifying database mode via command-line + have pruning of OpenSearch indices (based on size) include "other" Malcolm indices as well (e.g., nginx logs, system resources, third-party logs, etc.) * Configuration changes (in [environment variables](https://idaholab.github.io/Malcolm/docs/malcolm-config.html#MalcolmConfigEnvVars) in [`./config/`](https://github.com/cisagov/Malcolm/tree/v24.0.1/config)) + added the following variables with relation to idaholab#313 - added `ARKIME_ROTATE_INDEX` to [`arkime.env`](https://github.com/cisagov/Malcolm/tree/v24.0.1/arkime.env.example) with default value of `daily` (see [Arkime docs on rotateIndex](https://arkime.com/settings#rotateIndex)) - added the following variables and defaults to [`opensearch.env`](https://github.com/cisagov/Malcolm/tree/v24.0.1/opensearch.env.example): ``` # OpenSearch index patterns and timestamp fields # Index pattern for network traffic logs written via Logstash (e.g., Zeek logs, Suricata alerts) MALCOLM_NETWORK_INDEX_PATTERN=arkime_sessions3-* # Default time field to use for network traffic logs in Logstash and Dashboards MALCOLM_NETWORK_INDEX_TIME_FIELD=firstPacket # Suffix used to create index to which network traffic logs are written (supports Ruby strftime strings in %{}) MALCOLM_NETWORK_INDEX_SUFFIX=%{%y%m%d} # Index pattern for other logs written via Logstash (e.g., nginx, beats, fluent-bit, etc.) MALCOLM_OTHER_INDEX_PATTERN=malcolm_beats_* # Default time field to use for other logs in Logstash and Dashboards MALCOLM_OTHER_INDEX_TIME_FIELD=@timestamp # Suffix used to create index to which other logs are written (supports Ruby strftime strings in %{}) MALCOLM_OTHER_INDEX_SUFFIX=%{%y%m%d} # Index pattern used specifically by Arkime (will probably match MALCOLM_NETWORK_INDEX_PATTERN, should probably be arkime_sessions3-*) ARKIME_NETWORK_INDEX_PATTERN=arkime_sessions3-* # Default time field used by for sessions in Arkime viewer ARKIME_NETWORK_INDEX_TIME_FIELD=firstPacket ``` + changed default for `EXTRACTED_FILE_HTTP_SERVER_KEY` to `infected` in [`zeek-secret.env`](https://github.com/cisagov/Malcolm/tree/v24.0.1/zeek-secret.env.example) + added `EXTRACTED_FILE_HTTP_SERVER_ZIP` with default value of `false` in [`zeek.env`](https://github.com/cisagov/Malcolm/tree/v24.0.1/zeek.env.example), see (idaholab#288)
- Loading branch information