diff --git a/README.md b/README.md index 136c1c4..1ee0de1 100644 --- a/README.md +++ b/README.md @@ -29,6 +29,7 @@ A typical use case would be universities deploying a pDNSSOC client on their DNS * [:beetle: Issue tracker](../../issues) * [:loudspeaker: Community discussions](../../discussions) * [:question: Frequently asked questions](./FAQ.md) +* [:bar_chart: Presentations](./docs/presentations.md) ## Acknowledgments pDNSSOC would not exist without: diff --git a/docs/correlation_engine.md b/docs/correlation_engine.md deleted file mode 100644 index e9ead10..0000000 --- a/docs/correlation_engine.md +++ /dev/null @@ -1,97 +0,0 @@ -# Correlation Engine - -:warning: :construction: WORK IN PROGRESS :construction: :warning: - -## Contents - - - - - -- [Correlation Engine](#correlation-engine) - - [Contents](#contents) - - [Building blocks](#building-blocks) - - [`MISP` Caching implementation](#misp-caching-implementation) - - [Storage solution](#storage-solution) - - [Opt-in data pDNS data sharing](#opt-in-data-pdns-data-sharing) - - [Areas of work - `WIP`](#areas-of-work-wip) - - [Timeline - `WIP`](#timeline-wip) - - - - -## Building blocks - -```mermaid -flowchart LR; - - subgraph correlation [Correlation engine] - Ingest((Ingest)) - Storage(Storage) - Memcached(Memcached) - Correlate((Correlate)) - end - - subgraph participants [Participating Institutions] - pDNSSensor1(pDNS sensor) - pDNSSensor2(pDNS sensor) - pDNSSensor3(pDNS sensor) - end - - subgraph Intelligence - MISP(MISP) - end - - subgraph Alerting - AlertStorage(Alert Storage) - Alert((Alert)) - end - - subgraph pDNS based projects - pDNSBasedProjects(Other projects,\npDNS databases) - end - -pDNSSensor1(pDNS sensor)-->Ingest((Ingest)); -pDNSSensor2(pDNS sensor)-->Ingest((Ingest)); -pDNSSensor3(pDNS sensor)-->Ingest((Ingest)); - -MISP(MISP) --> Memcached(Memcached) - - -Ingest((Ingest)) --> Storage(Storage) -Storage(Storage) --> Correlate((Correlate)) -Memcached(Memcached) --> Correlate((Correlate)) -Correlate((Correlate)) --> AlertStorage(Alert Storage) -AlertStorage(Alert Storage) --> Alert((Alert)) - - -Alert((Alert)) --> participants - -Ingest((Ingest)) --> |Opt-in pDNS\ndata sharing| pDNSBasedProjects -``` - -### `MISP` Caching implementation - -Attributes are fetched from one or multiple MISP instances and stored in a KV store solution (`Memcached` at the moment) so as to be ready for correlation without putting pressure on the MISP instances - -### Storage solution - -DNS data is ingested from passive DNS sensors. This is where the relation between originating DNS recursive client and participating institution entity is stored so that alerting is possible. - -### Opt-in data pDNS data sharing - -Participating institutions will be given the choice to opt-in for pDNS data sharing with other projects that are based on pDNS data analysis. - -## Areas of work - `WIP` - -- [ ] Storage selection -- [ ] Define whether correlation will also be done for past attributes/DNS logs -- [ ] Define time window for valid MISP attributes -- [ ] Define types of MISP attributes -- [ ] Define different types of DNS logs (DNS Ttraffic above the recursive, passive DNS aggregated logs) -- [ ] Define data flow for pDNS data and Incident response related data - - -## Timeline - `WIP` - -TBD diff --git a/docs/log_collection.md b/docs/log_collection.md deleted file mode 100644 index ce770a9..0000000 --- a/docs/log_collection.md +++ /dev/null @@ -1,70 +0,0 @@ -# Log collection - -We are currently investigating available open source pDNS sensor and DNS probe solutions to integrate with pDNSSOC. There is a variety of implementations, among them: - -* [passivedns](https://github.com/gamelinux/passivedns/) -* [DNSMonster](https://github.com/mosajjal/dnsmonster) -* [gopassivedns](https://github.com/Phillipmartin/gopassivedns) - :warning: Unmaintained :warning: -* [Farsight SIE DNS Sensor](https://github.com/farsightsec/sie-dns-sensor/) -* [go-dns-collector](https://github.com/dmachard/go-dns-collector/) -* DNS data from Bro/Zeek -* [DNSTAP](https://dnstap.info/) - -## Sensor deployment - -The sensor is installed and operated by the client institute at the resolving servers, collecting only server-to-server traffic between recursive resolvers and authoritative servers: -* No link between people and traffic can be established by [Correlation Engine](./docs/correlation_engine.md). -* Only non-cached traffic is probed by the sensor, significantly reducing the data sent to the [Correlation Engine](./docs/correlation_engine.md). - -### Approaches -The sensor can be deployed: -* Directly on the DNS Server host -* On a network monitoring interface - -```mermaid -flowchart LR; - - dnsclients(DNS clients) - - subgraph dnsservers [DNS Servers] - DNS_Servers(DNS) - pDNS_int(a. Direct deployment\non DNS Server) - DNS_Servers --LOGS--> pDNS_int - end - - subgraph upstreamdnsservers [Upstream DNS] - Upstream_DNS(DNS) - end - - - dnsclients -->dnsservers - - dnsservers --> TAP/SPAN2((TAP/SPAN)) - TAP/SPAN2((tap/span)) ----> upstreamdnsservers - - dnsservers --> upstreamdnsservers - pDNS_ext[b. Sensor on\nnetwork monitoring\ninterface] --LOGS---> TAP/SPAN2 - - style pDNS_int stroke:#333,stroke-width:4px - style pDNS_ext stroke:#333,stroke-width:4px -``` - -## Collector implementations - -### DNSTAP -https://dnstap.info/ - -* Supported by most of the well-known DNS server implementations -* On some cases (e.g. BIND), it requires rebuilding with specific feature flags -* Offers filtering per message type and streaming using a well defined [Google Protobuf schema](https://github.com/dnstap/dnstap.pb/blob/master/dnstap.proto). -* There are variable tools that can act as DNSTAP Protobuf clients, converting Protobuf messages to the desired format. - -### sie-dns-sensor -https://www.farsightsecurity.com/technical/passive-dns/passive-dns-sensor/ - -* Capturing raw packets from a network interface (can be deployed either on the same host as the DNS server or on a network monitoring interface) -* Widely used and battle tested -* Implementation agnostic -* Packages for Debian, RHEL and FREEBSD -* Protobuf based messages -* Work in progress to have an nmsg to JSON implementation diff --git a/docs/log_format.md b/docs/log_format.md deleted file mode 100644 index bb2062c..0000000 --- a/docs/log_format.md +++ /dev/null @@ -1,5 +0,0 @@ -# Common Log Format - -Supporting multiple DNS log collection solutions requires using a standard and common log format for traced DNS query-response pairs. - -> WIP \ No newline at end of file diff --git a/docs/presentations.md b/docs/presentations.md new file mode 100644 index 0000000..c5e6815 --- /dev/null +++ b/docs/presentations.md @@ -0,0 +1,28 @@ +# Presentations + +## 2023 + +* pDNSSOC: Correlating DNS logs with threat intel from MISP as a poor man’s SOC. + * Christos Arvanitis, Pau Cutrina Vilalta, Romain Wartel + * 2023 NSF Cybersecurity Summit - Lawrence Berkeley National Laboratory|Berkeley, CA + * [Slides](./presentations/2023/pDNSSOC_NSF_2023.pdf) + +## 2022 + +* Plans for a pDNS based SOC deployment + * Christos Arvanitis, Romain Wartel + * International Symposium on Grids & Clouds 2022 + * [Slides](./presentations/2022/Plans%20for%20a%20pDNS%20based%20SOC%20deployment%20-%20ISGC%202022.pdf) + +## 2021 + +* WLCG Operational Security - pDNSSOC Plans + * Christos Arvanitis, Romain Wartel + * [GDB 2021](https://indico.cern.ch/event/876796/#15-pdns-soc-plans) + * [Slides](./presentations/2021/WLCG-SEC-OPS-COMBINED-GDB-DEC-2021_v4.pdf) + +* DNS-based SOC infrastructure for WLCG + * Romain Wartel + * HEPiX Autumn 2021 + * [Slides](./presentations/2021/DNS_SOC.pdf) + diff --git a/docs/presentations/DNS_SOC.pdf b/docs/presentations/2021/DNS_SOC.pdf similarity index 100% rename from docs/presentations/DNS_SOC.pdf rename to docs/presentations/2021/DNS_SOC.pdf diff --git a/docs/presentations/WLCG-SEC-OPS-COMBINED-GDB-DEC-2021_v4.pdf b/docs/presentations/2021/WLCG-SEC-OPS-COMBINED-GDB-DEC-2021_v4.pdf similarity index 100% rename from docs/presentations/WLCG-SEC-OPS-COMBINED-GDB-DEC-2021_v4.pdf rename to docs/presentations/2021/WLCG-SEC-OPS-COMBINED-GDB-DEC-2021_v4.pdf diff --git a/docs/presentations/Plans for a pDNS based SOC deployment - ISGC 2022.pdf b/docs/presentations/2022/Plans for a pDNS based SOC deployment - ISGC 2022.pdf similarity index 100% rename from docs/presentations/Plans for a pDNS based SOC deployment - ISGC 2022.pdf rename to docs/presentations/2022/Plans for a pDNS based SOC deployment - ISGC 2022.pdf diff --git a/docs/presentations/2023/pDNSSOC_NSF_2023.pdf b/docs/presentations/2023/pDNSSOC_NSF_2023.pdf new file mode 100644 index 0000000..8d1aa31 Binary files /dev/null and b/docs/presentations/2023/pDNSSOC_NSF_2023.pdf differ