Get stable up to date with dev (#23)

* drone * changing name of an attribute in flowresult to signify the last hop / destination node * Feature/general fixes and mods (#20) * just a few tidy ups and a comment * changing the name of an attribute to better suit * changing the name of an attribute to better suit * tweaks Co-authored-by: Jon Clayton <[email protected]> * Feature/ip owners check (#21) * adding to access checker * testing icmp specifically * walked the results of policy checker * now able to display only the permitted hits * all results for all nodes * built results dict * structuring the results into a new object * building access result * now showing all permits * merging permit and deny lists * pulling data from tracetreelist and putting into accept object * adding fields * meaningful data now populating the acceptresult object * adding black formatting and optional type for annotation * tidy ups and annotations * re-engineering parts and adding more functionality / bare bones that wil allow jimi to execute the access checks * Major overhall and squashed a load of commits via rebase interactive renaming snapshot dir renaming snapshot dir setting folder sane defaults in one place fixes fixes fixes fixes fixes fixes various fixes to testing various fixes to testing various fixes to testing various fixes to testing various fixes to testing various fixes to testing various fixes to testing changes fixes fixes fixes fixes testing seems to be working now other than json serializable error seems to be working now other than json serializable error seems to be working now other than json serializable error major deletions and some comments major overhaul, renaming, making more readable, more comments adding drone ci file drone drone drone update drone update adding kube deployment restart * changing remoteConnectBatfish to batfishConnect * changing remoteConnectBatfish to batfishConnect * upgrading plugin version * upgrading plugin version * upgrading plugin version * drone * adding dstPorts as a header constraint * major changes, checks on text inputs, also adding ipProtocols and dstPorts * adding in Route Check to actions to build model adding in eachability check fix typo in reachability check * fixing things by removing old batfishops references that caused exceptions2 * fixing route check via debug * fixing reachability check * major works on deny results * sort reachability data structures and create a new class * WIP: testing simplyfying the returned results and putting into custom class/struct * testing creating new tr class/struct, seems to be working as it should now * more refactoring of reachability checker * a few tweaks to variables/args * work on route checker * latest route tweaks * happy with the traceroute checker in its current form. Its a bit bare bones and you have to ghand feed it the correct data but is working. * some moving around of file, refractoring, simplification and fixing a few issues with the returned data / dicts. Still need to fix the stack overflow that is happening when accessing the returned data. * change version to force re-register models * change version to force re-register models * change version to force re-register models * change version to force re-register models * tidy ups, better commenting, deletions, polishing, fixes, refactoring * moving files around * fixed problems with nodespec being a list but needing to be a string * lots of comments, deletions, tidy-ups and tweaks * remove unneeded brackets * more changes, try/catch, better comments, better naming of functions, adding dstPorts to the returned data * minor tweaks * some class name tweaks to follow best practice * some class name tweaks to follow best practice * docstrings, type hints and some fixes that flake8 reported * refactored and fixed typed hints, but discovered a possible bug with passing in multiple ports which should match more than one accept rule * more type hints * moved some very generic helpers out of a class into helper class with staticmethods, better checking and exceptino handling, more if/elif and logics to check data before starting, added a pre-flight check function * latest tests to find out why single port getting hit in results * adding dataclasses * removed an old helper that was causing an issue and isn't really needed (stopped ports working * have a working version that will iterate through ports separately. * renaming to make more sense * builder class testing (#7) * builder class testing * deleting old data class from w/end tests * moved a couple of helper functions out of data builder and into helpers * fixed an issue with the imports and a few other var naming issues Co-authored-by: Jon Clayton <[email protected]> Co-authored-by: Jon Clayton <[email protected]> * Feature/node properties check (#22) * adding to access checker * testing icmp specifically * walked the results of policy checker * now able to display only the permitted hits * all results for all nodes * built results dict * structuring the results into a new object * building access result * now showing all permits * merging permit and deny lists * pulling data from tracetreelist and putting into accept object * adding fields * meaningful data now populating the acceptresult object * adding black formatting and optional type for annotation * tidy ups and annotations * re-engineering parts and adding more functionality / bare bones that wil allow jimi to execute the access checks * Major overhall and squashed a load of commits via rebase interactive renaming snapshot dir renaming snapshot dir setting folder sane defaults in one place fixes fixes fixes fixes fixes fixes various fixes to testing various fixes to testing various fixes to testing various fixes to testing various fixes to testing various fixes to testing various fixes to testing changes fixes fixes fixes fixes testing seems to be working now other than json serializable error seems to be working now other than json serializable error seems to be working now other than json serializable error major deletions and some comments major overhaul, renaming, making more readable, more comments adding drone ci file drone drone drone update drone update adding kube deployment restart * changing remoteConnectBatfish to batfishConnect * changing remoteConnectBatfish to batfishConnect * upgrading plugin version * upgrading plugin version * upgrading plugin version * drone * adding dstPorts as a header constraint * major changes, checks on text inputs, also adding ipProtocols and dstPorts * adding in Route Check to actions to build model adding in eachability check fix typo in reachability check * fixing things by removing old batfishops references that caused exceptions2 * fixing route check via debug * fixing reachability check * major works on deny results * sort reachability data structures and create a new class * WIP: testing simplyfying the returned results and putting into custom class/struct * testing creating new tr class/struct, seems to be working as it should now * more refactoring of reachability checker * a few tweaks to variables/args * work on route checker * latest route tweaks * happy with the traceroute checker in its current form. Its a bit bare bones and you have to ghand feed it the correct data but is working. * some moving around of file, refractoring, simplification and fixing a few issues with the returned data / dicts. Still need to fix the stack overflow that is happening when accessing the returned data. * change version to force re-register models * change version to force re-register models * change version to force re-register models * change version to force re-register models * tidy ups, better commenting, deletions, polishing, fixes, refactoring * moving files around * fixed problems with nodespec being a list but needing to be a string * lots of comments, deletions, tidy-ups and tweaks * remove unneeded brackets * more changes, try/catch, better comments, better naming of functions, adding dstPorts to the returned data * minor tweaks * some class name tweaks to follow best practice * some class name tweaks to follow best practice * docstrings, type hints and some fixes that flake8 reported * refactored and fixed typed hints, but discovered a possible bug with passing in multiple ports which should match more than one accept rule * more type hints * moved some very generic helpers out of a class into helper class with staticmethods, better checking and exceptino handling, more if/elif and logics to check data before starting, added a pre-flight check function * latest tests to find out why single port getting hit in results * adding dataclasses * removed an old helper that was causing an issue and isn't really needed (stopped ports working * have a working version that will iterate through ports separately. * renaming to make more sense * builder class testing (#7) * builder class testing * deleting old data class from w/end tests * moved a couple of helper functions out of data builder and into helpers * fixed an issue with the imports and a few other var naming issues Co-authored-by: Jon Clayton <[email protected]> Co-authored-by: Jon Clayton <[email protected]> Co-authored-by: Jon Clayton <[email protected]>
bodleytunes · Jun 3, 2021 · 4c47481 · 4c47481
1 parent b09a7db
commit 4c47481
Show file tree

Hide file tree

Showing 9 changed files with 131 additions and 4 deletions.
diff --git a/.gitignore b/.gitignore
@@ -223,6 +223,8 @@ _Pvt_Extensions
 
 __pycache__/snmp.cpython-38.pyc
 
+
+
 **/__pycache__/**
 **.cpython*
-*cpython*
+*cpython*
diff --git a/includes/batfish.py b/includes/batfish.py
@@ -11,6 +11,7 @@
 
 class Batfish:
     def __init__(self, NETWORK_NAME=None, host=None, snapshot_folder=None):
+
         self.NETWORK_NAME: str = "Firewalls"
         self.host: str = host
         self.snapshot_folder: str = snapshot_folder

diff --git a/includes/queries/access_check.py b/includes/queries/access_check.py
@@ -1,17 +1,23 @@
 from logging import exception
 from typing import Optional, List, Tuple, DefaultDict, Any
+
 from collections import defaultdict
 
 import pandas as pd
 
 from pybatfish.exception import BatfishException
 
 from plugins.batfish.includes.batfish import Batfish
+
+
 from plugins.batfish.includes.bat_helpers import BatHelpers
 from plugins.batfish.includes.data.builder import AccessDataBuilder
 
 
 class AccessCheck(Batfish):
+
+
+
     def __init__(
         self,
         batfish_server: Optional[str] = None,
@@ -75,6 +81,8 @@ def get_results(
         self._pre_flight_checks()
 
         # create empty list for returned results (Accept and Deny results)
+
+
         self.results_dict: dict = defaultdict(list)
 
         # Loop through all passed in nodes(Network devices/Firewalls)
@@ -133,14 +141,16 @@ def _query(self, nodes: Optional[list] = None) -> pd.DataFrame:
             flow = self.b_fish.hc(
                 srcIps=self.src_ip, dstIps=self.dst_ip, applications=self.applications
             )
+
             self._make_query(flow, nodes)
         elif len(self.dst_ports) > 0 and len(self.ip_protocols) > 0:
             # send dst_ports to splitter helper
             self.dst_ports_list = BatHelpers._split_ports(self.dst_ports)
             # there are more than one port returned in the list then loop through ports and make a query on each one
             if len(self.dst_ports_list) > 1:
+
+
                 # run queries on multiple ports
-                print("multiple ports")
                 for port in self.dst_ports_list:
                     flow = self.b_fish.hc(
                         srcIps=self.src_ip,
@@ -149,6 +159,8 @@ def _query(self, nodes: Optional[list] = None) -> pd.DataFrame:
                         ipProtocols=BatHelpers.make_upper(self.ip_protocols),
                     )
                     self._make_query(flow, nodes)
+
+
             # single port
             else:
                 flow = self.b_fish.hc(
@@ -166,6 +178,8 @@ def _query(self, nodes: Optional[list] = None) -> pd.DataFrame:
             )
             self._make_query(flow, nodes)
 
+
+        # ip protocols
         elif len(self.ip_protocols) > 0:
             flow = self.b_fish.hc(
                 srcIps=self.src_ip,

diff --git a/includes/queries/node_properties.py b/includes/queries/node_properties.py
@@ -18,3 +18,93 @@ def check_node_properties(self, snapshot_folder=None):
         result = df
 
         return result
+
+
+
+
+####################################################
+
+
+from logging import exception
+from typing import Optional, List, Tuple, DefaultDict, Any
+from collections import defaultdict
+
+import pandas as pd
+
+from pybatfish.exception import BatfishException
+
+from plugins.batfish.includes.batfish import Batfish
+from plugins.batfish.includes.bat_helpers import BatHelpers
+from plugins.batfish.includes.data.builder import AccessDataBuilder
+
+
+class NodePropertiesCheck(Batfish):
+    def __init__(
+        self,
+        batfish_server: Optional[str] = None,
+        host: Optional[str] = None,
+        src_ip: Optional[str] = None,
+        dst_ip: Optional[str] = None,
+        applications: Optional[list] = None,
+        dst_ports: Optional[str] = None,
+        ip_protocols: List[Any] = None,
+        nodes: Optional[str] = None,
+        node: str = None,
+        snapshot_folder: Optional[str] = None,
+        b_fish=None,
+    ):
+
+        self.batfish_server = batfish_server
+        self.host = host
+        self.src_ip = src_ip
+        self.dst_ip = dst_ip
+        self.applications = applications
+        self.dst_ports = dst_ports
+        self.ip_protocols = ip_protocols
+        self.snapshot_folder = snapshot_folder
+        # self.nodes = "hub2"
+
+        # Instance of a batfish object
+        self.b_fish = b_fish
+
+        pass
+
+    def run(
+        self,
+        node: Optional[str] = None,
+    ):
+
+        # create empty list for returned results (Accept and Deny results)
+        self.results_dict: dict = defaultdict(list)
+
+        results = self._query(node)
+
+        self._build_results(results)
+
+        return results
+
+    def _query(self, nodes: Optional[list] = None) -> pd.DataFrame:
+
+        self._make_query(nodes)
+
+    def _make_query(self, nodes):
+        """
+        make query
+        """
+        # nodes is actually a single node here, not sure why batfish have named it "nodes"?
+        try:
+            query = self.b_fish.bfq.nodeProperties(nodes=nodes)
+            result = query.answer().frame()
+            # Append each nodes query result to the results_dict list
+            self.results_dict[nodes].append(result)
+
+        except BatfishException as e:
+            print(e)
+            raise BatfishException(f"Batfish Query failure :  {e}")
+
+    def _build_results(
+        self, results_dict: DefaultDict[str, List[Any]]
+    ) -> Tuple[List[dict], List[dict]]:
+
+
+
diff --git a/includes/queries/reachability_check.py b/includes/queries/reachability_check.py
@@ -1,4 +1,6 @@
 from typing import Any, Dict
+
+
 import json
 
 from plugins.batfish.includes.batfish import Batfish
@@ -38,6 +40,8 @@ def __init__(
         self.tr = TraceResult()
         self.fl = FlowResult()
 
+
+
         self.trace_result: dict
 
         self.b_fish = b_fish
@@ -68,6 +72,8 @@ def check(
         # separate out Flow and Traces
         try:
             flow = result.iloc[0]["Flow"]
+
+
         except BaseException as e:
             raise BaseException(f"out of bounds {e}")
 
@@ -93,7 +99,9 @@ def _generate_flow_data(self, flow) -> FlowResult:
         self.fl.src_ip = flow.srcIp
         self.fl.dst_ip = flow.dstIp
         self.fl.ip_protocol = flow.ipProtocol
-        self.fl.ingress_node = flow.ingressNode
+
+
+        self.fl.destination_ingress_node = flow.ingressNode
         self.fl.ingress_vrf = flow.ingressVrf
 
     def _generate_trace_data(self, traces) -> TraceResult:
@@ -190,6 +198,8 @@ def _generate_trace_data(self, traces) -> TraceResult:
             # finally append trace to reachability result
             self.rr.trace_result.append(self.tr)
 
+
+
             # return a dictionary of the nested objects to be consumed by jimi
             self.trace_result: dict = self._build_dict(self.rr)
 

diff --git a/includes/result_models/__pycache__/reachability.cpython-37.pyc b/includes/result_models/__pycache__/reachability.cpython-37.pyc
diff --git a/includes/result_models/access.py b/includes/result_models/access.py
@@ -1,3 +1,5 @@
+
+
 from typing import Any
 from dataclasses import dataclass
 

diff --git a/includes/result_models/reachability.py b/includes/result_models/reachability.py
@@ -89,13 +89,17 @@ def __init__(
         self.src_ip: str
         self.dst_ip: str
         self.dst_port: str
-        self.ingress_node: str
+
+
+        self.destination_ingress_node: str
         self.ingress_interface: str
         self.ingress_vrf: str
         self.ip_protocol: str
 
         pass
 
 
+
+
 # @dataclass
 # class ReachabilityResultDataView:
diff --git a/models/action.py b/models/action.py
@@ -146,6 +146,7 @@ def setAttribute(self, attr, value, sessionData=None):
 
 class _batfishTraceRouteCheck(action._action):
 
+
     """
     * Connect to existing batfish Batfish() object
     * Create TraceRouteCheck() and pass it the Batfish() client
@@ -247,6 +248,8 @@ def doAction(self, data):
 
         try:
             b_fish = data["eventData"]["remote"]["client"]
+
+
         except BaseException as e:
             b_fish = None
             raise BaseException(f"error {e}")
@@ -266,6 +269,7 @@ def doAction(self, data):
                 dstIps=self.dst_ips,
             )
 
+
             data["eventData"]["remote"]["trace_results"] = rc.trace_result
             data["eventData"]["remote"]["flow_results"] = rr.flow_result.__dict__