neops.io as a network automation software was built with the following principles in mind
- network engineers know their network best, so network design and logic of tasks/todo’s are in their responsibility
- neops.io itself has no, or as minimalistic as possible, knowledge of the network and the functionality of its devices. Information and logic about the network is brought to neops.io by tasks that are collecting information from the network or external systems.
- Facts are a central element in neops.io. They are the base upon to build the logic of tasks to deploy changes in the network
- neops.io should bring flexibility to implement functionality at different levels
- device level: change the methods of how to read and write from network devices to fit your vendors and your needs
- provider level: bring your own task providers to implement APIs of your peripheral systems or bring new features (like AI ;)) to neops.io
- task level: define the todo in your network within your tasks, so they will represent the prerequisites of your network
- permission, workflow level (planned) and triggers (planned): easy automate or delegate tasks
graph TD
f1([Frontend])
b1([Backend])
w1([Worker])
w2([Worker])
db1[(Postgres)]
db2[(Elastic)]
n1[Network Devices]
n2[Network Devices]
f1 --> | GraphQL | b1
b1 --> db1
b1 --> db2
b1 --- | Redis | w1
b1 --- | Redis | w2
w1 --> db1
w1 --> db2
w2 --> db1
w2 --> db2
w1 --> n1
w2 --> n1
w1 --> n2
w2 --> n2
We ship one container with different functionality. Frontend (VueJS) and Backend (python) are packed in one container.
Depending on its start parameters the neops.io container runs as:
- Frontend/Backend
- Worker
- Scheduler
neops.io depends on external ressources/services/containers:
- Redis - message brocker between backend/worker/scheduler
- Postgres - relational database
- Elastic Search - search database
- Keycloak - authentication service
graph TD
rp([Traefik Reverse Proxy])
bf1([neops.io Backend/Frontend])
bf2([neops.io Backend/Frontend])
w1([neops.io Worker])
w2([neops.io Worker])
s1([neops.io Scheduler])
db1[(Postgres)]
db2[(Elastic)]
r1[(Redis)]
a1([Authentication/Keycloak])
rp --> bf1
rp --> bf2
bf1 & bf2 --> r1
r1 --> w1 & w2
bf1 & bf2 --- db1
bf1 & bf2 --- db2
w1 & w2 --- db1
w1 & w2 --- db2
s1 --- db1
s1 --> r1
bf1 & bf2 --> a1
style bf1 stroke,stroke-dasharray: 5, 5
style w1 stroke,stroke-dasharray: 5, 5
neops.io has all its persistent data in the database, so neops.io containers can be easily run in parallel for scaling reasons.
For scaling and/or redundancy setup of 3rd party applications (Postgres, Elastic Search, Redis and Keycloak) please refer to their documentation.
In reference to network components neops.io is currently working with the following entities:
- Device Groups - Entity to group devices
- Devices - Main entity to access network components
- Interfaces - Entity for interconnecting Devices
- Clients - Entity connected to the network
Locations are Device Groups with additional properties like address and coordinates.
The frontend is build with Vue.js and accesses the backend over a GraphQL API.
Backend is based on Python and realized with Django, Celery and nornir as its main components.
most relevant database models:
classDiagram
Interface "1" -- "1" Interface
Device "1" -- "n" Interface
Device "n" -- "1" Platform
Platform "n" -- "n" Library
DeviceGroup "n" -- "n" Device
DeviceGroup -- Location
ProcessExecution "1" -- "n" Execution
Execution "1" -- "n" DeviceExecution
Execution "n" -- "1" Task
DeviceExecution "n" -- "1" Device
Execution "1" -- "n" DeviceExecutionResult
Device "1" -- "n" DeviceExecutionResult
Execution "1" -- "n" DeviceGroupExecutionResult
DeviceGroup "1" -- "n" DeviceGroupExecutionResult
Execution "1" -- "n" InterfaceExecutionResult
Interface "1" -- "n" InterfaceExecutionResult
Execution "1" -- "n" ClientExecutionResult
Client "1" -- "n" ClientExecutionResult
Device "1" -- "n" DeviceConfiguration
Interface "1" -- "n" InterfaceConfiguration
Client "1" -- "n" Interface
Client "n" -- "1" Location
class ProcessExecution{
name
}
class Execution{
state
}
class DeviceExecution{
state
log
}
class DeviceExecutionResult{
state
message
}
class DeviceGroupExecutionResult{
state
message
}
class InterfaceExecutionResult{
state
message
}
class ClientExecutionResult{
state
message
}
class Device{
hostname
ip
facts
}
class Platform{
name
}
class Library{
name
}
class Interface{
name
neighbor
facts
}
class Client{
mac
facts
}
class DeviceGroup{
name
facts
}
class Task{
name
provider
task arguments
}
class DeviceConfiguration{
date
configuration
}
class InterfaceConfiguration{
date
configuration
}
class Location {
address
lon/lat
}
DeviceGroup, Device, Interface and Clients including their facts are stored as separated Indexes in Elastic Search
Celery is used for the Workers they are executing the neops.io Tasks/Providers. The workers have direct access to the databases. Scheduled tasks are distributed with a redis queue. To inform the Frontend about updated information GraphQL subscription updates are triggered over different redis queues.
All backend Operations are available over the GraphQL API. Testing and Documentation of the API is provided by GraphiQL on your installation under http(s)://NEOPS_URL/graphiql/
see provider
We use flake8 linting and black formatting in the backend and eslint in the frontend to style our code.