Skip to content

Commit

Permalink
Readme
Browse files Browse the repository at this point in the history
  • Loading branch information
byrnedo committed Jul 15, 2024
1 parent 483f89b commit f539a52
Show file tree
Hide file tree
Showing 7 changed files with 85 additions and 18 deletions.
56 changes: 48 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,54 @@ Supported architectures: amd64, arm64

Could be described as one-shot scheduling.

### Networking

Static routes between hosts, maintained by a systemd unit file.
All containers attached to the default `podman` network which we modify.

### DNS

Dns is coredns with fanout between all nodes along with serving from file.

Hosts are maintained via a CNI plugin that adds/removes the ip to the hosts file.

Pods get a hostname of `<labels.app>.<metadata.namespace>.cluster.skate.`

### Ingress

Nginx container listening on port 80 and 443

Use an Ingress resource to enable.


```yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: foo-external
spec:
rules:
- host: foo.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: foo
port:
number: 80
```
Service resources are ignored and it's implicit that a pod has a service with url: `<labels.name>.<metadata.namespace>.cluster.skate`

Plan:
- Nginx container mounts /etc/skate/ingress.conf
- nginx reloads on file change
- skatelet updates the file on ingress resource change
- use letsencrypt and http verification


## Registering nodes

```shell
Expand Down Expand Up @@ -91,11 +139,3 @@ sudo apt-get install -y gcc make libssl-dev pkg-config
- [x] container dns
- [ ] ingress
- [ ] modded fanout to wait for all and round robin all

### Networking

Dns is coredns with fanout between all nodes along with serving from file.

Hosts are maintained via a CNI plugin that adds/removes the ip to the hosts file.

Good enough.
3 changes: 1 addition & 2 deletions hack/test-deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,7 @@ metadata:
labels:
app: nginx
spec:
replicas: 1

replicas: 3
selector:
matchLabels:
app: nginx
Expand Down
6 changes: 3 additions & 3 deletions manifests/coredns.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -19,12 +19,12 @@ spec:
volumes:
- name: cni
hostPath:
path: /var/lib/skatelet/cni/podman
path: /var/lib/skate/cni/podman
containers:
- name: coredns
image: ghcr.io/skateco/coredns
volumeMounts:
- mountPath: /var/lib/skatelet/cni/podman
- mountPath: /var/lib/skate/cni/podman
name: cni
env:
- name: CORE_FILE
Expand All @@ -33,7 +33,7 @@ spec:
bind lo 0.0.0.0
hosts /var/lib/skatelet/cni/podman/addnhosts
hosts /var/lib/skate/cni/podman/addnhosts
}
cluster.skate:53 {
Expand Down
28 changes: 28 additions & 0 deletions manifests/ingress.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: nginx
namespace: skate
labels:
app: nginx
spec:
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
hostNetwork: true
volumes:
- name: cni
hostPath:
path: /var/lib/skate/ingress
containers:
- name: coredns
image: ghcr.io/skateco/coredns
volumeMounts:
- mountPath: /var/lib/skate/ingress
name: ingress
6 changes: 3 additions & 3 deletions src/skate.rs
Original file line number Diff line number Diff line change
Expand Up @@ -104,9 +104,9 @@ impl SupportedResources {
// whether there's host network set
pub fn host_network(&self) -> bool {
match self {
SupportedResources::Pod(p) => p.clone().spec.unwrap().host_network.unwrap(),
SupportedResources::Deployment(d) => d.clone().spec.unwrap().template.spec.unwrap().host_network.unwrap(),
SupportedResources::DaemonSet(d) => d.clone().spec.unwrap().template.spec.unwrap().host_network.unwrap(),
SupportedResources::Pod(p) => p.clone().spec.unwrap_or_default().host_network.unwrap_or_default(),
SupportedResources::Deployment(d) => d.clone().spec.unwrap_or_default().template.spec.unwrap_or_default().host_network.unwrap_or_default(),
SupportedResources::DaemonSet(d) => d.clone().spec.unwrap_or_default().template.spec.unwrap_or_default().host_network.unwrap_or_default(),
}
}
fn fixup_metadata(meta: ObjectMeta, extra_labels: Option<HashMap<String, String>>) -> Result<ObjectMeta, Box<dyn Error>> {
Expand Down
2 changes: 1 addition & 1 deletion src/skatelet/cni.rs
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ use crate::skate::exec_cmd;


fn conf_path_str() -> String {
"/var/lib/skatelet/cni".to_string()
"/var/lib/skate/cni".to_string()
}

fn lock<T>(network_name: &str, cb: &dyn Fn() -> Result<T, Box<dyn Error>>) -> Result<T, Box<dyn Error>> {
Expand Down
2 changes: 1 addition & 1 deletion src/skatelet/skatelet.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ use crate::skatelet::apply::{ApplyArgs, remove, RemoveArgs};
use crate::skatelet::cni::cni;
use crate::skatelet::system::{system, SystemArgs};

pub const VAR_PATH: &str = "/var/lib/skatelet";
pub const VAR_PATH: &str = "/var/lib/skate";

#[derive(Debug, Parser)]
#[command(name = "skatelet")]
Expand Down

0 comments on commit f539a52

Please sign in to comment.