Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

more contracts usage in Nextcloud and docs enhancement #345

Merged
merged 2 commits into from
Nov 13, 2024
Merged

more contracts usage in Nextcloud and docs enhancement #345

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
17c8b0a
more readme refactoring
Nov 12, 2024
45017ee
use more contracts in nextcloud and update docs
Nov 13, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
137 changes: 64 additions & 73 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,20 +12,58 @@ that is not like the other server management tools.
## TOC

<!--toc:start-->
- [Unified Interfaces](#unified-interfaces)
- [Incremental Adoption](#incremental-adoption)
- [Usage](#usage)
- [More Benefits of SHB](#more-benefits-of-shb)
- [Manual](#manual)
- [Server Management](#server-management)
- [Unified Interfaces](#unified-interfaces)
- [Incremental Adoption](#incremental-adoption)
- [More Benefits of SHB](#more-benefits-of-shb)
- [Roadmap](#roadmap)
- [Available Blocks](#available-blocks)
- [Provided Services](#provided-services)
- [Demos](#demos)
- [Community](#community)
- [License](#license)
<!--toc:end-->

## Unified Interfaces
## Usage

> **Caution:** You should know that although I am using everything in this repo for my personal
> production server, this is really just a one person effort for now and there are most certainly
> bugs that I didn't discover yet.

Self Host Blocks is available as a flake.
To use it in your project, add the following flake input:

```nix
inputs.selfhostblocks.url = "github:ibizaman/selfhostblocks";
```

To get started using Self Host Blocks,
follow [the usage section](https://shb.skarabox.com/usage.html) of the manual.
It goes over how to deploy with [Colmena][], [nixos-rebuild][]
and also goes over secrets management with [SOPS][].

[Colmena]: https://colmena.cli.rs/
[nixos-rebuild]: https://nixos.org/manual/nixos/stable/#sec-changing-config
[SOPS]: https://github.com/Mic92/sops-nix

Then, to actually configure services, you can choose which one interests you in
[the services section](https://shb.skarabox.com/services.html) of the manual.

Head over to the [matrix channel](https://matrix.to/#/#selfhostblocks:matrix.org)
for any remaining question, or just to say hi :)

## Server Management

Self Host Blocks provides a standardized configuration for [some services](https://shb.skarabox.com/services.html) provided by nixpkgs.
The goal is to help spread adoption of self-hosting by providing an opinionated configuration with best practices by default.

Self Host Blocks takes care of common self-hosting needs:
- Backup for all services.
- LDAP and SSO integration for most services.
- Monitoring with Grafana and Prometheus stack with provided dashboards.
- Automatic reverse proxy and certificate management for HTTPS.
- VPN and proxy tunneling services.

### Unified Interfaces

SHB's first goal is to provide unified [building blocks](#available-blocks)
and by extension configuration interface, for self-hosting.
Expand Down Expand Up @@ -92,7 +130,7 @@ As you can see, they are pretty similar!
SHB provides an ever growing list of [services](#provided-services)
that are configured in the same way.

## Incremental Adoption
### Incremental Adoption

SHB's second goal is to facilitate testing NixOS
and slowly switching an existing installation to NixOS.
Expand All @@ -104,35 +142,15 @@ any reverse proxy you want or any database you want,
without requiring work from maintainers of the services you want to self host.
(See [manual][contracts] for a complete explanation)

[contracts]: https://shb.skarabox.com/contracts.html

## Usage

> **Caution:** You should know that although I am using everything in this repo for my personal
> production server, this is really just a one person effort for now and there are most certainly
> bugs that I didn't discover yet.

Self Host Blocks is available as a flake.
To use it in your project, add the following flake input:

```nix
inputs.selfhostblocks.url = "github:ibizaman/selfhostblocks";
```
Two videos exist of me presenting the topic,
the first at [NixCon North America in spring of 2024][NixConNA2024]
and the second at [NixCon in Berlin in fall of 2024][NixConBerlin2024].

More information is provided in [the manual](https://shb.skarabox.com/usage.html),
like secrets management.

- You are new to self hosting and want pre-configured services to deploy easily.
Look at the [services section](https://shb.skarabox.com/services.html).
- You are a seasoned self-hoster but want to enhance some services you deploy already.
Go to the [blocks section](https://shb.skarabox.com/blocks.html).
- You are a user of Self Host Blocks but would like to use your own implementation for a block.
Go to the [contracts section](https://shb.skarabox.com/contracts.html).

Head over to the [matrix channel](https://matrix.to/#/#selfhostblocks:matrix.org)
for any remaining question, or just to say hi :)
[contracts]: https://shb.skarabox.com/contracts.html
[NixConNA2024]: https://www.youtube.com/watch?v=lw7PgphB9qM
[NixConBerlin2024]: https://www.youtube.com/watch?v=CP0hR6w1csc

## More Benefits of SHB
### More Benefits of SHB

By using Self Host Blocks, you get all the benefits of NixOS
which are, for self hosted applications specifically:
Expand Down Expand Up @@ -163,51 +181,23 @@ from other server management projects:
the `nixpkgs` input in the root `flakes.nix`, runs the tests and merges a PR with the new input if
the tests pass.

## Manual

The manual can be found at [shb.skarabox.com](https://shb.skarabox.com/).

Work is in progress to document everything in the manual but I'm not there yet. For what's not yet
documented, unfortunately the source code is the best place to read about them.
[Here](./modules/services) for services and [here](./modules/blocks) for blocks.

## Roadmap

Currently, the Nextcloud service and SSL block are the most advanced and most documented.
Currently, the Nextcloud, Vaultwarden services and the SSL and backup blocks are the most advanced and most documented.

Documenting all services and blocks will be done as I make all blocks and services use the
contracts.

Upstreaming changes is also on the roadmap.

Check [the issues](https://github.com/ibizaman/selfhostblocks/issues) to see planned works. Feel
free to add more!

That being said, I am personally using all the blocks and services in this project, so they do work
to some extent.
Check the [issues][] and the [milestones]() to see planned work.
Feel free to add more or to contribute!

## Available Blocks
[issues]: (https://github.com/ibizaman/selfhostblocks/issues)
[milestones]: https://github.com/ibizaman/selfhostblocks/milestones

- [`authelia.nix`](./modules/blocks/authelia.nix) for Single Sign On.
- [`backup.nix`](./modules/blocks/backup.nix).
- [`ldap.nix`](./modules/blocks/ldap.nix) for user management.
- [`monitoring.nix`](./modules/blocks/monitoring.nix) for dashboards, logs and alerts.
- [`nginx.nix`](./modules/blocks/nginx.nix) for reverse proxy with SSL termination.
- [`postgresql.nix`](./modules/blocks/postgresql.nix) for database setup.
- [`ssl.nix`](./modules/blocks/ssl.nix) for maintaining self-signed SSL certificates or certificates provided by Let's Encrypt.
- [`tinyproxy.nix`](./modules/blocks/tinyproxy.nix) to forward traffic to a VPN tunnel.
- [`vpn.nix`](./modules/blocks/vpn.nix) to setup a VPN tunnel.

## Provided Services

- [`arr.nix`](./modules/services/arr.nix) for finding media https://wiki.servarr.com/.
- [`deluge.nix`](./modules/services/deluge.nix) for downloading linux isos https://deluge-torrent.org/.
- [`hledger.nix`](./modules/services/hledger.nix) for managing finances https://hledger.org/.
- [`home-assistant.nix`](./modules/services/home-assistant.nix) for private IoT https://www.home-assistant.io/.
- [`jellyfin.nix`](./modules/services/jellyfin.nix) for watching media https://jellyfin.org/.
- [Nextcloud Server](https://shb.skarabox.com/services-nextcloud.html) for private documents, contacts, calendar, etc https://nextcloud.com.
- [`vaultwarden.nix`](./modules/services/vaultwarden.nix) for passwords https://github.com/dani-garcia/vaultwarden.
- [`audiobookshelf.nix`](./modules/services/audiobookshelf.nix) for hosting podcasts and audio books https://www.audiobookshelf.org/.
All blocks and services have NixOS tests.
Also, I am personally using all the blocks and services in this project, so they do work to some extent.

## Demos

Expand All @@ -228,6 +218,7 @@ when I'm satisfied with how things look, I'll upstream changes.

## License

I'm following the [Nextcloud](https://github.com/nextcloud/server) license which is AGPLv3. See
[this article](https://www.fsf.org/bulletin/2021/fall/the-fundamentals-of-the-agplv3) from the FSF that explains what this license adds to the GPL
one.
I'm following the [Nextcloud](https://github.com/nextcloud/server) license which is AGPLv3.
See [this article][why agplv3] from the FSF that explains what this license adds to the GPL one.

[why agplv3]: (https://www.fsf.org/bulletin/2021/fall/the-fundamentals-of-the-agplv3)
9 changes: 9 additions & 0 deletions docs/contracts.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,15 @@ In practice, a contract is a set of options that any user of a contract expects
values of these options dictate the behavior of the implementation. This is enforced with NixOS VM
tests.

## Videos {#contracts-videos}

Two videos exist of me presenting the topic,
the first at [NixCon North America in spring of 2024][NixConNA2024]
and the second at [NixCon in Berlin in fall of 2024][NixConBerlin2024].

[NixConNA2024]: https://www.youtube.com/watch?v=lw7PgphB9qM
[NixConBerlin2024]: https://www.youtube.com/watch?v=CP0hR6w1csc

## Provided contracts {#contracts-provided}

Self Host Blocks is a proving ground of contracts. This repository adds a layer on top of services
Expand Down
20 changes: 10 additions & 10 deletions docs/usage.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -279,15 +279,15 @@ One way to setup secrets management using `sops-nix`:
selfhostblocks.inputs.sops-nix.nixosModules.default
];
```
6. Reference the secrets in nix:
6. Set default sops file:
```bash
sops.defaultSopsFile = ./secrets.yaml;
```
Setting the default this way makes all sops instances use that same file.
7. Reference the secrets in nix:
```nix
shb.nextcloud.adminPassFile = config.sops.secrets."nextcloud/adminpass".path;

sops.secrets."nextcloud/adminpass" = {
sopsFile = ./secrets.yaml;
mode = "0440";
owner = "nextcloud";
group = "nextcloud";
restartUnits = [ "phpfpm-nextcloud.service" ];
};
shb.nextcloud.adminPass.result.path = config.sops.secrets."nextcloud/adminpass".path;

sops.secrets."nextcloud/adminpass" = config.shb.nextcloud.adminPass.request;
```
The above snippet uses the [secrets contract](./contracts-secret.html) to ease configuration.
82 changes: 38 additions & 44 deletions modules/services/nextcloud-server.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -95,10 +95,11 @@ in
default = "root";
};

adminPassFile = lib.mkOption {
type = lib.types.nullOr lib.types.path;
description = "File containing the Nextcloud admin password. Required.";
default = null;
adminPass = contracts.secret.mkOption {
description = "Nextcloud admin password.";
mode = "0400";
owner = "nextcloud";
restartUnits = [ "phpfpm-nextcloud.service" ];
};

maxUploadSize = lib.mkOption {
Expand All @@ -121,7 +122,11 @@ in
postgresSettings = lib.mkOption {
type = lib.types.nullOr (lib.types.attrsOf lib.types.str);
default = null;
description = "Settings for the PostgreSQL database. Go to https://pgtune.leopard.in.ua/ and copy the generated configuration here.";
description = ''
Settings for the PostgreSQL database.

Go to https://pgtune.leopard.in.ua/ and copy the generated configuration here.
'';
example = lib.literalExpression ''
{
# From https://pgtune.leopard.in.ua/ with:
Expand Down Expand Up @@ -369,14 +374,11 @@ in
default = "admin";
};

adminPasswordFile = lib.mkOption {
type = lib.types.path;
description = ''
File containing the admin password of the LDAP server.

Must be readable by the nextcloud system user.
'';
default = "";
adminPassword = contracts.secret.mkOption {
description = "LDAP server admin password.";
mode = "0400";
owner = "nextcloud";
restartUnits = [ "phpfpm-nextcloud.service" ];
};

userGroup = lib.mkOption {
Expand Down Expand Up @@ -439,24 +441,17 @@ in
default = "one_factor";
};

secretFile = lib.mkOption {
type = lib.types.path;
description = ''
File containing the secret for the OIDC endpoint.

Must be readable by the nextcloud system user.
'';
default = "";
secret = contracts.secret.mkOption {
description = "OIDC shared secret.";
mode = "0400";
owner = "nextcloud";
restartUnits = [ "phpfpm-nextcloud.service" ];
};

secretFileForAuthelia = lib.mkOption {
type = lib.types.path;
description = ''
File containing the secret for the OIDC endpoint, must be readable by the Authelia user.

Must be readable by the authelia system user.
'';
default = "";
secretForAuthelia = contracts.secret.mkOption {
description = "OIDC shared secret. Content must be the same as `secretFile` option.";
mode = "0400";
owner = "authelia";
};

fallbackDefaultAuth = lib.mkOption {
Expand All @@ -478,9 +473,15 @@ in
extraApps = lib.mkOption {
type = lib.types.raw;
description = ''
Extra apps to install. Should be a function returning an attrSet of appid to packages
generated by fetchNextcloudApp. The appid must be identical to the “id” value in the apps
appinfo/info.xml. You can still install apps through the appstore.
Extra apps to install.

Should be a function returning an `attrSet` of `appid` as keys to `packages` as values,
like generated by `fetchNextcloudApp`.
The appid must be identical to the `id` value in the apps'
`appinfo/info.xml`.
Search in [nixpkgs](https://github.com/NixOS/nixpkgs/tree/master/pkgs/servers/nextcloud/packages) for the `NN.json` files for existing apps.

You can still install apps through the appstore.
'';
default = null;
example = lib.literalExpression ''
Expand Down Expand Up @@ -576,13 +577,6 @@ in

config = lib.mkMerge [
(lib.mkIf cfg.enable {
assertions = [
{
assertion = !(isNull cfg.adminPassFile);
message = "Must set shb.nextcloud.adminPassFile.";
}
];

users.users = {
nextcloud = {
name = "nextcloud";
Expand Down Expand Up @@ -617,7 +611,7 @@ in
config = {
dbtype = "pgsql";
adminuser = cfg.adminUser;
adminpassFile = toString cfg.adminPassFile;
adminpassFile = cfg.adminPass.result.path;
};
database.createLocally = true;

Expand Down Expand Up @@ -819,7 +813,7 @@ in
systemd.services.nextcloud-setup.script = ''
${occ} app:install files_external || :
${occ} app:enable files_external
'' + lib.optionalString (cfg.apps.externalStorage.userLocalMount != "") (
'' + lib.optionalString (cfg.apps.externalStorage.userLocalMount != null) (
let
cfg' = cfg.apps.externalStorage.userLocalMount;

Expand Down Expand Up @@ -859,7 +853,7 @@ in
${occ} ldap:set-config "${cID}" 'ldapAgentName' \
'uid=${cfg'.adminName},ou=people,${cfg'.dcdomain}'
${occ} ldap:set-config "${cID}" 'ldapAgentPassword' \
"$(cat ${cfg'.adminPasswordFile})"
"$(cat ${cfg'.adminPassword.result.path})"
${occ} ldap:set-config "${cID}" 'ldapBase' \
'${cfg'.dcdomain}'
${occ} ldap:set-config "${cID}" 'ldapBaseGroups' \
Expand Down Expand Up @@ -936,7 +930,7 @@ in
mkdir -p ${cfg.dataDir}/config
cat <<EOF > "${cfg.dataDir}/config/secretFile"
{
"oidc_login_client_secret": "$(cat ${cfg.apps.sso.secretFile})"
"oidc_login_client_secret": "$(cat ${cfg.apps.sso.secret.result.path})"
}
EOF
'';
Expand Down Expand Up @@ -1001,7 +995,7 @@ in
{
client_id = cfg.apps.sso.clientID;
client_name = "Nextcloud";
client_secret.source = cfg.apps.sso.secretFileForAuthelia;
client_secret.source = cfg.apps.sso.secretForAuthelia.result.path;
public = false;
authorization_policy = cfg.apps.sso.authorization_policy;
redirect_uris = [ "${protocol}://${fqdnWithPort}/apps/oidc_login/oidc" ];
Expand Down
Loading