Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

resumeDevice for swap on btrfs #651

Open
MithicSpirit opened this issue May 28, 2024 · 17 comments
Open

resumeDevice for swap on btrfs #651

MithicSpirit opened this issue May 28, 2024 · 17 comments
Labels
enhancement New feature or request question Not a bug or issue, but a question asking for help or information

Comments

@MithicSpirit
Copy link

How does one enable resumeDevice for a swapfile that is in a btrfs subvolume? Looking through https://github.com/nix-community/disko/blob/master/lib/types/btrfs.nix, there seem to be no options for this. I think it would also be nice to support other options from https://github.com/nix-community/disko/blob/master/lib/types/swap.nix if possible, such as priority, randomEncryption, and discardPolicy (although I am not as interested in these).

@Anomalocaridid
Copy link

Anomalocaridid commented Aug 26, 2024

Unfortunately, as far as I am aware, there is currently no way to enable this with disko. Instead, I use LVM and a dedicated swap volume instead of a btrfs subvolume, if that helps at all.

@nvmd
Copy link
Contributor

nvmd commented Aug 30, 2024

Support for resumeDevice would be a very nice feature indeed.

It doesn't make much sense, unfortunately – in order for suspend/hibernate to work with btrfs, kernel needs not only resume=... option to be specified on the command line, but also resume_offset=..., that needs to be computed with btrfs inspect-internal map-swapfile -r swap_file on live filesystem.

It doesn't look like resume_offset=... could be known in advance so that it can be specified in a declarative configuration prior to the deployment.

You can read more here: https://wiki.archlinux.org/title/Power_management/Suspend_and_hibernate#Acquire_swap_file_offset

@nvmd
Copy link
Contributor

nvmd commented Aug 30, 2024

@MithicSpirit some of the options you mentioned are a part of PR #752

@fhilgers
Copy link

fhilgers commented Sep 29, 2024

Hey, just wanted to let you know that with boot.initrd.systemd.enable swap just works.
Systemd automatically uses the swap device as resume target and also caclulates the resume offset when hibernating. On an efi device this information will be stored in the efivars and systemd reads it again on resume.
So you do not have to specify any kernel parameters or resumeDevice manually.
Read more here: https://wiki.archlinux.org/title/Power_management/Suspend_and_hibernate#Hibernation

@KingKrouch
Copy link

Hey, just wanted to let you know that with boot.initrd.systemd.enable swap just works. Systemd automatically uses the swap device as resume target and also caclulates the resume offset when hibernating. On an efi device this information will be stored in the efivars and systemd reads it again on resume. So you do not have to specify any kernel parameters or resumeDevice manually. Read more here: https://wiki.archlinux.org/title/Power_management/Suspend_and_hibernate#Hibernation

Got a good guide on how I'd enable swap hibernate on my Nix config then? I'm using a swapfile on a LUKS encrypted BTRFS subvolume. I already have systemd enabled in the initrd.

@fhilgers
Copy link

fhilgers commented Sep 29, 2024

I just created my swapfile using disko like in this example: https://github.com/nix-community/disko/blob/master/example/luks-btrfs-subvolumes.nix

I neither have boot.resumeDevice nor resume and resume_offset kernel params.

Then when i do systemctl hibernate everything just works. On boot I select the latest generation in systemd boot (the autoselected one) and enter the luks passphrase.
Maybe it has something to do with the systemd version. Storing resume_offset in efivars is a rather recent feature.

@KingKrouch
Copy link

KingKrouch commented Sep 30, 2024

I just created my swapfile using disko like in this example: https://github.com/nix-community/disko/blob/master/example/luks-btrfs-subvolumes.nix

I neither have boot.resumeDevice nor resume and resume_offset kernel params.

Then when i do systemctl hibernate everything just works. On boot I select the latest generation in systemd boot (the autoselected one) and enter the luks passphrase. Maybe it has something to do with the systemd version. Storing resume_offset in efivars is a rather recent feature.

Okay, I checked that, and I have my swapfile set up in a similar manner, but when I attempt to run systemctl hibernate it gives me this (for reference in case I need it, I have 32GB of RAM):

❯ sudo systemctl hibernate       
Call to Hibernate failed: Not enough suitable swap space for hibernation available on compatible block devices and file systems
❯ swapon --show
NAME               TYPE SIZE  USED PRIO
/.swapvol/swapfile file  20M 19.9M   -2

I have the swap subvolume set up like this:
https://gist.github.com/KingKrouch/aba13fb3517fbe5e302b91aa4c0e488e

@fhilgers
Copy link

It shows that your swapfile is only 20 Megabytes. Normally one should have as much swap space as RAM so you have to change that to 32G.

@KingKrouch
Copy link

It shows that your swapfile is only 20 Megabytes. Normally one should have as much swap space as RAM so you have to change that to 32G.

Yep, I changed that in my disko config to 38G, and that change makes it work.

So for my BTRFS subvolume, since I already partitioned my drive, I couldn't run that, so doing this made it work:

sudo swapoff /.swapvol/swapfile
sudo rm /.swapvol/swapfile
sudo truncate -s 0 /.swapvol/swapfile
sudo chattr +C /.swapvol/swapfile
sudo fallocate -l 38G /.swapvol/swapfile
sudo chmod 600 /.swapvol/swapfile
sudo mkswap /.swapvol/swapfile
sudo swapon /.swapvol/swapfile

@iFreilicht
Copy link
Contributor

One additional note: I disagree that it would be in-scope for disko to handle swapfiles. Swap partitions, sure, but if you want a swapfile you can just add

  swapDevices = [
    {
      device = "/var/lib/swapfile";
      size = 16 * 1024; # Same as available RAM
    }
  ];

To your regular config, and this has nothing to do with partitioning. Using this option will automatically set boot.resumeDevice as well.

Unless another maintainer disagrees, I would close this issue.

@iFreilicht iFreilicht added enhancement New feature or request question Not a bug or issue, but a question asking for help or information labels Sep 30, 2024
@MithicSpirit
Copy link
Author

One additional note: I disagree that it would be in-scope for disko to handle swapfiles. Swap partitions, sure, but if you want a swapfile you can just add

  swapDevices = [
    {
      device = "/var/lib/swapfile";
      size = 16 * 1024; # Same as available RAM
    }
  ];

To your regular config, and this has nothing to do with partitioning. Using this option will automatically set boot.resumeDevice as well.

This doesn't work on btrfs. According to the wiki, it's necessary to run some commands beforehand due to btrfs nuances. If that's not in scope for disko, then I'm not sure what is.

@iFreilicht
Copy link
Contributor

iFreilicht commented Sep 30, 2024

According to the wiki, it's necessary to run some commands beforehand due to btrfs nuances.

What wiki are we talking about here? According to https://wiki.nixos.org/wiki/Btrfs#Swap_file:

Optionally, create a separate subvolume for the swap file.

This is definitely in scope, but all of the commands that follow are basic creation of a subvolume, which is already supported:

# mkdir -p /mnt
# mount /dev/sdXY /mnt
# btrfs subvolume create /mnt/swap
# umount /mnt
# mkdir /swap
# mount -o noatime,subvol=swap /dev/sdXY /swap

Further:

Then, create the swap file and adjust its size as desired:

# btrfs filesystem mkswapfile --size 8g --uuid clear /swap/swapfile

So to me it seems there is a bug in nixpkgs and the activation script in NixOS should check what filesystem the swapfile is supposed to be on, and if it's btrfs, use that to create the file instead of dd.

I'm not opposed to writing a PR for this in nixpkgs, but I need your support with testing as I don't use btrfs for my root filesystem.

Also, if you have differing information, let me know.

@fhilgers
Copy link

fhilgers commented Sep 30, 2024

I do not quite understand the problem, disko already creates the swapfile on btrfs automatically which is shown in the example btrfs config.

For going into hibernation, nothing has to be specified, because systemd detects it automatically. One can specify resume and resume_offset on the kernel cmdline or write them to /sys/power/resume and /sys/power/resume_offset prior to hibernating to avoid auto detection.

For resuming from hibernation, nothing has to be specified in the case of using systemd in the intitrd, as systemd persists the values used when it started hibernation to the efivars.
In case of not using systemd in the initrd, the kernel parameters are the way to go.

As the resume offset can only be calculated after creating the swapfile, one would need to have to run a script for dynamically generating kernel parameters, which I do not know whether that is currently possible in NixOS.

More infos are in these issues from systemd: systemd/systemd#23177 systemd/systemd#27247

The simplest way is probably to document the behavior:

  1. if boot.initrd.systemd.enable: do nothing, it should just work
  2. if not: use btrfs inspect-internal map-swapfile -r swap_file after disko format and add kernel parameters

The correct way is probably to dynamically add the resume offset to the kernel parameters on nixos-install or nixos-rebuild switch. I do not know whether that is currently possible.

@MithicSpirit
Copy link
Author

Then, create the swap file and adjust its size as desired:

# btrfs filesystem mkswapfile --size 8g --uuid clear /swap/swapfile

Yeah, this is what I meant

So to me it seems there is a bug in nixpkgs and the activation script in NixOS should check what filesystem the swapfile is supposed to be on, and if it's btrfs, use that to create the file instead of dd.

That's fair. If it's fixed in nixpkgs, then I think it's fine to remove from disko.

I'm not opposed to writing a PR for this in nixpkgs, but I need your support with testing as I don't use btrfs for my root filesystem.

I'd probably end up having to test in a VM, but sure.

@iFreilicht
Copy link
Contributor

iFreilicht commented Sep 30, 2024

I do not quite understand the problem, disko already creates the swapfile on btrfs automatically which is shown in the example btrfs config.

I did not realize that, sorry. This seems to be a unique feature in disko's btrfs module, and indeed it creates the swapfile with btrfs filesystem mkswapfile. In that case, this behavior is partly broken and we do indeed have to fix it.

The simplest way is probably to document the behavior:

  1. if boot.initrd.systemd.enable: do nothing, it should just work
  2. if not: use btrfs inspect-internal map-swapfile -r swap_file after disko format and add kernel parameters

Just to make sure, if boot.initrd.systemd.enable is the only thing that's required to make this work, we could set that option as soon as one swapfile is defined in a disko btrfs subvolume to fix this issue, right?

The correct way is probably to dynamically add the resume offset to the kernel parameters on nixos-install or nixos-rebuild switch. I do not know whether that is currently possible.

The activation script also changes the boot configuration, so I'd say yes, though I haven't checked yet how well this fits in with the current structure. Would that be what's required for making btrfs swapfiles compatible with Grub as well?

That's fair. If it's fixed in nixpkgs, then I think it's fine to remove from disko.

I would like that, it seems kinda odd that disko would do this one thing that doesn't have anything to do with formatting or disk layout for this one filesystem only. Would need a bit of a transition period, though.

@nvmd
Copy link
Contributor

nvmd commented Oct 1, 2024

Just to make sure, if boot.initrd.systemd.enable is the only thing that's required to make this work, we could set that option as soon as one swapfile is defined in a disko btrfs subvolume to fix this issue, right?

I don't think enabling boot.initrd.systemd.enable just for a resume functionality is a good idea:

I think documenting this behavior, maybe even just as a comment in the mentioned example file, should be enough.

@NikodemMarek
Copy link

Hi, I have a btrfs partitioned disk without lvm (kind of like in this example https://github.com/nix-community/disko/blob/master/example/btrfs-subvolumes.nix), I've tried to enable hibernation via boot.initrd.systemd.enable, and it hibernates just fine, but resuming doesn't seem to work. Could anybody provide some help, to fix this problem?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request question Not a bug or issue, but a question asking for help or information
Projects
None yet
Development

No branches or pull requests

7 participants