Cannot upgrade a jail to newer FreeBSD release

[madu41]

Upgrading a jail to a new version of FreeBSD fails. The new version is available for Iocage.

FreeBSD version installed:

$ freebsd-version -kru
13.3-RELEASE-p7
13.3-RELEASE-p7
13.3-RELEASE-p7

Iocage version installed and used:

$ iocage -v
Version	1.8

$ pkg info | grep iocage
py311-iocage-1.8               FreeBSD jail manager written in Python3

I want to upgrade a jail still on FreeBSD 13.2:

$ iocage list -l | grep webbmail
| 15  | webbmail         | on   | up    | jail | 13.2-RELEASE-p12 | vnet0|10.2.50.57/24  | -   | -        | yes      |

Now I try to upgrade that jail to 13.3.

$ iocage upgrade webbmail -r 13.3-RELEASE
etcupdate failed! Rolling back snapshot.

No upgrade. If I try again I get a new error message:

# iocage upgrade webbmail -r 13.3-RELEASE
Snapshot already exists
Traceback (most recent call last):
  File "/usr/local/bin/iocage", line 10, in <module>
    sys.exit(cli())
             ^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/iocage_cli/upgrade.py", line 40, in cli
    ioc.IOCage(jail=jail, skip_jails=skip_jails).upgrade(release)
  File "/usr/local/lib/python3.11/site-packages/iocage_lib/iocage.py", line 2081, in upgrade
    ).upgrade_basejail()
      ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/iocage_lib/ioc_upgrade.py", line 239, in upgrade_basejail
    self.__snapshot_jail__()
  File "/usr/local/lib/python3.11/site-packages/iocage_lib/ioc_upgrade.py", line 406, in __snapshot_jail__
    ioc.IOCage(jail=self.uuid, skip_jails=True, silent=True).snapshot(name)
  File "/usr/local/lib/python3.11/site-packages/iocage_lib/iocage.py", line 1732, in snapshot
    ioc_common.logit(
  File "/usr/local/lib/python3.11/site-packages/iocage_lib/ioc_common.py", line 109, in logit
    callback(content, exception)
  File "/usr/local/lib/python3.11/site-packages/iocage_lib/ioc_common.py", line 94, in callback
    raise callback_exception(message)
iocage_lib.ioc_exceptions.Exists: Snapshot already exists

[Defenso-QTH]

From what I understand freebsd-update is not able to deal with some conflicts in your configuration files all by itself.

I would try to upgrade interactively from inside the jail.

[dgeo]

You have no output before etcupdate failed! Rolling back snapshot. ?
For the second error, I think you should find a snapshot with iocage snaplist webbmail, named ioc_upgrade..., that you could suppress to get rid of this error.

[madu41]

Thank you for responses!

@Defenso-QTH, All my jails are basejails. Is it even possible to undate those from the inside?

@dgeo, no there is no output before etcupdate failed! Rolling back snapshot. I have been looking for a verbose filag, but now I see a "--debug" to Iocage. Maybe that could give something.

The second error is because the snapshot for the update is not removed, if I understand it correctly.

Has anyone upgraded 13.2 jails to 13.3 under 13.3?

[dgeo]

I did that something like 160 times this year, with only two "bugs", one due to a previous interrupted ugdate (cleaning /var/db/freebsd-update in the jail did the job), the second still unexplained where install command took several seconds for each file.

[dgeo]

@madu41 when you write "all my jails are basejails", you mean independant jails, not cloned from a release (I'd name that thickjails in current iocage code) ?
If I remember well, older "basejail" system (from shell iocage version) were jails with a read-only /basejail mountpoint and symbolic links pointing there for /usr/bin, /usr/lib and so on... those ones are trickier to upgrade using freebsd-update !

[madu41]

Yes, I mean "basejail" with nullfs mounting of shared data. I have created those with iocage create -b (...). I created those under FreeBSD 12 or earlier and I have done several updates without problem, even from 12 to 13.

I have some feeling that something changed between 13.2 and 13.3. Wasn't the $FreeBSD$ CVS tag removed? Normal upgrade 13.2 to 13.3 worked, but not in Iocage?

[dgeo]

the cvs tag should not harm, (apart from making you validate many diff in some cases), but not blocking whole process...
I did not test basejail upgrade, only thin (zfs clones) and thick jails.
If I find some time I'll try some testing...
If I got it well, you do not need binary upgrades on every jail, only etcupdate (knowing that freebsd-update makes the glue to avoid re-building /var/db/etcupdate, which should be possible to script out?)

[madu41]

I had this problem already with previous version of Iocage. I will try again to see if I see something. I rather not recreate the jails.

[madu41]

If I create a new jail on RELEASE-13.2 I can upgrade that to RELEASE-13.3 or RELEASE-13.4, but the "old" jails cannot be updated (error message etcupdate failed).

[htmue]

I had the same problem (etcupdate failed) when trying to upgrade from 13.3 to 13.4 after a successful upgrade from 13.2 to 13.3.

Running etcupdate in the jail showed the reason: Conflicts remain from previous update, aborting. Running etcupdate resolve in the jail solved it.

[madu41]

@htmue, did you run etcupdate in each jail before running the update with iocage? What kind of jails do you run? "Base jail" as I do?

[htmue]

@madu41 Yes, in every jail, and yes, base jails.