Tuesday, February 12, 2019

Upgrading Qubes to latest fedora 29 template on an old (4.0), but not turned on in months laptop

I had qubes-os 4.0 on a laptop I hadn't turned on in months. It was still running fedora-26.

I wanted to update directly to the latest template (fedora-29). A direct in-place upgrade (26->29) isn't recommended by fedora, but totally worked and even says it should work in the Qubes upgrade docs.

Summary

 

I only did two things different than the stock standard directions in these three docs:
https://www.qubes-os.org/doc/templates/
https://www.qubes-os.org/doc/template/fedora/upgrade-28-to-29/
https://www.qubes-os.org/doc/disposablevm-customization/
  1. The following command errored, saying I was missing rpmfusion gpg keys. I added them successfully and the upgrade completed.
    sudo dnf --releasever=29 --best --allowerasing distro-sync
  2. I am using two template vms so I have a more-trusted standard fedora-29 vm with less installed software exposed in my sys-* vms.
The Qubes documentation is excellent and I saw even more this time around re-reading them while making this blog post. If you have more questions, the mailing list is very helpful and searchable here: https://groups.google.com/forum/#!forum/qubes-users

The purpose of this blog post is to document my step by step process for this upgrade so I can use it as a reference and others can learn what I learned. I'm sure some things can be more efficient and automated, but I'm first learning the GUIs provided. I'm not opposed to cli, just trying to understand the different mangement features provided by qubes.

Upgrading


I wanted to try in this less stressful environment, so I chose this laptop, since I don't need it for my day-to-day work and I wanted to compare and contrast in-place upgrades with downloading a brand new template.

First, I upgraded dom0

> sudo qubes-dom0-upgrade

This got me to: Kernel 4.14.74, xen 4.8.4, and still Qubes 4.0.

I then rebooted because I like rebooting a lot to make sure I am exactly where I expect to be, nothing pending.

I installed fedora-29 two different ways

 

Method 1: Install the new template:

I followed directions here: https://www.qubes-os.org/doc/templates/
> sudo qubes-dom0-update qubes-template-fedora-29

This was super easy. I tested by creating and launching a vm from this template using the gui "Create Qubes vm" launcher item and manually choosing the "fedora-29" VM template. Success. I updated it internally by booting the terminal in the qubesvm and running "sudo dnf update".

Note: I just realized I don't know if I type update or upgrade usually, and it turns out that it's the same command, but update is a deprecated alias for upgrad according to "man dnf".

Method 2: I upgraded my template VM for fedora-26 in-place

I used the same instructions here: https://www.qubes-os.org/doc/template/fedora/upgrade-28-to-29/

Notes and one error I encountered and fixed in my process:

I named my template fedora-29-1 because I didn't want the simultaneous download of the fedora-29 template that dom0 was doing to have any naming issues/errors.
I encountered an error upgrading that stated I didn't have fedora29 rpmfusion free and nonfree signing keys imported. I checked /etc/pki/rpm-gpg and sure enough, they were missing, it automatically added fedora 25-28, but ignored fedora29, strangely enough.

I imported the keys with the following commands:
  • [user@fedora-29-1 rpm-gpg]$ sudo rpm --import 'https://rpmfusion.org/keys?action=AttachFile&do=get&target=RPM-GPG-KEY-rpmfusion-free-fedora-29'
  • [user@fedora-29-1 rpm-gpg]$ sudo rpm --import 'https://rpmfusion.org/keys?action=AttachFile&do=get&target=RPM-GPG-KEY-rpmfusion-nonfree-fedora-29'
As mentioned might be a necessity in the link, I did need to add disk space after downloading all the files. I got this message after restarting my template upgrade command:
Error Summary
-------------
Disk Requirements:
At least 904MB more space needed on the / filesystem.
I tried copying over the cache folder so I wouldn't have to waste another 20+ minutes waiting for 1.4GB of files to download, but I was unable to get the cache to copy. If someone knows how to copy the cache with a lot of cached downloads, let me know. ** I recommend adding disk space pre-emptively, because it if you've installed much, the stock VM probably doesn't have space for the in-place upgrade. ** It takes an extra 30 seconds up front and saves 10-30 minutes later.

Summary: The documentation is great for this. I recommend preemptively adding the disk-cache. Then import the rpm-fusion keys before upgrading.

So far we haven't changed anything that we're relying on. I love the segmentation of Qubes that allows us to install a new template and upgrade a cloned template in-place, and have nothing I was relying on risked because of this. The segmentation is awesome. Now we switch VMs to the new templates

Qubes Template Manager is AWESOME 

 

Q icon> System Tools > Qubes Template Manager
https://www.qubes-os.org/doc/templates/#how-to-switch-templates-40
It makes switching to a different/new template effortless.
I did have to shutdown some VMs in order to switch them over.
I also rebooted a couple times because I wanted to make sure everything was working properly.

My don't-put-all-my-eggs-in-one-basket sequence that worked:  

First: A spot check

I spot-checked both templates by creating a new-vm from those templates and tested out network connectivity by opening firefox to a new page. I upgrade the templates a well and tested out VLC in the 29-1 template since it should still be installed. All worked. Great.

Second: Move over easy VMs


  • Spot check one VM that has persistence. I use a VM that had some files in Downloads that were still there in the migrated AppVM with the new Fedora-29-1 template and I confirmed in the CLI that cat /etc/fedora-release as 29.
  • Do the rest of the vms besides sys-* and disposible-vms.
  • Reboot

Third: Move sys-* after a reboot

  • I'm cautions/still learning how restart services and had trouble letting Qubes Manager let me kill sys-network. I decided to reboot and then update the sys-* vms.
  • I booted, shut down anything that had automatically started, and then I was able to shutdown/kill the sys-* vms to update their template.
  • I rebooted again to get the full start scripts running in the right order because that seemed like less work than doing it manually and this fells like a fuller-test if they work.

Disposable VMs need a bit extra work:

This link shows the few commands for updating and includes deletion.
https://www.qubes-os.org/doc/disposablevm-customization/

I consciously chose fedora-29-1 for the dvm-29-1 and not the clean, fedora-29 vm because I wanted VLC and a few other disposable applications that I don't want in my sys-vms.

I did end up with two launcher items/VMs for different disposable VMs. I deleted the old one following directions in the link above.

Notes on what I did differently than stock-standard:

I'm running two templates, the upgraded-in-place one (fedora-29-1) and the downloaded fedora-29 template. Anything involving sys-* or display-mgmt vms I set to fedora-29. For things that need installed software, I used fedora-29-1.
Eventually I might get minimal and/or disposible VMs working for these. I wasn't able to get a minimal VM working for network or firewall on my latest attempt. I haven't given up yet though.

I set personal use (work/personal/untrusted/etc) vms to fedora-29-1 because that has extra packages installed that I want to stay around.