Encrypted Ubuntu Installation with "Manual" Partitioning
If you've ever installed a recent version of Ubuntu, you'll know that there is an option for Full-Disk Encryption (FDE) shortly after you begin the installation process. However, if you want to manually partition things (like I do), then you're out of luck! The Ubiquity installer does not support FDE and manual partitioning. That being said, you can still do it. I could be wrong, but I believe I was the first person to document the exact process. I was wrong. I've since found at least two other posts that document this process and predate my initial documentation considerably. If you follow that link and check out my answer, you'll see I note that the process of setting up FDE with LVM is not for novice users. Today I take that back!
In this post I present what I call the LGMP (LUKS guided manual partitioning) installation script. I set up different systems from time to time, and it was annoying having to go through the system prep and finalization steps manually each time, so I wrote a script to help automate the process and decided to share. The link to my GitHub where you can find the script is at the bottom of this post.
This script can install to a system with multiple disks attached, but it will only install the system to one of those disks. In other words, if you have two disks, you can install Ubuntu to /dev/sdb with my script, but if you want to utilize /dev/sda in your installation, you'll have to configure that manually.
With the pre-installation stage complete, the script informs the user to run the Ubuntu installer noting how to manually configure the partitions during installation. The user must complete the installation process and choose the option to "Continue Testing."
Now the user goes back to the open terminal window running my script. The post-installation stage begins by pressing the [Enter] key. The script will re-mount the installed system, configure the crypttab file and regenerate the boot files. That's it! Reboot, and you're ready to roll!
If a key file was created during system installation, this is saved to the installed user's desktop. This key file can be used in place of the passphrase for system decryption at boot time (or any other time). I won't document this process here as you can find various ways to do it documented elsewhere online. One such clever implementation can be found here.
To address the issue of changing the LUKS encryption passphrase and recover from a lost passphrase, my script dumps a script called Change-LUKS-passphrase.sh on the desktop. If a key file was not created during installation, this can only be used to change the LUKS passphrase using the current passphrase. If however, a key file was generated, it gets embedded in this script, allowing this script to be used for recovery in the event that the LUKS passphrase is forgotten.
There is also a file named Reinstallation.sh. This script can be run from a Live USB/DVD and helps decrypt the drive so that the existing partition structure and LUKS encryption can be reused with a new installation. This is useful for reinstalling while keeping the /home partition intact. When run, it provides the necessary prompts to guide the user through the reinstallation process.
While I was in the mood of trying to prevent utter disaster of the kind caused by forgotten passwords, I remembered that if the LUKS header gets corrupted, the consequences may be just as terrible (ie. no more possibility of decryption). Therefore, I configured my LGMP script to drop a copy of the LUKS header on the installed user's desktop as well so that it's not as easy to forget to save it on a separate backup medium.
To finish everything off, I created a LUKS-README.html file that goes into much further detail than this post about the LUKS implementation, which files need to be stored in a secure location, and how to use the scripts that are saved to the installed user's desktop.
In this post I present what I call the LGMP (LUKS guided manual partitioning) installation script. I set up different systems from time to time, and it was annoying having to go through the system prep and finalization steps manually each time, so I wrote a script to help automate the process and decided to share. The link to my GitHub where you can find the script is at the bottom of this post.
This script can install to a system with multiple disks attached, but it will only install the system to one of those disks. In other words, if you have two disks, you can install Ubuntu to /dev/sdb with my script, but if you want to utilize /dev/sda in your installation, you'll have to configure that manually.
How does it work?
My script operates in two stages, the pre-installation stage and the post-installation stage. When you boot the system from a Live OS, you first need to copy my script to the desktop, make it executable, and run it. This is done before running the Ubuntu installer. At this time, the pre-installation stage will ask the user a series of questions to determine how big the /boot, /boot/efi (if applicable), LVM (LUKS), / (root), /home, and swap partitions should be. Then it automates the process of creating all of the partitions and logical volumes. At this point the first stage is completed and the script will pause. The script window must stay open.With the pre-installation stage complete, the script informs the user to run the Ubuntu installer noting how to manually configure the partitions during installation. The user must complete the installation process and choose the option to "Continue Testing."
Now the user goes back to the open terminal window running my script. The post-installation stage begins by pressing the [Enter] key. The script will re-mount the installed system, configure the crypttab file and regenerate the boot files. That's it! Reboot, and you're ready to roll!
The added perks
I finished putting all of this together and realized, I wasn't actually done. What if a user wants to change the encryption passphrase and isn't familiar with cryptsetup and LUKS? What if the user forgets his/her passphrase? What if the user wants to reinstall the OS leaving the /home partition intact? What if a users LUKS header gets corrupted? As a result, I decided to add some extras and they become obvious upon first boot after installation. There are a number of files that get dumped on the installed users desktop.If a key file was created during system installation, this is saved to the installed user's desktop. This key file can be used in place of the passphrase for system decryption at boot time (or any other time). I won't document this process here as you can find various ways to do it documented elsewhere online. One such clever implementation can be found here.
To address the issue of changing the LUKS encryption passphrase and recover from a lost passphrase, my script dumps a script called Change-LUKS-passphrase.sh on the desktop. If a key file was not created during installation, this can only be used to change the LUKS passphrase using the current passphrase. If however, a key file was generated, it gets embedded in this script, allowing this script to be used for recovery in the event that the LUKS passphrase is forgotten.
There is also a file named Reinstallation.sh. This script can be run from a Live USB/DVD and helps decrypt the drive so that the existing partition structure and LUKS encryption can be reused with a new installation. This is useful for reinstalling while keeping the /home partition intact. When run, it provides the necessary prompts to guide the user through the reinstallation process.
While I was in the mood of trying to prevent utter disaster of the kind caused by forgotten passwords, I remembered that if the LUKS header gets corrupted, the consequences may be just as terrible (ie. no more possibility of decryption). Therefore, I configured my LGMP script to drop a copy of the LUKS header on the installed user's desktop as well so that it's not as easy to forget to save it on a separate backup medium.
To finish everything off, I created a LUKS-README.html file that goes into much further detail than this post about the LUKS implementation, which files need to be stored in a secure location, and how to use the scripts that are saved to the installed user's desktop.
hey, i'm glad to see that. actually I've been following the link you made on the question I asked on stackexchange. I'll definitly use that and this is not just to please you, I needed this, and I'm sure a lot of people do.
ReplyDeleteDo you have it on github, I could fork it and that'll make at least a few people more to be aware of that. I don't know which country you are from but if you'r from the east you'll be glad (i hope) to know that I felt inspired by the zhou yi and named a program after it. https://github.com/e2002e/zhou. Good day/night
I'm taking the opportunity of my failure to see that it was on github and coming to appologize to ask you one thing: the community working for jumbo-john have been proficient in making all king of hash available to john but there is the luks-header for which they never could find a solution, could you take the time to see about that or give me simple directive (appart form do it yourself). Thanks
ReplyDeleteI'm don't quite follow your question. What exactly are you asking about LUKS headers?
DeleteHi there,
ReplyDeleteIs is possible to alter this script to choose a specific partition on a disk that already contains data, so as not to wipe them, and dual boot OS's?
Certainly this is possible, but it's a little trickier to automate, and since I originally wrote this for my own benefit as a single-OS new installation script, I did not provide for that functionality. At the moment I do not have time to put into this, but feel free to submit a request on github, and if I have time to address it in the future, I may consider rewriting some code to make this possible. As for right now, you should be able to accomplish this with the steps I list in the link to my askUbuntu answer (https://askubuntu.com/a/918030/606758). While those steps are also geared toward a new install, if you're not too unfamiliar with the partitioning side of things, they could probably be easily adapted to a dual-boot environment.
DeleteWhen running the script I get the following error:
ReplyDeleteCreating 2G boot partition ... done
Creating 100M efi partition ... done
Creating -1MB lvm partition ... done
Setting up encryption:
Encrypting /dev/nvme0n13 with your passphrase ... Device /dev/nvme0n13 doesn't exist or access denied.
done
Adding key file as a decryption option for /dev/nvme0n13 ... Device /dev/nvme0n13 doesn't exist or access denied.
done
Device /dev/nvme0n13 doesn't exist or access denied.
Setting up LVM:
Creating 10 swap logical volume ... failed
Creating 50 root logical volume ... failed
(standard_in) 1: syntax error
Creating home logical volume ... failed
At this point, you should KEEP THIS WINDOW OPEN and start the installation
process. When you reach the "Installation type" page, select "Something else"
and continue to manual partition setup.
/dev/nvme0n11 should be used as ext2 for /boot
/dev/nvme0n12 should be used as EFI System Partition
/dev/mapper/vg0-home should be used as ext4 for /home
/dev/mapper/vg0-root should be used as ext4 for /
/dev/mapper/vg0-swap should be used as swap
/dev/nvme0n1 should be selected as the "Device for boot loader installation"
Not sure why it's referencing nvme0n13 etc when fdisk shows the following partitions were created:
Device Start End Sectors Size Type
/dev/nvme0n1p1 2048 4196351 4194304 2G Linux filesystem
/dev/nvme0n1p2 4196352 4401151 204800 100M Microsoft basic data
/dev/nvme0n1p3 4401152 500115455 495714304 236.4G Linux filesystem
Can this be fixed in the script?
Based on what I am seeing, yes, this can be fixed. It appears to be an issue with the naming system for NVMe SSD partitions. I do not have the time or a system with an NVMe drive that I can play with right now, but when I have time, I'll look into this more. I've added an issue for it in GitHub to see if anyone else might want to add a patch for NVMe support. Once a fix is in place, I'll try to remember to write back here.
DeleteI just committed the patch that adds NVMe support, but since I don't have a system to test on, I would be grateful if you would tell me how it works, should you decide to give it a go.
DeleteIt works!
DeleteHi there,
ReplyDeleteThanks for making such a useful script!
There's a couple of questions I wanted to ask, though. These aren't really complains and are more of a lack of understanding on my part.
1. Why do you ask for /boot to be set to 'ext2' as opposed to 'ext4'? I have read that ext2 does not support trim, and I would like to have /boot set to ext4 if at all possible. This leads me to my next question:
2. Why do you recommend that trim should not be used alongside LVM snapshots? Is there some shortcoming or danger to using trim in conjunction with LVM snapshots?
Thank you!
Hey, I'm glad others are benefiting from it! Regarding your questions:
Delete1) I recommend ext2 primarily because I am mirroring the behavior of the default installer as closely as possible. Can ext4 be used? Yes, so long as you are using Grub 2. I've read that older versions of grub would not support ext4. Ultimately, I have assumed that whoever configured Ubiquity to format /boot as ext2 must have had a (good?) reason for not opting for ext4.
2) I have not verified this, but there are reports that the trim command will not run on systems where LVM snapshots have been taken.
Any chance to see this script on Kubuntu with nvme support?
ReplyDeleteWorks great on Ubuntu 19.04 with nvme but Kubuntu gives error...
This was very likely affecting all flavors of Ubuntu!
DeleteIf you saw my previous comment before I deleted it, ignore it. I discovered that VirtualBox can emulate NVMe. I ran a trial install and discovered what I am assuming is the same error you were getting. It was complaining that the NVMe partition was available as LUKS was being configured. NVMe is apparently fast enough to create a race condition. Will the OS pick up on the newly created partition before the script tries to configure LUKS on that partition? In my virtual machine, it wasn't. I have updated the script so that LUKS configuration will wait until the partition is available to the OS. I'm testing my install now, but I don't foresee any further problem. If there is, I'll come back and edit this comment.
If this is not the same error that you were seeing, please submit an issue in GitHub with the error you're seeing (preferred) or comment back here with the error text so I can dig into it further.