Oracle Free Tier: Install Debian on the x86 micro instances with full disk encryption

Oracle Cloud offers AMD VPSes with 0,1 virtual CPU core and 1GB RAM for free through their Oracle Free Tier program. Unfortunately, Debian is not part of their standard range of OSes available to install. For the free ARM nodes Oracle offers, reinstalling is very simple through the use of netboot.xyz and selecting a netinstall image. Unfortunately, reinstalling their x86 instances is a bit more cumbersome.

This document shows how to do an in-place install of Debian 12 with full disk encryption and remote unlocking on an instance after first choosing the Ubuntu 22.04 minimal image in the Oracle Cloud console. As root file system, a choice is given between ext4 and btrfs.

Installing Alpine Linux

In order to reinstall the instance, we will need to minimise the amount of resources the system uses. By installing Alpine Linux and then using that to install Debian, we get a very minimal temporary OS using less than 100 MB of RAM and can be moved from the boot volume into RAM while continuing to run stable.

First, SSH into the Ubuntu instance you deployed on Oracle Cloud.

ssh ubuntu@REPLACE_WITH_YOUR_INSTANCES_PUBLIC_IP_ADDRESS

Change to the root user:

sudo -i

Then, execute the following commands to replace Ubuntu with Alpine Linux:

cd /
wget https://dl-cdn.alpinelinux.org/alpine/v3.18/releases/x86_64/alpine-virt-3.18.4-x86_64.iso
dd if=alpine-virt-3.18.4-x86_64.iso of=/dev/sda
sync
reboot

The system will now reboot into an unconfigured Alpine Linux install.

Configure Alpine Linux

After the system is rebooted, you’ll lose SSH access. Therefore, use the Oracle Cloud Console to access the Alpine Linux VM. Deploy the cloud console by going to Compute -> Instances -> Instance details (by clicking on the instance you want to install). Scroll down and click ‘Console connection’ -> ‘Launch Cloud Shell connection’.

A new connection will deploy. Hit enter to get a login prompt after the console prints Instance Console Connection reached state: ACTIVE

On the resulting command prompt (localhost login:), login using username root and without a password.

Next, configure networking:

vi /etc/network/interfaces

Press i to start editing. The Oracle Cloud shell supports copy pasting.

auto eth0
iface eth0 inet dhcp

Press escape and type :wq to save and exit.

Restart networking

/etc/init.d/networking restart

Setup SSH using the Alpine SSH wizard.

setup-sshd
Which ssh server? ('openssh', 'dropbear' or 'none') [openssh] 
Allow root ssh login? ('?' for help) [prohibit-password] 
Enter ssh key or URL for root (or 'none') [none] 
 * service sshd added to runlevel default
 * Caching service dependencies ...
 [ ok ]
ssh-keygen: generating new host keys: RSA ECDSA ED25519 
 * Starting sshd ...
 [ ok ]

Add your SSH public key

mkdir .ssh
vi .ssh/authorized_keys

Paste in your SSH public key and save.

Remove Alpine from the boot volume

SSH into the Alpine Linux install. Remove old IP and matching fingerprint lines from your .ssh/known_hosts if you get any errors and retry.

ssh root@REPLACE_WITH_YOUR_INSTANCES_PUBLIC_IP_ADDRESS

Move Alpine into RAM and unmount the boot volume by executing the following commands line by line.

mkdir /media/setup
cp -a /media/sda/* /media/setup
mkdir /lib/setup
cp -a /.modloop/* /lib/setup
/etc/init.d/modloop stop
umount /dev/sda
mv /media/setup/* /media/sda/
mv /lib/setup/* /.modloop/

Setup APK by entering the following command and accepting the default settings.

setup-apkrepos

Install Alpine packages

apk update
apk add dosfstools e2fsprogs debootstrap cryptsetup nano gptfdisk partx btrfs-progs
modprobe btrfs

Partition the disk

gdisk /dev/sda

Press 3 to create a new GPT partition table. Press n to create the boot partition. Make it 1024MB by answering the wizard as detailed below.

Command (? for help): n
Partition number (1-128, default 1): 
First sector (34-104857566, default = 2048) or {+-}size{KMGTP}: 
Last sector (2048-104857566, default = 104855551) or {+-}size{KMGTP}: +1024M
Current type is 8300 (Linux filesystem)
Hex code or GUID (L to show codes, Enter = 8300): 
Changed type of partition to 'Linux filesystem'

Press n to create the EFI partition. Make it 512MB and set it to EFI.

Command (? for help): n
Partition number (2-128, default 2): 
First sector (34-104857566, default = 2099200) or {+-}size{KMGTP}: 
Last sector (2099200-104857566, default = 104855551) or {+-}size{KMGTP}: +512M
Current type is 8300 (Linux filesystem)
Hex code or GUID (L to show codes, Enter = 8300): EF00
Changed type of partition to 'EFI system partition'

Press n to create the root file system partition. Make it use all available remaining disk space.

Command (? for help): n
Partition number (3-128, default 3): 
First sector (34-104857566, default = 3147776) or {+-}size{KMGTP}: 
Last sector (3147776-104857566, default = 104855551) or {+-}size{KMGTP}: 
Current type is 8300 (Linux filesystem)
Hex code or GUID (L to show codes, Enter = 8300): 
Changed type of partition to 'Linux filesystem'

Press p to display the partition table. It should give the following result:

ommand (? for help): p
Disk /dev/sda: 104857600 sectors, 50.0 GiB
Model: BlockVolume     
Sector size (logical/physical): 512/4096 bytes
Disk identifier (GUID): 57C983BB-7A42-4B74-BD69-F9A0AEFD5199
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 104857566
Partitions will be aligned on 2048-sector boundaries
Total free space is 4029 sectors (2.0 MiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048         2099199   1024.0 MiB  8300  Linux filesystem
   2         2099200         3147775   512.0 MiB   EF00  EFI system partition
   3         3147776       104855551   48.5 GiB    8300  Linux filesystem

If this is not the case or if you’d like to make changes to this setup, start over by pressing o to recreate the partition table. Otherwise, press w and y to exit and save your changes.

Formatting the new partitions

Remove the old partition table

partx -v -d /dev/sda
partition: none, disk: /dev/sda, lower: 0, upper: 0
/dev/sda: partition #1 removed
/dev/sda: partition #2 removed
/dev/sda: partition #3 removed

Add the new partition table

partx -v -a /dev/sda
partition: none, disk: /dev/sda, lower: 0, upper: 0
/dev/sda: partition table type 'gpt' detected
range recount: max partno=3, lower=0, upper=0
/dev/sda: partition #1 added
/dev/sda: partition #2 added
/dev/sda: partition #3 added

Format the boot and EFI partitions

mkfs.ext4 /dev/sda1
mkfs.vfat /dev/sda2

Encrypt the root partition using a strong password.

cryptsetup luksFormat -c aes-xts-plain64 -s 512 -h sha256 --pbkdf argon2i /dev/sda3

Unlock the newly created encrypted partition

cryptsetup luksOpen /dev/sda3 luks1

Format root filesystem

Depending on your preference, you can use ext4 or btrfs, or your own choice as the main file system. The first two are options I’ve personally tried. Both work well. Choose one.

Option A: Format and mount the root partition (ext4)

mkfs.ext4 /dev/mapper/luks1

mount -t ext4 /dev/mapper/luks1 /mnt

Option B: Format and mount the root partition (btrfs)

Format the root partition (btrfs)

mkfs.btrfs /dev/mapper/luks1

mount /dev/mapper/luks1 /mnt

btrfs subvolume create /mnt/@rootfs
btrfs subvolume create /mnt/@swap

umount /mnt

mount -t btrfs -o defaults,subvol=@rootfs /dev/mapper/luks1 /mnt

Add the /boot partition as a mount

mkdir /mnt/boot
mount -t ext4 /dev/sda1 /mnt/boot

And finally, the EFI partition

mkdir /mnt/boot/efi
mount -t vfat /dev/sda2 /mnt/boot/efi

We are now ready to start debootstrapping Debian.

Debootstrapping Debian

mkdir -p /mnt/var/tmp
chmod 1777 /mnt/var/tmp
debootstrap bookworm /mnt

This step will take a while and it may sometimes even seem like it is hanging, especially when resolving dependencies and unpacking the base system. For me, the debootstrap command took around 15 minutes.

Configuring hostname

echo YOUR_PREFERRED_HOSTNAME > /mnt/etc/hostname

cat <<EOF >> /mnt/etc/hosts
127.0.0.1 YOUR_PREFERRED_HOSTNAME.YOUR_DOMAIN YOUR_PREFERRED_HOSTNAME
::1 YOUR_PREFERRED_HOSTNAME.YOUR_DOMAIN YOUR_PREFERRED_HOSTNAME
EOF

Configuring networking

Choose one of these two:

Option A: ifupdown

cat <<EOF >> /mnt/etc/network/interfaces

auto lo
iface lo inet loopback

auto ens3
iface ens3 inet dhcp
EOF

Option B: systemd-networkd

rm /mnt/etc/network/interfaces

cat <<EOF >> /mnt/etc/systemd/network/ens3.network

[Match]
Name=ens3

[Network]
DHCP=yes
EOF

Configuring apt

cat  <<EOF > /mnt/etc/apt/sources.list 
deb http://ftp.nl.debian.org/debian/ bookworm main contrib non-free non-free-firmware
deb-src http://ftp.nl.debian.org/debian/ bookworm main contrib non-free non-free-firmware
deb http://security.debian.org/ bookworm-security main contrib non-free non-free-firmware
deb-src http://security.debian.org/ bookworm-security main contrib non-free non-free-firmware

# bookworm-updates, previously known as 'volatile'
deb http://ftp.nl.debian.org/debian/ bookworm-updates main contrib non-free non-free-firmware
deb-src http://ftp.nl.debian.org/debian/ bookworm-updates main contrib non-free non-free-firmware

# bookworm-backports, previously on backports.debian.org
deb http://ftp.nl.debian.org/debian/ bookworm-backports main contrib non-free non-free-firmware
deb-src http://ftp.nl.debian.org/debian/ bookworm-backports main contrib non-free non-free-firmware
EOF

Enter chroot

Create system bind mounts

mount --rbind /dev  /mnt/dev
mount --rbind /proc /mnt/proc
mount --rbind /sys  /mnt/sys

enter Debian

LANG=C chroot /mnt /bin/bash
apt update
apt dist-upgrade -y
apt install locales -y
dpkg-reconfigure locales

Select en_US.UTF-8 and possible other languages and set a default language.

dpkg-reconfigure tzdata

Choose your timezone

Enabling systemd-networkd if desired

If you chose to use systemd-networkd above, enable it here, otherwise don’t

systemctl enable systemd-networkd

Setting up mounts

Let’s first include the description you would see when installing Debian the conventional way.

cat <<EOF > /etc/fstab
# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
EOF

and add the /, /boot and /boot/efi mount points

cat <<EOF >> /etc/fstab

# / partition
EOF

Option A: Ext4 root partition


echo UUID=$(blkid -s UUID -o value \
      /dev/mapper/luks1) \
      / ext4 defaults 0 2 >> /etc/fstab

Option B: BTRFS root partition

echo UUID=$(blkid -s UUID -o value \
      /dev/mapper/luks1) \
      / btrfs defaults,subvol=@rootfs 0 2 >> /etc/fstab

/boot and /boot/efi partition

cat <<EOF >> /etc/fstab

# /boot partition
EOF

echo UUID=$(blkid -s UUID -o value \
      /dev/sda1) \
      /boot ext4 defaults 0 2 >> /etc/fstab

cat <<EOF >> /etc/fstab

# UEFI /boot/efi partition (fat32)
EOF
echo PARTUUID=$(blkid -s PARTUUID -o value \
      /dev/sda3) \
      /boot/efi vfat nofail,x-systemd.device-timeout=1 0 1 >> /etc/fstab

Crypttab

echo luks1 UUID=$(blkid -s UUID -o value \
      /dev/sda3) none \
      luks,discard,initramfs >> /etc/crypttab

cat <<EOF >> /etc/udev/rules.d/99-local-crypt.rules
ENV{DM_NAME}!="", SYMLINK+="$env{DM_NAME}"
ENV{DM_NAME}!="", SYMLINK+="dm-name-$env{DM_NAME}"
EOF

Create a swap file

The Oracle free tier micro instances only have 1GB of RAM. To prevent out-of-memory errors, I created a 2GB swap file.

Option A: Create a swapfile on ext4

fallocate -l 2G /swapfile
chmod 600 /swapfile
mkswap -f /swapfile

cat <<EOF >> /etc/fstab

# Swapfile
/swapfile none swap sw 0 0
EOF

swapon -av      

Option B: Create a swapfile on btrfs

apt install btrfs-progs

First, we mount the earlier created @swap subvolume.

mkdir /swap

cat <<EOF >> /etc/fstab

# /swap subvolume
EOF

echo UUID=$(blkid -s UUID -o value \
      /dev/mapper/luks1) \
      /swap btrfs defaults,subvol=@swap 0 2 >> /etc/fstab

mount -o defaults,subvol=@swap /dev/mapper/luks1 /swap
btrfs filesystem mkswapfile --size 2G /swap/swapfile
cat <<EOF >> /etc/fstab

# Swapfile
/swap/swapfile none swap sw 0 0
EOF

swapon -av

Install GRUB bootloader

apt install -y grub-efi
grub-install --target=x86_64-efi --efi-directory=/boot/efi \
      --bootloader-id=debian --recheck --no-floppy

Enable SSD Trim support

sed -i "s/GRUB_CMDLINE_LINUX_DEFAULT=\"quiet\"/GRUB_CMDLINE_LINUX_DEFAULT=\"rd.luks.options=discard\"/g" /etc/default/grub

Enable the serial console, in order to be able to continue using the Oracle cloud shell

cat <<EOF >> /etc/default/grub
GRUB_TERMINAL_INPUT="console serial"
GRUB_TERMINAL_OUTPUT="gfxterm serial"
GRUB_SERIAL_COMMAND="serial --unit=0 --speed=115200"
GRUB_CMDLINE_LINUX_DEFAULT="\${GRUB_CMDLINE_LINUX_DEFAULT} console=tty0 console=ttyS0,115200"
EOF
update-grub2

Install Linux kernel and basic system packages

apt install -y linux-image-amd64
apt install -y cryptsetup ssh nano sudo man fail2ban screen ntp pollinate haveged wget curl net-tools git pwgen gpg build-essential dosfstools btrfs-progs qemu-guest-agent

Fix Fail2ban SSH error

sed -i "s/backend = auto/backend = systemd/g" /etc/fail2ban/jail.conf

Harden SSH and prevent language setting errors

sed -i "s/#PasswordAuthentication yes/PasswordAuthentication no/g" /etc/ssh/sshd_config
sed -i "s/#PermitRootLogin prohibit-password/PermitRootLogin no/g" /etc/ssh/sshd_config
sed -i "s/AcceptEnv LANG LC_*/#AcceptEnv LANG LC_*/g" /etc/ssh/sshd_config

Add an admin user

adduser management

Give sudo privileges

adduser management sudo

Become the management user.

su - management

Add your public SSH keys

mkdir ~/.ssh
nano ~/.ssh/authorized_keys

exit

Setup remote unlocking

apt install dropbear-initramfs initramfs-tools -y
nano /etc/dropbear/initramfs/dropbear.conf

Adjust the following settings

DROPBEAR_OPTIONS="-I 180 -j -k -p 2222 -s -c cryptroot-unlock"
IFDOWN=*
DEVICE=ens3
DROPBEAR=y

Also make sure to open port 2222 in the VCN security list in the Oracle cloud console.

nano /etc/dropbear/initramfs/authorized_keys

Add your SSH public key in here. You will now be able to unlock the encrypted root disk at boot by going to root@REPLACE_WITH_YOUR_INSTANCES_PUBLIC_IP_ADDRESS

Update the initramfs to add these changes and update grub just to be sure:

update-initramfs -u -k all
update-grub2

Cleanup

apt --purge autoremove

Rebooting into Debian

exit the chroot environment

exit

reboot

reboot

Fixing known_hosts errors

You should now be able to see Debian booting in the Oracle Cloud console. Unlocking the root file system is possible through the console or by creating an SSH connection to root@REPLACE_WITH_YOUR_INSTANCES_PUBLIC_IP_ADDRESS. You may be getting known_hosts errors every time you use this. To mitigate that, go through the following steps:

  1. Add the setting UpdateHostKeys no to your ~/.ssh/config entry for the instance.
  2. Delete all existing IP address with fingerprints of this instance from your ~/.ssh/known_hosts.
  3. Boot the instance and connect to root@REPLACE_WITH_YOUR_INSTANCES_PUBLIC_IP_ADDRESS to unlock the disk and choose yes to add the fingerprint.
  4. Open ~/.ssh/known_hosts and comment the new entry with #
  5. Next, connect to the booted machine with management@REPLACE_WITH_YOUR_INSTANCES_PUBLIC_IP_ADDRESS and choose yes to add the fingerprint.
  6. Open ~/.ssh/known_hosts again and uncomment the previously commented entry.

Example ~/.ssh/config entry, which allows unlocking using ssh oracle-unlock and logging in using ssh oracle:

Host oracle-unlock
    Hostname 10.0.0.10
    User root  
    Port 22
    IdentityFile ~/.ssh/id_ed25519
    UpdateHostKeys no

Host oracle
    Hostname 10.0.0.10
    User management  
    Port 22
    IdentityFile ~/.ssh/id_ed25519
    UpdateHostKeys no

Conclusion

If you enjoyed reading this post and/or have suggestions for improvement, please let me know by leaving a comment down below or contacting me through the contact form.

As a final word of caution, be careful with how much you rely on freely offered services and take proper precautions wrt backups and recovery in case these offerings change.

Next steps could be to disable the sudo password prompt or to install zsh.

If you used BTRFS as your file system, now, with a clean installation, might be a good time to create your first snapshot. That way, you can repurpose this VM as often as you like without having to start from scratch.

sudo mkdir /.snapshots
sudo chmod 750 /.snapshots
sudo btrfs subvolume snapshot / /.snapshots/2023-11-10-base-install

References