How to Autoinstall Ubuntu Linux on a Device Without a Screen or a Keyboard

1. Overview

In this tutorial, we’ll explore how to perform an unattended Ubuntu installation on devices that don’t have a screen or a keyboard. Unlike the network-based method, which focuses on large-scale deployments using netbooting and cloud-init, this guide offers an alternative approach.

We’ll create a reliable and flexible autoinstall.yaml configuration file and a new Ubuntu ISO that includes it. This allows unattended installation in a headless environment. We’ll also get the installer to send us a signal when the installation is complete.

We’ll illustrate this process using a Linux Mint 21 machine to create an autoinstaller ISO for Ubuntu Server 24.04.1 LTS. However, we can also automate the installation of other recent versions of Ubuntu Desktop or Ubuntu Server.

2. Use Cases for Autoinstall

Autoinstall is especially useful in scenarios that require automation and consistency. Here are a few common use cases:

• Cloud provider automation → Automates the deployment of pre-configured Ubuntu instances in cloud environments
• CI/CD pipelines → Integrates with CI/CD tools to automatically install and configure machines for testing or deployment
• Edge computing and IoT → Automates OS setup for headless devices in edge or IoT networks without manual intervention
• On-premise data centers → Ensures consistent server configuration across large data center deployments
• Disaster recovery systems → Quickly restores servers with predefined settings after system failures

Autoinstall simplifies deployment in automated infrastructures, both in the cloud and on-premises.

3. Compatibility and System Requirements

Our goal is to create an autoinstaller ISO that works on both BIOS/Legacy and UEFI machines.

While the steps we’ll follow are compatible with recent versions of Ubuntu Server and Desktop editions, it’s important to perform the autoinstaller creation on a machine running Ubuntu (or a derivative) that is as close as possible to the version and type of distribution we want to automate.

For example, since we want to automate the installation of Ubuntu Server 24.04.1 LTS, it’s best to use the same version or a closely related derivative on our development machine. In this case, we have found that Linux Mint 21, which is a derivative of Ubuntu Desktop 22.04 LTS, is fine.

This warning is crucial because the tools and scripts we use in this process, such as xorriso, grub-mkstandalone, grub-efi, and others, may have version-specific behaviors and dependencies. These utilities may differ in features, defaults, or file locations between Ubuntu versions. Using a system that’s closely matched to our target distribution minimizes the risk of compatibility issues such as boot failures or installation errors, and ensures that the autoinstaller ISO works as expected on our headless devices.

4. A Flexible autoinstall.yaml

The autoinstall.yaml file is at the heart of automating Ubuntu installations. Introduced with Ubuntu Server 20.04 and Ubuntu Desktop 23.04, it’s in the YAML format and allows us to specify everything from disk partitioning and network settings to user creation and package selection.

To keep things simple, it’s best to start with this minimal but comprehensive configuration and customize it later as needed:

autoinstall:
  version: 1
  identity:
    hostname: baeldung
    username: baeldung
    password: $6$EmUjCxkffcnA4b4k$GJ1wubOVaK45NSjEzos1hvecyLvfIsnhaI6dzJOHmKnFPYHo21qoTz2yin7rOqkFxueFhZCTvZGZybSWR6Tis/
    realname: Baeldung User
  ssh:
    install-server: true
    allow-pw: true
  storage:
    layout:
      name: direct
  locale: en_US.UTF-8
  keyboard:
    layout: us
  packages:
    - net-tools
    - p7zip-full
    - curl
  network:
    version: 2
    ethernets:
      all-eth:
        match:
          name: "*"
        dhcp4: true
  updates: security
  late-commands:
    - |
      curtin in-target -- /bin/bash -c 'IP=$(hostname -I | awk "{print \$1}"); curl -X GET "https://www.mydomain.com/signalCompletedInstallation.php?IP=$IP"'

Let’s break down the different sections:

• version → autoinstall configuration version
• identity → ensures that a user with sudo privileges is available after installation; passwords are hashed for security, in this case, the password is baeldung
• ssh → install-server: true ensures that the SSH server is installed, while allow-pw: true allows password-based login
• storage → defines the storage layout for the installation as direct, which uses the entire disk and creates a swap file instead of a dedicated partition
• locale → ensures the appropriate locale is applied by default
• keyboard → specifies the keyboard layout
• packages → lists any additional packages to be installed at the end of the setup
• network → uses DHCP to dynamically assign IP addresses to all available network interfaces, matching the device names with *
• updates → update type during installation: security = critical updates only
• late-commands → uses curtin to get the IP address and send a notification to a URL when installation is complete

The curtin in-target command ensures that the specified command, in this case /bin/bash, runs within the context of the newly installed target system, and not from the live installation environment. This is important because any command executed without curtin in-target may not reflect the state or configuration of the installed system.

This is the command for generating a password hash, in this case for the baeldung user:

$ python3 -c 'import crypt; print(crypt.crypt("baeldung", crypt.mksalt(crypt.METHOD_SHA512)))'

As for signalCompletedInstallation.php, we’ll discuss it later.

5. Creating Autoinstalling ISO

Creating a new ISO for autoinstallation requires precision, as the process involves several steps, each of which is critical to producing a fully functional ISO.

Building the ISO manually can be complex, and it’s easy to make mistakes, especially when updating boot files and performing checksum calculations. This well-tested script simplifies the task by ensuring that each step is handled automatically and efficiently. Let’s save it as create-autoinstall-iso.sh:

#!/bin/bash

# Script to create a custom Ubuntu autoinstall ISO for Ubuntu 24.04
# Assumes ubuntu-24.04.1-live-server-amd64.iso and autoinstall.yaml are in the current directory

# Exit immediately if a command exits with a non-zero status
set -e

# Variables
ISO_NAME="ubuntu-24.04.1-live-server-amd64.iso"
ISO_CUSTOM="$(pwd)/ubuntu-24.04.1-live-server-autoinstall.iso"
WORK_DIR="$(pwd)/ubuntu-autoinstall"
MOUNT_DIR="$WORK_DIR/mount"
ISO_DIR="$WORK_DIR/iso"
AUTOINSTALL_FILE="autoinstall.yaml"

# Check for required files
if [[ ! -f "$ISO_NAME" ]]; then
    echo "ISO file '$ISO_NAME' not found in the current directory."
    exit 1
fi

if [[ ! -f "$AUTOINSTALL_FILE" ]]; then
    echo "Autoinstall file '$AUTOINSTALL_FILE' not found in the current directory."
    exit 1
fi

# Install required packages
echo "Installing required packages..."
sudo apt-get update
sudo apt-get install -y xorriso grub-pc-bin grub-efi-amd64-bin grub-efi-ia32-bin mtools rsync

# Create working directories
echo "Creating working directories..."
mkdir -p "$MOUNT_DIR" "$ISO_DIR"

# Mount the original ISO
echo "Mounting the original ISO..."
sudo mount -o loop "$ISO_NAME" "$MOUNT_DIR"

# Copy the contents of the ISO to the working directory
echo "Copying ISO contents..."
sudo rsync -a "$MOUNT_DIR/" "$ISO_DIR/"

# Unmount the ISO
echo "Unmounting the original ISO..."
sudo umount "$MOUNT_DIR"

# Change ownership to current user for editing
echo "Adjusting permissions..."
sudo chown -R "$(whoami):$(whoami)" "$ISO_DIR/"
sudo chmod -R u+w "$ISO_DIR/"

# Copy autoinstall.yaml to the root of the ISO
echo "Adding autoinstall configuration..."
cp "$AUTOINSTALL_FILE" "$ISO_DIR/"

# Modify GRUB configuration for both BIOS and UEFI boot
echo "Modifying GRUB configuration..."
GRUB_CFG="$ISO_DIR/boot/grub/grub.cfg"
sudo tee "$GRUB_CFG" > /dev/null <<EOF
search --set=root --file /casper/vmlinuz
set default=0
set timeout=5

menuentry "Autoinstall Ubuntu Server" {
    linux /casper/vmlinuz autoinstall ---
    initrd /casper/initrd
}
EOF

# Create GRUB EFI image
echo "Creating GRUB EFI image..."
mkdir -p "$ISO_DIR/EFI/boot"
grub-mkstandalone \
    --format=x86_64-efi \
    --output="$ISO_DIR/EFI/boot/bootx64.efi" \
    --locales="" \
    --fonts="" \
    "boot/grub/grub.cfg=$GRUB_CFG"

# Create UEFI boot image
echo "Creating UEFI boot image..."
dd if=/dev/zero of="$ISO_DIR/boot/grub/efi.img" bs=1M count=10
mkfs.vfat "$ISO_DIR/boot/grub/efi.img"
mmd -i "$ISO_DIR/boot/grub/efi.img" efi efi/boot
mcopy -i "$ISO_DIR/boot/grub/efi.img" "$ISO_DIR/EFI/boot/bootx64.efi" ::efi/boot/

# Updating MD5 checksums
echo "Updating MD5 checksums..."
cd "$ISO_DIR"
sudo chmod u+w md5sum.txt
sudo rm md5sum.txt
sudo find . -type f ! -name 'md5sum.txt' ! -path './ubuntu/*' -exec md5sum {} \; | sudo tee md5sum.txt

# Create the new ISO image
echo "Creating the custom ISO..."
cd "$WORK_DIR"
xorriso -as mkisofs \
    -iso-level 3 \
    -o "$ISO_CUSTOM" \
    -full-iso9660-filenames \
    -volid "UBUNTU_24_04_1_SERVER_AMD64" \
    -eltorito-boot boot/grub/i386-pc/eltorito.img \
        -no-emul-boot \
        -boot-load-size 4 \
        -boot-info-table \
    -eltorito-alt-boot \
    -e boot/grub/efi.img \
        -no-emul-boot \
    -isohybrid-gpt-basdat \
    "$ISO_DIR"


# Clean up
echo "Cleaning up temporary files..."
sudo rm -rf "$WORK_DIR"

echo "Custom autoinstall ISO created successfully: $ISO_CUSTOM"

Here are the key steps performed by this script:

• Check for required files → Ensures the Ubuntu ISO and autoinstall.yaml are present
• Install required packages → Installs necessary tools like xorriso, grub, and rsync
• Create working directories → Sets up directories for mounting and modifying the ISO
• Mount the original ISO → Mounts the original Ubuntu ISO file
• Copy ISO contents → Copies the contents of the original ISO to a working directory
• Unmount the ISO → Unmounts the original ISO to proceed with modifications
• Adjust permissions → Changes ownership and permissions for editing files
• Add autoinstall configuration → Copies autoinstall.yaml into the ISO root
• Modify GRUB configuration → Updates GRUB settings for BIOS and UEFI boot
• Create GRUB EFI image → Generates a GRUB EFI image for UEFI systems
• Create UEFI boot image → Builds a bootable image for UEFI boot support
• Update MD5 checksums → Recalculates checksums for the modified files
• Create the custom ISO → Uses xorriso to generate the new autoinstall ISO
• Clean up temporary files → Removes working directories and temporary files

To run this script, let’s place it in the same directory as the Ubuntu ISO file and the autoinstall.yaml configuration file. Let’s make sure the script is executable, then run it from a terminal with ./create-autoinstall-iso.sh. The script creates a custom ISO whose installation process doesn’t require any input.

6. Running the Autoinstall

Now that our ubuntu-24.04.1-live-server-autoinstall.iso file is ready, we can use it in the cloud or locally. We just need to make sure that the target machine boots from this ISO, and then the installation starts automatically.

If we have a hypervisor like VirtualBox to monitor what happens before deployment in a truly headless environment, the beginning of the installation will look like this:

Based on the parameters in autoinstall.yaml, our target machine needs an active Internet connection with DHCP enabled to install the security updates.

6.1. Installation End Signal

The setup signals our https://www.mydomain.com/signalCompletedInstallation.php?IP=$IP when the installation is complete. Here is a minimal implementation of signalCompletedInstallation.php, just to have a log on the server running mydomain.com:

<?php
error_log("Installation completed for IP: $ip");
?>

If we’re running Apache on a Linux machine, we can easily monitor the Apache logs in real-time and see output similar to the following at the end of the autoinstall:

$ [email protected]
[...]
$ tail -f /var/log/apache2/error.log
[...]
[Sun Oct 27 ... 2024] [...] Installation completed for IP: 10.0.2.15

Of course, we can replace this example PHP with a REST endpoint tailored to our needs. However, there are two problems to consider:

• This way we won’t see the installer logs, which we’ll discuss shortly
• If the installation fails for some reason, we won’t get this end signal

If the installation is successful, the system automatically reboots and waits for us to log in via SSH.

6.2. Check the Autoinstaller Logs

If we’re using a headless environment and the installer doesn’t send the termination signal in a reasonable amount of time, we can suspect something has gone wrong. If our cloud provider supports VNC, it’s a good and easy way to visually monitor the installation. Otherwise, we can use SSH, but the baeldung user won’t be able to log in until the installation is complete.

If we’re using a virtualized test environment, we may have difficulty knowing the machine’s IP and how to expose it to the host. In the case of VirtualBox, we can use a bridged network adapter and find out the IP by running a ping scan in the host:

$ ifconfig
[...]
enxda8f7670beec: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 172.20.10.6  netmask 255.255.255.240  broadcast 172.20.10.15

$ nmap -sn 172.20.10.0/28
[...]
Nmap scan report for 172.20.10.3
Host is up (0.0023s latency).
[...]

Now we know the IP to connect to the machine. The installer leaves several log files:

$ ssh [email protected]
[...]
$ cd /var/log/installer
$ ls
autoinstall-user-data  media-info
block                  subiquity-client-debug.log
casper-md5check.json   subiquity-client-debug.log.1567
cloud-init-output.log  subiquity-client-info.log
cloud-init.log         subiquity-client-info.log.1567
curtin-install         subiquity-server-debug.log
curtin-install.log     subiquity-server-debug.log.1570
device-map.json        subiquity-server-info.log
installer-journal.txt  subiquity-server-info.log.1570

However, the entire installation process from beginning to end is in installer-journal.txt. It starts with the kernel parameters used to boot the installer:

$ sudo cat installer-journal.txt
Running as unit: run-u84.service
Oct 30 13:56:05 ubuntu-server kernel: Linux version 6.8.0-41-generic (buildd@lcy02-amd64-100) (x86_64-linux-gnu-gcc-13 (Ubuntu 13.2.0-23ubuntu4) 13.2.0, GNU ld (GNU Binutils for Ubuntu) 2.42) #41-Ubuntu SMP PREEMPT_DYNAMIC Fri Aug  2 20:41:06 UTC 2024 (Ubuntu 6.8.0-41.41-generic 6.8.12)
Oct 30 13:56:05 ubuntu-server kernel: Command line: BOOT_IMAGE=/casper/vmlinuz autoinstall ---
Oct 30 13:56:05 ubuntu-server kernel: KERNEL supported cpus:
Oct 30 13:56:05 ubuntu-server kernel:   Intel GenuineIntel
[...]

If we really needed to log in via SSH during the installation to monitor logs in real-time, we could use useradd in conjunction with chpasswd in the early-commands section of autoinstall.yaml to create a temporary user with SSH access.

7. Conclusion

In this article, we discussed the step-by-step process of creating an unattended Ubuntu installation ISO for devices without a display or keyboard. By creating a custom autoinstall.yaml file and including it in an Ubuntu ISO, we enable seamless installation in headless environments, ensuring automated configuration and setup without user intervention.

We detailed how to create an autoinstaller ISO and make it compatible with both BIOS/Legacy and UEFI systems. Following these instructions, we can deploy Ubuntu Server or Desktop versions in different environments, increasing deployment reliability and reducing setup time.

In addition, we ensured that a signal confirmed successful installation, streamlining the deployment process for various IT and development scenarios.