How to Avoid Conflicts Between dnsmasq and systemd-resolved

1. Introduction

When setting up a local DNS server with dnsmasq, we might encounter a common conflict with systemd-resolved. Both services attempt to bind to port 53, the default DNS port, causing one or both to fail. This issue can be frustrating, especially if the conflict reappears after a reboot.

In this tutorial, we’ll explore why this conflict happens and provide practical solutions to resolve it. Whether we prefer to turn off the conflicting behavior, configure both services to work side-by-side, or simply adjust their startup behavior, we’ll cover a range of approaches. Let’s get started!

2. Understanding dnsmasq and systemd-resolved

At its core, the conflict arises because both dnsmasq and systemd-resolved attempt to use port 53 to provide DNS resolution. But first, let’s briefly recap the roles of each service:

• dnsmasq: A lightweight DNS and DHCP server commonly used for local networks. By default, it binds to all interfaces and listens on port 53.
• systemd-resolved: A service provided by systemd to handle name resolution for the system. It includes a DNS stub resolver that listens on port 53 (specifically at 127.0.0.53).

When both services are active, they try to bind to port 53 simultaneously, resulting in an error for one of them – usually dnsmasq. This issue is especially noticeable on distributions like Ubuntu, where systemd-resolved is enabled by default.

Now, the problem is tied to how the underlying design and operating mechanism of systemd-resolved. Its DNS stub listener is meant to provide a local caching layer for DNS queries.

On the other hand, dnsmasq is often used as the primary DNS server for local network devices. Because neither service is aware of the other by default, they end up conflicting over the same port.

3. Checking System’s Configuration

Before applying any fixes, first and foremost, we need to gather some basic information about the services and their current behavior on our system. This will help us diagnose the conflict and choose the appropriate solution. Let’s walk through each process.

3.1. Identifying Active Services

First, let’s check whether both dnsmasq and systemd-resolved are running on our system with systemctl:

$ systemctl list-units --type=service | grep -E 'dnsmasq|resolved'
dnsmasq.service                  loaded active running   dnsmasq - A lightweight DHCP and caching DNS server
systemd-resolved.service         loaded active running   Network Name Resolution

The output should indicate the status of both services. If both are active, as seen in our example, the conflict is likely present.

3.2. Checking Port Usage

To confirm which service is using port 53, we can use netstat or ss.

Let’s see an example with netstat:

$ sudo netstat -tulpn | grep :53
tcp        0      0 127.0.0.1:53          0.0.0.0:*        LISTEN      1234/dnsmasq
tcp        0      0 127.0.0.53:53         0.0.0.0:*        LISTEN      5678/systemd-resol

The output will show which process is bound to port 53.

In this example, which is a typical conflict scenario, dnsmasq and systemd-resolved are both trying to use port 53 on different IP addresses, which will lead to issues.

3.3. Determining systemd Version

Finally, as part of our prerequisite checking process, we should check our systemd version to ensure compatibility with specific fixes:

$ systemctl --version
systemd 247 (247.3-3ubuntu3.11)

In this case, the systemd version is 247, which supports the DNSStubListener=no option. If the version is older than 232, we may need to rely on alternate methods, such as adjusting service dependencies.

Ultimately, these preliminary steps will help us clearly understand the conflict and the tools needed to address it.

4. Disabling systemd-resolved’s DNS Stub Listener

When running both dnsmasq and systemd-resolved, one frequent source of conflict is the DNS stub listener in systemd-resolved. One of the simplest ways to resolve the conflict is to disable the DNS stub listener in systemd-resolved. But what exactly is the DNS stub listener?

4.1. The DNS Stub Listener

The DNS stub listener is a lightweight DNS server built into systemd-resolved. By default, it listens for DNS queries on port 53 (the standard port for DNS) at the local IP address 127.0.0.53. This allows systemd-resolved to act as a DNS resolver for applications on our system by forwarding queries to upstream DNS servers or handling cached responses.

However, while the stub listener is convenient for some setups, it can create a major conflict when we want to run dnsmasq. Since both services attempt to bind to port 53, they clash, causing DNS resolution to fail. To prevent this issue, we can now turn off the DNS stub listener in systemd-resolved, effectively freeing up port 53 for dnsmasq.

4.2. Modifying resolved.conf

To turn off the DNS stub listener, we’ll need to edit the resolved.conf configuration file, /etc/systemd/resolved.conf. This file contains options that control the behavior of systemd-resolved. If it doesn’t already exist or is empty, we can create it.

Now, let’s open the file in a text editor (like the vi editor) and locate or add the [Resolve] section:

$ vi /etc/systemd/resolved.conf

# Editing the /etc/systemd/resolved.conf File
...

[Resolve]
DNSStubListener=no

Under this section, we add or update the DNSStubListener=no setting to turn off the stub listener. We should remember that the [Resolve] section header is mandatory; the configuration won’t be applied without it.

Now, systemd-resolved no longer listens for DNS queries on port 53.

After making the changes, we should save the file and restart systemd-resolved to apply the new settings:

$ systemctl restart systemd-resolved

Notably, even after turning off the stub listener and restarting the system, systemd-resolved continues to manage DNS resolution. It will now use the DNS servers configured globally (for all network interfaces) or per-link (for specific network interfaces). These configurations are typically set via:

• The NetworkManager
• systemd-networkd
• Manual entries in /etc/systemd/resolved.conf under the DNS= and FallbackDNS= options

Applications on the system bypass the stub listener and query these upstream DNS servers directly. This solution is particularly useful in hybrid setups where we want both services to coexist without completely turning off systemd-resolved.

5. Binding dnsmasq to Specific Interfaces

If turning off the DNS stub listener is not an option, perhaps because we need systemd-resolved for other purposes, we can configure dnsmasq to avoid conflicts by changing how it binds to interfaces and IP addresses.

To do this, we can update the dnsmasq configuration to explicitly bind it to specific interfaces or addresses.

Let’s now edit the dnsmasq configuration file, /etc/dnsmasq.conf, or create it if it doesn’t exist:

$ vi /etc/dnsmasq.conf

# Editing the /etc/dnsmasq.conf File
...

bind-interfaces
listen-address=127.0.0.1

Here, bind-interfaces ensures that dnsmasq only binds to the specified interfaces. Then, listen-address=127.0.0.1 restricts dnsmasq to listening on the localhost interface. If necessary, we can add additional IP addresses or interfaces.

After making these changes, we should restart the dnsmasq service:

$ systemctl restart dnsmasq

With this configuration, dnsmasq and systemd-resolved can operate on the same system without conflict, as they no longer attempt to bind to the same addresses.

6. Managing Boot-Time Conflicts

Even after turning off the DNS stub listener or configuring dnsmasq, we might encounter conflicts at boot if the services start in the wrong order. Let’s see how to adjust service dependencies and startup behavior to avoid such issues.

6.1. Adjusting Service Start Order

To ensure that dnsmasq starts before systemd-resolved, we modify the systemd unit file for dnsmasq.

First, let’s open the dnsmasq service file:

$ systemctl edit dnsmasq

In the file, we can now add the configuration:

# Editing the dnsmasq service file
...

[Unit]
Before=systemd-resolved.service

Afterward, we save the file, reload the systemd daemon, and restart the dnsmasq service:

$ systemctl daemon-reload
$ systemctl restart dnsmasq

Reloading the systemd daemon and restarting dnsmasq ensures that the updated service configuration takes effect.

6.2. Using Drop-In Configurations

As an alternative to modifying the primary service file, we can use a drop-in configuration for dnsmasq to achieve the same effect.

First, we create a directory for the drop-in file if it doesn’t already exist:

$ mkdir -p /etc/systemd/system/dnsmasq.service.d/

Then, we create a drop-in configuration file, for example, resolved-fix.conf:

$ vi /etc/systemd/system/dnsmasq.service.d/resolved-fix.conf

Next, we can add the configuration to specify the service dependencies:

# Editing the drop-in configuration file
...

[Unit]
Before=systemd-resolved.service

Lastly, we should reload the systemd configuration and restart the services:

$ systemctl daemon-reload
$ systemctl restart dnsmasq
$ systemctl restart systemd-resolved

With these processes, we ensure that dnsmasq starts before systemd-resolved, and prevent conflicts during the boot process.

7. Allowing dnsmasq and systemd-resolved to Coexist

If we want both dnsmasq and systemd-resolved to work together, we can configure them to complement each other rather than conflict. This involves redirecting systemd-resolved to use dnsmasq as its upstream DNS resolver or carefully configuring both services to manage their responsibilities without overlapping.

7.1. Redirecting systemd-resolved to Use dnsmasq

A common approach to allow the two services to coexist is configuring systemd-resolved to forward all DNS queries to dnsmasq. This ensures that systemd-resolved still operates as the system’s local DNS resolver but forwards queries to dnsmasq for actual resolution.

To start, we need to modify /etc/resolv.conf file. By default, systemd-resolved manages /etc/resolv.conf as a symbolic link to /run/systemd/resolve/stub-resolv.conf. However, we’ll need to modify /etc/resolv.conf to direct DNS queries to dnsmasq.

To do this, first, we remove the existing symbolic link:

$ rm /etc/resolv.conf

Then, we create a new /etc/resolv.conf file, and add the primary DNS resolver:

$ vi /etc/resolv.conf

Editing the /etc/resolv.conf file
...

nameserver 127.0.0.1

This tells the system to use dnsmasq (listening on 127.0.0.1) as the primary DNS resolver.

Next, we need to prevent overwrites by NetworkManager. If we’re using NetworkManager, it may overwrite /etc/resolv.conf during boot. To prevent this, we can create a configuration file to turn off DNS management by NetworkManager.

To do this, we open or create a new file at /etc/NetworkManager/conf.d/disableresolv.conf:

$ vi /etc/NetworkManager/conf.d/disableresolv.conf

# Editing the /etc/NetworkManager/conf.d/disableresolv.conf file
...

[main]
dns=none

Afterward, we restart NetworkManager to apply the changes:

$ systemctl restart NetworkManager

Now, NetworkManager will no longer interfere with /etc/resolv.conf.

Finally, we can now restart the necessary services:

$ systemctl restart dnsmasq
$ systemctl restart systemd-resolved

With this, we restart dnsmasq and systemd-resolved to ensure they use the latest configurations.

To complete the coexistence setup, we need to configure dnsmasq to use external upstream DNS servers for queries it cannot resolve locally. This is especially useful if dnsmasq is configured to handle only local domains. To do this, there are two options.

7.2. Managing Upstream DNS Servers

Our first option is to create a custom resolv file (for example, /etc/dnsmasq-resolv.conf) with our upstream DNS servers:

$ vi /etc/dnsmasq-resolv.conf

# Editing the /etc/dnsmasq-resolv.conf file
...
nameserver 8.8.8.8
nameserver 8.8.4.4

Next, we can now update the dnsmasq configuration, /etc/dnsmasq.conf, to use this resolv.conf file:

$ vi /etc/dnsmasq.conf

# Editing the /etc/dnsmasq.conf file
...

resolv-file=/etc/dnsmasq-resolv.conf

Now, we can restart dnsmasq with systemctl restart dnsmasq.

7.3. Disabling resolv.conf in dnsmasq

Our second option is to turn off dnsmasq’s reliance on /etc/resolv.conf entirely and directly specify upstream servers in its configuration.

To do this, first, we open /etc/dnsmasq.conf and add the upstream servers:

$ vi /etc/dnsmasq.conf

# Editing the /etc/dnsmasq.conf file
...

no-resolv
server=8.8.8.8
server=8.8.4.4

Finally, we restart dnsmasq with systemctl restart dnsmasq.

In short, we can allow both services to coexist by redirecting systemd-resolved to use dnsmasq and carefully managing upstream DNS servers. This setup ensures that dnsmasq handles DNS resolution while systemd-resolved continues to provide system-wide name resolution capabilities.

8. Conclusion

In this article, we explored the root cause of the conflict between dnsmasq and systemd-resolved, both of which attempt to bind to port 53 for DNS services. We walked through practical solutions, including turning off the DNS stub listener in systemd-resolved, configuring dnsmasq to bind to specific interfaces, and adjusting service startup behavior to prevent conflicts.

However, if we want both services to coexist, we covered how to redirect systemd-resolved to use dnsmasq as its upstream DNS resolver and manage upstream servers effectively.