Export/Import of NFS Shares Having Subdirectories as Mount Points

1. Introduction

Network filesystem (NFS) is a crucial tool in the Linux ecosystem that enables file sharing across different machines in a network. For us as system administrators, setting up NFS is a routine task. However, when we need to export and import NFS shares that contain subdirectories as mount points, things can get a bit tricky.

In this tutorial, we’ll explore configuring NFS exports and mounts to ensure that subdirectories are correctly handled and visible on client machines. Whether we’re managing a diskless client setup or need to share complex directory structures, we’ll discuss the necessary insights.

Let’s get started!

2. Understanding NFS Exports and Mount Points

Before we discuss the specifics of handling subdirectory mount points, let’s briefly recap the essential concepts we need to understand.

NFS allows us to share directories from a server to one or more clients over a network. On the server side, we refer to these directories as “exports”. Clients can then “mount” these exports, making the remote files and directories accessible as though they were part of the local filesystem.

Typically, exporting a simple directory via NFS is straightforward. We define the directory in the server’s /etc/exports file, specify the clients that can access it, and apply the configuration using the exportfs command.

However, when dealing with directories that contain subdirectories as mount points, such as directories mounted using the mount –bind command, we need extra care.

When a client mounts such an NFS export, it may not always see the contents of the subdirectories if they are themselves mounted directories. This behavior stems from the way NFS handles filesystem hierarchies and the visibility of nested mounts.

3. Scenario Setup

Suppose we have server A, which exports a directory /srv via NFS. Within /srv, there is a subdirectory /srv/foo that is a mount point for another directory on the same server, let’s say /bar/foo.

To achieve this setup, we create it using this mount command on the server:

$ mount --bind /bar/foo /srv/foo

Then, on Client B, this export (/srv) is mounted under /mnt/srv. The expectation is that when we list the contents of /mnt/srv/foo on the client, we should see the contents of /bar/foo from the server. However, what we might encounter instead is an empty directory at /mnt/srv/foo.

To better understand our situation, let’s see a visual structure:

Server A:                     Client B:
/                             /
├── srv/                      └── mnt/
│   └── foo/ ──┐                  └── srv/
│              │                      └── foo/ (expected to show
└── bar/       │                            contents of /bar/foo,
    └── foo/ ──┘                            but may appear empty)

On Server A, /srv/foo is bind-mounted to /bar/foo
Server A exports /srv via NFS
Client B mounts Server A's /srv to /mnt/srv

This situation can be frustrating, especially if the mounted subdirectory contains critical data that the client needs to access. The root cause lies in how NFS manages exports and mount points. To resolve this, we need to adjust the NFS server configuration to ensure that all subdirectories, including those that are mount points, are correctly exported and accessible to the client.

4. Exporting NFS Shares With Subdirectory Mount Points

A standard configuration often falls short when exporting NFS shares that include subdirectories as mount points. Without the proper setup, these subdirectories may not be visible to clients, leading to empty directories where critical data should be.

4.1. Basic Export Configuration

Let’s start with a basic export configuration to understand where it might go wrong.

Suppose we have the following in our /etc/exports file on server A:

/srv *(rw,no_subtree_check,nohide)

In this example, we export /srv to all clients using the asterisk (*) wildcard. The asterisk means that the export is available to any client machine on the network without restricting access to specific hosts or IP addresses.

Also, we use the nohide option, which is intended to allow clients to see the contents of mounted subdirectories like /srv/foo. However, this doesn’t always work as expected, particularly when dealing with complex directory structures or multiple subdirectories.

But a question arises here: Why isn’t nohide enough?

The nohide option seems like it should solve our problem by making the mount point visible, but there are scenarios where it doesn’t fully function as intended. The issue stems from how NFS handles filesystems within a shared directory.

Specifically, if /srv/foo is a separate mount point, NFS may still treat it as a distinct filesystem. This leads to issues with visibility from the client side.

4.2. Improving the Configuration With nohide and fsid

To address the issue of subdirectory visibility, one common workaround is to export both the parent directory and the subdirectory separately:

/srv        *(rw,no_subtree_check,fsid=0)
/srv/foo    *(rw,no_subtree_check,nohide)

Here, we use the fsid=0 option on the parent directory /srv to ensure it’s treated as the root of the exported filesystem. Then, we explicitly export the subdirectory /srv/foo with nohide.

However, while this setup may work in some cases, it’s not foolproof. The nohide option has limitations, particularly with complex or nested mounts. This is where the crossmnt option, which we’ll discuss next, comes into play, offering a more reliable and manageable solution.

5. Using the crossmnt Option

The crossmnt option is often the best approach to reliably exporting NFS shares with subdirectory mount points. It tells NFS to allow clients to traverse from one mounted filesystem to another seamlessly.

Specifically, the crossmnt option changes how NFS handles the transition between the exported directory and its subdirectories, which are mount points. When we set the crossmnt option on a directory, it effectively makes the contents of any mounted subdirectories visible to the client, just as if they were part of the same filesystem.

5.1. Sample Illustration

Let’s see how we can modify the /etc/exports file on server A to use the crossmnt option now:

/srv        *(rw,no_subtree_check,fsid=0,crossmnt)
/srv/foo    *(rw,no_subtree_check)

In this new configuration, we apply the crossmnt option to the parent directory /srv. Although we still export the subdirectory /srv/foo, it doesn’t need the nohide option. This is because crossmnt on /srv takes care of visibility.

After we finish editing the /etc/exports file, we’ll need to apply the changes with the exportfs command:

$ exportfs -rav
exporting *:/srv
exporting *:/srv/foo

Here, our output indicates the successful export of directories /srv and /srv/foo to all clients with the asterisk (*) wildcard, similar to our previous interaction with nohide.

Notably, for our options with exportfs:

• -r – tells exportfs to re-export all directories (it’s useful when we want to refresh the export list without rebooting the server because it re-reads the /etc/exports file and applies any changes made to it)
• -a – tells exportfs to export all directories listed in /etc/exports, and if there are multiple entries, all will be exported
• -v – provides detailed output, showing what directories we’re exporting and to which clients

In short, this command re-exports all directories listed in /etc/exports, ensuring the new configuration takes effect.

5.2. Mounting the NFS Export

To verify the setup on the client side, we can mount the NFS export as usual:

$ mount -t nfs server:/srv /mnt/srv

Here, we use the mount command to mount a remote NFS share on a local directory:

• -t nfs – specifies the type of filesystem to mount, which in this case is NFS (we need to tell the mount command that we’re mounting an NFS share)
• server:/srv – specifies the NFS export we want to mount (server is the hostname or IP address of the NFS server, and /srv is the path to the exported directory on the server)
• /mnt/srv – indicates the local mount point on the client system where the NFS share will be mounted, and once mounted, the client can access the remote files as if they were local, under /mnt/srv.

Now, when we list the contents of /mnt/srv/foo, we should see the contents of /bar/foo as expected without encountering the empty directory issue.

With the crossmnt option, we ensure that clients can move between the parent directory and its mounted subdirectories without losing the visibility of any data. This makes crossmnt a robust solution for more complex NFS setups.

6. Alternative Approaches

While using the crossmnt option is a reliable way to handle NFS shares with subdirectory mount points, it’s not the only method. Depending on our specific needs and environment, there are other possible approaches worth considering. Let’s explore some of these options.

6.1. Mounting Each Subdirectory Separately on the Client

One straightforward alternative we can consider is to mount each subdirectory individually on the client, instead of relying on the server’s NFS configuration to make subdirectories visible through a single mount.

For example, let’s say on client B, instead of just mounting /srv and expecting /srv/foo to be visible, we can consider this approach:

$ mount -t nfs server:/srv /mnt/srv
$ mount -t nfs server:/srv/foo /mnt/srv/foo

Our approach here has a few advantages:

• Simplicity: We avoid potential issues with crossmnt or nohide by handling each directory individually.
• Granular control: We gain more precise control over which directories are mounted and how they are accessed.

However, there are also some downsides we should note:

• Increased management overhead: Managing multiple mounts can become cumbersome, especially in large environments with many subdirectories.
• Potential for configuration errors: Each additional mount introduces a potential point of failure or misconfiguration.

In short, this approach can be particularly useful in simpler environments or when we only need to access a few subdirectories. Contrarily, it may not scale well for larger setups. It’s a trade-off between simplicity and manageability, where the choice depends on the specific needs of the environment.

6.2. Using Symbolic Links

Another alternative we can consider is to use symbolic links (symlinks) to connect the necessary directories.

On the NFS server, instead of binding directories with mount –bind, we could create symlinks:

$ ln -s /bar/foo /srv/foo

Here, ln with the -s option (symbolic) creates a symbolic link. With this setup, when the client mounts /srv, it will see /srv/foo as a symbolic link pointing to /bar/foo. The client can then access the contents of /bar/foo through /srv/foo without additional configuration.

Here are this method’s benefits that we should be aware of:

• Transparency: Symlinks are transparent to users and applications on the client.
• Reduced complexity: We don’t need to worry about the intricacies of NFS export options like crossmnt or nohide.

However, symlinks also come with their own set of challenges:

• Dependency on server path structure: The symlink points to an absolute path on the server, which can introduce issues if the server’s directory structure changes.
• Limited use cases: Symlinks may not be suitable for all environments, especially where strict path consistency is required.

Ultimately, these alternative approaches offer flexibility, allowing us to tailor the NFS setup to our specific needs. However, each comes with trade-offs we must carefully consider before implementation.

7. Diskless Clients

Diskless clients (which boot from a network rather than a local hard drive) are a common use case for NFS in enterprise settings. In a typical diskless setup, the client’s root filesystem is stored on the NFS server and accessed over the network during boot. This setup requires the NFS server to export the root filesystem in such a way that all necessary subdirectories and mount points are accessible to the client.

7.1. Configuring NFS for Diskless Clients

Let’s consider the following scenario. Server A hosts the root filesystem image for the diskless client. The image is mounted on /srv/des1, and the server needs to export this directory for the client to boot correctly.

To configure NFS for diskless clients, we’ll need to carefully set up the server’s exports. Using the crossmnt option can ensure that any subdirectory mount points within the root filesystem are correctly handled.

Let’s see an example /etc/exports configuration for this scenario:

/srv/      *(rw,no_root_squash,no_subtree_check,fsid=0,crossmnt)
/srv/des1  *(rw,no_root_squash,no_subtree_check)

In this setup, we export the parent directory /srv/ with crossmnt to ensure that all subdirectories, including the root filesystem at /srv/des1, are visible to the client. Similarly, we export the root filesystem itself (/srv/des1) to allow direct access.

After setting this up, we can apply the changes with exportfs -rav. Afterward, we proceed to configure the diskless client to mount the NFS root during boot.

7.2. Key Considerations and Pitfalls

When setting up NFS for diskless clients, there are some considerations we should keep in mind:

• Performance: Diskless clients rely heavily on network performance. We should ensure that our network infrastructure can handle the increased load, particularly during boot times.
• Permissions: If the client needs root access to the NFS share, we should ensure that we correctly apply the no_root_squash option.
• Multiple Clients: If we’re supporting multiple diskless clients, we need to consider how we’ll manage their unique configurations. This might involve setting up separate NFS exports or using boot-time scripts to adjust mounts dynamically.

With these considerations, we can effectively set up diskless clients that rely on NFS, ensuring a smooth boot process and reliable access to necessary files and directories.

8. Conclusion

In this article, we’ve explored the intricacies of exporting and importing such NFS shares. By carefully configuring our exports, whether using crossmnt, nohide, or the alternative methods we’ve discussed, we can ensure that our NFS shares function as expected, even in complex setups.