Authors Top

If you have a few years of experience in the Linux ecosystem, and you’re interested in sharing that experience with the community, have a look at our Contribution Guidelines.

1. Overview

In this article, we’ll see how to check if a file write is in progress in Linux. Knowing if a file writing operation is completed depends on each application. Thus, we’ll focus on two ways we can infer this – checking if the file is still open by a process and directly inspecting the file.

2. Is There a Process With the File Opened?

A writing operation may be in progress if any process has opened a certain file. However, even if a file is open, we still cannot be sure that the writing operation has been completed.

As a general guideline, if we’re concerned about the completion of a writing operation, it’s always better to use an advisory lock. If we can modify the program that’s writing the file, it’s better to have it state when the file writing operation is over.

2.1. lsof: Listing Open Files

We can use the utility lsof to know if there’s anything still using the file:

$ lsof -f -- sample_file
COMMAND   PID     USER   FD  TYPE  DEVICE SIZE/OFF     NODE NAME
less    61728 username   4r   REG     8,6        0  7347202 sample_file
less    61833 username   4r   REG     8,6        0  7347202 sample_file

We’ve used the -f — argument to restrict the search to those files whose name is exactly sample_file. The flag in this case may be overkill but if the file name is provided as a path (containing /), the use of the flag is required.

We can see that the file is in use twice by the less command. Moreover, we can know the file descriptor (4 in the output) and the mode (r for reading). If we see a w, it means that the process has opened the file to write it. If the file is being written, it has to be open with the file descriptor of w. However, if the file is not in use (and we get an empty output of lsof), we’ll get no information on why it’s no longer open.

Alternatively, we can get a similar output by considering all the files under a directory:

$ lsof +d directory/
COMMAND   PID     USER  FD TYPE DEVICE  SIZE/OFF     NODE NAME
bash    61044 username cwd  DIR    8,6      4096  7346869 directory
less    61442 username cwd  DIR    8,6      4096  7346869 directory
less    61442 username  4r  REG    8,6         0  7346873 directory/sample_file_1

Another major drawback of the lsof command is the way it knows a file is being used. lsof searches in the kernel memory for open files. Thus, rapid changes in kernel memory will most probably result in unpredictable results.

2.2. fuser: Which Processes Are Using a File

There’s another command in Linux, fuser, that returns the PID of the running commands that have a file open:

$ fuser sample_file 
/home/user/sample_file: 61728 61833 

There are two commands (those with PID 61728 and 61833) that have the file sample_file opened. From there, we can use the ps command to know more about these processes.

2.3. Inspecting the Process Directories

There’s yet another way to know more about commands using a certain file by inspecting the /proc directory. Inside this directory, we have, among others, one directory for each running process in Linux. Inside each directory, there’s one directory named fd. This fd directory contains symbolic links created to the files that the process is using.

The permissions of the symbolic link represent the file mode in which the program is using the file. If it has permission for writing, this means that the file was potentially open for writing. We can see the permissions with the ls -l flag.

For example, let’s see how this looks for PID 61728 from the example in the previous section:

$ ls -l /proc/61728/fd/
total 0
lrwx------ 1 user user 64 Oct 21 17:50 0 -> /dev/pts/2
lrwx------ 1 user user 64 Oct 21 17:50 1 -> /dev/pts/2
lrwx------ 1 user user 64 Oct 21 17:50 2 -> /dev/pts/2
lr-x------ 1 user user 64 Oct 21 17:50 3 -> /dev/pts/2
lr-x------ 1 user user 64 Oct 21 17:50 4 -> /home/user/sample_file

We can see that the file of interest (file descriptor 4) is a symbolic link pointing to the original final location. We see in the permissions that there is no w permission (for writing). The file only has r (for read) and x (for execute) permissions. This means that this process won’t be able to write to the file.

We can apply this method to list all the file descriptors in all PID (combining ls with the -d flag to print the full path and -l to display the file permissions) and then search for the file of interest (with grep):

$ sudo ls -d /proc/*[0-9]*/fd/* -l | grep sample_file
lr-x------ 1 user user 64 Oct 21 17:50 /proc/61728/fd/4 -> /home/user/sample_file
lr-x------ 1 user user 64 Oct 21 17:52 /proc/61833/fd/4 -> /home/user/sample_file

We see that two commands have the file opened (the same as we got from lsof and fuser).

Only processes owned by the current user can be inspected (unless we’re root or have sudo rights). This method might also be quite slow. Moreover, if the process opens and closes the file several times, we cannot know if the writing operation is over.

3. Is the File Still Changing?

Another way we can see if a write operation is completed is to check if the file is changing. These methods aren’t as robust as the previous ones: when a file is changing, there is certainly something modifying, it but we won’t know which is the process of modifying the file. These methods are useful when we’re transferring data and we suspect that the process may have halted.

3.1. Looping to See File Size Changes

We can see if a file is changing is with a for loop that uses the stat command to get the total size in bytes (with the -c%s flag) and compares its evolution in time:

$ for (( ; ; )); do
 before_bytes=$(stat -c%s sample_file)
 sleep 0.5
 after_bytes=$(stat -c%s sample_file)
 if [ $before_bytes -eq $after_bytes ]; 
 then
  break;
 fi
done

Whenever the terminal is freed, the writing operations will be over. If any command is writing the file, the file’s size will be increasing. We can increase the waiting time (set to 0.5 seconds) to allow for slower write operations.

3.2. Inspect the File Itself

An even simpler way to see if the file is changing is to check the last bytes of the file with tail. This command will show the last 10 lines of the file. We can compare these between multiple calls to the command. We might know that our file has some trailing characters or expressions. This lets us know that the writing operation is over.

Moreover, there are some file formats (like pdf) that have data in them to determine if the file is complete or not. This requires potentially reading the whole file, which is a time-consuming task – but it might prove helpful in some cases.

4. Conclusion

In this tutorial, we’ve discussed ways to gather information on the file statuses in Linux: what processes are using it and if they are writing to it.

Authors Bottom

If you have a few years of experience in the Linux ecosystem, and you’re interested in sharing that experience with the community, have a look at our Contribution Guidelines.

Comments are closed on this article!