Set Environment Variables From File of Key/Value Pairs in Linux

1. Introduction

As a best practice, we usually store secret credentials and other application configurations in a file that is not a part of the application source code. A popular way to do this is by using a .env file, in which each line consists of a key-value pair separated by an equal (=) sign. For the application to be able to read variables from this file, we need to set each key-value pair as a Linux environment variable.

In this tutorial, we’ll look at different ways to set environment variables from a file of key-value pairs in Linux.

2. Our .env File

For this tutorial, let’s assume there is a file .env in our current working directory, with one key-value pair per line, similar to the one below:

$ cat .env 
VAR1="hello"
VAR2="world"
VAR3="thisIsMyEnvFile"

Sometimes, we may comment out some lines in the file using # and expect that our method ignores these lines:

$ cat .env 
VAR1="hello"
VAR2="world"
VAR3="thisIsMyEnvFile"
#VAR4="toBeIgnored"

Now, let’s look at different ways to accomplish our task.

3. Using xargs and export

The xargs command stands for eXtended ARGumentS, and it converts the input we give it into arguments for a command. It takes input from standard input (STDIN). The export command is used to set environment variables. We can use the xargs command to convert each line of our .env file into an argument to the export command. Let’s see how to accomplish this.

3.1. Basic Usage

The most basic way to set environment variables from our .env file is as follows:

$ export $(cat .env | xargs) && env
... other environment variables ...
VAR1=hello
VAR2=world
VAR3=thisIsMyEnvFile
... other environment variables ...

Let’s take a look at what happened above. Firstly, to set the environment variables, we used the xargs command on the .env file contents to set the key-value pairs as arguments to the export command. Subsequently, the export command sets the environment variables we need.

Then, we can run our application, and the variables will be accessible to the application. For demonstration purposes, we ran the env command instead of our application. This command prints all the environment variables currently set, and we verified that our method is working as expected because the variables from our file were seen in the output.

3.2. Ignoring Comments

If we comment out a line in our .env file by adding a hash (#) at the beginning, the above method will fail:

$ export $(cat .env | xargs) && env
bash: export: `#VAR4=toBeIgnored': not a valid identifier

To overcome this, we need to use grep instead of cat and filter out the lines that start with a hash character:

$ export $(grep -v '^#' .env | xargs) && env
... other environment variables ...
VAR1=hello
VAR2=world
VAR3=thisIsMyEnvFile
... other environment variables ...

In our command, we matched the lines that started with a hash and used the -v option of grep to invert the selection. This will select all the lines that do not start with a hash and send it as input to xargs.

3.3. Limitations

We must note that one critical limitation of using xargs is that it is very buggy when the values have spaces in them. We’ll see other methods below that work well with spaces.

4. Using the set Command

The set command helps us modify shell options. Specifically, the -a option causes all variables that are defined after the command to be exported to all subshells and scripts. Let’s see how we can set the environment variables from our .env file using this method:

$ set -a
$ source .env
$ set +a
$ env
... other environment variables ...
VAR1=hello
VAR2=world
VAR3=thisIsMyEnvFile
... other environment variables ...

In the above snippet, after running set -a, we run our .env file, which causes all the key-value pairs to be set as environment variables. This is possible because the syntax of our .env file is valid Bash syntax for setting variables. Then, we turn off the “a” option using set +a so that the variables defined after this are not exported. Further, we run the env command to verify if the variables have been set. Since we’re using the source command to execute our file with the key-value pairs, this operation will ignore comment lines starting with “#”.

5. Using the env Command

We can use the env command to temporarily set variables before running another command:

$ env VAR1=hello env
... other environment variables ...
VAR1=hello

In the above code, the first env command sets the variable VAR1=hello before the second env command runs. In a real-life scenario, we would run our application command instead of the second env command.

Now, the env command provides us with an -S option that splits the argument into separate arguments. We can pass in the contents of our .env file and use this option to set all the variables:

$ env -S $(grep -v '^#' .env) env
... other environment variables ...
VAR1=hello
VAR2="world"
VAR3="thisIsMyEnvFile"

Note that we used grep to filter out the lines starting with a hash (#) as we did earlier.

6. Conclusion

In this article, we looked at different ways to set environment variables from a file containing key-value pairs. The xargs method is straightforward but has some limitations when the values contain spaces. Using the set command overcomes these limitations but is a multi-step process. Using the env command with the -S options is a very brief method that also overcomes the limitations of using xargs.