Authors Top

If you have a few years of experience in the Linux ecosystem, and you’re interested in sharing that experience with the community, have a look at our Contribution Guidelines.

1. Overview

In this tutorial, we’ll look at the special shell variable Internal Field Separator (IFS). First, we’ll discuss the default values of the IFS variable and understand its various use cases. Then, we’ll explore some implementations by setting custom values in IFS.

2. IFS Variable and Its Default Values

The special shell variable IFS determines how Bash recognizes word boundaries while splitting a sequence of character strings. The default value of IFS is a three-character string comprising a space, tab, and newline:

$ echo "$IFS" | cat -et
 ^I$
$

Here we used the -e and -t options of the cat command to display the special character values of the IFS variable.

Now, Let’s run an example to demonstrate word splitting on a space-delimited string:

$ string="foo bar foobar"
$ for i in $string
> do
>  echo "'$i' is the substring"
> done
'foo' is the substring 
'bar' is the substring
'foobar' is the substring

As we looped over the string, we could get the individual substring because space is one of the IFS variable’s default values.

We’ll observe an output similar to above if the input contains tab-separated values:

 for i in `echo -e "foo\tbar\tfoobar"`; do echo "'$i' is the substring"; done

Alongside space and tab, the IFS variable also holds a newline character as a default value. Let’s verify the field splitting effect on a newline-separated input:

$ string="foo
> bar
> foobar"
$ printf "%q\n" "$string"
$'foo\nbar\nfoobar'
$ for i in $string
> do
> echo "'$i' fetched from the string"
> done
'foo' fetched from the string
'bar' fetched from the string
'foobar' fetched from the string

Here we could split the newline-separated fields from the multi-line input.

3. Setting Custom Values in IFS

In the previous section, we learned about the default value of the IFS variable. Further, we can also set custom values in the IFS variable. This helps us to handle various field delimited strings.

Let’s learn how to do that with an example:

$ string=foo:bar:foobar
$ old_ifs="$IFS"
$ IFS=":"
$ for i in $string
> do
>   echo "'$i' is the splitted word"
> done
'foo' is the splitted word
'bar' is the splitted word
'foobar' is the splitted word

In this example, before updating the IFS variable’s value, we stored its value in another variable old_ifs.

Consequently, we can use this variable later to reset IFS to its original value:

IFS="$old_ifs"

We can also unset IFS to reset the Bash behavior to its default:

$ echo "$IFS" | cat -et
 ^I$
$
$ string="foo bar foo:bar"
$ for i in $string; do echo "[$i] extracted"; done
[foo] extracted
[bar] extracted
[foo:bar] extracted
$ IFS=":"  && echo "$IFS" | cat -et
:$
$ for i in $string; do echo "[$i] extracted"; done
[foo bar foo] extracted
[bar] extracted
$ unset IFS  && echo "$IFS" | cat -et
$
$ for i in $string; do echo "[$i] extracted"; done
[foo] extracted
[bar] extracted
[foo:bar] extracted

Note that, after we unset IFS, Bash processed word splitting as if IFS holds the default value.

However, if we set IFS to an empty string, then Bash will not perform any splitting operation:

$ IFS=""
$ for i in $string; do echo "[$i] extracted"; done
[foo bar foo:bar] extracted

3.1. Default IFS vs. Custom IFS

There’s a subtle difference in IFS behavior when it holds the default whitespace characters values vs. when it holds the custom values.

Leading and trailing whitespace characters are stripped off from the input string when IFS is set to the default value. Along with this sequence of consecutive whitespace characters is treated as a single separator. However, this behavior is not observed when custom values are set in the IFS variable.

Let’s verify the default IFS behavior with an example:

string="foo  bar    foobar     "
for i in $string
do
  echo "[$i] extracted"
done
[foo] extracted
[bar] extracted
[foobar] extracted

Here the input string consists of trailing spaces and multiple spaces between the words. However, Bash ignored trailing whitespace characters and treated multiple spaces between words as a single delimiter.

Let’s check another example by setting a custom value (colon) in the IFS variable:

$ string="foo:bar::foobar::"
$ IFS=":"
$ for i in $string
> do
>   echo "[$i] extracted"
> done
[foo] extracted
[bar] extracted
[] extracted
[foobar] extracted
[] extracted

In this case, the multiple colons between words of the input string weren’t treated as a single delimiter.

3.2. Use Cases With the read Command

Let’s use an example from our article on parsing CSVs to understand the use case of the IFS variable with the read command:

#! /bin/bash
while IFS="," read -r rec_column1 rec_column2 rec_column3 rec_column4
do
 echo "Displaying Record-$rec_column1"
 echo "Quantity: $rec_column2"
 echo "Price: $rec_column3"
 echo "Value: $rec_column4"
 echo ""
done < <(tail -n +2 input.csv)

Notably, we’re setting IFS to “,” in a while loop to break each line of input.csv into tokens. As a result, we can then parse the comma-delimited field values into Bash variables using the read command.

3.3. IFS and Positional Parameters

Special command-line arguments [email protected] and $* hold the list of arguments (positional parameters) passed to a Bash script or function.

Let’s create a sample script:

#!/bin/bash
echo " Input Parameters using \[email protected] : [email protected]"
echo " Input Parameters using \$* : $*"

Now, let’s execute the script with some positional parameters:

$ ./inputs.sh arg1 arg2 arg3
 Input Parameters using [email protected] : arg1 arg2 arg3
 Input Parameters using $* : arg1 arg2 arg3

As we can see, both [email protected] and $* hold the input arguments to the script in a similar fashion. We’ll now update our script to set the IFS variable to a custom value:

#!/bin/bash
IFS='|'
echo " Input Parameters using \[email protected] : [email protected]"
echo " Input Parameters using \$* : $*"

Let’s observe the execution output:

$ ./inputs.sh arg1 arg2 arg3
 Input Parameters using [email protected] : arg1 arg2 arg3
 Input Parameters using $* : arg1|arg2|arg3

The difference here is the join between the input arguments. The $IFS acts as a separator to the positional parameters when using $*. The parameters are expanded to $1c$2 and so on, where c is the first character set in IFS.

4. Conclusion

In this tutorial, we learned how Bash recognizes word boundaries and performs splitting.

In the beginning, we discussed the default values of the IFS variable and understood its various use cases. Then, we implemented some case-studies by setting custom values in IFS.

Authors Bottom

If you have a few years of experience in the Linux ecosystem, and you’re interested in sharing that experience with the community, have a look at our Contribution Guidelines.

2 Comments
Oldest
Newest
Inline Feedbacks
View all comments
Comments are closed on this article!