1. Overview
In this tutorial, we’ll look at the special shell variable Internal Field Separator (IFS). First, we’ll discuss the default values of the IFS variable and understand its various use cases. Then, we’ll explore some implementations by setting custom values in IFS.
2. IFS Variable and Its Default Values
The special shell variable IFS determines how Bash recognizes word boundaries while splitting a sequence of character strings. The default value of IFS is a three-character string comprising a space, tab, and newline:
$ echo "$IFS" | cat -et
^I$
$Here we used the -e and -t options of the cat command to display the special character values of the IFS variable.
Now, Let’s run an example to demonstrate word splitting on a space-delimited string:
$ string="foo bar foobar"
$ for i in $string
> do
> echo "'$i' is the substring"
> done
'foo' is the substring
'bar' is the substring
'foobar' is the substringAs we looped over the string, we could get the individual substring because space is one of the IFS variable’s default values.
We’ll observe an output similar to above if the input contains tab-separated values:
for i in `echo -e "foo\tbar\tfoobar"`; do echo "'$i' is the substring"; doneAlongside space and tab, the IFS variable also holds a newline character as a default value. Let’s verify the field splitting effect on a newline-separated input:
$ string="foo
> bar
> foobar"
$ printf "%q\n" "$string"
$'foo\nbar\nfoobar'
$ for i in $string
> do
> echo "'$i' fetched from the string"
> done
'foo' fetched from the string
'bar' fetched from the string
'foobar' fetched from the stringHere we could split the newline-separated fields from the multi-line input.
3. Setting Custom Values in IFS
In the previous section, we learned about the default value of the IFS variable. Further, we can also set custom values in the IFS variable. This helps us to handle various field delimited strings.
Let’s learn how to do that with an example:
$ string=foo:bar:foobar
$ old_ifs="$IFS"
$ IFS=":"
$ for i in $string
> do
> echo "'$i' is the splitted word"
> done
'foo' is the splitted word
'bar' is the splitted word
'foobar' is the splitted wordIn this example, before updating the IFS variable’s value, we stored its value in another variable old_ifs.
Consequently, we can use this variable later to reset IFS to its original value:
IFS="$old_ifs"We can also unset IFS to reset the Bash behavior to its default:
$ echo "$IFS" | cat -et
^I$
$
$ string="foo bar foo:bar"
$ for i in $string; do echo "[$i] extracted"; done
[foo] extracted
[bar] extracted
[foo:bar] extracted
$ IFS=":" && echo "$IFS" | cat -et
:$
$ for i in $string; do echo "[$i] extracted"; done
[foo bar foo] extracted
[bar] extracted
$ unset IFS && echo "$IFS" | cat -et
$
$ for i in $string; do echo "[$i] extracted"; done
[foo] extracted
[bar] extracted
[foo:bar] extractedNote that, after we unset IFS, Bash processed word splitting as if IFS holds the default value.
However, if we set IFS to an empty string, then Bash will not perform any splitting operation:
$ IFS=""
$ for i in $string; do echo "[$i] extracted"; done
[foo bar foo:bar] extracted3.1. Default IFS vs. Custom IFS
There’s a subtle difference in IFS behavior when it holds the default whitespace characters values vs. when it holds the custom values.
Leading and trailing whitespace characters are stripped off from the input string when IFS is set to the default value. Along with this sequence of consecutive whitespace characters is treated as a single separator. However, this behavior is not observed when custom values are set in the IFS variable.
Let’s verify the default IFS behavior with an example:
string="foo bar foobar "
for i in $string
do
echo "[$i] extracted"
done
[foo] extracted
[bar] extracted
[foobar] extracted
Here the input string consists of trailing spaces and multiple spaces between the words. However, Bash ignored trailing whitespace characters and treated multiple spaces between words as a single delimiter.
Let’s check another example by setting a custom value (colon) in the IFS variable:
$ string="foo:bar::foobar::"
$ IFS=":"
$ for i in $string
> do
> echo "[$i] extracted"
> done
[foo] extracted
[bar] extracted
[] extracted
[foobar] extracted
[] extractedIn this case, the multiple colons between words of the input string weren’t treated as a single delimiter.
3.2. Use Cases With the read Command
Let’s use an example from our article on parsing CSVs to understand the use case of the IFS variable with the read command:
#! /bin/bash
while IFS="," read -r rec_column1 rec_column2 rec_column3 rec_column4
do
echo "Displaying Record-$rec_column1"
echo "Quantity: $rec_column2"
echo "Price: $rec_column3"
echo "Value: $rec_column4"
echo ""
done < <(tail -n +2 input.csv)Notably, we’re setting IFS to “,” in a while loop to break each line of input.csv into tokens. As a result, we can then parse the comma-delimited field values into Bash variables using the read command.
3.3. IFS and Positional Parameters
Special command-line arguments $@ and $* hold the list of arguments (positional parameters) passed to a Bash script or function.
Let’s create a sample script:
#!/bin/bash
echo " Input Parameters using \$@ : $@"
echo " Input Parameters using \$* : $*"Now, let’s execute the script with some positional parameters:
$ ./inputs.sh arg1 arg2 arg3
Input Parameters using $@ : arg1 arg2 arg3
Input Parameters using $* : arg1 arg2 arg3As we can see, both $@ and $* hold the input arguments to the script in a similar fashion. We’ll now update our script to set the IFS variable to a custom value:
#!/bin/bash
IFS='|'
echo " Input Parameters using \$@ : $@"
echo " Input Parameters using \$* : $*"Let’s observe the execution output:
$ ./inputs.sh arg1 arg2 arg3
Input Parameters using $@ : arg1 arg2 arg3
Input Parameters using $* : arg1|arg2|arg3The difference here is the join between the input arguments. The $IFS acts as a separator to the positional parameters when using $*. The parameters are expanded to $1c$2 and so on, where c is the first character set in IFS.
4. Conclusion
In this tutorial, we learned how Bash recognizes word boundaries and performs splitting.
In the beginning, we discussed the default values of the IFS variable and understood its various use cases. Then, we implemented some case-studies by setting custom values in IFS.