Cleartext Passwords in the Shell and How to Handle Special Characters

1. Introduction

Passwords are essential for security. Hence, it’s best to choose them with a high level of entropy. This process may involve picking a character from every available category, including symbols. Yet, some of those may include characters with special meanings, resulting in issues.

In this tutorial, we’ll explore when, why, and how we should escape which symbols when using passwords. First, we discuss cleartext passwords, why we should avoid them, and how to escape their characters. Next, we discuss passwords on the command line. After that, we explain sensitive string quoting. Then, we turn to a specific character in passwords. Finally, we briefly mention a way to debug cleartext password entry.

We tested the code in this tutorial on Debian 11 (Bullseye) with GNU Bash 5.1.4. It should work in most POSIX-compliant environments.

2. Cleartext Passwords

Supplying and using cleartext passwords is a bad practice anywhere and at any time.

In fact, there are a number of reasons for this:

• shoulder surfing
• compromised system password discovery
• shell history mechanism
• secondary memory may preserve cleartext passwords even if deleted

That’s why /etc/shadow contains hashed passwords.

Yet, there are still many instances where passwords are cleartext:

• SSH key password switches, the -p switch of mysql, and similar
• using HTTP without SSL, FTP instead of SFTP, and other security-weak protocols
• insecurely-written scripts
• local password storage

In all cases, we might need to consider where we supply our password and what it consists of.

3. Escape Password Characters

How a password entry is handled depends on the tool and implementation. Still, we may have to escape some password characters, especially when not quoting or when using double quotes in the shell:

• whitespace characters
• ( and ) parentheses subshell operator
• ` backtick command substitution operator
• ! exclamation point operator
• ‘ single quotes
• “ double quotes
• | pipe character
• < and > redirection operators
• & ampersand backgrounding operator
• # octothorp comment character
• ; semicolon command-line delimiter
• . period source built-in
• $ dollar sign substitution operator
• \ backslash escape character

As usual, escaping happens with the \ backslash escape character itself:

$ echo !text
-bash: !text: event not found
$ echo \!text
!text
$ echo "!text"
-bash: !text: event not found
$ echo "\!text"
\!text

Of course, for this example of the exclamation point operator, we can also disable Bash history expansion with set +H or set +o histexpand. However, doing similar configurations and escapes for each character can be tedious and error-prone.

Let’s see how we can improve and avoid that.

4. Password on the Command Line with read

The read built-in or external command is the standard way to get input in most shells.

Even in its most standard POSIX form, read provides the -r switch, which avoids treating backslashes in any special way, allowing us to use any character up to a newline:

$ read var
\\\\
$ echo $var
\\
$ read -r var
\\\\
$ echo $var
\\\\

Some implementations also provide the -s switch, which prevents the command from echoing its input.

Combining both -r and -s with read enables a fairly robust and secure password entry:

$ read -r -s var
$ echo $var
$Pipes (<`#P|pes.$S $p $a $c $e $s & Q"'otes\ackslashBone$!`>)

Naturally, the $var variable holds our password in cleartext, so passing it around might be a technical and security challenge. Still, using the standard read with variables can avoid many of the implementation issues of some tools.

5. Quoting Passwords

As usual, the best way to use a variable, password, or any standalone sensitive string involves quotes.

5.1. Variables

Especially in the case of variables, using no quotes often presents challenges like wildcard expansion, whitespace splitting, and misbehaving special characters. On the other hand, single quotes prevent interpolation.

Thus, double quotes are, in many cases, the best way to supply a variable as a password:

$ echo "$var"
$Pipes (<`#P|pes.$S $p $a $c $e $s & Q"'otes\ackslashBone$!`>)
$ ssh $var
ssh: Could not resolve hostname $pipes: Name or service not known
$ ssh "$var"
ssh: Could not resolve hostname $pipes (<;`#p|pes.$s $p $a $c $e $s & q"'otes\\ackslashbone$!`>;): Name or service not known

Notably, this avoids word splitting and special character interpretation.

5.2. Standalone Strings

There’s no need to escape the characters of any standalone string in single quotes:

$ echo '$Pipes (<`#P|pes.$S $p $a $c $e $s & Q"'otes\ackslashBone$!`>)'
-bash: syntax error near unexpected token `('
$ echo '$Pipes (<`#P|pes.$S $p $a $c $e $s & Q"'"'"'otes\ackslashBone$!`>)'
'$Pipes (<`#P|pes.$S $p $a $c $e $s & Q"'"'"'otes\ackslashBone$!`>)'

However, we can’t use or escape a single quote within a single-quoted string. To solve this, we glue both kinds of quotes to produce ‘”‘”‘ for every embedded single quote:

• First ‘ closes the open single-quoted string
• First “ opens a new double-quoted string
• Second ‘ is the first and only character of the double-quoted string
• Second “ closes the double-quoted string
• Third ‘ reopens the closed single-quoted string

In essence, we insert a single quote into a single-quoted string by terminating the single-quoted string, gluing a double-quoted single quote to it, and reopening the single-quoted string.

6. Newline in Passwords

It’s rare to see a newline character as part of a password mainly because of two reasons:

• password submission and setting usually rely on a terminating character
• commonly, shells use newlines as terminating characters

Let’s try to work around these limitations. First, we encode the newline character as part of the password string. After that, we also submit or set the password itself via automated commands or configuration files instead of interactive prompts.

Even then, we might encounter tools interpreting a newline character in the middle of our password as its end. To see how the shell sees, we can employ another technique.

7. Debugging Cleartext Password Submission

In Bash, there is a convenient way to monitor what’s being passed for interpretation after all interpolations are complete but before the actual interpreting is done:

$ set -x
$ ssh $var
+ ssh '$Pipes' '(<;`#P|pes.$S' '$p' '$a' '$c' '$e' '$s' '&' 'Q"'\''otes\ackslashBone$!`>;)'
ssh: Could not resolve hostname $pipes: Name or service not known

By using set -x, we can see what our command line looks like just before being run. In this case, we see all escapes, as well as additional quoting, which breaks up the original string into several ones. To turn the option off, we simply issue set +x.

8. Summary

In this article, we talked about passwords with special characters and how to handle them.

In conclusion, it’s usually best to use double quotes for passwords in variables and single quotes for passwords as standalone strings, avoiding the need to escape many characters.