1. Introduction
Linux provides many security mechanisms. One of the most basic is the /etc/shadow file, which holds the hashed passwords of users in /etc/passwd.
In this tutorial, we’ll explore /etc/shadow and ways to generate an encrypted password in a proper format. First, we interpret the structure of the /etc/shadow file as well as how to work with it manually and via standard utilities. Next, we explore encrypted password generation in general and with common tools and languages.
Importantly, we use the terms encryption and hashing interchangeably in this article, despite the fact that the former is reversible and the latter isn’t. In fact, this is common practice in manuals and functions for hashing and encryption of passwords.
We tested the code in this tutorial on Debian 11 (Bullseye) with GNU Bash 5.1.4. It should work in most POSIX-compliant environments.
2. /etc/shadow
Following the basic format of /etc/passwd, /etc/shadow contains basic user password data in eight colon-separated columns per unique user row:
- Username – an already existing account name from /etc/passwd
- Password – the actual hashed password when the respective password entry in /etc/passwd is x
- Change – date of the last password change (number of days since 1 Jan 1970) with 0 forcing a change on the next login, and an empty entry meaning no password aging
- Minimum – minimum number of days between password changes with 0 or an empty entry omitting the constraint
- Maximum – maximum number of days a password is valid with 99999 as the default
- Warning – number of days prior to password expiry that should be left for a user to receive a warning about it
- Inactive – number of days after password expiry, following which the user will become disabled
- Expire – date after which the user becomes disabled (number of days since 1 Jan 1970) with 7 as the default
Let’s see an example entry:
user1:$y$j9T$zxM[...]aS.$XM6[...]66E:19278:0:99999:7::19285:When editing /etc/shadow directly, it’s best to use the -s switch to vipw, which locks /etc/shadow similar to visudo and the /etc/sudoers file. Critically, vipw doesn’t automatically check the syntax after any edits.
2.1. chage and Password Aging
The chage tool can display and configure password age settings without needing to manually calculate days and dates.
Let’s use the –list flag to check all settings for the root user:
$ chage --list root
Last password change : Oct 01, 2022
Password expires : never
Password inactive : never
Account expires : never
Minimum number of days between password change : 0
Maximum number of days between password change : 99999
Number of days of warning before password expires : 7As of now, there are no limits imposed on root.
Consequently, we can change any field via its associated flag:
- -d or –lastday for the last change date
- -m or –mindays for the days between password changes
- -M or –maxdays for the maximum password validity
- -W or –warndays for the warning period
- -I or –inactive for the inactivity period
- -E or –expiredate for the expiration period
Of course, most importantly, any dates can be in a more human-readable format like YYYY-MM-DD.
Let’s set an expiry date for a user:
$ chage --expiredate 2022-10-20 user1
$ chage --list user1
Last password change : Oct 13, 2022
Password expires : never
Password inactive : never
Account expires : Oct 20, 2022
[...]Here, we set and confirm 20 Oct 2022 as the expiry date of user1. To reset the expiry date to never, we can use the same command but with -1 (negative one) as the date.
Apart from the username and password aging fields, we are only left with the password.
2.2. chpasswd and Passwords
The chpasswd tool is complementary to chage because it enables us to show and manipulate the password field of /etc/shadow.
To do so, chpasswd receives and applies a list of USERNAME:PASSWORD pairs. Further, the tool can select and use encryption methods and specify some of their options:
$ echo 'user1:PASSWORD' | chpasswd --crypt-method SHA512In this case, we use SHA-512 to set the password of user1. Internally, chpasswd uses a call to a standard UNIX encryption function.
2.3. crypt() and Encryption Algorithms
There are many algorithms for encryption. Which one /etc/shadow uses depends on several factors.
Usually, the default encryption algorithm can be read or defined via the ENCRYPT_METHOD variable of /etc/login.defs. Alternatively, we can use the pam_unix.so pluggable authentication module (PAM) and change the default hashing algorithm via /etc/pam.d/common-password.
Another consideration is the presence and version of glibc. In fact, crypt(), as the main password encryption function, leverages glibc. By default, it uses the insecure Data Encryption Standard (DES), but depending on the second argument, we can employ many others.
Essentially, the initial characters of the password field value in /etc/shadow identify the encryption algorithm:
- $1$ is Message Digest 5 (MD5)
- $2a$ is blowfish
- $5$ is 256-bit Secure Hash Algorithm (SHA-256)
- $6$ is 512-bit Secure Hash Algorithm (SHA-512)
- $y$ (or $7$) is yescrypt
- none of the above means DES
Critically, as of this writing, yescrypt with its contest entry yescrypt v2 and current specification, is widely-adopted and the default password hashing scheme for many recent versions of major distributions like Debian 11, Fedora 35+, Kali Linux 2021.1+, and Ubuntu 22.04+. Further, it’s supported on Fedora 29+ and RHEL 9+. Still, many standard tools still don’t support yescrypt.
In all cases except DES, the whole format of the field becomes more complex:
$id$param$salt$encryptedOf these dollar-delimited subfields, we already explored id. The param field can contain options for algorithms that support them.
Finally, a salt in the password encryption context is a random supplemental value used when going through the process of hashing. Its main idea is to introduce complexity beyond that of a regular human to reduce the odds of a successful cracking.
3. Generating an /etc/shadow Password
After understanding the format of /etc/shadow and its password field, we move on to manual password generation.
Naturally, we can use the passwd and chpasswd commands to automatically populate /etc/shadow as one of the steps for user creation.
Still, we can also set the value of the password field by hand via vipw, especially for newer hashing algorithms like yescrypt. Of course, we have to calculate it correctly based on our password and a salt.
Critically, to avoid storing passwords in the Bash history, we can set the HISTCONTROL environment variable to ignorespace. After that, any command with a leading space is ignored, as history confirms:
$ history -c
$ history
1 history
$ HISTCONTROL=ignorespace
$ echo Ignored.
$ history
1 history
2 HISTCONTROL=ignorespace
3 historyIn addition to disabling history, we can use stdin, targeted hidden password entry mechanisms, or reading from temporary files, where possible.
In all cases, we use SHA-512 as an example since it’s widely considered the safest of the standard algorithms. Notably, despite being provably strong, blowfish can be seen as an outlier and not as common.
4. Generate /etc/shadow Passwords with OpenSSL
We can easily generate most standard password types directly via OpenSSL and its openssl toolkit.
First, we install openssl. Next, we can generate a password with the passwd subcommand:
$ openssl passwd PASSWORD
kyP4M07HmoQHE
$ openssl passwd PASSWORD
EFqb5NwMiyE2gBy default, openssl generates a random salt each time. However, we can supply a salt via -salt to produce the same results for equivalent passwords and salts:
$ openssl passwd -salt SALT PASSWORD
SA4DMc/4L/jVQ
$ openssl passwd -salt SALT PASSWORD
SA4DMc/4L/jVQMoreover, recent versions of OpenSSL support several types of hashing:
- -crypt for the standard UNIX crypt, i.e., DES (default)
- -apr1 for the Apache-specific MD5 variant
- -1 for MD5
- -5 for SHA-256
- -6 for SHA-512
Since yescrypt doesn’t have a Request for Comments (RFC), it’s not part of the suite. For all but the first variant above, we get all fields as described earlier:
$ openssl passwd -6 -salt SALT PASSWORD
$1$SALT$YQNBYRN9kIvLkQIp4SpsO0Now, let’s increase security by removing the manual salting and using stdin alone and with a file:
$ openssl passwd -6 -stdin
PASSWORD
$6$jbLx5r051YAp08Oo$hwD8avGKTzCWX4QUCrCluDZde6RBZwbr0F6WEGGcZwtpV16h1irFewkHxZykS7r0RPZtqHO.Vv9RiWGTbkrfN.
^C
$ cat passwordfile
PASSWORD
$ openssl passwd -6 -stdin < passwordfile
$6$ve2o67Cutv97NNTI$SsqCb.7zYcYSjWWuXNeQSCK.f9PIxf/K9DBNHNJyATq1mj8q/Fshsb8zu5Rze/jCbLTELrn/MU77AbsKLOy0b0
$ clearIn fact, we should be able to plug the result directly into our /etc/shadow for all supported algorithms.
5. Generate /etc/shadow Passwords with Interpreters
Of course, installing an additional package like OpenSSL might not be possible, desired, or even necessary. Let’s try to generate passwords using the system or external encryption library via commonly preinstalled interpreters.
5.1. Perl
Ever since 1987, the Practical Extraction and Report Language (Perl) with its perl interpreter has been the swiss knife in many UNIX systems through the years.
Perl has a direct interface to crypt() without any necessary imports. The format of the call is more or less self-explanatory:
$ perl -e 'print crypt "PASSWORD", "\$6\$SALT\$"'
$6$SALT$io0TPmhM8ythCm7Idt0AfYvTuFCLyA1CMVmeT3EUqarf2NQcTuLKEgP9.4Q8fgClzP7OCnyOY1wo1xDw0jtyH1
$ perl -e 'print crypt "PASSWORD", "\$6\$SALT\$"'
$6$SALT$io0TPmhM8ythCm7Idt0AfYvTuFCLyA1CMVmeT3EUqarf2NQcTuLKEgP9.4Q8fgClzP7OCnyOY1wo1xDw0jtyH1Here, we use the one-liner flag -e to print the result of combining PASSWORD as the password with SALT as the salt through SHA-512 (6). Hence, we get the same result.
Of course, we can use any valid construct to generate the salt and supply the password via stdin (<>):
$ perl -e 'print crypt <>, "\$6\$".(
join "", (".", "/", 0..9, "A".."Z", "a".."z")[
rand 64, rand 64, rand 64, rand 64, rand 64, rand 64
])."\$"'
PASSWORD
$6$qWByzg$PolYXthwP.hYqGN0z44Q9UBtNRAq9rggEdTeImNd2KV2sMy19UpFReLKWisDjOmsLCYyhGYvFQGhC5oL1Be1T1This way, we get a unique string on each call and can also use files. Naturally, we can employ MD5 ($1$) and SHA-256 ($5$) as well. Moreover, modules like Crypt::Blowfish offer more algorithms.
Still, we can generate yescrypt hashes as long as our system supports them and we follow the proper format:
$ perl -e 'print crypt "PASSWORD", "\$y\$j9T\$SALT\$"'
$y$j9T$SALT$HIA0o5.HmkE9HhZ4H8X1r0aRYrqdcv0IJEZ2PLpqpz6
$ perl -e 'print crypt "PASSWORD", "\$7\$9/..../..../\$SALTS\$"'
$7$9/..../..../$SALTS$uZRYjuH0DtWQPgNx0StRUYTkmbVbzv4LxkE7YeHPAl1Critically, since crypt() is a thin wrapper, its implementation might vary. For portable SHA-512 encryption, Digest::SHA is probably a better choice.
5.2. Python
Python began in 1991 and became famous around 2003. Critically, it’s vital for several major Linux distributions like Red Hat, Debian, and Ubuntu. Similar to Perl, Python is common on UNIX in general.
Unlike Perl, Python needs a separate import statement for crypt(), but that’s part of the default installation. That’s valid for both version 2 and version 3 of the interpreter. Notably, we show examples with both, each using the -c one-liner flag.
Again, we can choose both the password and the salt:
$ python3 -c 'import crypt; print(crypt.crypt("PASSWORD", "$6$SALT"))'
$6$SALT$io0TPmhM8ythCm7Idt0AfYvTuFCLyA1CMVmeT3EUqarf2NQcTuLKEgP9.4Q8fgClzP7OCnyOY1wo1xDw0jtyH1
$ python2 -c 'import crypt; print(crypt.crypt("PASSWORD", "$6$SALT$"))'
$6$SALT$io0TPmhM8ythCm7Idt0AfYvTuFCLyA1CMVmeT3EUqarf2NQcTuLKEgP9.4Q8fgClzP7OCnyOY1wo1xDw0jtyH1Yet, we can also generate the salt and use stdin directly or via getpass() for even better security:
$ python3 -c 'import crypt,getpass; print(crypt.crypt(getpass.getpass(),crypt.mksalt(crypt.METHOD_SHA512)))'
Password:
$6$5rl.0/Aeo6EkWhiC$jeWoXYG67DMRHJNlZnD4NWwXN0Z3ZjLROpLZ2zkfdr3d4wgTsAwH06AXe2LoSDhGItYKapiPXv2XiBu4718bM0
$ python2 -c 'import crypt,getpass,random; print(crypt.crypt(getpass.getpass(), "$6$" + ("".join(random.choice([chr(i) for i in range(97,123)] + [chr(i) for i in range(65,91)] + [chr(i) for i in range(48,58)] + ["/", "."]) for _ in range(6))) + "$"))'
Password:
$6$lxN6u6$gvEHAwmv30RzKyHZ3TOGxhCvLaFUu5uobE4YfZ0eE/jjTa47OQmxolm3cQr7St75VhJ9xGv/kMuEmZFDzCWsa0When using hidden input, it’s generally good practice to enter the password twice and compare the results:
$ python -c 'import getpass; p=getpass.getpass(); print("Match." if p==getpass.getpass("Again: ") else "No match.")'
Password:
Again:
Match.Verifying the password can be vital to avoid locking ourselves out of a system.
5.3. Ruby
The Ruby programming language is also commonly available on UNIX platforms with its interpreter: ruby.
In this case, we call crypt() directly on the password string and with the salt as its argument to get the same result:
$ ruby -e 'puts "PASSWORD".crypt("$6$SALT")'
$6$SALT$io0TPmhM8ythCm7Idt0AfYvTuFCLyA1CMVmeT3EUqarf2NQcTuLKEgP9.4Q8fgClzP7OCnyOY1wo1xDw0jtyH1Again, we can also use a random salt and stdin for the password entry:
$ ruby -e 'require "io/console"; puts IO::console.getpass.crypt("$6$" + rand(36 ** 8).to_s(36))'Importantly, unlike Python, Ruby’s getpass() doesn’t provide a prompt by default.
6. Summary
In this article, we dived deep into the /etc/shadow file, exploring each of its options, as well as how to change them via tools and by hand. Finally, we explored different ways to create an encrypted (hashed) version of a password so it’s compatible with the file.
In conclusion, /etc/shadow protects the system, so understanding it can prevent unauthorized access.