1. Overview
As system administrators, we regularly work with different types of servers remotely. Needless to say, accessing the remote servers using scripts on a local machine becomes a necessity. Notably, there are many Linux-based tools available to get the data from the remote servers. Thus, these are extensively employed in the management of hundreds of remote servers for easy orchestration.
This tutorial will elucidate the easiest ways of accessing and executing the commands on a remote SSH machine.
Now, let’s get into the nitty-gritty of it.
2. Using the expect Tool
Generally, we use the expect script for automating the user-interactive CLI prompts. It works by expecting a CLI prompt pattern, which automatically sends out responses without any user interaction.
2.1. Writing the Script
We use four main expect commands to automate this interaction:
- First, the spawn invokes a new process or session
- Second, the expect waits for the spawned process output in the expected pattern
- Third, the send writes to the spawned process’ stdin
- Finally, they interact the control back to the current process so that stdin is sent to the current process, and subsequently, stdout and stderr are returned
Let’s see a quick example:
$ cat remote_login.exp
#!/usr/bin/expect
set timeout 60
spawn ssh -p [lindex $argv 3] [lindex $argv 1]@[lindex $argv 0]
expect "*?assword" {
send "[lindex $argv 2]\r"
}
expect ":~$ " {
send "hostname\r\r"
}
expect ":~$ " {
send "df -h | grep sd\r\r"
}
expect ":~$ " {
send "tail -2 /var/log/dpkg.log\r\r"
}
expect ":~$ " {
send "exit\r"
}
interactThe above snippet shows the simple expect script that logs in to the SSH server. The script starts with the shebang (#!) followed by the absolute interpreter path. It helps in identifying the interpreter for executables in Linux-based environments.
With the support of the first line, the program loader executes the specified interpreter program and passes the script as the first argument. It is recommended to set a timeout as some server prompts are slow to load. Here, we’ve set it to one minute while the infinite wait time loads the remote system. Ten seconds is the default timeout in the expect.
Next, we invoke a new SSH session using spawn. lindex gets the individual elements from a positional argument list. The script expects four positional arguments on the remote SSH server.
- IP address [ Position 0 ]
- Username [ Position 1 ]
- Password [ Position 2 ]
- SSH Port [ Position 3 ]
After executing the script with all four valid positional parameters, the remote server inquires for the password. The first letter “P” in the “password” prompt might be uppercase in some systems and lowercase in others. To eliminate such ambiguity, we match only a part of the pattern, “password”. Then, the send command directs the password to the terminal stdin. Further on, depending on the terminal patterns, we keep sending the commands for execution.
In this case, the hostname obtains the name of the remote SSH machine. The df command provides the disk usage of the remote server. We filter out the “sda” volume using the grep command. Lastly, we execute the tail command in the log files on the remote server and close the session using the exit command. As mentioned earlier, interact returns the control to the current parent process so that stdin is mapped back to the parent process.
2.2. Executing the Script
Now, let’s execute the script by providing four positional parameters – IP address (10.149.20.11), username (tools), password (Bael@123), and SSH port (4455):
$ ./remote_login.exp 10.149.20.11 tools Bael@123 4455
spawn ssh -p 4455 [email protected]
[email protected]'s password:
Welcome to Ubuntu 18.04.6 LTS (GNU/Linux 5.4.0-87-generic x86_64)
...
... output truncated ...
...
Last login: Thu Oct 21 15:55:11 2021 from 180.98.67.141
$ hostname
REMOTE-SERVER
$
$ df -h | grep sd
/dev/sda1 98G 15G 79G 16% /
$
$ tail -2 /var/log/dpkg.log
2021-10-19 05:17:21 status half-configured man-db:amd64 2.8.3-2ubuntu0.1
2021-10-19 05:17:24 status installed man-db:amd64 2.8.3-2ubuntu0.1
$
$ exit
logout
Connection to 10.149.20.11 closed.
$As we can see, the script executed all the commands (hostname, df, tail) on the remote SSH server. Finally, the session between the local and remote servers was terminated using the exit command.
2.3. Using autoexpect
Instead of writing a program manually using an expect script, we can automatically create scripts using the autoexpect tool. This is because it monitors our interaction with another program or CLI prompts and automatically creates an expect script that exactly replicates our interaction. It hits the sweet spot even with seasoned veterans, as it automates various mindless interaction pieces.
For the sake of discussion, let’s use the autoexpect tool to create the expect script automatically. Now, we’ll type autoexpect and execute the required commands on the remote SSH server:
$ autoexpect ssh [email protected] -p4455 'hostname; df -h | grep sd; tail -2 /var/log/dpkg.log'
autoexpect started, file is script.exp
[email protected]'s password:
REMOTE-SERVER
/dev/sda1 98G 15G 79G 16% /
2021-10-19 05:17:21 status half-configured man-db:amd64 2.8.3-2ubuntu0.1
2021-10-19 05:17:24 status installed man-db:amd64 2.8.3-2ubuntu0.1
autoexpect done, file is script.exp
$The first line of the output enunciates that the tools started recording the interaction, and script.exp stores the final expect script.
Subsequently, let’s open up the file to check the records of the transaction in expect form:
$ cat script.exp
#!/usr/bin/expect -f
...
... output truncated ...
...
set force_conservative 0 ;# set to 1 to force conservative mode even if
;# script wasn't run conservatively originally
if {$force_conservative} {
set send_slow {1 .1}
proc send {ignore arg} {
sleep .1
exp_send -s -- $arg
}
}
...
... output truncated ...
...
set timeout -1
spawn ssh [email protected] -p4455 {hostname; df -h | grep sd; tail -2 /var/log/dpkg.log}
match_max 100000
expect -exact "[email protected]'s password: "
send -- "Bael@123\r"
expect eofFinally, without providing any input arguments, we can execute the script to perform the said tasks:
$ ./script.exp
spawn ssh [email protected] -p4455 hostname; df -h | grep sd; tail -2 /var/log/dpkg.log
[email protected]'s password:
REMOTE-SERVER
/dev/sda1 98G 15G 79G 16% /
2021-10-19 05:17:21 status half-configured man-db:amd64 2.8.3-2ubuntu0.1
2021-10-19 05:17:24 status installed man-db:amd64 2.8.3-2ubuntu0.1
$3. Using sshpass
In the previous section, we saw how to automate the end-to-end transaction between the servers. Now, we’ll see how to create a non-interactive SSH authentication session enabled using sshpass.
The sshpass tool automatically provides the password for SSH, based on the login, and takes us to the final server terminal. The tool helps with the user ssh interaction and shell scripting as well. Simply put, sshpass can support but can’t supplant SSH. Option -p of sshpass provides the password inline, which in turn passes simultaneously to the ssh password prompt on the remote server:
$ sshpass -p 'Bael@123' ssh [email protected] -p4455 'hostname; df -h | grep sd; tail -2 /var/log/dpkg.log';
Welcome to Ubuntu 18.04.6 LTS (GNU/Linux 5.4.0-87-generic x86_64)
...
... output truncated ...
...
Last login: Thu Oct 19 12:15:31 2021 from 180.98.67.141
REMOTE-SERVER
/dev/sda1 98G 15G 79G 16% /
2021-10-19 05:17:21 status half-configured man-db:amd64 2.8.3-2ubuntu0.1
2021-10-19 05:17:24 status installed man-db:amd64 2.8.3-2ubuntu0.13.1. Using Environment Variables
In the above case, we provide the password in the command line, which is insecure and not a recommended practice. To circumvent this, we can either use SSHPASS environment variable with ssh flag -e or refer to a file with ssh flag -f.
Now, let’s explore the usage of the environment variable to transmit the password:
$ export SSHPASS="Bael@123"
$ echo $SSHPASS
Bael@123Option -e of the sshpass command directs the tool to refer to the session environment variable with the default name SSHPASS for the password:
$ sshpass -e ssh [email protected] -p4455 'hostname; df -h | grep sd; tail -2 /var/log/dpkg.log';
Welcome to Ubuntu 18.04.6 LTS (GNU/Linux 5.4.0-87-generic x86_64)
...
... output truncated ...
...
Last login: Sat Oct 23 12:14:42 2021 from 180.98.67.141
REMOTE-SERVER
/dev/sda1 98G 15G 79G 16% /
2021-10-19 05:17:21 status half-configured man-db:amd64 2.8.3-2ubuntu0.1
2021-10-19 05:17:24 status installed man-db:amd64 2.8.3-2ubuntu0.1
$3.2. Using File in Encrypted Form
Environment variables are available for the current session only. All values will get lost if we open a new session or log out from the current session. Nevertheless, we have an option to redirect the password reference to a static file:
$ echo 'Bael@123' > .sshpasswd
$ cat .sshpasswd
Bael@123Now, let’s keep the password in the .sshpasswd file in the $HOME directory. With the help of option -f in sshpass, we supply the password file path in the command:
$ sshpass -f .sshpasswd ssh [email protected] 'hostname; df -h | grep sd; tail -2 /var/log/dpkg.log';
Welcome to Ubuntu 18.04.6 LTS (GNU/Linux 5.4.0-87-generic x86_64)
...
... output truncated ...
...
Last login: Sat Oct 23 08:43:16 2021 from 180.98.67.141
REMOTE-SERVER
/dev/sda1 98G 15G 79G 16% /
2021-10-19 05:17:21 status half-configured man-db:amd64 2.8.3-2ubuntu0.1
2021-10-19 05:17:24 status installed man-db:amd64 2.8.3-2ubuntu0.1
$A plain static file is an open book and can be read by any user who has access to that local machine. It would be sensible to encrypt the password file to restrict unconditional usage.
First, let’s encrypt the .sshpasswd file using the OpenPGP encryption and signing tool. Option -c of the gpg will encrypt the password with a symmetric cipher using a passphrase. By default, it uses AES-128, but we can change it using the –cipher-algo option:
$ echo 'Bael@123' > .sshpasswd
$
$ gpg -c .sshpasswd
gpg: keybox '/home/tools/.gnupg/pubring.kbx' created
$
$ ls -la | grep sshpasswd
-rw-rw-r-- 1 tools tools 10 Oct 23 08:59 .sshpasswd
-rw-rw-r-- 1 tools tools 90 Oct 23 08:59 .sshpasswd.gpgAfter creating the .sshpasswd.gpg file, we can remove the .sshpasswd file from the machine:
$ rm -f .sshpasswd
$ ls -la | grep sshpasswd
-rw-rw-r-- 1 tools tools 90 Oct 23 08:59 .sshpasswd.gpgNow, when we try to open the .sshpasswd.gpg, it will be in encrypted form:
$ cat .sshpasswd.gpg
-}▒▒▒Iſ▒Y▒▒:▒=8E0▒▒▒▒▒
▒▒:i(▒▒+▒mM▒},w.b=zi▒!▒▒ݴ▒t▒▒n▒▒*▒▒▒_YB9▒F▒7However, we can decrypt the text inside the file using the gpg command:
$ gpg -d -q .sshpasswd.gpg
Bael@123 Now, let’s integrate gpg, sshpass, and ssh:
$ gpg -d -q .sshpasswd.gpg > inlinepass; sshpass -f inlinepass ssh [email protected] 'hostname; df -h | grep sd; tail -2 /var/log/dpkg.log';
Welcome to Ubuntu 18.04.6 LTS (GNU/Linux 5.4.0-87-generic x86_64)
...
... output truncated ...
...
Last login: Sat Oct 23 08:58:39 2021 from 180.98.67.141
REMOTE-SERVER
/dev/sda1 98G 15G 79G 16% /
2021-10-19 05:17:21 status half-configured man-db:amd64 2.8.3-2ubuntu0.1
2021-10-19 05:17:24 status installed man-db:amd64 2.8.3-2ubuntu0.1First, the password gets decrypted and copied into the inlinepass variable. Further on, sshpass uses this variable to forward the password to the ssh session login prompt for successful login. Finally, the session authentication occurs and the system executes the commands.
4. Passwordless SSH
In this section, we’ll briefly touch upon creating a passwordless ssh between the server, and it further eases our job in logging into the remote SSH server.
4.1. Public and Private Key Generation
First, let’s set the passwordless ssh between the machine by creating a public and private RSA key pair for the specific user in that machine. With the ssh-keygen command, we can easily make the private key in the id_rsa file and the public key in id_rsa.pub. These files are placed under the .ssh directory in the $HOME path:
$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/localuser/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/localuser/.ssh/id_rsa.
Your public key has been saved in /home/localuser/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:d9AGZC4DdUPcgOwoX6duD65wxjFk2X7HU1gXWLjYxFg localuser@LOCAL-MACHINE
The key's randomart image is:
+---[RSA 2048]----+
| .o.=Oo+E=oo|
...
... output truncated ...
...
+----[SHA256]-----+4.2. Key Exchange and Execution
Now, let’s upload the local machine public key to the remote machine. With the help of the ssh-copy-id command, it can be shipped with the OpenSSH client package. After executing this command, the public key of the local machine will be automatically saved under the .ssh/authorized_keys file on the remote server:
$ ssh-copy-id [email protected] -p4455
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/home/tools/.ssh/id_rsa.pub"
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
[email protected]'s password:
Number of key(s) added: 1
Now try logging into the machine, with: "ssh -p '4455' '[email protected]'"
and check to make sure that only the key(s) you wanted were added.Once the underlying key exchanges are in place, then we are all good to execute the command or script on the remote server:
$ ssh -p '4455' '[email protected]’
Welcome to Ubuntu 18.04.6 LTS (GNU/Linux 5.4.0-87-generic x86_64)
...
... output truncated ...
...
tools@REMOTE-SERVER:~$ exit
logout
Connection to 10.149.20.11 closed.
$With the help of the passwordless ssh option, whenever we try to access the remote server using SSH protocol, it will no longer ask for a password:
$ ssh [email protected] 'hostname; df -h | grep sd; tail -2 /var/log/dpkg.log';
Welcome to Ubuntu 18.04.6 LTS (GNU/Linux 5.4.0-87-generic x86_64)
...
... output truncated ...
...
REMOTE-SERVER
/dev/sda1 98G 15G 79G 16% /
2021-10-19 05:17:21 status half-configured man-db:amd64 2.8.3-2ubuntu0.1
2021-10-19 05:17:24 status installed man-db:amd64 2.8.3-2ubuntu0.1
$5. Conclusion
In summary, we saw multiple ways to log in to the remote SSH server. Along the way, we also saw a detailed illustration of how we can use the expect script to access and execute the commands on remote machines. Additionally, autoexpect is like a feather in the cap as it automatically creates the expect script by recording our CLI interactions.
Further on, we also looked upon how sshpass helps the ssh tool in non-interactive logins and supports encrypted password file reference.
Finally, as a one-stop solution, we also explored steps for setting up the passwordless ssh between machines by exchanging keys. On the whole, depending on our use case, we may choose any one of these options to access and manage the remote servers.