1. Overview
In this tutorial, we discuss four commonly used RSA public key serialization formats, their backgrounds, examples, and applications in the real world.
2. Introduction
Cryptography studies secure communication through insecure or secure channels and techniques for preventing unauthorized access to information. There are two categories in cryptography:
- Private key cryptography (Symmetric key cryptography)
- Public key cryptography (Asymmetric key cryptography)
In symmetric key cryptography, the two entities should share the key through a secure channel before sharing the message. The history of symmetric key cryptography goes back hundreds of years. The Caesar cipher is an example of an ancient symmetric key cipher that Julius Caesar used.
The concept of public key cryptography was introduced a few decades ago, in 1976, by Whitfield Diffie and Martin Hellman.
In public key cryptography, there are two keys: private and public keys. The encryption key is shared publicly, so it is called the public key. However, the decryption key is secret and kept private, so it is called the private key. Using the public key, the message can be passed through an insecure channel.
Since 1976, many public key cryptosystems have been proposed. But few of them are widely used and have become more popular. RSA public key cryptosystem is the most popular and widely used among them.
3. RSA Algorithm
RSA is a public key cryptosystem invented by Ronald Rivest, Adi Shamir and Leonard Adleman in 1977. It was the first practical public key scheme.
3.1. RSA Key Generation
The pseudocode for RSA key generation is as follows:
3.2. RSA Message Encryption
The pseudocode for RSA message encryption is as follows:
3.3. RSA Message Decryption
The pseudocode for RSA message decryption is as follows:
3.4. The Use of the Keys
The use of the public key and the private key in the RSA algorithm is as follows:
Next, we’ll discuss some of the most commonly used RSA public key (that is, 
) serialization formats.
4. Supported RSA Public Key Formats
Depending on the application, there are various formats in which the RSA keys can be stored. In this section, we discuss four RSA public key serialization formats. The supported formats we discuss here are as follows:
- PEM (Privacy-enhanced Electronic Mail)
- DER-encoded (Distinguished Encoding Rules-encoded)
- SSH (the Secure Shell public key file)
- SSH2 (an improved version of the SSH format)
4.1. PEM Format
The most commonly used storage format is the PEM (Privacy-Enhanced Electronic Mail) format. A PEM file is a text file with one or more elements encoded in ASCII Base64 with headers and footers in plaintext. Here, we consider the PEM-encoded (X509 SubjectPublicKeyInfo) public key and the PEM-encoded PKCS#1 (Public Key Cryptography Standard #1) RSAPublicKey. (You may find their ASN.1 (Abstract Syntax Notation One) definitions in RFC 5280 and RFC 8017, respectively.). This article uses a 1024-bit RSA key. The key size determines the security strength of a cryptosystem. Note that the RSA certificate with 1024 bits is outdated and not recognized by browsers. We use a 1024-bit RSA key only for demonstration purposes. At the time of writing, the 2048-bit RSA certificate is the typical key length in real-world use. Considering security, 2048-bit RSA keys offer sufficient security for now. However, a 3072-bit RSA key is recommended for beyond 2030. The derived public key in PEM format (X509 SubjectPublicKeyInfo) is as follows:
This is the X509 SubjectPublicKeyInfo format that OpenSSL uses by default when generating a public key. OpenSSL is a software library for applications. It provides secure communication over computer networks. OpenSSL also supports several various cryptographic algorithms. OpenSSL is a highly helpful open-source command line toolkit for handling cryptography keys and X.509 (its internet form, PKIX) certificates. The derived public key in PEM format (PKCS#1) is as follows:
Considering the RSA public key in PEM format examples, PKCS#1 has “BEGIN RSA PUBLIC KEY” and “END RSA PUBLIC KEY” in the header and the footer, respectively. Whereas X509 SubjectPublicKeyInfo has “BEGIN PUBLIC KEY” and “END PUBLIC KEY”. Furthermore, we can also demonstrate the public key and the private key in PEM using the RSA module with OpenSSL as follows:
Email and other text-based communication frequently use the PEM format.
4.2. DER-encoded Format
The DER-encoded (Distinguished Encoding Rules-encoded) format is a binary encoding method for data described by ASN.1. Therefore, it is sometimes called the ASN.1 DER-encoded format. In general, to obtain the DER format is to convert the certificate from another format, like PEM, to DER using a capable conversion tool. The following represents the above public key in PEM format in the DER-encoded structure with the OpenSSL command:
Note that the RSA public key in PEM format has plaintext statements such as:
-----BEGIN CERTIFICATE-----
-----END CERTIFICATE-----But the RSA public key in DER format doesn’t have such plaintext statements.
DER-encoded format is often used in smart cards and other embedded devices.
4.3. SSH Format (The Secure Shell Public Key File Format)
SSH is a secure network protocol. Using SSH, we can connect with a remote machine and execute commands on the machine. The RSA public key format used by OpenSSH is as follows:
Considering the RSA public key in the SSH format, it does not have headers or footers like PEM or SSH2 formats do. Furthermore, RFC4253 thoroughly describes the OpenSSH public key format.
The SSH format is primarily used for secure remote login and server command execution.
4.4. SSH2 Format (Improved Version of SSH)
The SSH2 format, also known as the RFC4716 format, is a more secure, efficient, and portable SSH version.
The following shows the RSA public key in SSH2 format:
The RSA public key in SSH2 format looks like the RSA public key in the PEM format that we discussed previously. But note that, in the SSH2 format, the header and the footer sentences have four dashes and a space.
The SSH2 format is commonly used in modern systems and networks.
5. Conclusion
In this article, we discussed different RSA public key formats: PEM format, DER format, SSH format, and SSH2 format. The SSH and SSH2 formats are often used to authenticate the identities of interactive users and remote systems. Both PEM and DER formats are used for exchanging public keys and storing public keys in files. Moreover, text-based communication frequently uses the PEM format. Smart cards and other embedded devices use the DER format. Considering security, it is recommended to use 2048-bit RSA keys.