The previous post in this series explained how password authentication works in the user authentication layer of the SSH-2 architecture , which enables the client to identify itself to the server, and be authenticated.

Another very commonly used authentication method is publickey authentication, which is based on public key encryption (discussed in an earlier post). Public key encryption uses two separate but related keys, known as the keypair. One key, known as the private key, is kept secret, and the other key, the public key, is made widely available.

Why is public key authentication so popular? The primary reason is that passwords are unnecessary – all that is required to authenticate with the server is the username and private key. This means that weak passwords that can easily be cracked are not an issue. This can have a downside, of course – the private key, which is usually kept in a file by the client application,  must be stored securely.

How does public key authentication work in SSH-2 to authenticate a user?

Firstly, the user’s public key must be registered against the username in the server, so that the server knows which public key corresponds to which user.

Next, the client digitally signs a block of data known to both client and server with the user’s private key.  This signature is sent to the server, along with the user’s public key. The server checks that the correct public key is being used for the named user (which it can easily do as the public key should be registered to this user), and then verifies the digital signature with the stored public key.

There are two commonly used public key algorithms – ssh-rsa and ssh-dss. Users could use either algorithm or both, i.e. they could have a public key registered for each algorithm.

The message the client sends to the server is shown below:

byte      SSH_MSG_USERAUTH_REQUEST
string    "enterprisedt" [user name]
string    "ssh-userauth" [service name]
string    "publickey"
boolean   TRUE
string    "ssh-rsa" [public key algorithm name]
string    public key to be used for authentication
string    signature

If the authentication attempt is successful, the server returns the following success message:

byte      SSH_MSG_USERAUTH_SUCCESS

Again, as in password authentication, at this point authentication is complete, and other services can be requested such as TCP/IP forwarding requests, channels for terminal access, process execution and subsystems such as SFTP. These are part of the connection protocol to be discussed in the next post in this series.