How to Fix SSH Permission Denied (publickey) Error in Ansible: A Deep Guide

Introduction

When working with Ansible, a common and frustrating error is “SSH Error: Permission denied (publickey)”. This problem usually arises when Ansible, which relies on SSH to manage remote servers, fails to authenticate using a public key. SSH is the cornerstone of Ansible’s agentless architecture, and if it cannot establish a connection, your automation tasks will not execute properly.

This in-depth guide will walk you through every possible cause of this error, provide practical fixes ranging from basic to advanced, and cover common SSH configurations that might be the root of the issue. Whether you are new to Ansible or a seasoned user, this guide will help you navigate and resolve SSH permission problems, ensuring uninterrupted connectivity and workflow automation.

What Is the “Permission Denied (publickey)” Error?

In simple terms, the “Permission denied (publickey)” error occurs when the SSH client (in this case, Ansible) fails to authenticate the connection with the remote server using a public key. Ansible uses SSH to communicate with managed nodes, and if the public key authentication is denied, Ansible will be unable to execute its playbooks on the remote servers.

Common Causes of SSH Permission Denied (publickey) in Ansible

Here are the most frequent reasons why you may encounter this error:

• No SSH key pair exists on the control machine.
• Incorrect permissions on your private or public SSH key.
• The public key is not copied to the remote server or it is not located in the correct directory.
• SSH agent not loaded with the correct key.
• Misconfiguration of the ansible_user or ansible_ssh_private_key_file in the inventory file.
• SSH key forwarding issues, particularly when using SSH from a jump host or a bastion.
• SSH key mismatches between different environments, especially if you’re managing multiple servers.

Let’s explore each of these in detail, along with the solutions to fix them.

Basic Troubleshooting Steps for SSH Permission Denied (publickey)

Before diving into advanced configurations and Ansible-specific fixes, it’s important to start with basic troubleshooting steps. These are often enough to resolve the problem.

1. Verify SSH Key Pair Exists on Control Node

To establish an SSH connection, the control node (your local machine) needs to have an SSH key pair. Run the following command to verify if an SSH key already exists:

ls ~/.ssh/id_rsa

If the file doesn’t exist, create a new key pair:

ssh-keygen -t rsa -b 4096

This command generates a 4096-bit RSA key pair, which is suitable for most modern applications. Make sure not to overwrite an existing key unless necessary.

Why Do You Need an SSH Key Pair?

SSH key pairs are critical for Ansible to securely connect to remote servers without a password prompt. If no key pair exists, Ansible won’t be able to authenticate with remote servers, leading to the “Permission denied (publickey)” error.

2. Ensure Correct Permissions on SSH Keys

SSH will reject your connection if the private key (id_rsa) or public key (id_rsa.pub) files have overly permissive permissions. To fix this, set the appropriate permissions on both files:

chmod 600 ~/.ssh/id_rsa
chmod 644 ~/.ssh/id_rsa.pub

This restricts access to the private key to the current user and allows public reading of the public key.

Why Does SSH Require Strict Permissions?

SSH ensures that your private keys are secured. If the permissions are too permissive, the key may be accessible by other users on the system, which creates a security risk. Thus, SSH enforces strict permission rules to safeguard key usage.

3. Copy Public Key to Remote Server

If the public key is not present on the remote server, you won’t be able to authenticate via SSH. Use the ssh-copy-id command to upload the public key:

ssh-copy-id user@remote_server

This command will append your public key to the remote server’s ~/.ssh/authorized_keys file, which is necessary for key-based authentication.

4. Test the SSH Connection Manually

Before attempting to run your Ansible playbooks, manually verify that you can establish an SSH connection:

ssh user@remote_server

If the connection succeeds, then Ansible should also be able to communicate with the remote host. If not, the issue likely lies within your SSH configuration.

Intermediate Ansible-Specific Solutions

If you’ve completed the basic troubleshooting steps and are still encountering the “Permission denied (publickey)” error, the issue might be specific to your Ansible configuration.

1. Set the Correct SSH User in Ansible Inventory

Ansible’s inventory file defines which hosts to connect to and how to connect to them. If the ansible_user is incorrect or missing, Ansible might try to use the wrong user to connect via SSH.

Here’s an example of a correct inventory entry:

[servers]
server1 ansible_host=192.168.1.10 ansible_user=user

In this example, Ansible will attempt to connect to the server using the user account. Make sure the SSH user is the one authorized to log in via SSH on the remote machine.

Incorrect User? Fixing Ansible User Issues

Often, the SSH user set in the Ansible inventory file doesn’t match the authorized user on the remote server. Ensure that the user specified as ansible_user is the correct one.

2. Specify Private Key Path in Inventory

If Ansible is using the wrong private key for authentication, specify the correct private key in your inventory file:

[servers]
server1 ansible_host=192.168.1.10 ansible_user=user ansible_ssh_private_key_file=~/.ssh/id_rsa

By explicitly telling Ansible which key to use, you can avoid situations where it attempts to use the wrong key.

3. Check SSH Agent and Add Key if Necessary

Ansible relies on the SSH agent to manage private keys. If your key isn’t added to the agent, you can add it with the following commands:

ssh-agent bash
ssh-add ~/.ssh/id_rsa

To verify that the key is loaded, run:

ssh-add -l

This command will list all SSH keys currently managed by the SSH agent. Ensure your key appears in the list.

Why Use SSH Agent?

The SSH agent allows Ansible to manage private keys efficiently without prompting for a password each time it connects to a remote server. If the agent is not loaded, Ansible may fail to connect, resulting in the permission denied error.

Advanced Troubleshooting Techniques

If the error persists after performing basic and intermediate troubleshooting, it’s time to delve into more advanced techniques.

1. Increase SSH Verbosity for Detailed Debugging

To gain more insights into why the SSH connection is failing, increase the verbosity of Ansible’s SSH output by running playbooks with the -vvvv option:

ansible-playbook -i inventory playbook.yml -vvvv

This command enables verbose mode and prints detailed logs that show exactly what’s happening during the SSH authentication process. Look for specific messages related to public key authentication.

2. Check the Remote Server’s authorized_keys File

Sometimes, the public key on the remote server might be corrupted or misconfigured. Check the ~/.ssh/authorized_keys file on the remote server and ensure that:

• The public key is listed correctly.
• There are no extra spaces or invalid characters.

3. Use paramiko SSH Backend in Ansible

By default, Ansible uses OpenSSH as the SSH backend. In certain cases, switching to paramiko can help resolve authentication issues. You can configure this in your Ansible playbook or inventory file by adding:

ansible_ssh_common_args: '-o StrictHostKeyChecking=no'

Alternatively, to force paramiko for all connections, modify your ansible.cfg:

[defaults]
transport = paramiko

4. Forward SSH Key (If Using Jump Hosts)

If you are connecting to remote servers via a jump host or bastion, you may need to forward your SSH key to the remote server. Enable SSH key forwarding by adding this to your inventory file:

[servers]
server1 ansible_host=192.168.1.10 ansible_user=user ansible_ssh_extra_args='-o ForwardAgent=yes'

Key forwarding allows the remote server to use your SSH credentials from the jump host, solving authentication problems that arise in such scenarios.

Common SSH Configuration Issues and Fixes

1. Missing SSH Configurations

If you’re managing multiple SSH keys or servers, it’s beneficial to configure ~/.ssh/config. Here’s an example configuration:

Host server1
  HostName 192.168.1.10
  User user
  IdentityFile ~/.ssh/id_rsa

This configuration ensures that the correct user and key are used for specific hosts.

2. Incorrect File Permissions on Remote Server

Check the permissions of the ~/.ssh/authorized_keys file on the remote server:

chmod 600 ~/.ssh/authorized_keys
chown user:user ~/.ssh/authorized_keys

These commands set the correct ownership and permissions for the file, ensuring SSH can authenticate using the stored public key.

FAQs

Why am I still getting “Permission denied (publickey)” even after verifying permissions?

Ensure that your SSH agent is running and that the correct private key is loaded into the agent. Also, double-check the public key is copied correctly to the remote server’s authorized_keys file.

How can I debug SSH key authentication issues?

Use the following command for verbose debugging of SSH connections:

ssh -i ~/.ssh/id_rsa user@remote_server -v

This will provide detailed output about each step in the authentication process.

Can I disable public key authentication and use passwords?

While you can configure password-based authentication in SSH, it’s not recommended for production environments due to security risks. If necessary, you can enable password authentication in the SSH configuration, but this should be a last resort.

Conclusion

The “SSH Error: Permission denied (publickey)” is a common issue in Ansible, but by following this deep guide, you now have a range of solutions at your disposal. Whether the problem lies in SSH key permissions, Ansible inventory configurations, or advanced SSH setups like key forwarding, these strategies will help you resolve the error and ensure smooth automation with Ansible. Thank you for reading the DevopsRoles page!

By mastering these techniques, you can overcome SSH authentication problems and maintain a reliable, scalable infrastructure managed by Ansible.

Ansible, DevOps