I got a message like “Server refused our key” when I tried to SSH my server using a PuTTY with a RSA key.

After a log check in /var/log/auth.log (Ubuntu):

sshd[2910]: Authentication refused: bad ownership or modes for file /home/user/.ssh/authorized_keys

Ownership

Ownership of .ssh folder and authorized_keys file should be:

drwx------ 2 user user 4096 Aug 24 16:30 .ssh
-rw------- 1 user user 753 Aug 24 16:30 authorized_keys

Where user user is your username.

If it’s not the case you can fix ownership thanks to chown command:

cd
chown user:user .ssh
chown user:user .ssh/authorized_keys

Permissions

You can fix permissions using chmod command.

cd  
chmod 700 .ssh
chmod 600 .ssh/authorized_keys

That’s it =)