SSH over VPN
Managing servers on the public internet brings a lot of security threats with it. RDP is so bad it isn't recommended to even expose it to the public internet. With SSH, things are a bit more safe, but only after you perform a set of SSH hardening tasks, e.g.:
- Change default SSH port (security through obscurity, only helps reduce number of bots attempting to connect)
- Disable root login
- Disable password login, only use PubKey authentication
- Setup 2FA for SSH
- etc.
In order to add another layer of protection, you can setup VPN to protect your SSH connections (while leaving your website available to the public internet).
Having SSH over VPN shouldn't make you forget about the aforementioned SSH hardening steps. VPN is just one of them.