Opening a port isn't really bad if you have your firewall configured properly. You will have to open a port either way with jellyfin or wireguard. If you have a TLS/SSL certificate then just doing jellyfin is fine (but have good passwords since it's public facing), otherwise a VPN like wireguard will handle encryption for you.
As for managing traffic on the VPN you can follow this advice: https://serverfault.com/questions/1075973/wireguard-how-to-only-tunnel-some-of-the-traffic
Basically setup your firewall to stop extra traffic on your end, and change accessible IPs in wireguard to your service(s) so the peer knows not to talk on that interface for unrelated things.
If it's not as secure, then using a VPN is a great way to make sure everything works securely.