From what I'm seeing on the MS and Skype support sites, I don't think you are going to be able to restrict standard Skype at all with firewall rules. They basically reserve the right to go to any ip address on the internet from your site to connect these calls.
They offer some hope of supported restrictions if you subscribe to Office 365 Skype Business, but even that seems to be limited support.
The only sure fire way seems to be to have Skype for Business combined with an internal Skype server at your site. And the setup on that seems a little complicated.