I played multiplayer for years now and after lot of testing just figured issue was with my IP for some reason, Those servers I never joined was not able to join as well, I switched to VPN and that seems to resolve the problem, I assume in this case could be NAT settings on my router or issue from ISP end