WSL2 doesn't seem to play well with some VPNs, in this case "Global Protect".
TLDR:
Make sure IPv6 is enabled, and create %UserProfile%\.wslconfig with
[WSL2]
networkingMode=mirrored
WSL1 works better, but that wont run Docker, and is now deprecated. The issue seems to be that the WSL2 networking is unusual. It doesn't (currrently) show up in any of the networking utilities, and the VM doesn't show up in the HyperV manager. So it's invisibly somewhere, doing some form of NAT that isn't normal NetNat... So it appears to work when it goes out the normal ethernet interface, but doesn't NAT and thus fails when it gets routed out of the soft interface of the VPN. It appears that IPv6 is needed for "mirrored" to work properly. Noone seems to know why, but MS say it's mandatory now. So that's that.