Is there a practical limit to wing loading?

I'm looking for aero dynamic information, not statements about max gross
weight. Can you load your glider up to the point that, even with strong and
reliable lift, you're at a disadvantage to lighter ships?

I was just thinking of the old days when I read that some contest pilots
tried, or considered, using salt water for ballast because it's heavier.