Dizziness disorientation

Hi folks,

Does anyone have any experience of the following or know about it.

I recently got a ppl licence and generally are fine doing all the
flying type tasks required to safely take off fly and land.
One thing that seems to get me every time is if I am for example
flying straight and level at 2 or 3 thousand feet (height just as an
arbitory figure) and I encounter a crosswind the aircraft slowly but
surely starts to yaw as the wind hits the tail fin. this bit I
understand but the bit I don't is when this situation happens
I feel dizzy and disorientated for a few moments as the view from the
window in VFR starts to rotate.
I can reduce this by a bit of rudder to stop the rotation and keep the
ball in the centre.

Is this normal or should I not be flying?

thanks