If the aircraft is banked and a component of the lift is then
horizontal, why doesn't the aircraft just go sideways
over into the next county? We have to bring gravity, centrifugal
force, and the effect of the tail feathers into this picture.
We need a good mental picture of what is happening.......

"The math" doesn't give a "good mental picture".
Gravity provides the thrust for a glider. It is sliding downhill.

Centrifugal force is provided by part of the lifting force of the
wings. It causes the glider to turn because it is always at
right angles to the direction of flight. Don't think of it as pulling
you sideways but rather as pulling you round. (sic.)

The "tail feathers": The rudder is used to counteract the adverse
yaw of the wings caused by the differences of drag on the two
wings during turning flight.

The elevator is used to counteract the loss of some of the
"upwards" lift being used to create "inwards" lift (towards the
center of the circle) during banking. This loss is made
up for by increasing the angle of attack.

The use of the rudder and elevator during turning is not entirely
necessary but it does make flying safer and more efficient.

David