The germans used the 'shrage musik' cannons to fire upwards (around 45°) for
night fighter use.
The japanese also often used this system on modified J2M3, Ki-45 and J1N1-S.
I also heard about 'shadow detection' upward firing experimental guns on
Fw190, perhaps other types (Bf110).

Has there ever been an aircraft with a gun permanently aligned
off-center? I don't mean swivel mounted, but permanently fixed at a
point other than straight ahead.

Wouldn't a gun pointed at a downward angle make ground attack easier?