All I can tell you is that I typically get asked for my certificate
and medical when I cross the border and go through Canadian Customs.
Canada doesn't have a sport pilot, but they do honor US pilot
certificates by ICAO treaty or some such.

As for no transponder, I think that would be ok. People have flown
J-3's and such with no electrics up to Alaska and back. Just stay out
of higher classes of airspace that require them. No need for an FCC
radio license, that one has dropped away. They are in the process of
requiring a passport to reenter the US, but I don't think its a
requirement, YET.