If a law enforcement officer asks to see your certificate and you don't
happen to have it with you, what is he going to do about it? Report to the
FAA that you were driving too fast without a pilot's license?

Keep your certificate wherever you can be sure it will be with you when you
are going flying, and stop worrying so much.