IMO, one of the attributes of a good pilot is the ability to make (and carry
through with) decisions that in your best judgement are the safest course of
action - regardless of peer (and other) pressures.

I'd be a rich man if I had a dollar for everytime an aircraft comes out of
the care of engineers with a different set of issues than those it was being
maintained for - all too often (engineers included) folks default to
laziness and try to pretend the problem doesn't exist rather than making
their own (probably unpopular) decision and getting to the root cause of
it - at the end of the day it's NOT normal to have to slam a cargo door with
ever increasing velocity to get it to close properly.

You erred on the side of caution and safety because you saw something that
(a) "Was NOT the norm" and (b) "Could have had serious in-flight
consequences".

Good call in my opinion.