Right of Way in the pattern?

Had this happen on Sunday up in Ontario. We were flying into Muskoka (a
non-towered field) in the Pilatus and were "cleared" in by Canadian
FSS. (We were IFR but in good VMC wx) We enter on the right downwind to
Rwy 18 and hear another acft on freq - apparently he's NE of the field
entering on the left base. We figure we're quite a bit faster than the
172 and should reach final first. Turns out we're both on opposite
bases at the same altitude at which point my partner says we're lower
and faster and we procede to turn final. Well, the guy in the 172 gets
a bug up his ass and is whining to FSS that he had to take "evasive
action" to avoid us - not even close - and makes some comment about
calling the RCMP (like they actually have jurisdiction here?). I know
the rule about the acft on final having right of way, but neither of us
had reached that point yet, and both acft were at the same altitude on
opposite bases. Who had the ROW?