performance tuning - Faster list duplicate 'limiter'

Given some list, e.g., {5, 3, 2, 3, 1, 0, 1, 3, 5, 5, 4, 5, 1, 2, 5, 2, 2, 0, 1, 4}, I need to limit the duplicates, if any, to an arbitrarily chosen number. The excess duplicates are to be chopped from the end of the list, order of list must be otherwise untouched. For example, with a limit of 2, the above example would become {5, 3, 2, 3, 1, 0, 1, 5, 4, 2, 0, 4}.

List can contain pretty much anything that would make sense. I'm using

Module[{o = Ordering@#},
   o[[o]] = Join @@ Range /@ Tally[#[[o]]][[All, 2]];
   Pick[#, UnitStep[#2 - o], 1]] &[yourListHere, dupeLengthLimitHere]

which is fairly quick, wondering if there's a better can-opener for this.

Answer

Thanks for the responses. It appears my original effort is appropriate for a pure MM implementation: some of the suggestions mirrored another method I tried that sometimes was slightly faster but at the expense of much heavier memory use, and often much slower.

For reference, code reposted:

Module[{o = Ordering@#},

   o[[o]] = Join @@ Range /@ Tally[#[[o]]][[All, 2]];
   Pick[#, UnitStep[#2 - o], 1]] &[yourListHere, dupeLengthLimitHere]