The engine speed should remain nearly the same, no matter what the load (within the engine's ability). Sounds like a governor issue - assuming there is nothing wrong with the deck, like bad bearings, belts, or crud under the deck. Check your deck first, then go from there. Even at the lower RPM's, does the engine run strong or weak?