In-form striker Colin Larkin hit his seventh goal in seven games to extend Mansfield's unbeaten run to four games.
Larkin pounced five minutes from time, drilling a low 12-yard drive through a crowded penalty area after the visitors failed to clear a Scott McNiven cross.
However three minutes later Dale appeared to be denied a clear penalty when Grant Holt's header from Jamie Clarke's free-kick was handled on the line by a Mansfield defender.
The decision was cruel on Rochdale, who had twice gone close to a late matchwinner themselves, only for Mansfield keeper Kevin Pilkington to pull out a vital save.
Dale had almost grabbed a fourth minute lead when Brian Cash, in his last game of a loan spell at the club, just failed to touch in Holt's right-wing cross.
A well-worked free-kick at the other end saw Neil McKenzie find Larkin, who skipped past a defender before firing into the sidenetting from a narrow angle.
This sparked the Stags into life and they then created four clear chances before half-time.
On the half-hour Larkin crossed to the near post where Derek Asamoah's clever flick was tipped over by Neil Edwards. From the resulting corner Dave Artell headed in, but was ruled offside.
Less than 60 seconds later Larkin shaved the outside of the post after a mazy run.
With the Stags dominating a slip by Daryl Burgess then allowed Asamoah a free shot at an empty net, but with keeper Edwards stranded the striker blazed high and wide.
In first-half injury-time Asamoah again squandered a great change shooting tamely at the keeper after neat passing from Larkin and Wayne Corden.
The second half was a copy of the first 45 minutes, with all the chances coming in the latter stages.
Dale substitute Leighton McGivern was denied a goal with his first touch in the 68th minute by Pilkington's brave save and two minutes later the keeper repeated his acrobatics to block Ernie Cooksey at the far post.
In the 74th minute as Mansfield hit back, Corden's acrobatic volley being pushed over by the flying Edwards, who was eventually beaten by Larkin's matchwinner.