Playing in their last game at The Lawn, Rovers knew they had to win and hope other results would go their way to stay up.
Thanks to goals from Bruno Teixeira and Julian Alsop the home side achieved victory and thankfully other results did go their way.
Fellow strugglers Southport were beaten 1-0 and both Tamworth and Scarborough drew, leaving Rovers safe.
Having not won a game in their last 11 outings, Rovers were faced with a mountain to climb.
A 5-1 win over Scarborough on February 11 was the last time Rovers had won before meeting Stevenage.
But knowing their only chance of survival was to win, Rovers rose to the occasion and can deservedly savour their superb rescue act.
Both teams were left with 10 men after Rovers' Darren Jones and Stevenage's Jefferson Louis were sent off for violent conduct after a needless clash.
However, the home side's win was never in doubt and thanks to Teixeira's and Alsop's smart finishes, full points were bagged when they were needed the most.
Stevenage started the better and Luke Oliver header inches wide in the sixth minute, before at the other end Guy Madjo fired wide for Rovers.
George Boyd missed the target for Stevenage in the eighth minute, before goalscorer Teixeira shot wide for Rovers five minutes later.
Adam Miller forced Rovers keeper Ryan Clarke to make a smart stop in the 15th minute and then five minutes later fired over as the visitors looked for the opener.
Jon Nurse headed wide for Stevenage in the 27th minute, before Rovers came close to scoring through Madjo, only for visiting keeper Alan Julian to make an excellent block.
Oliver headed wide at the other end, before Teixeira broke the deadlock with a superb strike two minutes before the interval.
The second-half started scrappily, but Alsop rose above the Stevenage defence to head home and double the home side's advantage.
Louis saw a shot saved by Clarke, while Teixeira had an effort blocked by Julian.
Jones and Louis were then sent off as the match turned ugly, but Rovers had done enough to stay up.