In a game of many twists and turns, Boro dominated the early stages and looked to have sewn up the points after they raced into a two-goal lead inside 12 minutes.
But Forest Green hit back and temporary boss Paul Wanless was delighted with the effort of his side, who grabbed their first goals and first point of the season with a performance which merited even more.
Stevenage began with a neat passing game which saw them tear into their visitors early on, and from the moment that they grabbed the opener on six minutes it appeared there could only be one result.
Steve Morison was the man who found the net, notching his fifth of the campaign when he out-muscled Alex Lawless after being put clear from a George Boyd flick-on and slotted the ball past Steve Williams in the Forest Green goal.
Stevenage continued to pile on the pressure and soon doubled their lead when Boyd crashed home a fierce volley from a Steve Guppy cross.
The hosts looked set to run riot but Rovers clawed their way back into the match and began to forage upfield, with Allan Russell testing Alan Julian in the home goal with two efforts.
Forest Green kept up the pressure and pulled one back on 24 minutes when Russell turned and fired in from the edge of the box after good work on the right from Alex Meechan.
With their confidence restored, the visitors went in search of an equaliser and got one seven minutes before the break when Russell fired the ball across the face of goal for Simon Clist to touch home.
The home side began the second half in the same way they started the first, but Craig Dobson squandered two great chances to regain the lead.
Stevenage were made to regret his lack of composure when Tony Butler rose to head home from a Meechan cross to put Rovers in front after 54 minutes.
Rovers looked to be on their way to victory when Steve Williams produced a stunning diving save to deny Morison.
However, a final cruel twist prevented the visitors heading home with maximum points, as Morison latched on to a loose ball and fired home to level the scores.