Action flowed thick and fast in an entertaining second half marred only be decisions from Jock Waugh that frustrated both sides.
Quinn was lucky only to be booked for bringing down Rovers striker Allan Russell on the edge of the box on 64 minutes. Quinn was not the last man but it is debatable whether Jake Wright would have been able to reach Russell in time as the front man bore down on goal.
Forest Green's on loan Morecambe striker Adriano Rigoglioso's blasted free-kick that resulted in Quinn's foul was deflected wide by the wall.
At the other end Halifax were denied a certain penalty by Mr Waugh when he waved away appeals after Martin Foster's shot appeared to be clawed off the line by a defender's hand on the line on 71 minutes.
In a frantic closing spell, both teams could have nicked a late winner. Halifax's Andy Campbell should have scored on 70 minutes when he was put through by substitute Shane Smeltz's first touch - but the ex-Middlesbrough striker wastefully blasted over.
Town's Greg Young had a header tipped over in stoppage time by Forest keeper Steve Williams and Rovers's ex-Halifax striker Alex Meechan went close as well.
It was in contrast to a more pedestrian opening period in which Halifax cruised to 2-0 lead.
Ryan Sugden opened the scoring on 24 minutes with a textbook striker's goal. He flicked on keeper Craig Mawson's goal-kick into Campbell's path and the ex-Cardiff front man hit the bar with an attempted lob, Sugden pounced on the rebound. The ball came down off the woodwork at speed from a difficult angle but Sugden managed to steer in a low shot through the legs of Christian Edwards on the line.
Campbell, who cost the Bluebirds £950,000 four years ago, could have made it 2-0 short after but his attempted chip floated wide after a storming win from deep through some static defending from Rovers.
The second goal, however, was not long in coming and it resulted it Rovers's persistence down the right on 34 minutes. He chased and won a lost cause, raced to the by-line and curled over a cross that unmarked Danny Forrest gleefully slammed in right-footed from close range.
A couple of half chances followed as Halifax turned the screw - only for the visitors to be given the lifeline of a penalty just before half-time.
Forest Green had started the game well, denying time and space on the ball to Halifax's players, and they could have taken a 13th minute lead. Town defender Young headed a long goal-kick by Rovers keeper Williams into the path of Adriano Rigoglioso but he blasted wide from about 20 yards.
The visitors seemed to go to pieces after Halifax scored but they were soon buzzing with confidence again when Young's trip on Meechan on the right-hand edge of the penalty box saw referee Mr Waugh point to the spot.
Kevin Nicholson rammed the penalty home high into the roof of the net on 42 minutes to give Forest Green some hope.
Only 7 minutes after the restart, the visitors were level through Pitman's strike. Chances galore followed at both ends but neither side were able to make a decisive breakthrough.