The match was given the go-ahead after a lunchtime pitch inspection, with Rovers hoping to build on their shock victory over League Two Rochdale in round two.
Rams' caretaker boss Chris Hutchings handed a start to young defender Jason Beardsley after he returned from a loan spell with Notts County.
The home side made a flying start, with Adriano Rigoglioso having a shot charged down in the opening minute.
Rovers took a shock lead with 14 minutes on the clock, as midfielder Jonathan Smith unleashed a shot from nearly 25 yards which beat Rams goalkeeper Roy Carroll all ends up.
Unbelievably it was 2-0 six minutes further on as Alex Lawless took advantage of some slack defending from the visitors to place a low shot past the exposed Carroll.
The Rams pulled a goal back five minutes before the break as Rob Hulse headed home the rebound after Terry Burton had touched Kris Commons' free-kick onto the crossbar.
Right on half-time it was 2-2, as another Commons free-kick caused chaos in the home defence and Martin Albrechtsen bundled the ball home from close range.
After the restart, the home side took their time to recover from the blow of letting a two-goal lead slip, with Hulse threatening with a header as the Rams came close to taking the lead.
Home keeper Burton then had to deal with efforts from Hulse and Commons as Rovers came under sustained pressure.
However, they hit back with 18 minutes left on the clock as Paul Stonehouse made the score 3-2 to Jim Harvey's side.
Left-back Stonehouse moved forward to take a pass from Simon Clist and his scuffed right-foot shot somehow found its way past the rooted Carroll.
Just four minutes later it was all-square at 3-3 as Paul Green latched on to some good work by sub Nacer Barazite to leave Burton with no chance.
Three minutes from the end, Rovers' dreams died as Darren Jones was sent off for a foul on Commons and Steve Davies converted from the penalty spot to give Derby a place in round four.