Rotherham manager Steve Evans said his side deserved to beat 10-man Cardiff by a bigger margin than 2-1 .
The Millers grabbed a much needed Sky Bet Championship win in added time through Matthew Connolly's own goal.
Vadis Odjidja-Ofoe and Peter Whittingham each converted from the spot to ensure the game went into half-time at 1-1 - Cardiff goalkeeper David Marshall was sent off for the incident that led to Rotherham's penalty and the opening goal two minutes before the break.
Evans said after his bottom-placed team's first league success: "The win has been coming for me.
"I think if it would have been anything other than a win it would have been a travesty. We've been the best side by a million miles from the first minute.
"The referee has got the two big decisions right.
"We played some super football and played on the front foot and as Rotherham United under Steve Evans. That was more like us.
On Marshall's dismissal, Evans added: "You can't kick out. He's been a bit foolish and he'll be disappointed with that.
"I thought we were battering them at 11 versus 11. They showed no interest in going forward in the second half and were camped behind the ball.
"The question was in the second half could we get the goal that our momentum deserved? You think it's not going to be our day.
"I think we were fantastic. We beat a top side full of quality players. We gave them a bit of a hiding."
Bluebirds boss Russell Slade said he could have no complaints about the dismissal but felt a foul should have been awarded prior to that after his keeper tangled with Rotherham's Matt Derbyshire and aimed a kick at him.
He said: "It's clearly a free-kick to ourselves because the player (Derbyshire) has pulled David Marshall and then David has lashed out. So yes it is a sending off - but it's not a penalty, it's a free-kick to ourselves.
"Should we have been down to 10 men? Yes we should. It's frustrating but he's left the 10 out there in a very difficult position.
"I'm sure he'll be very regretful of what's happened today. Occasionally players see the red mist.
"We didn't expect to be down to 10 men in the manner that we did. That put us in a difficult situation but I think we responded well to that and got straight back in the game.
"I thought we'd done enough in terms of digging in and defending as a group. We made big sacrifices to ensure that we could take a point back down the motorway but it wasn't to be.
"We reduced them to very little. I'll be honest, I thought they had run out of ideas. In all fairness there was very little else that troubled us."