The row broke out on Tuesday as Mr Key faced hostile questions from opposition MPs who don't think he's being tough enough with Australia over the treatment of New Zealanders with criminal records who are waiting to be deported.
Earlier, as Mr Key walked into parliament, Labour's Kelvin Davis tried to confront him and called out "you're gutless, prime minister".
In parliament, Mr Key said the government had repeatedly raised with Australia the plight of detainees and had been given assurances they could return to New Zealand if they wanted and appeal their deportations from there.
"Some of them are rapists, some of them are child molesters, some of them are murderers and these are the people the Labour Party is saying are more important to support than New Zealanders who need protecting when they come back here," he said.
"And Mr Davis - if you want to put yourself on the side of sex offenders go ahead, my son, but we will defend New Zealanders."
Following a question from new Green Party MP Marama Davidson, Mr Key repeated his earlier assertions.
"If those members want to protect sex offenders, rapists and murderers, go ahead. I'm not going to."
Mr Key said some of the detainees were so dangerous they couldn't be sent to New Zealand on commercial flights.
"What the Labour Party is saying is `to hell with the rest of New Zealand, these people should be put on a commercial aircaft'," he said.
Labour MPs complained to Speaker David Carter that they were "deeply offended" by Mr Key's remarks, and asked him to order the prime minister to apologise.
When he didn't, several MPs walked out leaving just 13 Labour members in their seats.
Labour leader Andrew Little later said Mr Key's behaviour had been "weak and gutless ... he has failed to stand up for what's right and his performance was a new low."
source: data archive