While Attorney General Eric Holder's Capitol Hill appearance Wednesday was marked by a heated argument with his chief Republican critic, more and more Democrats are joining the fray -- in a sign that the Justice Department's seizure of journalists' phone records may trigger a bipartisan challenge to his leadership.

Rep. Zoe Lofgren, D-Calif., was among the Democrats who sharply questioned Holder Wednesday on the controversy. She questioned why and how Holder was able to recuse himself from that case, and said it appears the department's actions "have, in fact, impaired the First Amendment."

"Reporters who might have previously believed that a confidential source would speak to them would no longer have that level of confidence, because those confidential sources are now going to be chilled in their relationship with the press," she said.

The toughest criticism of Holder is still coming from Republicans. To date, Holder has fended off a succession of Republican-led efforts to oust him. At the hearing Wednesday, he dismissed a recent call from the GOP chairman for his resignation as ill-informed.

And on Thursday, President Obama backed up his long-time ally.

"I have complete confidence in Eric Holder as attorney general," he said at a press conference, defending the importance of leak investigations as well as press freedom. Obama added that he would make "no apologies" on the issue of pursuing leaks.

But the volume of Democratic complaints may be rising.

Liberal radio host Bill Press took the left's criticism to a new level Thursday when he publicly called for Holder to be forced out.

On his Twitter account, Press wrote: "What 'breach of national security' are we talking about re the AP story? It's BS and Holder should be fired."

Making clear the tweet wasn't just a slip of the keyboard, he then wrote: "I'll repeat what I said earlier. POTUS should fire Eric Holder."

"POTUS" is shorthand for the president.

Democratic lawmakers have stopped short of calling for Holder's resignation, but have pressed him for answers on why the department found it necessary to go around the back of AP bosses and obtain their phone records -- which was done as part of an investigation into sensitive intelligence leaks.

Holder, in defending the records grab, has called the leak one of the most serious he's ever seen, while noting that he recused himself from the case and his deputy is actually the one leading the investigation.

Other Democrats have been more defensive of Holder, and suggested the latest controversy would be used for partisan purposes. Rep. Jerrold Nadler, D-N.Y., said it was Republicans who demanded the leak investigation in the first place.

"Now, of course, it is convenient to attack the attorney general for being too aggressive or the Justice Department for being too aggressive," he said.

Holder similarly went after Republicans for their scrutiny on Wednesday. He lashed out at Rep. Darrell Issa, R-Calif., after he accused Holder of purposely and repeatedly keeping information from Congress.

"No, that's what you typically do," Holder responded. Following crosstalk, Holder added, "That is inappropriate and is too consistent with the way in which you conduct yourself as a member of Congress. It's unacceptable and it's shameful."

Issa led the charge in Congress against Holder and his department over the botched Operation Fast and Furious anti-gun-trafficking program. Republicans eventually voted to hold him in contempt of Congress over resistance to providing certain documents pertaining to that case.

But Holder had since been able to keep his head down -- until the Associated Press revealed this week that Justice had secretly sought its phone records.

Prominent Democrats swiftly joined Republicans in questioning whether that step was necessary.

"The burden is always on the government when they go after private information -- especially information regarding the press or its confidential sources. I want to know more about this case, but on the face of it, I am concerned that the government may not have met that burden," Senate Judiciary Committee Chairman Patrick Leahy, D-Vt., said Monday.