Speaker Paul Ryan said Wednesday the House will vote on whether to impeach Internal Revenue Service Commissioner John Koskinen, and he indicated members won't be pressured to vote one way or the other.

The entire House Republican conference is to meet behind closed doors next week to discuss the issue, but conservatives led by Rep. John Fleming of Louisiana say the process of introducing a "privileged resolution" to force the vote will begin before that.

Ryan, who has previously hedged on whether the House should take such action, said on Wednesday that he won't stop it. The House hasn't impeached a cabinet official since the mid-1870s and has never impeached an executive branch official, such as Koskinen, who doesn't hold cabinet rank. Any House action would be largely symbolic, because the effort is expected to be blocked in the Senate if it passes the House.

"This is a privileged resolution. So it will come to the floor," said Ryan during a news conference. He said there are members on both sides of the issue. "This is something where Congress is going to work its will."

Republicans remain angry at Koskinen, who they accuse of impeding an investigation into whether the tax agency improperly targeted conservative non-profits. Their allegations include failing to prevent the IRS from destroying evidence and providing false and misleading information to Congress. Koskinen and the Treasury Department have said the allegations are meritless.

In March 2014, despite congressional subpoenas seeking IRS emails, agency employees in West Virginia magnetically erased 422 backup tapes, eliminating as many as 24,000 e-mails. Subsequent investigations by the Justice Department and the Treasury Department's inspector general found that the destruction was accidental.

Regardless, Koskinen testified to Congress in June 2014 that "since the start of this investigation, every email has been preserved. Nothing has been destroyed." He has said since that his testimony reflected his understanding at the time.

Fleming said Wednesday that he and other members of the conservative House Freedom Caucus plan to move forward with their privileged resolution to force an impeachment vote "early next week," probably Tuesday. Once it's introduced, action must be taken within two days, and Fleming said he doesn't know if the vote will occur before the planned House Republican conference.

Fleming said he and other members are feeling no pressure to back off.

In fact, Fleming said that in a closed-door meeting of Republicans Wednesday morning, Ryan described the matter as "more or less a conscience vote. Something that we all have to decide on our own."

"My sense of it was is that there would be no persuading, no whipping, the usual thing we do on bills," said Fleming, who believes there are enough House votes for impeachment.

"Look, our constituents back home, Democrat or Republicans, are demanding we do something about corruption," he said.

But another Republican, Rep. Richard Hanna of New York, said in an interview that he believes the impeachment effort is "a waste of time." Hanna, who is not seeking re-election, noted that Koskinen came into office two weeks after Lois Lerner — the central figure in the controversy over the IRS targeting conservative groups — left the agency.

"They're angry at the IRS and I think they want to make a point," said Hanna. "But impeaching him — or holding him responsible — I'm not sure how that solves that."

Rep. John Conyers of Michigan, the top Democrat on the House Judiciary Committee, is among those who say that even if the House votes to impeach, the Senate wouldn't provide a two-thirds vote to convict him at trial.

In July, Fleming and Rep. Tim Huelskamp of Kansas had sought to force a vote one day before Congress left for its summer break, but they ran out of time to see it through.

That resolution contained four separate articles of impeachment. They included one accusing Koskinen of "engaging in a pattern of conduct showing he is unfit," including false statements to Congress.

The Treasury Department, which oversees the IRS, called the July impeachment effort baseless and a distraction, and added that Treasury Secretary Jack Lew continues to have full confidence in Koskinen.

Migrated