Updated

Kody Brown and his four wives want what any family wants, to live in the privacy of their own home free from government intrusion, and out from under the threat of criminal prosecution for — as they see it — just loving each other.

The polygamous family, stars of the TLC show "Sister Wives," has sued Utah and the county they fled from, hoping to persuade a federal judge to overturn the state's bigamy law as unconstitutional.

The case could potentially decriminalize a way of life for tens of thousands of self-described Mormon fundamentalists, most of whom live in Utah where bigamy is a third-degree felony punishable by up to five years in prison.

The state, meanwhile, has publicly said it won't prosecute consenting adult polygamists unless there are other crimes involved, but insists the law doesn't overreach.

"It is not protected under religious freedom because states have the right to regulate marriage," said Paul Murphy, spokesman for Utah Attorney General Mark Shurtleff.

More On This...

A hearing was set for Wednesday on a motion to dismiss the case after prosecutors in Utah County, where the family had been living until last year, announced no criminal charges would be filed against the Browns under the state's bigamy statute.

The move came after Shurtleff assured the Browns they wouldn't be prosecuted by the state under his policy that consenting adult polygamists won't be charged as long as they're not committing other crimes.

U.S. District Judge Clark Waddoups then dismissed Shurtleff and Gov. Gary Herbert from the lawsuit, but allowed it to continue against the county since, at the time, prosecutors there hadn't officially adopted the same policy. That changed in May when Utah County Attorney Jeff Buhman announced the closing of his criminal investigation into the Brown's lifestyle, simultaneously adopting the state policy.

Buhman, who didn't return a call from The Associated Press, is seeking to have the lawsuit dismissed outright. He claims the Browns have no standing since they are no longer subject to prosecution.

No matter, claims their attorney, Washington, D.C., constitutional law professor Jonathan Turley.

Brown and his wives — Meri, Janelle, Christine and Robyn — remain victims and continue to live under the stigma of being considered felons, Turley said, noting they fled to Nevada last year.

It's no longer just about being prosecuted, Turley argued.

"The underlying statute is facially unconstitutional," he said.

Turley argues that under previous U.S. Supreme Court rulings, such as one that struck down Texas' sodomy law, private intimate relationships between consenting adults are constitutionally protected.

While all states outlaw bigamy, some like Utah have laws that not only prohibit citizens from having more than one marriage license, but also make it illegal to even purport to be married to multiple partners. Utah's bigamy statute even bans unmarried adults from living together and having a sexual relationship.

The latter provision could include same-sex couples, unmarried heterosexual couples and those, like the Browns, who don't have multiple licenses but live in a marriage-like relationship.

Their lawsuit doesn't aim to challenge Utah's right to refuse recognition of multiple licenses, nor are the Browns seeking them, Turley said. They just want to be left alone, he added, and not labeled felons.

In fact, Utah's statehood was granted in the 1890s under the condition that plural marriage — which was then openly practiced by members of The Church of Jesus Christ of Latter-day Saints — would be banned in the state constitution.

The practice has been illegal under federal law in the U.S. since the 1860s. In 1887 and 1890, the U.S. Supreme Court rejected legal challenges of the ban that argued that First Amendment protections should guarantee the right to continue the practice.

The Browns are members of the Apostolic United Brethren and say they practice polygamy as part of their religious beliefs.

And like most polygamists, Brown only has a valid marriage license with his first wife, Meri. He married the other three in religious ceremonies. They consider themselves "spiritually married."

Fewer than a dozen Utah cases have challenged the law — none successfully. It has most commonly been used to prosecute polygamists typically when the defendants were also charged with other crimes, like child abuse. The criminal investigation into Warren Jeffs, for instance, and his polygamous Fundamentalist Church of Jesus Christ of Latter Day Saints group, began as a probe into child sex abuse allegations.

He is now serving a life sentence in Texas after convictions on child sex and bigamy charges.

Some experts liken the issue to gay marriage or sodomy bans, but note neither carries the historical stigma that comes with polygamy.

"In many ways, the movement for the rights of polygamists has just not advanced enough to the point where the claim is going to seem legitimate to the courts," said Stanford University constitutional law expert Jane Schacter.

She noted there is "much more acceptance of gay couples, of homosexuality in general."

Polygamy "opens up a whole can of legal worms that is not really opened with the same sex marriage issue," Schacter said, noting polygamy throughout history has been associated with underage marriages and child sex abuse.

Regardless of whether there are polygamous relationships among consenting adults where no other crimes are occurring, the historical stigma persists, she said.

"It's not an issue I think will be laughed out of court, but I don't think it is poised for a victory at this point," Schacter added. "Contemporary law under the First Amendment regarding the exercise of religion is just not very friendly to the claims of polygamists."