Breaking up can be hard to do if the other party doesn’t want to let you go. People who move out of high-tax states may learn this the hard way — through a residency audit.

States such as New York, California and Illinois use the audits to claim that your recent interstate move was just a tax dodge and that you still owe their state income taxes. Proving you’ve actually moved and plan to make the new place your permanent home — yes, the burden of proof is on you in a residency audit — often requires far more than flashing your new driver’s license or spending a certain number of days outside the old state.

Who is most at risk

Technically, anyone who moves out of a high-tax state could face scrutiny, but tax experts say the residency audit risk increases if:

• You moved to a state with a much lower tax burden.

• You still have a home or business ties in the old state.

• You moved just before selling a business, a bunch of stock or some other valuable asset.

• You’re in a high tax bracket.

Wealthy people who move away from high-tax states are virtually certain to face a residency audit, says tax attorney Mark Klein, a partner at Hodgson Russ in New York. The stakes can be substantial: New York collected about $1 billion from residency audits from 2013 to 2017, according to Monaeo, a company that sells a location-tracking app for proving tax residency. More than half of the 3,000 or so people audited each year lose their cases, and the average amount collected per audit was $144,270, Monaeo calculated.

Auditors go where the money is. You’re unlikely to be audited if you’re already in a low tax bracket and cut all ties to your old state. But the more you have to gain from a move away from a high-tax state, the more careful you should be about making that move, tax experts say.

What really matters in a residency audit

Many people mistakenly believe they need only spend 183 days of each year outside their former state to win a residency audit, Klein says. But if you spend more days in the high-tax state than you do elsewhere, you could still be considered a resident. That can be a particular problem for the “migratory rich” who own homes in multiple states, or even for more ordinary people who travel a lot. Klein advises his clients to spend at least twice as much time in their new home state as in their old one.

Auditors look at a wide range of factors for evidence of where your true home lies. Are you still seeing doctors and dentists in your old location? Does your family celebrate holidays there? Where do you keep your most treasured items — your photo albums, family heirlooms, pets? Where’s your safe deposit box?

Create a good paper trail

Creating a substantial paper trail can be key to winning your case. Register to vote and get a driver’s license in your new state, but don’t stop there. You also should change vehicle registrations, update the address where you receive bank statements, bills and other mail and revise your estate-planning documents to reflect the laws of your new state.

People under residency audits typically need to prove where they were each day of the year in question, Klein says. Cell phone records — which can show where you were with each text or call — can be used by taxpayers to prove their case but also can be subpoenaed by the tax agency. Other potentially rich (and subpoenable) data sources include travel records, credit card receipts and toll collection devices, such as E-Z Pass.

You may need to maintain records indefinitely. Although most audits happen within a few years of the last tax return you filed, there’s often no statute of limitations if a state finds you should have filed a return but didn’t.

People at high risk of audit also should consult a tax professional who specializes in residency audits, especially if they’re keeping a home or business in their old state or if their move might not be their last. If you start in California and move to Nevada, but residency auditors don’t catch up to you until you’ve moved again to Arizona, your stay in Nevada could be deemed temporary and you could owe California taxes for that time period.

“You need to stick the landing,” Klein says.