This article originally appeared in Bloomberg Tax. 

Just because you think you moved from New York doesn’t mean the state or city see it that way. Nishant Mittal of Topia explains the “automated desk audit” and why taxpayers who thought they had parted ways with the state or city may have received a notice challenging a claimed change of residency.

In May, former residents of New York state and New York City who claimed to be nonresidents in 2020 began to receive computer-generated letters from Albany inquiring about whether they had, in fact, “moved.” Two things were very unusual about these letters.

First, they were sent less than a month after tax day. Normally, residency audits aren’t initiated until two to three years later.

Second, they were automated desk audits. A typical residency case results in a field audit with teams of auditors who visit your residence(s) and review your records in person. The point of a desk audit is to cast a wide net and see who it catches.

If you received a letter demanding 10% to 15% of your income—what New York’s highest earners pay in combined state and city income tax—you would debate it. Thousands of New Yorkers are about to be in that situation if they aren’t already.

Many people believe that if you spend 183 or more days working outside of New York, you’re not a resident and therefore don’t owe state and city taxes. That is the statutory residency test. It is important but insufficient. To not pay New York state and New York City taxes, your move must be indefinite in the eyes of state auditors. You must demonstrate the intention to relocate your permanent home, or in tax-speak, your domicile.

The criteria for domicile change are subjective. New York auditors consider five primary factors: the existence of a residence in New York, your participation in New York businesses, where you spend your time, where you keep things “near and dear” to your family (e.g., family heirlooms, photos, pets, etc.), and where your immediate family members are located.

If you left New York in April 2020 but return in September 2021, New York’s tax auditors will argue that you never moved in the first place. As New York-based tax lawyer Timothy Noonan put it, “You didn’t stick the landing.”

Noonan says that the number of callers asking about New York residency and tax is up 30-fold this year. Of those inbound callers, Noonan reports that about 70% believe they have indeed changed their domicile and won’t need to pay New York taxes. Among New York residents who requested a change of address with the U.S. Postal Service between March and November 2020, about 333,000 left the state entirely. That is a lot of tax revenue to lose in one year.

About 30% of Noonan’s callers realize after the conversation that they didn’t really “move” in the eyes of the law. If your Manhattan apartment housed your family members and prized possessions throughout 2020 while you chased 183 days in Florida, don’t bother fighting. Suck it up and pay your New York taxes.

The desk audits tell us that Albany is determined to prevent New Yorkers from changing their residency status incorrectly. How many New Yorkers will get audited, and how many will lose?

Through a Freedom of Information Act (FOIA) request, my colleagues and I at Topia found that New York conducted 15,122 residency audits between 2013 and 2017. Audits tend to occur two to three years after the tax year of concern, so returns filed between 2011 to 2015 most likely triggered them. In those years, New York’s five boroughs saw net out-migration of about 450,000 people according to U.S. Census Bureau data. That means more than 450,000 people left, but this gives us a conservative baseline.

New York state won 52% of those residency audits and collected about $1 billion in total. Now that hedge-fund managers and other Wall Street titans are fleeing New York taxes—and making a show of it—the potential upside for Albany is likely higher.

If the number of residency audits is correlated with migration, then the 330,000 people who left New York in 2020 ought to trigger over 11,000 residency audits for 2020 alone, and at least half will lose their audit—with an important caveat.

Historically, residency audits were all field audits and hence limited by the number of auditors available. Desk auditing enables New York to scale up the number of residency audits without hiring significantly more personnel. Albany is bound to conduct a record number of residency audits.

Thanks to the state budget deal struck in April, New York City’s high earners face the highest income tax in America. Many, with encouragement from their employers, will leave because of it. If you’re among them, plan to spend 183 days outside the state and talk to a lawyer about how to pass the domicile test. Be ready to prove where you were located each day of the year.

In the short term, New York might recover significant income tax revenue through residency audits. In the long term, its high earners have never been more motivated—and more able—to pick up and leave permanently. The return of in-person work won’t save New York if it takes place in Florida.

New Yorkers, beware: If you want to move to another state in order to reduce your tax bill, you actually have to move. And whether you actually move or not, you’ll probably hear from the auditors. What do they have to lose?