The IRS From the Inside: What Most Taxpayers Never Know

The Internal Revenue Service employs roughly 80,000 people. It processes more than 260 million tax returns annually. It collects approximately $4.7 trillion in revenue each year.

And yet, for most taxpayers, the IRS is a black box — an institution they interact with reluctantly, understand poorly, and fear disproportionately.

That fear is understandable. It is also, in most cases, misplaced.

Having worked inside the agency, we can tell you that the IRS is not a monolithic enforcement machine. It is a large bureaucracy with limited resources, specific priorities, and well-defined processes. Understanding how it actually works — not how it is portrayed in popular culture — is one of the most valuable things a taxpayer can know.

The most persistent myth about IRS audits is that they are random.

They are not.

The IRS uses a scoring system called the Discriminant Information Function (DIF) to rank returns by their audit potential. Returns with high DIF scores — those that deviate significantly from statistical norms for similar taxpayers — are more likely to be selected for review.

What drives a high DIF score? Among the most common factors:

Unusually high deductions relative to income. If your charitable contributions, business expenses, or miscellaneous deductions are significantly higher than the average for your income level, the return will stand out.

Unreported income. The IRS receives information returns — W-2s, 1099s, K-1s — from thousands of payers. When the income on those documents does not match what appears on your return, the discrepancy is flagged automatically.

Schedule C losses. Self-employment income and losses are among the most audited items on individual returns, particularly when losses appear year after year or when the ratio of expenses to income is unusually high.

Large cash transactions. Banks are required to file Currency Transaction Reports for cash transactions over $10,000. Patterns of cash activity that do not align with reported income attract attention.

Certain industries. Some industries — restaurants, car washes, nail salons, and other cash-intensive businesses — receive heightened scrutiny because of historically high rates of underreporting.

Understanding these triggers does not mean avoiding legitimate deductions. It means documenting them thoroughly and being prepared to substantiate them if asked.

IRS auditors are not trying to find fraud in most cases. They are trying to verify that the return accurately reflects the taxpayer's income and deductions.

The most common audit issues are not criminal. They are documentation failures — deductions that were legitimate but cannot be substantiated because the taxpayer did not keep adequate records.

What auditors want to see:

• Receipts and invoices for business expenses
• Mileage logs for vehicle deductions
• Bank statements that corroborate reported income
• Written acknowledgment letters for charitable contributions over $250
• Contemporaneous records for travel and entertainment expenses

The word contemporaneous matters. Records created after the fact — reconstructed from memory or assembled in response to an audit notice — carry far less weight than records created at the time of the transaction.

The single most effective audit defense is documentation created before the audit ever begins.

Not all audits are created equal.

Correspondence audits are conducted entirely by mail. The IRS sends a letter asking for documentation of a specific item — a charitable deduction, a business expense, a claimed credit. You respond with the requested documentation. These are the most common type of audit and, for well-documented returns, the most manageable.

Office audits require you to appear at an IRS office with your records. They are more comprehensive than correspondence audits but still focused on specific issues.

Field audits involve an IRS agent visiting your home or business. These are reserved for complex returns — typically businesses with significant revenue, high-net-worth individuals, or cases where the IRS suspects significant underreporting. Field audits are the most thorough and the most consequential.

Knowing which type of audit you are facing — and responding appropriately — is critical. Many taxpayers make the mistake of providing more information than requested in a correspondence audit, inadvertently opening issues that were not originally under review.

Fear of the IRS is often based on misconceptions about its powers.

The IRS cannot:

• Arrest you for a tax debt. Criminal prosecution for tax evasion is handled by the Department of Justice, not the IRS, and requires willful intent to defraud. Owing taxes — even significant amounts — is not a crime.
• Seize your assets without notice. The IRS must follow a specific process before levying wages or seizing property, including multiple notices and an opportunity to respond.
• Ignore an installment agreement. If you enter into a payment plan and comply with its terms, the IRS is bound by that agreement.
• Audit you indefinitely. The standard statute of limitations for audit is three years from the filing date. It extends to six years if the IRS believes you omitted more than 25% of your gross income, and there is no limit in cases of fraud — but the standard window is three years.

Understanding these limits reduces fear and enables rational decision-making when dealing with the agency.

When you receive an IRS notice or face an audit, you have the right to be represented by a CPA, Enrolled Agent, or attorney.

You should exercise that right.

Not because the IRS is adversarial — in most cases, it is not. But because the process has rules, and knowing those rules changes outcomes.

An Enrolled Agent or CPA who has worked inside the IRS understands:

• How to communicate with the agency in the language it responds to
• Which issues are worth contesting and which are not
• How to present documentation in the format auditors expect
• When to escalate to Appeals and when to accept a proposed adjustment
• How to negotiate installment agreements, offers in compromise, and penalty abatements

That knowledge is not available in a self-help guide. It comes from experience — ideally, from experience on both sides of the table.

Our team includes former IRS and US Treasury professionals. We have sat across the table from taxpayers as agents. We now sit beside taxpayers as advocates.

That experience gives our clients an advantage that no amount of software or generic tax advice can replicate.

If you are facing an audit, a notice, or a complex tax situation, the most valuable thing you can do is speak with someone who has been inside the agency. Not to game the system — but to navigate it with precision, confidence, and the knowledge that comes from having seen it from the inside.

Related Services: Individual Tax Preparation | Small Business Tax & Accounting | Contact Us

Facing an IRS notice or audit? Contact us for a free consultation with former IRS and Treasury professionals.