The Workhorse of Modern Healthcare Analytics

Administrative claims data represents the financial transactions generated every time a patient receives healthcare services—medical visits, procedures, hospitalizations, and pharmacy fills. These claims flow from providers to insurers, creating a comprehensive record of healthcare utilization. Medical claims capture all doctor visits, hospitalizations, and medical procedures regardless of the office or hospital in which the encounter occurred. Likewise, pharmacy claims capture all of a patient’s medications, regardless of which pharmacy dispensed the drugs. Claims represent the only practical means of generating a complete view of a patient’s medical treatment and as a result represent the de facto source of truth for a patient’s healthcare journey across the entire system.

The World is Awash in Claims Data

Whether for profit or public health, virtually every player in healthcare leverages claims data as their primary operational intelligence. Publicly-traded pharmaceutical manufacturers like Eli Lilly use claims for market research, sales force optimization, competitive intelligence, and to assess how well their drugs are working in the ‘real-world’. Health insurers rely on historical claims data for their actuarial tables, rate-setting, and financial forecasting, making it the backbone of their underwriting. State and federal government agencies operate their own claims ecosystems: each state administers its Medicaid program essentially as a government-run health insurer, while the federal government administers Medicare (I like to think of the US government as an insurance company with an army). Health tech companies across the spectrum—from point solutions like Sword Health (purchased by employers to prevent expensive musculoskeletal surgery) to direct-to-consumer apps like GoodRx—all depend on claims data to understand utilization patterns and identify opportunities to provide value.

What Do Medical Claims Actually Look Like?

Medical claims are generated each time a healthcare provider delivers services to a patient. After the encounter, the provider’s billing department translates the clinical documentation into a standardized claim format - a 1500 form if the claim is to be sent via mail or an 837 file if the claim is to be sent electronically (most commonly as of 2025). Each individual claim will contain the following information:

Although millions of individual claims are submitted each day, most analysts and data scientists won’t ever see an actual claim. The most common format of medical claims data are commonly referred to as claims extracts. These are all paid claims (which have been processed and settled by insurance providers) and come in large flat files - often CSVs or Excel files. The claims extracts that insurance carriers generate are guaranteed to look a bit different from one another, mostly in the column names and ordering of the columns will vary. However, the content is largely standardized between them.

No standard for storing medical claims has gained widespread adoption, perhaps because standardizing medical claims is trivial compared to other health data streams like EHR data. While both FHIR and OMOP Data Models (to be discussed at length in Chapter X) include dedicated tables for claims and cost data, these are used only in specific use cases and applications. In practice, claims data remains stored in proprietary relational schemas.

Warning! Don’t Trust The Diagnostic Information In Claims

The simplicity and comprehensiveness of medical claims can be enticing, as it looks like a long-term representation of a patient’s healthcare experience. While this data provides a comprehensive view of patient encounters across the healthcare system, the diagnostic codes in claims are very often inaccurate. Validation studies have consistently shown that diagnosis codes are often assigned to individuals who lack that diagnosis. HIV diagnosis codes are a classic example, as studies have found that only about 80% of patients with such a code in medical claims actually have the disease. In such cases, HIV is often incorrectly assigned as a diagnosis code for HIV screening procedures and never corrected. While workarounds exist - algorithms that combine multiple diagnosis codes have improved accuracy - the challenge of using medical claims as robust clinical indicators remain.

Quick Reference