Insurance Policy Data Extraction: Capture Coverage, Limits, Deductibles, and Exclusions
Extract coverages, limits, deductibles and endorsements from two insurance policies with Unsiloed, then use Python to compare the renewal with the expiring policy. Every reduced limit, raised deductible and new exclusion comes with the page it came from.
An insurance policy can renew with less cover than the year before. The insurer might lower the most it will pay, ask the business to pay more of each claim itself, or stop covering a kind of claim altogether. The new policy's summary page can look almost identical to last year's, because some of these changes sit in amendments further back in the PDF. We use the Unsiloed extraction API to pull the coverage terms from the old and new policies, then use Python to list every change with the page it came from.
What's in an Insurance Policy
The example follows a small business, Alder Workshop LLC, from its expiring 2026 policy to its 2027 renewal. Both are commercial general liability policies, which pay out when the business is blamed for injuring someone or damaging their property.
Each policy has three parts.
- The declarations page is a one-page summary. It names the business and the insurer, gives the dates the policy runs and its price (the premium), and has a table of what is covered.
- The coverage form is the full contract wording, which sets out what the insurer pays for and when. A list of every attached form, called the forms schedule, comes with it.
- Endorsements are amendments attached to the policy. Each one changes the contract, for example by lowering a payout or ruling out a kind of claim, and is identified by a form number and edition date.
Each row of the summary table names a type of cover and gives two amounts.
- A limit is the most the insurer will pay. A per-occurrence limit applies to each incident, and an aggregate limit caps the total across the policy year. A sublimit is a smaller cap on one kind of claim inside a larger limit.
- A deductible is the part of each claim the business pays before the insurer pays anything.
An exclusion is a kind of claim the policy doesn't pay for at all.
Why the Numbers Need Their Labels
Store each dollar amount with the type of cover it belongs to and whether it applies per incident or per year. General liability has both a per-occurrence limit and an aggregate limit, so each needs its own row.
Both summary pages show a USD 100,000 limit for damage to premises the business rents, but an amendment on page 3 of the renewal says this.
The Damage to premises rented to you limit shown in the Declarations is replaced by USD 50,000 each occurrence.
Reading the summary page alone would miss that cut. The amendment also says the USD 50,000 counts toward the general liability limit rather than adding to it, so adding the two amounts would overstate the cover. Apply each amendment to the row it names before comparing policies. The guide to reconciling contract amendments covers this pattern in more depth.
The renewal also adds this exclusion on page 4.
This insurance does not apply to bodily injury or property damage arising out of unauthorized access to, or disclosure of, confidential or personal information.
Keep the full wording. The amendment's title only names the form, and the wording says which claims are no longer covered.
Extract the Summary Page
A schema describes the fields and data types to extract. We extract the summary page and the amendments separately, so each change stays traceable to its own page.
Download the expiring policy and renewal into one folder. The snippets use Python with requests installed and an Unsiloed API key in the UNSILOED_API_KEY environment variable.
To start with saved results, download the expiring declarations, renewal declarations, expiring endorsements, and renewal endorsements into that folder and skip to the comparison.
Create extract.py in that folder. The amount description tells the model to read each limit as printed on the summary page, before any amendment changes it.
def field(kind, description):
return {"type": kind, "description": description}
def obj(properties):
return {"type": "object", "properties": properties,
"required": list(properties), "additionalProperties": False}
coverage = obj({
"name": field("string", "Coverage label exactly as printed"),
"limit_type": field("string", "Limit type as printed, not inferred"),
"amount": field("number", "Limit in USD on the declarations page, before endorsements"),
"deductible": field("number", "Deductible in USD in this row, including explicit zero"),
})
declarations_schema = obj({
"policy_number": field("string", "Policy number"),
"insurer": field("string", "Insurer name"),
"named_insured": field("string", "Named insured"),
"policy_period": field("string", "YYYY-MM-DD to YYYY-MM-DD"),
"premium": field("number", "Annual premium in USD"),
"coverages": {"type": "array", "items": coverage,
"description": "Every declarations table row in printed order. Do not apply endorsements."},
})
Append this request helper and declarations loop to extract.py. The extraction endpoint takes the PDF and schema and returns a job ID. We check the job every three seconds until it completes.
import json, os, time
import requests
BASE = "https://prod.visionapi.unsiloed.ai"
HEADERS = {"api-key": os.environ["UNSILOED_API_KEY"]}
def extract(path, schema):
with open(path, "rb") as f:
job = requests.post(f"{BASE}/v2/extract", headers=HEADERS,
files={"pdf_file": (path, f, "application/pdf")},
data={"schema_data": json.dumps(schema), "model": "gamma",
"enable_citations": "true"})
job.raise_for_status()
while True:
response = requests.get(f"{BASE}/extract/{job.json()['job_id']}", headers=HEADERS)
response.raise_for_status()
result = response.json()
if result["status"] in ("failed", "review"):
raise SystemExit(f"Check extraction job: {result['status']}")
if result["status"] == "completed":
return result["result"]
time.sleep(3)
for name in ("expiring", "renewal"):
result = extract(f"{name}.pdf", declarations_schema)
with open(f"{name}-declarations.json", "w") as f:
json.dump(result, f, indent=2)
gamma selects the extraction model, and enable_citations returns the page and box each value came from. Each saved file holds the completed job's result object.
Each field has a value and two confidence scores. The grounding_score expresses confidence in locating the value in the document. The extraction_score expresses confidence in the extracted value. Scores run from zero to one and are review signals, not proof that a value is correct.
Extract the Amendments With Their Wording
Each amendment record needs its form number and edition, so a reviewer can find it, plus what it changes and which cover it affects. The schema asks for the paragraph that makes the change, not just the amendment's entry in the forms schedule.
Append this schema and loop to extract.py. It covers the two kinds of change in these policies, adding an exclusion and changing a limit. Other kinds need their own fields. A null amount on an exclusion means no new limit applies, not a limit of zero. This schema reads amendments only. The coverage form has exclusions of its own, so if the renewal switches to a different edition of that form, extract its exclusions too before comparing.
endorsement = obj({
"form_number": field("string", "Form number without edition"),
"edition": field("string", "Edition as printed, MM YY"),
"title": field("string", "Title exactly as printed"),
"operation": {"type": "string", "enum": ["adds_exclusion", "changes_limit"]},
"coverage": field("string", "Affected coverage label exactly as printed"),
"limit_type": field(["string", "null"], "Limit type for a limit change, null for exclusions"),
"amount": field(["number", "null"], "New limit in USD, null for exclusions"),
"wording": field("string", "Verbatim operative paragraph, excluding headings and definitions"),
})
endorsements_schema = obj({
"endorsements": {"type": "array", "items": endorsement,
"description": "Every endorsement in document order, once each. Read its operative page, not just the schedule. Exclude the base coverage form."},
})
for name in ("expiring", "renewal"):
result = extract(f"{name}.pdf", endorsements_schema)
with open(f"{name}-endorsements.json", "w") as f:
json.dump(result, f, indent=2)
Run the file from the folder holding the PDFs.
python3 extract.py
The coverage list is in coverages["value"], and the endorsement list is in endorsements["value"]. Each declarations file has four coverage rows. The expiring policy has one amendment and the renewal has three.
Here is the amount field for the renewal's USD 50,000 cap, with its citation.
{
"score": {
"grounding_score": 0.99,
"extraction_score": 0.99
},
"value": 50000,
"citation": {
"bbox": [
455,
153,
487,
169
],
"page": 3,
"page_width": 612.0,
"page_height": 792.0
}
}
The bbox gives the left, top, right and bottom edges of the value, measured from the page's top-left corner, and page counts from 1. Scale it by page_width and page_height to draw it in a viewer. Labels the model infers, such as changes_limit, have no citation, so review them against the cited wording.
Compare the Expiring Policy With the Renewal
Apply each policy's amendments first, then compare like with like. Both policies come from the same insurer and use the same coverage names and currency, so rows can be matched directly.
Apply Each Policy's Amendments
Create compare.py next to the four saved JSON files. We key each row by coverage name and limit type, then replace a limit only where an amendment names that exact pair. Labels can pick up stray punctuation from the line they were printed on, so we trim it before matching. If two amendments target the same row, the script stops rather than guess which one applies.
import json
def load(path):
with open(path) as f:
return json.load(f)
def v(field):
return field["value"]
def label(field):
return v(field).strip().rstrip(".")
def key(row):
return label(row["name"]), v(row["limit_type"])
def policy(name):
declarations = load(f"{name}-declarations.json")
changes = v(load(f"{name}-endorsements.json")["endorsements"])
rows = v(declarations["coverages"])
coverages = {key(row): row for row in rows}
if len(coverages) != len(rows):
raise ValueError("Review duplicate coverage keys")
changed = set()
for change in changes:
if v(change["operation"]) != "changes_limit":
continue
target = label(change["coverage"]), v(change["limit_type"])
if target not in coverages or target in changed:
raise ValueError(f"Review endorsement target: {target}")
coverages[target]["amount"] = change["amount"]
changed.add(target)
exclusions = {(v(c["form_number"]), v(c["edition"])): c for c in changes
if v(c["operation"]) == "adds_exclusion"}
return declarations, coverages, exclusions
old, old_rows, old_exclusions = policy("expiring")
new, new_rows, new_exclusions = policy("renewal")
Replacing the whole field carries over the amendment's score and citation. The declarations file on disk keeps the original amount for the extraction audit trail. To compare policies from different insurers, map their coverage names to each other first.
Report the Changes
Append this to compare.py to collect reduced limits and raised deductibles, with the source field from each policy. A coverage that appears in only one policy stops the script for review.
changes = []
def evidence(field, document):
return {"value": v(field), "document": document, "citation": field["citation"]}
def record(label, before=None, after=None):
changes.append({"change": label,
"before": evidence(before, "expiring.pdf") if before else None,
"after": evidence(after, "renewal.pdf") if after else None})
for target in sorted(old_rows.keys() | new_rows.keys()):
if target not in old_rows or target not in new_rows:
raise ValueError(f"Review coverage present in only one policy: {target}")
before, after = old_rows[target], new_rows[target]
if v(after["amount"]) < v(before["amount"]):
record(f"Reduced limit | {target[0]} | {target[1]}", before["amount"], after["amount"])
if v(after["deductible"]) > v(before["deductible"]):
record(f"Raised deductible | {target[0]} | {target[1]}", before["deductible"], after["deductible"])
Append this to compare exclusions and the premium. We match exclusions by form number and edition, and record any matching form whose wording changed.
for form in sorted(old_exclusions.keys() | new_exclusions.keys()):
before, after = old_exclusions.get(form), new_exclusions.get(form)
if before is None:
record(f"Added exclusion | {' '.join(form)}", after=after["wording"])
elif after is None:
record(f"Removed exclusion | {' '.join(form)}", before=before["wording"])
elif v(before["wording"]) != v(after["wording"]):
record(f"Changed exclusion wording | {' '.join(form)}", before["wording"], after["wording"])
if v(old["premium"]) != v(new["premium"]):
record("Premium change", old["premium"], new["premium"])
Append this to save coverage-diff.json and print each change with its source location.
with open("coverage-diff.json", "w") as f:
json.dump(changes, f, indent=2)
def display(item):
if item is None:
return "Not listed in extracted endorsements"
value = item["value"]
value = f"USD {value:,.0f}" if isinstance(value, (int, float)) else value
citation = item["citation"]
return f"{value} [{item['document']} p.{citation['page']}, bbox {citation['bbox']}]"
for change in changes:
print(change["change"])
print(" Before: " + display(change["before"]))
print(" After: " + display(change["after"]))
Run the comparison from the same folder.
python3 compare.py
The saved extractions produce this output.
Reduced limit | Damage to premises rented to you | each occurrence
Before: USD 100,000 [expiring.pdf p.1, bbox [350, 518, 388, 531]]
After: USD 50,000 [renewal.pdf p.3, bbox [455, 153, 487, 169]]
Raised deductible | General liability | each occurrence
Before: USD 1,000 [expiring.pdf p.1, bbox [460, 430, 489, 443]]
After: USD 2,500 [renewal.pdf p.1, bbox [460, 430, 489, 443]]
Added exclusion | UP 21 09 01 27
Before: Not listed in extracted endorsements
After: This insurance does not apply to bodily injury or property damage arising out of unauthorized access to, or disclosure of, confidential or personal information. [renewal.pdf p.4, bbox [52, 336, 528, 367]]
Premium change
Before: USD 4,200 [expiring.pdf p.1, bbox [232, 285, 260, 298]]
After: USD 4,650 [renewal.pdf p.1, bbox [232, 285, 261, 298]]
Find the Right Pages in a Full Policy Package
The two policies so far are three and four pages long. The complete policy an insurer sends also includes a cover letter, the full contract wording, definitions and legal notices, and the summary page and amendments can sit anywhere among them. The full renewal package holds the same renewal in 15 pages, with the summary on page 2 and the amendments on pages 11, 13 and 14.
Rather than extracting the whole package, we use the Unsiloed split endpoint to sort its pages into categories first. Each category comes back as its own PDF, with a pages list of the package pages it holds. We extract the declarations and endorsements files with the same schemas, then use that list to map each citation back to its page in the package.
Download the package into the same folder and append this to extract.py.
def split(path, categories):
with open(path, "rb") as f:
job = requests.post(f"{BASE}/splitter", headers=HEADERS,
files={"file": (path, f, "application/pdf")},
data={"categories": json.dumps(categories)})
job.raise_for_status()
while True:
response = requests.get(f"{BASE}/splitter/{job.json()['job_id']}", headers=HEADERS)
response.raise_for_status()
result = response.json()
if result["status"] == "failed":
raise SystemExit("Check split job")
if result["status"] == "completed":
return result["result"]["files"]
time.sleep(3)
def remap(node, pages):
if isinstance(node, dict):
if isinstance(node.get("citation"), dict):
node["citation"]["page"] = pages[node["citation"]["page"] - 1]
for child in node.values():
remap(child, pages)
elif isinstance(node, list):
for child in node:
remap(child, pages)
categories = [
{"name": "declarations", "description": "Declarations page with the table of coverages, limits and deductibles"},
{"name": "endorsements", "description": "Endorsement pages that change the policy, each headed by a form number and edition"},
{"name": "other", "description": "Cover letter, forms schedule, base coverage form, conditions, definitions and notices"},
]
parts = {part["category"]: part for part in split("renewal-full.pdf", categories)}
for name, schema in (("declarations", declarations_schema), ("endorsements", endorsements_schema)):
part = parts[name]
with open(f"{name}.pdf", "wb") as f:
f.write(requests.get(part["full_path"]).content)
result = extract(f"{name}.pdf", schema)
remap(result, part["pages"])
with open(f"renewal-full-{name}.json", "w") as f:
json.dump(result, f, indent=2)
The split puts page 2 in declarations.pdf and pages 11, 13 and 14 in endorsements.pdf, so remap turns page 1 of endorsements.pdf into page 11 of the package. Running compare.py with policy("renewal-full") in place of policy("renewal") prints the same four changes, cited to pages 2, 11 and 14 of the package.
Flag What Needs a Person
Send these cases to a reviewer with the relevant pages.
- Low-scoring values. The guide to confidence score thresholds explains how to set review rules per field.
- Cover in only one version. It could be removed cover, a renamed row or a missing attachment, so check the forms schedule before calling it a removal.
- A limit with no clear type. An amount not marked as per incident or per year can't be compared with either, so keep it out of the diff until someone confirms it.
The output gives whoever reviews the cover, such as the business's insurance broker, a short list of changes to check. It doesn't replace that review.
