Higher-Order Extraction

Higher-Order Extraction returns values calculated from other fields in a document, including answers that aren’t printed on the page. Each calculated field comes back with the source values it used and where each source value appears on the page.

A University of South Carolina pay stub shows 54.17 hours and $839.64 in earnings but no hourly rate. Both values are highlighted, and the hourly rate is calculated as $839.64 divided by 54.17 hours, giving $15.50 per hour.

This feature is in beta. The playground and API call it calculated fields or computed fields. In the extraction playground, turn on Calculated Fields. For API requests, pass enable_computed_fields=true to /v2/extract.

By the end, we’ll have a script that sends a pay stub to /v2/extract, asks for an hourly rate that the stub doesn’t print, and saves the calculated rate with the hours and earnings it came from. Grab the full script from the dropdown below if you’d rather skip the walkthrough.

Show the Full Script

Set UNSILOED_API_KEY in your environment, and save the sample pay stub as pay-stub.png and the schema from Step 2 as schema.json in the same directory before running.

Python
cURL

extract_calculated_fields.py

import json
import os
import time

import requests

API_KEY = os.environ["UNSILOED_API_KEY"]
BASE_URL = "https://prod.visionapi.unsiloed.ai"

with open("schema.json") as f:
    schema = json.load(f)

with open("pay-stub.png", "rb") as f:
    response = requests.post(
        f"{BASE_URL}/v2/extract",
        headers={"api-key": API_KEY},
        files={"pdf_file": ("pay-stub.png", f, "image/png")},
        data={
            "schema_data": json.dumps(schema),
            "model": "alpha",
            "enable_computed_fields": "true",
            "enable_citations": "true",
        },
        timeout=120,
    )
response.raise_for_status()
job_id = response.json()["job_id"]
print(f"Job submitted: {job_id}")

# Calculated fields can take several minutes, so poll for up to 20 minutes.
deadline = time.time() + 20 * 60
while time.time() < deadline:
    job = requests.get(
        f"{BASE_URL}/extract/{job_id}", headers={"api-key": API_KEY}, timeout=60
    )
    job.raise_for_status()
    job = job.json()
    if job["status"] in ("completed", "failed"):
        break
    time.sleep(10)
else:
    raise TimeoutError(f"Job {job_id} is still running. Check it again later.")

if job["status"] == "failed":
    raise RuntimeError(f"Extraction failed: {job}")

with open("result.json", "w") as f:
    json.dump(job, f, indent=2)

rate = job["result"]["hourly_rate"]
print(f"hourly_rate: {rate['value']} ({rate['computed']['outcome']})")

# Submit the job and note the job_id in the response
curl --fail-with-body \
  "https://prod.visionapi.unsiloed.ai/v2/extract" \
  -H "api-key: $UNSILOED_API_KEY" \
  -F "pdf_file=@pay-stub.png;type=image/png" \
  -F "schema_data=<schema.json" \
  -F "model=alpha" \
  -F "enable_computed_fields=true" \
  -F "enable_citations=true"

# Repeat until "status" is "completed", replacing YOUR_JOB_ID
curl --fail-with-body \
  "https://prod.visionapi.unsiloed.ai/extract/YOUR_JOB_ID" \
  -H "api-key: $UNSILOED_API_KEY" > result.json

Step 1: Set Up Your Environment

We need an API key, the sample document, and Python’s requests package. The Extract quickstart covers the first and last of these in more detail.

1.1 Export Your API Key

Export your key as an environment variable named UNSILOED_API_KEY:

export UNSILOED_API_KEY="your-api-key"

1.2 Download the Sample Pay Stub

Download the sample pay stub and save it as pay-stub.png in your working directory. The stub’s Hours and Earnings table totals 54.17 hours and $839.64 for the pay period, but the stub doesn’t print an hourly rate. That’s the value we’ll calculate.

1.3 Install Dependencies

Python
cURL

Install the requests package:

pip install requests

Step 2: Describe the Calculation

A calculated field is an ordinary schema field whose description says how to combine other fields. It uses the same JSON Schema rules as the rest of the schema, and you don’t supply any executable code.

2.1 Add the Input Fields

Create a file called schema.json and add a field for each value the calculation needs:

schema.json

{
  "type": "object",
  "properties": {
    "current_hours": {
      "type": "number",
      "description": "Current Hours on the TOTAL row of the Hours and Earnings table."
    },
    "current_earnings": {
      "type": "number",
      "description": "Current Earnings on the TOTAL row of the Hours and Earnings table."
    },

These are normal extraction fields. Each description points at one printed value, so the API knows exactly which number to read.

2.2 Add the Calculated Field

Finish the file with the calculated field:

schema.json

    "hourly_rate": {
      "type": "number",
      "description": "Calculated field. Hourly pay rate in dollars: current_earnings divided by current_hours. The rate is not printed on the pay stub."
    }
  },
  "required": ["current_hours", "current_earnings", "hourly_rate"],
  "additionalProperties": false
}

The description does three jobs:

It names the input fields by their keys, current_earnings and current_hours.
It states the operation, dividing one by the other.
It says the value isn’t printed, so the API calculates it instead of searching the page for it.

In your own schemas, also include units and any rules the calculation depends on.

Step 3: Submit the Document

The request is a normal /v2/extract call with enable_computed_fields set to true. We also set enable_citations to true so that each input comes back with its location on the page.

3.1 Set Up the Script

Python
cURL

Create a file called extract_calculated_fields.py and start with the imports, configuration, and schema:

extract_calculated_fields.py

import json
import os
import time

import requests

API_KEY = os.environ["UNSILOED_API_KEY"]
BASE_URL = "https://prod.visionapi.unsiloed.ai"

with open("schema.json") as f:
    schema = json.load(f)

3.2 Upload the Pay Stub

Python
cURL

Next, upload the pay stub and schema, then print the job ID:

extract_calculated_fields.py

with open("pay-stub.png", "rb") as f:
    response = requests.post(
        f"{BASE_URL}/v2/extract",
        headers={"api-key": API_KEY},
        files={"pdf_file": ("pay-stub.png", f, "image/png")},
        data={
            "schema_data": json.dumps(schema),
            "model": "alpha",
            "enable_computed_fields": "true",
            "enable_citations": "true",
        },
        timeout=120,
    )
response.raise_for_status()
job_id = response.json()["job_id"]
print(f"Job submitted: {job_id}")

The file goes in the pdf_file form field even though it’s a PNG.

Run:

curl --fail-with-body \
  "https://prod.visionapi.unsiloed.ai/v2/extract" \
  -H "api-key: $UNSILOED_API_KEY" \
  -F "pdf_file=@pay-stub.png;type=image/png" \
  -F "schema_data=<schema.json" \
  -F "model=alpha" \
  -F "enable_computed_fields=true" \
  -F "enable_citations=true"

The file goes in the pdf_file form field even though it’s a PNG. Copy the job_id from the response for the next step.

Step 4: Poll for the Result

Calculated fields add a stage to the job. Processing time varies from seconds to several minutes, so allow a longer timeout than you would for ordinary extraction.

4.1 Write the Polling Loop

Python
cURL

Next, poll every 10 seconds for up to 20 minutes:

extract_calculated_fields.py

deadline = time.time() + 20 * 60
while time.time() < deadline:
    job = requests.get(
        f"{BASE_URL}/extract/{job_id}", headers={"api-key": API_KEY}, timeout=60
    )
    job.raise_for_status()
    job = job.json()
    if job["status"] in ("completed", "failed"):
        break
    time.sleep(10)
else:
    raise TimeoutError(f"Job {job_id} is still running. Check it again later.")

if job["status"] == "failed":
    raise RuntimeError(f"Extraction failed: {job}")

If the loop times out, keep the job ID and check its status later rather than submitting the document again.

Replace YOUR_JOB_ID with the value from Step 3.2 and run this command until status is completed:

curl --fail-with-body \
  "https://prod.visionapi.unsiloed.ai/extract/YOUR_JOB_ID" \
  -H "api-key: $UNSILOED_API_KEY" > result.json

4.2 Save and Print the Result

Python
cURL

Finally, save the full response and print the calculated rate:

extract_calculated_fields.py

with open("result.json", "w") as f:
    json.dump(job, f, indent=2)

rate = job["result"]["hourly_rate"]
print(f"hourly_rate: {rate['value']} ({rate['computed']['outcome']})")

Run the script:

python extract_calculated_fields.py

The output looks like this:

Job submitted: baf18251-7402-46e3-b921-c3d6d5b0637d
hourly_rate: 15.500092302012185 (calculated)

The polling command in Step 4.1 already saved the full response to result.json. The calculated rate is in result.hourly_rate.value.

Step 5: Read the Calculated Result

Open result.json. The hourly_rate field holds the answer, the inputs it used, and the checks the API ran.

5.1 Check the Outcome

"hourly_rate": {
  "value": 15.500092302012185,
  "citation": null,
  "computed": {
    "outcome": "calculated",
    "explanation": "Divides the current earnings by the current hours.",
    "inputs_used": 2,
    "inputs_empty": 0
  }
}

computed.outcome is calculated when the field has a value. Check it before you use value.
computed.explanation states the calculation in plain language.
computed.inputs_used and computed.inputs_empty count the inputs that had values and the inputs that were empty.
citation is null because the hourly rate isn’t printed anywhere to point to.

The value isn’t rounded. Round 15.500092302012185 to 15.50 in your application if you need a currency amount.

5.2 Trace the Inputs

"inputs": [
  {
    "path": "current_earnings",
    "value": 839.64,
    "citation": { "bbox": [520, 490, 556, 500], "page": 1, "page_width": 1118, "page_height": 923 }
  },
  {
    "path": "current_hours",
    "value": 54.17,
    "citation": { "bbox": [429, 490, 458, 500], "page": 1, "page_width": 1118, "page_height": 923 }
  }
]

Each input names its source field by path and gives the value used. The citation box shows where that value appears on the page, so you can verify the calculation against the document. An input’s citation is null when enable_citations is false. Drawing the two boxes on the sample pay stub highlights the TOTAL row values:

The Hours and Earnings table on the sample pay stub, with the current_hours box around 54.17 and the current_earnings box around 839.64 on the TOTAL row.

5.3 Review the Checks

"checks": [
  {
    "name": "printed_value",
    "detail": "no value printed as 'Hourly pay rate' was found; the judge does not find the value read for this field, 15.5, on the page",
    "result": "not_run"
  },
  {
    "name": "formulas_agree",
    "detail": "fewer than two formulas reached a figure to compare",
    "result": "not_run"
  }
]

printed_value compares the result against a matching value printed in the document, and formulas_agree compares alternative formulas. Both are not_run here, so neither validated this answer.

Handle Refused Calculations

A job can finish with status: "completed" while a calculated field is refused. For example, changing the hourly_rate description to ask for the rate “rounded to the nearest cent” returned this field:

"hourly_rate": {
  "value": null,
  "citation": null,
  "computed": {
    "cause": "not_expressible",
    "detail": "The operations do not include a way to round a number to the nearest cent.",
    "outcome": "refused"
  }
}

The beta doesn’t support every calculation. In testing, rounding and counting the days between two dates were both refused. A refused field has a null value, so treat it as missing, not as zero. Read cause and detail to decide whether to revise the field’s description or calculate the value in your application from the extracted inputs.

Getting Started

Document Processing

Integrations

FAQ

Higher-Order Extraction

Step 1: Set Up Your Environment

1.1 Export Your API Key

1.2 Download the Sample Pay Stub

1.3 Install Dependencies

Step 2: Describe the Calculation

2.1 Add the Input Fields

2.2 Add the Calculated Field

Step 3: Submit the Document

3.1 Set Up the Script

3.2 Upload the Pay Stub

Step 4: Poll for the Result

4.1 Write the Polling Loop

4.2 Save and Print the Result

Step 5: Read the Calculated Result

5.1 Check the Outcome

5.2 Trace the Inputs

5.3 Review the Checks

Handle Refused Calculations

​Step 1: Set Up Your Environment

​1.1 Export Your API Key

​1.2 Download the Sample Pay Stub

​1.3 Install Dependencies

​Step 2: Describe the Calculation

​2.1 Add the Input Fields

​2.2 Add the Calculated Field

​Step 3: Submit the Document

​3.1 Set Up the Script

​3.2 Upload the Pay Stub

​Step 4: Poll for the Result

​4.1 Write the Polling Loop

​4.2 Save and Print the Result

​Step 5: Read the Calculated Result

​5.1 Check the Outcome

​5.2 Trace the Inputs

​5.3 Review the Checks

​Handle Refused Calculations

Step 1: Set Up Your Environment

1.1 Export Your API Key

1.2 Download the Sample Pay Stub

1.3 Install Dependencies

Step 2: Describe the Calculation

2.1 Add the Input Fields

2.2 Add the Calculated Field

Step 3: Submit the Document

3.1 Set Up the Script

3.2 Upload the Pay Stub

Step 4: Poll for the Result

4.1 Write the Polling Loop

4.2 Save and Print the Result

Step 5: Read the Calculated Result

5.1 Check the Outcome

5.2 Trace the Inputs

5.3 Review the Checks

Handle Refused Calculations