> ## Documentation Index
> Fetch the complete documentation index at: https://www.unsiloed.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Higher-Order Extraction

> Calculate fields from values in a document and inspect the inputs used for each result.

Higher-Order Extraction returns values calculated from other fields in a document, including answers that aren't printed on the page. Each calculated field comes back with the source values it used and where each source value appears on the page.

![A University of South Carolina pay stub shows 54.17 hours and \$839.64 in earnings but no hourly rate. Both values are highlighted, and the hourly rate is calculated as \$839.64 divided by 54.17 hours, giving \$15.50 per hour.](https://mintcdn.com/unsiloed/2Jrh-8CC6TGiSSl5/images/calculated-fields-demo.gif?s=3fe603e38b4a5d3e1717420d2693dca2)

> [!NOTE]
>
> This feature is in beta. The playground and API call it calculated fields or computed fields. In the [extraction playground](https://app.unsiloed.ai/playground/Extractor), turn on **Calculated Fields**. For API requests, pass `enable_computed_fields=true` to `/v2/extract`.

By the end, we'll have a script that sends a pay stub to `/v2/extract`, asks for an hourly rate that the stub doesn't print, and saves the calculated rate with the hours and earnings it came from. Grab the full script from the dropdown below if you'd rather skip the walkthrough.

> [accordion] Show the Full Script
>
> Set `UNSILOED_API_KEY` in your environment, and save the [sample pay stub](https://www.unsiloed.ai/docs/images/calculated-fields-pay-stub.png) as `pay-stub.png` and the schema from [Step 2](#step-2-describe-the-calculation) as `schema.json` in the same directory before running.
>
> > [tab] Python
> >
> > ```python extract_calculated_fields.py theme={null}
> > import json
> > import os
> > import time
> >
> > import requests
> >
> > API_KEY = os.environ["UNSILOED_API_KEY"]
> > BASE_URL = "https://prod.visionapi.unsiloed.ai"
> >
> > with open("schema.json") as f:
> >     schema = json.load(f)
> >
> > with open("pay-stub.png", "rb") as f:
> >     response = requests.post(
> >         f"{BASE_URL}/v2/extract",
> >         headers={"api-key": API_KEY},
> >         files={"pdf_file": ("pay-stub.png", f, "image/png")},
> >         data={
> >             "schema_data": json.dumps(schema),
> >             "model": "alpha",
> >             "enable_computed_fields": "true",
> >             "enable_citations": "true",
> >         },
> >         timeout=120,
> >     )
> > response.raise_for_status()
> > job_id = response.json()["job_id"]
> > print(f"Job submitted: {job_id}")
> >
> > # Calculated fields can take several minutes, so poll for up to 20 minutes.
> > deadline = time.time() + 20 * 60
> > while time.time() < deadline:
> >     job = requests.get(
> >         f"{BASE_URL}/extract/{job_id}", headers={"api-key": API_KEY}, timeout=60
> >     )
> >     job.raise_for_status()
> >     job = job.json()
> >     if job["status"] in ("completed", "failed"):
> >         break
> >     time.sleep(10)
> > else:
> >     raise TimeoutError(f"Job {job_id} is still running. Check it again later.")
> >
> > if job["status"] == "failed":
> >     raise RuntimeError(f"Extraction failed: {job}")
> >
> > with open("result.json", "w") as f:
> >     json.dump(job, f, indent=2)
> >
> > rate = job["result"]["hourly_rate"]
> > print(f"hourly_rate: {rate['value']} ({rate['computed']['outcome']})")
> > ```
>
> > [tab] cURL
> >
> > ```bash theme={null}
> > # Submit the job and note the job_id in the response
> > curl --fail-with-body \
> >   "https://prod.visionapi.unsiloed.ai/v2/extract" \
> >   -H "api-key: $UNSILOED_API_KEY" \
> >   -F "pdf_file=@pay-stub.png;type=image/png" \
> >   -F "schema_data=<schema.json" \
> >   -F "model=alpha" \
> >   -F "enable_computed_fields=true" \
> >   -F "enable_citations=true"
> >
> > # Repeat until "status" is "completed", replacing YOUR_JOB_ID
> > curl --fail-with-body \
> >   "https://prod.visionapi.unsiloed.ai/extract/YOUR_JOB_ID" \
> >   -H "api-key: $UNSILOED_API_KEY" > result.json
> > ```

## Step 1: Set Up Your Environment

We need an API key, the sample document, and Python's `requests` package. The [Extract quickstart](https://www.unsiloed.ai/docs/document-processing/extraction/quickstart.md#step-1-set-up-your-environment) covers the first and last of these in more detail.

### 1.1 Export Your API Key

Export your key as an environment variable named `UNSILOED_API_KEY`:

```bash theme={null}
export UNSILOED_API_KEY="your-api-key"
```

### 1.2 Download the Sample Pay Stub

Download the [sample pay stub](https://www.unsiloed.ai/docs/images/calculated-fields-pay-stub.png) and save it as `pay-stub.png` in your working directory.

The stub's Hours and Earnings table totals 54.17 hours and \$839.64 for the pay period, but the stub doesn't print an hourly rate. That's the value we'll calculate.

### 1.3 Install Dependencies

> [tab] Python
>
> Install the `requests` package:
>
> ```bash theme={null}
> pip install requests
> ```

> [tab] cURL
>
> cURL is preinstalled on macOS and most Linux distributions. No other packages are needed.

## Step 2: Describe the Calculation

A calculated field is an ordinary schema field whose `description` says how to combine other fields. It uses the same [JSON Schema rules](https://www.unsiloed.ai/docs/document-processing/extraction/schemas.md) as the rest of the schema, and you don't supply any executable code.

### 2.1 Add the Input Fields

Create a file called `schema.json` and add a field for each value the calculation needs:

```json schema.json theme={null}
{
  "type": "object",
  "properties": {
    "current_hours": {
      "type": "number",
      "description": "Current Hours on the TOTAL row of the Hours and Earnings table."
    },
    "current_earnings": {
      "type": "number",
      "description": "Current Earnings on the TOTAL row of the Hours and Earnings table."
    },
```

These are normal extraction fields. Each description points at one printed value, so the API knows exactly which number to read.

### 2.2 Add the Calculated Field

Finish the file with the calculated field:

```json schema.json theme={null}
    "hourly_rate": {
      "type": "number",
      "description": "Calculated field. Hourly pay rate in dollars: current_earnings divided by current_hours. The rate is not printed on the pay stub."
    }
  },
  "required": ["current_hours", "current_earnings", "hourly_rate"],
  "additionalProperties": false
}
```

The description does three jobs:

* It names the input fields by their keys, `current_earnings` and `current_hours`.
* It states the operation, dividing one by the other.
* It says the value isn't printed, so the API calculates it instead of searching the page for it.

In your own schemas, also include units and any rules the calculation depends on.

## Step 3: Submit the Document

The request is a normal `/v2/extract` call with `enable_computed_fields` set to `true`. We also set `enable_citations` to `true` so that each input comes back with its location on the page.

### 3.1 Set Up the Script

> [tab] Python
>
> Create a file called `extract_calculated_fields.py` and start with the imports, configuration, and schema:
>
> ```python extract_calculated_fields.py theme={null}
> import json
> import os
> import time
>
> import requests
>
> API_KEY = os.environ["UNSILOED_API_KEY"]
> BASE_URL = "https://prod.visionapi.unsiloed.ai"
>
> with open("schema.json") as f:
>     schema = json.load(f)
> ```

> [tab] cURL
>
> cURL doesn't need a setup step. Each command below includes the API key and base URL.

### 3.2 Upload the Pay Stub

> [tab] Python
>
> Next, upload the pay stub and schema, then print the job ID:
>
> ```python extract_calculated_fields.py theme={null}
> with open("pay-stub.png", "rb") as f:
>     response = requests.post(
>         f"{BASE_URL}/v2/extract",
>         headers={"api-key": API_KEY},
>         files={"pdf_file": ("pay-stub.png", f, "image/png")},
>         data={
>             "schema_data": json.dumps(schema),
>             "model": "alpha",
>             "enable_computed_fields": "true",
>             "enable_citations": "true",
>         },
>         timeout=120,
>     )
> response.raise_for_status()
> job_id = response.json()["job_id"]
> print(f"Job submitted: {job_id}")
> ```
>
> The file goes in the `pdf_file` form field even though it's a PNG.

> [tab] cURL
>
> Run:
>
> ```bash theme={null}
> curl --fail-with-body \
>   "https://prod.visionapi.unsiloed.ai/v2/extract" \
>   -H "api-key: $UNSILOED_API_KEY" \
>   -F "pdf_file=@pay-stub.png;type=image/png" \
>   -F "schema_data=<schema.json" \
>   -F "model=alpha" \
>   -F "enable_computed_fields=true" \
>   -F "enable_citations=true"
> ```
>
> The file goes in the `pdf_file` form field even though it's a PNG. Copy the `job_id` from the response for the next step.

## Step 4: Poll for the Result

Calculated fields add a stage to the job. Processing time varies from seconds to several minutes, so allow a longer timeout than you would for ordinary extraction.

### 4.1 Write the Polling Loop

> [tab] Python
>
> Next, poll every 10 seconds for up to 20 minutes:
>
> ```python extract_calculated_fields.py theme={null}
> deadline = time.time() + 20 * 60
> while time.time() < deadline:
>     job = requests.get(
>         f"{BASE_URL}/extract/{job_id}", headers={"api-key": API_KEY}, timeout=60
>     )
>     job.raise_for_status()
>     job = job.json()
>     if job["status"] in ("completed", "failed"):
>         break
>     time.sleep(10)
> else:
>     raise TimeoutError(f"Job {job_id} is still running. Check it again later.")
>
> if job["status"] == "failed":
>     raise RuntimeError(f"Extraction failed: {job}")
> ```
>
> If the loop times out, keep the job ID and check its status later rather than submitting the document again.

> [tab] cURL
>
> Replace `YOUR_JOB_ID` with the value from Step 3.2 and run this command until `status` is `completed`:
>
> ```bash theme={null}
> curl --fail-with-body \
>   "https://prod.visionapi.unsiloed.ai/extract/YOUR_JOB_ID" \
>   -H "api-key: $UNSILOED_API_KEY" > result.json
> ```

### 4.2 Save and Print the Result

> [tab] Python
>
> Finally, save the full response and print the calculated rate:
>
> ```python extract_calculated_fields.py theme={null}
> with open("result.json", "w") as f:
>     json.dump(job, f, indent=2)
>
> rate = job["result"]["hourly_rate"]
> print(f"hourly_rate: {rate['value']} ({rate['computed']['outcome']})")
> ```
>
> Run the script:
>
> ```bash theme={null}
> python extract_calculated_fields.py
> ```
>
> The output looks like this:
>
> ```text theme={null}
> Job submitted: baf18251-7402-46e3-b921-c3d6d5b0637d
> hourly_rate: 15.500092302012185 (calculated)
> ```

> [tab] cURL
>
> The polling command in Step 4.1 already saved the full response to `result.json`. The calculated rate is in `result.hourly_rate.value`.

## Step 5: Read the Calculated Result

Open `result.json`. The `hourly_rate` field holds the answer, the inputs it used, and the checks the API ran.

### 5.1 Check the Outcome

```json theme={null}
"hourly_rate": {
  "value": 15.500092302012185,
  "citation": null,
  "computed": {
    "outcome": "calculated",
    "explanation": "Divides the current earnings by the current hours.",
    "inputs_used": 2,
    "inputs_empty": 0
  }
}
```

* `computed.outcome` is `calculated` when the field has a value. Check it before you use `value`.
* `computed.explanation` states the calculation in plain language.
* `computed.inputs_used` and `computed.inputs_empty` count the inputs that had values and the inputs that were empty.
* `citation` is `null` because the hourly rate isn't printed anywhere to point to.

The value isn't rounded. Round `15.500092302012185` to `15.50` in your application if you need a currency amount.

### 5.2 Trace the Inputs

```json theme={null}
"inputs": [
  {
    "path": "current_earnings",
    "value": 839.64,
    "citation": { "bbox": [520, 490, 556, 500], "page": 1, "page_width": 1118, "page_height": 923 }
  },
  {
    "path": "current_hours",
    "value": 54.17,
    "citation": { "bbox": [429, 490, 458, 500], "page": 1, "page_width": 1118, "page_height": 923 }
  }
]
```

Each input names its source field by `path` and gives the value used. The `citation` box shows where that value appears on the page, so you can verify the calculation against the document. An input's `citation` is `null` when `enable_citations` is `false`.

Drawing the two boxes on the sample pay stub highlights the TOTAL row values:

![The Hours and Earnings table on the sample pay stub, with the current\_hours box around 54.17 and the current\_earnings box around 839.64 on the TOTAL row.](https://mintcdn.com/unsiloed/2Jrh-8CC6TGiSSl5/images/calculated-fields-input-citations.png?fit=max\&auto=format\&n=2Jrh-8CC6TGiSSl5\&q=85\&s=16440107dd77ca8bf29d9656bbdf1ed1)

### 5.3 Review the Checks

```json theme={null}
"checks": [
  {
    "name": "printed_value",
    "detail": "no value printed as 'Hourly pay rate' was found; the judge does not find the value read for this field, 15.5, on the page",
    "result": "not_run"
  },
  {
    "name": "formulas_agree",
    "detail": "fewer than two formulas reached a figure to compare",
    "result": "not_run"
  }
]
```

`printed_value` compares the result against a matching value printed in the document, and `formulas_agree` compares alternative formulas. Both are `not_run` here, so neither validated this answer.

## Handle Refused Calculations

A job can finish with `status: "completed"` while a calculated field is refused. For example, changing the `hourly_rate` description to ask for the rate "rounded to the nearest cent" returned this field:

```json theme={null}
"hourly_rate": {
  "value": null,
  "citation": null,
  "computed": {
    "cause": "not_expressible",
    "detail": "The operations do not include a way to round a number to the nearest cent.",
    "outcome": "refused"
  }
}
```

The beta doesn't support every calculation. In testing, rounding and counting the days between two dates were both refused. A refused field has a `null` value, so treat it as missing, not as zero. Read `cause` and `detail` to decide whether to revise the field's description or calculate the value in your application from the extracted inputs.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.