> ## Documentation Index
> Fetch the complete documentation index at: https://docs.costory.io/llms.txt
> Use this file to discover all available pages before exploring further.

# BigQuery jobs

> Beta: attach BigQuery job metadata to GCP cost, reallocate slots to the user who ran each query, and see on-demand and reservation spend in one view.

> [!NOTE]
>
> BigQuery jobs is in beta.

The GCP billing export shows BigQuery compute as one **Analysis** line, billed per TiB scanned on demand, plus a separate reservation line if you buy slots. Neither line says who ran the queries, which SQL they ran, or how much slot time they used.

BigQuery jobs enrichment (Costory reads BigQuery job metadata and attaches it to GCP Analysis billing lines so you can group that cost by user, statement, destination table, and related job fields.) reads job metadata from the projects you grant and attaches it to those billing lines. Costory then reallocates slot cost to the user who actually ran the job. On-demand Analysis and reservation spend sit in the same [Explorer (Costory's main interface for exploring cloud costs across all connected providers.)](/features/cost-explorer.md) view, grouped by `cos_query_user`, statement, or destination table.

This is a different connector from [BigQuery](/setup/usage-metrics/bigquery.md). That page queries any table live as an external metric. This page attaches jobs to the GCP bill.

Use this when you need to:

* See all BigQuery compute in a single view: on-demand and slots, attributed to the person or service account that ran the query
* Attribute cost to a user, service account, statement type, or destination table
* Build unit economics (Financial metrics expressed on a per-unit basis: cost per customer, cost per API call, cost per transaction.) such as cost per query, cost per TiB billed, or slot-hours per TiB
* See spend on failed or stopped queries
* Estimate on-demand vs Editions slots per workload
* Ask Costory Agent why a query used too many slots, from the SQL and the expensive stages

## Key terms used in this article

* **Analysis**: the GCP billing SKU for on-demand BigQuery compute.
* **BigQuery jobs enrichment**: job metadata attached to those billing lines, including slot reallocation to the user who ran the job.
* **Unlabelled**: cost with `cos_query_user` empty, usually a project Costory cannot list jobs on.
* **Compute intensity (Slot-hours per TiB billed. High intensity means the query used a lot of slots relative to the data it scanned.)**: slot-hours per TiB billed.

See the [Glossary](/docs/glossary.md) for a full list of terms.

## Prerequisites

* A [GCP billing export](/setup/billing/gcp.md) already connected to Costory.
* Admin access in Costory.
* Permission to grant IAM on **every GCP project that runs queries**, not only the billing-export project.

## Connect BigQuery jobs

**Manual setup**

**1. Open BigQuery jobs**

Go to [Integrations](https://app.\[REDACTED]/integration), click **+ Add integration**, then under **External metric** choose **GCP**. On **Choose what to connect**, pick **BigQuery jobs**. The connect URL is `/integration/new/gcp/bigquery-jobs`. You need to be an admin. 

**2. Grant job access on every query project**

Copy the **Service account** from the connect screen. Grant **BigQuery Resource Viewer** (`roles/bigquery.resourceViewer`) on every project that runs queries, so Costory can list all jobs (`bigquery.jobs.listAll`). Access on the billing-export dataset is not enough.

Run the command shown on the page, or paste this and replace the placeholders:

```bash theme={null}
gcloud projects add-iam-policy-binding PROJECT_ID \
  --member="serviceAccount:SERVICE_ACCOUNT_EMAIL" \
  --role="roles/bigquery.resourceViewer"
```

Repeat for each project that submits jobs. A project without this grant still has cost, and that cost shows up as Unlabelled.

**3. Enter connection details**

Set **Connection name** and **GCP project ID**. Click **Validate**, then **Save connection**. **Save connection** stays disabled until validation succeeds.

**4. Group cost by user in Explorer**

After the next billing refresh, open [Explorer](/features/cost-explorer.md), filter to BigQuery, and group by `cos_query_user`. On-demand Analysis and reallocated slot cost appear in the same query. Add usage metrics such as `query_count` or `query_compute_ms` in **Advanced Explorer**.

> [!NOTE]
>
> BigQuery keeps job history for 180 days. Costory starts attaching jobs from the moment you connect. There is no older backfill from Google.

**Terraform**

Not supported.

## What Costory adds

### Dimensions

| Field | Meaning |
| - | - |
| `cos_query_user` | User or service account that ran the job. Empty when Costory has cost but no job access. |
| `cos_query_statement` | Statement type: SELECT, MERGE, CTAS, DELETE, and similar. |
| `cos_query_workload` | QUERY, LOAD, or EXTRACT. |
| `cos_query_target` | Destination table. |
| `cos_query_reservation_id` | Reservation the job ran on, if any. |
| `cos_query_parent_id` | Parent job id for script children. |
| `cos_query_error` | Error on failed or stopped jobs. |

These fields follow [Standard Columns](/docs/standard-columns.md) naming (`cos_*`). They are attached by this connector. They are not in the raw GCP billing export.

### Usage metrics

| Field | Meaning |
| - | - |
| `query_count` | Number of jobs. |
| `query_bytes_billed` | Bytes billed on demand. |
| `query_bytes_scanned` | Bytes scanned. |
| `query_compute_ms` | Slot-milliseconds. |
| `query_duration_ms` | Wall-clock duration. |

## On-demand and slots in one view

GCP bills slots as a reservation line, not per query. Costory reallocates that spend onto the user who ran the job, using `query_compute_ms`. The same `cos_query_*` dimensions then apply to on-demand Analysis and to slot cost.

Group by `cos_query_user` (or statement, destination table, workload) and you get a single view of all your BigQuery compute: who ran it, what they ran, and whether it was paid per TiB or from the reservation.

That is the point of connecting jobs even if you already buy Editions slots. The invoice stays lumped. Explorer shows the people and pipelines that consumed the slots.

### Compare on-demand with Editions

If you are still on demand and want a slots estimate per workload:

```text theme={null}
(query_compute_ms + overhead × query_duration_ms) / 3,600,000 × price_per_slot_hour
```

Use **25** slots of overhead per job as the central fluid-autoscaling estimate, or **50** as a worst case. `price_per_slot_hour` is the list pay-as-you-go Editions price for your region. Edit it per customer. It is not a Costory default.

Subtract that estimate from on-demand Analysis cost to see whether a workload would be cheaper on slots.

> [!WARNING]
>
> Always filter `cos_query_user != null` before you compare on-demand with slots. Unlabelled cost has no slot time, so the estimate looks like 100% savings.

## Job drill-down and Costory Agent

Open a job by id (`project:location.job_id`) to see its SQL text, the tables it referenced, and the stages that used the most slots.

That is what lets **Costory Agent** suggest performance work. Ask Costory in the app, or any assistant on [Costory MCP](/features/mcp.md), to inspect a heavy job. The agent reads the SQL and the slot-heavy stages, then explains why the query used too many slots (full table scans, exploding joins, missing filters, script children) and can suggest a rewrite.

> [!TIP]
>
> Try: *"This job `my-project:EU.bquxjob_xxx` used too many slots. Show the SQL, the stages with the most slot-ms, and suggest how to reduce slot time."*

## What you can build

* One Explorer view of on-demand and slot cost, grouped by the user who ran the query.
* Cost per user, service account, statement type, or destination table on the `cos_query_*` fields.
* Unit metrics in Explorer formulas: cost per query (`cost / query_count`), cost per TiB billed, and slot-hours per TiB (compute intensity).
* Spend on failed or stopped queries by filtering `cos_query_error`.
* A coverage KPI: cost where `cos_query_user` is set, divided by total BigQuery compute. Connected orgs often sit in the mid-90s.
* A [virtual dimension (Rule-based dimensions whose output values are defined in Costory.)](/features/tagging/dimensions.md#virtual-dimensions) on `cos_query_target` (or other job fields) to group jobs into workloads, for example by destination-table name patterns.

A **BigQuery Enriched Support** dashboard template will package these views. Until it is in the [template library](/features/dashboards/templates.md), start from Explorer and save the widgets you need.

## Pitfalls

| Issue | What to do |
| - | - |
| **Coverage gaps** | A project without job access still has cost. It appears as Unlabelled (`cos_query_user` empty). Grant Resource Viewer on that project and watch the coverage KPI. Slot reallocation cannot assign Unlabelled spend. |
| **Phantom savings** | Filter `cos_query_user != null` on every on-demand vs slots comparison. |
| **Wrong grain** | A service account that runs many pipelines can look worse on slots overall, while some destination tables are cheaper there. Decide per destination table or per workload, not per user. |
| **Slot prices** | List PAYG prices vary by region. Replace the example price in the formula for each customer. |
| **Reservation invoice vs Explorer** | GCP still bills the reservation as one line. Costory reallocates that line to `cos_query_user` using `query_compute_ms`. Use Explorer for who consumed the slots, not the raw SKU. |

## Troubleshooting

| Issue | What to check |
| - | - |
| **Validate fails** | Confirm the service account from the connect screen has `roles/bigquery.resourceViewer` on the project ID you entered. Follow any extra roles listed on that screen. |
| **Most cost is Unlabelled** | Grant job access on every project that runs queries, not only the billing-export project. Recheck coverage after the next refresh. |
| **No `cos_query_*` fields** | Confirm the jobs integration is saved, then wait for the next billing refresh. Group BigQuery compute, including Analysis and reservation SKUs. |
| **Empty usage metrics** | Open **Advanced Explorer**. Classic greys out non-cost metrics. |

## Related pages

> **[GCP Billing Setup](/setup/billing/gcp.md)**
>
> Connect Detailed Usage Cost export before you attach jobs.

> **[BigQuery (tables)](/setup/usage-metrics/bigquery.md)**
>
> Query any BigQuery table live. Different connector.

> **[BigQuery and dbt visibility](/use-cases/dbt_bigquery_visibility/increase_visibility.md)**
>
> Label dbt models, then use jobs for per-query cost.

> **[Costory MCP](/features/mcp.md)**
>
> Ask Costory Agent why a job used too many slots.

> **[Virtual dimensions](/features/tagging/dimensions.md#virtual-dimensions)**
>
> Group jobs into workloads from destination tables.

> **[Unit Economics](/features/unit-economics.md)**
>
> Cost per query, cost per TiB, slot-hours per TiB.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.