Skip to content

BigQuery Read Credentials Setup

The Looker migrator compare flow (just looker_migrate eval with render) needs live BigQuery access to render the Dataface Port side of the compare UI. Without credentials, every render fails with "Rendered Dataface artifact not found".

How credentials work

Dataface uses google-cloud-bigquery and relies on Google's Application Default Credentials (ADC). The library picks credentials up automatically from the environment — there is no Dataface-specific credential config key.

Resolution order:

  1. GOOGLE_APPLICATION_CREDENTIALS — path to a service account JSON file.
  2. User credentials from gcloud auth application-default login.
  3. Workload identity / metadata server (GCE, Cloud Run only).

The BigQuery project and dataset come from apps/looker_migrate/runtime/dataface.yml, which reads two env vars:

Env var Purpose
LOOKER_COMPARE_BIGQUERY_PROJECT GCP project that hosts the analytics dataset
LOOKER_COMPARE_BIGQUERY_DATASET BigQuery dataset to query

Ask a teammate for the correct project and dataset values — they are Fivetran-internal.

Setup — user credentials (local dev)

gcloud auth application-default login

Then add to your .env:

LOOKER_COMPARE_BIGQUERY_PROJECT=your-gcp-project
LOOKER_COMPARE_BIGQUERY_DATASET=your-dataset

Reload the shell (or source .env) and run just creds-check to confirm.

Setup — service account (persistent / agents / CI)

  1. Obtain a service account JSON key with at least BigQuery Data Viewer and BigQuery Job User roles on the relevant GCP project.
  2. Save the file outside the repo (e.g. ~/.config/gcloud/dataface-bq-sa.json). Do not commit it to the repo.
  3. Add to your .env:
GOOGLE_APPLICATION_CREDENTIALS=/path/to/service-account.json
LOOKER_COMPARE_BIGQUERY_PROJECT=your-gcp-project
LOOKER_COMPARE_BIGQUERY_DATASET=your-dataset

Self-check

just creds-check

Exit 0 means the connection is working. Non-zero exit prints the failure and points back to this doc.

Common failure modes

Symptom Cause Fix
LOOKER_COMPARE_BIGQUERY_PROJECT is not set Env var missing Add to .env and reload shell
DefaultCredentialsError / google.auth.exceptions No ADC and no JSON file Run gcloud auth application-default login or set GOOGLE_APPLICATION_CREDENTIALS
403 Permission denied on bigquery.jobs.create Account lacks BigQuery Job User Grant role in GCP IAM
404 Not found: Dataset Wrong project or dataset name Check LOOKER_COMPARE_BIGQUERY_PROJECT / LOOKER_COMPARE_BIGQUERY_DATASET
401 invalid authentication credentials ADC token expired Re-run gcloud auth application-default login