◐ Shell
clean mode source ↗

dbt integration: Add support for data source connection configuration

YassinNouh21

Context

PR #5827 added dbt integration with basic data source support.

Problem

The generated data sources use default/inferred configurations but don't allow specifying:

  • BigQuery: GCP project, dataset overrides
  • Snowflake: account, warehouse, role, connection parameters
  • File: base path, file format options

This means generated sources may not connect to the actual data.

Current Limitation

# BigQuery example - no way to override project
return BigQuerySource(
    name=f"{model.name}_source",
    table=model.full_table_name,  # Uses dbt's database.schema.table
    # Missing: project override, dataset override
)

Proposed Enhancement

Add CLI options for data source configuration:

# BigQuery
feast dbt import -m manifest.json -e driver_id \
  --data-source-type bigquery \
  --bigquery-project my-gcp-project

# Snowflake  
feast dbt import -m manifest.json -e driver_id \
  --data-source-type snowflake \
  --snowflake-account my-account \
  --snowflake-warehouse my-warehouse \
  --snowflake-role my-role

# File
feast dbt import -m manifest.json -e driver_id \
  --data-source-type file \
  --file-format parquet \
  --file-base-path /data/warehouse

Or support reading from feature_store.yaml:

offline_store:
  type: bigquery
  project: my-gcp-project
  
dbt:
  inherit_offline_store_config: true

Related