◐ Shell
clean mode source ↗

dbt integration: Add validation for timestamp field data type

YassinNouh21

Context

PR #5827 added dbt integration that requires a timestamp field for point-in-time correctness.

Problem

The code validates that the timestamp field exists in the model columns but doesn't validate that it's actually a timestamp type. This could lead to runtime errors or incorrect behavior.

Current Behavior

# In dbt_import.py:183-189
if timestamp_field not in column_names:
    click.echo(
        f"{Fore.YELLOW}Warning: Model '{model.name}' missing timestamp "
        f"field '{timestamp_field}'. Skipping.{Style.RESET_ALL}"
    )
    continue

No type checking is performed.

Proposed Solution

Add validation that the timestamp field is one of the timestamp types:

  • TIMESTAMP
  • TIMESTAMP_NTZ / TIMESTAMP_LTZ / TIMESTAMP_TZ
  • DATETIME
  • DATE

Warn users if the field exists but has an incompatible type.

Example

timestamp_col = next((c for c in model.columns if c.name == timestamp_field), None)
if timestamp_col:
    normalized_type = timestamp_col.data_type.upper()
    valid_ts_types = ['TIMESTAMP', 'TIMESTAMP_NTZ', 'TIMESTAMP_LTZ', 'TIMESTAMP_TZ', 'DATETIME', 'DATE']
    if not any(t in normalized_type for t in valid_ts_types):
        click.echo(f"{Fore.YELLOW}Warning: Timestamp field '{timestamp_field}' has type '{timestamp_col.data_type}' which may not be suitable{Style.RESET_ALL}")

Related