fix: SQLite online store deletes tables from other projects in shared registry scenarios by aniketpalu · Pull Request #5766 · feast-dev/feast

aniketpalu

Test Results: Before vs After Fix

Before Fix (Bug Present)

Test Case 1

Applying real_snake project (shares registry with modern_hermit)
Applying changes for project real_snake
Created sqlite table real_snake_driver_hourly_stats_fresh
Created sqlite table real_snake_driver_hourly_stats
Deleted sqlite table modern_hermit_driver_hourly_stats_fresh ❌ BUG
Deleted sqlite table modern_hermit_driver_hourly_stats ❌ BUG

Test Case 2

Re-applying modern_hermit project
Applying changes for project modern_hermit
No changes to registry
Created sqlite table modern_hermit_driver_hourly_stats_fresh
Created sqlite table modern_hermit_driver_hourly_stats
Deleted sqlite table real_snake_driver_hourly_stats_fresh ❌ BUG
Deleted sqlite table real_snake_driver_hourly_stats ❌ BUG
Issue: Each feast apply deletes tables from the other project sharing the same registry.

Terminal Output

(dev) apaluska@apaluska-mac feast % feast init

Creating a new Feast repository in /Users/apaluska/Work/feast/modern_hermit.

(dev) apaluska@apaluska-mac feast % feast init

Creating a new Feast repository in /Users/apaluska/Work/feast/real_snake.

(dev) apaluska@apaluska-mac feast % cd modern_hermit/feature_repo; feast apply;
/Users/apaluska/Work/feast/modern_hermit/feature_repo/example_repo.py:27: DeprecationWarning: Entity value_type will be mandatory in the next release. Please specify a value_type for entity 'driver'.
  driver = Entity(name="driver", join_keys=["driver_id"])
Applying changes for project modern_hermit
/Users/apaluska/Work/feast/sdk/python/feast/feature_store.py:583: RuntimeWarning: On demand feature view is an experimental feature. This API is stable, but the functionality does not scale well for offline retrieval
  warnings.warn(
Created project modern_hermit
Created entity driver
Created feature view driver_hourly_stats
Created feature view driver_hourly_stats_fresh
Created on demand feature view transformed_conv_rate
Created on demand feature view transformed_conv_rate_fresh
Created feature service driver_activity_v3
Created feature service driver_activity_v1
Created feature service driver_activity_v2

Created sqlite table modern_hermit_driver_hourly_stats_fresh
Created sqlite table modern_hermit_driver_hourly_stats

(dev) apaluska@apaluska-mac feature_repo % cd ../../modern_hermit/feature_repo; cat feature_store.yaml 
project: modern_hermit
# By default, the registry is a file (but can be turned into a more scalable SQL-backed registry)
registry: data/registry.db
# The provider primarily specifies default offline / online stores & storing the registry in a given cloud
provider: local
online_store:
    type: sqlite
    path: data/online_store.db
entity_key_serialization_version: 3
# By default, no_auth for authentication and authorization, other possible values kubernetes and oidc. Refer the documentation for more details.
auth:
    type: no_auth
(dev) apaluska@apaluska-mac feature_repo % cd ../../real_snake/feature_repo; cat feature_store.yaml 
project: real_snake
# By default, the registry is a file (but can be turned into a more scalable SQL-backed registry)
registry: /Users/apaluska/Work/feast/modern_hermit/feature_repo/data/registry.db
# The provider primarily specifies default offline / online stores & storing the registry in a given cloud
provider: local
online_store:
    type: sqlite
    path: data/online_store.db
entity_key_serialization_version: 3
# By default, no_auth for authentication and authorization, other possible values kubernetes and oidc. Refer the documentation for more details.
auth:
    type: no_auth
(dev) apaluska@apaluska-mac feature_repo % feast apply
/Users/apaluska/Work/feast/real_snake/feature_repo/example_repo.py:27: DeprecationWarning: Entity value_type will be mandatory in the next release. Please specify a value_type for entity 'driver'.
  driver = Entity(name="driver", join_keys=["driver_id"])
Applying changes for project real_snake
/Users/apaluska/Work/feast/sdk/python/feast/feature_store.py:583: RuntimeWarning: On demand feature view is an experimental feature. This API is stable, but the functionality does not scale well for offline retrieval
  warnings.warn(
Created project real_snake
Created entity driver
Created feature view driver_hourly_stats
Created feature view driver_hourly_stats_fresh
Created on demand feature view transformed_conv_rate
Created on demand feature view transformed_conv_rate_fresh
Created feature service driver_activity_v2
Created feature service driver_activity_v3
Created feature service driver_activity_v1

Created sqlite table real_snake_driver_hourly_stats_fresh
Created sqlite table real_snake_driver_hourly_stats
Deleted sqlite table modern_hermit_driver_hourly_stats_fresh
Deleted sqlite table modern_hermit_driver_hourly_stats

(dev) apaluska@apaluska-mac feature_repo % cd ../../modern_hermit/feature_repo; feast apply
/Users/apaluska/Work/feast/modern_hermit/feature_repo/example_repo.py:27: DeprecationWarning: Entity value_type will be mandatory in the next release. Please specify a value_type for entity 'driver'.
  driver = Entity(name="driver", join_keys=["driver_id"])
Applying changes for project modern_hermit
/Users/apaluska/Work/feast/sdk/python/feast/feature_store.py:583: RuntimeWarning: On demand feature view is an experimental feature. This API is stable, but the functionality does not scale well for offline retrieval
  warnings.warn(
No changes to registry
Created sqlite table modern_hermit_driver_hourly_stats_fresh
Created sqlite table modern_hermit_driver_hourly_stats
Deleted sqlite table real_snake_driver_hourly_stats_fresh
Deleted sqlite table real_snake_driver_hourly_stats

After Fix (Bug Resolved)

Test Case 1

Applying fit_impala project
Applying changes for project fit_impala
Created sqlite table fit_impala_driver_hourly_stats_fresh
Created sqlite table fit_impala_driver_hourly_stats ✅ No deletions of other projects' tables

Test Case 2: Applying premium_dory project (shares registry with fit_impala)

Applying changes for project premium_dory
Created sqlite table premium_dory_driver_hourly_stats_fresh
Created sqlite table premium_dory_driver_hourly_stats ✅ No deletions of fit_impala tables
Result: Projects sharing a registry no longer delete each other's tables. Each project only manages its own infrastructure.

Terminal Output

(dev) apaluska@apaluska-mac feast % feast init
feast i
Creating a new Feast repository in /Users/apaluska/Work/feast/fit_impala.

ni%                                                                                                                 
(dev) apaluska@apaluska-mac feast % feast init

Creating a new Feast repository in /Users/apaluska/Work/feast/premium_dory.

(dev) apaluska@apaluska-mac feast % cd fit_impala/feature_repo; feast apply
/Users/apaluska/Work/feast/fit_impala/feature_repo/example_repo.py:27: DeprecationWarning: Entity value_type will be mandatory in the next release. Please specify a value_type for entity 'driver'.
  driver = Entity(name="driver", join_keys=["driver_id"])
Applying changes for project fit_impala
/Users/apaluska/Work/feast/sdk/python/feast/feature_store.py:583: RuntimeWarning: On demand feature view is an experimental feature. This API is stable, but the functionality does not scale well for offline retrieval
  warnings.warn(
Created project fit_impala
Created entity driver
Created feature view driver_hourly_stats_fresh
Created feature view driver_hourly_stats
Created on demand feature view transformed_conv_rate_fresh
Created on demand feature view transformed_conv_rate
Created feature service driver_activity_v3
Created feature service driver_activity_v1
Created feature service driver_activity_v2

Created sqlite table fit_impala_driver_hourly_stats_fresh
Created sqlite table fit_impala_driver_hourly_stats

(dev) apaluska@apaluska-mac feature_repo % 
(dev) apaluska@apaluska-mac feature_repo % 
(dev) apaluska@apaluska-mac feature_repo % 
(dev) apaluska@apaluska-mac feature_repo % cd ../../premium_dory/feature_repo; cat feature_store.yaml  
project: premium_dory
# By default, the registry is a file (but can be turned into a more scalable SQL-backed registry)
registry: **/Users/apaluska/Work/feast/fit_impala/feature_repo/data/registry.db**
# The provider primarily specifies default offline / online stores & storing the registry in a given cloud
provider: local
online_store:
    type: sqlite
    path: data/online_store.db
entity_key_serialization_version: 3
# By default, no_auth for authentication and authorization, other possible values kubernetes and oidc. Refer the documentation for more details.
auth:
    type: no_auth
(dev) apaluska@apaluska-mac feature_repo % feast apply
/Users/apaluska/Work/feast/premium_dory/feature_repo/example_repo.py:27: DeprecationWarning: Entity value_type will be mandatory in the next release. Please specify a value_type for entity 'driver'.
  driver = Entity(name="driver", join_keys=["driver_id"])
Applying changes for project premium_dory
/Users/apaluska/Work/feast/sdk/python/feast/feature_store.py:583: RuntimeWarning: On demand feature view is an experimental feature. This API is stable, but the functionality does not scale well for offline retrieval
  warnings.warn(
Created project premium_dory
Created entity driver
Created feature view driver_hourly_stats_fresh
Created feature view driver_hourly_stats
Created on demand feature view transformed_conv_rate_fresh
Created on demand feature view transformed_conv_rate
Created feature service driver_activity_v2
Created feature service driver_activity_v3
Created feature service driver_activity_v1

**Created sqlite table premium_dory_driver_hourly_stats_fresh
Created sqlite table premium_dory_driver_hourly_stats**