feat: Add OnlineStore for MongoDB by caseyclements · Pull Request #6025 · feast-dev/feast
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
…t.toml Any change to pyproject.toml invalidates the pixi.lock manifest hash, causing 'pixi install --locked' to fail in CI even when the changed section (mongodb optional extras) is not used by any pixi environment. Regenerated with: pixi install (v0.63.2) Signed-off-by: Casey Clements <casey.clements@mongodb.com>
Stores like mongodb, redis, and dynamodb raise FeastExtrasDependencyImportError at import time when their optional Python extras are not installed. test_all.py only caught ModuleNotFoundError, so any such import caused the entire test_docstrings() function to abort rather than gracefully skipping the module. Extend the except clause to also catch FeastExtrasDependencyImportError so the doctest run completes for all other modules when an optional extra is absent in the test environment. Signed-off-by: Casey Clements <casey.clements@mongodb.com>
Directory was renamed from tests/integration/feature_repos to tests/universal/feature_repos in upstream/master. Update the PYTEST_PLUGINS module path in test-python-universal-mongodb-online to match the new location. Signed-off-by: Casey Clements <casey.clements@mongodb.com>
…eError and other third-party import errors qdrant_client raises TypeError at import time when a gRPC EnumTypeWrapper is used with the | union operator on Python < 3.12. The previous fix only caught ModuleNotFoundError and FeastExtrasDependencyImportError, leaving the test runner vulnerable to any other exception raised by third-party libraries during pkgutil.walk_packages. Changes: - Catch bare Exception in the import try/except block so any import-time error from an optional dependency causes a graceful skip rather than an abort of the entire test run. - Initialize temp_module = None before the try block and add a continue guard so that a failed import never leaves a stale module reference to be used in the subsequent doctest execution block. Signed-off-by: Casey Clements <casey.clements@mongodb.com>
…r package import failures FeastExtrasDependencyImportError inherits from FeastError -> Exception, not from ImportError. pkgutil.walk_packages calls __import__ internally when recursing into sub-packages, and only silently swallows ImportError subclasses; any other exception is re-raised, crashing the entire test run. Passing onerror=lambda _: None tells walk_packages to skip any package that fails to import during the walk phase, regardless of the exception type. The inner importlib.import_module try/except already handles the same errors for the explicit import step used to collect doctests. Signed-off-by: Casey Clements <casey.clements@mongodb.com>
…iversal.feature_repos The upstream directory tests/integration/feature_repos was renamed to tests/universal/feature_repos. Three files still referenced the old path: - mongodb_repo_configuration.py: IntegrationTestRepoConfig and MongoDBOnlineStoreCreator imports - tests/.../online_store/mongodb.py: OnlineStoreCreator import - tests/unit/online_store/test_mongodb_online_retrieval.py: TAGS import Updating all three unblocks test collection and allows make test-python-universal-mongodb-online to run locally. Signed-off-by: Casey Clements <casey.clements@mongodb.com>
…-operator The Python ONLINE_STORE_CLASS_FOR_TYPE now includes 'mongodb', so the operator's parity check (test-datasources) fails unless the Go API also lists it. Add 'mongodb' to ValidOnlineStoreDBStorePersistenceTypes in both api/v1 and api/v1alpha1. Signed-off-by: Casey Clements <casey.clements@mongodb.com>
Add DriverInfo with the Feast name and version to both the sync MongoClient and async AsyncMongoClient so that MongoDB can identify traffic originating from a Feast AI integration. Signed-off-by: Casey Clements <casey.clements@mongodb.com>
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
…rePersistence The +kubebuilder:validation:Enum annotation on the Type field of OnlineStoreDBStorePersistence was not updated when mongodb was added to ValidOnlineStoreDBStorePersistenceTypes. This annotation drives CRD OpenAPI schema validation at Kubernetes admission time, so any FeatureStore CR specifying type: mongodb would be rejected by the API server. Updated both api/v1 and api/v1alpha1. Signed-off-by: Casey Clements <casey.clements@mongodb.com>
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
…onsistent with sync counterpart Signed-off-by: Casey Clements <casey.clements@mongodb.com>
Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
…ially identical. Signed-off-by: Casey Clements <casey.clements@mongodb.com>
ntkathole added a commit to red-hat-data-services/feast that referenced this pull request
* Initial commit on INTPYTHON-297-MongoDB-Feast-Integration
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Added mongodb to project.optional-dependencies in pyproject.toml. Now pymongo is found as extra
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Checkpoint. Passing tests.unit.online_store.test_online_writes.TestOnlineWrites with default args of MongoDBOnlineStoreConfig. Lots to do.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Handle Nan in dfs for test_online_writes.py. Now all tests in the module pass
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Removed suffix of implementation: mongodb_openai -> mongodb
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Moved MongoDBOnlineStore to feast.infra.online_store.contrib
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Formatting
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Refactor online_read that converts bson to proto. Left two methods for now. simply and transformming
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Remove file added early during discovery
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Formatting
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Added version of test that uses testcontainers mongodb instead of assuming one is running
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Create Make target for universal tests
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Cleanup
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Removed temporary integration tests requiring one to spin up own mongodb server.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Format
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Typing
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Implemented ASync API and Tests
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Removed offline store stubs. The first PR will only contain the OnlineStore
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Moved mongodb_online_store out of cobtrib package.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Add documentation.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Cleanups and docstrings
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Fixed another reference to contrib dir
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Typos
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Made _convert_raw_docs_to_proto staticmethods private
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* After benchmarking two alogithm for conevrting read results from bson to proto, removed the naive one. It was outcompeted 3X across dimensions
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Add extra unit tests
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Formatting
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Fixes in pyproject.toml
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Fixed Detect secrets false positives.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Update sdk/python/feast/infra/online_stores/mongodb_online_store/mongodb.py
Set _client and _collection to None upon synchronous teardown.
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Fix CI: guard pymongo imports and skip test module when pymongo unavailable
In the unit-test-python CI job, pymongo is not installed (it is an optional
extra). This caused ModuleNotFoundError during pytest collection.
Two changes:
1. mongodb.py: wrap pymongo imports in try/except -> FeastExtrasDependencyImportError
(consistent with redis.py, dynamodb.py, datastore.py pattern)
2. test_mongodb_online_retrieval.py: add pytest.importorskip(\pymongo\) so
the entire test module is skipped gracefully when pymongo is absent
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Fix: return (None, None) when entity doc exists but feature view was never written
MongoDB stores all feature views for an entity in a single document.
If FV 'driver_stats' is written, an entity doc exists for driver_1.
A subsequent read for FV 'pricing' (never written) was previously
returning (None, {all_features: ValueProto()}) - a truthy feature dict
with all-empty protos - instead of the correct (None, None) sentinel.
Feast and downstream callers use (None, None) to signal entity absence.
A non-None feature dict means 'entity found, values are null', which
causes different behaviour in the feature retrieval pipeline.
Fix: after confirming the entity doc exists, also check that the
feature view key is present in doc['features']. If absent, return
(None, None) rather than a dict of empty protos.
Adds test_convert_raw_docs_entity_exists_but_fv_not_written to cover
the multi-feature-view colocation scenario identified in code review.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Update pixi.lock after adding mongodb optional dependency to pyproject.toml
Any change to pyproject.toml invalidates the pixi.lock manifest hash,
causing 'pixi install --locked' to fail in CI even when the changed
section (mongodb optional extras) is not used by any pixi environment.
Regenerated with: pixi install (v0.63.2)
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: catch FeastExtrasDependencyImportError in doctest runner
Stores like mongodb, redis, and dynamodb raise FeastExtrasDependencyImportError
at import time when their optional Python extras are not installed.
test_all.py only caught ModuleNotFoundError, so any such import caused the
entire test_docstrings() function to abort rather than gracefully skipping
the module.
Extend the except clause to also catch FeastExtrasDependencyImportError so
the doctest run completes for all other modules when an optional extra is
absent in the test environment.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: update PYTEST_PLUGINS path for mongodb online store tests
Directory was renamed from tests/integration/feature_repos to
tests/universal/feature_repos in upstream/master. Update the
PYTEST_PLUGINS module path in test-python-universal-mongodb-online
to match the new location.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: broaden import exception handling in doctest runner to catch TypeError and other third-party import errors
qdrant_client raises TypeError at import time when a gRPC EnumTypeWrapper
is used with the | union operator on Python < 3.12. The previous fix only
caught ModuleNotFoundError and FeastExtrasDependencyImportError, leaving
the test runner vulnerable to any other exception raised by third-party
libraries during pkgutil.walk_packages.
Changes:
- Catch bare Exception in the import try/except block so any import-time
error from an optional dependency causes a graceful skip rather than an
abort of the entire test run.
- Initialize temp_module = None before the try block and add a continue
guard so that a failed import never leaves a stale module reference to
be used in the subsequent doctest execution block.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: pass onerror to pkgutil.walk_packages to suppress non-ImportError package import failures
FeastExtrasDependencyImportError inherits from FeastError -> Exception, not
from ImportError. pkgutil.walk_packages calls __import__ internally when
recursing into sub-packages, and only silently swallows ImportError subclasses;
any other exception is re-raised, crashing the entire test run.
Passing onerror=lambda _: None tells walk_packages to skip any package that
fails to import during the walk phase, regardless of the exception type. The
inner importlib.import_module try/except already handles the same errors for
the explicit import step used to collect doctests.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: update stale tests.integration.feature_repos imports to tests.universal.feature_repos
The upstream directory tests/integration/feature_repos was renamed to
tests/universal/feature_repos. Three files still referenced the old path:
- mongodb_repo_configuration.py: IntegrationTestRepoConfig and
MongoDBOnlineStoreCreator imports
- tests/.../online_store/mongodb.py: OnlineStoreCreator import
- tests/unit/online_store/test_mongodb_online_retrieval.py: TAGS import
Updating all three unblocks test collection and allows
make test-python-universal-mongodb-online to run locally.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* feat: add mongodb to ValidOnlineStoreDBStorePersistenceTypes in feast-operator
The Python ONLINE_STORE_CLASS_FOR_TYPE now includes 'mongodb', so the
operator's parity check (test-datasources) fails unless the Go API also
lists it. Add 'mongodb' to ValidOnlineStoreDBStorePersistenceTypes in
both api/v1 and api/v1alpha1.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* feat: add Feast driver metadata to MongoDB client instantiations
Add DriverInfo with the Feast name and version to both the sync
MongoClient and async AsyncMongoClient so that MongoDB can identify
traffic originating from a Feast AI integration.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* docs: update MongoDB online store status from alpha to preview
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: add mongodb to kubebuilder Enum annotations for OnlineStoreDBStorePersistence
The +kubebuilder:validation:Enum annotation on the Type field of
OnlineStoreDBStorePersistence was not updated when mongodb was added to
ValidOnlineStoreDBStorePersistenceTypes. This annotation drives CRD
OpenAPI schema validation at Kubernetes admission time, so any
FeatureStore CR specifying type: mongodb would be rejected by the API
server. Updated both api/v1 and api/v1alpha1.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Update +GOLANGCI_LINT_VERSION to fix upstream issue golang/go#74462
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: raise ValueError in _get_client_async for invalid config type, consistent with sync counterpart
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: Added mongodb to operator yamls
Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>
* Small change suggested by ntkathole
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Factor out write logic into utility function making sync/async essentially identical.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
---------
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: ntkathole <nikhilkathole2683@gmail.com>
ntkathole pushed a commit to red-hat-data-services/feast that referenced this pull request
ntkathole added a commit to red-hat-data-services/feast that referenced this pull request
* Initial commit on INTPYTHON-297-MongoDB-Feast-Integration
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Added mongodb to project.optional-dependencies in pyproject.toml. Now pymongo is found as extra
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Checkpoint. Passing tests.unit.online_store.test_online_writes.TestOnlineWrites with default args of MongoDBOnlineStoreConfig. Lots to do.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Handle Nan in dfs for test_online_writes.py. Now all tests in the module pass
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Removed suffix of implementation: mongodb_openai -> mongodb
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Moved MongoDBOnlineStore to feast.infra.online_store.contrib
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Formatting
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Refactor online_read that converts bson to proto. Left two methods for now. simply and transformming
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Remove file added early during discovery
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Formatting
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Added version of test that uses testcontainers mongodb instead of assuming one is running
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Create Make target for universal tests
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Cleanup
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Removed temporary integration tests requiring one to spin up own mongodb server.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Format
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Typing
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Implemented ASync API and Tests
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Removed offline store stubs. The first PR will only contain the OnlineStore
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Moved mongodb_online_store out of cobtrib package.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Add documentation.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Cleanups and docstrings
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Fixed another reference to contrib dir
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Typos
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Made _convert_raw_docs_to_proto staticmethods private
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* After benchmarking two alogithm for conevrting read results from bson to proto, removed the naive one. It was outcompeted 3X across dimensions
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Add extra unit tests
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Formatting
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Fixes in pyproject.toml
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Fixed Detect secrets false positives.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Update sdk/python/feast/infra/online_stores/mongodb_online_store/mongodb.py
Set _client and _collection to None upon synchronous teardown.
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Fix CI: guard pymongo imports and skip test module when pymongo unavailable
In the unit-test-python CI job, pymongo is not installed (it is an optional
extra). This caused ModuleNotFoundError during pytest collection.
Two changes:
1. mongodb.py: wrap pymongo imports in try/except -> FeastExtrasDependencyImportError
(consistent with redis.py, dynamodb.py, datastore.py pattern)
2. test_mongodb_online_retrieval.py: add pytest.importorskip(\pymongo\) so
the entire test module is skipped gracefully when pymongo is absent
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Fix: return (None, None) when entity doc exists but feature view was never written
MongoDB stores all feature views for an entity in a single document.
If FV 'driver_stats' is written, an entity doc exists for driver_1.
A subsequent read for FV 'pricing' (never written) was previously
returning (None, {all_features: ValueProto()}) - a truthy feature dict
with all-empty protos - instead of the correct (None, None) sentinel.
Feast and downstream callers use (None, None) to signal entity absence.
A non-None feature dict means 'entity found, values are null', which
causes different behaviour in the feature retrieval pipeline.
Fix: after confirming the entity doc exists, also check that the
feature view key is present in doc['features']. If absent, return
(None, None) rather than a dict of empty protos.
Adds test_convert_raw_docs_entity_exists_but_fv_not_written to cover
the multi-feature-view colocation scenario identified in code review.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Update pixi.lock after adding mongodb optional dependency to pyproject.toml
Any change to pyproject.toml invalidates the pixi.lock manifest hash,
causing 'pixi install --locked' to fail in CI even when the changed
section (mongodb optional extras) is not used by any pixi environment.
Regenerated with: pixi install (v0.63.2)
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: catch FeastExtrasDependencyImportError in doctest runner
Stores like mongodb, redis, and dynamodb raise FeastExtrasDependencyImportError
at import time when their optional Python extras are not installed.
test_all.py only caught ModuleNotFoundError, so any such import caused the
entire test_docstrings() function to abort rather than gracefully skipping
the module.
Extend the except clause to also catch FeastExtrasDependencyImportError so
the doctest run completes for all other modules when an optional extra is
absent in the test environment.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: update PYTEST_PLUGINS path for mongodb online store tests
Directory was renamed from tests/integration/feature_repos to
tests/universal/feature_repos in upstream/master. Update the
PYTEST_PLUGINS module path in test-python-universal-mongodb-online
to match the new location.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: broaden import exception handling in doctest runner to catch TypeError and other third-party import errors
qdrant_client raises TypeError at import time when a gRPC EnumTypeWrapper
is used with the | union operator on Python < 3.12. The previous fix only
caught ModuleNotFoundError and FeastExtrasDependencyImportError, leaving
the test runner vulnerable to any other exception raised by third-party
libraries during pkgutil.walk_packages.
Changes:
- Catch bare Exception in the import try/except block so any import-time
error from an optional dependency causes a graceful skip rather than an
abort of the entire test run.
- Initialize temp_module = None before the try block and add a continue
guard so that a failed import never leaves a stale module reference to
be used in the subsequent doctest execution block.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: pass onerror to pkgutil.walk_packages to suppress non-ImportError package import failures
FeastExtrasDependencyImportError inherits from FeastError -> Exception, not
from ImportError. pkgutil.walk_packages calls __import__ internally when
recursing into sub-packages, and only silently swallows ImportError subclasses;
any other exception is re-raised, crashing the entire test run.
Passing onerror=lambda _: None tells walk_packages to skip any package that
fails to import during the walk phase, regardless of the exception type. The
inner importlib.import_module try/except already handles the same errors for
the explicit import step used to collect doctests.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: update stale tests.integration.feature_repos imports to tests.universal.feature_repos
The upstream directory tests/integration/feature_repos was renamed to
tests/universal/feature_repos. Three files still referenced the old path:
- mongodb_repo_configuration.py: IntegrationTestRepoConfig and
MongoDBOnlineStoreCreator imports
- tests/.../online_store/mongodb.py: OnlineStoreCreator import
- tests/unit/online_store/test_mongodb_online_retrieval.py: TAGS import
Updating all three unblocks test collection and allows
make test-python-universal-mongodb-online to run locally.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* feat: add mongodb to ValidOnlineStoreDBStorePersistenceTypes in feast-operator
The Python ONLINE_STORE_CLASS_FOR_TYPE now includes 'mongodb', so the
operator's parity check (test-datasources) fails unless the Go API also
lists it. Add 'mongodb' to ValidOnlineStoreDBStorePersistenceTypes in
both api/v1 and api/v1alpha1.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* feat: add Feast driver metadata to MongoDB client instantiations
Add DriverInfo with the Feast name and version to both the sync
MongoClient and async AsyncMongoClient so that MongoDB can identify
traffic originating from a Feast AI integration.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* docs: update MongoDB online store status from alpha to preview
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: add mongodb to kubebuilder Enum annotations for OnlineStoreDBStorePersistence
The +kubebuilder:validation:Enum annotation on the Type field of
OnlineStoreDBStorePersistence was not updated when mongodb was added to
ValidOnlineStoreDBStorePersistenceTypes. This annotation drives CRD
OpenAPI schema validation at Kubernetes admission time, so any
FeatureStore CR specifying type: mongodb would be rejected by the API
server. Updated both api/v1 and api/v1alpha1.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Update +GOLANGCI_LINT_VERSION to fix upstream issue golang/go#74462
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: raise ValueError in _get_client_async for invalid config type, consistent with sync counterpart
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: Added mongodb to operator yamls
Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>
* Small change suggested by ntkathole
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Factor out write logic into utility function making sync/async essentially identical.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
---------
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: ntkathole <nikhilkathole2683@gmail.com>
ntkathole pushed a commit to red-hat-data-services/feast that referenced this pull request
Shizoqua pushed a commit to Shizoqua/feast that referenced this pull request
* Initial commit on INTPYTHON-297-MongoDB-Feast-Integration
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Added mongodb to project.optional-dependencies in pyproject.toml. Now pymongo is found as extra
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Checkpoint. Passing tests.unit.online_store.test_online_writes.TestOnlineWrites with default args of MongoDBOnlineStoreConfig. Lots to do.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Handle Nan in dfs for test_online_writes.py. Now all tests in the module pass
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Removed suffix of implementation: mongodb_openai -> mongodb
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Moved MongoDBOnlineStore to feast.infra.online_store.contrib
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Formatting
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Refactor online_read that converts bson to proto. Left two methods for now. simply and transformming
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Remove file added early during discovery
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Formatting
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Added version of test that uses testcontainers mongodb instead of assuming one is running
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Create Make target for universal tests
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Cleanup
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Removed temporary integration tests requiring one to spin up own mongodb server.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Format
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Typing
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Implemented ASync API and Tests
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Removed offline store stubs. The first PR will only contain the OnlineStore
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Moved mongodb_online_store out of cobtrib package.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Add documentation.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Cleanups and docstrings
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Fixed another reference to contrib dir
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Typos
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Made _convert_raw_docs_to_proto staticmethods private
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* After benchmarking two alogithm for conevrting read results from bson to proto, removed the naive one. It was outcompeted 3X across dimensions
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Add extra unit tests
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Formatting
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Fixes in pyproject.toml
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Fixed Detect secrets false positives.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Update sdk/python/feast/infra/online_stores/mongodb_online_store/mongodb.py
Set _client and _collection to None upon synchronous teardown.
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Fix CI: guard pymongo imports and skip test module when pymongo unavailable
In the unit-test-python CI job, pymongo is not installed (it is an optional
extra). This caused ModuleNotFoundError during pytest collection.
Two changes:
1. mongodb.py: wrap pymongo imports in try/except -> FeastExtrasDependencyImportError
(consistent with redis.py, dynamodb.py, datastore.py pattern)
2. test_mongodb_online_retrieval.py: add pytest.importorskip(\pymongo\) so
the entire test module is skipped gracefully when pymongo is absent
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Fix: return (None, None) when entity doc exists but feature view was never written
MongoDB stores all feature views for an entity in a single document.
If FV 'driver_stats' is written, an entity doc exists for driver_1.
A subsequent read for FV 'pricing' (never written) was previously
returning (None, {all_features: ValueProto()}) - a truthy feature dict
with all-empty protos - instead of the correct (None, None) sentinel.
Feast and downstream callers use (None, None) to signal entity absence.
A non-None feature dict means 'entity found, values are null', which
causes different behaviour in the feature retrieval pipeline.
Fix: after confirming the entity doc exists, also check that the
feature view key is present in doc['features']. If absent, return
(None, None) rather than a dict of empty protos.
Adds test_convert_raw_docs_entity_exists_but_fv_not_written to cover
the multi-feature-view colocation scenario identified in code review.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Update pixi.lock after adding mongodb optional dependency to pyproject.toml
Any change to pyproject.toml invalidates the pixi.lock manifest hash,
causing 'pixi install --locked' to fail in CI even when the changed
section (mongodb optional extras) is not used by any pixi environment.
Regenerated with: pixi install (v0.63.2)
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: catch FeastExtrasDependencyImportError in doctest runner
Stores like mongodb, redis, and dynamodb raise FeastExtrasDependencyImportError
at import time when their optional Python extras are not installed.
test_all.py only caught ModuleNotFoundError, so any such import caused the
entire test_docstrings() function to abort rather than gracefully skipping
the module.
Extend the except clause to also catch FeastExtrasDependencyImportError so
the doctest run completes for all other modules when an optional extra is
absent in the test environment.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: update PYTEST_PLUGINS path for mongodb online store tests
Directory was renamed from tests/integration/feature_repos to
tests/universal/feature_repos in upstream/master. Update the
PYTEST_PLUGINS module path in test-python-universal-mongodb-online
to match the new location.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: broaden import exception handling in doctest runner to catch TypeError and other third-party import errors
qdrant_client raises TypeError at import time when a gRPC EnumTypeWrapper
is used with the | union operator on Python < 3.12. The previous fix only
caught ModuleNotFoundError and FeastExtrasDependencyImportError, leaving
the test runner vulnerable to any other exception raised by third-party
libraries during pkgutil.walk_packages.
Changes:
- Catch bare Exception in the import try/except block so any import-time
error from an optional dependency causes a graceful skip rather than an
abort of the entire test run.
- Initialize temp_module = None before the try block and add a continue
guard so that a failed import never leaves a stale module reference to
be used in the subsequent doctest execution block.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: pass onerror to pkgutil.walk_packages to suppress non-ImportError package import failures
FeastExtrasDependencyImportError inherits from FeastError -> Exception, not
from ImportError. pkgutil.walk_packages calls __import__ internally when
recursing into sub-packages, and only silently swallows ImportError subclasses;
any other exception is re-raised, crashing the entire test run.
Passing onerror=lambda _: None tells walk_packages to skip any package that
fails to import during the walk phase, regardless of the exception type. The
inner importlib.import_module try/except already handles the same errors for
the explicit import step used to collect doctests.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: update stale tests.integration.feature_repos imports to tests.universal.feature_repos
The upstream directory tests/integration/feature_repos was renamed to
tests/universal/feature_repos. Three files still referenced the old path:
- mongodb_repo_configuration.py: IntegrationTestRepoConfig and
MongoDBOnlineStoreCreator imports
- tests/.../online_store/mongodb.py: OnlineStoreCreator import
- tests/unit/online_store/test_mongodb_online_retrieval.py: TAGS import
Updating all three unblocks test collection and allows
make test-python-universal-mongodb-online to run locally.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* feat: add mongodb to ValidOnlineStoreDBStorePersistenceTypes in feast-operator
The Python ONLINE_STORE_CLASS_FOR_TYPE now includes 'mongodb', so the
operator's parity check (test-datasources) fails unless the Go API also
lists it. Add 'mongodb' to ValidOnlineStoreDBStorePersistenceTypes in
both api/v1 and api/v1alpha1.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* feat: add Feast driver metadata to MongoDB client instantiations
Add DriverInfo with the Feast name and version to both the sync
MongoClient and async AsyncMongoClient so that MongoDB can identify
traffic originating from a Feast AI integration.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* docs: update MongoDB online store status from alpha to preview
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: add mongodb to kubebuilder Enum annotations for OnlineStoreDBStorePersistence
The +kubebuilder:validation:Enum annotation on the Type field of
OnlineStoreDBStorePersistence was not updated when mongodb was added to
ValidOnlineStoreDBStorePersistenceTypes. This annotation drives CRD
OpenAPI schema validation at Kubernetes admission time, so any
FeatureStore CR specifying type: mongodb would be rejected by the API
server. Updated both api/v1 and api/v1alpha1.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Update +GOLANGCI_LINT_VERSION to fix upstream issue golang/go#74462
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: raise ValueError in _get_client_async for invalid config type, consistent with sync counterpart
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: Added mongodb to operator yamls
Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>
* Small change suggested by ntkathole
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Factor out write logic into utility function making sync/async essentially identical.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
---------
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: ntkathole <nikhilkathole2683@gmail.com>
Signed-off-by: Shizoqua <hr.lanreshittu@gmail.com>
aniketpalu pushed a commit to aniketpalu/feast that referenced this pull request
* Initial commit on INTPYTHON-297-MongoDB-Feast-Integration
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Added mongodb to project.optional-dependencies in pyproject.toml. Now pymongo is found as extra
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Checkpoint. Passing tests.unit.online_store.test_online_writes.TestOnlineWrites with default args of MongoDBOnlineStoreConfig. Lots to do.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Handle Nan in dfs for test_online_writes.py. Now all tests in the module pass
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Removed suffix of implementation: mongodb_openai -> mongodb
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Moved MongoDBOnlineStore to feast.infra.online_store.contrib
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Formatting
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Refactor online_read that converts bson to proto. Left two methods for now. simply and transformming
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Remove file added early during discovery
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Formatting
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Added version of test that uses testcontainers mongodb instead of assuming one is running
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Create Make target for universal tests
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Cleanup
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Removed temporary integration tests requiring one to spin up own mongodb server.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Format
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Typing
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Implemented ASync API and Tests
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Removed offline store stubs. The first PR will only contain the OnlineStore
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Moved mongodb_online_store out of cobtrib package.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Add documentation.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Cleanups and docstrings
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Fixed another reference to contrib dir
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Typos
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Made _convert_raw_docs_to_proto staticmethods private
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* After benchmarking two alogithm for conevrting read results from bson to proto, removed the naive one. It was outcompeted 3X across dimensions
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Add extra unit tests
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Formatting
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Fixes in pyproject.toml
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Fixed Detect secrets false positives.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Update sdk/python/feast/infra/online_stores/mongodb_online_store/mongodb.py
Set _client and _collection to None upon synchronous teardown.
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Fix CI: guard pymongo imports and skip test module when pymongo unavailable
In the unit-test-python CI job, pymongo is not installed (it is an optional
extra). This caused ModuleNotFoundError during pytest collection.
Two changes:
1. mongodb.py: wrap pymongo imports in try/except -> FeastExtrasDependencyImportError
(consistent with redis.py, dynamodb.py, datastore.py pattern)
2. test_mongodb_online_retrieval.py: add pytest.importorskip(\pymongo\) so
the entire test module is skipped gracefully when pymongo is absent
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Fix: return (None, None) when entity doc exists but feature view was never written
MongoDB stores all feature views for an entity in a single document.
If FV 'driver_stats' is written, an entity doc exists for driver_1.
A subsequent read for FV 'pricing' (never written) was previously
returning (None, {all_features: ValueProto()}) - a truthy feature dict
with all-empty protos - instead of the correct (None, None) sentinel.
Feast and downstream callers use (None, None) to signal entity absence.
A non-None feature dict means 'entity found, values are null', which
causes different behaviour in the feature retrieval pipeline.
Fix: after confirming the entity doc exists, also check that the
feature view key is present in doc['features']. If absent, return
(None, None) rather than a dict of empty protos.
Adds test_convert_raw_docs_entity_exists_but_fv_not_written to cover
the multi-feature-view colocation scenario identified in code review.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Update pixi.lock after adding mongodb optional dependency to pyproject.toml
Any change to pyproject.toml invalidates the pixi.lock manifest hash,
causing 'pixi install --locked' to fail in CI even when the changed
section (mongodb optional extras) is not used by any pixi environment.
Regenerated with: pixi install (v0.63.2)
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: catch FeastExtrasDependencyImportError in doctest runner
Stores like mongodb, redis, and dynamodb raise FeastExtrasDependencyImportError
at import time when their optional Python extras are not installed.
test_all.py only caught ModuleNotFoundError, so any such import caused the
entire test_docstrings() function to abort rather than gracefully skipping
the module.
Extend the except clause to also catch FeastExtrasDependencyImportError so
the doctest run completes for all other modules when an optional extra is
absent in the test environment.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: update PYTEST_PLUGINS path for mongodb online store tests
Directory was renamed from tests/integration/feature_repos to
tests/universal/feature_repos in upstream/master. Update the
PYTEST_PLUGINS module path in test-python-universal-mongodb-online
to match the new location.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: broaden import exception handling in doctest runner to catch TypeError and other third-party import errors
qdrant_client raises TypeError at import time when a gRPC EnumTypeWrapper
is used with the | union operator on Python < 3.12. The previous fix only
caught ModuleNotFoundError and FeastExtrasDependencyImportError, leaving
the test runner vulnerable to any other exception raised by third-party
libraries during pkgutil.walk_packages.
Changes:
- Catch bare Exception in the import try/except block so any import-time
error from an optional dependency causes a graceful skip rather than an
abort of the entire test run.
- Initialize temp_module = None before the try block and add a continue
guard so that a failed import never leaves a stale module reference to
be used in the subsequent doctest execution block.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: pass onerror to pkgutil.walk_packages to suppress non-ImportError package import failures
FeastExtrasDependencyImportError inherits from FeastError -> Exception, not
from ImportError. pkgutil.walk_packages calls __import__ internally when
recursing into sub-packages, and only silently swallows ImportError subclasses;
any other exception is re-raised, crashing the entire test run.
Passing onerror=lambda _: None tells walk_packages to skip any package that
fails to import during the walk phase, regardless of the exception type. The
inner importlib.import_module try/except already handles the same errors for
the explicit import step used to collect doctests.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: update stale tests.integration.feature_repos imports to tests.universal.feature_repos
The upstream directory tests/integration/feature_repos was renamed to
tests/universal/feature_repos. Three files still referenced the old path:
- mongodb_repo_configuration.py: IntegrationTestRepoConfig and
MongoDBOnlineStoreCreator imports
- tests/.../online_store/mongodb.py: OnlineStoreCreator import
- tests/unit/online_store/test_mongodb_online_retrieval.py: TAGS import
Updating all three unblocks test collection and allows
make test-python-universal-mongodb-online to run locally.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* feat: add mongodb to ValidOnlineStoreDBStorePersistenceTypes in feast-operator
The Python ONLINE_STORE_CLASS_FOR_TYPE now includes 'mongodb', so the
operator's parity check (test-datasources) fails unless the Go API also
lists it. Add 'mongodb' to ValidOnlineStoreDBStorePersistenceTypes in
both api/v1 and api/v1alpha1.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* feat: add Feast driver metadata to MongoDB client instantiations
Add DriverInfo with the Feast name and version to both the sync
MongoClient and async AsyncMongoClient so that MongoDB can identify
traffic originating from a Feast AI integration.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* docs: update MongoDB online store status from alpha to preview
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: add mongodb to kubebuilder Enum annotations for OnlineStoreDBStorePersistence
The +kubebuilder:validation:Enum annotation on the Type field of
OnlineStoreDBStorePersistence was not updated when mongodb was added to
ValidOnlineStoreDBStorePersistenceTypes. This annotation drives CRD
OpenAPI schema validation at Kubernetes admission time, so any
FeatureStore CR specifying type: mongodb would be rejected by the API
server. Updated both api/v1 and api/v1alpha1.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Update +GOLANGCI_LINT_VERSION to fix upstream issue golang/go#74462
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: raise ValueError in _get_client_async for invalid config type, consistent with sync counterpart
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: Added mongodb to operator yamls
Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>
* Small change suggested by ntkathole
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Factor out write logic into utility function making sync/async essentially identical.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
---------
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: ntkathole <nikhilkathole2683@gmail.com>
Signed-off-by: Aniket Paluskar <apaluska@redhat.com>
yuan1j pushed a commit to yuan1j/feast that referenced this pull request
* Initial commit on INTPYTHON-297-MongoDB-Feast-Integration
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Added mongodb to project.optional-dependencies in pyproject.toml. Now pymongo is found as extra
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Checkpoint. Passing tests.unit.online_store.test_online_writes.TestOnlineWrites with default args of MongoDBOnlineStoreConfig. Lots to do.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Handle Nan in dfs for test_online_writes.py. Now all tests in the module pass
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Removed suffix of implementation: mongodb_openai -> mongodb
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Moved MongoDBOnlineStore to feast.infra.online_store.contrib
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Formatting
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Refactor online_read that converts bson to proto. Left two methods for now. simply and transformming
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Remove file added early during discovery
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Formatting
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Added version of test that uses testcontainers mongodb instead of assuming one is running
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Create Make target for universal tests
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Cleanup
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Removed temporary integration tests requiring one to spin up own mongodb server.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Format
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Typing
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Implemented ASync API and Tests
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Removed offline store stubs. The first PR will only contain the OnlineStore
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Moved mongodb_online_store out of cobtrib package.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Add documentation.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Cleanups and docstrings
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Fixed another reference to contrib dir
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Typos
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Made _convert_raw_docs_to_proto staticmethods private
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* After benchmarking two alogithm for conevrting read results from bson to proto, removed the naive one. It was outcompeted 3X across dimensions
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Add extra unit tests
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Formatting
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Fixes in pyproject.toml
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Fixed Detect secrets false positives.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Update sdk/python/feast/infra/online_stores/mongodb_online_store/mongodb.py
Set _client and _collection to None upon synchronous teardown.
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Fix CI: guard pymongo imports and skip test module when pymongo unavailable
In the unit-test-python CI job, pymongo is not installed (it is an optional
extra). This caused ModuleNotFoundError during pytest collection.
Two changes:
1. mongodb.py: wrap pymongo imports in try/except -> FeastExtrasDependencyImportError
(consistent with redis.py, dynamodb.py, datastore.py pattern)
2. test_mongodb_online_retrieval.py: add pytest.importorskip(\pymongo\) so
the entire test module is skipped gracefully when pymongo is absent
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Fix: return (None, None) when entity doc exists but feature view was never written
MongoDB stores all feature views for an entity in a single document.
If FV 'driver_stats' is written, an entity doc exists for driver_1.
A subsequent read for FV 'pricing' (never written) was previously
returning (None, {all_features: ValueProto()}) - a truthy feature dict
with all-empty protos - instead of the correct (None, None) sentinel.
Feast and downstream callers use (None, None) to signal entity absence.
A non-None feature dict means 'entity found, values are null', which
causes different behaviour in the feature retrieval pipeline.
Fix: after confirming the entity doc exists, also check that the
feature view key is present in doc['features']. If absent, return
(None, None) rather than a dict of empty protos.
Adds test_convert_raw_docs_entity_exists_but_fv_not_written to cover
the multi-feature-view colocation scenario identified in code review.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Update pixi.lock after adding mongodb optional dependency to pyproject.toml
Any change to pyproject.toml invalidates the pixi.lock manifest hash,
causing 'pixi install --locked' to fail in CI even when the changed
section (mongodb optional extras) is not used by any pixi environment.
Regenerated with: pixi install (v0.63.2)
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: catch FeastExtrasDependencyImportError in doctest runner
Stores like mongodb, redis, and dynamodb raise FeastExtrasDependencyImportError
at import time when their optional Python extras are not installed.
test_all.py only caught ModuleNotFoundError, so any such import caused the
entire test_docstrings() function to abort rather than gracefully skipping
the module.
Extend the except clause to also catch FeastExtrasDependencyImportError so
the doctest run completes for all other modules when an optional extra is
absent in the test environment.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: update PYTEST_PLUGINS path for mongodb online store tests
Directory was renamed from tests/integration/feature_repos to
tests/universal/feature_repos in upstream/master. Update the
PYTEST_PLUGINS module path in test-python-universal-mongodb-online
to match the new location.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: broaden import exception handling in doctest runner to catch TypeError and other third-party import errors
qdrant_client raises TypeError at import time when a gRPC EnumTypeWrapper
is used with the | union operator on Python < 3.12. The previous fix only
caught ModuleNotFoundError and FeastExtrasDependencyImportError, leaving
the test runner vulnerable to any other exception raised by third-party
libraries during pkgutil.walk_packages.
Changes:
- Catch bare Exception in the import try/except block so any import-time
error from an optional dependency causes a graceful skip rather than an
abort of the entire test run.
- Initialize temp_module = None before the try block and add a continue
guard so that a failed import never leaves a stale module reference to
be used in the subsequent doctest execution block.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: pass onerror to pkgutil.walk_packages to suppress non-ImportError package import failures
FeastExtrasDependencyImportError inherits from FeastError -> Exception, not
from ImportError. pkgutil.walk_packages calls __import__ internally when
recursing into sub-packages, and only silently swallows ImportError subclasses;
any other exception is re-raised, crashing the entire test run.
Passing onerror=lambda _: None tells walk_packages to skip any package that
fails to import during the walk phase, regardless of the exception type. The
inner importlib.import_module try/except already handles the same errors for
the explicit import step used to collect doctests.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: update stale tests.integration.feature_repos imports to tests.universal.feature_repos
The upstream directory tests/integration/feature_repos was renamed to
tests/universal/feature_repos. Three files still referenced the old path:
- mongodb_repo_configuration.py: IntegrationTestRepoConfig and
MongoDBOnlineStoreCreator imports
- tests/.../online_store/mongodb.py: OnlineStoreCreator import
- tests/unit/online_store/test_mongodb_online_retrieval.py: TAGS import
Updating all three unblocks test collection and allows
make test-python-universal-mongodb-online to run locally.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* feat: add mongodb to ValidOnlineStoreDBStorePersistenceTypes in feast-operator
The Python ONLINE_STORE_CLASS_FOR_TYPE now includes 'mongodb', so the
operator's parity check (test-datasources) fails unless the Go API also
lists it. Add 'mongodb' to ValidOnlineStoreDBStorePersistenceTypes in
both api/v1 and api/v1alpha1.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* feat: add Feast driver metadata to MongoDB client instantiations
Add DriverInfo with the Feast name and version to both the sync
MongoClient and async AsyncMongoClient so that MongoDB can identify
traffic originating from a Feast AI integration.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* docs: update MongoDB online store status from alpha to preview
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: add mongodb to kubebuilder Enum annotations for OnlineStoreDBStorePersistence
The +kubebuilder:validation:Enum annotation on the Type field of
OnlineStoreDBStorePersistence was not updated when mongodb was added to
ValidOnlineStoreDBStorePersistenceTypes. This annotation drives CRD
OpenAPI schema validation at Kubernetes admission time, so any
FeatureStore CR specifying type: mongodb would be rejected by the API
server. Updated both api/v1 and api/v1alpha1.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Update +GOLANGCI_LINT_VERSION to fix upstream issue golang/go#74462
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: raise ValueError in _get_client_async for invalid config type, consistent with sync counterpart
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: Added mongodb to operator yamls
Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>
* Small change suggested by ntkathole
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Factor out write logic into utility function making sync/async essentially identical.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
---------
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: ntkathole <nikhilkathole2683@gmail.com>
Signed-off-by: yuanjun220 <1069645408@qq.com>
franciscojavierarceo pushed a commit that referenced this pull request
# [0.61.0](v0.60.0...v0.61.0) (2026-04-07) ### Bug Fixes * Add grpcio dependency group to transformation server Dockerfile ([2c2150a](2c2150a)) * Add https readiness check for rest-registry tests ([ea85e63](ea85e63)) * Add website build check for PRs and fix blog frontmatter YAML error ([#6079](#6079)) ([30a3a43](30a3a43)) * Added missing jackc/pgx/v5 entries ([94ad0e7](94ad0e7)) * Added MLflow metric charts across feature selection ([#6080](#6080)) ([a403361](a403361)) * Check duplicate names for feature view across types ([#5999](#5999)) ([95b9af8](95b9af8)) * Fix integration tests ([#6046](#6046)) ([02d5548](02d5548)) * Fix missing error handling for resource_counts endpoint ([d9706ce](d9706ce)) * Fix non-specific label selector on metrics service ([a1a160d](a1a160d)) * fix path feature_definitions.py ([7d7df68](7d7df68)) * Fix regstry Rest API tests intermittent failure ([d53a339](d53a339)) * Fixed IntegrityError on SqlRegistry ([#6047](#6047)) ([325e148](325e148)) * Fixed intermittent failures in get_historical_features ([c335ec7](c335ec7)) * Fixed pre-commit check ([114b7db](114b7db)) * Fixed the intermittent FeatureViewNotFoundException ([661ecc7](661ecc7)) * Fixed uv cache permission error for docker build on mac ([ad807be](ad807be)) * Fixes a `PydanticDeprecatedSince20` warning for trino_offline_store ([#5991](#5991)) ([abfd18a](abfd18a)) * Handle existing RBAC role gracefully in namespace registry ([b46a62b](b46a62b)) * Ignore ipynb files during apply ([#6151](#6151)) ([4ea123d](4ea123d)) * Integration test failures ([#6040](#6040)) ([9165870](9165870)) * Mount TLS volumes for init container ([080a9b5](080a9b5)) * **postgres:** Use end_date in synthetic entity_df for non-entity retrieval ([#6110](#6110)) ([088a802](088a802)), closes [#6066](#6066) * Ray offline store tests are duplicated across 3 workflows ([54f705a](54f705a)) * Reenable tests ([#6036](#6036)) ([82ee7f8](82ee7f8)) * SSL/TLS mode by default for postgres connection ([4844488](4844488)) * Use commitlint pre-commit hook instead of a separate action ([35a81e7](35a81e7)) ### Features * Add Claude Code agent skills for Feast ([#6081](#6081)) ([1e5b60f](1e5b60f)), closes [#5976](#5976) [#6007](#6007) * Add complex type support (Map, JSON, Struct) with schema validation ([#5974](#5974)) ([1200dbf](1200dbf)) * Add decimal to supported feature types ([#6029](#6029)) ([#6226](#6226)) ([cff6fbf](cff6fbf)) * Add feast apply init container to automate registry population on pod start ([#6106](#6106)) ([6b31a43](6b31a43)) * Add feature view versioning support to PostgreSQL and MySQL online stores ([#6193](#6193)) ([940e0f0](940e0f0)), closes [#6168](#6168) [#6169](#6169) [#2728](#2728) * Add materialization, feature freshness, request latency, and push metrics to feature server ([2c6be18](2c6be18)) * Add metadata statistics to registry api ([ef1d4fc](ef1d4fc)) * Add non-entity retrieval support for ClickHouse offline store ([4d08ddc](4d08ddc)), closes [#5835](#5835) * Add OnlineStore for MongoDB ([#6025](#6025)) ([bf4e3fa](bf4e3fa)), closes [golang/go#74462](golang/go#74462) * Add Oracle DB as Offline store in python sdk & operator ([#6017](#6017)) ([9d35368](9d35368)) * Add RBAC aggregation labels to FeatureStore ClusterRoles ([daf77c6](daf77c6)) * Add ServiceMonitor auto-generation for Prometheus discovery ([#6126](#6126)) ([56e6d21](56e6d21)) * Add typed_features field to grpc write request (([#6117](#6117)) ([#6118](#6118)) ([eeaa6db](eeaa6db)), closes [#6116](#6116) * Add UUID and TIME_UUID as feature types ([#5885](#5885)) ([#5951](#5951)) ([5d6e311](5d6e311)) * Add version indicators to lineage graph nodes ([#6187](#6187)) ([73805d3](73805d3)) * Add version tracking to FeatureView ([#6101](#6101)) ([ed4a4f2](ed4a4f2)) * Added Agent skills for AI Agents ([#6007](#6007)) ([99008c8](99008c8)) * Added CodeQL SAST scanning and detect-secrets pre-commit hook ([547b516](547b516)) * Added odfv transformations metrics ([8b5a526](8b5a526)) * Adding optional name to Aggregation (feast-dev[#5994](#5994)) ([#6083](#6083)) ([56469f7](56469f7)) * Created DocEmbedder class ([#5973](#5973)) ([0719c06](0719c06)) * Extended OIDC support to extract groups & namespaces and token injection with multiple methods ([#6089](#6089)) ([7c04026](7c04026)) * Feature Server High-Availability on Kubernetes ([#6028](#6028)) ([9c07b4c](9c07b4c)), closes [Hi#Availability](https://github.com/Hi/issues/Availability) [Hi#Availability](https://github.com/Hi/issues/Availability) * **go:** Implement metrics and tracing for http and grpc servers ([#5925](#5925)) ([2b4ec9a](2b4ec9a)) * Horizontal scaling support to the Feast operator ([#6000](#6000)) ([3ec13e6](3ec13e6)) * Making feature view source optional (feast-dev[#6074](#6074)) ([#6075](#6075)) ([76917b7](76917b7)) * Replace ORJSONResponse with Pydantic response models for faster JSON serialization ([65cf03c](65cf03c)) * Support arm docker build ([#6061](#6061)) ([1e1f5d9](1e1f5d9)) * Support distinct count aggregation [[#6116](#6116)] ([3639570](3639570)) * Support HTTP in MCP ([#6109](#6109)) ([e72b983](e72b983)) * Support nested collection types (Array/Set of Array/Set) ([#5947](#5947)) ([#6132](#6132)) ([ab61642](ab61642)) * Support podAnnotations on Deployment pod template ([1b3cdc1](1b3cdc1)) * Use orjson for faster JSON serialization in feature server ([6f5203a](6f5203a)) * Utilize date partition column in BigQuery ([#6076](#6076)) ([4ea9b32](4ea9b32)) ### Performance Improvements * Online feature response construction in a single pass over read rows ([113fb04](113fb04)) * Optimize protobuf parsing in Redis online store ([#6023](#6023)) ([59dfdb8](59dfdb8)) * Optimize timestamp conversion in _convert_rows_to_protobuf ([33a2e95](33a2e95)) * Parallelize DynamoDB batch reads in sync online_read ([#6024](#6024)) ([9699944](9699944)) * Remove redundant entity key serialization in online_read ([d87283f](d87283f))
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters