fix(spark): Replace mapInArrow with foreachPartition for materialization by abhijeet-dhumal · Pull Request #6370 · feast-dev/feast
…config forwarding for vector store materialization Signed-off-by: abhijeet-dhumal <abhijeetdhumal652@gmail.com>
abhijeet-dhumal
changed the title
fix(spark): replace mapInArrow with foreachPartition and fix session …
fix(spark): Replace mapInArrow with foreachPartition for materialization
SparkSource previously required exactly one of table/query/path. This relaxes the constraint to allow query + path together: - query: used for reading raw data during materialization - path: used for offline write-back (offline=True) and as pre-computed read source in get_historical_features Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters