◐ Shell
clean mode source ↗

fix: Add __eq__, __hash__ to SparkSource for correct comparison by ElliotNguyen68 · Pull Request #4028 · feast-dev/feast

Those are good points, but I think we should still simply check for absolute equality, so priority doesn't really matter, if there is a change in query or if user changes query to None and switches to using table, this method should return False. In some cases, the change might not actually have an impact i guess, but it's still a change. So a single expression should be fine.

As for the question about 3 competing parameters (table, query and path), some sort of a constraint is probably a good idea, but imho the whole idea of using SparkSource for everything was a really bad decision in the first place. If the user wants to register a parquet folder as a data source for example, i believe the way to do that should be by providing FileSource object instead of a SparkSource one... To achieve that, we need to teach SparkOfflineStore how to read FileSource and deprecate path parameter from SparkSource... but that's a discussion for another day and a little out of scope here :)