feat: Use ASOF JOIN in Snowflake offline store query by nanohanno · Pull Request #4850 · feast-dev/feast
Conversation
What this PR does / why we need it:
The query template for the Snowflake Offline Store is updated to use the recently added ASOF JOIN (https://www.snowflake.com/en/blog/time-series-analytics-asof-join-generally-available/). We have seen a very substantial improvement in feature queries using the new template. Especially, for large feature tables that have many historical entries for certain entity values the performance is even better than using the ttl parameter in the current query template.
Which issue(s) this PR fixes:
Fixes #4732
Misc
@bimtauer and @miianiemela have shown interest in the feature, are you able to take a look and test it in your setup?
Signed-off-by: hkuepers <hanno.kuepers@ratepay.com>
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice, thank you for this!
Signed-off-by: hkuepers <hanno.kuepers@ratepay.com>
Looks like integration tests are failing.
I did not realize that there are integation tests for Snowflake, that is great.
There was a bug in the template for features without entities that we did not check before. Hopefully the fix solves the failing test now.
hkuepers added 3 commits
Signed-off-by: hkuepers <hanno.kuepers@ratepay.com>
Signed-off-by: hkuepers <hanno.kuepers@ratepay.com>
Signed-off-by: hkuepers <hanno.kuepers@ratepay.com>
Looks like integration tests are failing.
I did not realize that there are integation tests for Snowflake, that is great. There was a bug in the template for features without entities that we did not check before. Hopefully the fix solves the failing test now.
Hmm. Looks like it's still failing fyi.
Signed-off-by: hkuepers <hanno.kuepers@ratepay.com>
Looks like integration tests are failing.
I did not realize that there are integation tests for Snowflake, that is great. There was a bug in the template for features without entities that we did not check before. Hopefully the fix solves the failing test now.
Hmm. Looks like it's still failing fyi.
Yeah, I did not consider setting up the Snowflake integration tests locally and debugging from the CI logs for the template query is a little painful. I have done some improvements today and am still positive that it is close to being there 🙏 . If not I will set up the Snowflake environment in the next days to see the errors in Snowsight.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for this!