◐ Shell
clean mode source ↗

feat: Use ASOF JOIN in Snowflake offline store query by nanohanno · Pull Request #4850 · feast-dev/feast

Conversation

@nanohanno

What this PR does / why we need it:

The query template for the Snowflake Offline Store is updated to use the recently added ASOF JOIN (https://www.snowflake.com/en/blog/time-series-analytics-asof-join-generally-available/). We have seen a very substantial improvement in feature queries using the new template. Especially, for large feature tables that have many historical entries for certain entity values the performance is even better than using the ttl parameter in the current query template.

Which issue(s) this PR fixes:

Fixes #4732

Misc

@bimtauer and @miianiemela have shown interest in the feature, are you able to take a look and test it in your setup?

Signed-off-by: hkuepers <hanno.kuepers@ratepay.com>

franciscojavierarceo

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, thank you for this!

@franciscojavierarceo

Looks like integration tests are failing.

Signed-off-by: hkuepers <hanno.kuepers@ratepay.com>

@nanohanno

Looks like integration tests are failing.

I did not realize that there are integation tests for Snowflake, that is great.
There was a bug in the template for features without entities that we did not check before. Hopefully the fix solves the failing test now.

hkuepers added 3 commits

December 19, 2024 12:51
Signed-off-by: hkuepers <hanno.kuepers@ratepay.com>
Signed-off-by: hkuepers <hanno.kuepers@ratepay.com>
Signed-off-by: hkuepers <hanno.kuepers@ratepay.com>

@franciscojavierarceo

Looks like integration tests are failing.

I did not realize that there are integation tests for Snowflake, that is great. There was a bug in the template for features without entities that we did not check before. Hopefully the fix solves the failing test now.

Hmm. Looks like it's still failing fyi.

Signed-off-by: hkuepers <hanno.kuepers@ratepay.com>

@nanohanno

Looks like integration tests are failing.

I did not realize that there are integation tests for Snowflake, that is great. There was a bug in the template for features without entities that we did not check before. Hopefully the fix solves the failing test now.

Hmm. Looks like it's still failing fyi.

Yeah, I did not consider setting up the Snowflake integration tests locally and debugging from the CI logs for the template query is a little painful. I have done some improvements today and am still positive that it is close to being there 🙏 . If not I will set up the Snowflake environment in the next days to see the errors in Snowsight.

franciscojavierarceo

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for this!

tmihalac pushed a commit to opendatahub-io/feast that referenced this pull request

Jan 21, 2025

Labels

2 participants

@nanohanno @franciscojavierarceo