feat: Make arrow primary interchange for offline ODFV execution by tokoko · Pull Request #4083 · feast-dev/feast

What this PR does / why we need it:

ODFV execution in offline flow revolves around pandas as both pandas and substrait transformations take pandas dataframes as an input. substrait has an unnecessary performance penalty as a consequence because it has to then convert pandas to arrow before execution. This PR flips the order around and makes arrow the primary interchange for execution, therefore substrait transformation can work without ever touching pandas.

P.S. This PR only touches offline flow for now just to make the scope smaller. I'll make necessary changes to the online side later in a separate PR and get rid of some of the dead code that will be left unused.