◐ Shell
reader mode source ↗
Skip to content

feat: Unify Feature Transformations and Feature Views#5747

Open
franciscojavierarceo wants to merge 34 commits into
masterfrom
refactor-odfv
Open

feat: Unify Feature Transformations and Feature Views#5747
franciscojavierarceo wants to merge 34 commits into
masterfrom
refactor-odfv

Conversation

@franciscojavierarceo

@franciscojavierarceo franciscojavierarceo commented Nov 28, 2025 •
edited
Loading

Copy link
Copy Markdown
Member

What this PR does / why we need it:

This PR refactors the transformation system with a cleaner architecture that separates transformation logic from execution. Transformations should focus on HOW to transform data, while FeatureViews handle WHEN and WHERE to execute.

Changes:

  • Added: feature_transformation field to FeatureView for transformation logic
  • Added: transform: bool = True parameter to API methods for per-call control
  • Added: Schema validation for pre-transformed data (transform=False)
  • Added: Transformation execution logic in online/offline feature retrieval
  # Define transformation logic
  @transformation(mode="python")
  def driver_features(inputs):
      return {"conv_plus_acc": [c + a for c, a in zip(inputs["conv_rate"], inputs["acc_rate"])]}

  # Create FeatureView with transformation
  fv = FeatureView(
      name="driver_computed_features",
      source=driver_source,
      entities=[driver],
      schema=[Field(name="conv_plus_acc", dtype=Float64)],
      feature_transformation=driver_features,  # ← NEW
      online=True,
  )

  # API-level transformation control
  # Execute transformations (default)
  features = store.get_online_features(
      features=["driver_computed_features:conv_plus_acc"],
      entity_rows=entity_rows,
      transform=True  # ← NEW (default)
  )

  # Skip transformations for external batch jobs
  features = store.get_online_features(
      features=["driver_computed_features:conv_plus_acc"],
      entity_rows=spark_transformed_rows,
      transform=False  # ← NEW: skip + validate schema
  )

Which issue(s) this PR fixes:

#4584, #5716, #5689

Misc

@franciscojavierarceo franciscojavierarceo requested a review from a team as a code owner November 28, 2025 18:20
@franciscojavierarceo franciscojavierarceo force-pushed the refactor-odfv branch 2 times, most recently from 2899413 to 649c429 Compare December 1, 2025 21:47
@HaoXuAI

HaoXuAI commented Dec 16, 2025 •
edited
Loading

Copy link
Copy Markdown
Collaborator

I think when can be a bit confusing. even for 'batch', it can be 'on_read' or 'on_write'. I think on a different angle, should transformation be in the scope of writing, and isn't transformation only responsible for transforming the data?

FeatureView as the actually data assets, I think makes more sense to having the API.

@franciscojavierarceo

franciscojavierarceo commented Dec 23, 2025

Copy link
Copy Markdown
Member Author

I think when can be a bit confusing. even for 'batch', it can be 'on_read' or 'on_write'. I think on a different angle, should transformation be in the scope of writing, and isn't transformation only responsible for transforming the data?

FeatureView as the actually data assets, I think makes more sense to having the API.

Yeah, that's fair. @HaoXuAI I updated the PR, lmk what you think.

@franciscojavierarceo franciscojavierarceo requested a review from a team as a code owner January 5, 2026 04:17
@franciscojavierarceo franciscojavierarceo requested review from ejscribner, robhowley and shuchu and removed request for a team January 5, 2026 04:17
@franciscojavierarceo franciscojavierarceo force-pushed the refactor-odfv branch 2 times, most recently from 5abbec2 to e5f074e Compare January 5, 2026 15:18
@franciscojavierarceo
feat: Add unified transformation …
837e34f 
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
@franciscojavierarceo
feat: Unify transformations …
d12fbfd 
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
@franciscojavierarceo
feat: Unify Transformations …
9a2df49 
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
@franciscojavierarceo
feat: Unify Transformations …
9aceb7f 
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
@franciscojavierarceo
updated docs …
5c8b93c 
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
@franciscojavierarceo
refactor: separate transformation logic from execution decisions with… …
6d5ce47 
… auto-inference

Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
@franciscojavierarceo
format …
9380cf9 
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
@franciscojavierarceo
incorporaitng feedback …
5b759ed 
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
@franciscojavierarceo
updated …
2d7a43b 
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
@franciscojavierarceo
fix …
b6299d2 
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
@franciscojavierarceo
linter …
3726733 
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
@franciscojavierarceo
cleanup …
f55d4b4 
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
@franciscojavierarceo
cleanup …
716e692 
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
@franciscojavierarceo
more fix …
a4f2e0a 
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
@franciscojavierarceo
more fix …
02ae40b 
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
@franciscojavierarceo
updated …
74de467 
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
@franciscojavierarceo
fix …
c360aef 
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
@franciscojavierarceo
fix …
f73431c 
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
@franciscojavierarceo
fix …
50e536a 
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
@franciscojavierarceo
fix …
a87c4b4 
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
franciscojavierarceo and others added 14 commits January 5, 2026 11:42
@franciscojavierarceo
Merge branch 'master' into refactor-odfv
e2f722c
@franciscojavierarceo
lint …
dde05bd 
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
@franciscojavierarceo
fix …
0e1f037 
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
@franciscojavierarceo
linter …
662e21b 
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
@franciscojavierarceo
fix linter …
9f48c75 
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
@franciscojavierarceo
fix …
a058aac 
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
@jatin5251 @franciscojavierarceo
fix(redis): Preserve millisecond timestamp precision for Redis online… …
b8771c9 
… store (#5807)

* Update redis.py

Add millisecond-precision timestamp support to Redis online store

Signed-off-by: Jatin Kumar <jatink.5251@gmail.com>

* Update redis.py

sub-second precision when returning timestamps to client

Signed-off-by: Jatin Kumar <jatink.5251@gmail.com>

* Update redis.py

fix(redis): preserve millisecond timestamp precision

Signed-off-by: Jatin Kumar <jatink.5251@gmail.com>

* Update redis.py

fix: Remove whitespace on blank lines (W293)

Signed-off-by: Jatin Kumar <jatink.5251@gmail.com>

---------

Signed-off-by: Jatin Kumar <jatink.5251@gmail.com>
@samuelkim7 @franciscojavierarceo
feat: Add GCS registry store in Go feature server (#5818) …
34d9b52 
Signed-off-by: samuelkim7 <samuel.kim@goflink.com>
@franciscojavierarceo
chore: Refactor some unit tests into integration tests (#5820) …
6c35d45 
* chore: Refactor some unit tests into integration tests

Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>

* chore: Refactor some unit tests into integration tests

Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>

* rename TestConfig

Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>

* rename TestConfig

Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>

* add integration flag

Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>

* update paths

Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>

* update paths

Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>

---------

Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
@Srihari1192 @franciscojavierarceo
test: Remove e2e_rhoai package tests …
5bcd6e6 
Signed-off-by: Srihari <svenkata@redhat.com>
@franciscojavierarceo
fix …
61812ba 
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
@franciscojavierarceo
fix …
d4adcd5 
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
@franciscojavierarceo
fix …
a007be3 
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
@franciscojavierarceo
uploading progress …
416b15c 
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ntkathole ntkathole ntkathole left review comments

@HaoXuAI HaoXuAI HaoXuAI left review comments

@shuchu shuchu Awaiting requested review from shuchu shuchu is a code owner automatically assigned from feast-dev/maintainers

@ejscribner ejscribner Awaiting requested review from ejscribner ejscribner is a code owner automatically assigned from feast-dev/maintainers

@robhowley robhowley Awaiting requested review from robhowley robhowley is a code owner automatically assigned from feast-dev/maintainers

Assignees

No one assigned

Labels

ok-to-test

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@franciscojavierarceo @HaoXuAI @ntkathole @jatin5251 @samuelkim7 @Srihari1192