feat: Add Set as feature type#5888

Copilot

What this PR does / why we need it:

This PR adds support for Set as a feature type in Feast, allowing users to store collections of unique values with automatic deduplication.

Changes Made

1. Protobuf Definitions (Value.proto)

Added 8 new SET type variants for primitive types:
- INT32_SET, INT64_SET, FLOAT_SET, DOUBLE_SET
- STRING_SET, BYTES_SET, BOOL_SET, UNIX_TIMESTAMP_SET
Added corresponding protobuf message types (Int32Set, Int64Set, etc.)
Updated the Value message to include set fields
Compiled with protobuf version 4.25.1 and mypy-protobuf 3.3.0 (matching CI environment)

2. Python SDK Updates

value_type.py: Added set types to the ValueType enum and defined SetType union
types.py:
- Created new Set class (ComplexFeastType) similar to Array
- Added set type mappings in VALUE_TYPES_TO_FEAST_TYPES
- Set types support all primitive types except Map
type_map.py:
- Added PYTHON_SET_VALUE_TYPE_TO_PROTO_VALUE mapping
- Implemented Python → Proto conversions for sets with automatic duplicate removal
- Implemented Proto → Python conversions returning native Python sets
- Updated PROTO_VALUE_TO_VALUE_TYPE_MAP with set field mappings
- Added SET types to pandas type mapping (returns "object" type)
- Added set type inference in python_type_to_feast_value_type to properly handle Python sets passed in entity_rows
- Fixed None value handling in all set and list value conversions to prevent "TypeError: 'NoneType' object is not iterable" errors

3. Tests

Added unit tests in test_types.py for Set type creation and round-trip conversion
Added comprehensive tests in test_type_map.py for:
- Set conversion (string, int, float, bool sets)
- Duplicate removal from lists when converting to sets
- Empty and null set handling
- Multiple set value conversions
Added test_python_transformation_with_set_types in test_on_demand_python_transformation.py for:
- Set types in On Demand Feature Views
- Automatic deduplication of set values
- Conversion from lists to sets
- Online feature retrieval with Set types

4. Documentation

Updated docs/reference/type-system.md with:
- Set type table showing all supported set variants
- Usage examples demonstrating duplicate removal
- Updated complete feature view example with set fields

Usage Example

from feast.types import Set, String, Int32
from feast import Field

# Define fields with set types
schema = [
    Field(name="visited_pages", dtype=Set(String)),
    Field(name="unique_categories", dtype=Set(Int32)),
]

# Data with duplicates
data = {
    "visited_pages": {"home", "products", "checkout", "products"},  # duplicate removed
    "unique_categories": {1, 2, 3, 2, 1}  # duplicates removed
}
# Feast stores: {"home", "products", "checkout"} and {1, 2, 3}

Benefits

Automatic deduplication: Sets automatically remove duplicate values
Type safety: Full type checking and validation
Consistent API: Set types follow the same patterns as Array types
Comprehensive support: All primitive types (except Map) have set variants

Misc

Protobuf changes are backward compatible. Existing LIST and MAP types are unaffected.
Generated protobuf files use protobuf 4.25.1 and mypy-protobuf 3.3.0, matching the versions specified in CI requirements.
All linting checks pass (make format-python and make lint-python)
Unit tests pass for set type functionality
Fixed multiple None value handling bugs in type conversion logic to ensure robustness

Original prompt

This section details on the original issue you should resolve

<issue_title>Add set as feature type</issue_title>
<issue_description>Is your feature request related to a problem? Please describe.
Support for set features as feast feature types is missing.

Describe the solution you'd like
Add support for set type features in feast.

Describe alternatives you've considered
We have considered storing set features as list or maps but then burden is on calling client to covert back.
</issue_description>

<agent_instructions>please add this to the feast types and update the docs</agent_instructions>

Comments on the Issue (you are @copilot in this section)

Fixes Add set as feature type #5886

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

franciscojavierarceo

@copilot we should not update the protobuf version

Copilot

@copilot we should not update the protobuf version

Fixed! Reverted all generated protobuf files and recompiled with protobuf version 4.25.0 instead of 6.31.1. The set types are now properly generated without updating the protobuf version. See commit e11795d.

franciscojavierarceo

@copilot linter failed, please run make lint-python and make format-python

Copilot

@copilot linter failed, please run make lint-python and make format-python

Fixed formatting issues by running make format-python. The test file had some line length violations that have been corrected. All ruff checks now pass for the modified files. See commit a72e37d.

franciscojavierarceo

@copilot linter and unit tests are still failing, please fix.

franciscojavierarceo

@copilot look at the failed test:

FAILED sdk/python/tests/unit/test_on_demand_python_transformation.py::test_python_transformation_with_set_types - TypeError: 'NoneType' object is not iterable

Copilot

@copilot linter and unit tests are still failing, please fix.

Fixed two bugs that were causing failures:

UNIX_TIMESTAMP_SET field name: Was incorrectly using "int64_set_val" instead of "unix_timestamp_set_val" in the type mapping
Pandas type mapping: Added SET types to feast_value_type_to_pandas_type function to return "object" type

All ruff checks pass. See commit 055457b.

- Add 8 Set value types (BYTES_SET, STRING_SET, INT32_SET, INT64_SET, DOUBLE_SET, FLOAT_SET, BOOL_SET, UNIX_TIMESTAMP_SET) - Implement Set class with base type validation (no nested Sets/Maps) - Add type conversion logic with duplicate removal - Generate protobuf bindings and update type stubs - Add comprehensive tests and documentation for Set types Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com>

- Fix IndexParams len() check by tracking indices_added flag - Fix Unicode decode error for Set types by including them in numeric_vector_list_types - Fix Unicode decode error for complex types by using base64 encoding instead of decode() - Fix duplicate timestamp fields in Milvus insertion by filtering them from values_dict - Add missing timestamp field names to exclusion list in schema creation Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com>

The test was providing string values for item_id field which is defined as Int64 in the schema, causing protobuf serialization issues that could manifest as Unicode decode errors. Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com>

Extract the long set type handling code from _python_value_to_proto_value into a dedicated _python_set_to_proto_values function for better readability and maintainability. Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com>

devin-ai-integration

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no potential bugs to report.

View in Devin Review to see 5 additional flags.

franciscojavierarceo

overall this took a long time with Copilot but Claude was able to clean it up right away.

Copilot was close with some good prompting so that's a fun lesson.

devin-ai-integration

Devin Review found 1 new potential issue.

View issue and 9 additional flags in Devin Review.

Signed-off-by: yassinnouh21 <yassinnouh21@gmail.com>

# [0.60.0](v0.59.0...v0.60.0) (2026-02-17) ### Bug Fixes * Added a flag to correctly download the go binaries ([0f77135](0f77135)) * Adds mapping of date Trino's type into string Feast's type ([531e839](531e839)) * **ci:** Use uv run for pytest in master_only benchmark step ([#5957](#5957)) ([5096010](5096010)) * Disable materialized odfvs for historical retrieval ([#5880](#5880)) ([739d28a](739d28a)) * Fix linting and formatting issues ([#5907](#5907)) ([42ca14a](42ca14a)) * Make timestamp field handling compatible with Athena V3 ([#5936](#5936)) ([e2bad34](e2bad34)) * Support pgvector under non-default schema ([#5970](#5970)) ([c636cd4](c636cd4)) * unit tests not running on main branch ([#5909](#5909)) ([62fe664](62fe664)) * Update java dep which blocking release ([#5903](#5903)) ([a5b8186](a5b8186)) * Update the dockerfile with golang 1.24.12. ([#5918](#5918)) ([be1b522](be1b522)) * Use context.Background() in client constructors ([#5897](#5897)) ([984f93a](984f93a)) ### Features * Add blog post for PyTorch ecosystem announcement ([#5906](#5906)) ([d2eb629](d2eb629)) * Add blog post on Feast dbt integration ([#5915](#5915)) ([b3c8138](b3c8138)) * Add DynamoDB in-place list update support for array-based features ([#5916](#5916)) ([aa5973f](aa5973f)) * Add HTTP connection pooling for remote online store client ([#5895](#5895)) ([e022bf8](e022bf8)) * Add integration tests for dbt import ([#5899](#5899)) ([a444692](a444692)) * Add lazy initialization and feature service caching ([#5924](#5924)) ([b37b7d0](b37b7d0)) * Add multiple entity support to dbt integration ([#5901](#5901)) ([05a4fb5](05a4fb5)), closes [#5872](#5872) * Add PostgreSQL online store support for Go feature server ([#5963](#5963)) ([b8c6f3d](b8c6f3d)) * Add publish docker image of Go feature server. ([#5923](#5923)) ([759d8c6](759d8c6)) * Add Set as feature type ([#5888](#5888)) ([52458fc](52458fc)) * Added online server worker config support in operator ([#5926](#5926)) ([193c72a](193c72a)) * Added support for OpenLineage integration ([#5884](#5884)) ([df70d8d](df70d8d)) * Adjust ray offline store to support abfs(s) ADLS Azure Storage ([#5911](#5911)) ([d6c0b2d](d6c0b2d)) * Batch_engine config injection in feature_store.yaml through operator ([#5938](#5938)) ([455d56c](455d56c)) * Consolidate Python packaging - remove setup.py/setup.cfg, standardize on pyproject.toml and uv ([16696b8](16696b8)) * **go:** Add MySQL registry store support for Go feature server ([#5933](#5933)) ([19f9bb8](19f9bb8)) * Improve local dev experience with file-aware hooks and auto parallelization ([#5956](#5956)) ([839b79e](839b79e)) * Modernize precommit hooks and optimize test performance ([#5929](#5929)) ([ea7d4fa](ea7d4fa)) * Optimize container infrastructure for production ([#5881](#5881)) ([5ebdac8](5ebdac8)) * Optimize DynamoDB online store for improved latency ([#5889](#5889)) ([fcc8274](fcc8274))

# [0.60.0](feast-dev/feast@v0.59.0...v0.60.0) (2026-02-17) ### Bug Fixes * Added a flag to correctly download the go binaries ([0f77135](feast-dev@0f77135)) * Adds mapping of date Trino's type into string Feast's type ([531e839](feast-dev@531e839)) * **ci:** Use uv run for pytest in master_only benchmark step ([feast-dev#5957](feast-dev#5957)) ([5096010](feast-dev@5096010)) * Disable materialized odfvs for historical retrieval ([feast-dev#5880](feast-dev#5880)) ([739d28a](feast-dev@739d28a)) * Fix linting and formatting issues ([feast-dev#5907](feast-dev#5907)) ([42ca14a](feast-dev@42ca14a)) * Make timestamp field handling compatible with Athena V3 ([feast-dev#5936](feast-dev#5936)) ([e2bad34](feast-dev@e2bad34)) * Support pgvector under non-default schema ([feast-dev#5970](feast-dev#5970)) ([c636cd4](feast-dev@c636cd4)) * unit tests not running on main branch ([feast-dev#5909](feast-dev#5909)) ([62fe664](feast-dev@62fe664)) * Update java dep which blocking release ([feast-dev#5903](feast-dev#5903)) ([a5b8186](feast-dev@a5b8186)) * Update the dockerfile with golang 1.24.12. ([feast-dev#5918](feast-dev#5918)) ([be1b522](feast-dev@be1b522)) * Use context.Background() in client constructors ([feast-dev#5897](feast-dev#5897)) ([984f93a](feast-dev@984f93a)) ### Features * Add blog post for PyTorch ecosystem announcement ([feast-dev#5906](feast-dev#5906)) ([d2eb629](feast-dev@d2eb629)) * Add blog post on Feast dbt integration ([feast-dev#5915](feast-dev#5915)) ([b3c8138](feast-dev@b3c8138)) * Add DynamoDB in-place list update support for array-based features ([feast-dev#5916](feast-dev#5916)) ([aa5973f](feast-dev@aa5973f)) * Add HTTP connection pooling for remote online store client ([feast-dev#5895](feast-dev#5895)) ([e022bf8](feast-dev@e022bf8)) * Add integration tests for dbt import ([feast-dev#5899](feast-dev#5899)) ([a444692](feast-dev@a444692)) * Add lazy initialization and feature service caching ([feast-dev#5924](feast-dev#5924)) ([b37b7d0](feast-dev@b37b7d0)) * Add multiple entity support to dbt integration ([feast-dev#5901](feast-dev#5901)) ([05a4fb5](feast-dev@05a4fb5)), closes [feast-dev#5872](feast-dev#5872) * Add PostgreSQL online store support for Go feature server ([feast-dev#5963](feast-dev#5963)) ([b8c6f3d](feast-dev@b8c6f3d)) * Add publish docker image of Go feature server. ([feast-dev#5923](feast-dev#5923)) ([759d8c6](feast-dev@759d8c6)) * Add Set as feature type ([feast-dev#5888](feast-dev#5888)) ([52458fc](feast-dev@52458fc)) * Added online server worker config support in operator ([feast-dev#5926](feast-dev#5926)) ([193c72a](feast-dev@193c72a)) * Added support for OpenLineage integration ([feast-dev#5884](feast-dev#5884)) ([df70d8d](feast-dev@df70d8d)) * Adjust ray offline store to support abfs(s) ADLS Azure Storage ([feast-dev#5911](feast-dev#5911)) ([d6c0b2d](feast-dev@d6c0b2d)) * Batch_engine config injection in feature_store.yaml through operator ([feast-dev#5938](feast-dev#5938)) ([455d56c](feast-dev@455d56c)) * Consolidate Python packaging - remove setup.py/setup.cfg, standardize on pyproject.toml and uv ([16696b8](feast-dev@16696b8)) * **go:** Add MySQL registry store support for Go feature server ([feast-dev#5933](feast-dev#5933)) ([19f9bb8](feast-dev@19f9bb8)) * Improve local dev experience with file-aware hooks and auto parallelization ([feast-dev#5956](feast-dev#5956)) ([839b79e](feast-dev@839b79e)) * Modernize precommit hooks and optimize test performance ([feast-dev#5929](feast-dev#5929)) ([ea7d4fa](feast-dev@ea7d4fa)) * Optimize container infrastructure for production ([feast-dev#5881](feast-dev#5881)) ([5ebdac8](feast-dev@5ebdac8)) * Optimize DynamoDB online store for improved latency ([feast-dev#5889](feast-dev#5889)) ([fcc8274](feast-dev@fcc8274)) Signed-off-by: soojin <soojin@dable.io>

# [0.60.0](feast-dev/feast@v0.59.0...v0.60.0) (2026-02-17) ### Bug Fixes * Added a flag to correctly download the go binaries ([0f77135](feast-dev@0f77135)) * Adds mapping of date Trino's type into string Feast's type ([531e839](feast-dev@531e839)) * **ci:** Use uv run for pytest in master_only benchmark step ([feast-dev#5957](feast-dev#5957)) ([5096010](feast-dev@5096010)) * Disable materialized odfvs for historical retrieval ([feast-dev#5880](feast-dev#5880)) ([739d28a](feast-dev@739d28a)) * Fix linting and formatting issues ([feast-dev#5907](feast-dev#5907)) ([42ca14a](feast-dev@42ca14a)) * Make timestamp field handling compatible with Athena V3 ([feast-dev#5936](feast-dev#5936)) ([e2bad34](feast-dev@e2bad34)) * Support pgvector under non-default schema ([feast-dev#5970](feast-dev#5970)) ([c636cd4](feast-dev@c636cd4)) * unit tests not running on main branch ([feast-dev#5909](feast-dev#5909)) ([62fe664](feast-dev@62fe664)) * Update java dep which blocking release ([feast-dev#5903](feast-dev#5903)) ([a5b8186](feast-dev@a5b8186)) * Update the dockerfile with golang 1.24.12. ([feast-dev#5918](feast-dev#5918)) ([be1b522](feast-dev@be1b522)) * Use context.Background() in client constructors ([feast-dev#5897](feast-dev#5897)) ([984f93a](feast-dev@984f93a)) ### Features * Add blog post for PyTorch ecosystem announcement ([feast-dev#5906](feast-dev#5906)) ([d2eb629](feast-dev@d2eb629)) * Add blog post on Feast dbt integration ([feast-dev#5915](feast-dev#5915)) ([b3c8138](feast-dev@b3c8138)) * Add DynamoDB in-place list update support for array-based features ([feast-dev#5916](feast-dev#5916)) ([aa5973f](feast-dev@aa5973f)) * Add HTTP connection pooling for remote online store client ([feast-dev#5895](feast-dev#5895)) ([e022bf8](feast-dev@e022bf8)) * Add integration tests for dbt import ([feast-dev#5899](feast-dev#5899)) ([a444692](feast-dev@a444692)) * Add lazy initialization and feature service caching ([feast-dev#5924](feast-dev#5924)) ([b37b7d0](feast-dev@b37b7d0)) * Add multiple entity support to dbt integration ([feast-dev#5901](feast-dev#5901)) ([05a4fb5](feast-dev@05a4fb5)), closes [feast-dev#5872](feast-dev#5872) * Add PostgreSQL online store support for Go feature server ([feast-dev#5963](feast-dev#5963)) ([b8c6f3d](feast-dev@b8c6f3d)) * Add publish docker image of Go feature server. ([feast-dev#5923](feast-dev#5923)) ([759d8c6](feast-dev@759d8c6)) * Add Set as feature type ([feast-dev#5888](feast-dev#5888)) ([52458fc](feast-dev@52458fc)) * Added online server worker config support in operator ([feast-dev#5926](feast-dev#5926)) ([193c72a](feast-dev@193c72a)) * Added support for OpenLineage integration ([feast-dev#5884](feast-dev#5884)) ([df70d8d](feast-dev@df70d8d)) * Adjust ray offline store to support abfs(s) ADLS Azure Storage ([feast-dev#5911](feast-dev#5911)) ([d6c0b2d](feast-dev@d6c0b2d)) * Batch_engine config injection in feature_store.yaml through operator ([feast-dev#5938](feast-dev#5938)) ([455d56c](feast-dev@455d56c)) * Consolidate Python packaging - remove setup.py/setup.cfg, standardize on pyproject.toml and uv ([16696b8](feast-dev@16696b8)) * **go:** Add MySQL registry store support for Go feature server ([feast-dev#5933](feast-dev#5933)) ([19f9bb8](feast-dev@19f9bb8)) * Improve local dev experience with file-aware hooks and auto parallelization ([feast-dev#5956](feast-dev#5956)) ([839b79e](feast-dev@839b79e)) * Modernize precommit hooks and optimize test performance ([feast-dev#5929](feast-dev#5929)) ([ea7d4fa](feast-dev@ea7d4fa)) * Optimize container infrastructure for production ([feast-dev#5881](feast-dev#5881)) ([5ebdac8](feast-dev@5ebdac8)) * Optimize DynamoDB online store for improved latency ([feast-dev#5889](feast-dev#5889)) ([fcc8274](feast-dev@fcc8274)) Signed-off-by: Shizoqua <hr.lanreshittu@gmail.com>

Copilot AI assigned Copilot and franciscojavierarceo Jan 22, 2026

Copilot started work on behalf of franciscojavierarceo January 22, 2026 01:53 View session

Copilot AI changed the title ~~[WIP] Add support for set as feature type in feast~~ Jan 22, 2026

Copilot AI requested a review from franciscojavierarceo January 22, 2026 02:04

Copilot finished work on behalf of franciscojavierarceo January 22, 2026 02:04

franciscojavierarceo reviewed Jan 22, 2026

View reviewed changes

Copilot started work on behalf of franciscojavierarceo January 22, 2026 05:26 View session

Copilot AI requested a review from franciscojavierarceo January 22, 2026 05:30

Copilot finished work on behalf of franciscojavierarceo January 22, 2026 05:30

franciscojavierarceo reviewed Jan 22, 2026

View reviewed changes

Copilot started work on behalf of franciscojavierarceo January 22, 2026 18:16 View session

franciscojavierarceo reviewed Jan 22, 2026

View reviewed changes

Copilot AI requested a review from franciscojavierarceo January 22, 2026 18:20

Copilot finished work on behalf of franciscojavierarceo January 22, 2026 18:20

Copilot started work on behalf of franciscojavierarceo January 22, 2026 18:20 View session

Copilot finished work on behalf of franciscojavierarceo January 22, 2026 18:22

Copilot started work on behalf of franciscojavierarceo January 22, 2026 18:52 View session

Copilot finished work on behalf of franciscojavierarceo January 22, 2026 18:56

franciscojavierarceo added the ok-to-test label Jan 22, 2026

Copilot started work on behalf of franciscojavierarceo January 22, 2026 21:56 View session

Copilot finished work on behalf of franciscojavierarceo January 22, 2026 21:59

Copilot started work on behalf of franciscojavierarceo January 23, 2026 02:16 View session

Copilot finished work on behalf of franciscojavierarceo January 23, 2026 02:23

Copilot finished work on behalf of franciscojavierarceo January 26, 2026 04:18

franciscojavierarceo force-pushed the copilot/add-set-as-feature-type branch from 8c02e44 to 821b1f2 Compare January 27, 2026 05:10

franciscojavierarceo and others added 8 commits January 27, 2026 08:10

Fix item_id type mismatch in milvus test …

a56b774

The test was providing string values for item_id field which is defined as Int64 in the schema, causing protobuf serialization issues that could manifest as Unicode decode errors. Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com>

Apply ruff formatting

325edb7

Merge branch 'master' into copilot/add-set-as-feature-type

947bfb7

Apply ruff formatting to refactored code

45251de

Delete test_registry

2ad1eec

Merge branch 'master' into copilot/add-set-as-feature-type

4f7128a

franciscojavierarceo marked this pull request as ready for review January 28, 2026 04:49

franciscojavierarceo requested a review from a team as a code owner January 28, 2026 04:49

franciscojavierarceo reviewed Jan 28, 2026

View reviewed changes

franciscojavierarceo approved these changes Jan 28, 2026

View reviewed changes

devin-ai-integration Bot reviewed Jan 28, 2026

View reviewed changes

franciscojavierarceo approved these changes Jan 28, 2026

View reviewed changes

Merge branch 'master' into copilot/add-set-as-feature-type

5e87fc9

devin-ai-integration Bot reviewed Jan 28, 2026

View reviewed changes

Merge branch 'master' into copilot/add-set-as-feature-type

4a45b75

HaoXuAI reviewed Jan 29, 2026

View reviewed changes

YassinNouh21 pushed a commit to YassinNouh21/feast that referenced this pull request Feb 7, 2026

feat: Add Set as feature type (feast-dev#5888) …

59a041d

Signed-off-by: yassinnouh21 <yassinnouh21@gmail.com>

YassinNouh21 pushed a commit to YassinNouh21/feast that referenced this pull request Feb 7, 2026

feat: Add Set as feature type (feast-dev#5888) …

8779f55

Signed-off-by: yassinnouh21 <yassinnouh21@gmail.com>

soooojinlee mentioned this pull request Feb 8, 2026

feat: Add UUID and TIME_UUID as feature types (#5885) #5951

Merged

jyejare pushed a commit to opendatahub-io/feast that referenced this pull request Mar 9, 2026

feat: Add Set as feature type (feast-dev#5888)

0bbfb81

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Add Set as feature type#5888