feat: Add complex type support (Map, JSON, Struct) with schema validation by ntkathole · Pull Request #5974 · feast-dev/feast

ntkathole · 2026-02-16T12:23:43Z

What this PR does / why we need it:

This PR adds comprehensive support for complex data types in Feast and implements schema validation across all compute engines:

Commit 1 (970650b): Fix Map/Dict support and implement schema validation

Fixed Map/Dict type support across offline stores, online stores, and type mappings
Added enable_validation parameter to FeatureView, BatchFeatureView, and StreamFeatureView
Implemented schema validation nodes in Local, Ray, and Spark compute engines (missing columns, type mismatch warnings)
Added Spark and Milvus map type mappings
Updated documentation for Map type backend mappings

Commit 2 (ac28d4d): Add JSON and Struct complex data types

Added JSON, JSON_LIST, STRUCT, STRUCT_LIST to the protobuf ValueType enum and Value message
Introduced Json primitive type and Struct class (schema-aware structured type with named, typed fields)
Implemented full type mapping pipeline: Feast ↔ Proto ↔ PyArrow ↔ backend-native types (PostgreSQL, BigQuery, Snowflake, Redshift, Spark, MSSQL, Athena, DynamoDB, Milvus)
Added JSON well-formedness validation at both proto conversion level (always active) and validation node level (when enable_validation=True)
Implemented Spark-native type validation using from_feast_to_spark_type() and _spark_types_compatible() — eliminates the PyArrow-to-Spark type mismatch gap
Persisted Struct field schemas through the registry via Field tags (feast:struct_schema)
Added Go SDK type conversion support for JSON and Struct types
Added automatic serialization/deserialization for JSON type (Python dict/list ↔ JSON string)
Single _validate_schema() method per engine (missing columns + type checks + JSON content validation)

Commit 3 (a131b73): Modified default template with different types

Added Json, Map and Struct in driver template

Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>

franciscojavierarceo

lgtm

huge pr tho

…tion (feast-dev#5974) * feat: Fix Map/Dict support and implement schema validation Signed-off-by: ntkathole <nikhilkathole2683@gmail.com> * feat: Modified default example with different data types Signed-off-by: ntkathole <nikhilkathole2683@gmail.com> --------- Signed-off-by: ntkathole <nikhilkathole2683@gmail.com> Co-authored-by: Francisco Javier Arceo <arceofrancisco@gmail.com> Signed-off-by: Aniket Paluskar <apaluska@redhat.com>

ntkathole self-assigned this Feb 16, 2026

ntkathole added the ok-to-test label Feb 16, 2026

ntkathole force-pushed the fix_complex_type branch from 05522ce to 970650b Compare February 16, 2026 12:59

ntkathole changed the title ~~feat: Fix Map/Dict support and implement schema validation~~ feat: Add complex type support (Map, JSON, Struct) with schema validation Feb 16, 2026

ntkathole marked this pull request as ready for review February 16, 2026 15:49

ntkathole requested a review from a team as a code owner February 16, 2026 15:49