Current Behavior
The feature server's /get-online-features endpoint uses google.protobuf.json_format.MessageToDict to convert protobuf responses to JSON. MessageToDict is a generic converter designed for any protobuf type, but for the known GetOnlineFeaturesResponse structure, a type-specific converter is 4x faster.
Steps to reproduce
- Deploy Feast feature server to Kubernetes
- Call /get-online-features with 100 entities and 50 features
- Profile the endpoint and isolate MessageToDict time
- Observe ~12ms spent in MessageToDict (98% of serialization time)
- Compare with proto binary serialization which takes only 0.1ms
- Implement custom dict builder and observe 2.9ms (4x improvement)
Specifications
- Feast version: 0.60.0
- Python version: 3.11
- Tested on: OpenShift/Kubernetes with Redis, PostgreSQL, DynamoDB
Benchmark Results
Tested with 100 entities, 50 features in-cluster:
Current (MessageToDict + orjson): 12.0ms
- MessageToDict only: 11.8ms (98% of time!)
- orjson (dict→JSON): 0.2ms
Custom dict builder + orjson: 2.9ms
- Custom builder: 2.7ms
- orjson (dict→JSON): 0.2ms
Proto binary (for comparison): 0.1ms
Why MessageToDict is Slow
- Reflection-based field discovery for ANY protobuf message
- Dynamic type checking for each field
- Generic handling of all value types (oneofs, repeated, maps)
- CamelCase/snake_case conversion logic
- For GetOnlineFeaturesResponse, we know the exact schema, so all this reflection is unnecessary.
Possible Solution
Add sdk/python/feast/feature_server_utils.py with a custom dict builder
| Metric |
Before |
After |
Improvement |
| Serialization (100 entities) |
12.0ms |
2.9ms |
4.1x faster |
| Serialization (500 entities) |
~55ms |
~13ms |
4.2x faster |