◐ Shell
clean mode source ↗

Control mini-batch size when writing rows to online store from bytewax job

Is your feature request related to a problem? Please describe.
Bytewax Materialization job tries to write all rows at a time from a single pyarrow parquet dataset

Describe the solution you'd like
Should break into mini-batches before writing to online store, and this number should be configurable

Describe alternatives you've considered
No

Additional context
image