◐ Shell
clean mode source ↗

feat: Add batch commit mode for MySQL OnlineStore by chimeyrock999 · Pull Request #5699 · feast-dev/feast

What this PR does / why we need it:

Adds optional batch insert and commit mode to the MySQLOnlineStore to improve materialization performance for TiDB and MySQL backends.
By default, Feast writes each row individually and commits per record, which causes large overhead on MySQL compatible Raft-based databases like TiDB.
This PR introduces a configurable batch mode to group inserts and commit per batch, improving write throughput.

Changes

  • Added config flags:
    batch_mode: true
    batch_size: 1000
    
  • Implemented batched insert using executemany() / multi-row SQL.
  • Each batch is committed atomically.
  • Backward compatible (default = per-row mode).

Misc

  • Verified on TiDB v7.5.0 and MySQL 8.0.44.
  • Thanks @ntkathole confirming this direction.