◐ Shell
clean mode source ↗

feat: Add support for extra options for stream source by Prathap-P · Pull Request #5618 · feast-dev/feast

Prathap-P

What this PR does / why we need it:

This pr exposes runtime options that users can give before ingesting stream data.

Which issue(s) this PR fixes:

  • Our team have certificate way of authn/authz set for kafka topic and currently feast doesn't accept those options while registering a kafka source.
  • For ex: "kafka.security.protocol", "kafka.ssl.truststore.location", "kafka.ssl.truststore.password", "kafka.ssl.keystore.location", "kafka.ssl.keystore.password", "kafka.ssl.key.password"
  • So I thought we can expose an optional stream source options that users can provide while ingesting, so that they dont have to register again multiple times when options change.

Misc

I think this addition will be essential as spark supports numerous options which are used in real world scenarios.