fix: FeastExtrasDependencyImportError when using SparkOfflineStore without S3 by aloysius-lim · Pull Request #4594 · feast-dev/feast

franciscojavierarceo

What this PR does / why we need it:

When using SparkOfflineStore without S3, the following error is raised:

❯ python -c "from feast.infra.offline_stores.contrib.spark_offline_store.spark import SparkOfflineStore"

Traceback (most recent call last):
  File "[REDACTED]/python3.10/site-packages/feast/infra/utils/aws_utils.py", line 28, in <module>
    import boto3
ModuleNotFoundError: No module named 'boto3'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "[REDACTED]/python3.10/site-packages/feast/infra/offline_stores/contrib/spark_offline_store/spark.py", line 34, in <module>
    from feast.infra.utils import aws_utils
  File "[REDACTED]/python3.10/site-packages/feast/infra/utils/aws_utils.py", line 34, in <module>
    raise FeastExtrasDependencyImportError("aws", str(e))
feast.errors.FeastExtrasDependencyImportError: No module named 'boto3'
You may need run pip install 'feast[aws]'

Installing the AWS libraries should not be required when not using AWS, to avoid unnecessary bloat. Since Spark is commonly used in many different environments (e.g. GCS, Azure or on premise), this change allows SparkOfflineStore to be used without installing the AWS libraries.