fix: Update the pyarrow to latest v14.0.1 regarding the CVE-2023-47248.#3835
fix: Update the pyarrow to latest v14.0.1 regarding the CVE-2023-47248.#3835shuchu wants to merge 14 commits into
Conversation
Signed-off-by: Shuchu Han <shuchu.han@gmail.com>
|
A little bit worried about the unit test coverage. please be aware that I unpin the pyarrow version. py3.8-requirements.txt and py3.8-ci-requirements.txt were updated manually. (regarding the DASK version issue for python 3.8) |
Sorry, something went wrong.
Signed-off-by: Shuchu Han <shuchu.han@gmail.com>
…le write to temporary parquet file. Signed-off-by: Shuchu Han <shuchu.han@gmail.com>
…ng pyarrow v10.0.1 Signed-off-by: Shuchu Han <shuchu.han@gmail.com>
|
I meet a very interesting problem. I only update the Pyarrow version and snowflake api, the integration test results show me that the timestamp range is error while running Redshift SQL query. it happens while run "get_historical_features()" and the timestamp range were inferenced from the "entity_df": |
Sorry, something went wrong.
Signed-off-by: Shuchu Han <shuchu.han@gmail.com>
Signed-off-by: Shuchu Han <shuchu.han@gmail.com>
Signed-off-by: Shuchu Han <shuchu.han@gmail.com>
Signed-off-by: Shuchu Han <shuchu.han@gmail.com>
|
please do not merge this PR. @sudohainguyen |
Sorry, something went wrong.
Signed-off-by: Shuchu Han <shuchu.han@gmail.com>
|
No worries, looking forward to seeing this works |
Sorry, something went wrong.
Signed-off-by: Shuchu Han <shuchu.han@gmail.com>
Signed-off-by: Shuchu Han <shuchu.han@gmail.com>
Signed-off-by: Shuchu Han <shuchu.han@gmail.com>
Signed-off-by: Shuchu Han <shuchu.han@gmail.com>
Signed-off-by: Shuchu Han <shuchu.han@gmail.com>
|
Finally, I found the fix way. It's about the setting of parameter "coerce_timestamps" of "pyarrow.parquet.write_table". Let me close this PR and create a clean new one. |
Sorry, something went wrong.
What this PR does / why we need it:
Update the pyarrow to latest version v14.0.1 which has the fix for CVE-2023-47248
Fixes #3832