PyAthena

What's Changed

• Bump urllib3 from 1.26.17 to 1.26.18 by @dependabot in https://github.com/laughingman7743/PyAthena/pull/481
• Support json_serializer and json_deserializer engine/dialect paremeters by @edgarrmondragon in https://github.com/laughingman7743/PyAthena/pull/486
• Update GitHub Actions & Remove secrets by @laughingman7743 in https://github.com/laughingman7743/PyAthena/pull/487
• Update poetry.lock by @laughingman7743 in https://github.com/laughingman7743/PyAthena/pull/489

New Contributors

• @edgarrmondragon made their first contribution in https://github.com/laughingman7743/PyAthena/pull/486

Full Changelog: https://github.com/laughingman7743/PyAthena/compare/v3.0.9...v3.0.10

• Update s3fs dependencies #342
  Thanks @nikhil27472 #340

• Allow the boto3 session object to pass as an argument to the connection method.
  Thanks @benkehoe. (#248)

• Support SQLAlchemy 1.4

• Change schema name (database name) to optional.
• Support for catalog.
  Thanks @milanaleksic. (#220)

• Support duration_seconds, poll_interval and kill_on_interrupt options in SQLAlchemy connection string.
  Thanks @acountrec. (#214)
• Support passing the execute method keyword arguments to the pandas.read_csv method arguments.
  Thanks @bjcttam, @austinlostinboston. (#169, #210)

• Support duration_seconds, poll_interval and kill_on_interrupt options in SQLAlchemy connection string.
  Thanks @acountrec. (#214)
• Support passing the execute method keyword arguments to the pandas.read_csv method arguments.
  Thanks @bjcttam, @austinlostinboston. (#169, #210)

• Change the default value of na_values in PandasCursor from None to empty character.
  If you want to change the behavior to that of the previous version, explicitly set na_values to None in the argument of the execute method.

  import pyathena
  from pyathena.pandas.cursor import PandasCursor
  cursor = pyathena.connect(s3_staging_dir="s3://YOUR_S3_BUCKET/path/to/",
                            region_name="us-west-2",
                            cursor_class=PandasCursor).cursor()
  df = cursor.execute("SELECT * FROM many_rows",
                      na_values=None).as_pandas()
  

  Thanks @EdwardJRoss. (#204)

• Change the default value of na_values in PandasCursor from None to empty character.
  If you want to change the behavior to that of the previous version, explicitly set na_values to None in the argument of the execute method.

  import pyathena
  from pyathena.pandas_cursor import PandasCursor
  cursor = pyathena.connect(s3_staging_dir="s3://YOUR_S3_BUCKET/path/to/",
                            region_name="us-west-2",
                            cursor_class=PandasCursor).cursor()
  df = cursor.execute("SELECT * FROM many_rows",
                      na_values=None).as_pandas()
  

  Thanks @EdwardJRoss. (#204)

• Support verify option in SQLAlchemy connection string.
  Thanks @tamersalama. (#188)

• Support Multi-factor authentication.
  https://github.com/laughingman7743/PyAthena#multi-factor-authentication
  https://github.com/laughingman7743/PyAthena#assume-role-provider-with-mfa
• Implement cache expiration based on time.
  Thanks @rubenssoto. (#187)

• Fix poll_interval type from int to float.
  Thanks @naikordian. (#198)

• Fix extra package names to lowercase.
  Thanks @Akeboshiwind. (#189)
• Support verify option in SQLAlchemy connection string.
  Thanks @tamersalama. (#188)
• Implement DictCursor & AsyncDictCursor.
  Thanks @jerome-white. (#194)

• Drop support for Python2.7 and Python3.5.
• Add type hints.
• Move the classes related to Pandas to the package pyathena.pandas.
• Remove the execution_time_in_millis field from the cursor and result set object. (Same as the engine_execution_time_in_millis field)

• Fix for empty string decimal value in PandasCursor.
    Thanks @hoffrocket. (laughingman7743/PyAthena/pull/157)
• Changed to escape % characters, even if there are no parameters to bind in SQLAlchemy.
    Thanks @diego-serrano-glean. (laughingman7743/PyAthena/issues/160)

• Add keep_default_na, na_values and quoting arguments to the PandasCursor execute method.

• Support for canceling query execution on KeyboardInterrupt.

• Fix types mismatch in the pandas.DataFrame.to_sql method.
  Thanks @RiccardoDiGuida, @thibault-ketterer. (laughingman7743/PyAthena/issues/145, laughingman7743/PyAthena/issues/146)

• Fix an overflow error in PandaCursor.
  Thanks @juviasuisei @KneeShard. (laughingman7743/PyAthena/issues/141)

• Add the following query execution statistics.
  Thanks @navi86. (laughingman7743/PyAthena/pull/140)
  • query_queue_time_in_millis
  • total_execution_time_in_millis
  • query_planning_time_in_millis
  • service_processing_time_in_millis
  • data_manifest_location