siuba

What's Changed

• Misc fixes by @machow in https://github.com/machow/siuba/pull/476

Full Changelog: https://github.com/machow/siuba/compare/v0.4.3...v0.4.4

What's Changed

• fix(sql): sqlalchemy 2.0 compat imports by @machow in https://github.com/machow/siuba/pull/472
  • Note that pandas v2 removed a few functions (such as .dt.week). The translations for these are still supported.
• ci: do not upload to airtable by @machow in https://github.com/machow/siuba/pull/477

Full Changelog: https://github.com/machow/siuba/compare/v0.4.2...v0.4.3

What's Changed

• fix: summarize raising error when a grouping col is all NA (or mostly NA) (https://github.com/machow/siuba/pull/459; issue #458, #251)

Full Changelog: https://github.com/machow/siuba/compare/v0.4.1...v0.4.2

What's Changed

• fix: across correctly handles grouped data in sql (https://github.com/machow/siuba/pull/455)
• fix: across handles pandas mutate with aggregation (https://github.com/machow/siuba/pull/455)
• ci: remove doc publishing (https://github.com/machow/siuba/pull/456)

Full Changelog: https://github.com/machow/siuba/compare/v0.4.0...v0.4.1

See https://github.com/machow/siuba/releases/tag/v0.4.0rc1 for details

This PR mainly implements the functions tbl(), across(), pivot_longer(), pivot_wider(). It ensures verbs work with grouped data. As part of implementing across(), it refactored tidyselection and SQL support.

What's Changed

• feat(tbl): add tbl() function for pandas and sqla (https://github.com/machow/siuba/pull/446)
• feat pivot_longer() and pivot_wider() (https://github.com/machow/siuba/pull/440)
• feat across() by (https://github.com/machow/siuba/pull/452)
• feat: count() and add_count() support the name argument. (#452)
• feat: a new symbolic "formula" object, Fx, now exposed as a top level import. This is used to represent operations in across() and the pivot functions. (#452)
• feat: implement grouped distinct for both pandas and sql. (#452)
• feat(tidyselect)!: a lambda can no longer be used to create a tidyselection specifier. Instead we match dplyr's behavior (#452):
  • old: lambda _: _.startswith("abc") was equivalent to _.startswith("abc")
  • Now, when select() is given a callable function, it passes each column of data to it, and expects back a boolean.
  • new: select(cars, lambda ser: ser.dtype == "int")
• feat: forcats functions: fct_inseq(), fct_infreq() (https://github.com/machow/siuba/pull/435)
• feat: show_query simplify option uses "*" when possible (https://github.com/machow/siuba/pull/344)
• feat(siu): new pipe() and call() functions (https://github.com/machow/siuba/pull/413)
  • pipe() allows you to manually create a pipe
  • call() call you to represent a call to an external function (e.g. call(pd.to_datetime, _.some_col))
• feat: Add new datasets (https://github.com/machow/siuba/pull/434)
• feat(sql): add LazyTbl.last_select to simplify queries (https://github.com/machow/siuba/pull/449)

.

• fix(pandas)!: gather default selection is all columns (https://github.com/machow/siuba/pull/451)
• fix!: if_else and case_when now return Series (https://github.com/machow/siuba/pull/432)
• fix(sql)!: calling arrange() twice now resets the order_by variables set by the first call (matching dbplyr behavior) (#452).
• fix: add_count() now correctly handles named arguments, is tested for most cases count() is tested on (#452).
• fix: support negative index, multislice in tidyselect (https://github.com/machow/siuba/pull/430)
• fix(pandas): ensure verbs accept DataFrameGroupBy (https://github.com/machow/siuba/pull/445)
• fix(experimental): fix tab completions (https://github.com/machow/siuba/pull/444)

.

• tests: speed snowflake tests by removing reflection (https://github.com/machow/siuba/pull/428)
• tests(sql): update failing test, due to duckdb change (https://github.com/machow/siuba/pull/441)

.

• docs: using an OR condition in filter (https://github.com/machow/siuba/pull/427)
• docs: complete docstrings for all verbs (https://github.com/machow/siuba/pull/439)
• chore: Fix typos by @kianmeng in https://github.com/machow/siuba/pull/433

New Contributors

• @kianmeng made their first contribution in https://github.com/machow/siuba/pull/433

Full Changelog: https://github.com/machow/siuba/compare/v0.3.0...v0.4.0rc1

What's Changed

• feat: add duckdb as a SQL backend by @machow in https://github.com/machow/siuba/pull/422
  • breaking: LazyTbl still takes a sqlalchemy Engine, but can no longer take a sqlalchemy Connection.
• feat: allow head() over grouped data by @machow in https://github.com/machow/siuba/pull/420
• docs: update import for plotnine example by @isabelizimm in https://github.com/machow/siuba/pull/421
• ci: add wheel build to release by @machow in https://github.com/machow/siuba/pull/423
• refactor: prepare for sql win_over by cleaning up sql Over clause code @machow in https://github.com/machow/siuba/pull/412

New Contributors

• @isabelizimm made their first contribution in https://github.com/machow/siuba/pull/421

Full Changelog: https://github.com/machow/siuba/compare/v0.2.3...v0.3.0

Also adds more tests of calling numpy functions over symbolics

What's Changed

• chore: deprecation warning in metahook (https://github.com/machow/siuba/pull/414)
• feat: symbolic support numpy ufuncs (https://github.com/machow/siuba/pull/415)

Full Changelog: https://github.com/machow/siuba/compare/v0.2.1...v0.2.2

Fixes

• fix: mock_sqlalchemy_engine across dialects, collecting (https://github.com/machow/siuba/pull/410)

Full Changelog: https://github.com/machow/siuba/compare/v0.2.0...v0.2.1

What's Changed

• Refactor siu submodule (https://github.com/machow/siuba/pull/338)
  • made siu.py a folder, split into different submodules for visitors, dispatchers, calls, etc..
• feat(ci): publish main branch docs to netlify (https://github.com/machow/siuba/pull/395)
• docs(siu): flesh out docs for calls, symbolics, dispatchers (https://github.com/machow/siuba/pull/397)
• ci: Pin setup-gcloud to v0 instead of master (https://github.com/machow/siuba/pull/405)
• feat: use codata dispatch in SQL translation (https://github.com/machow/siuba/pull/398)
  • all sql column operation concrete signatures changed, to allow operations in operations
  • old:

  @mean.register
  def _mean(col: SqlColumn): ...
  

  • new:

  @mean.register
  def _mean(codata: SqlColumn, col): ...
  

• feat: support sqlite window functions (https://github.com/machow/siuba/pull/406)
• ci: bigquery tests run in parallel. Reduces time from 15 mins to about 2 mins.

This is a dev release to support...

• enable creating a LazyTbl from a subquery (for dbcooper-py)
• early preview of snowflake dialect

Fixes

• Error in pandas semi_join errored when no join cols were specified (#351; PR #374)

Misc

• Add an extra test for fast_mutate (#355 #372 )

⚠️ Note: this is a re-release of alpha v1.0.0a3 to be v0.1.1. This will allow users to easily install these releases using pip. Changelog copied below.

Fixes

• fix(ci): allow testing multiple bigquery test branches at the same time (#360)
• fix(pandas): fast methods support DataFrame aggs like n() (#363)
• fix(pandas): support pandas 1.3 (#366)
• fix(install): fix windows install breaking due to utf-8 in README (#370)

⚠️ Note: this is a re-release of alpha v1.0.0a2 to be v0.1.0. This will allow users to easily install these releases using pip. Changelog copied below.

This is an alpha release for v1.0.0. There will likely be extensive changes through January as I work to refactor the core API.

Big changes

• Added ops submodule (and removed spec submodule).
  • This contains generic functions for pandas methods.
  • Has method data needed to translate pandas expressions to SQL (e.g. whether something is a property, or uses an accessor).
• Refactored SQL translation mechanism:
  • Base dialect that others extend.
  • Over clauses in translate.py now have a func class method. This generates a constructor for a specific sql translation.
  • Now uses a pandas translator from ops submodule.

Features:

• Support bigquery as a SQL dialect (#322)
• Support MySQL as a SQL dialect. (https://github.com/machow/siuba/commit/a9b97a9009650ddd483dd2dde9c2d838a840f9e6)
• ci: add test deployment to action (#354)

Fixes

• fix(ci): allow testing multiple bigquery test branches at the same time (#360)
• fix(pandas): fast methods support DataFrame aggs like n() (#363)
• fix(pandas): support pandas 1.3 (#366)
• fix(install): fix windows install breaking due to utf-8 in README (#370)

This is an alpha release for v1.0.0. There will likely be extensive changes through January as I work to refactor the core API.

Big changes

• Added ops submodule (and removed spec submodule).
  • This contains generic functions for pandas methods.
  • Has method data needed to translate pandas expressions to SQL (e.g. whether something is a property, or uses an accessor).
• Refactored SQL translation mechanism:
  • Base dialect that others extend.
  • Over clauses in translate.py now have a func class method. This generates a constructor for a specific sql translation.
  • Now uses a pandas translator from ops submodule.

Features:

• Support bigquery as a SQL dialect (#322)
• Support MySQL as a SQL dialect. (https://github.com/machow/siuba/commit/a9b97a9009650ddd483dd2dde9c2d838a840f9e6)
• ci: add test deployment to action (#354)

This release adds support for SQLAlchemy 1.4, does some light refactoring, and fixes a slicing issue.

Fixes

• Make compatible with SQLAlchemy 1.4. #327
• Properly represent symbolics when indexing with multiple slices (e.g. _[1:2, _.a:_.b]). #325

Features

• Refactored fast group operations to use generic functions (regroup(), broadcast_agg(), is_compatible()). #310
• Enable piping to attributes (e.g. pipe(_.some_col)). #325

QA

• Migrated to github actions. #299

Much of this release is setting up for...

• fast methods to become drop-in replacements for their slow, reference implementation counterparts!
• SQL fixes and features for funneljoin-py

Fixes

• Pandas anti_join breaking when on argument is a string (#264)
• Sql now resets order by after joins (#277)
• Fixed major regression for sql where filter with multiple arguments did strange and terrible things (#277)
• Sql summarize correctly makes a subquery when group_by references a recently created column (#278)
• Sql right_join had to switch lhs and rhs, but was not correctly handling all implications of that. Specifically how key columns are kept, and suffixes used. (#279)

Features

• Fast grouped methods determine whether they can run quickly, fallback if not, along with a warning. Lambdas can be used to explicitly take the slower, reference implementation route (#268).
• Fast grouped methods allow property operations that return dim 0 results, like dtype. (#269)
• - in sql arrange now produces the correct DESC, rather than being a - operation (#280)
• pandas arrange now works on grouped data. i.e. DataFrameGroupBy. (#280)

QA

• a slew of tests!

Fixes

• anti_join in pandas no longer breaks when on argument is a string. E.g. anti_join(df1, df2, on = "some_col") #264
• fct_collapse now correctly keeps missing values (#262)