airbyte

5e4178e remove legacy migration code (#8499)
804fe6b Bump s3 version to remove excessive logging (#8501)
8ebd3bd 🎉 advanced auth impl (#8345)
7b81e11 Publish new GCS region (#8392)
2c34c70 🎉 Destination GCS: add new region (#8386)

Note: This release contains a bug affecting GCS Logs on Kubernetes. Please update to 0.33.2-alpha and later.

bc230ee Format and generate seed resources. (#8493)
341f505 Rename env vars for better readability. (#8447)
5393070 🎉 Refactor Pokeapi Source to CDK (#8432)
d3ac897 🎉 Snowflake Destination internal staging support (#8253)
776ca69 🎉 Source Python HTTP Tutorial: Migrate to CDK, remove deprecated dependencies (#8433)
9a26f8a Remove deprecated dependencies for connectors in build.gradle (#8430)
c438e01 Revert "🐛 📝 DAT: verify that a destination is able to write any ISO8601-compliant date string (#5814)" (#8484)
9270f32 🐛 Source Google Analytics: Fix issue with custom error and simplify configuration (#8175)
a11e31a 🎉 Destination BigQuery and BigQuery denormalized: Use byte-based buffering batching (#8199)
b60a2d0 Fix OAuth Injection (#8482)
cfa6815 Get default configured catalog from discover command (#8232)
4e44166 Salesforce make start_date optional and change format to YYYY-MM-DD (#8191)
298c8b2 Allow dev-tagged connectors to always be updateable (#8219)
64bd0a6 🐛 Source Intercom: backoff for companies' scrolling (#8395)
5ce8339 🎉 Source Facebook pages: updated all empty objects in Page and Post streams (#8267)
0fcc2dc fix: formatting issues (#8435)
d12f919 adding-prefect-in-roadmap (#8467)
d7ef91f Source Amazon Seller Partner: Add GET_SELLER_FEEDBACK_DATA report (#8021)
1eba280 upgrade source-harvest definition/specs (#8464)
31d9bca enviroment -> environment (#8436)
cdfe294 Fix casing of Advanced Auth for source-harvest (#8428)
9f685b9 🐛 Source LinkedIn Ads: hands 429 response status code (#8382)
8726d39 #7727 source postgres: add several types to integration tests (#7913)
9a54c4d 🐛 Source Zenloop: re-add missing seed files (#8299)

Breaking Changes

The following configuration environment variables were renamed to better reflect usage,

• GCP_STORAGE_BUCKET -> GCS_LOG_BUCKET
• MAX_SYNC_JOB_ATTEMPTS -> SYNC_JOB_MAX_ATTEMPTS.
• MAX_SYNC_TIMEOUT_DAYS -> SYNC_JOB_MAX_TIMEOUT_DAYS.
• WORKER_POD_TOLERATIONS -> JOB_POD_TOLERATIONS.
• WORKER_POD_NODE_SELECTORS -> JOB_POD_NODE_SELECTORS.
• JOB_IMAGE_PULL_POLICY to JOB_POD_MAIN_CONTAINER_IMAGE_PULL_POLICY.
• JOBS_IMAGE_PULL_SECRET to JOB_POD_MAIN_CONTAINER_IMAGE_PULL_SECRET.
• JOB_SOCAT_IMAGE to JOB_POD_SOCAT_IMAGE.
• JOB_BUSYBOX_IMAGE to JOB_POD_BUSYBOX_IMAGE.
• JOB_CURL_IMAGE to JOB_POD_CURL_IMAGE.
• KUBE_NAMESPACE to JOB_POD_KUBE_NAMESPACE.
• RESOURCE_CPU_REQUEST to JOB_POD_MAIN_CONTAINER_CPU_REQUEST.
• RESOURCE_CPU_LIMIT to JOB_POD_MAIN_CONTAINER_CPU_LIMIT.
• RESOURCE_MEMORY_REQUEST to JOB_POD_MAIN_CONTAINER_MEMORY_REQUEST.
• RESOURCE_MEMORY_LIMIT -> JOB_POD_MAIN_CONTAINER_MEMORY_LIMIT.

Users configuring these variables should update their respective configuration files to the new variable names.

Note: This release contains a bug affecting GCS Logs on Kubernetes. Please update to 0.33.2-alpha and later.

67f7cf0 Bump GCS version with avro/parquet timestamp conversion (#8360)
1286c8c Better breaking changes instructions.
d5adbf4 Better Env Var Organisation. (#8390)
3afb39d Fix to pass two_sequential_reads test and skip test state_with_abnormally_large_values (#8264)
b424c1a 🐛 Fix incremental normalization with empty tables (#8394)

c4c92bd Fix normalization when un-nesting (#8378)
15fe4dd Source Trello: Pull data only for configured board_ids (#8338)
e4e95e3 🐛 📝 DAT: verify that a destination is able to write any ISO8601-compliant date string (#5814)
8775cbe Make spec field required on definitions + remove SpecFetcher usage (#8140)
912e5a7 Change synchronisationProgress to synchronizationProgress (#8354)
757650f 🐛 Source Hubspot: remove 'skip_dynamic_fields' config param (#8329)
97594c7 Added temporal build script (#8325)
f2f224f Update SAT for validate_data_points Test Basic Read option. (#8342)
b265b6c SAT: Implement assert that all fields in all streams contained at least one data point (#8272)
a55ef51 Source Shopify: fix for missing properties & items in schemas (#7783)
ff87cff 🎉 SAT: Add assert that output catalog does not have any $ref in it (#8294)
65d471a Output run time per stream (#8173)
ff0c09a s3-source add option to not infer datatypes (#7892)
065bbf6 🎉 Destination S3 Support writing timestamps (#7732)
c97fe68 Deprecate Legacy Migration System (#8121)
ada2e17 Refactor MigrationAcceptanceTest to test for major version bumps (#8154)
6787777 Publish PR 7973: shopify new stream add items (#8320)
24b55dc Update HowTo-ConnectionSpecification.md
74015ec fix dependabot problems (#8323)
a229ab9 Bump tmpl from 1.0.4 to 1.0.5 in /airbyte-webapp (#7654)
e92c78d Bump tar from 6.1.0 to 6.1.11 in /airbyte-webapp (#7655)
8f05f7f Publish PR 8168: new zendesk support stream TicketMetricEvents (#8316)
f781df8 🎉 Source Google Ads: clarify setup fields (#8178)

Changelog:

0082efd Bmoric/add slash command (#8150)
9428b40 🐛 Source Mixpanel: Fix schemas and UI issues (#8256)
a2b31cc Source Trello: enable specifying board ids in configuration (bump version) (#8259)
72b2f8b Source Trello: enable specifying board ids in configuration (#8183)
b295c42 Remove auto-assigning of issues to deprecated project (#8314)
9e087c6 🐛 Source MongoDB - Added Milliseconds to Cursor for mongo date type (#8306)

Changelog:

afbbd6e Source Linnworks: refactor stream StockLocations (#8169)
afaa963 CDK: Fix 'emitted_at' from 'seconds 1000' to correct milliseconds. (#8295)
b727bb2 Allow using branded oauth buttons (#8187)
852c453 DAT destination acceptance tests & custom dbt transformations (#8131)
aeb15ce 🐛 Source postgres: fix money type with amount > 1000 (#7995)
908be3f Source Iterable: Update iterable.md docs (#8273)
3549140 Add the needs-triage label to new issue templates (#8283)
67e857c Update builds.md
0d7e69e 🐛 Source Hubspot: unset 'hs_time_in_*' properties (#8216)
9d62959 🐛 Source Google Search Console: set default end_date param value (#7431)
6e47a8f 🐛 Source Postgres: views are not listed during schema discovery (#8266)
f08c447 UI: Fix for missing connectors logos in UI (#8251)
a4a5cb8 🐛 Source Stripe: Rearrange setup fields (#8250)
12d8757 Remove insights_export from Facebook Pages field (#8265)

ea4f10e Use snake_case for analytics variable naming (#8068)
15fd920 Publishing new source: Outreach (#8181)
4b59fe4 🎉 New Source: Outreach (#7507)
bcab466 Add integration test annotations (#8035)
fd90c95 🎉 New Source: Zenloop (#7380)
c790a7f 🎉 Source Drift: add OAuth 2.0 authentication option (#7337)
831304e 🎉 Source Zendesk Chat: make Chats stream incremental + add tests for all streams (#8166)
cf3be4a Iterable adjustable stream slices (#8091)
c0cb8f3 Fix Github check connection for organizations with large number of re… (#8170)
0eba52e 🎉 New destination: Kinesis (#7799)
1f7a42b added firestore credentials to CI and fix grpc installation (#8158)
519ea2a 🎉 New Destination: Google Firestore (#7231)
5b0b584 🎉 New Destination: MQTT (#7718)
141524d 🐛 Source Monday: fix pagination and schema bug (#8016)
8a957ca 🎉 New destination: Redis (#7653)

817ec6d handle major version bump in automigration test (#8152)
aac54da 🎉 Source Facebook: add the "videos" stream. (#7855)
bec3b3b Add Delighted source docv to SUMMARY.md (#8148)
005e4f2 remove todo to handle bug in old version of docker compose; not an issue in new versions (#8122)
d67afa7 hard code auto migration test version (#8146)
dcc72ba Source Jira: add options to stream (small fixes) (#8141)
15b8501 Source Jira: Add option to render fields in HTML format and fix sprint_issue ids (#7398)
d618ff9 fix where version bump check takes place (#8142)
f447183 Bmoric/add endpoint to give feedback (#8005)
778b0f2 🎉 Source Harvest: add OAuth 2.0 (#7952)
74c13b5 🎉 New Source: Airtable (#7639)
678b731 change cart naming to cart.com (#8115)
93c63a3 improve documentation around logging in kubernetes (#8072)
dc04654 🎉 Source PostgreSQL: Handle the case when DB user doesn't has privileges to select the table (#8010)
80e0a8c Removed the limitation in Freshdesk streams pagination: max_page = 500 (#8017)
c5a7267 🐛🐌 Optimize incremental normalization runtime with snowflake (#8088)
3226b22 🎉 Source Snapchat Marketing: support oauth (#7811)
530773f 🎉 New Source: Youtube analytics (#7407)
ad3b7be Document how to order spec fields in the UI
7b2429d 🎉 New Source: Pardot (#7091)
3f00a3e improve javadocs in replication worker (#7942)

Changelog:

a1b8bd2 allow hubspot & intercom in cloud (#8032)
dcd3b93 fix npe (#8101)

6fb79cc fix issue with oneOf with sub schemas with same key but different type (#8034)
34565c6 Add better command for generating seed specs to readme
fda932d fix source specs again (#8103)
49adae0 Update roadmap for November. (#8074)
2aea2e0 Fix a bunch of broken links. (#8083)
9eb78a4 Akorotkov/6544 mongo performance issues (#8098)
82c4c82 🎉 Linkedin Ads: support of oAuth2 (#7839)
d8deca9 Add repo scope for github oAuth authorization. (#8092)
006c34c 🐛 Source Zendesk Support: Fix wrong types for schemas, add Transformer (#8050)
b76d155 🎉 Survey Monkey: Improve 'check' using '/users/me' API call (#7868)
46e1f65 🐛 Source MongoDB - Added milliseconds to convert timestamp to datetime format (#8046)
7b54480 Add how to copy files from Kubernetes pods. (#7238)
8e0e0b4 🐛 Updated source-mongodb-v2 performance (#7982)

e15c2a9 Fix UX handbook assets (#8020)
d6c684b Enhance API for use by cloud to provide per-connector billing info (#7893)
8934f3e add document for sources (#8000)
fcba526 🎉 New destination: ScyllaDB (#7468)
90a5dfb Add section about user impact to PR template (#8003)
c99335f Include requester and stats fields in tickets stream (#7486)
51c03da prevent gitbook from running all of our builds twice (#7993)
a3a87ed Fix package lock (#7991)
655df2c Fix scaffold format files (#7989)
011ec29 move doc link checker to cron / workflow dispatch (#7992)
ea738f1 🎉 New Source: Linnworks (#7588)
c8e206d Silence javadoc warnings (#7891)
35e6795 fix: provide worspaceId to AnalyticsInitializer (#7642)
495202e Pipedrive oAuth support (#7968)
86ca36c Format code (#7978)
d9135b0 🎉 Source Salesforce: Add Transform for output records (#7885)
fa040da Implement protocol change for OAuth outputs (#7917)
cdb476e Add auth config to oauth api (#7798)
3a0eaaf 🎉 Source Mysql - added basic MySql performance tests (#7820)
63c6249 Revert "🎉 Extend logic for JDBC connectors to provide additional properties in JSON schema (#7859)" (#7969)
f6d81ff Fix API docs for Boards stream (#7950)
7c28f25 🚀 Publish new source: Azure table storage (#7953)
f4e3687 Update on-oci-vm.md (#7890)
5a24ef5 🐛 Source Zendesk Talk: migrate to CDK (#7173)
2fe927b 🎉 Extend logic for JDBC connectors to provide additional properties in JSON schema (#7859)
6166164 Add to the new connection flow form with an existing source and dest… (#7831)
f40e536 restore airbyte-cli (#7935)
fbf08b6 Fix server errors (#7931)

2744c823f5 Bump Airbyte version from 0.31.0-alpha to 0.32.0-alpha (#7928)
558bc272a4 Source Facebook Marketing: implement retry logic for async reports (#7904)
0ba35fa1a2 bump AirbyteTestContainer timeout to accommodate migration times (#7927)
1c8af6226f for docker compose tests: add service name, log level (#7896)
a65b493f5a Add ability to configure job container images (#7869)
4c5a5d88a0 move force env var to bottom of .env to avoid conflicting with bump script (#7925)
9b31f7caa6 remove airbyte cli from build (#7924)
2e743d999c Added new slash command for triggering performance tests (#7920)
7fc21620f6 update doc to point to version 0.32.0 instead of 0.31.0
c9642a4882 🐛 Fix Migration Bug in v0.31.0-alpha (#7923)
6a2672985c Remove source and destination verification from create connection (#7756)
c19de44489 7513 oauth backend quickbooks (#7815)
2215523a81 add webapp docker build / tagging to README (#7919)
5846565f5a use tag in url instead of commit hash (#7921)
5566bdec82 update upgrade docs with instructions for faux major version bump (#7884)
26e44d1561 fix lodash (#7918)
e56e86dd28 Fix schema transformation false warnings. (#7863)
0c41542429 oAuth input parameters UI (#7325)
b295868b23 🐛 Source CockroachDB: Fix cockroach Datatypes (#7819)
d71871edc6 Bmoric/fix missing doc (#7882)

Changelog:

cf5d236 final file migration; version 0.31.0. good riddance (#7894)
4af4180 add specs (#7889)
fcb2ff4 Resolve linting errors in the javadoc contents (#7612)
4e4f58a do not format specs (#7886)
5b43c00 add attention banner to upgrade errors
9d05b1c Faux Major Version Bump (#7876)
b05c068 Fix airbyte-cli build on Mac M1 (#7878)
421908c Check in latest db2 source version
6d30ba4 🐛 Source IBM Db2: Connector provides wrong values for a datatype (#7670)
c511abd Fix master build (#7873)
e4120e7 Enforce building javadoc for platform only (#7647)
5f03d32 fix buffered stream consumer tests (#7834)
3194927 CDK: fix logging configuration inside source and streams (#7808)
70ed18b 🐛 Fix handling binary data type for Mssql source (#7748)
bea9bb5 Github: resolve refs for some schemas (#7856)

70f8521 Add logging to make clear what is happening when server starts. (#7858)
def1667 🎉 New Source DB2 secure-only (#7792)
760f429 Source OneSignal: correct get_update_state and unit test (#7617)
d3ba217 Update README.md (#7833)
73a9131 Revert "Custom auto-setup temporal docker image (#7681)" (#7835)
e20d98f Custom auto-setup temporal docker image (#7681)
d988399 Add doc on how to handle versioning with docker (#7824)
412426c Do not ignore tools/bin (#7801)
dac49d1 Source Cart: add stream addresses (#7767)
7f6d12a Davinchia/tracking client refactor (#7595)
504580d Remove base-python gradle dependencies in connectors where base-python is not used (#7499)
e8b8530 Add Snapchat Marketing oAuth backend (#7813)
6f70b6b Iterable: split email send stream into slices. (#7780)
c0cc288 🎉 CI secrets: migration to GSM (#7528)
15c56c3 Add note for deprecated definitions (#7805)
cf126ce 🐛 Destination BQ Denormalized: handle null values in fields described by a $ref schema (#7804)

Changelog:

62992bf Fix BufferedStreamConsumer tests (#7773)
c3c3e31 🎉 intercom added oauth support (#7060)
aa11238 Reduce CLI docker image size (#7695)
1b38fc7 bugfix 'Hubspot' -> 'HubSpot' (#7683)
fdda80c 🎉 Source DB2: added ssl support (#7355)
25c89a5 Source Facebook Marketing: Fix fail when async job takes too long (#7744)
6b0e9da 📚 Fix typo in the cdk documentation (#7784)
21d6dd9 Aesthetic clean-up / format OAuth code (#7782)
3d4f730 🐛 Source Salesforce: Fix types for anyType fields (#7778)
48d8250 🐛 Destination BigQuery-Denormalized: Fix JSON with $ref Definition keys (#7736)
2d2965b 🎉 New Source: Pinterest (#7493)
84a5328 Jamakase/refactor initial load (#7713)
16b626c Resolve discover catalog ref fields (#7734)

Changelog:

363a047 Update connector versions from #7719 (#7771)
efb5151 🐛 Make all JDBC destinations (SF, RS, PG, MySQL, MSSQL, Oracle) handle wide rows by using byte-based record buffering (#7719)
46fa44a 🎉 New Source: Commerce Tools (#5957)
658d984 Source Amazon Seller Partner: change check_connection step (#7752)
5d4c6c3 implement query to fetch config with metadata (#7632)

42b378f Revert "Revert "Bmoric/remove docker compose for build (#7500)" (#7698)" (#7746)
6f12778 Fix package lock (#7758)
c40a6f2 regenerate source specs
cc038cd 4280/connectors guides (#7312)
ce3426c upgrade to node 16 (#7724)
ef1278a Document how gitbook is kept in sync
3b77d03 Rename sync-branches to sync-branches.yml
eae84bc Attempt to autosync master with docs branch (#7753)
291dccb split sync worlflow (#7690)
0a43fb7 Source Stripe: remove \n from field tax_id (#7729)
88e2d2c 🎉 Source Tempo: migrate to the CDK
3cbfb99 🎉 Source Google Workspace Admin Reports: migrate to the CDK
8a19833 Regen SQL (#7743)
affea7f 🐛 Minor fixes to incremental normalization and nesting (#7669)
1e4bde8 Slack OAuthFlow Backend implementation (#7328)
c918f52 Fix Hubspot and Asana oAuth flow. (#7730)
f0c54a7 🎉 Source Greenhouse: Implement demographics streams support (#7607)
af903f2 🎉 Source Recharge: improve 'backoff' for HTTP requests (#7626)
154ecce improve link checker (#6797)
4a5751a m1 build fixes (#7720)
0b9b8ba 🎉 New Source: Monday (#7168)
1b08e30 🎉 Source Stripe: add checkout_sessions stream, checkout_sessions_line_item stream and promotion_codes stream (#7345)

0e3fd83 restore spec backfill (#7702)

remove spec backfill

76cb844 format (#7699)
797d11a Revert "Bmoric/remove docker compose for build (#7500)" (#7698)
d84f33a Revert "have docker-compose.build.yaml to use new docker context (#7696)" (#7697)
b5af3f1 have docker-compose.build.yaml to use new docker context (#7696)
058c8f8 Backfill specs into definitions (#7616)
9791a14 🎉 Source Shopify: Add FulfillmentOrders and Fulfillments streams (#7107)
9bda6a7 🐛 Bugfix: inject http client into server to prevent file churn (#7688)
45f6559 Fixed Mac M1 build (#7687)
16d9d15 🎉 New Destination: Pulsar (#7315)
2efef3c 🐛 Source Salesforce: Fix getting anyType fields using BULK API (#7592)
9f750c2 Publish PR 7186: config files for destination cassandra (#7685)
f53fd5e 🎉 New destination: Cassandra (#7186)
1f295f2 🎉 Source Google Directory: support oauth
497858d Source Facebook Marketing: Bump docker version (#7682)
4e17fa2 Bmoric/remove docker compose for build (#7500)
23f3d3e Source Facebook Marketing: Fix number and integer fields in schemas (#7605)
c6edf13 Eliziario/hubspot oauth (#7279)
7fd1804 Fix numerous create calls (#7659)
56db806 🐛 Source MSSQL: fix data type (smalldatetime, smallmoney) conversion from mssql source (#5609) (#7386)
af1c62f Publish new source Confluence #7241 (#7666)
4a99ac3 🎉 New Source: Confluence (#7241)
93afe18 Remove onboarding call reference. (#7665)