dbt-spark v1.6.0b3

dbt-spark 1.6.0-b3 - June 09, 2023

Features

• Standardize the _connection_keys and debug_query for dbt debug. (#PR754)

Fixes

• wrap expression for check constraints in parentheses (#7480)

Under the Hood

• drop support for python 3.7 (#792)

dbt-spark v1.3.2

dbt-spark 1.3.2 - May 26, 2023

Fixes

• Fixed issue where table materialization was not always properly refreshing for non-admin users on Databricks (#725)

Contributors

• @mikealfare (#725)

dbt-spark v1.4.2

dbt-spark 1.4.2 - May 26, 2023

Fixes

• Fixed issue where table materialization was not always properly refreshing for non-admin users on Databricks (#725)

Contributors

• @mikealfare (#725)

dbt-spark v1.6.0b2

dbt-spark 1.6.0-b2 - May 25, 2023

Features

• All constraint types are supported, but not enforced. (#656, #657)

dbt-spark v1.6.0b1

dbt-spark 1.6.0-b1 - May 12, 2023

Features

• Support insert_overwrite strategy with delta (#1013)

Fixes

• Fixed issue where table materialization was not always properly refreshing for non-admin users on Databricks (#725)
• Remove dead code 💀 (#758)

Under the Hood

• Remove unneeded type ignore (#753)

Dependencies

• Update black requirement from ~=23.1 to ~=23.3 (#701)
• Bump mypy from 1.1.1 to 1.2.0 (#708)
• Update pytest requirement from ~=7.2 to ~=7.3 (#726)
• Update types-pytz requirement from ~=2023.2 to ~=2023.3 (#704)
• Update pip-tools requirement from ~=6.12 to ~=6.13 (#737)
• Update pytz requirement from ~=2023.2 to ~=2023.3 (#702)

Dependency

• Bump pyodbc from 4.0.34 to 4.0.35 (#417)

Contributors

• @Fokko (#758, #753)
• @flvndh (#1013)

dbt-spark v1.5.0

dbt-spark 1.5.0 - April 27, 2023

Features

• Support for data types constraints in Spark following the dbt Core feature #6271 (#558)
• Enforce contracts on models materialized as tables and views (#639, #654)
• Modify adapter to support unified constraint fields (#655)
• Modify order of columns in create_table_as to match contract (#671)

Fixes

• Support for iceberg v2 tables. Added ability to use multiple join conditions to allow for multiple columns to make a row distinct. (#294)
• Use take() instead of collect on dataframe to improve the performance (#526)
• add merge_exclude_columns tests (#00)
• Fix pyodbc type_code -> data_type conversion (#665)
• Fixed issue where table materialization was not always properly refreshing for non-admin users on Databricks (#725)

Under the Hood

• remove tox call to integration tests (#584)
• Treat contract config as a python object (#645, #693)

Dependencies

• Allow thrift 0.16.0 (#605)
• Update ipdb requirement from ~=0.13.11 to ~=0.13.13 (#677)
• Update wheel requirement from ~=0.38 to ~=0.40 (#680)
• Update pre-commit requirement from ~=2.21 to ~=3.2 (#687)
• Update wheel requirement from ~=0.38 to ~=0.40 (#680)
• Bump mypy from 1.0.1 to 1.1.1 (#675)
• Update types-pytz requirement from ~=2022.7 to ~=2023.2 (#697)
• Update pytz requirement from ~=2022.7 to ~=2023.2 (#696)

Contributors

• @akashrn5 (#526)
• @b-per (#558)
• @dave-connors-3 (#00)
• @dparent1 (#294)

dbt-spark v1.5.0rc1

dbt-spark 1.5.0-rc1 - April 21, 2023

Features

• Support for data types constraints in Spark following the dbt Core feature #6271 (#558)
• Enforce contracts on models materialized as tables and views (#639, #654)
• Modify adapter to support unified constraint fields (#655)
• Modify order of columns in create_table_as to match contract (#671)

Fixes

• Support for iceberg v2 tables. Added ability to use multiple join conditions to allow for multiple columns to make a row distinct. (#294)
• Use take() instead of collect on dataframe to improve the performance (#526)
• add merge_exclude_columns tests (#00)
• Fix pyodbc type_code -> data_type conversion (#665)
• Fixed issue where table materialization was not always properly refreshing for non-admin users on Databricks (#725)

Under the Hood

• remove tox call to integration tests (#584)
• Treat contract config as a python object (#645, #693)

Dependencies

• Allow thrift 0.16.0 (#605)
• Update ipdb requirement from ~=0.13.11 to ~=0.13.13 (#677)
• Update wheel requirement from ~=0.38 to ~=0.40 (#680)
• Update pre-commit requirement from ~=2.21 to ~=3.2 (#687)
• Update wheel requirement from ~=0.38 to ~=0.40 (#680)
• Bump mypy from 1.0.1 to 1.1.1 (#675)
• Update types-pytz requirement from ~=2022.7 to ~=2023.2 (#697)
• Update pytz requirement from ~=2022.7 to ~=2023.2 (#696)

Contributors

• @akashrn5 (#526)
• @b-per (#558)
• @dave-connors-3 (#00)
• @dparent1 (#294)

dbt-spark v1.5.0b4

dbt-spark 1.5.0-b4 - March 30, 2023

Features

• Modify adapter to support unified constraint fields (#655)
• Modify order of columns in create_table_as to match contract (#671)

Under the Hood

• Treat contract config as a python object (#645, #693)

Dependencies

• Update ipdb requirement from ~=0.13.11 to ~=0.13.13 (#677)
• Update wheel requirement from ~=0.38 to ~=0.40 (#680)
• Update pre-commit requirement from ~=2.21 to ~=3.2 (#687)
• Update wheel requirement from ~=0.38 to ~=0.40 (#680)
• Bump mypy from 1.0.1 to 1.1.1 (#675)
• Update types-pytz requirement from ~=2022.7 to ~=2023.2 (#697)
• Update pytz requirement from ~=2022.7 to ~=2023.2 (#696)

dbt-spark v1.5.0b3

dbt-spark 1.5.0-b3 - March 16, 2023

Features

• Enforce contracts on models materialized as tables and views (#639, #654)

Fixes

• Use take() instead of collect on dataframe to improve the performance (#526)
• Fix pyodbc type_code -> data_type conversion (#665)

Contributors

• @akashrn5 (#526)

dbt-spark v1.5.0b2

dbt-spark 1.5.0-b2 - March 03, 2023

Fixes

• Support for iceberg v2 tables. Added ability to use multiple join conditions to allow for multiple columns to make a row distinct. (#294)

Under the Hood

• Rename constraints_enabled to contract (#645)

Contributors

• @dparent1 (#294)