Skip to content

Commit

Permalink
When importing in dbt format, add the dbt unique information as a d…
Browse files Browse the repository at this point in the history
…atacontract `unique` field (#558)

* When importing in dbt format, add the dbt unique information as a datacontract unique field

* Updated CHANGELOG
  • Loading branch information
syou6162 authored Dec 21, 2024
1 parent a18288f commit ff4813c
Show file tree
Hide file tree
Showing 3 changed files with 20 additions and 0 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]

### Added
- When importing in dbt format, add the dbt unique information as a datacontract unique field (#558)

### Changed

Expand Down
8 changes: 8 additions & 0 deletions datacontract/imports/dbt_importer.py
Original file line number Diff line number Diff line change
Expand Up @@ -155,4 +155,12 @@ def create_field(manifest: Manifest, model_unique_id: str, column: ColumnInfo, a
if required:
field.required = required

unique = False
if any(constraint.type == ConstraintType.unique for constraint in column.constraints):
unique = True
if [test for test in all_tests if test["test_type"] == "unique"]:
unique = True
if unique:
field.unique = unique

return field
11 changes: 11 additions & 0 deletions tests/test_import_dbt.py
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,7 @@ def test_import_dbt_manifest():
type: integer
description: This is a unique identifier for an order
required: true
unique: true
customer_id:
type: integer
description: Foreign key to the customers table
Expand Down Expand Up @@ -141,6 +142,7 @@ def test_import_dbt_manifest():
type: integer
description: ''
required: true
unique: true
first_name:
type: varchar
description: ''
Expand All @@ -155,6 +157,7 @@ def test_import_dbt_manifest():
type: integer
description: ''
required: true
unique: true
customer_id:
type: integer
description: ''
Expand All @@ -172,6 +175,7 @@ def test_import_dbt_manifest():
type: integer
description: ''
required: true
unique: true
order_id:
type: integer
description: ''
Expand All @@ -190,6 +194,7 @@ def test_import_dbt_manifest():
type: integer
description: This is a unique identifier for a customer
required: true
unique: true
first_name:
type: varchar
description: Customer's first name. PII.
Expand Down Expand Up @@ -238,6 +243,7 @@ def test_import_dbt_manifest_bigquery():
type: bigint
description: This is a unique identifier for an order
required: true
unique: true
customer_id:
type: bigint
description: Foreign key to the customers table
Expand Down Expand Up @@ -294,6 +300,7 @@ def test_import_dbt_manifest_bigquery():
type: bigint
description: ''
required: true
unique: true
first_name:
type: string
description: ''
Expand All @@ -308,6 +315,7 @@ def test_import_dbt_manifest_bigquery():
type: bigint
description: ''
required: true
unique: true
customer_id:
type: bigint
description: ''
Expand All @@ -325,6 +333,7 @@ def test_import_dbt_manifest_bigquery():
type: bigint
description: ''
required: true
unique: true
order_id:
type: bigint
description: ''
Expand All @@ -343,6 +352,7 @@ def test_import_dbt_manifest_bigquery():
type: bigint
description: This is a unique identifier for a customer
required: true
unique: true
first_name:
type: string
description: Customer's first name. PII.
Expand Down Expand Up @@ -413,6 +423,7 @@ def test_import_dbt_manifest_with_filter():
type: integer
description: This is a unique identifier for a customer
required: true
unique: true
first_name:
type: varchar
description: Customer's first name. PII.
Expand Down

0 comments on commit ff4813c

Please sign in to comment.