DGD-4258 - Merge tag 1.23.0 into our fork #7
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…e#2889) Signed-off-by: Jonathan Moraes <[email protected]>
* Add kafka resolver validation. (OpenLineage#2868) Signed-off-by: pawelkocinski <[email protected]> * Apply linter Signed-off-by: pawelkocinski <[email protected]> --------- Signed-off-by: pawelkocinski <[email protected]>
Signed-off-by: Pawel Leszczynski <[email protected]>
…LineageRelation extension interface (OpenLineage#2893) Signed-off-by: Dominik Dębowczyk <[email protected]> Co-authored-by: Dominik Dębowczyk <[email protected]>
* adding descriptions and marquez-ui to docker compose file Signed-off-by: Jonathan Moraes <[email protected]> * Adding support for docker-config file. Changes in the README file to explain how to use this configuration Signed-off-by: Jonathan Moraes <[email protected]> * Adding support for docker-config file. Changes in the README file to explain how to use this configuration Signed-off-by: Jonathan Moraes <[email protected]> --------- Signed-off-by: Jonathan Moraes <[email protected]>
Signed-off-by: Pawel Leszczynski <[email protected]>
…age#2892) * The field "preCanonicalized" which may cause infinite recursive calls is skipped during serialization. * Modified the test to return truly valid JSON from the serialization method Signed-off-by: Artur Owczarek <[email protected]>
* support Spark 4.0 Signed-off-by: Pawel Leszczynski <[email protected]>
…lecting input column lineage (OpenLineage#2900) Signed-off-by: Dominik Dębowczyk <[email protected]> Co-authored-by: Dominik Dębowczyk <[email protected]>
Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Pawel Leszczynski <[email protected]>
* Add Kinesis streaming support for open lineage. Signed-off-by: pawelkocinski <[email protected]> * Apply linter Signed-off-by: pawelkocinski <[email protected]> * Apply linter Signed-off-by: pawelkocinski <[email protected]> * Add missing spark stop which was causing tests to fail. Signed-off-by: pawelkocinski <[email protected]> * Apply linter Signed-off-by: pawelkocinski <[email protected]> --------- Signed-off-by: pawelkocinski <[email protected]>
* adding descriptions and marquez-ui to docker compose file Signed-off-by: Jonathan Moraes <[email protected]> * Adding support for docker-config file. Changes in the README file to explain how to use this configuration Signed-off-by: Jonathan Moraes <[email protected]> * Adding support for docker-config file. Changes in the README file to explain how to use this configuration Signed-off-by: Jonathan Moraes <[email protected]> --------- Signed-off-by: Jonathan Moraes <[email protected]>
* Update COMMITTERS.md adding Jens and Sheeri * Update COMMITTERS.md adding Jens and Sheeri Signed-off-by: Sheeri K. Cabral <[email protected]> --------- Signed-off-by: Sheeri K. Cabral <[email protected]>
* Add mongo read replica support in spark structured streaming. Signed-off-by: pawelkocinski <[email protected]> * Rebase with main. Signed-off-by: pawelkocinski <[email protected]> * Apply linter Signed-off-by: pawelkocinski <[email protected]> * Disable test for spark 4.0. Signed-off-by: pawelkocinski <[email protected]> * Apply rebase. Signed-off-by: pawelkocinski <[email protected]> * Apply linter Signed-off-by: pawelkocinski <[email protected]> --------- Signed-off-by: pawelkocinski <[email protected]>
* add handling for Window Signed-off-by: tnazarew <[email protected]> * add handling for Window Signed-off-by: tnazarew <[email protected]> * update changelog Signed-off-by: tnazarew <[email protected]> * fix test query Signed-off-by: tnazarew <[email protected]> --------- Signed-off-by: tnazarew <[email protected]>
Signed-off-by: Artur Owczarek <[email protected]>
* Missing AWS Glue symlinks are generated for every storage format. Signed-off-by: Artur Owczarek <[email protected]>
Bumps [aiohttp](https://github.com/aio-libs/aiohttp) from 3.9.4 to 3.10.2. - [Release notes](https://github.com/aio-libs/aiohttp/releases) - [Changelog](https://github.com/aio-libs/aiohttp/blob/master/CHANGES.rst) - [Commits](aio-libs/aiohttp@v3.9.4...v3.10.2) --- updated-dependencies: - dependency-name: aiohttp dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…penLineage#2930) Bumps plugin.serialization from 2.0.0 to 2.0.10. --- updated-dependencies: - dependency-name: plugin.serialization dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…penLineage#2931) Bumps plugin.serialization from 2.0.0 to 2.0.10. --- updated-dependencies: - dependency-name: plugin.serialization dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…e#2928) Bumps [io.freefair.lombok](https://github.com/freefair/gradle-plugins) from 8.6 to 8.7.1. - [Release notes](https://github.com/freefair/gradle-plugins/releases) - [Commits](freefair/gradle-plugins@8.6...8.7.1) --- updated-dependencies: - dependency-name: io.freefair.lombok dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…enLineage#2926) Bumps org.slf4j:slf4j-simple from 2.0.13 to 2.0.16. --- updated-dependencies: - dependency-name: org.slf4j:slf4j-simple dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…enLineage#2933) Signed-off-by: Charles Kuzmiak <[email protected]>
…ge#2936) * add static code analysis and software composition analysis to help improve code and security on the repo Signed-off-by: Harel Shein <[email protected]> * update app name Signed-off-by: Harel Shein <[email protected]> --------- Signed-off-by: Harel Shein <[email protected]>
Signed-off-by: Jakub Dardzinski <[email protected]>
…or options (OpenLineage#2934) Signed-off-by: Dominik Dębowczyk <[email protected]>
Signed-off-by: pawelkocinski <[email protected]>
…eage#2943) * Fix AWS Glue Iceberg symlinks Signed-off-by: Artur Owczarek <[email protected]>
… replicating building dependencies (OpenLineage#2788) * test ol changes Signed-off-by: Maciej Obuchowski <[email protected]> * release on main Signed-off-by: Maciej Obuchowski <[email protected]> --------- Signed-off-by: Maciej Obuchowski <[email protected]>
* Configure Python client with environment variables. Signed-off-by: Jakub Dardzinski <[email protected]> * Alias `message_key` to `messageKey` in KafkaConfig. Signed-off-by: Jakub Dardzinski <[email protected]> --------- Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Jakub Dardzinski <[email protected]>
…ineage#3120) Signed-off-by: Artur Owczarek <[email protected]>
…#3117) Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Dominik Dębowczyk <[email protected]>
Signed-off-by: Emmanuel Ferdman <[email protected]>
…e#3124) Bumps the client-java group in /client/java with 5 updates: | Package | From | To | | --- | --- | --- | | [org.junit.jupiter:junit-jupiter](https://github.com/junit-team/junit5) | `5.11.0` | `5.11.1` | | [com.google.guava:guava](https://github.com/google/guava) | `33.3.0-jre` | `33.3.1-jre` | | [com.google.cloud:google-cloud-datalineage](https://github.com/googleapis/google-cloud-java) | `0.41.0` | `0.43.0` | | [com.google.cloud:google-cloud-storage](https://github.com/googleapis/java-storage) | `2.42.0` | `2.43.1` | | [com.google.cloud:google-cloud-nio](https://github.com/googleapis/java-storage-nio) | `0.123.10` | `0.127.23` | Updates `org.junit.jupiter:junit-jupiter` from 5.11.0 to 5.11.1 - [Release notes](https://github.com/junit-team/junit5/releases) - [Commits](junit-team/junit5@r5.11.0...r5.11.1) Updates `com.google.guava:guava` from 33.3.0-jre to 33.3.1-jre - [Release notes](https://github.com/google/guava/releases) - [Commits](https://github.com/google/guava/commits) Updates `com.google.cloud:google-cloud-datalineage` from 0.41.0 to 0.43.0 - [Release notes](https://github.com/googleapis/google-cloud-java/releases) - [Changelog](https://github.com/googleapis/google-cloud-java/blob/main/CHANGELOG.md) - [Commits](googleapis/google-cloud-java@v0.41.0...v0.43.0) Updates `com.google.cloud:google-cloud-storage` from 2.42.0 to 2.43.1 - [Release notes](https://github.com/googleapis/java-storage/releases) - [Changelog](https://github.com/googleapis/java-storage/blob/main/CHANGELOG.md) - [Commits](googleapis/java-storage@v2.42.0...v2.43.1) Updates `com.google.cloud:google-cloud-nio` from 0.123.10 to 0.127.23 - [Release notes](https://github.com/googleapis/java-storage-nio/releases) - [Changelog](https://github.com/googleapis/java-storage-nio/blob/main/CHANGELOG.md) - [Commits](googleapis/java-storage-nio@v0.123.10...v0.127.23) --- updated-dependencies: - dependency-name: org.junit.jupiter:junit-jupiter dependency-type: direct:production update-type: version-update:semver-patch dependency-group: client-java - dependency-name: com.google.guava:guava dependency-type: direct:production update-type: version-update:semver-patch dependency-group: client-java - dependency-name: com.google.cloud:google-cloud-datalineage dependency-type: direct:production update-type: version-update:semver-minor dependency-group: client-java - dependency-name: com.google.cloud:google-cloud-storage dependency-type: direct:production update-type: version-update:semver-minor dependency-group: client-java - dependency-name: com.google.cloud:google-cloud-nio dependency-type: direct:production update-type: version-update:semver-minor dependency-group: client-java ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…OpenLineage#3125) Bumps the integration-spark group in /integration/spark with 4 updates: [org.junit.jupiter:junit-jupiter-api](https://github.com/junit-team/junit5), [org.junit.jupiter:junit-jupiter](https://github.com/junit-team/junit5), [org.junit:junit-bom](https://github.com/junit-team/junit5) and [org.xerial:sqlite-jdbc](https://github.com/xerial/sqlite-jdbc). Updates `org.junit.jupiter:junit-jupiter-api` from 5.11.0 to 5.11.1 - [Release notes](https://github.com/junit-team/junit5/releases) - [Commits](junit-team/junit5@r5.11.0...r5.11.1) Updates `org.junit.jupiter:junit-jupiter` from 5.11.0 to 5.11.1 - [Release notes](https://github.com/junit-team/junit5/releases) - [Commits](junit-team/junit5@r5.11.0...r5.11.1) Updates `org.junit:junit-bom` from 5.11.0 to 5.11.1 - [Release notes](https://github.com/junit-team/junit5/releases) - [Commits](junit-team/junit5@r5.11.0...r5.11.1) Updates `org.junit.jupiter:junit-jupiter` from 5.11.0 to 5.11.1 - [Release notes](https://github.com/junit-team/junit5/releases) - [Commits](junit-team/junit5@r5.11.0...r5.11.1) Updates `org.junit:junit-bom` from 5.11.0 to 5.11.1 - [Release notes](https://github.com/junit-team/junit5/releases) - [Commits](junit-team/junit5@r5.11.0...r5.11.1) Updates `org.xerial:sqlite-jdbc` from 3.46.1.0 to 3.46.1.3 - [Release notes](https://github.com/xerial/sqlite-jdbc/releases) - [Changelog](https://github.com/xerial/sqlite-jdbc/blob/master/CHANGELOG) - [Commits](xerial/sqlite-jdbc@3.46.1.0...3.46.1.3) --- updated-dependencies: - dependency-name: org.junit.jupiter:junit-jupiter-api dependency-type: direct:production update-type: version-update:semver-patch dependency-group: integration-spark - dependency-name: org.junit.jupiter:junit-jupiter dependency-type: direct:production update-type: version-update:semver-patch dependency-group: integration-spark - dependency-name: org.junit:junit-bom dependency-type: direct:production update-type: version-update:semver-patch dependency-group: integration-spark - dependency-name: org.junit.jupiter:junit-jupiter dependency-type: direct:production update-type: version-update:semver-patch dependency-group: integration-spark - dependency-name: org.junit:junit-bom dependency-type: direct:production update-type: version-update:semver-patch dependency-group: integration-spark - dependency-name: org.xerial:sqlite-jdbc dependency-type: direct:production update-type: version-update:semver-patch dependency-group: integration-spark ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* When flag columnLineage.deprecatedMechanismEnabled, then the dataset dependencies are extracted from property field into separate field (dataset) Signed-off-by: Artur Owczarek <[email protected]>
Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Dominik Dębowczyk <[email protected]>
* Adds S3 transport as additional dependency Signed-off-by: Artur Owczarek <[email protected]>
* add support for dbt-athena adapter Signed-off-by: Moritz <[email protected]> * fix athena namespace format for dbt integration Signed-off-by: Moritz <[email protected]> --------- Signed-off-by: Moritz <[email protected]>
Signed-off-by: Maciej Obuchowski <[email protected]>
Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Maciej Obuchowski <[email protected]>
Signed-off-by: Maciej Obuchowski <[email protected]>
…o avoid signing during build
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR merges upstream tag 1.23.0 into our fork, keeping the fork's customizations and making a few adjustments to Gradle build configuration.
-SNAPSHOTstring was appended to all gradle.properties version value in order to avoid signing during build.Tests were made by running
integration/spark/docker-compose.ymland a custom notebook to force lineage events, which were properly stored into marquez.