KFP Provider does not populate artifacts in run completion events

[jmendesky]

Describe the bug

Run completion events emitted by the KFP provider only contain ServingModelArtifacts, not Artifacts. This means that dependent run configurations do not work using this provider.

[grahamia]

On starting to develop this it became apparent that a functional regression has been made on upgrading to TFX > 1.2.0 which stops us from implementing a fix.

The "name" custom property which is relied upon to do the filtering in the metadata store in getArtifacts is now not set.

When executing the following gRPC calls:

grpcurl -d '{"type_name": "pipeline_run", "context_name": "kfp-operator-quickstart-kfp-k49nf"}' -proto ml_metadata/proto/metadata_store_service.proto -plaintext 127.0.0.1:8085 ml_metadata.MetadataStoreService.GetContextByTypeAndName
> returns the content_id to use in following call

grpcurl -d '{"context_id":"143456"}' -proto ml_metadata/proto/metadata_store_service.proto -plaintext 127.0.0.1:8085 ml_metadata.MetadataStoreService.GetArtifactsByContext

In older 1.2.0 TFX working version:

    {
      "id": "83917",
      "typeId": "33",
      "uri": ".../kfp-quickstart/Pusher/pushed_model/60170",
      "customProperties": {
        "name": {
          "stringValue": "kfp-quickstart:kfp-quickstart-kfp-877lkshjj4-6168-502898645:Pusher:pushed_model:0"
        },
        "pushed": {
          "intValue": "1"
        },
        "pushed_destination": {
          "stringValue": ".../kfp-quickstart/serving/1709567369"
        },
        "pushed_version": {
          "stringValue": "1709567369"
        },
        "tfx_version": {
          "stringValue": "1.2.0"
        }
      },
      "state": "LIVE",
      "createTimeSinceEpoch": "1709567384577",
      "lastUpdateTimeSinceEpoch": "1709567384577"
    }

In newer 1.13.0 no name property is included:

    {
      "id": "83922",
      "typeId": "33",
      "uri": ".../kfp-operator-quickstart-kfp/Pusher/pushed_model/60174",
      "customProperties": {
        "is_external": {
          "intValue": "0"
        },
        "pushed": {
          "intValue": "1"
        },
        "pushed_destination": {
          "stringValue": ".../kfp-operator-quickstart-kfp/serving/1709570854"
        },
        "pushed_version": {
          "stringValue": "1709570854"
        },
        "tfx_version": {
          "stringValue": "1.13.0"
        }
      },
      "state": "LIVE",
      "createTimeSinceEpoch": "1709570871359",
      "lastUpdateTimeSinceEpoch": "1709570871359"
    }