Test NM GPU image with KFP

[pacospace]

Depends-On: #91

[pacospace]

Currently blocked because pipeline fails to start. I checked with @Shreyanand that we were using same runtime and runtime image, but for me the pipeline cannot start with the following issue:

Traceback (most recent call last):
  File "/opt/app-root/lib64/python3.8/site-packages/elyra/pipeline/processor_kfp.py", line 157, in process
    kfp_tekton_compiler.TektonCompiler().compile(pipeline_function, pipeline_path)
  File "/opt/app-root/lib64/python3.8/site-packages/kfp_tekton/compiler/compiler.py", line 740, in compile
    super().compile(pipeline_func, package_path, type_check, pipeline_conf=pipeline_conf)
  File "/opt/app-root/lib64/python3.8/site-packages/kfp/compiler/compiler.py", line 948, in compile
    self._create_and_write_workflow(
  File "/opt/app-root/lib64/python3.8/site-packages/kfp_tekton/compiler/compiler.py", line 833, in _create_and_write_workflow
    workflow = self._create_workflow(
  File "/opt/app-root/lib64/python3.8/site-packages/kfp_tekton/compiler/compiler.py", line 661, in _create_workflow
    pipeline_func(*args_list)
  File "/opt/app-root/lib64/python3.8/site-packages/elyra/pipeline/processor_kfp.py", line 151, in <lambda>
    pipeline_function = lambda: self._cc_pipeline(pipeline,  # nopep8 E731
  File "/opt/app-root/lib64/python3.8/site-packages/elyra/pipeline/processor_kfp.py", line 450, in _cc_pipeline
    self._upload_dependencies_to_object_store(runtime_configuration,
  File "/opt/app-root/lib64/python3.8/site-packages/elyra/pipeline/processor.py", line 306, in _upload_dependencies_to_object_store
    raise ex from ex
  File "/opt/app-root/lib64/python3.8/site-packages/elyra/pipeline/processor.py", line 281, in _upload_dependencies_to_object_store
    cos_client = CosClient(config=runtime_configuration)
  File "/opt/app-root/lib64/python3.8/site-packages/elyra/util/cos.py", line 41, in __init__
    self.client = self.__initialize_object_store()
  File "/opt/app-root/lib64/python3.8/site-packages/elyra/util/cos.py", line 53, in __initialize_object_store
    if not self.client.bucket_exists(self.bucket):
  File "/opt/app-root/lib64/python3.8/site-packages/minio/api.py", line 404, in bucket_exists
    self._url_open('HEAD', bucket_name=bucket_name)
  File "/opt/app-root/lib64/python3.8/site-packages/minio/api.py", line 2226, in _url_open
    raise ResponseError(response,
minio.error.AccessDenied: AccessDenied: message: Access Denied

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/app-root/lib64/python3.8/site-packages/tornado/web.py", line 1704, in _execute
    result = await result
  File "/opt/app-root/lib64/python3.8/site-packages/elyra/pipeline/handlers.py", line 89, in post
    response = await PipelineProcessorManager.instance().process(pipeline)
  File "/opt/app-root/lib64/python3.8/site-packages/elyra/pipeline/processor.py", line 78, in process
    res = await asyncio.get_event_loop().run_in_executor(None, processor.process, pipeline)
  File "/usr/lib64/python3.8/asyncio/futures.py", line 260, in __await__
    yield self  # This tells Task to wait for completion.
  File "/usr/lib64/python3.8/asyncio/tasks.py", line 349, in __wakeup
    future.result()
  File "/usr/lib64/python3.8/asyncio/futures.py", line 178, in result
    raise self._exception
  File "/usr/lib64/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/opt/app-root/lib64/python3.8/site-packages/elyra/pipeline/processor_kfp.py", line 161, in process
    raise RuntimeError('Error compiling pipeline {} for engine {} at {}'.
RuntimeError: ('Error compiling pipeline untitled for engine Tekton at /tmp/tmpulpi50rl/untitled.tar.gz', 'AccessDenied: message: Access Denied')
Check the JupyterLab log for more details at 2022-03-13 19:32:25