C2D Docker

[alexcos20]

Breaking Changes proposed in this PR (requires other repos to be updated):

• ComputeAsset interface (will require ocean.js updates) (https://github.com/oceanprotocol/ocean-node/pull/705/files#diff-136208859964be379b346a07cbf27655403475ab701c3f4d0d6f0fa4c37726e8R89)
  Changes: instead of having url, we have fileObject now, so we can support multiple types of non published assets
• ComputeStartCommand interface (will require ocean.js updates) (

  ocean-node/src/@types/commands.ts

  Line 159 in d26a8c5

  export interface ComputeStartCommand extends Command {

  )
  Changes: instead of having dataset and additionalDatasets, we merged them all under datasets

Changes proposed in this PR:

• new freeStartCompute and getComputeStreamableLogs commands
• c2d using docker engine

All additional work should be based on PRs based against this one

Usage:

 export DOCKER_SOCKET_PATH='/var/run/docker.sock'

and then do a directCommand:

{
    "command": "freeStartCompute",
    "consumerAddress": "0xC7EC1970B09224B317c52d92f37F5e1E4fF6B687",
    "nonce": 1,
    "signature": "0x123",
    "datasets": [
        {
            "fileObject": {
                "type": "url",
                "url": "SOME_DATASET_URL",
                "method": "GET"
            }
        }
    ],
    "algorithm": {
        "fileObject": {
            "type": "url",
            "url": "SOME_ALGO_URL",
            "method": "GET"
        },
        "meta": {
            "container": {
                "image": "SOME_CONTAINER",
                "tag": "latest",
                "entrypoint": "python $ALGO'"
            }
        }
    }
}

[paulo-ocean]

• Changes: instead of having dataset and additionalDatasets, we merged them all under datasets

FYI, i've created a PR on Ocean JS related with the breaking changes:
oceanprotocol/ocean.js#1867

other changes also included in this PR here: #735 #737 #736 #739

[paulo-ocean]

Breaking Changes proposed in this PR (requires other repos to be updated):

• ComputeAsset interface (will require ocean.js updates) (https://github.com/oceanprotocol/ocean-node/pull/705/files#diff-136208859964be379b346a07cbf27655403475ab701c3f4d0d6f0fa4c37726e8R89)
  Changes: instead of having url, we have fileObject now, so we can support multiple types of non published assets
• ComputeStartCommand interface (will require ocean.js updates) (

  ocean-node/src/@types/commands.ts

  Line 159 in d26a8c5

  export interface ComputeStartCommand extends Command {

  
  )
  Changes: instead of having dataset and additionalDatasets, we merged them all under datasets

Changes proposed in this PR:

• new freeStartCompute and getComputeStreamableLogs commands
• c2d using docker engine

All additional work should be based on PRs based against this one

Usage:

 export DOCKER_SOCKET_PATH='/var/run/docker.sock'

and then do a directCommand:

{
    "command": "freeStartCompute",
    "consumerAddress": "0xC7EC1970B09224B317c52d92f37F5e1E4fF6B687",
    "nonce": 1,
    "signature": "0x123",
    "datasets": [
        {
            "fileObject": {
                "type": "url",
                "url": "SOME_DATASET_URL",
                "method": "GET"
            }
        }
    ],
    "algorithm": {
        "fileObject": {
            "type": "url",
            "url": "SOME_ALGO_URL",
            "method": "GET"
        },
        "meta": {
            "container": {
                "image": "SOME_CONTAINER",
                "tag": "latest",
                "entrypoint": "python $ALGO'"
            }
        }
    }
}

Hi @alexcos20 , what if the 'fileObject' is not present on the datasets or on the algorithm ? We need to get it from the service files object? I mean, get the ddos & decrypt the service.files to get these right?
Its also not clear to me, how we pass this, for instance from the CLI -> start compute ... We have no idea to know the fileObject info, without decrypting the service files.. or am i missing something here?
thanks

[jamiehewitt15]

We should update the documentation to explain how this works and how to set it up.

[jamiehewitt15]

I tried to test it via the directCommand endpoint but I found it causes the node to crash. I'll give it another go when it's marked as ready for review, as changes are still being made.

[jamiehewitt15]

Update: it seems to work via the direct command now but I will do some more testing to see if there are any more bugs

[paulo-ocean]

Update: it seems to work via the direct command now but I will do some more testing to see if there are any more bugs

sure, but this is a draft in progress :-) expect it