Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

C2D Docker #705

Draft
wants to merge 61 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
61 commits
Select commit Hold shift + click to select a range
4767ae9
start work on docker c2d
alexcos20 Oct 1, 2024
b355b2d
more on db
alexcos20 Oct 1, 2024
048c68a
fix c2d database
alexcos20 Oct 2, 2024
f866a10
BREAKING: refactor ComputeAsset and ComputeStartCommand interfaces
alexcos20 Oct 5, 2024
fd1f279
add freeCompute handler
alexcos20 Oct 5, 2024
92c0fd2
working POC of Docker compute
alexcos20 Oct 6, 2024
81e8ac5
small fixes
alexcos20 Oct 6, 2024
a69a07c
update deps
alexcos20 Oct 6, 2024
57ccbed
update ignores
alexcos20 Oct 6, 2024
2a7ac31
fix getComputeJobResult and getComputeJobStatus
alexcos20 Oct 7, 2024
5ad1953
wip: add sql lite db
paulo-ocean Oct 21, 2024
6d71253
more changes + refactor internal blob/body
paulo-ocean Oct 22, 2024
61459f6
add get running jobs
paulo-ocean Oct 22, 2024
d6c74e9
add logging
paulo-ocean Oct 23, 2024
90814f1
wip: adding cron to delete expired storage and expired jobs
paulo-ocean Oct 25, 2024
dab2a02
add test for getting expied and clean
paulo-ocean Oct 28, 2024
1a27b2f
refactor deletion, add .env properties
paulo-ocean Oct 28, 2024
b34319d
small refactor
paulo-ocean Oct 28, 2024
04064f2
wip: add env vars for docket compute envs
paulo-ocean Nov 5, 2024
a83e68f
wip: add env vars for docket compute envs
paulo-ocean Nov 5, 2024
ad605a1
add config options
paulo-ocean Nov 6, 2024
d61967d
basic cd2 free compute flow
paulo-ocean Nov 8, 2024
d562d90
add some comments
paulo-ocean Nov 11, 2024
72bf88e
add optional signature, for template 4 (need to get files object from…
paulo-ocean Nov 11, 2024
e0789a4
add consume data/signature for start/init c2d job
paulo-ocean Nov 12, 2024
8b6fd45
add some comments
paulo-ocean Nov 12, 2024
a834239
add c2d docker options on quick start script
paulo-ocean Nov 13, 2024
8427f6e
improve script for envs, improve config function to deal with empty s…
paulo-ocean Nov 13, 2024
2443ab4
add unit test for free docker env
paulo-ocean Nov 14, 2024
9aa6b6d
add integration test free docker start compute
paulo-ocean Nov 14, 2024
dfdba31
try fix test, debug
paulo-ocean Nov 14, 2024
4ea2372
refactor get compute envs, check chain filter
paulo-ocean Nov 14, 2024
5354082
fix test
paulo-ocean Nov 14, 2024
48dce80
enhance types + refactro config... still missing couple basic checks
paulo-ocean Nov 18, 2024
9ee1d96
add some basic checks, log the issues but proceed anyway
paulo-ocean Nov 18, 2024
d7a6544
add try catch, check container image
paulo-ocean Nov 19, 2024
4041171
wip: add intergation test with directCommand payload
paulo-ocean Nov 20, 2024
8d557b6
cleanup intergation test, debug
paulo-ocean Nov 20, 2024
6f20521
add support for env architectures
paulo-ocean Nov 20, 2024
ce9055d
increase maxJobDuration
FilipMasar Nov 20, 2024
8c7d2bd
add missing type declaraion
paulo-ocean Nov 22, 2024
93a406d
check algo image and manifest
paulo-ocean Nov 25, 2024
e18453a
fix imports issues
paulo-ocean Nov 25, 2024
bab3b04
fix typo in storage tests
paulo-ocean Nov 25, 2024
dc5497e
fix image parsing on meta
paulo-ocean Nov 25, 2024
bd1e37c
fix test, job should not start with invalid image
paulo-ocean Nov 25, 2024
d950c6a
add validation params as output of check image
paulo-ocean Nov 26, 2024
9159551
debug response on test
paulo-ocean Nov 26, 2024
2584990
fix promise resolve, use new return type
paulo-ocean Nov 26, 2024
21632b9
complete checkfor platform and arch if present on manifest
paulo-ocean Dec 2, 2024
644b1bf
add possible missing 0x prefix on agreementId
paulo-ocean Dec 4, 2024
9ad8b9a
Merge pull request #777 from oceanprotocol/issue-776-compute-status
paulo-ocean Dec 9, 2024
83d01f8
merge main fix conflicts
paulo-ocean Dec 10, 2024
3a85284
wip: get container stats
paulo-ocean Dec 6, 2024
9687a97
add hostconfig options for mem and cpu
paulo-ocean Dec 12, 2024
d8201c3
set swap same value, disable it
paulo-ocean Dec 13, 2024
63e2987
small refactor
paulo-ocean Dec 13, 2024
910c5dc
small refactor
paulo-ocean Dec 13, 2024
5651337
remove some commented lines
paulo-ocean Dec 13, 2024
ddfc8a5
Merge pull request #786 from oceanprotocol/issue-768-env-cpu-mem
paulo-ocean Dec 16, 2024
b96debe
small refactor, support multiple platforms, most for k8
paulo-ocean Dec 16, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,5 +2,6 @@ node_modules
/dist/*
!/dist/dashboard
logs
c2d_storage
.env.local
.env
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -158,3 +158,4 @@ html-report.html
# databases
*.sqlite
databases/*
c2d_storage/*
5 changes: 5 additions & 0 deletions docs/env.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,3 +84,8 @@ Environmental variables are also tracked in `ENVIRONMENT_VARIABLES` within `src/

- `NODE1_PRIVATE_KEY`: Used on test environments, specifically CI, represents the private key for node 1. Example: `"0xfd5c1ccea015b6d663618850824154a3b3fb2882c46cefb05b9a93fea8c3d215"`
- `NODE2_PRIVATE_KEY`: Used on test environments, specifically CI, represents the private key for node 2. Example: `"0x1263dc73bef43a9da06149c7e598f52025bf4027f1d6c13896b71e81bb9233fb"`

## Cron Jobs

- `CRON_DELETE_DB_LOGS`: Delete old logs from database Cron expression. Example: `0 0 * * *` (runs every day at midnight)
- `CRON_CLEANUP_C2D_STORAGE`: Clear c2d expired resources/storage and delete old jobs. Example: `*/5 * * * *` (runs every 5 minutes)
Loading
Loading