You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Current Bohrium executor runs an actual Pod with a go process or a DPDispatcher script running in the container to poll the job status. When massive steps with Bohrium executor are running simultaneously, the resource usage is considerable.
To reduce the cost, we can use virtual Kubelet to submit Bohrium job and poll the job status in the framework of https://github.com/dptech-corp/virtual-kubelet. A prerequisite of this project is a unified artifact storage (e.g. OSS) as the interface of Argo Workflows and Bohrium.
The text was updated successfully, but these errors were encountered:
Another possible solution is using executor plugins of Argo to write an executor for Bohrium tasks. It is needed to develop a server which implements APIs to submit Bohrium jobs and to acquire job statuses.
zjgemi
changed the title
Reform Bohrium executor with virtual Kubelet
Reform Bohrium executor with virtual Kubelet or Argo executor plugin
Jun 13, 2023
Current Bohrium executor runs an actual Pod with a go process or a DPDispatcher script running in the container to poll the job status. When massive steps with Bohrium executor are running simultaneously, the resource usage is considerable.
To reduce the cost, we can use virtual Kubelet to submit Bohrium job and poll the job status in the framework of https://github.com/dptech-corp/virtual-kubelet. A prerequisite of this project is a unified artifact storage (e.g. OSS) as the interface of Argo Workflows and Bohrium.
The text was updated successfully, but these errors were encountered: