Skip to content
This repository has been archived by the owner on Feb 8, 2023. It is now read-only.

Feature: collect subtask info into yaml format #70

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

hoarjour
Copy link

What do these changes do?

Collect subtask graph structure and runtime subtask information

Related issue number

Fixes #xxxx

Check code requirements

  • tests added / passed (if needed)
  • Ensure all linting tests pass, see here for how to run them

@hoarjour hoarjour requested a review from UranusSeven November 23, 2022 08:23
mars/services/task/supervisor/tests/test_task_manager.py Outdated Show resolved Hide resolved
mars/services/task/supervisor/tests/test_task_manager.py Outdated Show resolved Hide resolved
mars/services/task/supervisor/graph_visualizer.py Outdated Show resolved Hide resolved
mars/services/task/supervisor/graph_visualizer.py Outdated Show resolved Hide resolved
mars/services/cluster/api/core.py Outdated Show resolved Hide resolved
mars/services/cluster/api/oscar.py Outdated Show resolved Hide resolved
mars/services/cluster/yaml_dumper.py Outdated Show resolved Hide resolved
mars/services/task/supervisor/graph_visualizer.py Outdated Show resolved Hide resolved
mars/services/cluster/yaml_dumper.py Outdated Show resolved Hide resolved
mars/services/scheduling/worker/execution.py Outdated Show resolved Hide resolved
mars/services/scheduling/worker/execution.py Show resolved Hide resolved
mars/services/subtask/worker/processor.py Outdated Show resolved Hide resolved
mars/services/subtask/worker/processor.py Outdated Show resolved Hide resolved
mars/services/subtask/worker/processor.py Show resolved Hide resolved
mars/services/task/api/oscar.py Outdated Show resolved Hide resolved
mars/services/task/api/oscar.py Outdated Show resolved Hide resolved
mars/services/task/supervisor/processor.py Outdated Show resolved Hide resolved
mars/services/task/supervisor/processor.py Outdated Show resolved Hide resolved
mars/services/task/task_info_collector.py Outdated Show resolved Hide resolved
mars/services/subtask/worker/processor.py Outdated Show resolved Hide resolved
mars/services/task/api/oscar.py Outdated Show resolved Hide resolved
mars/services/task/api/oscar.py Outdated Show resolved Hide resolved
mars/services/task/api/oscar.py Outdated Show resolved Hide resolved
mars/services/task/supervisor/tests/test_task_manager.py Outdated Show resolved Hide resolved
mars/services/task/supervisor/tests/test_task_manager.py Outdated Show resolved Hide resolved
mars/services/task/task_info_collector.py Outdated Show resolved Hide resolved

class TaskInfoCollectorActor(mo.Actor):

def __init__(self):

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Try this:

mars_temp_dir = os.path.dirname(os.environ.get(MARS_LOG_PATH_KEY))
self.yaml_root_dir = os.path.join(mars_temp_dir, "mars_task_infos")

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add logs to tell the yaml locations.

mars/services/task/task_info_collector.py Outdated Show resolved Hide resolved
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants