feature(rjy): add crowd md env new, and multi-head policy #230

nighood · 2024-06-07T08:15:01Z

New Environment: CrowdSim
- Description: The CrowdSim environment is a grid world simulation where robots navigate through an environment populated with humans. The primary task for the robots is to minimize the average age of information (AoI) of the humans by moving to their locations and collecting data. Key features of the environment include:
  - Dynamic Interaction: Humans generate data at a constant rate, and robots must manage their limited energy supply while moving to collect this data.
  - Modes:
    - Easy Mode: Robots can only collect data from humans within a certain range, and collecting data resets the AoI of a human to zero.
    - Hard Mode: Robots can collect data from humans even when not within range, and collecting data does not reset the total AoI.
  - Initialization: The environment starts with a dataset of human locations and timestamps. Robots aim to minimize the average AoI by efficiently collecting data.
  - Completion Criteria: The environment is considered solved when the average AoI is minimized to a certain threshold or the time limit is reached.
  - Additional Features: Methods for resetting, closing, and stepping through the environment, seeding for reproducibility, saving replay videos, and generating random actions. Detailed properties for accessing observation space, action space, and reward space.
Multi-Head Policy Version for MuZero, EfficientZero, and Sampled EfficientZero
- Modification: Introduced multi-head policy versions for the MuZero, EfficientZero, and Sampled EfficientZero algorithms.

lzero/agent/efficientzero.py

fix(rjy): fix crowd env scale/add entropy info

…nto rjy-crowd-md-env-new

puyuan1996 · 2024-06-25T09:23:22Z

lzero/agent/sampled_efficientzero.py

@@ -93,7 +93,12 @@ def __init__(
            cfg.main_config.exp_name = exp_name
        self.origin_cfg = cfg
        self.cfg = compile_config(
-            cfg.main_config, seed=seed, env=None, auto=True, policy=SampledEfficientZeroPolicy, create_cfg=cfg.create_config
+            cfg.main_config,
+            seed=seed,


多了一个空行吗

puyuan1996 · 2024-06-25T09:24:30Z

lzero/mcts/utils.py

        if observation_array.ndim == 3:
            # Flatten the last two dimensions
            observation_array = observation_array.reshape(batch_size, -1)
        else:
            raise ValueError("For 'mlp' model_type, the observation must have 3 dimensions [B, S, O]")

+    elif model_type == 'rgcn':
+        if observation_array.ndim == 4:
+            # TODO(rjy): strage process


strage process是什么意思？

puyuan1996 · 2024-06-25T09:25:29Z

lzero/model/common.py

-            activation: Optional[nn.Module] = nn.ReLU(inplace=True),
-            last_linear_layer_init_zero: bool = True,
-            norm_type: Optional[str] = 'BN',
+        self,


bash format.sh一下

puyuan1996 · 2024-06-25T09:26:27Z

lzero/model/common.py

+        output_support_size: int = 601,
+        last_linear_layer_init_zero: bool = True,
+        activation: Optional[nn.Module] = nn.ReLU(inplace=True),
+        norm_type: Optional[str] = 'BN',


这些缩进还是换成原来的格式哈

puyuan1996 · 2024-06-25T09:27:25Z

lzero/model/common_gcn.py

+    """
+    Overview:
+        Relational graph convolutional network layer.
+    """


给一下这里代码实现的参考链接

puyuan1996 · 2024-06-25T09:34:27Z

zoo/CrowdSim/envs/CrowdSim_env.py

@@ -0,0 +1,113 @@
+from typing import Union, Optional


文件名改为 crowdsim_env

puyuan1996 · 2024-06-25T09:34:50Z

zoo/CrowdSim/envs/CrowdSim_env.py

+@ENV_REGISTRY.register('crowdsim_lightzero')
+class CrowdSimEnv(BaseEnv):
+
+    def __init__(self, cfg: dict = {}) -> None:


增加overview注释

puyuan1996 · 2024-06-25T09:35:55Z

zoo/CrowdSim/envs/Crowdsim/env/model/agent.py

+        return len(self.queue)
+
+
+# # Example of using the InformationQueue class


确认这里的example能够正常运行

puyuan1996 · 2024-06-25T09:37:41Z

zoo/CrowdSim/envs/crowdsim_lightzero_env.py

+@ENV_REGISTRY.register('crowdsim_lightzero')
+class CrowdSimEnv(BaseEnv):
+
+    def __init__(self, cfg: dict = {}) -> None:


增加overview注释，将之前的文档中英文版本放在这里的envs/路径下面哈

puyuan1996 · 2024-06-25T09:38:05Z

zoo/CrowdSim/envs/test_crowdsim_lightzero_env.py

+        mcfg['obs_mode'] = '1-dim-array'
+        env = CrowdSimEnv(mcfg)
+        env.seed(314)
+        env.enable_save_replay('/home/nighoodRen/LightZero/result/test_replay')


个人信息替换成template

puyuan1996 · 2024-06-25T09:42:54Z

lzero/model/muzero_model_md.py

+
+
+@MODEL_REGISTRY.register('MuZeroModelMD')
+class MuZeroModelMD(nn.Module):


所有增加的文件都需要继承自已有的文件，以避免冗余代码哈，只重写修改过的method。例如这里需要继承自MuZeroModel。相应的注释也需要更新一下。

nighood and others added 16 commits June 8, 2023 21:48

env(rjy): add crowdsim env

c27ae92

config(rjy): add mz/ez config for crowdsim

c6acd7d

Merge branch 'main' into rjy-crowd-2

0d235ab

env(rjy): add crowdsim env

ad0cd02

feature(rjy): add RGCN for represent net

14542a1

feature(rjy): add obs/action env mode. fix rgcn pipeline.

dc4a774

feature(rjy): add multi-head policy(combine logits)

c99db40

feature(rjy): modify new env with transmitted data

61831f1

feature(rjy): add rough vis of crowdsim

9599faa

polish(rjy): fix new env info in collecter

15d9a44

feature(rjy): add sez mlp_multi-head

3c8804d

feature(rjy): set the environment to two modes

c6723a0

Merge branch 'rjy-crowd-md-com-sez' into rjy-crowd-md-env-new

e100fe4

feature(rjy): add ez multi-head model

c4e9d58

Merge branch 'rjy-crowd-md-com-ez' into rjy-crowd-md-env-new

f677af1

polish(rjy): add v_trans in config

cb044af

puyuan1996 added environment New or improved environment config New or improved configuration labels Jun 7, 2024

fix(rjy): fix env bug

715b5b8

puyuan1996 reviewed Jun 12, 2024

View reviewed changes

lzero/agent/efficientzero.py Show resolved Hide resolved

puyuan1996 mentioned this pull request Jun 12, 2024

feature(rjy): add crowdsim env and related configs #208

Closed

nighood and others added 4 commits June 14, 2024 15:43

feature(rjy): add entropy info/set margin

fecf5d3

Merge pull request #1 from nighood/rjy-crowd-md-env-scale

c4da015

fix(rjy): fix crowd env scale/add entropy info

polish(rjy): polish code according to comments

63d37a8

Merge branch 'rjy-crowd-md-env-new' of github.com:nighood/LightZero i…

745f0a8

…nto rjy-crowd-md-env-new

puyuan1996 requested changes Jun 25, 2024

View reviewed changes

puyuan1996 reviewed Jun 25, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feature(rjy): add crowd md env new, and multi-head policy #230

feature(rjy): add crowd md env new, and multi-head policy #230

nighood commented Jun 7, 2024 •

edited

Loading

puyuan1996 Jun 25, 2024

puyuan1996 Jun 25, 2024

puyuan1996 Jun 25, 2024

puyuan1996 Jun 25, 2024

puyuan1996 Jun 25, 2024

puyuan1996 Jun 25, 2024

puyuan1996 Jun 25, 2024

puyuan1996 Jun 25, 2024

puyuan1996 Jun 25, 2024

puyuan1996 Jun 25, 2024

puyuan1996 Jun 25, 2024

		return len(self.queue)


		# # Example of using the InformationQueue class



		@MODEL_REGISTRY.register('MuZeroModelMD')
		class MuZeroModelMD(nn.Module):

feature(rjy): add crowd md env new, and multi-head policy #230

Are you sure you want to change the base?

feature(rjy): add crowd md env new, and multi-head policy #230

Conversation

nighood commented Jun 7, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nighood commented Jun 7, 2024 •

edited

Loading