New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Adds Factory contact-rich manipulation tasks to IsaacLab #1520

Merged

kellyguo11 merged 16 commits into isaac-sim:main from noseworm:factory

Dec 17, 2024

Contributor

noseworm commented Dec 10, 2024 •

edited by kellyguo11

Loading

Description

This MR adds new tasks for contact-rich manipulation based on the Factory line of work. Tasks include peg insertion, gear meshing, and nut threading.

Type of change

New feature (non-breaking change which adds functionality)
This change requires a documentation update

Screenshots

peg_insertion

gear_meshing

nut_threading

Checklist

I have run the pre-commit checks with ./isaaclab.sh --format
I have made corresponding changes to the documentation
My changes generate no new warnings
I have added tests that prove my fix is effective or that my feature works
I have updated the changelog and the corresponding version in the extension's config/extension.toml file
I have added my name to the CONTRIBUTORS.md or my name already exists there

noseworm and others added 7 commits

December 8, 2024 11:02


          Initial factory commit

3e6537d


          Use table already in omniverse

e33bea0


          Change assets to articulations

b40062f


          adding contact-rich environments (from Factory) to documentation

fe76be3


          Run pre-commit formatting

a287a99


          Update docs and changelog

f500931


          Add contributors for Factory envs

79f5544

noseworm marked this pull request as ready for review

December 10, 2024 15:44

noseworm requested review from jsmith-bdai, Dhoeller19, kellyguo11 and Mayankm96 as code owners

December 10, 2024 15:44

noseworm added 2 commits

December 11, 2024 08:18


          Fix indentation of player config

2aa8aa5


          Register each contact-rich task using a separate configclass

e81909a

noseworm commented

View reviewed changes

source/standalone/workflows/rl_games/play.py

@@ @@ -155,7 +155,7 @@ def main(): @@
                           # convert obs to agent format
                           obs = agent.obs_to_torch(obs)
                           # agent stepping
-                          actions = agent.get_action(obs, is_deterministic=True)
+                          actions = agent.get_action(obs, is_deterministic=agent.is_deterministic)

Contributor Author

noseworm Dec 11, 2024

For Factory environments, we evaluate policies stochastically.

agent.is_deterministic is specified in the rl_games_ppo_cfg.yaml under params.config.player.deterministic. If this is not specified, the default value will be True -- as such, this change should not affect other environments.

kbkartik commented Dec 12, 2024

Hi noseworm,

newbie to sim2real world. I am following this PR and had two questions:

For commit b40062f, why have you changed Cfg for rigid bodies from RigidObjectCfg to ArticulationCfg?
Where do I get the usd files for the rigid objects?

Thanks!


          Update ASSET_DIR for Factory assets

3fdd2de

kellyguo11 changed the title ~~Add Factory contact-rich manipulation tasks to IsaacLab~~ Adds Factory contact-rich manipulation tasks to IsaacLab

kellyguo11 reviewed

View reviewed changes

Contributor

kellyguo11 left a comment

Could you please also add the environment names into this table? https://isaac-sim.github.io/IsaacLab/main/source/overview/environments.html#comprehensive-list-of-environments

docs/source/overview/environments.rst Outdated Show resolved Hide resolved

source/extensions/omni.isaac.lab_tasks/docs/CHANGELOG.rst

+              * Added ``Factory-Direct-v0`` environment as a direct RL env that
+                implements contact-rich manipulation tasks including peg insertion,
+                gear meshing, and nut threading.

Contributor

kellyguo11 Dec 12, 2024

Suggested change

source/extensions/omni.isaac.lab_tasks/omni/isaac/lab_tasks/direct/factory/__init__.py Outdated Show resolved Hide resolved

source/extensions/omni.isaac.lab_tasks/omni/isaac/lab_tasks/direct/factory/factory_control.py Outdated

		return translated_pos


		def axis_angle_from_euler(euler):

Contributor

kellyguo11 Dec 12, 2024

I wonder if these can become more general utilities? We also have a bunch of rotation utilities in omni/isaac/lab/utils/math.py, there are likely some duplicates here with what's already in math.py. would be good to integrate them together.

Contributor Author

noseworm Dec 16, 2024

There are also several methods here that we are not using. I will remove those and use the methods from math.py when available.

source/extensions/omni.isaac.lab_tasks/omni/isaac/lab_tasks/direct/factory/factory_env.py Outdated Show resolved Hide resolved

source/extensions/omni.isaac.lab_tasks/omni/isaac/lab_tasks/direct/factory/factory_env.py Outdated Show resolved Hide resolved

source/extensions/omni.isaac.lab_tasks/omni/isaac/lab_tasks/direct/factory/factory_env.py Outdated Show resolved Hide resolved

source/extensions/omni.isaac.lab_tasks/omni/isaac/lab_tasks/direct/factory/torch_jit_utils.py Outdated Show resolved Hide resolved

source/extensions/omni.isaac.lab_tasks/omni/isaac/lab_tasks/direct/factory/factory_env.py Show resolved Hide resolved

This comment was marked as duplicate.

Sign in to view

kbkartik reviewed

View reviewed changes

source/extensions/omni.isaac.lab_tasks/omni/isaac/lab_tasks/direct/factory/factory_tasks_cfg.py Outdated

kbkartik Dec 13, 2024

Supposed to be ISAAC_NUCLEUS_DIR instead of ISAACLAB_NUCLEUS_DIR?

Contributor Author

noseworm Dec 16, 2024

ISAACLAB_NUCLEUS_DIR should be correct. This is equivalent to ISAAC_NUCLEUS_DIR/IsaacLab.

kellyguo11 reviewed

View reviewed changes

source/extensions/omni.isaac.lab_tasks/omni/isaac/lab_tasks/direct/factory/factory_env.py Outdated

+                          bad_envs = bad_envs[any_error.nonzero(as_tuple=False).squeeze(-1)]
+                          if bad_envs.shape[0] == 0:
+                              print("Done IK")

Contributor

kellyguo11 Dec 13, 2024

may be cleaner to remove this?

Contributor Author

noseworm Dec 16, 2024

Removed print statement.

kellyguo11 reviewed

View reviewed changes

source/extensions/omni.isaac.lab_tasks/omni/isaac/lab_tasks/direct/factory/factory_env.py Outdated

+                      self._set_gains(self.default_gains)
+                      physics_sim_view.set_gravity(carb.Float3(*self.cfg.sim.gravity))
+                      print("Done Reset")

Contributor

kellyguo11 Dec 13, 2024

might be good to remove this as well

Contributor Author

noseworm Dec 16, 2024

Removed the print statement.

kellyguo11 reviewed

View reviewed changes

source/extensions/omni.isaac.lab_tasks/omni/isaac/lab_tasks/direct/factory/factory_env.py Outdated Show resolved Hide resolved

noseworm and others added 5 commits

December 16, 2024 13:08


          Update from code review

66c912a

- Isaac- prefix added to task names
- Documentation updated with jpg images
- Use math.py rotation utilities
- Misc code cleanup


          Update comprehensive environment list

f01fa94


          Merge branch 'main' into factory

09b0902

Signed-off-by: Kelly Guo <[email protected]>


          Update extension.toml

59cdba1

Signed-off-by: Kelly Guo <[email protected]>


          Update per_test_timeouts.py

a2bc641

Signed-off-by: Kelly Guo <[email protected]>

kellyguo11 requested a review from hhansen-bdai as a code owner

December 16, 2024 22:42


          Update per_test_timeouts.py

ce0f385

Signed-off-by: Kelly Guo <[email protected]>

kellyguo11 merged commit 3c55db4 into isaac-sim:main

4 of 5 checks passed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

kbkartik kbkartik left review comments

kellyguo11 kellyguo11 left review comments

jsmith-bdai Awaiting requested review from jsmith-bdai jsmith-bdai is a code owner

Dhoeller19 Awaiting requested review from Dhoeller19 Dhoeller19 is a code owner

Mayankm96 Awaiting requested review from Mayankm96 Mayankm96 is a code owner

hhansen-bdai Awaiting requested review from hhansen-bdai hhansen-bdai is a code owner

Labels

None yet