Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Missing warnings with pytest --forked option #63

Open
kir0ul opened this issue Feb 28, 2022 · 1 comment
Open

Missing warnings with pytest --forked option #63

kir0ul opened this issue Feb 28, 2022 · 1 comment

Comments

@kir0ul
Copy link

kir0ul commented Feb 28, 2022

Hi,

I found out in openai/gym#2647 that the pytest --forked option seems to miss to report many warnings.

With the --forked option I get only 2 warnings:

❯ docker run gym-docker pytest --forked --import-mode=append
[...]
============================= test session starts ==============================
platform linux -- Python 3.10.2, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /usr/local/gym
plugins: forked-1.4.0
collected 623 items / 5 skipped / 618 selected
[...]
=============================== warnings summary ===============================
../lib/python3.10/site-packages/gym/wrappers/monitoring/video_recorder.py:9
  /usr/local/lib/python3.10/site-packages/gym/wrappers/monitoring/video_recorder.py:9: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
    import distutils.spawn

../lib/python3.10/site-packages/gym/spaces/box.py:78
  /usr/local/lib/python3.10/site-packages/gym/spaces/box.py:78: UserWarning: WARN: Box bound precision lowered by casting to float32
    logger.warn(f"Box bound precision lowered by casting to {self.dtype}")

-- Docs: https://docs.pytest.org/en/stable/warnings.html
================= 621 passed, 7 skipped, 2 warnings in 39.42s ==================

Now without the --forked option I get ~1500 warnings:

❯ docker run gym-docker pytest --import-mode=append
[...]
============================= test session starts ==============================
platform linux -- Python 3.10.2, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /usr/local/gym
plugins: forked-1.4.0
collected 623 items / 5 skipped / 618 selected
[...]
=============================== warnings summary ===============================
../lib/python3.10/site-packages/gym/wrappers/monitoring/video_recorder.py:9
  /usr/local/lib/python3.10/site-packages/gym/wrappers/monitoring/video_recorder.py:9: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
    import distutils.spawn

../lib/python3.10/site-packages/gym/spaces/box.py:78
tests/wrappers/test_normalize.py::test_normalize_observation[False]
tests/wrappers/test_normalize.py::test_normalize_observation[True]
tests/wrappers/test_normalize.py::test_normalize_reset_info
tests/wrappers/test_normalize.py::test_normalize_return
tests/wrappers/test_normalize.py::test_normalize_observation_vector_env
tests/wrappers/test_normalize.py::test_normalize_return_vector_env
  /usr/local/lib/python3.10/site-packages/gym/spaces/box.py:78: UserWarning: WARN: Box bound precision lowered by casting to float32
    logger.warn(f"Box bound precision lowered by casting to {self.dtype}")

tests/envs/test_determinism.py: 80 warnings
tests/envs/test_envs.py: 137 warnings
tests/spaces/test_spaces.py: 266 warnings
tests/spaces/test_utils.py: 176 warnings
tests/utils/test_env_checker.py: 2 warnings
tests/vector/test_async_vector_env.py: 16 warnings
tests/vector/test_numpy_utils.py: 64 warnings
tests/vector/test_shared_memory.py: 128 warnings
tests/vector/test_sync_vector_env.py: 8 warnings
tests/wrappers/test_normalize.py: 6 warnings
tests/wrappers/test_record_episode_statistics.py: 181 warnings
tests/wrappers/test_record_video.py: 398 warnings
tests/wrappers/test_time_aware_observation.py: 2 warnings
tests/wrappers/test_transform_observation.py: 1 warning
tests/wrappers/test_transform_reward.py: 14 warnings
  /usr/local/lib/python3.10/site-packages/gym/utils/seeding.py:47: DeprecationWarning: WARN: Function `rng.randint(low, [high, size, dtype])` is marked as deprecated and will be removed in the future. Please use `rng.integers(low, [high, size, dtype])` instead.
    deprecation(

tests/envs/test_envs.py::test_env[spec1]
tests/envs/test_envs.py::test_env[spec2]
tests/envs/test_envs.py::test_env[spec10]
tests/envs/test_envs.py::test_env[spec11]
  /usr/local/lib/python3.10/site-packages/gym/utils/env_checker.py:116: UserWarning: WARN: Agent's minimum observation space value is -infinity. This is probably too low.
    logger.warn(

tests/envs/test_envs.py::test_env[spec1]
tests/envs/test_envs.py::test_env[spec2]
tests/envs/test_envs.py::test_env[spec10]
tests/envs/test_envs.py::test_env[spec11]
  /usr/local/lib/python3.10/site-packages/gym/utils/env_checker.py:120: UserWarning: WARN: Agent's maxmimum observation space value is infinity. This is probably too high
    logger.warn(

tests/envs/test_envs.py::test_env[spec4]
tests/envs/test_envs.py::test_env[spec14]
  /usr/local/lib/python3.10/site-packages/gym/utils/env_checker.py:162: UserWarning: WARN: We recommend you to use a symmetric and normalized Box action space (range=[-1, 1]) cf https://stable-baselines3.readthedocs.io/en/master/guide/rl_tips.html
    logger.warn(

tests/envs/test_envs.py::test_env[spec14]
tests/envs/test_envs.py::test_env[spec14]
tests/envs/test_envs.py::test_env[spec14]
tests/envs/test_envs.py::test_env[spec14]
  /usr/local/lib/python3.10/site-packages/gym/envs/classic_control/pendulum.py:183: DeprecationWarning: In future, it will be an error for 'np.bool_' scalars to be interpreted as an index
    scale_img = pygame.transform.flip(scale_img, is_flip, True)

tests/spaces/test_spaces.py::test_box_dtype_check
tests/vector/test_spaces.py::test_iterate[Box0]
tests/vector/test_spaces.py::test_iterate[Box4]
tests/vector/test_spaces.py::test_iterate[Dict1]
tests/vector/test_spaces.py::test_iterate_custom_space[Tuple]
  /usr/local/lib/python3.10/site-packages/gym/spaces/box.py:153: UserWarning: WARN: Casting input x to numpy array.    logger.warn("Casting input x to numpy array.")

tests/vector/test_async_vector_env.py::test_custom_space_async_vector_env_shared_memory
  /usr/local/lib/python3.10/site-packages/_pytest/unraisableexception.py:78: PytestUnraisableExceptionWarning: Exception ignored in: <function AsyncVectorEnv.__del__ at 0x7f0366be9ea0>
  
  Traceback (most recent call last):
    File "/usr/local/lib/python3.10/site-packages/gym/vector/async_vector_env.py", line 631, in __del__
      self.close(terminate=True)
    File "/usr/local/lib/python3.10/site-packages/gym/vector/vector_env.py", line 193, in close
      self.close_extras(**kwargs)
    File "/usr/local/lib/python3.10/site-packages/gym/vector/async_vector_env.py", line 543, in close_extras
      if self._state != AsyncState.DEFAULT:
  AttributeError: 'AsyncVectorEnv' object has no attribute '_state'
  
    warnings.warn(pytest.PytestUnraisableExceptionWarning(msg))

tests/wrappers/test_record_video.py::test_record_video_reset_return_info
  /usr/local/lib/python3.10/site-packages/_pytest/unraisableexception.py:78: PytestUnraisableExceptionWarning: Exception ignored in: <function VideoRecorder.__del__ at 0x7f0366c06b90>
  
  Traceback (most recent call last):
    File "/usr/local/lib/python3.10/site-packages/gym/wrappers/monitoring/video_recorder.py", line 193, in __del__
      self.close()
    File "/usr/local/lib/python3.10/site-packages/gym/wrappers/monitoring/video_recorder.py", line 182, in close
      self.write_metadata()
    File "/usr/local/lib/python3.10/site-packages/gym/wrappers/monitoring/video_recorder.py", line 188, in write_metadata
      with open(self.metadata_path, "w") as f:
  FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/gym/videos/rl-video-step-0.meta.json'
  
    warnings.warn(pytest.PytestUnraisableExceptionWarning(msg))

tests/wrappers/test_video_recorder.py::test_record_breaking_render_method
  /usr/local/lib/python3.10/site-packages/gym/wrappers/monitoring/video_recorder.py:136: UserWarning: WARN: Env returned None on render(). Disabling further rendering for video recorder by marking as disabled: path=/tmp/tmpqrycjchl.mp4 metadata_path=/tmp/tmpqrycjchl.meta.json
    logger.warn(

-- Docs: https://docs.pytest.org/en/stable/warnings.html
================ 621 passed, 7 skipped, 1509 warnings in 21.03s ================

Steps to reproduce:

git clone [email protected]:openai/gym.git
cd gym
wget https://www.roboti.us/file/mjkey.txt
docker build -f py.Dockerfile --build-arg MUJOCO_KEY=./mjkey.txt --build-arg PYTHON_VERSION=3.10 --tag gym-docker .

Then run either docker run gym-docker pytest --forked --import-mode=append or docker run gym-docker pytest --import-mode=append

Let me know if I can do anything else to help troubleshoot!
@RonnyPfannschmidt
Copy link
Member

This is unlikely to be fixed unless someone takes over maintenance

@kir0ul kir0ul mentioned this issue Apr 6, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@RonnyPfannschmidt @kir0ul