Error in "Exercise 8.4*.ipynb" --> "TypeError: dispatcher for __array_function__ did not return an iterable"

[MariosGkMeng]

Line that triggers the error:

q0, policy0, history0 = dyna_q(env, n=50, num_episodes=400, alpha=0.5, gamma=0.95)

Python version: 3.10

Complete Error Message:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In [3], line 3
      1 env = gym.make('ShortcutMaze-v0')
      2 env.seed(3); np.random.seed(3)
----> 3 q0, policy0, history0 = dyna_q(env, n=50, num_episodes=400, alpha=0.5, gamma=0.95)
      5 env = gym.make('ShortcutMaze-v0')
      6 env.seed(3); np.random.seed(3)

File c:\Users\*\reinforcement-learning-an-introduction\chapter08\dyna.py:25, in dyna_q(env, n, num_episodes, eps, alpha, gamma)
     22 model = {}
     24 for episode in range(num_episodes):
---> 25     state = env.reset()
     27     done = False
     28     while not done:
     29         # Sample action according to the current policy and step the environment forward

File ~\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\gym\wrappers\order_enforcing.py:42, in OrderEnforcing.reset(self, **kwargs)
     40 """Resets the environment with `kwargs`."""
     41 self._has_reset = True
---> 42 return self.env.reset(**kwargs)

File ~\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\gym\wrappers\env_checker.py:45, in PassiveEnvChecker.reset(self, **kwargs)
     43 if self.checked_reset is False:
     44     self.checked_reset = True
---> 45     return env_reset_passive_checker(self.env, **kwargs)
     46 else:
     47     return self.env.reset(**kwargs)

File ~\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\gym\utils\passive_env_checker.py:192, in env_reset_passive_checker(env, **kwargs)
    187     logger.warn(
    188         "Future gym versions will require that `Env.reset` can be passed `options` to allow the environment initialisation to be passed additional information."
    189     )
    191 # Checks the result of env.reset with kwargs
--> 192 result = env.reset(**kwargs)
    194 if not isinstance(result, tuple):
    195     logger.warn(
    196         f"The result returned by `env.reset()` was not a tuple of the form `(obs, info)`, where `obs` is a observation and `info` is a dictionary containing additional information. Actual type: `{type(result)}`"
    197     )

File ~\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\gym\core.py:380, in ObservationWrapper.reset(self, **kwargs)
    378 """Resets the environment, returning a modified observation using :meth:`self.observation`."""
    379 obs, info = self.env.reset(**kwargs)
--> 380 return self.observation(obs), info

File c:\Users\*\reinforcement-learning-an-introduction\chapter08\maze.py:117, in MultiDiscreteObservationWrapper.observation(self, observation)
    116 def observation(self, observation):
--> 117     flat_observation = np.ravel_multi_index(observation, self.env.observation_space.nvec)
    118     return flat_observation

File <__array_function__ internals>:180, in ravel_multi_index(*args, **kwargs)

TypeError: dispatcher for __array_function__ did not return an iterable

[vojtamolda]

Hello @MariosGkMeng! Thanks a lot for reporting the error. Unfortunately, I don't have the bandwidth to investigate and work on this now.

But if you figure out what's wrong I'll be happy to merge a PR with a fix ;)

[LSB0798]

I tried with gym==0.13.1, it's ok