Bug report: dqn_atari.py can't run with your pretrained weights using TensorFlow

[allengee]

I think this bug is related to the one I reported previously, which you fixed (Thanks!).
Were your pretrained weights trained using Theano? Can they be made run under TensorFlow?
I get a ValueError exception when I run the following:
python dqn_atari.py --mode test --weights pretrained_weights/dqn_Breakout-v0_weights.h5f

Traceback (most recent call last):
File "dqn_atari.py", line 136, in
dqn.load_weights(weights_filename)
File "/home/allen/Tutorials/KerasRL/Doc/rl/agents/dqn.py", line 93, in load_weights
self.model.load_weights(filepath)
File "/home/allen/anaconda2/lib/python2.7/site-packages/keras/engine/topology.py", line 2500, in load_weights
self.load_weights_from_hdf5_group(f)
File "/home/allen/anaconda2/lib/python2.7/site-packages/keras/engine/topology.py", line 2585, in load_weights_from_hdf5_group
K.batch_set_value(weight_value_tuples)
File "/home/allen/anaconda2/lib/python2.7/site-packages/keras/backend/tensorflow_backend.py", line 963, in batch_set_value
assign_op = x.assign(assign_placeholder)
File "/home/allen/anaconda2/lib/python2.7/site-packages/tensorflow/python/ops/variables.py", line 501, in assign
return state_ops.assign(self._variable, value, use_locking=use_locking)
File "/home/allen/anaconda2/lib/python2.7/site-packages/tensorflow/python/ops/gen_state_ops.py", line 45, in assign
use_locking=use_locking, name=name)
File "/home/allen/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 703, in apply_op
op_def=op_def)
File "/home/allen/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2319, in create_op
set_shapes_for_outputs(ret)
File "/home/allen/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1711, in set_shapes_for_outputs
shapes = shape_func(op)
File "/home/allen/anaconda2/lib/python2.7/site-packages/tensorflow/python/ops/state_ops.py", line 210, in _AssignShape
return [op.inputs[0].get_shape().merge_with(op.inputs[1].get_shape())]
File "/home/allen/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/tensor_shape.py", line 570, in merge_with
(self, other))
ValueError: Shapes (8, 8, 4, 32) and (32, 4, 8, 8) are not compatible

[matthiasplappert]

The weights were obtained using Theano, which means that you'll have to convert them if you intend to use them with TensorFlow. Please refer to the Keras documentation, which describes how to do this: https://github.com/fchollet/keras/wiki/Converting-convolution-kernels-from-Theano-to-TensorFlow-and-vice-versa. There's also a (I think relatively new) utility method to convert an entire model: https://github.com/fchollet/keras/blob/99f564e9724d3456fc5b48b9dcd6254c83611242/keras/utils/layer_utils.py#L103

This should allow you to use the weights with your TensorFlow backend. Let me know if it doesn't work.

[kimardenmiller]

I'm using the Theano backend, but also getting an issue with the pre-trained weights.

 Traceback (most recent call last):
   File "/Users/kimardenmiller/Dropbox/PyCharm/Deep-QNetwork_gaming/DQN_v.2/Breakout.py", line 45, in <module>
     dqn.load_weights('dqn_{}_weights_750000.h5f'.format(ENV_NAME))
   File "/Users/kimardenmiller/anaconda/envs/tensorflow/lib/python3.5/site-packages/rl/agents/dqn.py", line 194, in load_weights
     self.model.load_weights(filepath)
   File "/Users/kimardenmiller/anaconda/envs/tensorflow/lib/python3.5/site-packages/keras/engine/topology.py", line 2520, in load_weights
     self.load_weights_from_hdf5_group(f)
   File "/Users/kimardenmiller/anaconda/envs/tensorflow/lib/python3.5/site-packages/keras/engine/topology.py", line 2572, in load_weights_from_hdf5_group
     str(len(flattened_layers)) + ' layers.')
 Exception: You are trying to load a weight file containing 5 layers into a model with 4 layers.

Seems to train fine starting from no pre-trained weights.