continue training after crash

[vinnitu]

When I train seq2seq in tensorflow in examples (an rnn in darknet) I use backup between epochs

But I don't sure keras_spell.py save model from time to time, how can I do it in properly way?

[parth126]

@vinnitu Adding this at the end of for loop in iterate_training function should do the trick
model.save_weights('SpellModel_{0}.h5'.format(iteration))
This will save the weights after each iteration. One iteration may have more than one epoch.

If you want to save it after each epoch instead, make sure that each iteration has only one epoch.
You could also you keras callback function ModelCheckpoint (https://keras.io/callbacks/).

You will also have to save the model itself once. Add the following snippet in generate_model function.

model_json = model.to_json()
with open("SpellModel.json", "w") as json_file:
json_file.write(model_json)

Later you can load the model from this json file and then load the weights from your checkpoint.
Look https://keras.io/models/about-keras-models/ for details

[MajorTal]

I added saving the model after each epoch.

…

On Feb 28, 2017 1:48 AM, "Parth Mehta" ***@***.***> wrote: @vinnitu <https://github.com/vinnitu> Adding this at the end of for loop in iterate_training function should do the trick model.save_weights('SpellModel_{0}.h5'.format(iteration)) This will save the weights after each iteration. One iteration may have more than one epoch. If you want to save it after each epoch instead, make sure that each iteration has only one epoch. You could also you keras callback function ModelCheckpoint ( https://keras.io/callbacks/). You will also have to save the model itself once. Add the following snippet in generate_model function. model_json = model.to_json() with open("SpellModel.json", "w") as json_file: json_file.write(model_json) Later you can load the model from this json file and then load the weights from your checkpoint. Look https://keras.io/models/about-keras-models/ for details — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#8 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA9lweIUKPODvyuDdoVOuRppzyp1f8voks5rg2DJgaJpZM4LeJma> .