ValueError: Layer #0 (named "efficientnetb0" in the current model) was found to correspond to layer efficientnet-b0 in the save file. However the new layer efficientnetb0 expects 312 weights, but the saved weights have 309 elements.

[ghost]

Hi, I am trying to train on a TPU with Kaggle, save the weights, and then predict on a GPU. The problem is that whenever I save my model's weights and try to load them up again I get an error like this:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-30-44f514f0c1e1> in <module>
      4     test_ids_ds = TEST.map(lambda image, idnum: idnum).unbatch()
      5 
----> 6     models = [get_model(model, EFFICIENTNET_MODELS[model]) for model in EFFICIENTNET_MODELS]
      7     predictions = [model.predict(test_images_ds) for model in models]
      8     results = pd.DataFrame(sum(predictions) / len(predictions))

<ipython-input-30-44f514f0c1e1> in <listcomp>(.0)
      4     test_ids_ds = TEST.map(lambda image, idnum: idnum).unbatch()
      5 
----> 6     models = [get_model(model, EFFICIENTNET_MODELS[model]) for model in EFFICIENTNET_MODELS]
      7     predictions = [model.predict(test_images_ds) for model in models]
      8     results = pd.DataFrame(sum(predictions) / len(predictions))

<ipython-input-29-6bdf6c1e54c8> in get_model(model_name, weight_path)
    100                           metrics=[tf.keras.metrics.AUC(name='AUC', multi_label=True)])
    101 
--> 102             model.load_weights(weight_path)
    103 
    104             return model

/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py in load_weights(self, filepath, by_name, skip_mismatch, options)
   2209             f, self.layers, skip_mismatch=skip_mismatch)
   2210       else:
-> 2211         hdf5_format.load_weights_from_hdf5_group(f, self.layers)
   2212 
   2213   def _updated_config(self):

/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/saving/hdf5_format.py in load_weights_from_hdf5_group(f, layers)
    704                        str(len(symbolic_weights)) +
    705                        ' weights, but the saved weights have ' +
--> 706                        str(len(weight_values)) + ' elements.')
    707     weight_value_tuples += zip(symbolic_weights, weight_values)
    708   K.batch_set_value(weight_value_tuples)

ValueError: Layer #0 (named "efficientnetb0" in the current model) was found to correspond to layer efficientnet-b0 in the save file. However the new layer efficientnetb0 expects 312 weights, but the saved weights have 309 elements.

I have tried using another EfficientNet model to see if it was just the one set of weights, but I get the same error, with an expected amount of weights being 3 more than provided in the weight file. Here is the code I use to load my weights:

EFFICIENTNET_MODELS = {'b0': '../input/ranzcr-weights/efficientnetb0.h5',
                                                'b1': '../input/ranzcr-weights/efficientnetb1.h5',
                                                'b2': '../input/ranzcr-weights/efficientnetb2.h5',
                                                'b3': '../input/ranzcr-weights/efficientnetb3.h5',
                                                'b4': '../input/ranzcr-weights/efficientnetb4.h5',
                                                'b5': '../input/ranzcr-weights/efficientnetb5.h5'} 

def get_model(model_name, weight_path):
        with strategy.scope():
            if model_name == 'b0':
                EfficientNet = tf.keras.applications.EfficientNetB0(
                    include_top=False,
                    weights=None,
                    input_shape=(*IMG_SIZE,3)
                )
            if model_name == 'b1':
                EfficientNet = tf.keras.applications.EfficientNetB1(
                    include_top=False,
                    weights=None,
                    input_shape=(*IMG_SIZE,3)
                )
            if model_name == 'b2':
                EfficientNet = tf.keras.applications.EfficientNetB2(
                    include_top=False,
                    weights=None,
                    input_shape=(*IMG_SIZE,3)
                )
            if model_name == 'b3':
                EfficientNet = tf.keras.applications.EfficientNetB3(
                    include_top=False,
                    weights=None,
                    input_shape=(*IMG_SIZE,3)
                )
            if model_name == 'b4':
                EfficientNet = tf.keras.applications.EfficientNetB4(
                    include_top=False,
                    weights=None,
                    input_shape=(*IMG_SIZE,3)
                )
            if model_name == 'b5':
                EfficientNet = tf.keras.applications.EfficientNetB5(
                    include_top=False,
                    weights=None,
                    input_shape=(*IMG_SIZE,3)
                )

            EfficientNet.trainable = True

            model = Sequential([EfficientNet,
                                Flatten(),
                                Dense(11, activation='sigmoid')])

            model.compile(optimizer='adam',
                          loss='binary_crossentropy',
                          metrics=[tf.keras.metrics.AUC(name='AUC', multi_label=True)])

            model.load_weights(weight_path)

            return model

models = [get_model(model, EFFICIENTNET_MODELS[model]) for model in EFFICIENTNET_MODELS]

For saving the weights, I simply set up the model in the same way and then train and save the weights. Is this issue in my code or is it an issue with EfficientNet? How can I fix this?

[king398]

Don't save the weights, I had the same problem. Also Don`t save it in .h5 format also. Save the model as a tensorflow checkpoint model.

model.save("path",include_optimizer=False)