How to solve Graph execution error during my model training?

Md Mahadi Hasan Sany

I build the inception v3 model from scratch. When I tried to train my model with a popular data set, dogs and cats from Kaggle, I faced Graph execution error. The reason might be my data set; I guess some data are not in the RGB scale, or it might be other causes.

Data Set

Model compile

model.compile(optimizer=Adam(learning_rate=0.0001),loss = 'categorical_crossentropy', metrics= ['accuracy'])

Data Preprocess

train_batches = ImageDataGenerator(preprocessing_function=tf.keras.applications.inception_v3.preprocess_input) \
    .flow_from_directory(directory=trin_path, target_size=(299,299), classes=['dogs', 'cats'], batch_size=10)
valid_batches = ImageDataGenerator(preprocessing_function=tf.keras.applications.inception_v3.preprocess_input) \
    .flow_from_directory(directory=valid_path, target_size=(299,299), classes=['dogs', 'cats'], batch_size=10)
test_batches = ImageDataGenerator(preprocessing_function=tf.keras.applications.inception_v3.preprocess_input) \
     .flow_from_directory(directory=test_path, target_size=(299,299), classes=['dogs', 'cats'], batch_size=10, shuffle=False)

Model Fit

r = model.fit(x=train_batches, validation_data=valid_batches, epochs=5)

The Error Occur

Epoch 1/5
 72/500 [===>..........................] - ETA: 1:14 - loss: 0.6929 - accuracy: 0.6042/usr/local/lib/python3.7/dist-packages/PIL/TiffImagePlugin.py:770: UserWarning: Possibly corrupt EXIF data.  Expecting to read 32 bytes but only got 0. Skipping tag 270
  " Skipping tag %s" % (size, len(data), tag)
/usr/local/lib/python3.7/dist-packages/PIL/TiffImagePlugin.py:770: UserWarning: Possibly corrupt EXIF data.  Expecting to read 5 bytes but only got 0. Skipping tag 271
  " Skipping tag %s" % (size, len(data), tag)
/usr/local/lib/python3.7/dist-packages/PIL/TiffImagePlugin.py:770: UserWarning: Possibly corrupt EXIF data.  Expecting to read 8 bytes but only got 0. Skipping tag 272
  " Skipping tag %s" % (size, len(data), tag)
/usr/local/lib/python3.7/dist-packages/PIL/TiffImagePlugin.py:770: UserWarning: Possibly corrupt EXIF data.  Expecting to read 8 bytes but only got 0. Skipping tag 282
  " Skipping tag %s" % (size, len(data), tag)
/usr/local/lib/python3.7/dist-packages/PIL/TiffImagePlugin.py:770: UserWarning: Possibly corrupt EXIF data.  Expecting to read 8 bytes but only got 0. Skipping tag 283
  " Skipping tag %s" % (size, len(data), tag)
/usr/local/lib/python3.7/dist-packages/PIL/TiffImagePlugin.py:770: UserWarning: Possibly corrupt EXIF data.  Expecting to read 20 bytes but only got 0. Skipping tag 306
  " Skipping tag %s" % (size, len(data), tag)
/usr/local/lib/python3.7/dist-packages/PIL/TiffImagePlugin.py:770: UserWarning: Possibly corrupt EXIF data.  Expecting to read 48 bytes but only got 0. Skipping tag 532
  " Skipping tag %s" % (size, len(data), tag)
/usr/local/lib/python3.7/dist-packages/PIL/TiffImagePlugin.py:788: UserWarning: Corrupt EXIF data.  Expecting to read 2 bytes but only got 0. 
  warnings.warn(str(msg))
500/500 [==============================] - ETA: 0s - loss: 0.6609 - accuracy: 0.6318
---------------------------------------------------------------------------
UnknownError                              Traceback (most recent call last)
<ipython-input-90-bd0e48768399> in <module>()
----> 1 r = model.fit(x=train_batches,validation_data=valid_batches,epochs=5)

1 frames
/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/execute.py in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name)
     53     ctx.ensure_initialized()
     54     tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
---> 55                                         inputs, attrs, num_outputs)
     56   except core._NotOkStatusException as e:
     57     if name is not None:

UnknownError: Graph execution error:

2 root error(s) found.
  (0) UNKNOWN:  UnidentifiedImageError: cannot identify image file <_io.BytesIO object at 0x7fb8672df290>
Traceback (most recent call last):

  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/script_ops.py", line 271, in __call__
    ret = func(*args)

  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/autograph/impl/api.py", line 642, in wrapper
    return func(*args, **kwargs)

  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/data/ops/dataset_ops.py", line 1004, in generator_py_func
    values = next(generator_state.get_iterator(iterator_id))

  File "/usr/local/lib/python3.7/dist-packages/keras/engine/data_adapter.py", line 830, in wrapped_generator
    for data in generator_fn():

  File "/usr/local/lib/python3.7/dist-packages/keras/engine/data_adapter.py", line 956, in generator_fn
    yield x[i]

  File "/usr/local/lib/python3.7/dist-packages/keras_preprocessing/image/iterator.py", line 65, in __getitem__
    return self._get_batches_of_transformed_samples(index_array)

  File "/usr/local/lib/python3.7/dist-packages/keras_preprocessing/image/iterator.py", line 230, in _get_batches_of_transformed_samples
    interpolation=self.interpolation)

  File "/usr/local/lib/python3.7/dist-packages/keras_preprocessing/image/utils.py", line 114, in load_img
    img = pil_image.open(io.BytesIO(f.read()))

  File "/usr/local/lib/python3.7/dist-packages/PIL/Image.py", line 2896, in open
    "cannot identify image file %r" % (filename if filename else fp)

PIL.UnidentifiedImageError: cannot identify image file <_io.BytesIO object at 0x7fb8672df290>


     [[{{node PyFunc}}]]
     [[IteratorGetNext]]
  (1) CANCELLED:  Function was cancelled before it was started
0 successful operations.
0 derived errors ignored. [Op:__inference_test_function_69081]
vscv

The EXIF data of some images are corrupt, so PIL reading the images fails during training.

Methods may fix it:

  1. PIL : Please remove the damaged images from the training set first, for example, PetImages/Cat/666.jpg, and PetImages/Dog/11702.jpg. Please also refer to https://www.kaggle.com/code/sushovansaha9/cat-dog-classification-transferlearning-ipynb.

  2. PIL : remove the EXIF data but keep the images.

import piexif  
piexif.remove(filename)
  1. tf.io : just read the image without remove any images or EXIF:
file = tf.io.read_file(filename)
image = tf.image.decode_jpeg(file)

Collected from the Internet

Please contact [email protected] to delete if infringement.

edited at
0

Comments

0 comments
Login to comment

Related

How to solve this django OneToOneField model error?

BigQuery: How to Avoid "Resources exceeded during query execution." error

Google ML Engine: Prediction failed: Error during model execution

solving graph disconnected error during making a model of autoencoder

how to merge my training and validation accuracy in one graph using tensorboard

how to solve LinAlgError & ValueError when training arima model with Python

How to convert model in eager execution to static graph and save in .pb file?

How to save and snapshot machine learning model during a single training?

How to solve " NameError: name 'model' is not defined " error?

How to solve the error with deploying a model in aws sagemaker?

How to solve error during include of component in Adobe Experience Manager?

How can i solve my error during importing pygame

How to solve this error "setState() or markNeedsBuild() called during build."?

How to splitting training data into smaller batches to solve memory error

How do I solve the BSOD that appears during my Windows install?

How to solve Execution Error in hive?

How to solve runtime error "No entry could be found" during Tycho?

How to solve the unexpected token error in my jsx?

No error during debug but error during normal execution

Polymorphic type error during recursion - how to solve?

how can i solve java fatal error during jframe

How to pad sequences during training for an encoder decoder model

How to solve the sql execution error in snowsql client?

ERROR: Error during SonarScanner execution

How to fix Memory Error while training model?

Spacy - problem during the training of my model, it seems to block at epoch 0

How to display loading message during the execution of my function?

how can reslove : InvalidArgumentError: Graph execution error?

During training the TensorFlow model(!!Not the Keras model), How to get the input and output of the intermediate layer(op) of the model?