如何使用TF2.0中的内置Keras生成CNN热图(tf.keras)

马蒂亚斯

我曾经基于TensorFlow 1之上的独立Keras库为我的卷积神经网络生成热图。这很好,但是,当我切换到TF2.0和内置tf.keras实现(渴望执行)之后,我无法不再使用我以前的热图生成代码。

因此,我为TF2.0重新编写了部分代码,并得到以下结果:

from tensorflow.keras.applications.vgg16 import preprocess_input
from tensorflow.keras.preprocessing.image import load_img
from tensorflow.keras.models import load_model

from tensorflow.keras import preprocessing
from tensorflow.keras import backend as K
from tensorflow.keras import models

import matplotlib.pyplot as plt
import tensorflow as tf
import numpy as np

image_size = 150
image_path = "/tmp/images/test-image.jpg"
model_path = "/tmp/models/prototype/basic_vgg16.h5"

# Load pre-trained Keras model and the image to classify
model = load_model(model_path)  # VGG16 CNN with custom classifier head
image = load_img(image_path, target_size=(image_size, image_size))
img_tensor = preprocessing.image.img_to_array(image)
img_tensor = np.expand_dims(img_tensor, axis=0)
img_tensor = preprocess_input(img_tensor)

input_layer = model.get_layer("model_input")
conv_layer = model.get_layer("block5_conv3")
heatmap_model = models.Model([model.inputs], [conv_layer.output, model.output])

# Get gradient of the winner class w.r.t. the output of the (last) conv. layer
with tf.GradientTape() as gtape:
    conv_output, predictions = heatmap_model(img_tensor)
    loss = predictions[:, np.argmax(predictions[0])]
    grads = gtape.gradient(loss, conv_output)
    pooled_grads = K.mean(grads, axis=(0, 1, 2))

# Get values of pooled grads and model conv. layer output as Numpy arrays
iterate = K.function([model.inputs], [pooled_grads, conv_layer.output[0]])
pooled_grads_value, conv_layer_output_value = iterate([img_tensor])

# Multiply each channel in the feature-map array by "how important it is"
for i in range(pooled_grads_value.shape[0]):
    conv_layer_output_value[:, :, i] *= pooled_grads_value[i]

# Channel-wise mean of resulting feature-map is the heatmap of class activation
heatmap = np.mean(conv_layer_output_value, axis=-1)
heatmap = np.maximum(heatmap, 0)
max_heat = np.max(heatmap)
if max_heat == 0:
    max_heat = 1e-10
heatmap /= max_heat

# Render heatmap via pyplot
plt.matshow(heatmap)
plt.show()

但是现在以下行:

iterate = K.function([model.inputs], [pooled_grads, conv_layer.output[0]])

导致此错误消息:

AttributeError: Tensor.op is meaningless when eager execution is enabled.

我一直使用Keras,没有直接与TF配合使用,因此我在这里迷失了方向。
有什么想法可能是这里的问题吗?


PS:如果您想使用此代码,则可以创建基于VGG16的模型,如下所示:

# Create Keras model from pre-trained VGG16 and custom classifier
input_layer = layers.Input(shape=(image_size, image_size, 3), name="model_input")
vgg16_model = VGG16(weights="imagenet", include_top=False, input_tensor=input_layer)
model_head = vgg16_model.output
model_head = layers.Flatten(name="model_head_flatten")(model_head)
model_head = layers.Dense(256, activation="relu")(model_head)
model_head = layers.Dense(3, activation="softmax")(model_head)
model = models.Model(inputs=input_layer, outputs=model_head)
model.compile(loss="categorical_crossentropy", optimizer=optimizers.Adam(), metrics=["accuracy"])
拉斐尔·默德克(Raphael Meudec)

在结束GradientTape循环,conv_outputgrads已持有的价值。迭代函数不再需要计算值。

下面的工作示例:

from tensorflow.keras.applications.vgg16 import preprocess_input
from tensorflow.keras.preprocessing.image import load_img
from tensorflow.keras.models import load_model

from tensorflow.keras import preprocessing
from tensorflow.keras import backend as K
from tensorflow.keras import models

import tensorflow as tf
import numpy as np

image_size = 224

# Load pre-trained Keras model and the image to classify
model = tf.keras.applications.vgg16.VGG16()
image = np.random.random((image_size, image_size, 3))
img_tensor = preprocessing.image.img_to_array(image)
img_tensor = np.expand_dims(img_tensor, axis=0)
img_tensor = preprocess_input(img_tensor)

conv_layer = model.get_layer("block5_conv3")
heatmap_model = models.Model([model.inputs], [conv_layer.output, model.output])

# Get gradient of the winner class w.r.t. the output of the (last) conv. layer
with tf.GradientTape() as gtape:
    conv_output, predictions = heatmap_model(img_tensor)
    loss = predictions[:, np.argmax(predictions[0])]
    grads = gtape.gradient(loss, conv_output)
    pooled_grads = K.mean(grads, axis=(0, 1, 2))

heatmap = tf.reduce_mean(tf.multiply(pooled_grads, conv_output), axis=-1)
heatmap = np.maximum(heatmap, 0)
max_heat = np.max(heatmap)
if max_heat == 0:
    max_heat = 1e-10
heatmap /= max_heat

print(heatmap.shape)

本文收集自互联网,转载请注明来源。

如有侵权,请联系 [email protected] 删除。

编辑于
0

我来说两句

0 条评论
登录 后参与评论

相关文章

TF2 / Keras切片张量使用[:,:,0]

如何在TF2中的Keras Lambda层中包装冻结的Tensoflow图?

在TF1训练中使用的Numpy数组-在TF2中,Keras的准确性低得多

未添加 TF2 指标的 Keras

更改中间激活/输出并观察TF2 / Keras中的预测

(tf2 / tf.keras)当一个模型包含另一个模型时,#参数如何减少

TF2 中的 Keras 实现是否支持本地 Keras 可以用 TF1 做的所有事情?

自定义学习率调度器TF2和Keras

在Keras / TF CNN中输入数字数组而不是图像

如何在Tensorflow中从tf.keras导入keras?

如何在tf.data.Dataset生成器中使用tf.keras模型?

在具有多个Keras模型的TF2自定义训练循环中应用渐变的正确方法

keras multi_gpu_model saved_model 未能在 TF2 代码中加载模型

TensorFlow 2-tf.keras:如何使用tf.data API和TFRecords训练像MTCNN这样的tf.keras多任务网络

如何使用TF2将加载的h5模型正确保存到pb

如何保存 TF2 训练模型并再次使用它进行推理?

鉴于有多个 GPU 可用,如何在 TF2 中使用专用 GPU?

如何在Tensorflow-2.0中绘制tf.keras模型图?

我应该使用独立的Keras库还是tf.keras?

在Keras.model中使用tf.keras.layers

如何使用tf.keras从saveedModel访问图层

如何使用tf.contrib.keras.optimizers.Adamax?

如何在Keras中使用tf.einsum?

如何使用tf.keras.layers通过Tensorflow conv2d批处理图像序列

使用tf.keras功能API时如何在Tensorflow / Keras中使用for循环?

tf.keras如何保存ModelCheckPoint对象

在TF估计器中使用Keras模型

在Keras中使用tf.metrics?

在Keras模型中使用Tf-Idf