Warm tip: This article is reproduced from stackoverflow.com, please click
keras opencv python-3.x computer-vision

How to create a custom keras loss function with opencv?

发布于 2020-03-27 10:25:21

I'm developing a machine learning model using keras and I notice that the available losses functions are not giving the best results on my test set.

I am using an Unet architecture, where I input a (16,16,3) image and the net also outputs a (16,16,3) picture (auto-encoder). I notice that maybe one way to improve the model would be if I used a loss function that compares pixel to pixel on the gradients (laplacian) between the net output and the ground truth. However, I did not found any tutorial that would handle this kind of application, because it would need to use opencv laplacian function on each output image from the net.

The loss function would be something like this:

def laplacian_loss(y_true, y_pred):

  # y_true already is the calculated gradients, only needs to compute on the y_pred
  # calculates the gradients for each predicted image
  y_pred_lap = []
  for img in y_pred:
    laplacian = cv2.Laplacian( np.float64(img), cv2.CV_64F )
    y_pred_lap.append( laplacian )

  y_pred_lap = np.array(y_pred_lap)

  # mean squared error, according to keras losses documentation
  return K.mean(K.square(y_pred_lap - y_true), axis=-1)

Has anyone done something like that for loss calculation?

Questioner
Lucas Kirsten
Viewed
78
分享
Lucas Kirsten 2019-07-03 22:48

I managed to reach a easy solution. The main feature was that the gradient calculation is actually a 2D filter. For more information about it, please follow the link about the laplacian kernel. In that matter, is necessary that the output of my network be filtered by the laplacian kernel. For that, I created an extra convolutional layer with fixed weights, exactly as the laplacian kernel. After that, the network will have two outputs (one been the desired image, and the other been the gradient's image). So, is also necessary to define both losses.

To make it clearer, I'll exemplify. In the end of the network you'll have something like:

channels = 3 # number of channels of network output
lap = Conv2D(channels , (3,3), padding='same', name='laplacian') (net_output)
model = Model(inputs=[net_input], outputs=[net_out, lap])

Define how you want to calculate the losses for each output:

# losses for output, laplacian and gaussian
losses = {
"enhanced": "mse",
"laplacian": "mse"
}
lossWeights = {"enhanced": 1.0, "laplacian": 0.6}

Compile the model:

model.compile(optimizer=Adam(), loss=losses, loss_weights=lossWeights)

Define the laplacian kernel, apply its values in the weights of the above convolutional layer and set trainable equals False (so it won't be updated).

bias = np.asarray([0]*3)
# laplacian kernel
l = np.asarray([
  [[[1,1,1],
  [1,-8,1],
  [1,1,1]
  ]]*channels
  ]*channels).astype(np.float32)
bias = np.asarray([0]*3).astype(np.float32)
wl = [l,bias]
model.get_layer('laplacian').set_weights(wl)
model.get_layer('laplacian').trainable = False

When training, remember that you need two values for the ground truth:

model.fit(x=X, y = {"out": y_out, "laplacian": y_lap})

Observation: Do not utilize the BatchNormalization layer! In case you use it, the weights in the laplacian layer will be updated!

Related issues

1
Create a dictionary from a list of lists with certain criteria
2
Removing Items From List while iterating with For loop
3
Determining the Distance between two matrices using numpy
4
I'm having trouble returning an HTML link as I pull links from a google search query in Python
5
Is it possible to create nested class attributes?
6
How Do I Sort Bar Chart by Month Names?
7
Connect to SQLite3 in PyCharm
8
EVENT LOOP is closed discord.py, possible token error
9
*args returns a list containing only those arguments that are even
10
Can not get client.command parameter to parse API response by key value in discord.py