Warm tip: This article is reproduced from stackoverflow.com, please click
activation-function keras sigmoid

Where should i define derivative from custom activation function in Keras

发布于 2020-04-05 23:36:16

I am beginner in python, deep learning and neural network. I had made custom activation function. What i want to know when i am making custom activation function that root from sigmoid, where should i define the derivative for my custom activation function?

I've tried reading about automatic differentation. but i am not sure does keras automatically derivative my custom sigmoid?

my custom activation function in keras/activation.py

def tempsigmoid(x, temp=1.0):
    return K.sigmoid(x/temp)

my model

def baseline_model():
    # create model
    model = Sequential()
    model.add(Conv2D(101, (5, 5), input_shape=(1, 28, 28), activation='relu'))
    model.add(MaxPooling2D(pool_size=(2, 2)))
    model.add(Dropout(0.2))
    model.add(Flatten())
    model.add(Dense(128, activation='relu'))
    model.add(Dense(num_classes, activation='tempsigmoid'))
    # Compile model
    model.compile(loss='mse', optimizer='adam', metrics=['accuracy'])
    return model
Questioner
astri
Viewed
99
Matias Valdenegro 2019-04-24 16:04

Yes, Keras uses automatic differentiation, as it only supports backends with this feature (like TensorFlow).

So you do not need to define the gradient or derivative at all, it will be computed for you automatically.