关键词: 激活函数, 卷积神经网络, 图像分类

Abstract: Aiming at the problem that the activation function in Convolutional Neural Networks can’t effectively provide a specific gradient response for pixels at different activation levels, an adaptive activation function is designed, which is composed of multiple piecewise linear functions. Firstly, according to the ranges of pixel activation value, multiple independent activation domains are adaptively generated, and the union of each activation domain contains the activation values of all pixels in the activation map; then a specific linear function is learned in each activation domain in order to provide a specific gradient response for the pixels in the activation domain; finally taking the NIN and ResNet18 as examples, on the CIFAR10 and CIFAR100 dataset, the performance of the proposed activation function is verified. Experimental results show that, compared with the existing activation function, the activation function proposed in this paper can provide a suitable gradient response for pixels with different activation levels, so that the classification accuracies reach 87.96% and 69.01% on the NIN respectively. The accuracies reach 88.56% and 73.54% on the ResNet18, respectively.