自适应参数化ReLU，是一种动态的ReLU（Dynamic ReLU），在2019年5月投稿至IEEE Trans. on Industrial Electronics，2020年1月录用，2020年2月在IEEE网站发布预印本。

本文在调参记录25的基础上，将自适应参数化ReLU中间层的神经元个数，从2个增加到4个，同时添加了一个Dropout层，继续测试其在Cifar10数据集上的效果。

自适应参数化ReLU的基本原理：

Keras程序：

#!/usr/bin/env python3

实验结果：

Using TensorFlow backend.
x_train shape: (50000, 32, 32, 3)
50000 train samples
10000 test samples
Epoch 1/1000
420s 840ms/step - loss: 6.8132 - acc: 0.3731 - val_loss: 6.0072 - val_acc: 0.4935
Epoch 2/1000
333s 667ms/step - loss: 5.5637 - acc: 0.5070 - val_loss: 4.9312 - val_acc: 0.5947
Epoch 3/1000
333s 667ms/step - loss: 4.6506 - acc: 0.5734 - val_loss: 4.1140 - val_acc: 0.6551
Epoch 4/1000
333s 667ms/step - loss: 3.9385 - acc: 0.6189 - val_loss: 3.4676 - val_acc: 0.6973
Epoch 5/1000
333s 667ms/step - loss: 3.3667 - acc: 0.6524 - val_loss: 2.9511 - val_acc: 0.7314
Epoch 6/1000
333s 667ms/step - loss: 2.8977 - acc: 0.6816 - val_loss: 2.5364 - val_acc: 0.7556
Epoch 7/1000
334s 667ms/step - loss: 2.5121 - acc: 0.7062 - val_loss: 2.1815 - val_acc: 0.7762
Epoch 8/1000
334s 667ms/step - loss: 2.2071 - acc: 0.7228 - val_loss: 1.8975 - val_acc: 0.7940
Epoch 9/1000
333s 667ms/step - loss: 1.9563 - acc: 0.7382 - val_loss: 1.6740 - val_acc: 0.8045
Epoch 10/1000
333s 667ms/step - loss: 1.7436 - acc: 0.7521 - val_loss: 1.4899 - val_acc: 0.8178
Epoch 11/1000
333s 667ms/step - loss: 1.5743 - acc: 0.7647 - val_loss: 1.3368 - val_acc: 0.8307
Epoch 12/1000
333s 667ms/step - loss: 1.4340 - acc: 0.7721 - val_loss: 1.2230 - val_acc: 0.8355
Epoch 13/1000
334s 667ms/step - loss: 1.3226 - acc: 0.7815 - val_loss: 1.1185 - val_acc: 0.8426
Epoch 14/1000
334s 667ms/step - loss: 1.2300 - acc: 0.7885 - val_loss: 1.0387 - val_acc: 0.8458
Epoch 15/1000
333s 667ms/step - loss: 1.1585 - acc: 0.7961 - val_loss: 1.0022 - val_acc: 0.8457
Epoch 16/1000
334s 667ms/step - loss: 1.0832 - acc: 0.8069 - val_loss: 0.9392 - val_acc: 0.8491
Epoch 17/1000
333s 667ms/step - loss: 1.0364 - acc: 0.8119 - val_loss: 0.8801 - val_acc: 0.8604
Epoch 18/1000
333s 667ms/step - loss: 0.9910 - acc: 0.8164 - val_loss: 0.8453 - val_acc: 0.8661
Epoch 19/1000
333s 666ms/step - loss: 0.9597 - acc: 0.8208 - val_loss: 0.8281 - val_acc: 0.8665
Epoch 20/1000
333s 666ms/step - loss: 0.9316 - acc: 0.8248 - val_loss: 0.7990 - val_acc: 0.8686
Epoch 21/1000
333s 666ms/step - loss: 0.9131 - acc: 0.8259 - val_loss: 0.7708 - val_acc: 0.8773
Epoch 22/1000
333s 666ms/step - loss: 0.8881 - acc: 0.8341 - val_loss: 0.7715 - val_acc: 0.8744
Epoch 23/1000
333s 666ms/step - loss: 0.8734 - acc: 0.8352 - val_loss: 0.7602 - val_acc: 0.8800
Epoch 24/1000
333s 667ms/step - loss: 0.8614 - acc: 0.8374 - val_loss: 0.7653 - val_acc: 0.8762
Epoch 25/1000
333s 666ms/step - loss: 0.8512 - acc: 0.8415 - val_loss: 0.7291 - val_acc: 0.8862
Epoch 26/1000
333s 666ms/step - loss: 0.8386 - acc: 0.8446 - val_loss: 0.7418 - val_acc: 0.8838
Epoch 27/1000
333s 666ms/step - loss: 0.8346 - acc: 0.8471 - val_loss: 0.7217 - val_acc: 0.8891
Epoch 28/1000
334s 668ms/step - loss: 0.8243 - acc: 0.8515 - val_loss: 0.7233 - val_acc: 0.8860
Epoch 29/1000
335s 670ms/step - loss: 0.8208 - acc: 0.8516 - val_loss: 0.7140 - val_acc: 0.8889
Epoch 30/1000
336s 671ms/step - loss: 0.8181 - acc: 0.8540 - val_loss: 0.7033 - val_acc: 0.8956
Epoch 31/1000
335s 669ms/step - loss: 0.8054 - acc: 0.8546 - val_loss: 0.7127 - val_acc: 0.8903
Epoch 32/1000
336s 672ms/step - loss: 0.8017 - acc: 0.8582 - val_loss: 0.7283 - val_acc: 0.8879
Epoch 33/1000
334s 669ms/step - loss: 0.8039 - acc: 0.8604 - val_loss: 0.7117 - val_acc: 0.8904
Epoch 34/1000
332s 663ms/step - loss: 0.8007 - acc: 0.8604 - val_loss: 0.7053 - val_acc: 0.8927
Epoch 35/1000
334s 668ms/step - loss: 0.7940 - acc: 0.8636 - val_loss: 0.7236 - val_acc: 0.8891
Epoch 36/1000
332s 663ms/step - loss: 0.7910 - acc: 0.8634 - val_loss: 0.6960 - val_acc: 0.8989
Epoch 37/1000
334s 669ms/step - loss: 0.7845 - acc: 0.8663 - val_loss: 0.7109 - val_acc: 0.8913
Epoch 38/1000
334s 668ms/step - loss: 0.7870 - acc: 0.8655 - val_loss: 0.7001 - val_acc: 0.8968
Epoch 39/1000
335s 670ms/step - loss: 0.7830 - acc: 0.8680 - val_loss: 0.7081 - val_acc: 0.8951
Epoch 40/1000
335s 670ms/step - loss: 0.7846 - acc: 0.8704 - val_loss: 0.6885 - val_acc: 0.9064
Epoch 41/1000
332s 664ms/step - loss: 0.7823 - acc: 0.8703 - val_loss: 0.6980 - val_acc: 0.9037
Epoch 42/1000
333s 667ms/step - loss: 0.7770 - acc: 0.8712 - val_loss: 0.7001 - val_acc: 0.9009
Epoch 43/1000
333s 665ms/step - loss: 0.7745 - acc: 0.8725 - val_loss: 0.6988 - val_acc: 0.9041
Epoch 44/1000
335s 671ms/step - loss: 0.7789 - acc: 0.8717 - val_loss: 0.6864 - val_acc: 0.9074
Epoch 45/1000
335s 670ms/step - loss: 0.7716 - acc: 0.8760 - val_loss: 0.6904 - val_acc: 0.9069
Epoch 46/1000
334s 669ms/step - loss: 0.7711 - acc: 0.8753 - val_loss: 0.6904 - val_acc: 0.9040
Epoch 47/1000
332s 665ms/step - loss: 0.7680 - acc: 0.8788 - val_loss: 0.7124 - val_acc: 0.9002
Epoch 48/1000
335s 670ms/step - loss: 0.7685 - acc: 0.8784 - val_loss: 0.6935 - val_acc: 0.9093
Epoch 49/1000
335s 670ms/step - loss: 0.7673 - acc: 0.8788 - val_loss: 0.7104 - val_acc: 0.8989
Epoch 50/1000
335s 670ms/step - loss: 0.7675 - acc: 0.8774 - val_loss: 0.7000 - val_acc: 0.9055
Epoch 51/1000
335s 670ms/step - loss: 0.7629 - acc: 0.8819 - val_loss: 0.7205 - val_acc: 0.8998
Epoch 52/1000
333s 666ms/step - loss: 0.7658 - acc: 0.8803 - val_loss: 0.6880 - val_acc: 0.9074
Epoch 53/1000
335s 670ms/step - loss: 0.7699 - acc: 0.8792 - val_loss: 0.7064 - val_acc: 0.9058
Epoch 54/1000
333s 665ms/step - loss: 0.7619 - acc: 0.8832 - val_loss: 0.7095 - val_acc: 0.9044
Epoch 55/1000
335s 670ms/step - loss: 0.7662 - acc: 0.8807 - val_loss: 0.6995 - val_acc: 0.9083
Epoch 56/1000
335s 671ms/step - loss: 0.7618 - acc: 0.8826 - val_loss: 0.7061 - val_acc: 0.9056
Epoch 57/1000
333s 665ms/step - loss: 0.7647 - acc: 0.8826 - val_loss: 0.6994 - val_acc: 0.9077
Epoch 58/1000
334s 668ms/step - loss: 0.7628 - acc: 0.8817 - val_loss: 0.6965 - val_acc: 0.9124
Epoch 59/1000
335s 670ms/step - loss: 0.7628 - acc: 0.8835 - val_loss: 0.7090 - val_acc: 0.9020
Epoch 60/1000
335s 669ms/step - loss: 0.7594 - acc: 0.8835 - val_loss: 0.7074 - val_acc: 0.9051
Epoch 61/1000
335s 669ms/step - loss: 0.7555 - acc: 0.8861 - val_loss: 0.7233 - val_acc: 0.8998
Epoch 62/1000
332s 665ms/step - loss: 0.7617 - acc: 0.8850 - val_loss: 0.6922 - val_acc: 0.9105
Epoch 63/1000
335s 670ms/step - loss: 0.7543 - acc: 0.8872 - val_loss: 0.7026 - val_acc: 0.9055
Epoch 64/1000
335s 671ms/step - loss: 0.7562 - acc: 0.8880 - val_loss: 0.6993 - val_acc: 0.9092
Epoch 65/1000
333s 665ms/step - loss: 0.7572 - acc: 0.8868 - val_loss: 0.6902 - val_acc: 0.9107
Epoch 66/1000
335s 670ms/step - loss: 0.7576 - acc: 0.8858 - val_loss: 0.6948 - val_acc: 0.9098
Epoch 67/1000
335s 670ms/step - loss: 0.7573 - acc: 0.8875 - val_loss: 0.7264 - val_acc: 0.9005
Epoch 68/1000
335s 670ms/step - loss: 0.7501 - acc: 0.8893 - val_loss: 0.7128 - val_acc: 0.9069
Epoch 69/1000
334s 669ms/step - loss: 0.7503 - acc: 0.8908 - val_loss: 0.7080 - val_acc: 0.9072
Epoch 70/1000
335s 670ms/step - loss: 0.7567 - acc: 0.8882 - val_loss: 0.7090 - val_acc: 0.9081
...
Epoch 292/1000
334s 668ms/step - loss: 0.7486 - acc: 0.9087 - val_loss: 0.7229 - val_acc: 0.9221
Epoch 293/1000
334s 668ms/step - loss: 0.7537 - acc: 0.9070 - val_loss: 0.7409 - val_acc: 0.9157
Epoch 294/1000
334s 667ms/step - loss: 0.7549 - acc: 0.9068 - val_loss: 0.7365 - val_acc: 0.9167
Epoch 295/1000
332s 664ms/step - loss: 0.7520 - acc: 0.9078 - val_loss: 0.7265 - val_acc: 0.9186
Epoch 296/1000
332s 663ms/step - loss: 0.7542 - acc: 0.9078 - val_loss: 0.7219 - val_acc: 0.9187
Epoch 297/1000
336s 671ms/step - loss: 0.7538 - acc: 0.9076 - val_loss: 0.7297 - val_acc: 0.9201
Epoch 298/1000
333s 665ms/step - loss: 0.7492 - acc: 0.9099 - val_loss: 0.7118 - val_acc: 0.9251
Epoch 299/1000
334s 669ms/step - loss: 0.7511 - acc: 0.9088 - val_loss: 0.7181 - val_acc: 0.9232
Epoch 300/1000
334s 667ms/step - loss: 0.7525 - acc: 0.9084 - val_loss: 0.7298 - val_acc: 0.9202
Epoch 301/1000
lr changed to 0.010000000149011612
334s 668ms/step - loss: 0.6468 - acc: 0.9450 - val_loss: 0.6315 - val_acc: 0.9491
Epoch 302/1000
335s 671ms/step - loss: 0.5879 - acc: 0.9639 - val_loss: 0.6157 - val_acc: 0.9529
Epoch 303/1000
333s 665ms/step - loss: 0.5643 - acc: 0.9690 - val_loss: 0.6061 - val_acc: 0.9528
Epoch 304/1000
335s 670ms/step - loss: 0.5450 - acc: 0.9722 - val_loss: 0.5968 - val_acc: 0.9525
Epoch 305/1000
336s 672ms/step - loss: 0.5290 - acc: 0.9744 - val_loss: 0.5860 - val_acc: 0.9559
Epoch 306/1000
336s 672ms/step - loss: 0.5164 - acc: 0.9766 - val_loss: 0.5746 - val_acc: 0.9556
Epoch 307/1000
334s 667ms/step - loss: 0.5026 - acc: 0.9779 - val_loss: 0.5686 - val_acc: 0.9548
Epoch 308/1000
332s 664ms/step - loss: 0.4952 - acc: 0.9779 - val_loss: 0.5672 - val_acc: 0.9566
Epoch 309/1000
332s 664ms/step - loss: 0.4826 - acc: 0.9799 - val_loss: 0.5587 - val_acc: 0.9563
Epoch 310/1000
332s 664ms/step - loss: 0.4717 - acc: 0.9812 - val_loss: 0.5531 - val_acc: 0.9546
Epoch 311/1000
332s 664ms/step - loss: 0.4622 - acc: 0.9825 - val_loss: 0.5421 - val_acc: 0.9564
Epoch 312/1000
334s 668ms/step - loss: 0.4557 - acc: 0.9813 - val_loss: 0.5400 - val_acc: 0.9541
Epoch 313/1000
333s 667ms/step - loss: 0.4473 - acc: 0.9829 - val_loss: 0.5319 - val_acc: 0.9572
Epoch 314/1000
333s 666ms/step - loss: 0.4363 - acc: 0.9848 - val_loss: 0.5311 - val_acc: 0.9553
Epoch 315/1000
333s 665ms/step - loss: 0.4285 - acc: 0.9844 - val_loss: 0.5253 - val_acc: 0.9545
Epoch 316/1000
334s 668ms/step - loss: 0.4214 - acc: 0.9851 - val_loss: 0.5162 - val_acc: 0.9572
Epoch 317/1000
332s 665ms/step - loss: 0.4124 - acc: 0.9860 - val_loss: 0.5116 - val_acc: 0.9553
Epoch 318/1000
333s 666ms/step - loss: 0.4077 - acc: 0.9858 - val_loss: 0.5074 - val_acc: 0.9548
Epoch 319/1000
335s 671ms/step - loss: 0.3978 - acc: 0.9875 - val_loss: 0.5031 - val_acc: 0.9539
Epoch 320/1000
334s 669ms/step - loss: 0.3944 - acc: 0.9857 - val_loss: 0.4901 - val_acc: 0.9569
Epoch 321/1000
332s 663ms/step - loss: 0.3840 - acc: 0.9883 - val_loss: 0.4926 - val_acc: 0.9554
Epoch 322/1000
333s 667ms/step - loss: 0.3826 - acc: 0.9861 - val_loss: 0.4884 - val_acc: 0.9580
Epoch 323/1000
332s 664ms/step - loss: 0.3751 - acc: 0.9871 - val_loss: 0.4797 - val_acc: 0.9560
Epoch 324/1000
332s 665ms/step - loss: 0.3695 - acc: 0.9878 - val_loss: 0.4787 - val_acc: 0.9554
Epoch 325/1000
333s 665ms/step - loss: 0.3639 - acc: 0.9876 - val_loss: 0.4677 - val_acc: 0.9575
Epoch 326/1000
331s 662ms/step - loss: 0.3563 - acc: 0.9885 - val_loss: 0.4603 - val_acc: 0.9573
Epoch 327/1000
334s 667ms/step - loss: 0.3514 - acc: 0.9886 - val_loss: 0.4573 - val_acc: 0.9569
Epoch 328/1000
333s 667ms/step - loss: 0.3465 - acc: 0.9888 - val_loss: 0.4584 - val_acc: 0.9561
Epoch 329/1000
331s 661ms/step - loss: 0.3449 - acc: 0.9878 - val_loss: 0.4575 - val_acc: 0.9556
Epoch 330/1000
332s 665ms/step - loss: 0.3371 - acc: 0.9889 - val_loss: 0.4506 - val_acc: 0.9575
Epoch 331/1000
333s 665ms/step - loss: 0.3314 - acc: 0.9888 - val_loss: 0.4413 - val_acc: 0.9565
Epoch 332/1000
332s 664ms/step - loss: 0.3275 - acc: 0.9891 - val_loss: 0.4430 - val_acc: 0.9553
Epoch 333/1000
332s 665ms/step - loss: 0.3246 - acc: 0.9884 - val_loss: 0.4350 - val_acc: 0.9560
Epoch 334/1000
334s 667ms/step - loss: 0.3180 - acc: 0.9893 - val_loss: 0.4385 - val_acc: 0.9547
Epoch 335/1000
333s 667ms/step - loss: 0.3179 - acc: 0.9880 - val_loss: 0.4342 - val_acc: 0.9556
Epoch 336/1000
332s 665ms/step - loss: 0.3107 - acc: 0.9897 - val_loss: 0.4227 - val_acc: 0.9578
Epoch 337/1000
332s 664ms/step - loss: 0.3082 - acc: 0.9886 - val_loss: 0.4264 - val_acc: 0.9553
Epoch 338/1000
334s 668ms/step - loss: 0.3031 - acc: 0.9895 - val_loss: 0.4251 - val_acc: 0.9563
Epoch 339/1000
332s 664ms/step - loss: 0.2994 - acc: 0.9899 - val_loss: 0.4240 - val_acc: 0.9542
Epoch 340/1000
334s 668ms/step - loss: 0.2966 - acc: 0.9895 - val_loss: 0.4186 - val_acc: 0.9547
Epoch 341/1000
332s 664ms/step - loss: 0.2945 - acc: 0.9891 - val_loss: 0.4112 - val_acc: 0.9548
Epoch 342/1000
334s 668ms/step - loss: 0.2888 - acc: 0.9904 - val_loss: 0.4151 - val_acc: 0.9532
Epoch 343/1000
335s 670ms/step - loss: 0.2866 - acc: 0.9895 - val_loss: 0.4121 - val_acc: 0.9556
Epoch 344/1000
334s 667ms/step - loss: 0.2835 - acc: 0.9896 - val_loss: 0.4018 - val_acc: 0.9561
Epoch 345/1000
334s 669ms/step - loss: 0.2811 - acc: 0.9892 - val_loss: 0.4020 - val_acc: 0.9547
Epoch 346/1000
332s 665ms/step - loss: 0.2768 - acc: 0.9897 - val_loss: 0.4021 - val_acc: 0.9536
Epoch 347/1000
334s 668ms/step - loss: 0.2755 - acc: 0.9888 - val_loss: 0.4000 - val_acc: 0.9536
Epoch 348/1000
335s 670ms/step - loss: 0.2729 - acc: 0.9889 - val_loss: 0.3904 - val_acc: 0.9571
Epoch 349/1000
333s 665ms/step - loss: 0.2679 - acc: 0.9905 - val_loss: 0.3914 - val_acc: 0.9542
Epoch 350/1000
334s 667ms/step - loss: 0.2672 - acc: 0.9892 - val_loss: 0.3913 - val_acc: 0.9564
Epoch 351/1000
334s 667ms/step - loss: 0.2647 - acc: 0.9890 - val_loss: 0.3941 - val_acc: 0.9546
Epoch 352/1000
332s 664ms/step - loss: 0.2624 - acc: 0.9893 - val_loss: 0.3894 - val_acc: 0.9529
Epoch 353/1000
332s 664ms/step - loss: 0.2581 - acc: 0.9901 - val_loss: 0.3924 - val_acc: 0.9522
Epoch 354/1000
334s 668ms/step - loss: 0.2564 - acc: 0.9897 - val_loss: 0.3930 - val_acc: 0.9518
Epoch 355/1000
333s 666ms/step - loss: 0.2560 - acc: 0.9898 - val_loss: 0.3842 - val_acc: 0.9536
Epoch 356/1000
331s 663ms/step - loss: 0.2569 - acc: 0.9885 - val_loss: 0.3790 - val_acc: 0.9524
Epoch 357/1000
333s 666ms/step - loss: 0.2509 - acc: 0.9895 - val_loss: 0.3791 - val_acc: 0.9548
Epoch 358/1000
333s 667ms/step - loss: 0.2499 - acc: 0.9895 - val_loss: 0.3759 - val_acc: 0.9552
Epoch 359/1000
333s 666ms/step - loss: 0.2519 - acc: 0.9876 - val_loss: 0.3735 - val_acc: 0.9541
Epoch 360/1000
333s 666ms/step - loss: 0.2476 - acc: 0.9884 - val_loss: 0.3767 - val_acc: 0.9516
Epoch 361/1000
331s 661ms/step - loss: 0.2478 - acc: 0.9877 - val_loss: 0.3617 - val_acc: 0.9554
Epoch 362/1000
331s 662ms/step - loss: 0.2410 - acc: 0.9899 - val_loss: 0.3677 - val_acc: 0.9530
Epoch 363/1000
333s 666ms/step - loss: 0.2420 - acc: 0.9887 - val_loss: 0.3678 - val_acc: 0.9524
Epoch 364/1000
334s 669ms/step - loss: 0.2412 - acc: 0.9885 - val_loss: 0.3599 - val_acc: 0.9552
Epoch 365/1000
333s 666ms/step - loss: 0.2382 - acc: 0.9887 - val_loss: 0.3655 - val_acc: 0.9536
Epoch 366/1000
332s 664ms/step - loss: 0.2359 - acc: 0.9892 - val_loss: 0.3612 - val_acc: 0.9543
Epoch 367/1000
332s 663ms/step - loss: 0.2349 - acc: 0.9891 - val_loss: 0.3675 - val_acc: 0.9526
Epoch 368/1000
334s 668ms/step - loss: 0.2315 - acc: 0.9899 - val_loss: 0.3626 - val_acc: 0.9516
Epoch 369/1000
335s 670ms/step - loss: 0.2295 - acc: 0.9902 - val_loss: 0.3695 - val_acc: 0.9526
Epoch 370/1000
333s 667ms/step - loss: 0.2323 - acc: 0.9880 - val_loss: 0.3635 - val_acc: 0.9530
Epoch 371/1000
337s 673ms/step - loss: 0.2310 - acc: 0.9890 - val_loss: 0.3666 - val_acc: 0.9531
Epoch 372/1000
336s 672ms/step - loss: 0.2307 - acc: 0.9881 - val_loss: 0.3628 - val_acc: 0.9515
Epoch 373/1000
331s 663ms/step - loss: 0.2324 - acc: 0.9865 - val_loss: 0.3545 - val_acc: 0.9551
Epoch 374/1000
332s 663ms/step - loss: 0.2304 - acc: 0.9879 - val_loss: 0.3614 - val_acc: 0.9524
Epoch 375/1000
337s 673ms/step - loss: 0.2280 - acc: 0.9876 - val_loss: 0.3659 - val_acc: 0.9521
Epoch 376/1000
332s 664ms/step - loss: 0.2246 - acc: 0.9884 - val_loss: 0.3631 - val_acc: 0.9504
Epoch 377/1000
332s 664ms/step - loss: 0.2285 - acc: 0.9870 - val_loss: 0.3560 - val_acc: 0.9500
...
Epoch 577/1000
335s 670ms/step - loss: 0.1963 - acc: 0.9889 - val_loss: 0.3402 - val_acc: 0.9501
Epoch 578/1000
334s 669ms/step - loss: 0.1966 - acc: 0.9887 - val_loss: 0.3376 - val_acc: 0.9499
Epoch 579/1000
335s 669ms/step - loss: 0.2001 - acc: 0.9875 - val_loss: 0.3440 - val_acc: 0.9472
Epoch 580/1000
334s 668ms/step - loss: 0.1990 - acc: 0.9884 - val_loss: 0.3421 - val_acc: 0.9494
Epoch 581/1000
333s 667ms/step - loss: 0.2014 - acc: 0.9869 - val_loss: 0.3274 - val_acc: 0.9535
Epoch 582/1000
334s 668ms/step - loss: 0.2016 - acc: 0.9871 - val_loss: 0.3455 - val_acc: 0.9487
Epoch 583/1000
334s 668ms/step - loss: 0.2005 - acc: 0.9877 - val_loss: 0.3417 - val_acc: 0.9483
Epoch 584/1000
332s 664ms/step - loss: 0.1984 - acc: 0.9877 - val_loss: 0.3477 - val_acc: 0.9466
Epoch 585/1000
335s 669ms/step - loss: 0.2025 - acc: 0.9867 - val_loss: 0.3508 - val_acc: 0.9441
Epoch 586/1000
334s 669ms/step - loss: 0.1995 - acc: 0.9879 - val_loss: 0.3432 - val_acc: 0.9480
Epoch 587/1000
334s 668ms/step - loss: 0.1969 - acc: 0.9888 - val_loss: 0.3486 - val_acc: 0.9476
Epoch 588/1000
332s 663ms/step - loss: 0.2015 - acc: 0.9871 - val_loss: 0.3385 - val_acc: 0.9488
Epoch 589/1000
332s 663ms/step - loss: 0.1994 - acc: 0.9877 - val_loss: 0.3444 - val_acc: 0.9456
Epoch 590/1000
334s 668ms/step - loss: 0.2010 - acc: 0.9878 - val_loss: 0.3489 - val_acc: 0.9457
Epoch 591/1000
334s 668ms/step - loss: 0.2010 - acc: 0.9878 - val_loss: 0.3371 - val_acc: 0.9497
Epoch 592/1000
332s 664ms/step - loss: 0.1966 - acc: 0.9887 - val_loss: 0.3478 - val_acc: 0.9455
Epoch 593/1000
334s 669ms/step - loss: 0.1993 - acc: 0.9875 - val_loss: 0.3397 - val_acc: 0.9492
Epoch 594/1000
332s 663ms/step - loss: 0.2002 - acc: 0.9877 - val_loss: 0.3415 - val_acc: 0.9504
Epoch 595/1000
332s 663ms/step - loss: 0.1973 - acc: 0.9885 - val_loss: 0.3457 - val_acc: 0.9482
Epoch 596/1000
334s 667ms/step - loss: 0.2007 - acc: 0.9877 - val_loss: 0.3558 - val_acc: 0.9450
Epoch 597/1000
334s 667ms/step - loss: 0.1961 - acc: 0.9890 - val_loss: 0.3313 - val_acc: 0.9523
Epoch 598/1000
334s 668ms/step - loss: 0.1961 - acc: 0.9889 - val_loss: 0.3423 - val_acc: 0.9487
Epoch 599/1000
332s 664ms/step - loss: 0.1957 - acc: 0.9887 - val_loss: 0.3471 - val_acc: 0.9463
Epoch 600/1000
332s 664ms/step - loss: 0.1979 - acc: 0.9886 - val_loss: 0.3447 - val_acc: 0.9484
Epoch 601/1000
lr changed to 0.0009999999776482583
334s 668ms/step - loss: 0.1872 - acc: 0.9920 - val_loss: 0.3180 - val_acc: 0.9540
Epoch 602/1000
332s 663ms/step - loss: 0.1760 - acc: 0.9955 - val_loss: 0.3097 - val_acc: 0.9569
Epoch 603/1000
332s 663ms/step - loss: 0.1743 - acc: 0.9963 - val_loss: 0.3071 - val_acc: 0.9586
Epoch 604/1000
333s 667ms/step - loss: 0.1715 - acc: 0.9969 - val_loss: 0.3063 - val_acc: 0.9585
Epoch 605/1000
331s 663ms/step - loss: 0.1706 - acc: 0.9973 - val_loss: 0.3042 - val_acc: 0.9592
Epoch 606/1000
331s 663ms/step - loss: 0.1700 - acc: 0.9974 - val_loss: 0.3082 - val_acc: 0.9582
Epoch 607/1000
331s 663ms/step - loss: 0.1689 - acc: 0.9977 - val_loss: 0.3073 - val_acc: 0.9596
Epoch 608/1000
332s 664ms/step - loss: 0.1679 - acc: 0.9976 - val_loss: 0.3063 - val_acc: 0.9584
Epoch 609/1000
335s 670ms/step - loss: 0.1671 - acc: 0.9980 - val_loss: 0.3053 - val_acc: 0.9597
Epoch 610/1000
332s 663ms/step - loss: 0.1669 - acc: 0.9981 - val_loss: 0.3050 - val_acc: 0.9595
Epoch 611/1000
332s 664ms/step - loss: 0.1660 - acc: 0.9982 - val_loss: 0.3054 - val_acc: 0.9599
Epoch 612/1000
334s 668ms/step - loss: 0.1646 - acc: 0.9984 - val_loss: 0.3064 - val_acc: 0.9591
Epoch 613/1000
333s 666ms/step - loss: 0.1655 - acc: 0.9983 - val_loss: 0.3022 - val_acc: 0.9598
Epoch 614/1000
334s 668ms/step - loss: 0.1639 - acc: 0.9987 - val_loss: 0.3009 - val_acc: 0.9603
Epoch 615/1000
334s 667ms/step - loss: 0.1640 - acc: 0.9983 - val_loss: 0.3035 - val_acc: 0.9594
Epoch 616/1000
334s 669ms/step - loss: 0.1635 - acc: 0.9985 - val_loss: 0.3031 - val_acc: 0.9606
Epoch 617/1000
335s 669ms/step - loss: 0.1635 - acc: 0.9983 - val_loss: 0.3056 - val_acc: 0.9596
Epoch 618/1000
333s 666ms/step - loss: 0.1629 - acc: 0.9986 - val_loss: 0.3040 - val_acc: 0.9593
Epoch 619/1000
334s 667ms/step - loss: 0.1622 - acc: 0.9987 - val_loss: 0.3030 - val_acc: 0.9605
Epoch 620/1000
333s 666ms/step - loss: 0.1621 - acc: 0.9985 - val_loss: 0.3035 - val_acc: 0.9598
Epoch 621/1000
332s 664ms/step - loss: 0.1617 - acc: 0.9986 - val_loss: 0.3034 - val_acc: 0.9611
Epoch 622/1000
334s 668ms/step - loss: 0.1613 - acc: 0.9986 - val_loss: 0.3025 - val_acc: 0.9612
Epoch 623/1000
334s 668ms/step - loss: 0.1610 - acc: 0.9986 - val_loss: 0.3012 - val_acc: 0.9610
Epoch 624/1000
332s 664ms/step - loss: 0.1601 - acc: 0.9988 - val_loss: 0.3005 - val_acc: 0.9621
Epoch 625/1000
332s 664ms/step - loss: 0.1609 - acc: 0.9984 - val_loss: 0.3018 - val_acc: 0.9606
Epoch 626/1000
334s 667ms/step - loss: 0.1593 - acc: 0.9990 - val_loss: 0.3002 - val_acc: 0.9607
Epoch 627/1000
332s 663ms/step - loss: 0.1592 - acc: 0.9989 - val_loss: 0.3015 - val_acc: 0.9611
Epoch 628/1000
331s 663ms/step - loss: 0.1591 - acc: 0.9987 - val_loss: 0.2975 - val_acc: 0.9616
Epoch 629/1000
332s 663ms/step - loss: 0.1583 - acc: 0.9990 - val_loss: 0.2989 - val_acc: 0.9618
Epoch 630/1000
334s 667ms/step - loss: 0.1579 - acc: 0.9990 - val_loss: 0.2999 - val_acc: 0.9610
Epoch 631/1000
332s 663ms/step - loss: 0.1576 - acc: 0.9990 - val_loss: 0.3003 - val_acc: 0.9598
Epoch 632/1000
334s 669ms/step - loss: 0.1579 - acc: 0.9988 - val_loss: 0.2996 - val_acc: 0.9611
Epoch 633/1000
332s 664ms/step - loss: 0.1576 - acc: 0.9990 - val_loss: 0.2974 - val_acc: 0.9614
Epoch 634/1000
334s 669ms/step - loss: 0.1566 - acc: 0.9990 - val_loss: 0.3010 - val_acc: 0.9612
Epoch 635/1000
332s 664ms/step - loss: 0.1565 - acc: 0.9990 - val_loss: 0.3000 - val_acc: 0.9608
Epoch 636/1000
334s 669ms/step - loss: 0.1566 - acc: 0.9990 - val_loss: 0.3001 - val_acc: 0.9606
Epoch 637/1000
333s 665ms/step - loss: 0.1558 - acc: 0.9991 - val_loss: 0.2986 - val_acc: 0.9612
Epoch 638/1000
334s 668ms/step - loss: 0.1559 - acc: 0.9989 - val_loss: 0.2975 - val_acc: 0.9605
Epoch 639/1000
334s 668ms/step - loss: 0.1550 - acc: 0.9991 - val_loss: 0.2985 - val_acc: 0.9618
Epoch 640/1000
335s 670ms/step - loss: 0.1550 - acc: 0.9992 - val_loss: 0.2998 - val_acc: 0.9613
Epoch 641/1000
334s 668ms/step - loss: 0.1546 - acc: 0.9991 - val_loss: 0.2982 - val_acc: 0.9609
Epoch 642/1000
332s 663ms/step - loss: 0.1543 - acc: 0.9992 - val_loss: 0.2988 - val_acc: 0.9606
Epoch 643/1000
333s 666ms/step - loss: 0.1537 - acc: 0.9992 - val_loss: 0.2999 - val_acc: 0.9604
Epoch 644/1000
331s 663ms/step - loss: 0.1536 - acc: 0.9992 - val_loss: 0.2996 - val_acc: 0.9611
Epoch 645/1000
333s 666ms/step - loss: 0.1535 - acc: 0.9990 - val_loss: 0.2981 - val_acc: 0.9606
Epoch 646/1000
331s 663ms/step - loss: 0.1530 - acc: 0.9991 - val_loss: 0.2986 - val_acc: 0.9619
Epoch 647/1000
334s 668ms/step - loss: 0.1527 - acc: 0.9992 - val_loss: 0.2996 - val_acc: 0.9612
Epoch 648/1000
334s 669ms/step - loss: 0.1529 - acc: 0.9989 - val_loss: 0.2992 - val_acc: 0.9607
Epoch 649/1000
335s 670ms/step - loss: 0.1527 - acc: 0.9991 - val_loss: 0.2995 - val_acc: 0.9603
Epoch 650/1000
333s 666ms/step - loss: 0.1522 - acc: 0.9991 - val_loss: 0.2980 - val_acc: 0.9594
Epoch 651/1000
334s 669ms/step - loss: 0.1517 - acc: 0.9991 - val_loss: 0.2965 - val_acc: 0.9597
Epoch 652/1000
333s 667ms/step - loss: 0.1518 - acc: 0.9992 - val_loss: 0.2968 - val_acc: 0.9610
Epoch 653/1000
332s 663ms/step - loss: 0.1510 - acc: 0.9992 - val_loss: 0.2958 - val_acc: 0.9600
Epoch 654/1000
334s 667ms/step - loss: 0.1510 - acc: 0.9991 - val_loss: 0.2962 - val_acc: 0.9605
Epoch 655/1000
332s 664ms/step - loss: 0.1507 - acc: 0.9991 - val_loss: 0.2951 - val_acc: 0.9609
Epoch 656/1000
335s 669ms/step - loss: 0.1502 - acc: 0.9991 - val_loss: 0.2944 - val_acc: 0.9601
Epoch 657/1000
333s 667ms/step - loss: 0.1503 - acc: 0.9990 - val_loss: 0.2938 - val_acc: 0.9610
Epoch 658/1000
332s 664ms/step - loss: 0.1495 - acc: 0.9993 - val_loss: 0.2957 - val_acc: 0.9613
Epoch 659/1000
333s 667ms/step - loss: 0.1501 - acc: 0.9990 - val_loss: 0.2962 - val_acc: 0.9603
Epoch 660/1000
332s 664ms/step - loss: 0.1492 - acc: 0.9992 - val_loss: 0.2962 - val_acc: 0.9607
Epoch 661/1000
333s 666ms/step - loss: 0.1488 - acc: 0.9993 - val_loss: 0.2975 - val_acc: 0.9593
Epoch 662/1000
331s 663ms/step - loss: 0.1487 - acc: 0.9992 - val_loss: 0.2947 - val_acc: 0.9608
Epoch 663/1000
334s 667ms/step - loss: 0.1483 - acc: 0.9993 - val_loss: 0.2956 - val_acc: 0.9598
Epoch 664/1000
334s 668ms/step - loss: 0.1483 - acc: 0.9991 - val_loss: 0.2938 - val_acc: 0.9608
Epoch 665/1000
334s 668ms/step - loss: 0.1479 - acc: 0.9992 - val_loss: 0.2913 - val_acc: 0.9607
Epoch 666/1000
334s 667ms/step - loss: 0.1478 - acc: 0.9992 - val_loss: 0.2925 - val_acc: 0.9607
Epoch 667/1000
332s 663ms/step - loss: 0.1471 - acc: 0.9993 - val_loss: 0.2930 - val_acc: 0.9606
Epoch 668/1000
334s 667ms/step - loss: 0.1471 - acc: 0.9992 - val_loss: 0.2920 - val_acc: 0.9609
Epoch 669/1000
333s 667ms/step - loss: 0.1467 - acc: 0.9992 - val_loss: 0.2909 - val_acc: 0.9613
Epoch 670/1000
334s 668ms/step - loss: 0.1463 - acc: 0.9994 - val_loss: 0.2927 - val_acc: 0.9613
Epoch 671/1000
334s 667ms/step - loss: 0.1461 - acc: 0.9993 - val_loss: 0.2942 - val_acc: 0.9601
Epoch 672/1000
331s 663ms/step - loss: 0.1455 - acc: 0.9993 - val_loss: 0.2936 - val_acc: 0.9611
Epoch 673/1000
331s 663ms/step - loss: 0.1452 - acc: 0.9994 - val_loss: 0.2908 - val_acc: 0.9613
Epoch 674/1000
331s 662ms/step - loss: 0.1453 - acc: 0.9992 - val_loss: 0.2925 - val_acc: 0.9595
Epoch 675/1000
331s 662ms/step - loss: 0.1449 - acc: 0.9992 - val_loss: 0.2913 - val_acc: 0.9615
Epoch 676/1000
333s 665ms/step - loss: 0.1450 - acc: 0.9993 - val_loss: 0.2908 - val_acc: 0.9613
Epoch 677/1000
333s 666ms/step - loss: 0.1442 - acc: 0.9995 - val_loss: 0.2916 - val_acc: 0.9612
Epoch 678/1000
332s 664ms/step - loss: 0.1439 - acc: 0.9995 - val_loss: 0.2944 - val_acc: 0.9609
Epoch 679/1000
334s 667ms/step - loss: 0.1440 - acc: 0.9994 - val_loss: 0.2921 - val_acc: 0.9612
Epoch 680/1000
331s 663ms/step - loss: 0.1432 - acc: 0.9996 - val_loss: 0.2919 - val_acc: 0.9613
Epoch 681/1000
334s 669ms/step - loss: 0.1436 - acc: 0.9992 - val_loss: 0.2935 - val_acc: 0.9599
Epoch 682/1000
332s 665ms/step - loss: 0.1431 - acc: 0.9993 - val_loss: 0.2910 - val_acc: 0.9608
Epoch 683/1000
333s 666ms/step - loss: 0.1434 - acc: 0.9991 - val_loss: 0.2910 - val_acc: 0.9613
Epoch 684/1000
333s 665ms/step - loss: 0.1427 - acc: 0.9993 - val_loss: 0.2907 - val_acc: 0.9612
Epoch 685/1000
332s 664ms/step - loss: 0.1427 - acc: 0.9993 - val_loss: 0.2910 - val_acc: 0.9611
Epoch 686/1000
333s 666ms/step - loss: 0.1423 - acc: 0.9991 - val_loss: 0.2946 - val_acc: 0.9606
Epoch 687/1000
331s 661ms/step - loss: 0.1417 - acc: 0.9994 - val_loss: 0.2935 - val_acc: 0.9616
Epoch 688/1000
334s 667ms/step - loss: 0.1416 - acc: 0.9995 - val_loss: 0.2922 - val_acc: 0.9612
Epoch 689/1000
333s 667ms/step - loss: 0.1413 - acc: 0.9994 - val_loss: 0.2932 - val_acc: 0.9609
Epoch 690/1000
334s 667ms/step - loss: 0.1413 - acc: 0.9992 - val_loss: 0.2914 - val_acc: 0.9616
Epoch 691/1000
332s 665ms/step - loss: 0.1413 - acc: 0.9990 - val_loss: 0.2935 - val_acc: 0.9619
Epoch 692/1000
333s 666ms/step - loss: 0.1410 - acc: 0.9993 - val_loss: 0.2952 - val_acc: 0.9612
Epoch 693/1000
333s 667ms/step - loss: 0.1407 - acc: 0.9992 - val_loss: 0.2952 - val_acc: 0.9602
Epoch 694/1000
331s 662ms/step - loss: 0.1404 - acc: 0.9992 - val_loss: 0.2940 - val_acc: 0.9600
Epoch 695/1000
333s 667ms/step - loss: 0.1400 - acc: 0.9995 - val_loss: 0.2934 - val_acc: 0.9603
Epoch 696/1000
334s 668ms/step - loss: 0.1396 - acc: 0.9994 - val_loss: 0.2940 - val_acc: 0.9602
Epoch 697/1000
333s 665ms/step - loss: 0.1394 - acc: 0.9994 - val_loss: 0.2925 - val_acc: 0.9599
Epoch 698/1000
333s 667ms/step - loss: 0.1393 - acc: 0.9994 - val_loss: 0.2920 - val_acc: 0.9611
Epoch 699/1000
333s 667ms/step - loss: 0.1392 - acc: 0.9992 - val_loss: 0.2884 - val_acc: 0.9611
Epoch 700/1000
333s 666ms/step - loss: 0.1383 - acc: 0.9994 - val_loss: 0.2877 - val_acc: 0.9609
Epoch 701/1000
331s 661ms/step - loss: 0.1385 - acc: 0.9993 - val_loss: 0.2887 - val_acc: 0.9605
Epoch 702/1000
332s 665ms/step - loss: 0.1387 - acc: 0.9992 - val_loss: 0.2883 - val_acc: 0.9606
Epoch 703/1000
331s 661ms/step - loss: 0.1380 - acc: 0.9992 - val_loss: 0.2892 - val_acc: 0.9603
Epoch 704/1000
331s 662ms/step - loss: 0.1376 - acc: 0.9994 - val_loss: 0.2886 - val_acc: 0.9598
Epoch 705/1000
334s 668ms/step - loss: 0.1379 - acc: 0.9993 - val_loss: 0.2891 - val_acc: 0.9608
Epoch 706/1000
333s 666ms/step - loss: 0.1372 - acc: 0.9994 - val_loss: 0.2902 - val_acc: 0.9595
Epoch 707/1000
333s 666ms/step - loss: 0.1370 - acc: 0.9995 - val_loss: 0.2895 - val_acc: 0.9594
Epoch 708/1000
333s 665ms/step - loss: 0.1370 - acc: 0.9992 - val_loss: 0.2892 - val_acc: 0.9594
Epoch 709/1000
331s 662ms/step - loss: 0.1366 - acc: 0.9993 - val_loss: 0.2897 - val_acc: 0.9594
Epoch 710/1000
331s 662ms/step - loss: 0.1362 - acc: 0.9996 - val_loss: 0.2913 - val_acc: 0.9589
Epoch 711/1000
334s 667ms/step - loss: 0.1362 - acc: 0.9993 - val_loss: 0.2890 - val_acc: 0.9609
Epoch 712/1000
333s 666ms/step - loss: 0.1360 - acc: 0.9993 - val_loss: 0.2924 - val_acc: 0.9584
Epoch 713/1000
333s 666ms/step - loss: 0.1361 - acc: 0.9992 - val_loss: 0.2912 - val_acc: 0.9587
Epoch 714/1000
331s 661ms/step - loss: 0.1354 - acc: 0.9993 - val_loss: 0.2904 - val_acc: 0.9581
Epoch 715/1000
331s 662ms/step - loss: 0.1354 - acc: 0.9994 - val_loss: 0.2901 - val_acc: 0.9581
Epoch 716/1000
333s 666ms/step - loss: 0.1351 - acc: 0.9992 - val_loss: 0.2914 - val_acc: 0.9582
Epoch 717/1000
332s 664ms/step - loss: 0.1349 - acc: 0.9992 - val_loss: 0.2881 - val_acc: 0.9585
Epoch 718/1000
331s 662ms/step - loss: 0.1345 - acc: 0.9995 - val_loss: 0.2870 - val_acc: 0.9596
Epoch 719/1000
331s 662ms/step - loss: 0.1344 - acc: 0.9994 - val_loss: 0.2863 - val_acc: 0.9600
Epoch 720/1000
333s 667ms/step - loss: 0.1343 - acc: 0.9993 - val_loss: 0.2881 - val_acc: 0.9592
Epoch 721/1000
334s 667ms/step - loss: 0.1338 - acc: 0.9995 - val_loss: 0.2869 - val_acc: 0.9595
Epoch 722/1000
331s 661ms/step - loss: 0.1336 - acc: 0.9995 - val_loss: 0.2853 - val_acc: 0.9599
Epoch 723/1000
334s 668ms/step - loss: 0.1334 - acc: 0.9994 - val_loss: 0.2856 - val_acc: 0.9603
Epoch 724/1000
331s 661ms/step - loss: 0.1330 - acc: 0.9995 - val_loss: 0.2843 - val_acc: 0.9600
Epoch 725/1000
332s 664ms/step - loss: 0.1330 - acc: 0.9993 - val_loss: 0.2853 - val_acc: 0.9601
Epoch 726/1000
331s 661ms/step - loss: 0.1329 - acc: 0.9994 - val_loss: 0.2819 - val_acc: 0.9600
Epoch 727/1000
331s 662ms/step - loss: 0.1326 - acc: 0.9994 - val_loss: 0.2816 - val_acc: 0.9596
Epoch 728/1000
333s 665ms/step - loss: 0.1324 - acc: 0.9993 - val_loss: 0.2826 - val_acc: 0.9598
Epoch 729/1000
333s 666ms/step - loss: 0.1323 - acc: 0.9994 - val_loss: 0.2840 - val_acc: 0.9597
Epoch 730/1000
331s 662ms/step - loss: 0.1321 - acc: 0.9993 - val_loss: 0.2831 - val_acc: 0.9604
Epoch 731/1000
333s 666ms/step - loss: 0.1314 - acc: 0.9995 - val_loss: 0.2833 - val_acc: 0.9597
Epoch 732/1000
333s 666ms/step - loss: 0.1315 - acc: 0.9994 - val_loss: 0.2808 - val_acc: 0.9609
Epoch 733/1000
331s 662ms/step - loss: 0.1309 - acc: 0.9994 - val_loss: 0.2831 - val_acc: 0.9593
Epoch 734/1000
332s 665ms/step - loss: 0.1308 - acc: 0.9994 - val_loss: 0.2840 - val_acc: 0.9592
Epoch 735/1000
331s 662ms/step - loss: 0.1306 - acc: 0.9995 - val_loss: 0.2836 - val_acc: 0.9596
Epoch 736/1000
332s 664ms/step - loss: 0.1305 - acc: 0.9993 - val_loss: 0.2838 - val_acc: 0.9590
Epoch 737/1000
331s 662ms/step - loss: 0.1302 - acc: 0.9994 - val_loss: 0.2829 - val_acc: 0.9593
Epoch 738/1000
333s 666ms/step - loss: 0.1304 - acc: 0.9993 - val_loss: 0.2845 - val_acc: 0.9580
Epoch 739/1000
333s 666ms/step - loss: 0.1296 - acc: 0.9996 - val_loss: 0.2878 - val_acc: 0.9579
Epoch 740/1000
333s 666ms/step - loss: 0.1298 - acc: 0.9993 - val_loss: 0.2851 - val_acc: 0.9587
Epoch 741/1000
333s 666ms/step - loss: 0.1294 - acc: 0.9995 - val_loss: 0.2859 - val_acc: 0.9593
Epoch 742/1000
333s 665ms/step - loss: 0.1290 - acc: 0.9995 - val_loss: 0.2840 - val_acc: 0.9589
Epoch 743/1000
333s 666ms/step - loss: 0.1288 - acc: 0.9995 - val_loss: 0.2836 - val_acc: 0.9591
Epoch 744/1000
332s 665ms/step - loss: 0.1286 - acc: 0.9995 - val_loss: 0.2868 - val_acc: 0.9587
Epoch 745/1000
331s 662ms/step - loss: 0.1288 - acc: 0.9993 - val_loss: 0.2851 - val_acc: 0.9587
Epoch 746/1000
331s 661ms/step - loss: 0.1285 - acc: 0.9993 - val_loss: 0.2841 - val_acc: 0.9589
Epoch 747/1000
331s 662ms/step - loss: 0.1282 - acc: 0.9995 - val_loss: 0.2841 - val_acc: 0.9589
Epoch 748/1000
334s 667ms/step - loss: 0.1281 - acc: 0.9994 - val_loss: 0.2846 - val_acc: 0.9585
Epoch 749/1000
333s 665ms/step - loss: 0.1276 - acc: 0.9995 - val_loss: 0.2846 - val_acc: 0.9597
Epoch 750/1000
333s 665ms/step - loss: 0.1272 - acc: 0.9996 - val_loss: 0.2839 - val_acc: 0.9589
Epoch 751/1000
333s 667ms/step - loss: 0.1272 - acc: 0.9995 - val_loss: 0.2831 - val_acc: 0.9594
Epoch 752/1000
332s 665ms/step - loss: 0.1273 - acc: 0.9994 - val_loss: 0.2828 - val_acc: 0.9592
Epoch 753/1000
333s 666ms/step - loss: 0.1271 - acc: 0.9994 - val_loss: 0.2823 - val_acc: 0.9592
Epoch 754/1000
333s 667ms/step - loss: 0.1267 - acc: 0.9995 - val_loss: 0.2826 - val_acc: 0.9598
...
Epoch 899/1000
333s 666ms/step - loss: 0.1018 - acc: 0.9994 - val_loss: 0.2702 - val_acc: 0.9570
Epoch 900/1000
333s 666ms/step - loss: 0.1019 - acc: 0.9992 - val_loss: 0.2697 - val_acc: 0.9573
Epoch 901/1000
lr changed to 9.999999310821295e-05
331s 661ms/step - loss: 0.1016 - acc: 0.9993 - val_loss: 0.2677 - val_acc: 0.9572
Epoch 902/1000
333s 666ms/step - loss: 0.1014 - acc: 0.9994 - val_loss: 0.2691 - val_acc: 0.9573
Epoch 903/1000
331s 662ms/step - loss: 0.1011 - acc: 0.9995 - val_loss: 0.2666 - val_acc: 0.9587
Epoch 904/1000
335s 669ms/step - loss: 0.1011 - acc: 0.9994 - val_loss: 0.2680 - val_acc: 0.9575
Epoch 905/1000
334s 669ms/step - loss: 0.1016 - acc: 0.9994 - val_loss: 0.2659 - val_acc: 0.9581
Epoch 906/1000
334s 667ms/step - loss: 0.1010 - acc: 0.9995 - val_loss: 0.2659 - val_acc: 0.9578
Epoch 907/1000
333s 666ms/step - loss: 0.1010 - acc: 0.9996 - val_loss: 0.2642 - val_acc: 0.9582
Epoch 908/1000
333s 665ms/step - loss: 0.1008 - acc: 0.9997 - val_loss: 0.2637 - val_acc: 0.9574
Epoch 909/1000
331s 662ms/step - loss: 0.1011 - acc: 0.9995 - val_loss: 0.2651 - val_acc: 0.9573
Epoch 910/1000
332s 665ms/step - loss: 0.1009 - acc: 0.9996 - val_loss: 0.2656 - val_acc: 0.9579
Epoch 911/1000
333s 666ms/step - loss: 0.1013 - acc: 0.9995 - val_loss: 0.2643 - val_acc: 0.9577
Epoch 912/1000
331s 662ms/step - loss: 0.1011 - acc: 0.9995 - val_loss: 0.2654 - val_acc: 0.9578
Epoch 913/1000
334s 667ms/step - loss: 0.1006 - acc: 0.9997 - val_loss: 0.2655 - val_acc: 0.9576
Epoch 914/1000
333s 665ms/step - loss: 0.1009 - acc: 0.9996 - val_loss: 0.2635 - val_acc: 0.9579
Epoch 915/1000
333s 667ms/step - loss: 0.1010 - acc: 0.9994 - val_loss: 0.2649 - val_acc: 0.9586
Epoch 916/1000
332s 665ms/step - loss: 0.1009 - acc: 0.9995 - val_loss: 0.2637 - val_acc: 0.9577
Epoch 917/1000
332s 664ms/step - loss: 0.1008 - acc: 0.9996 - val_loss: 0.2648 - val_acc: 0.9584
Epoch 918/1000
331s 662ms/step - loss: 0.1005 - acc: 0.9997 - val_loss: 0.2652 - val_acc: 0.9586
Epoch 919/1000
333s 666ms/step - loss: 0.1012 - acc: 0.9994 - val_loss: 0.2642 - val_acc: 0.9590
Epoch 920/1000
331s 662ms/step - loss: 0.1009 - acc: 0.9996 - val_loss: 0.2645 - val_acc: 0.9586
Epoch 921/1000
334s 669ms/step - loss: 0.1010 - acc: 0.9994 - val_loss: 0.2645 - val_acc: 0.9579
Epoch 922/1000
333s 665ms/step - loss: 0.1012 - acc: 0.9995 - val_loss: 0.2637 - val_acc: 0.9591
Epoch 923/1000
332s 665ms/step - loss: 0.1010 - acc: 0.9995 - val_loss: 0.2639 - val_acc: 0.9583
Epoch 924/1000
331s 663ms/step - loss: 0.1011 - acc: 0.9994 - val_loss: 0.2630 - val_acc: 0.9586
Epoch 925/1000
335s 669ms/step - loss: 0.1007 - acc: 0.9996 - val_loss: 0.2647 - val_acc: 0.9588
Epoch 926/1000
335s 670ms/step - loss: 0.1007 - acc: 0.9996 - val_loss: 0.2634 - val_acc: 0.9583
Epoch 927/1000
336s 673ms/step - loss: 0.1005 - acc: 0.9996 - val_loss: 0.2637 - val_acc: 0.9587
Epoch 928/1000
333s 667ms/step - loss: 0.1010 - acc: 0.9994 - val_loss: 0.2641 - val_acc: 0.9584
Epoch 929/1000
336s 672ms/step - loss: 0.1007 - acc: 0.9996 - val_loss: 0.2629 - val_acc: 0.9591
Epoch 930/1000
339s 679ms/step - loss: 0.1006 - acc: 0.9996 - val_loss: 0.2643 - val_acc: 0.9581
Epoch 931/1000
338s 676ms/step - loss: 0.1005 - acc: 0.9995 - val_loss: 0.2625 - val_acc: 0.9583
Epoch 932/1000
338s 675ms/step - loss: 0.1005 - acc: 0.9995 - val_loss: 0.2643 - val_acc: 0.9584
Epoch 933/1000
331s 662ms/step - loss: 0.1006 - acc: 0.9996 - val_loss: 0.2640 - val_acc: 0.9586
Epoch 934/1000
334s 668ms/step - loss: 0.1005 - acc: 0.9996 - val_loss: 0.2640 - val_acc: 0.9583
Epoch 935/1000
334s 667ms/step - loss: 0.1008 - acc: 0.9996 - val_loss: 0.2626 - val_acc: 0.9584
Epoch 936/1000
333s 667ms/step - loss: 0.1007 - acc: 0.9995 - val_loss: 0.2627 - val_acc: 0.9598
Epoch 937/1000
332s 665ms/step - loss: 0.1008 - acc: 0.9996 - val_loss: 0.2632 - val_acc: 0.9590
Epoch 938/1000
332s 663ms/step - loss: 0.1005 - acc: 0.9995 - val_loss: 0.2627 - val_acc: 0.9590
Epoch 939/1000
333s 666ms/step - loss: 0.1005 - acc: 0.9996 - val_loss: 0.2636 - val_acc: 0.9592
Epoch 940/1000
333s 666ms/step - loss: 0.1004 - acc: 0.9996 - val_loss: 0.2640 - val_acc: 0.9587
Epoch 941/1000
333s 665ms/step - loss: 0.1005 - acc: 0.9995 - val_loss: 0.2644 - val_acc: 0.9587
Epoch 942/1000
333s 666ms/step - loss: 0.1004 - acc: 0.9995 - val_loss: 0.2637 - val_acc: 0.9587
Epoch 943/1000
335s 669ms/step - loss: 0.1005 - acc: 0.9996 - val_loss: 0.2625 - val_acc: 0.9595
Epoch 944/1000
334s 668ms/step - loss: 0.1009 - acc: 0.9994 - val_loss: 0.2635 - val_acc: 0.9585
Epoch 945/1000
334s 668ms/step - loss: 0.1002 - acc: 0.9996 - val_loss: 0.2652 - val_acc: 0.9586
Epoch 946/1000
331s 662ms/step - loss: 0.1005 - acc: 0.9996 - val_loss: 0.2641 - val_acc: 0.9588
Epoch 947/1000
333s 666ms/step - loss: 0.1005 - acc: 0.9995 - val_loss: 0.2638 - val_acc: 0.9583
Epoch 948/1000
333s 667ms/step - loss: 0.1005 - acc: 0.9995 - val_loss: 0.2634 - val_acc: 0.9594
Epoch 949/1000
331s 662ms/step - loss: 0.1002 - acc: 0.9997 - val_loss: 0.2641 - val_acc: 0.9591
Epoch 950/1000
333s 667ms/step - loss: 0.1003 - acc: 0.9995 - val_loss: 0.2616 - val_acc: 0.9591
Epoch 951/1000
333s 666ms/step - loss: 0.1000 - acc: 0.9998 - val_loss: 0.2634 - val_acc: 0.9590
Epoch 952/1000
331s 662ms/step - loss: 0.1001 - acc: 0.9996 - val_loss: 0.2619 - val_acc: 0.9593
Epoch 953/1000
334s 668ms/step - loss: 0.0997 - acc: 0.9998 - val_loss: 0.2638 - val_acc: 0.9587
Epoch 954/1000
333s 666ms/step - loss: 0.1003 - acc: 0.9996 - val_loss: 0.2643 - val_acc: 0.9593
Epoch 955/1000
333s 666ms/step - loss: 0.1002 - acc: 0.9996 - val_loss: 0.2630 - val_acc: 0.9587
Epoch 956/1000
331s 662ms/step - loss: 0.1000 - acc: 0.9996 - val_loss: 0.2634 - val_acc: 0.9587
Epoch 957/1000
333s 667ms/step - loss: 0.1001 - acc: 0.9995 - val_loss: 0.2640 - val_acc: 0.9588
Epoch 958/1000
332s 665ms/step - loss: 0.1004 - acc: 0.9995 - val_loss: 0.2637 - val_acc: 0.9591
Epoch 959/1000
333s 667ms/step - loss: 0.1003 - acc: 0.9997 - val_loss: 0.2625 - val_acc: 0.9591
Epoch 960/1000
333s 665ms/step - loss: 0.1000 - acc: 0.9996 - val_loss: 0.2639 - val_acc: 0.9595
Epoch 961/1000
331s 662ms/step - loss: 0.1003 - acc: 0.9994 - val_loss: 0.2644 - val_acc: 0.9590
Epoch 962/1000
333s 667ms/step - loss: 0.0999 - acc: 0.9996 - val_loss: 0.2633 - val_acc: 0.9595
Epoch 963/1000
333s 666ms/step - loss: 0.1003 - acc: 0.9995 - val_loss: 0.2635 - val_acc: 0.9594
Epoch 964/1000
331s 662ms/step - loss: 0.0999 - acc: 0.9995 - val_loss: 0.2623 - val_acc: 0.9593
Epoch 965/1000
331s 662ms/step - loss: 0.1001 - acc: 0.9996 - val_loss: 0.2640 - val_acc: 0.9593
Epoch 966/1000
333s 666ms/step - loss: 0.1003 - acc: 0.9995 - val_loss: 0.2640 - val_acc: 0.9592
Epoch 967/1000
331s 661ms/step - loss: 0.0995 - acc: 0.9998 - val_loss: 0.2636 - val_acc: 0.9594
Epoch 968/1000
331s 663ms/step - loss: 0.0998 - acc: 0.9997 - val_loss: 0.2642 - val_acc: 0.9585
Epoch 969/1000
336s 672ms/step - loss: 0.0999 - acc: 0.9995 - val_loss: 0.2631 - val_acc: 0.9595
Epoch 970/1000
333s 667ms/step - loss: 0.1000 - acc: 0.9997 - val_loss: 0.2633 - val_acc: 0.9596
Epoch 971/1000
332s 665ms/step - loss: 0.0998 - acc: 0.9996 - val_loss: 0.2643 - val_acc: 0.9589
Epoch 972/1000
331s 662ms/step - loss: 0.0998 - acc: 0.9996 - val_loss: 0.2655 - val_acc: 0.9600
Epoch 973/1000
333s 666ms/step - loss: 0.0997 - acc: 0.9996 - val_loss: 0.2640 - val_acc: 0.9590
Epoch 974/1000
331s 662ms/step - loss: 0.0999 - acc: 0.9995 - val_loss: 0.2641 - val_acc: 0.9588
Epoch 975/1000
333s 667ms/step - loss: 0.0996 - acc: 0.9998 - val_loss: 0.2651 - val_acc: 0.9593
Epoch 976/1000
333s 666ms/step - loss: 0.1000 - acc: 0.9995 - val_loss: 0.2647 - val_acc: 0.9591
Epoch 977/1000
334s 668ms/step - loss: 0.0998 - acc: 0.9996 - val_loss: 0.2645 - val_acc: 0.9589
Epoch 978/1000
331s 662ms/step - loss: 0.0997 - acc: 0.9997 - val_loss: 0.2655 - val_acc: 0.9585
Epoch 979/1000
333s 665ms/step - loss: 0.0994 - acc: 0.9998 - val_loss: 0.2641 - val_acc: 0.9584
Epoch 980/1000
333s 666ms/step - loss: 0.0997 - acc: 0.9996 - val_loss: 0.2647 - val_acc: 0.9588
Epoch 981/1000
331s 662ms/step - loss: 0.0994 - acc: 0.9997 - val_loss: 0.2641 - val_acc: 0.9592
Epoch 982/1000
333s 666ms/step - loss: 0.0997 - acc: 0.9996 - val_loss: 0.2641 - val_acc: 0.9584
Epoch 983/1000
331s 662ms/step - loss: 0.0996 - acc: 0.9996 - val_loss: 0.2631 - val_acc: 0.9604
Epoch 984/1000
334s 667ms/step - loss: 0.0999 - acc: 0.9996 - val_loss: 0.2634 - val_acc: 0.9599
Epoch 985/1000
334s 668ms/step - loss: 0.0995 - acc: 0.9996 - val_loss: 0.2633 - val_acc: 0.9595
Epoch 986/1000
334s 668ms/step - loss: 0.0996 - acc: 0.9996 - val_loss: 0.2645 - val_acc: 0.9596
Epoch 987/1000
333s 666ms/step - loss: 0.0994 - acc: 0.9996 - val_loss: 0.2629 - val_acc: 0.9595
Epoch 988/1000
332s 665ms/step - loss: 0.0994 - acc: 0.9997 - val_loss: 0.2637 - val_acc: 0.9598
Epoch 989/1000
333s 666ms/step - loss: 0.0994 - acc: 0.9997 - val_loss: 0.2637 - val_acc: 0.9592
Epoch 990/1000
331s 662ms/step - loss: 0.0994 - acc: 0.9996 - val_loss: 0.2631 - val_acc: 0.9598
Epoch 991/1000
331s 662ms/step - loss: 0.0994 - acc: 0.9997 - val_loss: 0.2637 - val_acc: 0.9596
Epoch 992/1000
331s 662ms/step - loss: 0.0995 - acc: 0.9996 - val_loss: 0.2640 - val_acc: 0.9595
Epoch 993/1000
333s 667ms/step - loss: 0.0995 - acc: 0.9996 - val_loss: 0.2635 - val_acc: 0.9596
Epoch 994/1000
333s 666ms/step - loss: 0.0993 - acc: 0.9997 - val_loss: 0.2638 - val_acc: 0.9593
Epoch 995/1000
331s 662ms/step - loss: 0.0990 - acc: 0.9998 - val_loss: 0.2639 - val_acc: 0.9595
Epoch 996/1000
333s 666ms/step - loss: 0.0994 - acc: 0.9997 - val_loss: 0.2643 - val_acc: 0.9585
Epoch 997/1000
331s 662ms/step - loss: 0.0993 - acc: 0.9995 - val_loss: 0.2631 - val_acc: 0.9590
Epoch 998/1000
334s 667ms/step - loss: 0.0993 - acc: 0.9996 - val_loss: 0.2632 - val_acc: 0.9590
Epoch 999/1000
331s 662ms/step - loss: 0.0996 - acc: 0.9996 - val_loss: 0.2638 - val_acc: 0.9596
Epoch 1000/1000
333s 666ms/step - loss: 0.0991 - acc: 0.9997 - val_loss: 0.2636 - val_acc: 0.9596
Train loss: 0.0978851483464241
Train accuracy: 0.9999800000190735
Test loss: 0.26395464450120926
Test accuracy: 0.9592000007629394

最终的实验结果是95.92%。其实，在第670个epoch的时候，已经到了96.13%，最后反而降了。

参考文献：

Minghang Zhao, Shisheng Zhong, Xuyun Fu, Baoping Tang, Shaojiang Dong, Michael Pecht, Deep Residual Networks with Adaptively Parametric Rectifier Linear Units for Fault Diagnosis, IEEE Transactions on Industrial Electronics, DOI: 10.1109/TIE.2020.2972458, Date of Publication: 13 February 2020

Deep Residual Networks with Adaptively Parametric Rectifier Linear Units for Fault Diagnosis​ieeexplore.ieee.org

input自适应_一种Dynamic ReLU：自适应参数化ReLU（调参记录26）Cifar10~95.92%相关推荐

  1. input自适应_一种Dynamic ReLU：自适应参数化ReLU激活函数（调参记录13）

    自适应参数化ReLU,本身就是一种动态的ReLU(Dynamic ReLU,Dy-ReLU).该论文在2019年5月的时候,投稿到IEEE Transactions on Industrial Ele ...

  2. 深度残差网络+自适应参数化ReLU激活函数（调参记录26）Cifar10~95.92%

    本文在调参记录25的基础上,将自适应参数化ReLU中间层的神经元个数,从2个增加到4个,同时添加了一个Dropout层,继续测试其在Cifar10数据集上的效果. 自适应参数化ReLU的基本原理: K ...

  3. 深度残差网络+自适应参数化ReLU激活函数：调参记录26（Cifar10~95.92%）

    本文在调参记录25的基础上,将自适应参数化ReLU中间层的神经元个数,从2个增加到4个,同时添加了一个Dropout层,继续测试其在Cifar10数据集上的效果. 自适应参数化ReLU的基本原理: K ...

  4. 【哈工大版】Dynamic ReLU：Adaptively Parametric ReLU及Keras代码（调参记录26）Cifar10~95.92%

    本文介绍哈工大团队提出的一种动态ReLU(Dynamic ReLU)激活函数,即自适应参数化ReLU激活函数,原本是应用在基于一维振动信号的故障诊断,能够让每个样本有自己独特的ReLU参数,在2019 ...

  5. input自适应_深度残差网络+自适应参数化ReLU（调参记录18）Cifar10~94.28%

    本文在调参记录17的基础上,将残差模块的数量增加到27个.其实之前也这样做过,现在的区别在于,自适应参数化ReLU激活函数中第一个全连接层中的神经元个数设置成了特征通道数量的1/16.同样是在Cifa ...

  6. 【哈工大版】动态ReLU：自适应参数化ReLU及Keras代码（调参记录11）

    本文介绍哈工大团队提出的一种动态ReLU(Dynamic ReLU)激活函数,即自适应参数化ReLU激活函数,原本是应用在基于一维振动信号的故障诊断,能够让每个样本有自己独特的ReLU参数,在2019 ...

  7. 深度残差网络+自适应参数化ReLU激活函数（调参记录3）

    续上一篇: 深度残差网络+自适应参数化ReLU激活函数(调参记录2) https://blog.csdn.net/dangqing1988/article/details/105595917 本文继续 ...

  8. 深度残差网络+自适应参数化ReLU激活函数（调参记录8）

    续上一篇: 深度残差网络+自适应参数化ReLU激活函数(调参记录7) https://blog.csdn.net/dangqing1988/article/details/105670981 本文将层 ...

  9. 深度残差网络+自适应参数化ReLU激活函数：调参记录18（Cifar10~94.28%）

    本文将残差模块的数量增加到27个.其实之前也这样做过,现在的区别在于,自适应参数化ReLU激活函数中第一个全连接层中的神经元个数设置成了特征通道数量的1/16.同样是在Cifar10数据集上进行测试. ...

最新文章

  1. 超有趣！LSTM之父团队最新力作：将强化学习“颠倒”过来
  2. ArcEngine 图层标注 （根据字段、角度）
  3. ARM裸机开发环境搭建
  4. 使用ueditor实现多图片上传案例——Dao层（BaseDao）
  5. spark官方文档_这些未在 Spark SQL 文档中说明的优化措施，你知道吗？
  6. php静态分析工具window,window_SpeedPHP框架核心调试工具，在日常的编程开发当中，开发 - phpStudy...
  7. 数据结构——图的C语言实现
  8. dwc_otg驱动 BUG: sleeping function called from invalid context at mm/page_alloc.c
  9. 前端又省事了，Chrome直接支持lazyload延迟加载
  10. 已知先序和中序得出后序
  11. linux怎么测试一个脚本,一个Linux中用于监控的简易shell脚本
  12. Python之动态网页爬虫方案介绍及selenium+chrome详解
  13. fruit loops studio音乐宿主软件daw水果软件20.9中文版
  14. linux下搭建radius服务器,linux搭建freeradius服务器
  15. 使用yolov5训练PASCAL VOC2012数据集以及遇到的坑
  16. 计算机中if函数的使用方法,如何在Excel 中使用IF函数
  17. Excel 添加图片批注
  18. 做嵌入式编程，为什么用的是C语言而不是C++呢？
  19. 2010中国互联网十大搞笑段子，笑后请深思
  20. 散列表及散列冲突解决方案

热门文章

  1. 聊聊flink的Tumbling Window
  2. Python--day8--Socket编程/异常处理
  3. iOS 蓝牙开发和注意点
  4. Swift 4正式发布，新功能概览
  5. quartz+spring框架动态调整频率实践
  6. centOS6.7 /etc/profile
  7. FTP linux-500 OOPS问题解决 （转载）
  8. poj 1789 Truck History(最小生成树 prim）
  9. 当adobe flash player不能安装时
  10. PPT(五)-让你的图片靓起来！