Download as PDF, PPTX
![Hichem Felouat - hichemfel@gmail.com 14 Batch Normalization: This operation simply zero-centers and normalizes each input, then scales and shifts the result (before or after the activation function of each hidden layer). model = keras.models.Sequential([ keras.layers.Flatten(input_shape=[28, 28]), keras.layers.BatchNormalization(), keras.layers.Dense(300, activation="elu", kernel_initializer="he_normal"), keras.layers.BatchNormalization(), keras.layers.Dense(100, activation="elu", kernel_initializer="he_normal"), keras.layers.BatchNormalization(), keras.layers.Dense(10, activation="softmax") ]) Avoiding Overfitting](https://image.slidesharecdn.com/cnn-200926193538/75/Build-your-own-Convolutional-Neural-Network-CNN-14-2048.jpg)
![Hichem Felouat - hichemfel@gmail.com 16 Avoiding Overfitting Update the optimizer’s learning_rate attribute at the beginning of each epoch: lr_scheduler = keras.callbacks.ReduceLROnPlateau(factor=0.5, patience=5) history = model.fit(X_train, y_train, [...], callbacks=[lr_scheduler])](https://image.slidesharecdn.com/cnn-200926193538/75/Build-your-own-Convolutional-Neural-Network-CNN-16-2048.jpg)
![Hichem Felouat - hichemfel@gmail.com 19 How to increase your small image dataset trainAug = ImageDataGenerator( rotation_range=40, width_shift_range=0.2, height_shift_range = 0.2, shear_range=0.2, zoom_range=0.2, horizontal_flip=True, fill_mode='nearest') model.compile(loss="binary_crossentropy", optimizer=opt, metrics=["accuracy"]) H = model.fit_generator( trainAug.flow(trainX, trainY, batch_size=BS), steps_per_epoch=len(trainX) // BS,validation_data=(testX, testY), validation_steps=len(testX) // BS, epochs=EPOCHS)](https://image.slidesharecdn.com/cnn-200926193538/75/Build-your-own-Convolutional-Neural-Network-CNN-19-2048.jpg)
![Hichem Felouat - hichemfel@gmail.com 20 Avoiding Overfitting - Using Callbacks checkpoint_cb = keras.callbacks.ModelCheckpoint ("my_keras_model.h5", save_best_only=True) history = model.fit(X_train, y_train, epochs=10, validation_data=(X_valid, y_valid), callbacks=[checkpoint_cb]) # roll back to best model model = keras.models.load_model("my_keras_model.h5") • You can combine both callbacks to save checkpoints of your model (in case your computer crashes) and interrupt training early when there is no more progress (to avoid wasting time and resources): early_stopping_cb = keras.callbacks.EarlyStopping(patience=10, restore_best_weights=True) history = model.fit(X_train, y_train, epochs=100, validation_data=(X_valid, y_valid), callbacks=[checkpoint_cb, early_stopping_cb])](https://image.slidesharecdn.com/cnn-200926193538/75/Build-your-own-Convolutional-Neural-Network-CNN-20-2048.jpg)
Uploaded byHichem Felouat
Build your own Convolutional Neural Network CNN
This document provides an overview of building and training a convolutional neural network (CNN) from scratch in Keras and TensorFlow. It discusses CNN architecture including convolutional layers, pooling layers, and fully connected layers. It also covers techniques for avoiding overfitting such as regularization, dropout, data augmentation, early stopping, and callbacks. The document concludes with instructions on how to save and load a trained CNN model.
Build your own Convolutional Neural Network CNN
- 1. Build your own ConvolutionalNeural Network (CNN) Keras & TensorFlow Hichem Felouat hichemfel@gmail.com
- 2. Hichem Felouat -hichemfel@gmail.com 2 Machine Learning VS Deep Learning
- 3. Hichem Felouat -hichemfel@gmail.com 3 CNN Architecture
- 4. Hichem Felouat -hichemfel@gmail.com 4 Convolutional Layers • Padding ? • Stride ?
- 5. Hichem Felouat -hichemfel@gmail.com 5 Convolutional Layers This architecture allows the network to concentrate on small low-level features in the first hidden layer, then assemble them into larger higher-level features in the next hidden layer, and so on. # padding : same or valid conv = keras.layers.Conv2D(filters=32, kernel_size=3, strides=1, padding="same", activation="relu")
- 6. Hichem Felouat -hichemfel@gmail.com 6 Convolutional Layers
- 7. Hichem Felouat -hichemfel@gmail.com 7 Pooling Layers • Average pooling ?
- 8. Hichem Felouat -hichemfel@gmail.com 8 Pooling Layers • Reduce the computational load, memory usage, and the number of parameters (limiting the risk of overfitting). • Max pooling layer also introduces some level of invariance to small translations. max_pool = keras.layers.MaxPool2D(pool_size=2)
- 9. Hichem Felouat -hichemfel@gmail.com 9 Pooling Layers
- 10. Hichem Felouat -hichemfel@gmail.com 10 Fully Connected Layers Hyperparameter Binary classification Multilabel binary classification Multiclass classification input neurons One per input feature One per input feature One per input feature hidden layers + neurons per hidden layer Depends on the problem Depends on the problem Depends on the problem output neurons 1 1 per label 1 per class Hidden activation ReLU (relu) ReLU (relu) ReLU (relu) Output layer activation Logistic (sigmoid) Logistic (sigmoid) Softmax (softmax) Loss function Cross entropy Cross entropy Cross entropy Typical classification model architecture Binary classification : binary_crossentropy - categorical_crossentropy Multiclass classification : sparse_categorical_crossentropy
- 11. Hichem Felouat -hichemfel@gmail.com 11 CNN
- 12. Hichem Felouat -hichemfel@gmail.com 12 Avoiding Overfitting 1) Try tuning model hyperparameters such as (the number of layers, the number of neurons per layer, and the types of activation functions to use for each hidden layer. 2) Try tuning other hyperparameters, such as the number of epochs and the batch size. 3) Reusing parts of a pretrained network (possibly built on an auxiliary task or using unsupervised learning). 4) class_weight
- 13. Hichem Felouat -hichemfel@gmail.com 13 Avoiding Overfitting • Applying a good initialization strategy for the connection weights • kernel_initializer="he_uniform" or "he_normal", "glorot_uniform" keras.layers.Dense(10, activation="relu", kernel_initializer="he_normal")
- 14. Hichem Felouat -hichemfel@gmail.com 14 Batch Normalization: This operation simply zero-centers and normalizes each input, then scales and shifts the result (before or after the activation function of each hidden layer). model = keras.models.Sequential([ keras.layers.Flatten(input_shape=[28, 28]), keras.layers.BatchNormalization(), keras.layers.Dense(300, activation="elu", kernel_initializer="he_normal"), keras.layers.BatchNormalization(), keras.layers.Dense(100, activation="elu", kernel_initializer="he_normal"), keras.layers.BatchNormalization(), keras.layers.Dense(10, activation="softmax") ]) Avoiding Overfitting
- 15. Hichem Felouat -hichemfel@gmail.com 15 Avoiding Overfitting Using faster optimizer (Gradient Descent, Momentum Optimization, Nesterov Accelerated Gradient, AdaGrad, RMSProp, Adam, Nadam).
- 16. Hichem Felouat -hichemfel@gmail.com 16 Avoiding Overfitting Update the optimizer’s learning_rate attribute at the beginning of each epoch: lr_scheduler = keras.callbacks.ReduceLROnPlateau(factor=0.5, patience=5) history = model.fit(X_train, y_train, [...], callbacks=[lr_scheduler])
- 17. Hichem Felouat -hichemfel@gmail.com 17 Avoiding Overfitting - ℓ1 and ℓ2 Regularization layer_h_n = keras.layers.Dense(100, activation="relu",kernel_initializer="he_normal", kernel_regularizer=keras.regularizers.l2(0.01)) layer_output = keras.layers.Dense(10, activation="softmax",kernel_initializer="glorot_uniform") model.add(layer_h_n) model.add(layer_output) • The l2() function returns a regularizer that will be called at each step during training to compute the regularization loss. This is then added to the final loss. • l1 : keras.regularizers.l1(0.01) • l1 + l2 : keras.regularizers.l1_l2()
- 18. Hichem Felouat -hichemfel@gmail.com 18 Avoiding Overfitting - Dropout model.add(keras.layers.Dropout(0.4))
- 19. Hichem Felouat -hichemfel@gmail.com 19 How to increase your small image dataset trainAug = ImageDataGenerator( rotation_range=40, width_shift_range=0.2, height_shift_range = 0.2, shear_range=0.2, zoom_range=0.2, horizontal_flip=True, fill_mode='nearest') model.compile(loss="binary_crossentropy", optimizer=opt, metrics=["accuracy"]) H = model.fit_generator( trainAug.flow(trainX, trainY, batch_size=BS), steps_per_epoch=len(trainX) // BS,validation_data=(testX, testY), validation_steps=len(testX) // BS, epochs=EPOCHS)
- 20. Hichem Felouat -hichemfel@gmail.com 20 Avoiding Overfitting - Using Callbacks checkpoint_cb = keras.callbacks.ModelCheckpoint ("my_keras_model.h5", save_best_only=True) history = model.fit(X_train, y_train, epochs=10, validation_data=(X_valid, y_valid), callbacks=[checkpoint_cb]) # roll back to best model model = keras.models.load_model("my_keras_model.h5") • You can combine both callbacks to save checkpoints of your model (in case your computer crashes) and interrupt training early when there is no more progress (to avoid wasting time and resources): early_stopping_cb = keras.callbacks.EarlyStopping(patience=10, restore_best_weights=True) history = model.fit(X_train, y_train, epochs=100, validation_data=(X_valid, y_valid), callbacks=[checkpoint_cb, early_stopping_cb])
- 21. Hichem Felouat -hichemfel@gmail.com 21 How to Save and Load Your Model Keras use the HDF5 format to save both the model’s architecture (including every layer’s hyperparameters) and the values of all the model parameters for every layer (e.g., connection weights and biases). It also saves the optimizer (including its hyperparameters and any state it may have). model.save("my_keras_model.h5") Loading the model: model = keras.models.load_model("my_keras_model.h5")
- 22. Hichem Felouat -hichemfel@gmail.com 22 Avoiding Overfitting - CNN
- 23. Hichem Felouat -hichemfel@gmail.com 23 Thank you for your attention Code link Brain Tumor Detection CNN kaggle https://www.kaggle.com/hichemfelouat/brain-tumor-detection-cnn-01 Github https://github.com/hichemfelouat/my-codes-of-machine-learning/blob/master/Brain%20Tumor%20Detection%20CNN.py