Week 3: Transfer Learning
Welcome to this assignment! This week, you are going to use a technique called Transfer Learning in which you utilize an already trained network to help you solve a similar problem to the one it was originally trained to solve.
Let's get started!
NOTE: To prevent errors from the autograder, please avoid editing or deleting non-graded cells in this notebook . Please only put your solutions in between the ### START CODE HERE and ### END CODE HERE code comments, and refrain from adding any new cells.
# grader-required-cell
import os
import zipfile
import matplotlib.pyplot as plt
import tensorflow as tf
from tensorflow.keras import layers
from tensorflow.keras import Model
from tensorflow.keras.optimizers import RMSprop
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras.utils import img_to_array, load_imgDataset
For this assignment, you will use the Horse or Human dataset, which contains images of horses and humans.
Download the training and validation sets by running the cell below:
# Get the Horse or Human training dataset
!wget -q -P /content/ https://storage.googleapis.com/tensorflow-1-public/course2/week3/horse-or-human.zip
# Get the Horse or Human validation dataset
!wget -q -P /content/ https://storage.googleapis.com/tensorflow-1-public/course2/week3/validation-horse-or-human.zip
test_local_zip = './horse-or-human.zip'
zip_ref = zipfile.ZipFile(test_local_zip, 'r')
zip_ref.extractall('/tmp/training')
val_local_zip = './validation-horse-or-human.zip'
zip_ref = zipfile.ZipFile(val_local_zip, 'r')
zip_ref.extractall('/tmp/validation')
zip_ref.close()This dataset already has an structure that is compatible with Keras' flow_from_directory so you don't need to move the images into subdirectories as you did in the previous assignments. However, it is still a good idea to save the paths of the images so you can use them later on:
# grader-required-cell
# Define the training and validation base directories
train_dir = '/tmp/training'
validation_dir = '/tmp/validation'
# Directory with training horse pictures
train_horses_dir = os.path.join(train_dir, 'horses')
# Directory with training humans pictures
train_humans_dir = os.path.join(train_dir, 'humans')
# Directory with validation horse pictures
validation_horses_dir = os.path.join(validation_dir, 'horses')
# Directory with validation human pictures
validation_humans_dir = os.path.join(validation_dir, 'humans')
# Check the number of images for each class and set
print(f"There are {len(os.listdir(train_horses_dir))} images of horses for training.\n")
print(f"There are {len(os.listdir(train_humans_dir))} images of humans for training.\n")
print(f"There are {len(os.listdir(validation_horses_dir))} images of horses for validation.\n")
print(f"There are {len(os.listdir(validation_humans_dir))} images of humans for validation.\n")
'''
There are 500 images of horses for training.
There are 527 images of humans for training.
There are 128 images of horses for validation.
There are 128 images of humans for validation.
'''Now take a look at a sample image of each one of the classes:
# grader-required-cell
print("Sample horse image:")
plt.imshow(load_img(f"{os.path.join(train_horses_dir, os.listdir(train_horses_dir)[0])}"))
plt.show()
print("\nSample human image:")
plt.imshow(load_img(f"{os.path.join(train_humans_dir, os.listdir(train_humans_dir)[0])}"))
plt.show()
matplotlib makes it easy to see that these images have a resolution of 300x300 and are colored, but you can double check this by using the code below:
# grader-required-cell
# Load the first example of a horse
sample_image = load_img(f"{os.path.join(train_horses_dir, os.listdir(train_horses_dir)[0])}")
# Convert the image into its numpy array representation
sample_array = img_to_array(sample_image)
print(f"Each image has shape: {sample_array.shape}")
# Each image has shape: (300, 300, 3)As expected, the sample image has a resolution of 300x300 and the last dimension is used for each one of the RGB channels to represent color.
Training and Validation Generators
Now that you know the images you are dealing with, it is time for you to code the generators that will fed these images to your Network. For this, complete the train_val_generators function below:
Important Note: The images have a resolution of 300x300 but the flow_from_directory method you will use allows you to set a target resolution. In this case, set a target_size of (150, 150). This will heavily lower the number of trainable parameters in your final network, yielding much quicker training times without compromising the accuracy!
# grader-required-cell
# GRADED FUNCTION: train_val_generators
def train_val_generators(TRAINING_DIR, VALIDATION_DIR):
"""
Creates the training and validation data generators
Args:
TRAINING_DIR (string): directory path containing the training images
VALIDATION_DIR (string): directory path containing the testing/validation images
Returns:
train_generator, validation_generator: tuple containing the generators
"""
### START CODE HERE
# Instantiate the ImageDataGenerator class
# Don't forget to normalize pixel values and set arguments to augment the images
train_datagen = ImageDataGenerator(rescale=1/255,
rotation_range=40,
width_shift_range=0.2,
height_shift_range=0.2,
shear_range=0.2,
zoom_range=0.2,
horizontal_flip=True,
fill_mode='nearest')
# Pass in the appropriate arguments to the flow_from_directory method
train_generator = train_datagen.flow_from_directory(directory=TRAINING_DIR,
batch_size=32,
class_mode='binary',
target_size=(150, 150))
# Instantiate the ImageDataGenerator class (don't forget to set the rescale argument)
# Remember that validation data should not be augmented
validation_datagen = ImageDataGenerator(rescale=1/255)
# Pass in the appropriate arguments to the flow_from_directory method
validation_generator = validation_datagen.flow_from_directory(directory=VALIDATION_DIR,
batch_size=32,
class_mode='binary',
target_size=(150, 150))
### END CODE HERE
return train_generator, validation_generator# grader-required-cell
# Test your generators
train_generator, validation_generator = train_val_generators(train_dir, validation_dir)Expected Output:
Found 1027 images belonging to 2 classes. Found 256 images belonging to 2 classes.
Transfer learning - Create the pre-trained model
Download the inception V3 weights into the /tmp/ directory:
# Download the inception v3 weights
!wget --no-check-certificate \
https://storage.googleapis.com/mledu-datasets/inception_v3_weights_tf_dim_ordering_tf_kernels_notop.h5 \
-O /tmp/inception_v3_weights_tf_dim_ordering_tf_kernels_notop.h5Now load the InceptionV3 model and save the path to the weights you just downloaded:
# grader-required-cell
# Import the inception model
from tensorflow.keras.applications.inception_v3 import InceptionV3
# Create an instance of the inception model from the local pre-trained weights
local_weights_file = '/tmp/inception_v3_weights_tf_dim_ordering_tf_kernels_notop.h5'Complete the create_pre_trained_model function below. You should specify the correct input_shape for the model (remember that you set a new resolution for the images instead of the native 300x300) and make all of the layers non-trainable:
# grader-required-cell
# GRADED FUNCTION: create_pre_trained_model
def create_pre_trained_model(local_weights_file):
"""
Initializes an InceptionV3 model.
Args:
local_weights_file (string): path pointing to a pretrained weights H5 file
Returns:
pre_trained_model: the initialized InceptionV3 model
"""
### START CODE HERE
pre_trained_model = InceptionV3(input_shape = (150, 150, 3),
include_top = False,
weights = None)
pre_trained_model.load_weights(local_weights_file)
# Make all the layers in the pre-trained model non-trainable
for layer in pre_trained_model.layers:
layer.trainable = False
### END CODE HERE
return pre_trained_modelCheck that everything went well by comparing the last few rows of the model summary to the expected output:
# grader-required-cell
pre_trained_model = create_pre_trained_model(local_weights_file)
# Print the model summary
pre_trained_model.summary()Expected Output:
batch_normalization_v1_281 (Bat (None, 3, 3, 192) 576 conv2d_281[0][0]
activation_273 (Activation) (None, 3, 3, 320) 0 batch_normalization_v1_273[0][0]
mixed9_1 (Concatenate) (None, 3, 3, 768) 0 activation_275[0][0]
activation_276[0][0]
concatenate_5 (Concatenate) (None, 3, 3, 768) 0 activation_279[0][0]
activation_280[0][0]
activation_281 (Activation) (None, 3, 3, 192) 0 batch_normalization_v1_281[0][0]
mixed10 (Concatenate) (None, 3, 3, 2048) 0 activation_273[0][0]
mixed9_1[0][0]
concatenate_5[0][0]
activation_281[0][0]
===========================================================================================
Total params: 21,802,784
Trainable params: 0
Non-trainable params: 21,802,784
To check that all the layers in the model were set to be non-trainable, you can also run the cell below:
# grader-required-cell
total_params = pre_trained_model.count_params()
num_trainable_params = sum([w.shape.num_elements() for w in pre_trained_model.trainable_weights])
print(f"There are {total_params:,} total parameters in this model.")
print(f"There are {num_trainable_params:,} trainable parameters in this model.")Expected Output:
There are 21,802,784 total parameters in this model. There are 0 trainable parameters in this model.
Creating callbacks for later
You have already worked with callbacks in the first course of this specialization so the callback to stop training once an accuracy of 99.9% is reached, is provided for you:
# grader-required-cell
# Define a Callback class that stops training once accuracy reaches 99.9%
class myCallback(tf.keras.callbacks.Callback):
def on_epoch_end(self, epoch, logs={}):
if(logs.get('accuracy')>0.999):
print("\nReached 99.9% accuracy so cancelling training!")
self.model.stop_training = TruePipelining the pre-trained model with your own
Now that the pre-trained model is ready, you need to "glue" it to your own model to solve the task at hand.
For this you will need the last output of the pre-trained model, since this will be the input for your own. Complete the output_of_last_layer function below.
Note: For grading purposes use the mixed7 layer as the last layer of the pre-trained model. However, after submitting feel free to come back here and play around with this.
# grader-required-cell
# GRADED FUNCTION: output_of_last_layer
def output_of_last_layer(pre_trained_model):
"""
Gets the last layer output of a model
Args:
pre_trained_model (tf.keras Model): model to get the last layer output from
Returns:
last_output: output of the model's last layer
"""
### START CODE HERE
last_desired_layer = pre_trained_model.get_layer('mixed7')
print('last layer output shape: ', last_desired_layer.output_shape)
last_output = last_desired_layer.output
print('last layer output: ', last_output)
### END CODE HERE
return last_outputCheck that everything works as expected:
# grader-required-cell
last_output = output_of_last_layer(pre_trained_model)Expected Output (if mixed7 layer was used):
last layer output shape: (None, 7, 7, 768)
last layer output: KerasTensor(type_spec=TensorSpec(shape=(None, 7, 7, 768), dtype=tf.float32, name=None), name='mixed7/concat:0', description="created by layer 'mixed7'")
Now you will create the final model by adding some additional layers on top of the pre-trained model.
Complete the create_final_model function below. You will need to use Tensorflow's Functional API for this since the pretrained model has been created using it.
Let's double check this first:
# grader-required-cell
# GRADED FUNCTION: create_final_model
def create_final_model(pre_trained_model, last_output):
"""
Appends a custom model to a pre-trained model
Args:
pre_trained_model (tf.keras Model): model that will accept the train/test inputs
last_output (tensor): last layer output of the pre-trained model
Returns:
model: the combined model
"""
# Flatten the output layer to 1 dimension
x = layers.Flatten()(last_output)
### START CODE HERE
# Add a fully connected layer with 1024 hidden units and ReLU activation
x = layers.Dense(1024, activation='relu')(x)
# Add a dropout rate of 0.2
x = layers.Dropout(0.2)(x)
# Add a final sigmoid layer for classification
x = layers.Dense(1, activation='sigmoid')(x)
# Create the complete model by using the Model class
model = Model(inputs=pre_trained_model.input, outputs=x)
# Compile the model
model.compile(optimizer = RMSprop(learning_rate=0.0001),
loss = 'binary_crossentropy',
metrics = ['accuracy'])
### END CODE HERE
return model# grader-required-cell
# Save your model in a variable
model = create_final_model(pre_trained_model, last_output)
# Inspect parameters
total_params = model.count_params()
num_trainable_params = sum([w.shape.num_elements() for w in model.trainable_weights])
print(f"There are {total_params:,} total parameters in this model.")
print(f"There are {num_trainable_params:,} trainable parameters in this model.")Expected Output:
There are 47,512,481 total parameters in this model.
There are 38,537,217 trainable parameters in this model.
Wow, that is a lot of parameters!
After submitting your assignment later, try re-running this notebook but use the original resolution of 300x300, you will be surprised to see how many more parameters are for that case.
Now train the model:
# Run this and see how many epochs it should take before the callback
# fires, and stops training at 99.9% accuracy
# (It should take a few epochs)
callbacks = myCallback()
history = model.fit(train_generator,
validation_data = validation_generator,
epochs = 100,
verbose = 2,
callbacks=callbacks)
'''
Epoch 1/100
33/33 - 18s - loss: 0.2003 - accuracy: 0.9241 - val_loss: 0.0368 - val_accuracy: 0.9805 - 18s/epoch - 553ms/step
Epoch 2/100
33/33 - 12s - loss: 0.0346 - accuracy: 0.9864 - val_loss: 0.0072 - val_accuracy: 0.9961 - 12s/epoch - 368ms/step
Epoch 3/100
33/33 - 12s - loss: 0.0345 - accuracy: 0.9873 - val_loss: 0.0269 - val_accuracy: 0.9922 - 12s/epoch - 366ms/step
Epoch 4/100
33/33 - 12s - loss: 0.0182 - accuracy: 0.9942 - val_loss: 0.0547 - val_accuracy: 0.9805 - 12s/epoch - 368ms/step
Epoch 5/100
Reached 99.9% accuracy so cancelling training!
33/33 - 12s - loss: 0.0067 - accuracy: 0.9990 - val_loss: 0.0015 - val_accuracy: 1.0000 - 12s/epoch - 367ms/step
'''
The training should have stopped after less than 10 epochs and it should have reached an accuracy over 99,9% (firing the callback). This happened so quickly because of the pre-trained model you used, which already contained information to classify humans from horses. Really cool!
Now take a quick look at the training and validation accuracies for each epoch of training:
# Plot the training and validation accuracies for each epoch
acc = history.history['accuracy']
val_acc = history.history['val_accuracy']
loss = history.history['loss']
val_loss = history.history['val_loss']
epochs = range(len(acc))
plt.plot(epochs, acc, 'r', label='Training accuracy')
plt.plot(epochs, val_acc, 'b', label='Validation accuracy')
plt.title('Training and validation accuracy')
plt.legend(loc=0)
plt.figure()
plt.show()
Download your notebook for grading
You will need to submit your solution notebook for grading. The following code cells will check if this notebook's grader metadata (i.e. hidden data in the notebook needed for grading) is not modified by your workspace. This will ensure that the autograder can evaluate your code properly. Depending on its output, you will either:
- if the metadata is intact: Download the current notebook. Click on the File tab on the upper left corner of the screen then click on Download -> Download .ipynb. You can name it anything you want as long as it is a valid .ipynb (jupyter notebook) file.
- if the metadata is missing: A new notebook with your solutions will be created on this Colab workspace. It should be downloaded automatically and you can submit that to the grader.
# Download metadata checker
!wget -nc https://storage.googleapis.com/tensorflow-1-public/colab_metadata_checker.pyimport colab_metadata_checker
# Please see the output of this cell to see which file you need to submit to the grader
colab_metadata_checker.run('C2W3_Assignment_fixed.ipynb')Please disregard the following note if the notebook metadata is detected
Note: Just in case the download fails for the second point above, you can also do these steps:
- Click the Folder icon on the left side of this screen to open the File Manager.
- Click the Folder Refresh icon in the File Manager to see the latest files in the workspace. You should see a file ending with a _fixed.ipynb.
- Right-click on that file to save locally and submit it to the grader.
Congratulations on finishing this week's assignment!
You have successfully implemented a convolutional neural network that leverages a pre-trained network to help you solve the problem of classifying humans from horses.
Keep it up!