Comparing VGG Models

Machine Learning
Deep Learning
Computer Vision
Custom Implementation of VGG 1 Block, VGG 3 Block, VGG 16 and MLP 18 for a detailed comparative study.
Author

Soumyaratna Debnath

Published

April 19, 2023

GitHub

Code

# Importing the required libraries
from keras.preprocessing.image import ImageDataGenerator
from keras.models import Sequential
from keras.layers import Conv2D, MaxPooling2D, Flatten, Dense
from keras.callbacks import TensorBoard
import time
import csv
import os
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers
import numpy as np
import matplotlib.pyplot as plt
import shutil
2023-04-19 14:37:07.771981: I tensorflow/tsl/cuda/cudart_stub.cc:28] Could not find cuda drivers on your machine, GPU will not be used.
2023-04-19 14:37:07.807696: I tensorflow/tsl/cuda/cudart_stub.cc:28] Could not find cuda drivers on your machine, GPU will not be used.
2023-04-19 14:37:07.808495: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-04-19 14:37:08.701505: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT

Driver Code

print('Are you sure want to override previous report ', end='')
command = input()

if command == 'Yes':
    print('Overriding previous report')
    # Open the results file in write mode and adding the head
    with open('results.csv', mode='w', newline='') as results_file:
        results_writer = csv.writer(results_file)
        results_writer.writerow(['Model Name', 'Training Time', 'Train Loss', 'Train Acc', 'Test Acc', 'Num Params'])
    
    folders_to_remove = ["saved_models", "log_images", "log_stats"]
    for folder in folders_to_remove:
        if os.path.exists(folder):
            shutil.rmtree(folder)
            print(f"Directory {folder} removed.")
        else:
            print(f"Directory {folder} does not exist.")
Are you sure want to override previous report  Yes
Overriding previous report
Directory saved_models does not exist.
Directory log_images does not exist.
Directory log_stats does not exist.
# Define directories for training and testing data
train_dir = 'dataset/train/'
test_dir = 'dataset/test/'

# Define the image size to be used for resizing the images
img_size = (128, 128)

# Define the input image size (including the number of color channels)
input_img_size = (128, 128, 3)

# Define the batch size for training the model
batch_size = 20

# Define the number of epochs for training the model
num_epochs = 20
# Create an ImageDataGenerator object for data augmentation and normalization of training data
train_datagen = ImageDataGenerator(rescale=1./255)

# Create a generator for loading training data from the directory
train_generator = train_datagen.flow_from_directory(train_dir, 
                                                    target_size=img_size, # Resizes the images to a target size
                                                    batch_size=batch_size, # Defines the batch size
                                                    class_mode='binary') # Defines the type of labels to use

# Create an ImageDataGenerator object for normalization of testing data
test_datagen = ImageDataGenerator(rescale=1./255)

# Create a generator for loading testing data from the directory
test_generator = test_datagen.flow_from_directory(test_dir, 
                                                  target_size=img_size, # Resizes the images to a target size
                                                  batch_size=batch_size, # Defines the batch size
                                                  class_mode='binary') # Defines the type of labels to use

# Data generators for prediction
prediction_datagen = ImageDataGenerator(rescale=1./255)
preprocess_input = tf.keras.applications.vgg16.preprocess_input
prediction_datagen_vgg = keras.preprocessing.image.ImageDataGenerator(preprocessing_function=preprocess_input)
prediction_generator = test_datagen.flow_from_directory(test_dir, target_size=img_size, batch_size=1, class_mode='binary', shuffle=False) 
prediction_generator_vgg = prediction_datagen_vgg.flow_from_directory(test_dir, target_size=img_size, batch_size=1, class_mode='binary', shuffle=False) 
Found 160 images belonging to 2 classes.
Found 40 images belonging to 2 classes.
Found 40 images belonging to 2 classes.
Found 40 images belonging to 2 classes.
# Function for plotting the predictions and writing to TensorBoard
def plot_predictions(title, model, log_dir, test_generator):
    # Create a summary writer for TensorBoard
    writer = tf.summary.create_file_writer(log_dir)

    # Get the predicted classes for the test set
    y_pred = model.predict(test_generator)
    y_pred_classes = tf.round(y_pred).numpy().astype(int).flatten()

    # Get the true classes for the test set
    y_true = test_generator.classes.astype(int)

    # Get the class labels for the dataset
    class_labels = list(test_generator.class_indices.keys())

    # Get all the images and their corresponding labels from the test set generator
    images = []
    labels = []
    for i in range(len(test_generator)):
        batch = test_generator[i]
        images.extend(batch[0])
        labels.extend(batch[1].astype(int))

    for i in range(len(test_generator)):
        # Write the image to TensorBoard
        with tf.summary.create_file_writer(log_dir).as_default():
            tf.summary.image("{}   Image {}   Predicted: {}   True: {}".format(title, i+1, class_labels[y_pred_classes[i]], class_labels[labels[i]]), np.expand_dims(images[i], 0), step=0)

VGG 1 Block

# Define a function that creates a VGG block with one convolutional layer
def vgg_1_block():
    # Create a Sequential model object with a name
    model = Sequential(name = 'vgg_block_1')
    
    # Add a convolutional layer with 64 filters, a 3x3 kernel size, 'relu' activation, and 'same' padding
    model.add(Conv2D(64, (3, 3), activation='relu', padding='same', input_shape=input_img_size))
    
    # Add a max pooling layer with a 2x2 pool size
    model.add(MaxPooling2D((2, 2)))
    
    # Add a flatten layer to convert the 2D feature maps to a 1D feature vector
    model.add(Flatten())
    
    # Add a fully connected layer with 128 units and 'relu' activation
    model.add(Dense(128, activation='relu'))
    
    # Add an output layer with 1 unit and 'sigmoid' activation (for binary classification)
    model.add(Dense(1, activation='sigmoid'))
    
    # Return the model
    return model

# Create a VGG block with one convolutional layer
model1 = vgg_1_block()

# Print a summary of the model's architecture
model1.summary()

# Define a log directory for TensorBoard
log_dir = 'log_stats/vgg_1_block'

# Define the TensorBoard callback with update_freq='batch'
tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=log_dir, update_freq='batch')
Model: "vgg_block_1"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 conv2d (Conv2D)             (None, 128, 128, 64)      1792      
                                                                 
 max_pooling2d (MaxPooling2D  (None, 64, 64, 64)       0         
 )                                                               
                                                                 
 flatten (Flatten)           (None, 262144)            0         
                                                                 
 dense (Dense)               (None, 128)               33554560  
                                                                 
 dense_1 (Dense)             (None, 1)                 129       
                                                                 
=================================================================
Total params: 33,556,481
Trainable params: 33,556,481
Non-trainable params: 0
_________________________________________________________________
2023-04-19 14:38:27.877910: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1956] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
Skipping registering GPU devices...
# Compile the model with 'adam' optimizer, 'binary_crossentropy' loss function, and 'accuracy' metric
model1.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])

# Start timing the training
start_time = time.time()

# Train the model using the training generator for a specified number of epochs, and save the history
history = model1.fit(train_generator, steps_per_epoch=len(train_generator), epochs=num_epochs, callbacks=[tensorboard_callback])

# Stop timing the training
end_time = time.time()

# Calculate the training time by subtracting the start time from the end time
training_time = end_time - start_time

# Evaluate the model on the training set and get the training loss and accuracy
train_loss, train_acc = model1.evaluate(train_generator)

# Evaluate the model on the test set and get the test loss and accuracy
test_loss, test_acc = model1.evaluate(test_generator)

# Count the number of parameters in the model
num_params = model1.count_params()

# Open the results file in append mode and writing the results
with open('results.csv', mode='a', newline='') as results_file:
    results_writer = csv.writer(results_file)
    results_writer.writerow(['VGG 1 Block', training_time, train_loss, train_acc, test_acc, num_params])
    
model1.save('saved_models/vgg_1_block.h5')
Epoch 1/20
8/8 [==============================] - 4s 443ms/step - loss: 16.0090 - accuracy: 0.4938
Epoch 2/20
8/8 [==============================] - 4s 441ms/step - loss: 1.3641 - accuracy: 0.5063
Epoch 3/20
8/8 [==============================] - 4s 440ms/step - loss: 0.8025 - accuracy: 0.6000
Epoch 4/20
8/8 [==============================] - 4s 439ms/step - loss: 0.5900 - accuracy: 0.6875
Epoch 5/20
8/8 [==============================] - 4s 439ms/step - loss: 0.4382 - accuracy: 0.7750
Epoch 6/20
8/8 [==============================] - 4s 441ms/step - loss: 0.3841 - accuracy: 0.8687
Epoch 7/20
8/8 [==============================] - 4s 440ms/step - loss: 0.2853 - accuracy: 0.8875
Epoch 8/20
8/8 [==============================] - 4s 440ms/step - loss: 0.2322 - accuracy: 0.9438
Epoch 9/20
8/8 [==============================] - 4s 442ms/step - loss: 0.1883 - accuracy: 0.9563
Epoch 10/20
8/8 [==============================] - 4s 440ms/step - loss: 0.1287 - accuracy: 0.9812
Epoch 11/20
8/8 [==============================] - 4s 440ms/step - loss: 0.1186 - accuracy: 0.9875
Epoch 12/20
8/8 [==============================] - 4s 442ms/step - loss: 0.0991 - accuracy: 0.9625
Epoch 13/20
8/8 [==============================] - 4s 440ms/step - loss: 0.0737 - accuracy: 0.9812
Epoch 14/20
8/8 [==============================] - 4s 442ms/step - loss: 0.0589 - accuracy: 1.0000
Epoch 15/20
8/8 [==============================] - 4s 448ms/step - loss: 0.0466 - accuracy: 1.0000
Epoch 16/20
8/8 [==============================] - 4s 443ms/step - loss: 0.0412 - accuracy: 0.9937
Epoch 17/20
8/8 [==============================] - 4s 441ms/step - loss: 0.0302 - accuracy: 1.0000
Epoch 18/20
8/8 [==============================] - 4s 442ms/step - loss: 0.0279 - accuracy: 1.0000
Epoch 19/20
8/8 [==============================] - 4s 443ms/step - loss: 0.0257 - accuracy: 1.0000
Epoch 20/20
8/8 [==============================] - 4s 443ms/step - loss: 0.0187 - accuracy: 1.0000
1/8 [==>...........................] - ETA: 1s - loss: 0.0182 - accuracy: 1.00008/8 [==============================] - 1s 54ms/step - loss: 0.0206 - accuracy: 1.0000
2/2 [==============================] - 0s 58ms/step - loss: 0.5984 - accuracy: 0.8250
2023-04-19 14:38:31.364769: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'Placeholder/_0' with dtype int32
     [[{{node Placeholder/_0}}]]
2023-04-19 14:39:43.633980: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'Placeholder/_0' with dtype int32
     [[{{node Placeholder/_0}}]]
2023-04-19 14:39:44.234190: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'Placeholder/_0' with dtype int32
     [[{{node Placeholder/_0}}]]
plot_predictions('Vgg1', model1, 'log_images/vgg_1_block', prediction_generator)
 4/40 [==>...........................] - ETA: 0s40/40 [==============================] - 1s 20ms/step
2023-04-19 14:39:44.990430: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'Placeholder/_0' with dtype int32
     [[{{node Placeholder/_0}}]]

VGG 3 Block

# Define a function to create a VGG block with three convolutional layers
def vgg_3_block():
    # Create a Sequential model object with the name 'vgg_block_3'
    model = Sequential(name='vgg_block_3')
    
    # Add a convolutional layer with 64 filters, a kernel size of 3x3, 'same' padding, and ReLU activation,
    # and specify the input shape as the desired image size and 3 color channels
    model.add(Conv2D(64, (3, 3), activation='relu', padding='same', input_shape=input_img_size))
    
    # Add a max pooling layer with a pool size of 2x2
    model.add(MaxPooling2D((2, 2)))
    
    # Add another convolutional layer with 128 filters and 'same' padding
    model.add(Conv2D(128, (3, 3), activation='relu', padding='same'))
    
    # Add another max pooling layer
    model.add(MaxPooling2D((2, 2)))
    
    # Add a third convolutional layer with 256 filters and 'same' padding
    model.add(Conv2D(256, (3, 3), activation='relu', padding='same'))
    
    # Add a third max pooling layer
    model.add(MaxPooling2D((2, 2)))
    
    # Flatten the output of the convolutional layers
    model.add(Flatten())
    
    # Add a fully connected layer with 128 units and ReLU activation
    model.add(Dense(128, activation='relu'))
    
    # Add a final output layer with a single unit and sigmoid activation (for binary classification)
    model.add(Dense(1, activation='sigmoid'))
    
    # Return the completed model object
    return model

# Create an instance of the VGG block using the vgg_3_block function
model2 = vgg_3_block()

# Print a summary of the model's architecture
model2.summary()

# Define a log directory for TensorBoard
log_dir = 'log_stats/vgg_3_block'

# Define the TensorBoard callback with update_freq='batch'
tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=log_dir, update_freq='batch')
Model: "vgg_block_3"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 conv2d_1 (Conv2D)           (None, 128, 128, 64)      1792      
                                                                 
 max_pooling2d_1 (MaxPooling  (None, 64, 64, 64)       0         
 2D)                                                             
                                                                 
 conv2d_2 (Conv2D)           (None, 64, 64, 128)       73856     
                                                                 
 max_pooling2d_2 (MaxPooling  (None, 32, 32, 128)      0         
 2D)                                                             
                                                                 
 conv2d_3 (Conv2D)           (None, 32, 32, 256)       295168    
                                                                 
 max_pooling2d_3 (MaxPooling  (None, 16, 16, 256)      0         
 2D)                                                             
                                                                 
 flatten_1 (Flatten)         (None, 65536)             0         
                                                                 
 dense_2 (Dense)             (None, 128)               8388736   
                                                                 
 dense_3 (Dense)             (None, 1)                 129       
                                                                 
=================================================================
Total params: 8,759,681
Trainable params: 8,759,681
Non-trainable params: 0
_________________________________________________________________
# Compile the model with 'adam' optimizer, 'binary_crossentropy' loss function, and 'accuracy' metric
model2.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])

# Start timing the training
start_time = time.time()

# Train the model using the training generator for a specified number of epochs, and save the history
history = model2.fit(train_generator, steps_per_epoch=len(train_generator), epochs=num_epochs, callbacks=[tensorboard_callback])

# Stop timing the training
end_time = time.time()

# Calculate the training time by subtracting the start time from the end time
training_time = end_time - start_time

# Evaluate the model on the training set and get the training loss and accuracy
train_loss, train_acc = model2.evaluate(train_generator)

# Evaluate the model on the test set and get the test loss and accuracy
test_loss, test_acc = model2.evaluate(test_generator)

# Count the number of parameters in the model
num_params = model2.count_params()

# Open the results file in append mode and writing the results
with open('results.csv', mode='a', newline='') as results_file:
    results_writer = csv.writer(results_file)
    results_writer.writerow(['VGG 3 Block', training_time, train_loss, train_acc, test_acc, num_params])
    
model2.save('saved_models/vgg_3_block.h5')
Epoch 1/20
8/8 [==============================] - 4s 452ms/step - loss: 1.2802 - accuracy: 0.5250
Epoch 2/20
8/8 [==============================] - 4s 445ms/step - loss: 0.6984 - accuracy: 0.5437
Epoch 3/20
8/8 [==============================] - 4s 446ms/step - loss: 0.7064 - accuracy: 0.5000
Epoch 4/20
8/8 [==============================] - 4s 446ms/step - loss: 0.6933 - accuracy: 0.6125
Epoch 5/20
8/8 [==============================] - 4s 452ms/step - loss: 0.6818 - accuracy: 0.5437
Epoch 6/20
8/8 [==============================] - 4s 452ms/step - loss: 0.6226 - accuracy: 0.6562
Epoch 7/20
8/8 [==============================] - 4s 449ms/step - loss: 0.5584 - accuracy: 0.7250
Epoch 8/20
8/8 [==============================] - 4s 446ms/step - loss: 0.4817 - accuracy: 0.7625
Epoch 9/20
8/8 [==============================] - 4s 446ms/step - loss: 0.4709 - accuracy: 0.8188
Epoch 10/20
8/8 [==============================] - 4s 451ms/step - loss: 0.3938 - accuracy: 0.8000
Epoch 11/20
8/8 [==============================] - 4s 445ms/step - loss: 0.3187 - accuracy: 0.8438
Epoch 12/20
8/8 [==============================] - 4s 448ms/step - loss: 0.2332 - accuracy: 0.9062
Epoch 13/20
8/8 [==============================] - 4s 449ms/step - loss: 0.1594 - accuracy: 0.9438
Epoch 14/20
8/8 [==============================] - 4s 456ms/step - loss: 0.0972 - accuracy: 0.9688
Epoch 15/20
8/8 [==============================] - 4s 469ms/step - loss: 0.0486 - accuracy: 0.9812
Epoch 16/20
8/8 [==============================] - 4s 445ms/step - loss: 0.0302 - accuracy: 1.0000
Epoch 17/20
8/8 [==============================] - 4s 453ms/step - loss: 0.0142 - accuracy: 1.0000
Epoch 18/20
8/8 [==============================] - 4s 448ms/step - loss: 0.0089 - accuracy: 1.0000
Epoch 19/20
8/8 [==============================] - 4s 449ms/step - loss: 0.0033 - accuracy: 1.0000
Epoch 20/20
8/8 [==============================] - 4s 446ms/step - loss: 0.0023 - accuracy: 1.0000
8/8 [==============================] - 1s 78ms/step - loss: 0.0012 - accuracy: 1.0000
1/2 [==============>...............] - ETA: 0s - loss: 0.9936 - accuracy: 0.85002/2 [==============================] - 0s 79ms/step - loss: 1.0520 - accuracy: 0.8250
2023-04-19 14:39:46.379523: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'Placeholder/_0' with dtype int32
     [[{{node Placeholder/_0}}]]
2023-04-19 14:41:00.131167: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'Placeholder/_0' with dtype int32
     [[{{node Placeholder/_0}}]]
2023-04-19 14:41:00.947230: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'Placeholder/_0' with dtype int32
     [[{{node Placeholder/_0}}]]
plot_predictions('Vgg3', model2, 'log_images/vgg_3_block', prediction_generator)
 9/40 [=====>........................] - ETA: 0s40/40 [==============================] - 1s 14ms/step
2023-04-19 14:41:01.335855: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'Placeholder/_0' with dtype int32
     [[{{node Placeholder/_0}}]]

VGG 3 Block with Data Argumentation

# Define an ImageDataGenerator for data augmentation during training
train_datagen = ImageDataGenerator(
    rescale=1./255,                   # rescale pixel values to [0,1]
    rotation_range=45,                # random rotation between 0-45 degrees
    # width_shift_range=0.2,            # random shift horizontally up to 20% of the image width
    # height_shift_range=0.2,           # random shift vertically up to 20% of the image height
    shear_range=0.2,                  # random shear up to 20%
    zoom_range=0.2,                   # random zoom up to 20%
    # horizontal_flip=True,             # randomly flip images horizontally 
    fill_mode='nearest'               # fill any missing pixels with the nearest available pixel
)

# Create a flow of augmented training data from the training directory
train_generator = train_datagen.flow_from_directory(
    train_dir,                        # path to training data directory
    target_size=img_size,             # size of input images
    batch_size=batch_size,            # number of images per batch
    class_mode='binary'               # type of classification problem (binary or categorical)
)

# Define an ImageDataGenerator for rescaling pixel values in the test set
test_datagen = ImageDataGenerator(rescale=1./255)

# Create a flow of test data from the test directory
test_generator = test_datagen.flow_from_directory(
    test_dir,                         # path to test data directory
    target_size=img_size,             # size of input images
    batch_size=batch_size,            # number of images per batch
    class_mode='binary'               # type of classification problem (binary or categorical)
)
Found 160 images belonging to 2 classes.
Found 40 images belonging to 2 classes.
# Create an instance of the VGG block using the vgg_3_block function
model3 = vgg_3_block()

# Print a summary of the model's architecture
model3.summary()

# Define a log directory for TensorBoard
log_dir = 'log_stats/vgg_3_block_with_args'

# Define the TensorBoard callback with update_freq='batch'
tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=log_dir, update_freq='batch')
Model: "vgg_block_3"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 conv2d_4 (Conv2D)           (None, 128, 128, 64)      1792      
                                                                 
 max_pooling2d_4 (MaxPooling  (None, 64, 64, 64)       0         
 2D)                                                             
                                                                 
 conv2d_5 (Conv2D)           (None, 64, 64, 128)       73856     
                                                                 
 max_pooling2d_5 (MaxPooling  (None, 32, 32, 128)      0         
 2D)                                                             
                                                                 
 conv2d_6 (Conv2D)           (None, 32, 32, 256)       295168    
                                                                 
 max_pooling2d_6 (MaxPooling  (None, 16, 16, 256)      0         
 2D)                                                             
                                                                 
 flatten_2 (Flatten)         (None, 65536)             0         
                                                                 
 dense_4 (Dense)             (None, 128)               8388736   
                                                                 
 dense_5 (Dense)             (None, 1)                 129       
                                                                 
=================================================================
Total params: 8,759,681
Trainable params: 8,759,681
Non-trainable params: 0
_________________________________________________________________
# Compile the model with 'adam' optimizer, 'binary_crossentropy' loss function, and 'accuracy' metric
model3.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])

# Start timing the training
start_time = time.time()

# Train the model using the training generator for a specified number of epochs, and save the history
history = model3.fit(train_generator, steps_per_epoch=len(train_generator), epochs=num_epochs, callbacks=[tensorboard_callback])

# Stop timing the training
end_time = time.time()

# Calculate the training time by subtracting the start time from the end time
training_time = end_time - start_time

# Evaluate the model on the training set and get the training loss and accuracy
train_loss, train_acc = model3.evaluate(train_generator)

# Evaluate the model on the test set and get the test loss and accuracy
test_loss, test_acc = model3.evaluate(test_generator)

# Count the number of parameters in the model
num_params = model3.count_params()

# Open the results file in append mode and writing the results
with open('results.csv', mode='a', newline='') as results_file:
    results_writer = csv.writer(results_file)
    results_writer.writerow(['VGG 3 Block with Argumentation', training_time, train_loss, train_acc, test_acc, num_params])
    
model3.save('saved_models/vgg_3_block_with_args.h5')
Epoch 1/20
8/8 [==============================] - 4s 463ms/step - loss: 1.2631 - accuracy: 0.4812
Epoch 2/20
8/8 [==============================] - 4s 456ms/step - loss: 0.6834 - accuracy: 0.5750
Epoch 3/20
8/8 [==============================] - 4s 453ms/step - loss: 0.6574 - accuracy: 0.5813
Epoch 4/20
8/8 [==============================] - 4s 453ms/step - loss: 0.6267 - accuracy: 0.7375
Epoch 5/20
8/8 [==============================] - 4s 452ms/step - loss: 0.5854 - accuracy: 0.7125
Epoch 6/20
8/8 [==============================] - 4s 449ms/step - loss: 0.5650 - accuracy: 0.7500
Epoch 7/20
8/8 [==============================] - 4s 450ms/step - loss: 0.5137 - accuracy: 0.7875
Epoch 8/20
8/8 [==============================] - 4s 452ms/step - loss: 0.4996 - accuracy: 0.7812
Epoch 9/20
8/8 [==============================] - 4s 452ms/step - loss: 0.4974 - accuracy: 0.7688
Epoch 10/20
8/8 [==============================] - 4s 449ms/step - loss: 0.4721 - accuracy: 0.7688
Epoch 11/20
8/8 [==============================] - 4s 451ms/step - loss: 0.4361 - accuracy: 0.8250
Epoch 12/20
8/8 [==============================] - 4s 455ms/step - loss: 0.4523 - accuracy: 0.8125
Epoch 13/20
8/8 [==============================] - 4s 448ms/step - loss: 0.4220 - accuracy: 0.8125
Epoch 14/20
8/8 [==============================] - 4s 447ms/step - loss: 0.4201 - accuracy: 0.8000
Epoch 15/20
8/8 [==============================] - 4s 450ms/step - loss: 0.4078 - accuracy: 0.8125
Epoch 16/20
8/8 [==============================] - 4s 452ms/step - loss: 0.4350 - accuracy: 0.8375
Epoch 17/20
8/8 [==============================] - 4s 453ms/step - loss: 0.4067 - accuracy: 0.8125
Epoch 18/20
8/8 [==============================] - 4s 453ms/step - loss: 0.3679 - accuracy: 0.8813
Epoch 19/20
8/8 [==============================] - 4s 452ms/step - loss: 0.3514 - accuracy: 0.8313
Epoch 20/20
8/8 [==============================] - 4s 448ms/step - loss: 0.3851 - accuracy: 0.8188
8/8 [==============================] - 1s 109ms/step - loss: 0.3559 - accuracy: 0.8438
1/2 [==============>...............] - ETA: 0s - loss: 0.5282 - accuracy: 0.80002/2 [==============================] - 0s 80ms/step - loss: 0.5902 - accuracy: 0.7500
2023-04-19 14:41:02.555902: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'Placeholder/_0' with dtype int32
     [[{{node Placeholder/_0}}]]
2023-04-19 14:42:17.701998: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'Placeholder/_0' with dtype int32
     [[{{node Placeholder/_0}}]]
2023-04-19 14:42:18.761244: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'Placeholder/_0' with dtype int32
     [[{{node Placeholder/_0}}]]
plot_predictions('Vgg3_args', model3, 'log_images/vgg_3_block_with_args', prediction_generator)
 9/40 [=====>........................] - ETA: 0s40/40 [==============================] - 1s 14ms/step
2023-04-19 14:42:19.125891: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'Placeholder/_0' with dtype int32
     [[{{node Placeholder/_0}}]]

VGG 16 Transfer Learning

# Define the preprocessing function for VGG16 model
preprocess_input = tf.keras.applications.vgg16.preprocess_input

# Create a train data generator with the preprocessing function
train_datagen = keras.preprocessing.image.ImageDataGenerator(preprocessing_function=preprocess_input)

# Define the train generator by reading the images from the train directory
train_generator = train_datagen.flow_from_directory(
    train_dir,                        # path to training data directory
    target_size=img_size,             # size of input images
    batch_size=batch_size,            # number of images per batch
    class_mode='binary'               # type of classification problem (binary or categorical)
)

# Create a test data generator with the same preprocessing function as train generator
test_datagen = keras.preprocessing.image.ImageDataGenerator(preprocessing_function=preprocess_input)

# Define the test generator by reading the images from the test directory
test_generator = test_datagen.flow_from_directory(
    test_dir,                         # path to test data directory
    target_size=img_size,             # size of input images
    batch_size=batch_size,            # number of images per batch
    class_mode='binary'               # type of classification problem (binary or categorical)
)
Found 160 images belonging to 2 classes.
Found 40 images belonging to 2 classes.
def vgg_16_transfer_learning():
    # load model
    model = tf.keras.applications.vgg16.VGG16(include_top=False, input_shape=input_img_size)
    # mark loaded layers as not trainable
    for layer in model.layers:
        layer.trainable = False
    # add new classifier layers
    flat1 = Flatten()(model.layers[-1].output)
    class1 = Dense(128, activation='relu', kernel_initializer='he_uniform')(flat1)
    output = Dense(1, activation='sigmoid')(class1)
    # define new model
    model = keras.models.Model(inputs=model.inputs, outputs=output, name='vgg_16')
    return model

model4 = vgg_16_transfer_learning()

# Print a summary of the model's architecture
model4.summary()

# Define a log directory for TensorBoard
log_dir = 'log_stats/vgg_16'

# Define the TensorBoard callback with update_freq='batch'
tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=log_dir, update_freq='batch')
Model: "vgg_16"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 input_1 (InputLayer)        [(None, 128, 128, 3)]     0         
                                                                 
 block1_conv1 (Conv2D)       (None, 128, 128, 64)      1792      
                                                                 
 block1_conv2 (Conv2D)       (None, 128, 128, 64)      36928     
                                                                 
 block1_pool (MaxPooling2D)  (None, 64, 64, 64)        0         
                                                                 
 block2_conv1 (Conv2D)       (None, 64, 64, 128)       73856     
                                                                 
 block2_conv2 (Conv2D)       (None, 64, 64, 128)       147584    
                                                                 
 block2_pool (MaxPooling2D)  (None, 32, 32, 128)       0         
                                                                 
 block3_conv1 (Conv2D)       (None, 32, 32, 256)       295168    
                                                                 
 block3_conv2 (Conv2D)       (None, 32, 32, 256)       590080    
                                                                 
 block3_conv3 (Conv2D)       (None, 32, 32, 256)       590080    
                                                                 
 block3_pool (MaxPooling2D)  (None, 16, 16, 256)       0         
                                                                 
 block4_conv1 (Conv2D)       (None, 16, 16, 512)       1180160   
                                                                 
 block4_conv2 (Conv2D)       (None, 16, 16, 512)       2359808   
                                                                 
 block4_conv3 (Conv2D)       (None, 16, 16, 512)       2359808   
                                                                 
 block4_pool (MaxPooling2D)  (None, 8, 8, 512)         0         
                                                                 
 block5_conv1 (Conv2D)       (None, 8, 8, 512)         2359808   
                                                                 
 block5_conv2 (Conv2D)       (None, 8, 8, 512)         2359808   
                                                                 
 block5_conv3 (Conv2D)       (None, 8, 8, 512)         2359808   
                                                                 
 block5_pool (MaxPooling2D)  (None, 4, 4, 512)         0         
                                                                 
 flatten_3 (Flatten)         (None, 8192)              0         
                                                                 
 dense_6 (Dense)             (None, 128)               1048704   
                                                                 
 dense_7 (Dense)             (None, 1)                 129       
                                                                 
=================================================================
Total params: 15,763,521
Trainable params: 1,048,833
Non-trainable params: 14,714,688
_________________________________________________________________
# Compile the model with 'adam' optimizer, 'binary_crossentropy' loss function, and 'accuracy' metric
model4.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])

# Start timing the training
start_time = time.time()

# Train the model using the training generator for a specified number of epochs, and save the history
history = model4.fit(train_generator, steps_per_epoch=len(train_generator), epochs=num_epochs, callbacks=[tensorboard_callback])

# Stop timing the training
end_time = time.time()

# Calculate the training time by subtracting the start time from the end time
training_time = end_time - start_time

# Evaluate the model on the training set and get the training loss and accuracy
train_loss, train_acc = model4.evaluate(train_generator)

# Evaluate the model on the test set and get the test loss and accuracy
test_loss, test_acc = model4.evaluate(test_generator)

# Count the number of parameters in the model
num_params = model4.count_params()

# Open the results file in append mode and writing the results
with open('results.csv', mode='a', newline='') as results_file:
    results_writer = csv.writer(results_file)
    results_writer.writerow(['VGG 16', training_time, train_loss, train_acc, test_acc, num_params])
    
model4.save('saved_models/vgg_16_transfer_learning.h5')
Epoch 1/20
8/8 [==============================] - 3s 349ms/step - loss: 5.2201 - accuracy: 0.7875
Epoch 2/20
8/8 [==============================] - 3s 337ms/step - loss: 1.4520 - accuracy: 0.9250
Epoch 3/20
8/8 [==============================] - 3s 342ms/step - loss: 0.2833 - accuracy: 0.9625
Epoch 4/20
8/8 [==============================] - 3s 340ms/step - loss: 3.8766e-07 - accuracy: 1.0000
Epoch 5/20
8/8 [==============================] - 3s 335ms/step - loss: 2.5094e-04 - accuracy: 1.0000
Epoch 6/20
8/8 [==============================] - 3s 336ms/step - loss: 9.4699e-06 - accuracy: 1.0000
Epoch 7/20
8/8 [==============================] - 3s 335ms/step - loss: 5.3511e-07 - accuracy: 1.0000
Epoch 8/20
8/8 [==============================] - 3s 339ms/step - loss: 3.5025e-07 - accuracy: 1.0000
Epoch 9/20
8/8 [==============================] - 3s 336ms/step - loss: 2.1423e-07 - accuracy: 1.0000
Epoch 10/20
8/8 [==============================] - 3s 337ms/step - loss: 1.9476e-07 - accuracy: 1.0000
Epoch 11/20
8/8 [==============================] - 3s 337ms/step - loss: 1.4968e-07 - accuracy: 1.0000
Epoch 12/20
8/8 [==============================] - 3s 337ms/step - loss: 1.2451e-07 - accuracy: 1.0000
Epoch 13/20
8/8 [==============================] - 3s 335ms/step - loss: 1.1644e-07 - accuracy: 1.0000
Epoch 14/20
8/8 [==============================] - 3s 336ms/step - loss: 1.0194e-07 - accuracy: 1.0000
Epoch 15/20
8/8 [==============================] - 3s 337ms/step - loss: 9.6853e-08 - accuracy: 1.0000
Epoch 16/20
8/8 [==============================] - 3s 339ms/step - loss: 8.6054e-08 - accuracy: 1.0000
Epoch 17/20
8/8 [==============================] - 3s 338ms/step - loss: 7.8499e-08 - accuracy: 1.0000
Epoch 18/20
8/8 [==============================] - 3s 337ms/step - loss: 7.4105e-08 - accuracy: 1.0000
Epoch 19/20
8/8 [==============================] - 3s 337ms/step - loss: 6.9560e-08 - accuracy: 1.0000
Epoch 20/20
8/8 [==============================] - 3s 337ms/step - loss: 6.3877e-08 - accuracy: 1.0000
8/8 [==============================] - 3s 331ms/step - loss: 6.1388e-08 - accuracy: 1.0000
2/2 [==============================] - 1s 323ms/step - loss: 3.0647 - accuracy: 0.8500
2023-04-19 14:42:20.544130: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'Placeholder/_0' with dtype int32
     [[{{node Placeholder/_0}}]]
2023-04-19 14:43:16.201525: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'Placeholder/_0' with dtype int32
     [[{{node Placeholder/_0}}]]
2023-04-19 14:43:19.074663: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'Placeholder/_0' with dtype int32
     [[{{node Placeholder/_0}}]]
test_loss, test_acc = model4.evaluate(test_generator)
2023-04-19 14:43:19.915752: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'Placeholder/_0' with dtype int32
     [[{{node Placeholder/_0}}]]
2/2 [==============================] - 1s 332ms/step - loss: 3.0647 - accuracy: 0.8500
plot_predictions('Vgg16', model4, 'log_images/vgg_16', prediction_generator_vgg)
 4/40 [==>...........................] - ETA: 0s40/40 [==============================] - 1s 21ms/step
2023-04-19 14:43:20.693366: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'Placeholder/_0' with dtype int32
     [[{{node Placeholder/_0}}]]

MLP with 18 million trainable parameters

# Define the MLP model
def create_mlp_18():
    model = Sequential(name = 'MLP')
    model.add(Flatten(input_shape=input_img_size))
    model.add(Dense(256, activation='relu'))
    model.add(Dense(4096, activation='relu'))
    model.add(Dense(1024, activation='relu'))
    # model.add(Dropout(0.2))
    model.add(Dense(1, activation='sigmoid'))
    return model

model5 = create_mlp_18()
# Print the model summary to see the number of parameters
model5.summary()

# Define a log directory for TensorBoard
log_dir = 'log_stats/mlp_18'

# Define the TensorBoard callback with update_freq='batch'
tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=log_dir, update_freq='batch')
Model: "MLP"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 flatten_4 (Flatten)         (None, 49152)             0         
                                                                 
 dense_8 (Dense)             (None, 256)               12583168  
                                                                 
 dense_9 (Dense)             (None, 4096)              1052672   
                                                                 
 dense_10 (Dense)            (None, 1024)              4195328   
                                                                 
 dense_11 (Dense)            (None, 1)                 1025      
                                                                 
=================================================================
Total params: 17,832,193
Trainable params: 17,832,193
Non-trainable params: 0
_________________________________________________________________
# Compile the model with 'adam' optimizer, 'binary_crossentropy' loss function, and 'accuracy' metric
model5.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])

# Start timing the training
start_time = time.time()

# Train the model using the training generator for a specified number of epochs, and save the history
history = model5.fit(train_generator, steps_per_epoch=len(train_generator), epochs=num_epochs, callbacks=[tensorboard_callback])

# Stop timing the training
end_time = time.time()

# Calculate the training time by subtracting the start time from the end time
training_time = end_time - start_time

# Evaluate the model on the training set and get the training loss and accuracy
train_loss, train_acc = model5.evaluate(train_generator)

# Evaluate the model on the test set and get the test loss and accuracy
test_loss, test_acc = model5.evaluate(test_generator)

# Count the number of parameters in the model
num_params = model5.count_params()

# Open the results file in append mode and writing the results
with open('results.csv', mode='a', newline='') as results_file:
    results_writer = csv.writer(results_file)
    results_writer.writerow(['MLP18', training_time, train_loss, train_acc, test_acc, num_params])
    
model5.save('saved_models/mlp18.h5')
Epoch 1/20
8/8 [==============================] - 2s 181ms/step - loss: 828.1496 - accuracy: 0.5188
Epoch 2/20
8/8 [==============================] - 1s 181ms/step - loss: 16.2150 - accuracy: 0.5375
Epoch 3/20
8/8 [==============================] - 1s 180ms/step - loss: 15.6720 - accuracy: 0.7688
Epoch 4/20
8/8 [==============================] - 1s 177ms/step - loss: 12.2998 - accuracy: 0.7250
Epoch 5/20
8/8 [==============================] - 1s 179ms/step - loss: 5.2125 - accuracy: 0.7750
Epoch 6/20
8/8 [==============================] - 1s 180ms/step - loss: 0.6979 - accuracy: 0.7625
Epoch 7/20
8/8 [==============================] - 1s 180ms/step - loss: 0.4396 - accuracy: 0.8125
Epoch 8/20
8/8 [==============================] - 1s 177ms/step - loss: 0.2875 - accuracy: 0.8375
Epoch 9/20
8/8 [==============================] - 1s 179ms/step - loss: 0.2572 - accuracy: 0.8562
Epoch 10/20
8/8 [==============================] - 1s 176ms/step - loss: 0.2298 - accuracy: 0.8813
Epoch 11/20
8/8 [==============================] - 1s 176ms/step - loss: 0.2363 - accuracy: 0.8625
Epoch 12/20
8/8 [==============================] - 1s 176ms/step - loss: 0.2186 - accuracy: 0.8813
Epoch 13/20
8/8 [==============================] - 1s 178ms/step - loss: 0.2041 - accuracy: 0.8875
Epoch 14/20
8/8 [==============================] - 1s 182ms/step - loss: 0.1776 - accuracy: 0.8875
Epoch 15/20
8/8 [==============================] - 1s 180ms/step - loss: 0.1936 - accuracy: 0.8813
Epoch 16/20
8/8 [==============================] - 1s 181ms/step - loss: 0.1938 - accuracy: 0.8750
Epoch 17/20
8/8 [==============================] - 1s 178ms/step - loss: 0.2023 - accuracy: 0.8562
Epoch 18/20
8/8 [==============================] - 1s 178ms/step - loss: 0.2071 - accuracy: 0.8562
Epoch 19/20
8/8 [==============================] - 1s 179ms/step - loss: 0.2038 - accuracy: 0.8562
Epoch 20/20
8/8 [==============================] - 1s 177ms/step - loss: 0.2014 - accuracy: 0.8562
1/8 [==>...........................] - ETA: 1s - loss: 0.1485 - accuracy: 0.95008/8 [==============================] - 0s 34ms/step - loss: 0.1929 - accuracy: 0.8562
2/2 [==============================] - 0s 28ms/step - loss: 1.7975 - accuracy: 0.6750
2023-04-19 14:43:22.001196: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'Placeholder/_0' with dtype int32
     [[{{node Placeholder/_0}}]]
2023-04-19 14:43:51.992711: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'Placeholder/_0' with dtype int32
     [[{{node Placeholder/_0}}]]
2023-04-19 14:43:52.421763: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'Placeholder/_0' with dtype int32
     [[{{node Placeholder/_0}}]]
plot_predictions('MLP18', model5, 'log_images/mlp18', prediction_generator)
 6/40 [===>..........................] - ETA: 0s40/40 [==============================] - 1s 11ms/step
2023-04-19 14:43:53.022685: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'Placeholder/_0' with dtype int32
     [[{{node Placeholder/_0}}]]

MLP with 135 Million Trainable Parameters

# Define the MLP model
def create_mlp_135():
    model = Sequential(name = 'MLP')
    model.add(Flatten(input_shape=input_img_size))
    model.add(Dense(2500, activation='relu'))
    model.add(Dense(2500, activation='relu'))
    model.add(Dense(2500, activation='relu'))
    # model.add(Dropout(0.2))
    model.add(Dense(1, activation='sigmoid'))
    return model

model6 = create_mlp_135()
# Print the model summary to see the number of parameters
model6.summary()

# Define a log directory for TensorBoard
log_dir = 'log_stats/mlp_135'

# Define the TensorBoard callback with update_freq='batch'
tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=log_dir, update_freq='batch')
2023-04-19 14:55:06.216229: W tensorflow/tsl/framework/cpu_allocator_impl.cc:83] Allocation of 491520000 exceeds 10% of free system memory.
2023-04-19 14:55:06.543298: W tensorflow/tsl/framework/cpu_allocator_impl.cc:83] Allocation of 491520000 exceeds 10% of free system memory.
2023-04-19 14:55:06.622387: W tensorflow/tsl/framework/cpu_allocator_impl.cc:83] Allocation of 491520000 exceeds 10% of free system memory.
Model: "MLP"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 flatten_5 (Flatten)         (None, 49152)             0         
                                                                 
 dense_12 (Dense)            (None, 2500)              122882500 
                                                                 
 dense_13 (Dense)            (None, 2500)              6252500   
                                                                 
 dense_14 (Dense)            (None, 2500)              6252500   
                                                                 
 dense_15 (Dense)            (None, 1)                 2501      
                                                                 
=================================================================
Total params: 135,390,001
Trainable params: 135,390,001
Non-trainable params: 0
_________________________________________________________________
# Compile the model with 'adam' optimizer, 'binary_crossentropy' loss function, and 'accuracy' metric
model6.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])

# Start timing the training
start_time = time.time()

# Train the model using the training generator for a specified number of epochs, and save the history
history = model6.fit(train_generator, steps_per_epoch=len(train_generator), epochs=num_epochs, callbacks=[tensorboard_callback])

# Stop timing the training
end_time = time.time()

# Calculate the training time by subtracting the start time from the end time
training_time = end_time - start_time

# Evaluate the model on the training set and get the training loss and accuracy
train_loss, train_acc = model6.evaluate(train_generator)

# Evaluate the model on the test set and get the test loss and accuracy
test_loss, test_acc = model6.evaluate(test_generator)

# Count the number of parameters in the model
num_params = model6.count_params()

# Open the results file in append mode and writing the results
with open('results.csv', mode='a', newline='') as results_file:
    results_writer = csv.writer(results_file)
    results_writer.writerow(['MLP135', training_time, train_loss, train_acc, test_acc, num_params])
    
model5.save('saved_models/mlp135.h5')
Epoch 1/20
8/8 [==============================] - 11s 1s/step - loss: 8302.7051 - accuracy: 0.4750
Epoch 2/20
8/8 [==============================] - 10s 1s/step - loss: 871.6079 - accuracy: 0.6062
Epoch 3/20
8/8 [==============================] - 10s 1s/step - loss: 273.8789 - accuracy: 0.7188
Epoch 4/20
8/8 [==============================] - 10s 1s/step - loss: 134.2486 - accuracy: 0.7250
Epoch 5/20
8/8 [==============================] - 10s 1s/step - loss: 77.9153 - accuracy: 0.8062
Epoch 6/20
8/8 [==============================] - 10s 1s/step - loss: 115.3639 - accuracy: 0.8125
Epoch 7/20
8/8 [==============================] - 10s 1s/step - loss: 86.5102 - accuracy: 0.8813
Epoch 8/20
8/8 [==============================] - 10s 1s/step - loss: 5.3630 - accuracy: 0.9563
Epoch 9/20
8/8 [==============================] - 10s 1s/step - loss: 40.6399 - accuracy: 0.9187
Epoch 10/20
8/8 [==============================] - 10s 1s/step - loss: 8.1418 - accuracy: 0.9375
Epoch 11/20
8/8 [==============================] - 10s 1s/step - loss: 22.5314 - accuracy: 0.9375
Epoch 12/20
8/8 [==============================] - 10s 1s/step - loss: 50.7123 - accuracy: 0.9000
Epoch 13/20
8/8 [==============================] - 10s 1s/step - loss: 57.6497 - accuracy: 0.8813
Epoch 14/20
8/8 [==============================] - 10s 1s/step - loss: 34.0063 - accuracy: 0.9187
Epoch 15/20
8/8 [==============================] - 10s 1s/step - loss: 21.8980 - accuracy: 0.9187
Epoch 16/20
8/8 [==============================] - 10s 1s/step - loss: 0.7129 - accuracy: 0.9812
Epoch 17/20
8/8 [==============================] - 10s 1s/step - loss: 128.1349 - accuracy: 0.8875
Epoch 18/20
8/8 [==============================] - 10s 1s/step - loss: 12.1085 - accuracy: 0.9625
Epoch 19/20
8/8 [==============================] - 10s 1s/step - loss: 7.0070 - accuracy: 0.9750
Epoch 20/20
8/8 [==============================] - 10s 1s/step - loss: 36.5162 - accuracy: 0.9625
1/8 [==>...........................] - ETA: 1s - loss: 3.6694e-20 - accuracy: 1.00008/8 [==============================] - 0s 38ms/step - loss: 39.5354 - accuracy: 0.9812
2/2 [==============================] - 0s 49ms/step - loss: 298.6696 - accuracy: 0.7000
2023-04-19 14:55:09.682721: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'Placeholder/_0' with dtype int32
     [[{{node Placeholder/_0}}]]
2023-04-19 14:55:09.831306: W tensorflow/tsl/framework/cpu_allocator_impl.cc:83] Allocation of 491520000 exceeds 10% of free system memory.
2023-04-19 14:55:09.874048: W tensorflow/tsl/framework/cpu_allocator_impl.cc:83] Allocation of 491520000 exceeds 10% of free system memory.
2023-04-19 14:58:32.310921: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'Placeholder/_0' with dtype int32
     [[{{node Placeholder/_0}}]]
2023-04-19 14:58:32.793472: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'Placeholder/_0' with dtype int32
     [[{{node Placeholder/_0}}]]
plot_predictions('MLP135', model6, 'log_images/mlp135', prediction_generator)
 2/40 [>.............................] - ETA: 2s40/40 [==============================] - 2s 52ms/step
2023-04-19 14:58:33.204956: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'Placeholder/_0' with dtype int32
     [[{{node Placeholder/_0}}]]

Comparisons

import pandas as pd
df = pd.read_csv('results.csv', index_col=0)
display(df)
Training Time Train Loss Train Acc Test Acc Num Params
Model Name
VGG 1 Block 72.288201 2.056565e-02 1.00000 0.825 33556481
VGG 3 Block 73.744514 1.194572e-03 1.00000 0.825 8759681
VGG 3 Block with Argumentation 75.150163 3.559158e-01 0.84375 0.750 8759681
VGG 16 55.661988 6.138818e-08 1.00000 0.850 15763521
MLP18 29.990009 1.929377e-01 0.85625 0.675 17832193
MLP135 202.682255 3.953538e+01 0.98125 0.700 135390001

Are the results as expected? Why or why not?

VGG (1 Block)

In VGG (1 block) architecture, max-pooling and fully connected layers are placed after the single block of convolutional layers . Although this architecture has a limited number of parameters and trains quite quickly, it might not be able to recognise complicated aspects in the input.

VGG (3 blocks)

This architecture made up of three blocks of convolutional layers, and each block is followed by fully connected and max-pooling layers. This architecture can capture more complicated aspects in the data because it has more parameters than VGG (1 block). However, because there are more parameters, it takes longer to train than VGG (1 block).

VGG (3 blocks with data argumentation)

By adding different changes to the input images, such as rotation, scaling, and cropping, we inflate the size of the training set. By exposing the model to a larger range of input images when it is used with VGG (3 blocks), data augmentation can increase the model’s capacity to generalise to new data.

VGG 16 Tranfer Learning

When training a new model on a different dataset, transfer learning entails using a previously trained model as a starting point. Two pre-trained models that can be utilised for transfer learning are VGG16 and VGG19. These models have already learned to recognise a range of visual traits after being trained on massive datasets like ImageNet. For us, we used VGG16. We can reduce the amount of time and computational resources required to train a new model on a smaller dataset by starting with these pre-trained models.

Furthermore, when the target dataset is modest, transfer learning using VGG16 outperforms training a model from scratch in terms of performance.

MLP with 135 million parameters

The choice between using a MLP with 135 million parameters versus transfer learning with VGG16 depends on several factors, including the specific task, available resources, and the size of the dataset.

In general, using an MLP with 135 million parameters can potentially achieve higher accuracy compared to transfer learning with VGG16, especially for more complex tasks. However, training such a large MLP from scratch requires a significant amount of computational resources and time, making it impractical for many applications. Additionally, having such a large number of parameters increases the risk of overfitting, which can negatively impact model performance.

On the other hand, transfer learning with VGG16 can be a good choice for image classification tasks, especially if the dataset is small.

MLP with 18 million parameters

In general, an MLP with 135 million parameters is likely to have a higher capacity to model complex relationships within the data compared to an MLP with 18 million parameters. However, having a larger number of parameters does not necessarily guarantee better performance, and can increase the risk of overfitting, especially if the dataset is small.

On the other hand, an MLP with 18 million parameters is likely to require fewer computational resources and training time compared to an MLP with 135 million parameters. This can be beneficial, especially if the available resources are limited.

The number of layers in both MLPs being 3, it’s important to note that the depth of the neural network is also an important factor to consider. Increasing the depth of the network can help to model more complex relationships within the data. However, it can also increase the risk of vanishing or exploding gradients, making it more difficult to train the network.

Model Name Training Time Train Loss Train Acc Test Acc Num Params
VGG 1 Block 72.28820062 0.020565649 1 0.824999988 33556481
VGG 3 Block 73.74451399 0.001194572 1 0.824999988 8759681
VGG 3 Block with Argumentation 75.15016341 0.355915844 0.84375 0.75 8759681
VGG 16 55.6619885 6.14E-08 1 0.850000024 15763521
MLP18 29.99000883 0.192937687 0.856249988 0.675000012 17832193


Does data augmentation help? Why or why not?

Data augmentation is a method of creating additional training data from the existing datase by using various transformations including rotation, scaling, flipping, cropping, and other picture modificationst.

By doing this, the model is exposed to a wider variety of training data and can improve its ability to identify and generalise to fresh, unexplored data.

Data augmentation can reduce overfitting, which occurs when the model memorises the training data rather than generalising to new data, by expanding the quantity and diversity of the training dataset. This may enable the model to perform more effectively on the test data.

In other words, data augmentation makes the model more reliable and capable of identifying objects or patterns.

For us, the argumented model is working worse than the original one. One reason could be, less number of epochs to train with, perhaps may with with higher epoch, we will get better results.


Does it matter how many epochs you fine tune the model? Why or why not?

The term “number of epochs” refers to how many times the complete dataset was run through the model during training while fine-tuning a pre-trained model.

If the number of epoch is low, the model will underfit.

On the other side, if the number of epochs is too high , then model might begin to overfit to the training data, which would mean that it would memorise the training data rather than generalising to new data.

It is crucial to select the right number of epochs for fine-tuning in accordance with the difficulty of the task, the size of the dataset, and the unique properties of the model and data. Normally, this value is calculated by keeping track of the model’s performance on a validation set during training and halting when the validation accuracy stops increasing or begins to drop.

Even though we have not implemented it, but early stopping could be a good measure to utilize, when it comes to number of epochs.

Early stopping is a callback technique that can help prevent overfitting and save computational resources by stopping the training process before it completes all epochs. It can improve model performance, reduce training time, and help generalize the model better to unseen data.


Are there any particular images that the model is confused about? Why or why not?

This was the image for which each model got confused.