Applying Transfer Learning on ResNet using PyTorch
Recently I've written a Classifier that is able to distinguish dogs from cats.
Things I've learned:
- How to use a Pretrained Model
- Apply Transfer Learning
I've trained the model using Kaggle's classic dogs vs. cats dataset. The dataset can be found here
Well, I have to say it's not plug and play. At least not for the
torchvision has an
ImageFolder that's (right
now) the standard to load images as well as it's labels automatically.
The document suggests that
ImageFolder requires the dataset in the
root/dog/xxx.png root/dog/xxy.png root/dog/xxz.png root/cat/123.png root/cat/nsdf3.png root/cat/asd932_.png
So, I've arranged the dataset as the
ImageFolder requires me to.
The choice is arbitrary. I've considered some wired model. I don't know what to name it.
self.conv1 = nn.Conv2d(3, 8, 5) self.pool = nn.MaxPool(2, 2) self.conv2 = nn.Conv2d(8, 16, 3) self.conv3 = nn.Conv2d(16, 32, 3) self.conv4 = nn.Conv2d(32, 64, 3) self.conv5 = nn.Conv2d(64, 64, 3) self.fc1 = nn.Linear(64 * 5 * 5, 1000) self.fc2 = nn.Linear(1000, 500) self.fc3 = nn.Linear(500, 120) self.fc4 = nn.Linear(120, 84) self.fc5 = nn.Linear(84, 10) self.fc6 = nn.Linear(10, 2)
With ReLu everywhere and softmax layer at the end. It worked but not as I thought its gonna be. And its still smaller than starter ResNet which is ResNet18. So, I've considered ResNet18 over my network.
I've considered training my ResNet18 for the whole dataset. But the main problem is I don't have a GPU. (ノಠ益ಠ)ノ彡┻━┻
So Transfer Learning is the only choice I've got. I've opted for a pretrained Model which is trained on an ImageNet.
Choices I've got:
- Full Pretrained Model
- Fine-Tuning the ConvNet
- ConvNet as fixed feature extractor
I've got to tell you that using this is of no use. It just works. You don't have to do anything. You just have to write the predict function.
Taking the whole network and retraining the parameters just looked cool and painful too. Well, this one worked really good. The results were good too. It took so much time to train the model than I've thought it would take. I've trained the model but forgot to save it. I trained it another time though. But the time consumption was really too much.
Taking the whole network and adding a final layer and training just the last layer with softmax has done the job. The results are as expected. And also, It takes too less time compared to Full training and Fine-Tuning the ConvNet. The results are almost equal to the Fine-Tuning technique.
I've got more than 91% of accuracy.
I've tried some dog filter on praveen just to check whether it is able
to classify him correctly. Well, it has classified him as a
though. Look likes there's even more to do. ┬──┬ ノ( ゜-゜ノ)