CS230 final project. The instructor is Prof. Andrew Ng.
With the modern remote sensing technology, building and structural component images become accessible via aerial drones or related devices. In this project, the state-of-the-art deep learning technology for a civil engineering application is implemented, namely recognition of structural damage from images.
The big potential in deep learning application in structural engineering prompts the Kaggle Competition - ”PEER Hub ImageNet Challenge” held by University of California at Berkeley. Totally 8 classification tasks are raised.
We choose VGG-16 as our baseline model for transfer learning. The first step is to extract the bottleneck features and train one new-added layer with them (as in ”b”). Then, by releasing the higher level conv-blocks, we “fine-tune” the model to better fit each tasks (shown in “c”).
We tried different deep learning architectures and average the learning results at softmax layers to improve performance. The effect is significant for the tasks with smaller datasets (task 5, 6, 7 and 8).
The combination of modalities is found effective in improving the performance of small dataset. With in-depth hyperparameter tuning with VGG16 and Multi-model training, we rank at ~15% among other competitors. [Project poster]