Diver Action Recognition and Automated Surface Inspection using Enhanced Deep Transfer Learning Approach

With the growth of sensing, control and robotic technologies, robots have become useful assistants to humans in diverse application domains such as household, manufacturing, aerospace, underwater, and defense. In particular this thesis focuses on two applications: 1) autonomous vehicles for underwater tasks and 2) automated surface inspection in aerospace industries. The objective of the first application is to enable diver action recognition for efficient diver-AUV interaction. On the other hand, the objective of the second application is to enable automated surface texture recognition for aircraft lip skins. Both these applications deal with machine learning to detect and recognize various classes in a robust, reliable and computationally efficient manner. These classification problems face similar yet different challenges of uneven lighting, complex backgrounds and sensor uncertainties. In this regard, for the diver action recognition problem, this thesis constructed a multichannel convolutional neural network with a hierarchical tree-structured deep neural network classifier for fusion of stereo-pairs of AUV camera images and classification of diver actions. The implementation on real experimental data collected in ocean environments reveal that the proposed method supersedes the performance of existing deep learning architectures in terms of overall as well as individual class accuracies and F1-scores. On the other hand, for the automated metallic surface inspection problem, a deep transfer learning architecture with several image preprocessing steps is implemented for classification of conforming vs non-conforming surfaces. The results obtained on real experimental data reveal that the surfaces are classified correctly with high accuracy and low missed detection and false alarm rates.