Use of machine learning techniques in the detection of wall defects in South African heritage structures

Abstract

South Africa has a storied past, with remnants of it remaining in heritage structures such as the Castle of Good Hope and Robben Island. These heritage structures preserve the country’s past while also showing how it has progressed over time. Heritage structures play a role in the country's economy, contributing to both the tourism industry and job creation. Despite their importance, they are still prone to deterioration due to various factors and require monitoring to ensure their structural integrity. Currently, it is common practice to carry out monitoring through on site visual inspections. However, this has been found to produce inconsistent results while also incurring high costs due to the expertise required. This research therefore explored the use of machine learning techniques to classify wall defects in South African heritage structures. For this, machine learning and deep learning algorithms were used to classify crack, spall and intact areas. These algorithms included decision trees, support vector machines, k-nearest neighbours, artificial neural networks and a pre-trained AlexNet convolutional neural network (CNN) model. Additionally, the research focused on the use of image processing to determine the extent and characteristics of crack damage. Initially, a dataset of images made up of crack, spall and intact areas was collected from the Castle of Good Hope, and buildings on Robben Island and in Dal Josafat, Paarl. The Bag of Visual Words technique was then used to extract features from these images. Feature extraction was carried out using the speeded-up robust features (SURF) function, with k-means clustering subsequently carried out to create the visual vocabulary. Models were then trained to learn patterns and recognise the different damage types using the histogram of visual words obtained from the Bag of Visual Words as input. Models were then tested on a separate dataset where their accuracy, misclassification rate, precision and recall were obtained. A different approach was used to train and test the pre-trained AlexNet CNN model. For this, a transfer learning strategy was applied whereby the CNN’s output was modified. This was completed by adding a new specialised fully connected layer. The model's output layer weights were then fine-tuned during the training phase, after which testing was carried out to obtain the model's accuracy, misclassification rate, precision and recall. When comparing the results of the different models, it was observed that the CNN model produced the best results in terms of accuracy, precision and recall, obtaining 97.40% for each metric. Furthermore, the model also achieved a misclassification rate of 2.60% which was the lowest among all models. The performance of the model can be attributed to the specialised convolutional and pooling layers that the model processes, as well as the fact that the model was pre-trained, where prior learned information could be leveraged for this classification task. To determine the extent of crack damage in heritage structures, image processing techniques such as thresholding, binarisation, segmentation, skeletonisation and the Euclidean distance transform were used to extract measurements of the average crack width and the crack length. These techniques were used on a set of crack images collected from the Castle of Good Hope where the results obtained were then compared to the on site measurements. When comparing the average crack width measurements an average error of 10.73% was obtained. The comparison of crack length measurements was inconclusive as the image processing methodology produced high error rates which varied across all cracks. The error rates obtained can be attributed to various factors such as the image quality, limitations of the image processing techniques applied, sensitivity of the skeletonisation function and excessive noise in images. Although results varied in terms of the extraction of crack characteristics using image processing, the machine learning and deep learning models produced results which show their potential to be used as resources in the structural health monitoring field. Furthermore, their integration into structural health monitoring provides an alternative tool for engineers and heritage experts which can be used in hand with non-destructive data, sensor data and even images to allow for better and more precise decisions in terms of the conservation, restoration and repair and maintenance of heritage structures.