Hydrophobicity-based Grading of Industrial Composite Insulators Images using Cross Attention Vision Transformer with Knowledge Distillation

The hydrophobic property of composite insulators is a crucial attribute that prevents the accumulation of significant water content on the surface and ensures smooth operation and prevention of power line surges. However, since the weathering effect gradually reduces the hydrophobicity, periodic monitoring is necessary to ensure the smooth operation of the powerline. Efficient machine learning-based analysis of the composite insulator (CI) images collected by the spray method can potentially achieve very high efficacy. However, inconsistency in the shape and size of droplets, the intricate pattern of the droplets, the lack of color variation of the images, etc. pose a challenge to the prevalent computer vision techniques for categorizing the CIs. This treatise explored a novel, efficient deep learning paradigm, termed cross attention vision transformer (CA-ViT) for grading the CI images. The CA-ViT uses both small and large patches to blend spatial features corresponding to a different scale. The network uses a token fusion module to effectively combine the tokens obtained from small and large patches and understand the visual pattern, and characterize droplets of different shapes and sizes by exchanging cross-attentions, which demonstrates its ability to precisely detect small and large droplets, and accurately classify the droplet images. Further, the work also introduces a knowledge distillation strategy to reduce the computational run time of the model. Exhaustive experimental results have confirmed that our proposed work surpasses state-of-the-art methods.