Using deep learning for object recognition on hyperspectral data

Benefitting from the rapid expansion of consumer electronics (CE), plenty of new electronic products are introduced into the market annually. At the same time, due to the short life span of such products and the rapid emergence of new generations, large quantities of electronic waste are produced. The objective of this thesis is obtaining information on printed circuit boards (PCB) composition through non-invasive analysis using RGB (red, green and blue) and hyperspectral data to aid in recycling precious metals from recycled PCBs. The obtained information will be useful in more efficient processing of E-waste by harnessing deep learning networks. The goal of the thesis is to identify the surface mounted devices of a PCB such as the integrated circuits, connectors, capacitors, resistors etc. through the spatial information available in RGB images. The information obtained will be used in conjunction with HSI (Hyperspectral Images) to localise the detection area to recognise the composition of the PCB. We utilize spectral signatures of materials derived from HSI's and combining the spectral and spatial information to obtain a more precise recognition of the composition of a PCB. While hyperspectral data finds its use mostly in the remote sensing community to aid geological exploration, the application of this technology in the area of PCB recycling shows promise. The RGB images and HSI are collected by the multi-sensor system at the Helmholtz Institute Freiberg for Resource Technology (HIF) and supplements the training of the neural networks with the public data sets available from analogous research to localize the area of analysis for HSI based recognition. The neural network chosen for the application is the Faster Regional Convolutional Networks (Faster-RCNN) and we propose a guided anchoring method that utilizes Hyperspectral data to further improve the detection accuracy of the RGB based Faster RCNN, thereby including both spatial and spectral information of the PCBs. The results are then shown in comparison to demonstrate that the cross modal guided anchoring has a pronounced effect on the detection accuracy of the object detection network.