Attention-based end-to-end CNN framework for content-based X-ray image retrieval

Abstract

The widespread use of medical imaging devices allows deep analysis of diseases. However, the task of examining medical images increases the burden of specialist doctors. Computer-assisted systems provide an effective management tool that enables these images to be analyzed automatically. Although these tools are used for various purposes, today, they are moving towards retrieval systems to access increasing data quickly. In hospitals, the need for content-based image retrieval systems is seriously evident in order to store all images effectively and access them quickly when necessary. In this study, an attention-based end-to-end convolutional neural network (CNN)framework that can provide effective access to similar images from a large X-ray dataset is presented. In the first part of the proposed framework, a fully convolutional network architecture with attention structures is presented. This section contains several layers for determining the saliency points of X-ray images. In the second part of the framework, the modified image with X-ray saliency map is converted to representative codes in Euclidean space by the ResNet-18 architecture. Finally, hash codes are obtained by transforming these codes into hamming spaces. The proposed study is superior in terms of high performance and customized layers compared to current state-of-the-art X-ray image retrieval methods in the literature. Extensive experimental studies reveal that the proposed framework can increase the current precision performance by up to 13