Spinal Lesions Classification and Localization with ACAT-Net from X-ray Images

X-ray images play an important role in the diagnosis of spinal diseases because of their convenient collection and easy observation. But it is time-consuming and challenging for radiologists to examine the differences between the vertebrae to diagnose abnormalities and locate lesions. Many existing methods try to extract the global features of radiographs and do not make full use of adjacent vertebrae variations. In this paper, we propose a novel Axial-aware neural network with Consecutive Attention Transformer (CAT), namely ACAT-Net, which takes advantage of the convolutional neural network and transformer as a new deep learning framework. A deep convolutional network extracts features of anteroposterior and lateral X-ray images that may have abnormalities in them. The consecutive attention transformer block is then used to focus on the morphological differences of axial adjacent vertebrae on the spines. The ingenious structure we designed can significantly reduce the amount of network parameters. Extensive experiments on clinical and public datasets show that our method is remarkably superior to other existing approaches in the spine X-ray image analysis.