Research on line feature detection and matching method of mine blurred image by fusing event spike tensor

Aiming at the problem of image feature degradation caused by motion blur in the complex environment of underground coal mine, a fusion Event Spike Tensor (EST) is proposed as a line feature matching method for blur images of mine. First, a cross-modal spatiotemporal representation is built via data-layer fusion of EST and RGB images. Using a Multilayer Perceptron (MLP) to encode the timestamp and polarity information of the event stream, generating a four-dimensional tensor representation with spatiotemporal features. Upon this, a feature-fusion backbone combining a shallow fusion module and an hourglass module is employed. Weighted fusion of the dual-modal features is achieved by Channel-Prior Convolutional Attention (CPCA), while a lightweight multi-head self-attention mechanism captures global relations. Robust line detection under motion blur is finally realized with the Unified Line Segment Detection (ULSD). Secondly, a Graph Neural Network (GNN) matching model is constructed, employing SuperPoint to detect keypoints and their descriptors, which together with line segments form the nodes of the graph structure, enhancing the feature representation within a single image through the self-attention mechanism and establishing cross-image feature associations by using the cross-attention mechanism, and introducing the line segment information transfer module to reinforcement of geometric constraints. Finally, the effectiveness of the method is verified through experiments simulating three typical underground scenarios, namely transport roadways, digging faces and fully mechanized mining faces. The results show that: compared with LineTR algorithm, the proposed method improves the number of line segment detection by 36.7% on average, and reduces the distribution uniformity density variance by 77.5% on average; compared with LineTR and GlueStick methods, the mis-match rate of the proposed method is reduced to 0.7%‒2.7%, and the average reprojection error in the homography validation is 1.54 pixels, and the Frobenius paradigm is 77.69% lower than LineTR. The research can effectively overcome the motion blur problem in the underground complex environment and provide reliable technical support for the visual perception of coal mine intelligent equipment.