融合事件脉冲张量的矿井模糊图像线特征匹配方法研究

Research on line feature detection and matching method of mine blurred image by fusing event spike tensor

  • 摘要: 针对煤矿井下复杂环境中运动模糊导致的图像特征退化问题,提出一种融合事件脉冲张量(Event Spike Tensor,EST)的矿井模糊图像线特征匹配方法。首先,通过EST与RGB图像的数据层融合构建跨模态时空表征,利用多层感知器(Multilayer Perceptron,MLP)对事件流的时间戳和极性信息进行编码,生成具有时空特征的四维张量表示。在此基础上,由浅层融合模块和沙漏模块组成特征融合主干网络,通过通道先验卷积注意力机制(Channel-Prior Convolutional Attention,CPCA)实现双模态特征的加权融合,利用轻量级多头自注意力机制捕获全局特征关系,最终通过统一线段检测算法(Unified Line Segment Detection,ULSD)实现运动模糊场景下的鲁棒线特征检测。其次,构建图神经网络(Graph Neural Network,GNN)匹配模型,采用SuperPoint检测关键点及其描述子,与线段共同组成图结构的节点,通过自注意力机制增强单幅图像内的特征表示,并利用交叉注意力机制建立跨图像特征关联,同时引入线段信息传递模块强化几何约束。最后,通过模拟运输巷道、掘进工作面和综采工作面3种典型井下场景的实验验证方法的有效性。结果表明:相较于LineTR算法,所提方法的线段检测数量平均提升36.7%,分布均匀性密度方差平均降低77.5%;相较于LineTR和GlueStick方法,所提方法误匹配率降低至0.7%~2.7%,单应性验证中的平均重投影误差为1.54像素,Frobenius范数较LineTR降低77.69%。研究能够有效克服井下复杂环境中的运动模糊问题,为煤矿智能装备的视觉感知提供可靠技术支撑。

     

    Abstract: Aiming at the problem of image feature degradation caused by motion blur in the complex environment of underground coal mine, a fusion Event Spike Tensor (EST) is proposed as a line feature matching method for blur images of mine. First, a cross-modal spatiotemporal representation is built via data-layer fusion of EST and RGB images. Using a Multilayer Perceptron (MLP) to encode the timestamp and polarity information of the event stream, generating a four-dimensional tensor representation with spatiotemporal features. Upon this, a feature-fusion backbone combining a shallow fusion module and an hourglass module is employed. Weighted fusion of the dual-modal features is achieved by Channel-Prior Convolutional Attention (CPCA), while a lightweight multi-head self-attention mechanism captures global relations. Robust line detection under motion blur is finally realized with the Unified Line Segment Detection (ULSD). Secondly, a Graph Neural Network (GNN) matching model is constructed, employing SuperPoint to detect keypoints and their descriptors, which together with line segments form the nodes of the graph structure, enhancing the feature representation within a single image through the self-attention mechanism and establishing cross-image feature associations by using the cross-attention mechanism, and introducing the line segment information transfer module to reinforcement of geometric constraints. Finally, the effectiveness of the method is verified through experiments simulating three typical underground scenarios, namely transport roadways, digging faces and fully mechanized mining faces. The results show that: compared with LineTR algorithm, the proposed method improves the number of line segment detection by 36.7% on average, and reduces the distribution uniformity density variance by 77.5% on average; compared with LineTR and GlueStick methods, the mis-match rate of the proposed method is reduced to 0.7%‒2.7%, and the average reprojection error in the homography validation is 1.54 pixels, and the Frobenius paradigm is 77.69% lower than LineTR. The research can effectively overcome the motion blur problem in the underground complex environment and provide reliable technical support for the visual perception of coal mine intelligent equipment.

     

/

返回文章
返回