Hierarchical Multi-Scale Patch Attention and Global Feature-Adaptive Fusion for Robust Occluded Face Recognition

Abstract

Occluded face recognition remains a challenging problem in biometric identification, where real-world obstructions such as masks, sunglasses, scarves, and hands obscure key facial features. To address this, we introduce a dual-branch architecture that combines a Local Multi-Patch Attention Module (LMPAM) for extracting localized features with a Global Self-Attention Channel Module (GSACM) to enhance overall feature representation. The local branch utilizes Multi-Scale Patch Attention to adaptively emphasize visible facial regions, ensuring robust feature learning from unoccluded areas. Meanwhile, the global branch employs Self-Attention with Channel Recalibration to enhance discriminative features, capturing long-range dependencies while suppressing occlusion-induced noise. The two branches are integrated using Dynamic Weighted Local-Global Fusion (DW-LG), allowing the model to balance local and global information effectively. Unlike predefined occlusion-aware methods, our approach generalizes across occlusions of varying types, regions, and sizes and demonstrates robustness on multiple datasets with changes in illumination, pose, and facial expression—without requiring explicit localization. Extensive evaluations on CASIA-WebFace, LFW, and AR datasets demonstrate the effectiveness of our approach, achieving higher recognition performance under severe occlusion conditions.