Research project · 2026

SAMatcher: Dense Co-Visibility Modeling via Cross-View Fusion for Scale-Imbalance Image Matching

Xu Pan^1,2,†,‡Zhen Pang^1,†Qiyuan Ma¹Wei Ji³Shuhan Shen⁴Xianwei Zheng^1,*

State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing (LIESMARS), Wuhan University
School of Electrical and Electronic Engineering, Nanyang Technological University
National Key Laboratory of Space Target Awareness, Space Engineering University
Institute of Automation, Chinese Academy of Sciences

^† Equal contribution · ^‡ Work done at LIESMARS · ^* Corresponding author

Information Fusion (Under Review)

Abstract

Reliable correspondence estimation becomes difficult under cross-view scale imbalance: the same content occupies unequal spatial extents, so paired images provide different amounts of usable evidence. SAMatcher takes a region-first approach^[1]: it fuses paired features to predict dense co-visible masks and bounding boxes before downstream matching. Built on SAM-HQ^[2], the framework combines symmetric cross-view interaction, shared learnable prompts, and joint mask–box supervision to produce view-specific cropping priors. Across classical, sparse learned, and dense matchers, SAMatcher improves robustness; with RoMa, it raises AUC@5 and mAA@5 by 5.58 and 4.97 percentage points, respectively.

Motivation for co-visible region modeling — **Fig. 1.** Large scale differences create pixel-level confusion. Explicit co-visible regions retain mutually supported content and constrain matching to geometrically meaningful areas.

Method

SAMatcher framework — **Fig. 2.Overview of SAMatcher.** A shared encoder extracts both views; symmetric fusion exposes cross-view context; prompt-driven mask and box decoders predict complementary co-visible priors that guide correspondence estimation.

Symmetric cross-view fuser — **Fig. 3.Symmetric cross-view feature interaction.** Paired views are treated as interacting token sequences with shared parameters and view offsets, enabling bidirectional alignment without discarding view-specific structure.

Region first

Co-visible masks identify dense mutual support; bounding boxes provide a compact geometric envelope. Both are predicted before matching rather than inferred after correspondences fail.

Symmetric interaction

Bidirectional feature exchange aligns semantics across views while preserving their distinct spatial layouts, allowing co-visibility to emerge as a relation between images.

Joint constraints

Point-sampled mask loss, box regression, and mask–box consistency regularize dense coverage and geometric localization together.

Evaluation

We evaluate on MegaDepth^[3] using the cross-view scale-imbalance protocol: 10 test scenes and 500 image pairs with sufficient co-visible regions. We report pose AUC, accuracy, and mean Average Accuracy (mAA) at multiple thresholds, alongside precision (P) and matching score (MS). SAMatcher is attached to ten representative pipelines—from SIFT and nearest-neighbor matching to SuperGlue^[4] and LoFTR^[5]—plus RoMa^[7], and compared with both the unmodified pipeline and OETR^[6] region priors.

+5.58 ppRoMa AUC@5 gain over its base pipeline

+4.97 ppRoMa mAA@5 gain

78.86 / 82.38full-model mask / box IoU (%)

Pipeline	Prior	AUC@5	AUC@10	AUC@20	Acc@5	Acc@10	Acc@15	Acc@20	mAA@5	mAA@10	mAA@20	P	MS
CON+NN	None	28.07	43.91	58.91	49.80	66.60	74.80	77.80	49.80	58.20	67.25	70.76	9.77
CON+NN	+OETR	30.74	47.70	63.45	52.00	72.40	79.80	84.20	52.00	62.20	72.10	75.66	14.27
CON+NN	+SAMatcher	32.97	50.46	65.79	56.80	73.40	81.20	87.00	56.80	65.10	74.60	79.84	91.31
D2+NN	None	0.85	1.40	2.29	1.27	2.29	3.31	3.82	1.27	1.78	2.67	9.34	2.23
D2+NN	+OETR	7.33	12.04	18.11	13.32	20.34	23.97	27.85	13.32	16.83	21.37	34.18	3.07
D2+NN	+SAMatcher	11.16	19.02	28.37	20.93	31.40	37.91	42.56	20.93	26.16	33.20	49.82	89.99
DISK+NN	None	5.24	7.92	10.36	8.75	11.41	12.47	14.32	8.75	10.08	11.74	18.77	0.82
DISK+NN	+OETR	20.63	30.06	38.10	34.58	43.03	45.52	47.76	34.58	38.81	42.72	56.70	4.88
DISK+NN	+SAMatcher	27.78	40.71	53.36	45.52	59.34	66.75	69.57	45.52	52.43	60.29	75.09	89.65
DISK+SG	None	25.29	37.49	50.06	43.37	55.82	62.45	68.27	43.37	49.60	57.48	39.50	9.62
DISK+SG	+OETR	29.66	44.45	58.74	50.80	66.40	73.00	78.20	50.80	58.60	67.10	69.48	15.81
DISK+SG	+SAMatcher	32.27	48.14	62.79	54.60	70.80	77.20	82.80	54.60	62.70	71.35	78.39	62.82
SIFT+NN	None	5.85	11.13	19.79	11.20	20.60	28.80	35.00	11.20	15.90	23.90	17.06	85.93
SIFT+NN	+OETR	10.05	16.91	26.47	18.20	28.80	36.60	41.40	18.20	23.50	31.25	23.32	85.40
SIFT+NN	+SAMatcher	10.64	20.64	34.02	22.60	37.80	47.20	55.20	22.60	30.20	40.70	27.62	93.61
R2D2+NN	None	20.47	33.45	47.27	37.60	52.60	61.60	68.00	37.60	45.10	54.95	52.89	87.81
R2D2+NN	+OETR	30.34	46.03	62.22	51.40	70.00	79.20	83.60	51.40	60.70	71.05	73.59	91.95
R2D2+NN	+SAMatcher	34.12	52.54	69.07	59.80	78.60	86.00	90.00	59.80	69.20	78.60	82.59	97.23
SP+NN	None	3.74	5.81	9.64	5.60	10.40	13.40	17.00	5.60	8.00	11.60	17.31	3.94
SP+NN	+OETR	13.43	21.74	31.63	24.60	35.60	41.60	45.80	24.60	30.10	36.90	40.11	9.54
SP+NN	+SAMatcher	22.53	36.18	50.42	40.80	56.60	65.60	70.20	40.80	48.70	58.30	60.62	90.61
LoFTR	None	34.36	48.33	60.79	53.80	68.20	73.20	77.20	53.80	61.00	68.10	55.61	43.91
LoFTR	+OETR	36.64	53.04	67.30	61.00	75.40	81.60	85.80	61.00	68.20	75.95	74.12	50.06
LoFTR	+SAMatcher	42.34	58.93	73.74	66.20	83.40	89.00	92.60	66.20	74.80	82.80	93.34	55.11
SP+SG	None	33.46	49.64	64.75	55.20	71.60	80.80	85.20	55.20	63.40	73.20	85.19	13.54
SP+SG	+OETR	35.69	51.89	66.36	58.00	74.80	80.60	86.20	58.00	66.40	74.90	90.41	22.66
SP+SG	+SAMatcher	36.41	54.43	69.97	62.60	79.40	85.20	90.40	62.60	71.00	79.40	94.96	79.49
RoMa	None	50.22	64.25	77.76	72.40	81.48	85.46	91.44	72.40	76.94	82.69	96.03	93.50
RoMa	+OETR	44.91	62.17	76.51	67.99	86.14	90.15	92.70	67.99	77.07	84.25	81.37	98.97
RoMa	+SAMatcher	55.80	70.06	80.36	77.37	87.42	90.82	91.47	77.37	82.39	86.75	97.63	99.57

Table 1. Complete MegaDepth evaluation under cross-view scale imbalance (%). All ten feature–matcher pipelines are reported without a prior, with OETR, and with SAMatcher.

Variant	Mask IoU (%)	Box IoU (%)
w/o symmetric interaction	75.06	71.90
w/o joint constraint	76.67	80.68
w/o view offset	77.15	80.74
Full SAMatcher	78.86	82.38

Table 2. Ablation on MegaDepth. Symmetric interaction contributes most strongly to box localization, while joint constraints and view offsets provide complementary gains.

Results

The quantitative gains are consistent across descriptor families, but the visual evidence explains why: SAMatcher learns mutual visibility rather than image saliency, then uses this support to remove structurally invalid correspondences.

Relative performance gains — **Fig. 4.** Ridge-style relative gains on MegaDepth. SAMatcher improves every evaluated matching configuration more consistently than the box-only OETR prior.

Co-visible masks — **Fig. 5.** OETR box predictions versus SAMatcher masks. The learned masks follow mutually visible content under large viewpoint changes and partial overlap.

Region-guided matching — **Fig. 6.** Region-guided correspondence comparison. Green and red lines denote correct and incorrect matches; SAMatcher retains valid support where coarse boxes are missing or inaccurate.

Mask and box co-refinement — **Fig. 7.** Masks favor recall and boxes provide compact localization. Their intersection suppresses over-extended predictions and yields a more reliable co-visible region.

**Fig. 8.** Zero-shot generalization on unseen GL3D (top) and ScanNet (bottom). The model transfers co-visibility reasoning across outdoor aerial and indoor domains without fine-tuning.

Citation

@article{pan2026samatcher,
  title={SAMatcher: Dense Co-Visibility Modeling via Cross-View Fusion for Scale-Imbalance Image Matching},
  author={Pan, Xu and Pang, Zhen and Ma, Qiyuan and Ji, Wei and Shen, Shuhan and Zheng, Xianwei},
  journal={arXiv preprint arXiv:2606.03406},
  year={2026}
}

References

X. Pan, Z. Pang, Q. Ma, et al. “SAMatcher: Dense Co-Visibility Modeling via Cross-View Fusion for Scale-Imbalance Image Matching.” arXiv:2606.03406, 2026.arXiv
L. Ke, M. Ye, M. Danelljan, et al. “Segment Anything in High Quality.” NeurIPS, 2023.arXiv
Z. Li and N. Snavely. “MegaDepth: Learning Single-View Depth Prediction from Internet Photos.” CVPR, 2018.arXiv
P.-E. Sarlin, D. DeTone, T. Malisiewicz, and A. Rabinovich. “SuperGlue.” CVPR, 2020.arXiv
J. Sun, Z. Shen, Y. Wang, et al. “LoFTR: Detector-Free Local Feature Matching with Transformers.” CVPR, 2021.arXiv
H. Chen, Z. Luo, L. Zhou, et al. “A Span-Based Object-Level Region Proposal Network for Scene Matching.” CVPR, 2022.arXiv
J. Edstedt, Q. Sun, G. Bökman, M. Wadenbäck, and M. Felsberg. “RoMa: Robust Dense Feature Matching.” CVPR, 2024.arXiv