I’m working on an image registration problem where I need to align a low-quality, blurry infrared (IR) image with a high-resolution RGB image of the same scene. The challenge is that the scene contains repeating structural patterns, which causes traditional feature matching (SIFT / ORB) to produce incorrect correspondences. Also, the IR image is significantly(Read More)
I’m working on an image registration problem where I need to align a low-quality, blurry infrared (IR) image with a high-resolution RGB image of the same scene.
The challenge is that the scene contains repeating structural patterns, which causes traditional feature matching (SIFT / ORB) to produce incorrect correspondences. Also, the IR image is significantly blurred and lower contrast, making keypoint detection unstable.
I’ve tried basic OpenCV approaches like SIFT + FLANN, but the matches are inconsistent due to ambiguity in repetitive regions.
Current Code Attempt (Python + OpenCV)
import cv2
import numpy as np
# Load images
rgb = cv2.imread("rgb.png")
ir = cv2.imread("ir.png", cv2.IMREAD_GRAYSCALE)
# Convert RGB to grayscale for matching
rgb_gray = cv2.cvtColor(rgb, cv2.COLOR_BGR2GRAY)
# Feature detector
sift = cv2.SIFT_create()
# Detect keypoints and descriptors
kp1, des1 = sift.detectAndCompute(ir, None)
kp2, des2 = sift.detectAndCompute(rgb_gray, None)
# FLANN matcher
FLANN_INDEX_KDTREE = 1
index_params = dict(algorithm=FLANN_INDEX_KDTREE, trees=5)
search_params = dict(checks=50)
flann = cv2.FlannBasedMatcher(index_params, search_params)
matches = flann.knnMatch(des1, des2, k=2)
# Lowe's ratio test
good_matches = []
for m, n in matches:
if m.distance < 0.75 * n.distance:
good_matches.append(m)
print(f"Good matches found: {len(good_matches)}")
# Homography (fails often due to wrong matches)
if len(good_matches) > 10:
src_pts = np.float32([kp1[m.queryIdx].pt for m in good_matches]).reshape(-1, 1, 2)
dst_pts = np.float32([kp2[m.trainIdx].pt for m in good_matches]).reshape(-1, 1, 2)
H, mask = cv2.findHomography(src_pts, dst_pts, cv2.RANSAC, 5.0)
aligned_ir = cv2.warpPerspective(ir, H, (rgb.shape[1], rgb.shape[0]))
else:
print("Not enough reliable matches")
Problem
- Repeating structures cause ambiguous matches
- IR image blur reduces feature quality
- RANSAC still fails to find a stable homography in many cases




