Project 1(Computer Vision): Images of the Russian Empire

Background

Sergei Mikhailovich Prokudin-Gorskii (1863-1944) was a pioneering photographer whose work anticipated technological advancements in the field. He envisioned color photography as the next big thing and devised an innovative method involving capturing black-and-white images through red, green, and blue filters before projecting them in color to reconstruct vibrant images. At that time, color printing was fraught with challenges. Prokudin-Gorskii's greatest achievement came when he became famous for producing monochrome negatives using red, green, and blue filters. His artistic contributions were further highlighted when he received royal approval and financial backing to traverse vast landscapes across Russia, documenting them with his groundbreaking multicolored photographs. The Russian Empire’s cultural significance inspired him to seek recognition for his work. He obtained royal approval and financial backing to traverse vast landscapes across Russia, documenting them with his groundbreaking multicolored photographs. His RGB glass plate negatives were purchased in 1948 by the Library of Congress. Their digital archiving has made these negatives accessible online.

Photo Processing

Split and Merge

Since the images are arranged from top to bottom in BGR order, I divided each image into three segments and merged them to produce the colorized image.

Aligning

In order to overlay the 3 BGR images in alignment, there are some ways.

1. Sum of squared differences (SSD) between the intensities of the two features and the latter computes the correlation between the intensity values of the two features.
2. Mean squares method and the normalized correlation （NCC）
3. Enhanced Correlation Coefficient (ECC)

我采用ECC算法来对齐图像。该算法基于OpenCV 3中的ECC图像对齐方法，并参考了Evangelidis和Psarakis于2008年发表的一篇论文Parametric Image Alignment using Enhanced Correlation Coefficient Maximization。他们提出了一种新的相似性度量——Enhanced Correlation Coefficient（简称ECC），用于估计运动模型的参数。这种方法具有两个主要优势。

Unlike conventional pixel-based similarity measures that rely on differences in pixel intensities, ECC demonstrates remarkable robustness against variations in image brightness and contrast.
Despite the objective function being a complex nonlinear function of the parameters, their developed iterative method for solving the optimization problem remains linear. Meaningwise, they transformed what initially seemed like a computationally intensive challenge into an approach that can be solved more efficiently through iteration.

Automatic Cropping

Canny Edge Detector

Canny是一种非常有效的方法用于从不同视觉物体中提取有用的结构信息。
我应用它来获取边缘，在检测边缘时只关注边缘区域。
目的是裁剪白色、黑色或其他颜色的边框。
相反地，我将图像分为四个部分：顶部、底部、左侧和右侧。
每个部分仅包含Step2图像高度或宽度的10%。

(left)

(right)

Probabilistic Hough Transform

The Hough technique is used for identifying shapes. If a shape can be represented by a mathematical expression, then it can be detected in an image using the Hough Transform. Straight lines represent the simplest case; I employ the Probabilistic Hough Transform because it provides a more efficient approach compared to traditional Hough Transform for detecting curves.

By enhancing its intuitiveness for users to comprehend and utilize effectively. By first identifying all line segments across the entire Canny edge map and subsequently projecting these line segments onto a colorized version of the original image. However, in our project implementation, we divided these line segments into four distinct regions: top, bottom, left, and right as previously outlined.

Cropping

Once detected, I select specific regions from each image section. Specifically, I choose the lowest line from the upper section, and then select corresponding lines from other sections such as top, bottom, right and left. After that, I trim along these key edges to achieve uniformity.

Image Pyramid

When I try to run the above algorithm on a large image, the procedure was broken down. However, for a 500 X 500 image, it cost no more than 1 second. Thus, the idea is to resize the image by a factor of 0.5 recursively until the size of height is small than 500px and record the recurvise times N. If I resize the image 1 time by factor 0.5, the height and width [x, y] will become [0.5x, 0.5y]

Following this step, I obtain a small image and apply the aforementioned algorithm to it. Particularly, in ECC alignment, while obtaining a warp_matrix from the small image, I multiply this matrix by [[1 1 2]; [1 1 2]] for N iterations and then apply it to the large image to thereby expand the X,Y offsets.

Similarly, as I apply the Canny algorithm to small images, it necessitates multiplying 2^N times on either the X or Y axis of lines when cropping the image.

(partial)

There are some results:

Code:

https://gitee.com/sengo/Russian-Empire

Reference:

1.https://www.learnopencv.com/image-alignment-ecc-in-opencv-c-python/

2. Python OpenCV 教程 面向Python的OpenCV官方指南中文版 https://github.com/makelove/OpenCV-Python-Tutorial/blob/master/OpenCV-Python-Tutorial-中文版.pdf

3. http://robertpless.com/classes/ComputerVision/project1/description.html

4.https://en.wikipedia.org/wiki/Canny_edge_detector

5.https://en.wikipedia.org/wiki/Hough_transform

6.http://crcv.ucf.edu/videos/lectures/2012.php

7.http://www.jb51.net/article/33274.htm