A SIFT and RANSAC based panorama image stitcher

A panorama image stitcher employing the OpenCV implementation of SIFT. Broke the course record for the University of Amsterdam's Intro to Computer Vision BSc course in 2023.

Introduction

This was the first AI project I did. At the time i had only been coding for about 18 months, whenever i had time; My bachelor studies in Econometrics contained minimal programming education, and were quite time intensive (50 hour weeks were the norm). During my 2nd year, the Intro to Computer Vision was the first course i did as part of the minor in Computer Science at the Faculty of Science at the University of Amsterdam. It was a joy to finally apply some of my newly acquired math knowledge, and also get some practice in with building something significant in Python.

As this implementation broke the image stitching record of the ITCV course by a sizeable margin (the standing record was 4 at the time if i recall correctly, this version joins 9 images), I was invited by the course coordinators to present my results to my fellow students during a lecture. This was one of the highlights of my academic career up until that point, and i'd like to extend my gratitude to Dimitris Tzionas and Rein van den Boomgaard for giving me this opportunity.

Installation & usage

The currently available implementation is unlikely to be updated, but is functional under python 3.10, using OpenCV 4.11 and scikit-image 0.25.0. To install, run:

git clone http://github.com/mklblm/SIFT-based-Image-Stitcher
cd SIFT-based-Image-Stitcher
conda create -n itcvstitcher python=3.10
conda activate itcvstitcher
pip install -r requirements.txt

Then just run all the cells of the PanoramaStitcher.ipynb notebook to generate an image similar to the one shown above. An example set of images is provided.

Notes on the code

Using building blocks from the weekly assignments of the ITCV course, i built this implementation from the ground up during the 2023 May break. It is thus a mixture of code written by UvA Teaching Assistants that i attempted to optimise, and my own original work. The exception to this is the linear interpolation logic which is part of the cylindrical projection function, which was implemented by Rahul Kedia (reference below). An alternative version of the cylindrical projection function containing my own implementation of linear interpolation is also included, but Rahul's version is simply much quicker. It was also not my original intention to publish this code on a public platform; thus some of the comments in this code may not be useful for the reader - a number of them are simply there for my personal entertainment.

On computational performance and result quality

Because of my inexperience at the time with parallelising code, and not yet having learned how to build a project such as this in a modular fashion, the performance of this stitcher leaves much to be desired. The final version of the notebook stitches together 9 images of size 1512x2016, which were originally shot on an iPhone XR and subsequently scaled down. On a Windows 10 desktop PC with an Intel i7-12700k CPU the notebook takes about 4 minutes to run, with significant variance in the quality of the resulting panorama. The image posted above is cherrypicked from repeated runs.

References

Lowe, D. G. (2004). Distinctive image features from scale-invariant keypoints. International journal of computer vision, 60, 91-110. Direct download

UvA's Intro to Computer Vision online reader

Rahul Kedia's PanoramaStitching

ITCV Course coordinators:

Rein van den Boomgaard Rein's research

Dimitris Tzionas Dimitris' personal website