In this tutorial, we’ll talk about Harris Corner Detection, a mathematical approach to detecting corners in an image. First, we’ll illustrate the mathematical formulation. Then, we’ll show the step-by-step procedure to perform the Harris Corner Detector.
2. Mathematical Formulation
A corner is a point whose local neighborhood is characterized by large intensity variation in all directions. Corners are important features in computer vision because they are points stable over changes of viewpoint and illumination. The so-called Harris Corner Detector was introduced by Chris Harris and Mike Stephens in 1988 in the paper “A Combined Corner and Edge Detector”.
Let’s consider a two-dimensional image and a patch of size centered in . We want to evaluate the intensity variation occurred if the window is shifted by a small amount . Such variation can be estimated by computing the Sum of Squared Differences (SSD):
where is a window function that can be a rectangular or a Gaussian function. We need to maximize the function for corner detection. Since and are small, the shifted intensity can be approximated by the following first-order Taylor expansion:
where and are partial derivatives of in and direction, respectively. By substituting (2) in (1) we obtain:
The equation (3) can be expressed in the following matrix form:
where is a matrix computed from image derivatives:
The matrix is called structure tensor.
The Harris detector uses the following response function that scores the presence of a corner within the patch:
is a constant to chose in the range . Since is a symmetric matrix, and where and are the eigenvalues of . Hence, we can express the corner response as a function of the eigenvalues of the structure tensor:
So the eigenvalues determine whether a region is an edge, a corner or flat:
- if and are small, then is small and the region is flat;
- if or viceversa, then and the region is an edge;
- if and both eigenvalues are large, then R is large and the region is a corner.
The classification of the points using the eigenvalues of the structure tensor is represented in the following figure:
Harris corner detector consists of the following steps.
- Convert the original image into a grayscale image . The pixel values of are computed as a weighted sum of the corresponding , , values:
- Compute the derivatives and by convolving the image with the Sobel operator:
- Compute the products of the derivatives , , .
- Convolve the images , , with a Gaussian filter or a mean filter. Define the structure tensor for each pixel as expressed in eq. (5).
- Compute the response function for each pixel:
- Set a threshold on the value of and find pixels with responses above this threshold. Finally, compute the non-max suppression in order to pick up the optimal corners.
In the following figure we show a real application of the Harris Corner Detector. The corner response map (top-right image) was computed using Eq. 10 with k=0.04. The bottom-left image represents the thresholded corner response obtained with . The detected corners obtained after the application of the non-max suppression are shown in the bottom-right image.
In this article, we reviewed the Harris Corner Detector, a fundamental technique in computer vision. We explained the mathematical formulation, and finally, we explained how to implement it.