42

I am trying to find horizontal and vertical lines from an image which came from a "document". The documents are scanned pages from contracts and so the lines look like what you would see in a table or in a contract block.

I have been trying OpenCV for the job. The Hough transform implementation in OpenCV seemed useful for the job, but I could not find any combination of parameters that would allow it to cleanly find the vertical and horizontal lines. I tried with and without edge detection. No luck. If anyone has done anything similar I'm interested in knowing how.

See here an image of my before and after experimentation with HoughP in OpenCV. It's the best I could do, http://dl.dropbox.com/u/3787481/Untitled%201.png

So now I'm wondering whether there is another kind of transform I could use which would allow me to reliably find horizontal and vertical lines (and preferably dashed lines too).

I know this problem is solvable because I have Nuance and ABBYY OCR tools which can both reliably extract horizontal and vertical lines and return me the bounding box of the lines.

Thanks! Patrick.

Improve this question

6 Answers 6

Reset to default
41

Have you seen a code sample from HoughLinesP function documentation?

I think you can use it as starting point for your algorithm. To pick horizontal an vertical lines you just need to filter out other lines by line angle.

UPDATE:

As I see you need to find not the lines but horizontal an vertical edges on the page. For this task you need to combine several processing steps to get good results.

For your image I'm able to get good results by combining Canny edge detection with HoughLinesP. Here is my code (I've used python, but I think you see the idea):

img = cv2.imread("C:/temp/1.png") gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) edges = cv2.Canny(gray, 80, 120) lines = cv2.HoughLinesP(edges, 1, math.pi/2, 2, None, 30, 1); for line in lines[0]: pt1 = (line[0],line[1]) pt2 = (line[2],line[3]) cv2.line(img, pt1, pt2, (0,0,255), 3) cv2.imwrite("C:/temp/2.png", img)

Result looks like:

Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

Hi Andrey, thanks. Yes I tried HoughLinesP with many different variables. I have adjusted my original question and included a link to the best image I could get out of HoughLinesP. And yes I did try restricting to only near-horizontal lines.
Great. Clearly I was missing the Canny detector. That is a good result. I have also found an algorithm called Orthogonal Zig-Zag which can take the lines provided and then expand them to determine the width of the line as well (which is something that HoughP cannot do).
27

Here's a complete OpenCV solution using morphological operations.

  • Obtain binary image
  • Create horizontal kernel and detect horizontal lines
  • Create vertical kernel and detect vertical lines

Here's a visualization of the process. Using this input image:

Binary image

import cv2 # Load image, convert to grayscale, Otsu's threshold image = cv2.imread('1.png') result = image.copy() gray = cv2.cvtColor(image,cv2.COLOR_BGR2GRAY) thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)[1]

Detected horizontal lines highlighted in green

# Detect horizontal lines horizontal_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (40,1)) detect_horizontal = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, horizontal_kernel, iterations=2) cnts = cv2.findContours(detect_horizontal, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE) cnts = cnts[0] if len(cnts) == 2 else cnts[1] for c in cnts: cv2.drawContours(result, [c], -1, (36,255,12), 2)

Detected vertical lines highlighted in green

# Detect vertical lines vertical_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (1,10)) detect_vertical = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, vertical_kernel, iterations=2) cnts = cv2.findContours(detect_vertical, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE) cnts = cnts[0] if len(cnts) == 2 else cnts[1] for c in cnts: cv2.drawContours(result, [c], -1, (36,255,12), 2)

Result

Here's the output using another input image

Input -> Binary -> Detected Horizontal -> Detected Vertical -> Result

Note: Depending on the image, you may have to modify the kernel size. For instance to capture longer horizontal lines, it may be necessary to increase the horizontal kernel from (40, 1) to say (80, 1). If you wanted to detect thicker horizontal lines, then you could increase the width of the kernel to say (80, 2). In addition, you could increase the number of iterations when performing cv2.morphologyEx(). Similarly, you could modify the vertical kernels to detect more or less vertical lines. There is a trade-off when increasing or decreasing the kernel size as you may capture more or less of the lines. Again, it all varies depending on the input image

Full code for completeness

import cv2 # Load image, convert to grayscale, Otsu's threshold image = cv2.imread('1.png') result = image.copy() gray = cv2.cvtColor(image,cv2.COLOR_BGR2GRAY) thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)[1] # Detect horizontal lines horizontal_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (40,1)) detect_horizontal = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, horizontal_kernel, iterations=2) cnts = cv2.findContours(detect_horizontal, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE) cnts = cnts[0] if len(cnts) == 2 else cnts[1] for c in cnts: cv2.drawContours(result, [c], -1, (36,255,12), 2) # Detect vertical lines vertical_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (1,10)) detect_vertical = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, vertical_kernel, iterations=2) cnts = cv2.findContours(detect_vertical, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE) cnts = cnts[0] if len(cnts) == 2 else cnts[1] for c in cnts: cv2.drawContours(result, [c], -1, (36,255,12), 2) cv2.imshow('result', result) cv2.waitKey()
Improve this answer

Comments

12

If you just want the "lines" and not the "line segments", I would avoid using Canny, Hough, FindContours or any other such function in case you want more speed in your code. If your images are not rotated and what you want to find is always vertical or horizontal, I would just use cv::Sobel (one for vertical, and another for horizontal) and create accumulation arrays for columns and rows. Then you can search for maxima in such accumulations or profiles, for instance by setting a threshold, and you will know the row or column in which there is a vertical or horizontal edge lines.

Improve this answer

2 Comments

Do you know the parameters to be set for horizontal and vertical lines in cv::sobel?
just you need to try to change the parameter according to your image i am talking about these parameter try diff parameter and see the visual of image ((cv2.MORPH_RECT, (40,1))) these parameter for vertical and horizontal line
8

You might consider leaving the Hough line detection since this method looks for "global" lines, not necessarily line segments. I recently implemented an application that identified "parallelograms" - essentially squares that might be rotated and perspective fore-shortened due to viewing angle. You might consider something similar. My pipeline was:

  1. Convert from RGB to grayscale (cvCvtColor)
  2. Smooth (cvSmooth)
  3. Threshold (cvThreshold)
  4. Detect edges (cvCanny)
  5. Find contours (cvFindContours)
  6. Approximate contours with linear features (cvApproxPoly)

In your application, the resulting contour list will likely be large (depending upon the "aggressiveness" of smoothing and the feature enhancement of the Canny edge detector. You can prune this list by a variety of parameters: number of points returned from the contour finder, area of the contour (cvContourArea), etc. From my experience, I would expect that "valid" lines in your application would have well-defined area and vertex count properties. Additionally, you can filter out contours based on distance between end-points, angle defined by the line connecting end-points, etc.

Depending upon how much CPU "time" you have, you can always pair the Hough algorithm with an algorithm like that above to robustly identify horizontal and vertical lines.

Improve this answer

Comments

6

Don´t convert the RGB to grayscale. Sometimes, different colors in RGB can be merged to the same grayscale value, so it could miss some contours. You should analyze each of the RGB channels separately.

Improve this answer

Comments

1

Here is an approach that accumulates arrays for columns and rows. Then one can search for maxima in such accumulations (above a certain threshold) and deduce in which row or column there is a vertical or horizontal line.

If you want to quickly test the code, use the following Google Colab Notebook.
Google Colab Notebook

import numpy as np import cv2 import scipy from scipy.signal import find_peaks from matplotlib import pyplot as plt url = "https://i.sstatic.net/S00ap.png" !wget $url -q -O input.jpg fileName = 'input.jpg' img = cv2.imread(fileName) img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB) tmp = img.copy() gray = cv2.cvtColor(tmp, cv2.COLOR_BGR2GRAY) blurred = cv2.bilateralFilter(gray, 11, 61, 39) edges = cv2.Canny(blurred, 0, 255) v_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (1,3)) h_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (7,1)) v_morphed = cv2.morphologyEx(edges, cv2.MORPH_OPEN, v_kernel, iterations=2) v_morphed = cv2.dilate(v_morphed, None) h_morphed = cv2.morphologyEx(edges, cv2.MORPH_OPEN, h_kernel, iterations=2) h_morphed = cv2.dilate(h_morphed, None) v_acc = cv2.reduce(v_morphed, 0, cv2.REDUCE_SUM, dtype=cv2.CV_32S) h_acc = cv2.reduce(h_morphed, 1, cv2.REDUCE_SUM, dtype=cv2.CV_32S) def smooth(y, box_pts): box = np.ones(box_pts)/box_pts y_smooth = np.convolve(y, box, mode='same') return y_smooth s_v_acc = smooth(v_acc[0,:],9) s_h_acc = smooth(h_acc[:,0],9) v_peaks, v_props = find_peaks(s_v_acc, 0.70*np.max(np.max(s_v_acc))) h_peaks, h_props = find_peaks(s_h_acc, 0.70*np.max(np.max(s_h_acc))) for peak_index in v_peaks: cv2.line(tmp, (peak_index, 0), (peak_index, img.shape[0]), (255, 0, 0),2) for peak_index in h_peaks: cv2.line(tmp, (0, peak_index), (img.shape[1], peak_index), (0, 0, 255),2) v_height = v_props['peak_heights'] #list of the heights of the peaks h_height = h_props['peak_heights'] #list of the heights of the peaks def align_axis_x(ax, ax_target): """Make x-axis of `ax` aligned with `ax_target` in figure""" posn_old, posn_target = ax.get_position(), ax_target.get_position() ax.set_position([posn_target.x0, posn_old.y0, posn_target.width, posn_old.height]) def align_axis_y(ax, ax_target): """Make y-axis of `ax` aligned with `ax_target` in figure""" posn_old, posn_target = ax.get_position(), ax_target.get_position() ax.set_position([posn_old.x0, posn_target.y0, posn_old.width, posn_target.height]) fig = plt.figure(constrained_layout=False, figsize=(24,16)) spec = fig.add_gridspec(ncols=4, nrows=2, height_ratios=[1, 1]) ax1 = fig.add_subplot(spec[0,0]) ax1.imshow(tmp) ax2 = fig.add_subplot(spec[0, 1]) ax2.imshow(v_morphed) ax3 = fig.add_subplot(spec[0, 2]) ax3.imshow(h_morphed) ax4 = fig.add_subplot(spec[0, 3], sharey=ax3) ax4.plot(h_acc[:,0], np.arange(len(h_acc[:,0])), 'y', marker="o", ms=1, mfc="k", mec="k") ax4.plot(s_h_acc, np.arange(len(s_h_acc)), 'r', lw=1) ax4.plot(h_height, h_peaks, "x", lw="5") ax5 = fig.add_subplot(spec[1, 1], sharex=ax2) ax5.plot(np.arange(len(v_acc[0,:])), v_acc[0,:], 'y', marker="o", ms=1, mfc="k", mec="k") ax5.plot(np.arange(len(s_v_acc)), s_v_acc, 'r', lw=2) ax5.plot(v_peaks, v_height, "x", lw="5") plt.tight_layout() align_axis_y(ax4,ax3) align_axis_x(ax5,ax2)

Output

Improve this answer

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.