Python OpenCV line detection to detect `X` symbol in image

Question

I have an image where I need to detect a X symbol inside the line.

The image:

As you can see in the image above there is a X symbol inside a line. I want to know the X & Y coordinates of the symbol. Is there a way to find this symbol within this picture or is it to small?

import cv2 import numpy as np def calculateCenterSpot(results): startX, endX = results[0][0], results[0][2] startY, endY = results[0][1], results[0][3] centerSpotX = (endX - startX) / 2 + startX centerSpotY = (endY - startY) / 2 + startY return [centerSpotX, centerSpotY] img = cv2.imread('crop_1.png') res2 = img.copy() cords = [[1278, 704, 1760, 1090]] center = calculateCenterSpot(cords) cv2.circle(img, (int(center[0]), int(center[1])), 1, (0,0,255), 30) cv2.line(img, (int(center[0]), 0), (int(center[0]), img.shape[0]), (0,255,0), 10) cv2.line(img, (0, int(center[1])), (img.shape[1], int(center[1])), (255,0,0), 10) gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) # You can either use threshold or Canny edge for HoughLines(). _, thresh = cv2.threshold(gray,0,255,cv2.THRESH_BINARY_INV+cv2.THRESH_OTSU) #edges = cv2.Canny(gray, 50, 150, apertureSize=3) # Perform HoughLines tranform. lines = cv2.HoughLines(thresh,0.5,np.pi/180,1000) for line in lines: for rho,theta in line: a = np.cos(theta) b = np.sin(theta) x0 = a*rho y0 = b*rho x1 = int(x0 + 5000*(-b)) y1 = int(y0 + 5000*(a)) x2 = int(x0 - 5000*(-b)) y2 = int(y0 - 5000*(a)) if x2 == int(center[0]): cv2.circle(img, (x2,y1), 100, (0,0,255), 30) if y2 == int(center[1]): print('hell2o') # cv2.line(res2,(x1,y1),(x2,y2),(0,0,255),2) #Display the result. cv2.imwrite('h_res1.png', img) cv2.imwrite('h_res3.png', res2) cv2.imwrite('image.png', img)

I already tried doing it with HoughLines, but it wasn't a success.

Please share your code, so we can specifically dig into it and look for improvements or solutions. — AnsFourtyTwo
– AnsFourtyTwo, Commented Nov 13, 2019 at 12:30
Do you have a single image where you need to locate this x, or several ? And if there is multiple images to process, does the x symbol looks always the same, and have the same dimension ? — user7804910
– user7804910, Commented Nov 13, 2019 at 17:12
@VictorDeleau We have multiple. the X symbol is always the same. And it has the the same dimension. I hope this information helps. — mHvNG
– mHvNG, Commented Nov 13, 2019 at 17:28
Then you can look into matchTemplate (you can look for several versions of 'X' to account for a vertical line, etc.) — J.D.
– J.D., Commented Nov 13, 2019 at 19:03

nathancy · Accepted Answer · 2019-11-13 22:21:21Z

Instead of using cv2.HoughLines(), an alternative approach is to use template matching. The idea is to search and find the location of a template image in a larger image. To perform this method, the template slides over the input image (similar to 2D convolution) where comparison methods are performed to determine pixel similarity. This is the basic idea behind template matching. Unfortunately, this basic method has flaws since it only works if the template image size is the same as the desired item to find in the input image. So if your template image was smaller than the desired region to find in the input image, this method would not work.

To get around this limitation, we can dynamically rescale the image for better template matching using np.linspace(). With each iteration, we resize the input image and keep track of the ratio. We continue resizing until the template image size is larger than the resized image while keeping track of the highest correlation value. A higher correlation value means a better match. Once we iterate through various scales, we find the ratio with the largest match and then compute the coordinates of the bounding box to determine the ROI.

Using this screenshotted template image

Here's the result

import cv2 import numpy as np # Resizes a image and maintains aspect ratio def maintain_aspect_ratio_resize(image, width=None, height=None, inter=cv2.INTER_AREA): # Grab the image size and initialize dimensions dim = None (h, w) = image.shape[:2] # Return original image if no need to resize if width is None and height is None: return image # We are resizing height if width is none if width is None: # Calculate the ratio of the height and construct the dimensions r = height / float(h) dim = (int(w * r), height) # We are resizing width if height is none else: # Calculate the ratio of the 0idth and construct the dimensions r = width / float(w) dim = (width, int(h * r)) # Return the resized image return cv2.resize(image, dim, interpolation=inter) # Load template, convert to grayscale, perform canny edge detection template = cv2.imread('template.png') template = cv2.cvtColor(template, cv2.COLOR_BGR2GRAY) template = cv2.Canny(template, 50, 200) (tH, tW) = template.shape[:2] cv2.imshow("template", template) # Load original image, convert to grayscale original_image = cv2.imread('1.png') gray = cv2.cvtColor(original_image, cv2.COLOR_BGR2GRAY) found = None # Dynamically rescale image for better template matching for scale in np.linspace(0.1, 3.0, 20)[::-1]: # Resize image to scale and keep track of ratio resized = maintain_aspect_ratio_resize(gray, width=int(gray.shape[1] * scale)) r = gray.shape[1] / float(resized.shape[1]) # Stop if template image size is larger than resized image if resized.shape[0] < tH or resized.shape[1] < tW: break # Detect edges in resized image and apply template matching canny = cv2.Canny(resized, 50, 200) detected = cv2.matchTemplate(canny, template, cv2.TM_CCOEFF) (_, max_val, _, max_loc) = cv2.minMaxLoc(detected) # Uncomment this section for visualization ''' clone = np.dstack([canny, canny, canny]) cv2.rectangle(clone, (max_loc[0], max_loc[1]), (max_loc[0] + tW, max_loc[1] + tH), (0,255,0), 2) cv2.imshow('visualize', clone) cv2.waitKey(0) ''' # Keep track of correlation value # Higher correlation means better match if found is None or max_val > found[0]: found = (max_val, max_loc, r) # Compute coordinates of bounding box (_, max_loc, r) = found (start_x, start_y) = (int(max_loc[0] * r), int(max_loc[1] * r)) (end_x, end_y) = (int((max_loc[0] + tW) * r), int((max_loc[1] + tH) * r)) # Draw bounding box on ROI cv2.rectangle(original_image, (start_x, start_y), (end_x, end_y), (0,255,0), 2) cv2.imshow('detected', original_image) cv2.imwrite('detected.png', original_image) cv2.waitKey(0)

Is there also a way to do this when the template image can sometimes be a little different from the original image? I need to do this on multiple images but because of JPEG compression, the symbols are slightly different every time.
I have found another way to use multiple templates to detect symbols i have also posted my answer below.
It should still work if your template is slightly different due to JPEG compression since it determines the "best" correlation value so even if your template not exactly the same, you should still obtain a pretty good estimate. Take a look at template matching for multiple objects but this method assumes that the template and the desired object to find are exactly the same size.
@NizarB. I'm not sure what you mean, the bounding box coordinates are found in start_x, start_y, end_x, and end_y. You could calculate the centroid from there
No that calculation is wrong, to find centroid its x=start_x +(end_x - start_x)/2 and y=start_y + (end_y - start_y)/2. Draw the points out to visualize the box

NielsNL4 · Accepted Answer · 2019-11-14 11:59:52Z

For multiple template images you can use a for loop with the amount of different template images you have and then using the threshold to method find multiple template matches.

for i in range(templateAmount): template = cv2.imread('template{}.png'.format(i),0) w, h = template.shape[::-1] res = cv2.matchTemplate(img_gray,template,cv2.TM_CCOEFF_NORMED) threshold = 0.8 loc = np.where( res >= threshold) for pt in zip(*loc[::-1]): cv2.rectangle(img_rgb, pt, (pt[0] + w, pt[1] + h), (0,0,255), 2)

user7804910 · Accepted Answer · 2019-11-13 19:17:52Z

If you have multiple images where you need to detect this X symbol, and if this X symbol is always the same and has the same dimension, you could run a two-dimensional convolution over each image, where the kernel that you are convoluting is the X symbol, isolated, that you are trying to detect. You could then check the output of this two-dimensional convolution for the pixel of maximum intensity, whose normalize coordinate (x/w,y/h) would correspond with high probability to the normalize coordinate of the X symbol in the input image. Here is the mathematical expression for a two-dimensional convolution:

In opencv you can define your own kernel (make sure to only keep the cross and nothing else in the background), and then apply it to your image.

Collectives™ on Stack Overflow

Python OpenCV line detection to detect `X` symbol in image

3 Answers 3

5 Comments

Comments

Comments

Hot Network Questions