Picamera taking pictures fast and processing them

Sounds like a fascinating project!

I'm going to ignore the second script for now as it's a bit confused; but there's plenty that can be done to speed up your first script, and it's probably more useful to concentrate on improving that as you've obviously had it working before (albeit too slowly). So, first things first:

If I weren't sure where the script was spending too much time, I'd profile it. There are several tools built into to Python for this and some nice GUIs for visualizing the results too. However, if you don't want to profile everything (say you've got a hunch as to where the slow-down is), you can simply use time.time() around statements to figure out how long they're taking. For example:

import picamera import time # How long does camera initialization take? start = time.time() with picamera.PiCamera() as camera: pass print(time.time() - start) 

In this particular case, though, I can take a pretty good guess where the major slow-downs are! Firstly, camera initialization takes a long time (at least a second as you'll see if you try the script above). So ideally you want to initialize the camera once in your script (outside your detect_red function) instead of doing it again and again.

Next, you probably want to have a read of the sections in the picamera manual about rapid capture; ignore the later bits about threading and all that complex stuff - the important thing is that using the video port to capture images is much faster than using the still port (because it doesn't involve a mode switch; you can read about the camera's various ports in the hardware chapter). So, the only change necessary here is to add use_video_port=True to your capture call.

With these two simple changes the first script now looks like this:

def detect_red(camera): # saving the picture to an in-program stream rather than a file stream = io.BytesIO() #scale_down = 6 red = False # capture into stream camera.capture(stream, format='jpeg', use_video_port=True) # convert image into numpy array data = np.fromstring(stream.getvalue(), dtype=np.uint8) # turn the array into a cv2 image img = cv2.imdecode(data, 1) # Resizing the image, blur the image and convert it to HSV values for better recognition # img = cv2.resize(img, (len(img[0]) / scale_down, len(img) / scale_down)) # img = cv2.GaussianBlur(img, (5,5), 0) img = cv2.cvtColor(img, cv2.COLOR_BGR2HSV) #Defining the red color range and calculating if these values lie in the range red_lower = np.array([0, 150, 0], np.uint8) red_upper = np.array([5, 255, 255], np.uint8) red_binary = cv2.inRange(img, red_lower, red_upper) # Dilates the red space, making it larger dilation = np.ones((15, 15), "uint8") red_binary = cv2.dilate(red_binary, dilation) contours, _ = cv2.findContours(red_binary, cv2.RETR_LIST, cv2.CHAIN_APPROX_SIMPLE) if not contours == []: if not red: red = True print "Red surface detected!" else: print "No red surface detected." return red with picamera.PiCamera() as camera: # to speed things up, lower the resolution of the camera camera.resolution = (320, 240) detect_red(camera) detect_red(camera) detect_red(camera) detect_red(camera) # and so on... 

You should find repeated calls to detect_red much faster in this version. If it's fast enough, great! If it's not there's another trick you can use to speed things up which is to avoid JPEG encoding; at the moment, your script is capturing an image from the camera, having the camera encode it as a JPEG, then passing that JPEG to OpenCV to decode back into RGB values before processing it (OpenCV's image format is just a numpy array in BGR order). It would be faster to avoid all that encoding and decoding by having the camera return the RGB values directly.

Good luck!

You should also be able to directly set the size of the captured image, instead of doing a resize later; this will save a good chunk of compute time. Do this by setting: camera.resolution = (1024, 768) for whatever esolution you actually want.

Another trick which might be helpful would be just using the "red" channel of the RGB image as a greyscale image. Your captured image should be a numpy array with a row for each of the three color channels; you can just use the first channel as a greyscale image and save yourself some data processing.

I have used this code for cv2.cvtColor.

Getting image data from raspberry pi camera module in OpenCV with Python_ http://picamera.readthedocs.org/en/latest/api_array.html#pirgbarray

import cv2 import picamera import picamera.array import time with picamera.PiCamera() as camera: with picamera.array.PiRGBArray(camera) as stream: while True: camera.capture(stream, format='bgr') image = stream.array gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY) cv2.imshow('frame', gray) if cv2.waitKey(1) & 0xFF == ord('q'): break # reset the stream before the next capture stream.seek(0) stream.truncate() cv2.destroyAllWindows() 

The new version of Picamera has solve Picamera array lib. The picamera.array module provides a set of classes which aid in constructing n-dimensional numpy arrays from the camera output. Don't forget the stream.seek(0) and stream.truncate().

See Capturing to an OpenCV object