0

I have the following function:

def rescale(file): img = cv2.imread(file) return img

I am trying to upscale a series of images, but each image takes 20 seconds to process. I want to use multithreading in order to create a faster program.

I have a list of my images :

 file_list = ['page-136.jpg', 'page-0.jpg', 'page-11.jpg', 'page-12.jpg', 'page-13.jpg', 'page-14.jpg', 'page-37.jpg', 'page-58.jpg', 'page-62.jpg', 'page-64.jpg', 'page-134.jpg', 'page-135.jpg']

and I have the following code to add multithreading:

import tqdm from concurrent.futures import ThreadPoolExecutor, as_completed with ThreadPoolExecutor(max_workers=1000) as executor: future_to_response = { executor.submit(Upscale, i): i for i in file_list } t = tqdm.tqdm(total=len(future_to_response)) for future in as_completed(future_to_response): domain = future_to_response[future] result = future.result() t.update()

This however does not work, and I appear to be stuck. Can someone guide me in the right direction?

Improve this question
2
  • Please describe what you mean by "this however does not work." Do you get a traceback? Please post it. Does the output differ from what you expect? Tell us how it differs. Is it not any faster? Consider using processes instead of threads, this is likely a CPU-bound operation. Commented Dec 6, 2022 at 22:39
  • 1
    A few seconds with the search engine of your choice ("python threading") would tell you about the GIL and why threading is probably a bad idea for your use case. Also, what does "does not work" mean? Commented Dec 6, 2022 at 22:39

2 Answers 2

Reset to default
0

Instead of collecting the results in a container you simply assign it to a variable every iteration. When iteration stops result will only point-to the result of the last future to complete

.... result = [] for future in as_completed(future_to_response): ... result.append(future.result()) ...
Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

-1

You could use the threading library:

import threading threads_lst = [] for file in file_list: threads_lst.append(threading.Thread(target=Upscale, args=(file,))) threads_lst[-1].start() for t in threads_lst: t.join()
Improve this answer

1 Comment

You cannot use threads for parallel processing in stock Python. stackoverflow.com/a/55503917/801894