i have a task where i need selenium automation to do multiple searches inputs, each has to open the browser do some interactions and close, i can do that one after the other , but i thought that if i implement multithreading on this project it would be alot faster,i tried to implement it but it never worked as expected i did some searchs a read about queue and thread workers
but couldn't implement that either so can i make a queue and only 4 Threads work at time ? because i guess more than 4 browser would be alot. and would it be safe threaded ?
- 1Does this answer your question? Python selenium multiprocessing. Be sure to look at my answer to this which offers a much-needed improvement.The point is that if you have 4 threads and 20 tasks you want to run (20 URLs), you do not want to be opening and closing up 20 browsers but rather just 4 (one for each thread). This shows you how to reuse the browser for each new task (URL).Booboo– Booboo2021-04-10 16:52:23 +00:00Commented Apr 10, 2021 at 16:52
Add a comment |
1 Answer
You can use the module threading and a function:
import threading def main(): # your code to execute with Selenium for _ in range(4): threading.Thread(tagret=main).start() More than 4 browsers doing things at the same time could be indeed a lot, but it really depends on your PC and on how heavy the page is. You can always try out the code above with less threads and see how it goes.
