I am trying to search for elements on a webpage and have used various methods, including text and XPath. It seems that the timeout option does not work the way I expected, and no exception is raised even if the text or XPath is not present in the website's HTML. The code below demonstrates why I am confused.
There are three page.find queries. The first one just removes the cookies pop-up, so it can be ignored.
The second query:
page.find(("emailOrGhin", timeout = 1) works, and as shown in the results, it executes in about 0.01 seconds, which is great.
The third query is where I have questions:
page.find("emailOrGhinnowayjunkXyq", timeout=1) As shown in the results, this statement seems takes 1.52 seconds to execute, which is longer than the specified timeout of 1 second. Why does this happen?
Even more interesting, when the query fails to find a matching element, no exception (like ElementNotFound) is raised. I tried both except asyncio.TimeoutError and except Exception as e, but the statement completes successfully and returns None. I understand I can check for None, but I expected an exception to be triggered. Unfortunately, I cannot find a list of Nodriver exceptions anywhere.
I am hoping for insight/advice on two points:
Why the
timeoutoption behaves this way.What kind of error can be trapped if an element is not found, since the statement currently returns
None.
Here is the relevant code:
from datetime import timedelta from time import sleep import asyncio import datetime import sys import nodriver as uc passed_website = "https://www.google.com" # Main Processing Controller! = Mainline def a_main_processor(): loop =asyncio.new_event_loop() print("Line 20") driver = loop.run_until_complete(b_main_line_processing()) driver.stop() print("Line 28") return driver # Main Line Processing Controller! = Mainline async def b_main_line_processing(): driver = await uc.start() # maximize=True) # Start a new Chrome instance await driver.main_tab.maximize() sleep(1) ghin_url = "http://www.ghin.com/lookup.aspx" try: page = await driver.get(ghin_url, new_tab=False, new_window=False) except asyncio.TimeoutError: print("C100 - error", e) return driver # Find and Select Reject All Cookie Settings try: cookies = await page.find("Reject All") await cookies.click() except asyncio.TimeoutError: print("C105 - error:0, e") return driver # Find UserName Entry field name try: timepoint = datetime.datetime.now() user_name = await page.find("emailOrGhin", timeout = 1) print("First checkpoint =", str(datetime.datetime.now() - timepoint)[-9:-4], user_name) except Exception as e: print("First error took", str(datetime.datetime.now() - timepoint)[-9:-4], e) return driver # Pass bogus name to trip error try: timepoint = datetime.datetime.now() user_name = await page.find("emailOrGhinnowayjunkXyq", timeout=1) print("Second checkpoint =", str(datetime.datetime.now() - timepoint)[-9:-4], user_name) #except Exception as e: except asyncio.TimeoutError: print("Second error took", str(datetime.datetime.now() - timepoint)[-9:-4], e) return driver driver = a_main_processor() print("Line 62") The results are:
Line 20 First checkpoint = 00.01 <input type="text" id="emailOrGhin" name="emailOrGhin" aria-describedby="emailError" aria-invalid="false" aria-required="true" value="" style="margin-bottom: 0px;"></input> Second checkpoint = 01.52 None Line 28 Line 62 successfully removed temp profile C:\Users\pinev\AppData\Local\Temp\uc_5yaidgqs Process finished with exit code 0 The website is live, and the code should work.
timeoutoption, just that it will retry for that long.