Selenium: don't wait for async resources

Question

Selenium waits for async resource calls before proceeding to a new page.

Ex.

<script src="https://apis.google.com/js/platform.js" async defer></script>

On a website with many external apis (such as Google Analytics and share buttons from G+, Facebook, and Twitter). Selenium spends more time waiting for async calls than it does running tests.

Is there anyway to disable this behavior, so that selenium does not wait for async external api calls?

Community · Accepted Answer · 2017-05-23 12:02:38Z

What you see is page load timeout in action. You can tweak it and handle the timeout exception:

try: driver.set_page_load_timeout(5) # in seconds except TimeoutException: pass # continue with testing

In addition to it, you can also add an Explicit Wait to wait for a certain desired "action" element to appear, so that you can proceed with your tests immediately once the element appears.

You can also optimize it by blocking requests to certain domains that are non-relevant for your tests and would not hurt the page rendering and would not affect your tests. For instance, if you want to block Google Analytics requests:

How do you block Google Analytics from Selenium automated visits?

You can also disable images, CSS or flash (if this is applicable in your case):

This throws an error when the page takes too long to load. I want selenium to keep executing code and loading pages even if async resources are not loaded.
@PaintingInAir okay, what error are you seeing? Can you provide a reproducible example? Thanks.
Setting a lower page load timeout results in a timeout exception. According to the selenium docs, this is the purpose of driver.set_page_load_timeout. The second part of your answer is on the right track. Since all the async resources I'm loading are external, it achieves the same effect, without explicitly addressing the problem of waiting for async calls.
@PaintingInAir good, but why don't handle and ignore the timeout exception and continue testing? Thanks!
@PaintingInAir The "right" way to handle this is indeed to block the external resources call. That is (or should be) the point of the answer. The mechanics of how to accomplish that can be complicated, though I'd probably rely on a proxy. The "lazy" way to do it would be to set the page load timeout, then catch and ignore the exception then the timeout is hit. The piece that was missing from the answer was that if you go the timeout route, that you have to catch and ignore the exception.

Mahsum Akbas · Accepted Answer · 2015-09-20 18:16:32Z

with following code, you don't need wait to load complete page to make any process(i.e find_element functions)

FirefoxProfile fp = new FirefoxProfile(); fp.setPreference("webdriver.load.strategy", "unstable"); WebDriver driver = new FirefoxDriver(fp);

more details: https://code.google.com/p/selenium/wiki/FirefoxDriver#-Beta-_load_fast_preference

Collectives™ on Stack Overflow

Selenium: don't wait for async resources

2 Answers 2

6 Comments

Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

6 Comments

Comments

Linked

Related