3

Im using PhantomJS to collect data about a Html page. My code it`s something like this:

from selenium import webdriver class PageElements(): def __init__(self, url): self.driver = webdriver.PhantomJS() self.driver.get(url) self.elements, self.attribute_types = self._load_elements(self.driver) def _load_elements(self, self.driver) """"This is not relevant"""

So, after I execute the code on IPython Notebook sometimes, to test things out. After a while, i get this on my Activity Monitor:

enter image description here

And this:

enter image description here

The processes still run even after i add a destroyer like:

def __del__(self): self.driver.close()

What is happening? I would really appreciate a "why this is happening" answer, instead a "do this" one. Why my destroyer isn't working?

I opened @forivall links, and saw the Selenium code. The PhantomJS webdriver has it`s own destructor (thus making mine redundant). Why aren't they working in this case?

Improve this question
3
  • Is the code exiting before the end, maybe due to uncaught exceptions? Commented Sep 25, 2013 at 18:51
  • Nope. Added a print "exit" in the destructor, and it has been executed. Commented Sep 25, 2013 at 19:03
  • 1
    Check out this page. stackoverflow.com/questions/13287490/… Seems phantomJS support was possibly dropped (as suggested on that page.) Also, I noticed that code on that page was using .quit(), not .close() - Are multiple browsers being opened during the code? If that is the case, .close() will close the current window, but not all of the opened windows. .quit() will exit the current driver. Commented Sep 25, 2013 at 19:09

5 Answers 5

Reset to default
3

__del__() tends to be unreliable in python. Not only do you not know when it will be called, you don't even have any guarantees that it will ever be called. try/finally constructs, or (even better) with-blocks (a.k.a. context managers), are much more reliable.

That said, I had a similar issue even using context managers. phantomjs processes were left running all over the place. I was invoking phantomjs through selenium as follows:

from selenium import webdriver from contextlib import closing with closing(webdriver.PhantomJS()) as driver: do_stuff(driver)

contextlib's closing() function ensures that the close() method of its argument gets called whatever happens, but as it turns out, driver.close(), while available, is the wrong method for cleaning up a webdriver session. driver.quit() is the proper way to clean up. So instead of the above, do one of the following:

from selenium import webdriver from contextlib import contextmanager @contextmanager def quitting(quitter): try: yield quitter finally: quitter.quit() with quitting(webdriver.PhantomJS()) as driver: do_stuff(driver)

or

from selenium import webdriver driver = webdriver.PhantomJS() try: do_stuff(driver) finally: driver.quit()

(The above two snippets are equivalent)

Credit goes to @Richard's comment on the original question for pointing me toward .quit().

Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

tried to edit the second code example to remove the extra ")" for driver = webdriver.PhantomJS()) but edits need to be 6 chars or more.
1

As of July 2016, following the discussion on this GitHub issue, the best solution is to run:

import signal driver.service.process.send_signal(signal.SIGTERM) driver.quit()

Instead of driver.close(). Just running driver.quit() will kill the node process but not the phantomjs child process that it spawned.

Improve this answer

Comments

0 
 self.driver = webdriver.PhantomJS()

This creates a web browser that is then used by Selenium to run the tests. Each time Selenium runs, it opens a new instance of the web browser, rather than looking to see if there is a previous one it could re-use. If you do not use .close at the end of the test, then the browser will continue to run in the background.

As you have seen, running the test multiple times leaves multiple browsers orphaned.

Improve this answer

1 Comment

I get that. What i wanted to know is why this is happening.
0

What the difference between this case, and objects that Python usually destroy automatically with it`s garbage collector?

The difference is that it's creating something outside of Python's domain: it's creating a new OS-level process. Perhaps webdriver.PhantomJS should have its own __del__ that will shut itself down Perhaps the behaviour should be more robust, but that's not the design decision that the selenium developers went with, probably because most of the other drivers are not headless (so it's obvious that the windows are open).

Unfortunately, neither the selenium (or unofficial) documentation has much clarification/best practices on this. (see comments below on __del__ behaviour).

links to source: phantomjs/webdriver.py remote/webdriver.py (superclass)

Improve this answer

3 Comments

Thats's odd. I edited my question. Selenium Webdriver for PhantomJS has its destructor in the implementation. But it`s not working either.
It's been a while since I've played with __del__, and it turns out it has some edge cases: stackoverflow.com/questions/3554952/del-at-program-end/… . Perhaps the selenium python api would be nicer if it supported context manager usage (__enter__, __exit__, with ... as :. Using a context manager would require you to re-organize your code though, but for the better.
Huh, I got a drive-by downvote last week. Please add a comment on how my answer can be improved, or how it may be incorrect.
0

I was also struggling for the same problem and I solved it from this source link.

By replacing self.process.kill() in selenium/webdriver/phantomjs/service.py with self.process.send_signal(signal.SIGTERM).

By using driver.quit() will kill all process of phantomjs on completing program or cancel the program using CTR+C

Improve this answer

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.