18

I've developed a couple of Python scripts using Selenium and, at first, PhantomJS. While heading toward automated downloads, I switched for (headed) Firefox (which worked) and then Chrome with the headless option so I won't have the browser opening in front of me.

My first script, which accesses a page and a couple of HTML elements, works perfectly with headless Chrome.

The second one, however, works only with headed Chrome. If I add the "headless" option, it doesn't work anymore. When I try to print the HTML in headless mode to see why it cannot find the HTML element I'm looking for, all I have is :

<html xmlns="http://www.w3.org/1999/xhtml"><head></head><body></body></html>

With the headed Chrome, I have a complete HTML printed. This is how I start my headless Chrome :

options = webdriver.ChromeOptions() options.add_argument("--ignore-certificate-errors") options.add_argument("headless") driver = webdriver.Chrome(chrome_options=options)

Again, note that this works in another of my script. The only difference here is that I need to log in to access the page, but even then, why would it work with the head? My script is made to log in automatically anyway by filling the form.

Python : 3.6.1, Chrome : 60.0.3112.78 (64 bits), Selenium : 3.4.3

Any idea? Thanks.

** EDIT: Here is the beginning of the code**

url = 'https://10.11.227.21/tmui/' driver.get(url + "login.jsp") html_source = driver.page_source print(html_source) blocStatus = WebDriverWait(driver, TIMEOUT).until(EC.presence_of_element_located((By.ID, "username"))) inputElement = driver.find_element_by_id("username") inputElement.send_keys('actualLogin') inputElement = driver.find_element_by_id("passwd") inputElement.send_keys('actualPassword') inputElement.submit()
Improve this question
10
  • 1
    Can you show us your code block which you are trying to use as non-headed? Thanks Commented Jul 28, 2017 at 9:57
  • 1
    You want us to access an URL of Class A (10.11.227.21) Address? Thanks Commented Jul 28, 2017 at 10:16
  • 1
    -_- sorry about that, we are using a VPN that's why I forgot about it. So yeah basically I have no way to give it to you. Commented Jul 28, 2017 at 10:21
  • 2
    The issue could be due to the size of the window which is only 800x600 by default with the headless flag. Try to set the size flag: --window-size=1280,800 Commented Jul 28, 2017 at 10:50
  • 2
    @Gnougnou -- I'm curious if you've found a fix. I have exactly the same issue. Commented Aug 8, 2017 at 5:44

7 Answers 7

Reset to default
11

I had a same experience like you, and solved it by using xvfb and pyvirtualdisplay.

I use chromedrive=v2.3.1, chrome-browser=v60 and Selenium=3.4.3

In Headless chrome, some of script seems not to work as expected.

Please refer to vpassapera's comment in https://gist.github.com/addyosmani/5336747.

How about try it like below,

from pyvirtualdisplay import Display display = Display(visible=0, size=(800, 600)) display.start() # Do Not use headless chrome option # options.add_argument('headless') url = 'https://10.11.227.21/tmui/' driver.get(url + "login.jsp") html_source = driver.page_source print(html_source) blocStatus = WebDriverWait(driver, TIMEOUT).until(EC.presence_of_element_located((By.ID, "username"))) inputElement = driver.find_element_by_id("username") inputElement.send_keys('actualLogin') inputElement = driver.find_element_by_id("passwd") inputElement.send_keys('actualPassword') inputElement.submit() display.stop()

xvfb is required to use "pyvirtualdisplay"

$ sudo apt-get install -y xvfb
Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

I tried this but instead of html of the page it shows html of chrome official site.. why is that?
It's worth mentioning that this solution is only for *nix based OS
7

Headless Chrome does not support insecure certificates and hence, websites with insecure certificates does not open living it blank. You need to add capabilities as follow: 

from selenium import webdriver from selenium.webdriver import DesiredCapabilities from selenium.webdriver.chrome.options import Options chrome_options = Options() chrome_options.add_argument("--headless") capabilities = DesiredCapabilities.CHROME.copy() capabilities['acceptSslCerts'] = True capabilities['acceptInsecureCerts'] = True driver = webdriver.Chrome(chrome_options = chrome_options,executable_path='your path',desired_capabilities=capabilities) driver.get("yourWebsite")

This will do the work.

Improve this answer

1 Comment

It should also be noted if your local server doesn't redirect to https traffic to http make sure you're using the correct URL.
3

I had the same issue and setting the window size in conftest.py solved it. My code snippet:

@pytest.fixture def chrome_options(chrome_options, pytestconfig): if pytestconfig.getoption('headless'): chrome_options.add_argument('--headless') chrome_options.add_argument("window-size=1920,1080") else: chrome_options.add_argument("start-maximized"); return chrome_options
Improve this answer

Comments

0

Headless chrome may be faster on same machine than headed, try adding some wait before locating password element.

Improve this answer

Comments

0

For some if you remove the below, it would work. 

driver.fullscreen_window()
Improve this answer

Comments

0

For my situation, headless did not work because I was behind a proxy. Apparently, Chrome is able to use the system proxy, but headless does not use the system proxy.

I simply needed to provide the proxy, then headless (as well as Chrome) worked.

options.add_argument('--proxy-server=http://myproxy:port')
Improve this answer

Comments

-1

refer https://github.com/SeleniumHQ/selenium/issues/4477 add below code

self.chrome_options = webdriver.ChromeOptions() self.chrome_options.add_argument("--window-size=1920,1080") self.chrome_options.add_argument("--disable-extensions") self.chrome_options.add_argument("--proxy-server='direct://'") self.chrome_options.add_argument("--proxy-bypass-list=*") self.chrome_options.add_argument("--start-maximized") self.chrome_options.add_argument('--headless') self.chrome_options.add_argument('--disable-gpu') self.chrome_options.add_argument('--disable-dev-shm-usage') self.chrome_options.add_argument('--no-sandbox') self.chrome_options.add_argument('--ignore-certificate-errors') self.browser = webdriver.Chrome(options=self.chrome_options)
Improve this answer

1 Comment

As commented at the link you've provided, it's best for you to explain what these options do before suggesting them

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.