9

I would like to have Selenium run a headless instance of Google Chrome to mine data from certain websites without the UI overhead. I downloaded the ChromeDriver executable from here and copied it to my current scripting directory.

The driver appears to work fine with Selenium and is able to browse automatically, however I cannot seem to find the headless option. Most online examples of using Selenium with headless Chrome go something along the lines of:

import os from selenium import webdriver from selenium.webdriver.common.keys import Keys from selenium.webdriver.chrome.options import Options chrome_options = Options() chrome_options.add_argument("--headless") chrome_options.binary_location = '/Applications/Google Chrome Canary.app/Contents/MacOS/Google Chrome Canary'` driver = webdriver.Chrome(executable_path=os.path.abspath(“chromedriver"), chrome_options=chrome_options) driver.get("http://www.duo.com")`

However when I inspect the possible arguments for the Selenium WebDriver using the command chromedriver -h this is what I get:

D:\Jobs\scripts>chromedriver -h Usage: chromedriver [OPTIONS] Options --port=PORT port to listen on --adb-port=PORT adb server port --log-path=FILE write server log to file instead of stderr, increases log level to INFO --log-level=LEVEL set log level: ALL, DEBUG, INFO, WARNING, SEVERE, OFF --verbose log verbosely (equivalent to --log-level=ALL) --silent log nothing (equivalent to --log-level=OFF) --append-log append log file instead of rewriting --replayable (experimental) log verbosely and don't truncate long strings so that the log can be replayed. --version print the version number and exit --url-base base URL path prefix for commands, e.g. wd/url --whitelisted-ips comma-separated whitelist of remote IP addresses which are allowed to connect to ChromeDriver

No --headless option is available.

Does the ChromeDriver obtained from the link above allow for headless browsing?

Improve this question
2
  • 1
    Yup as far as I have tried, I could never run Google Chrome on headless mode. I just switched to firefox. Commented Jan 4, 2019 at 8:59
  • 1
    I am able to run in headless mode in macos too. Commented Jan 4, 2019 at 9:12

4 Answers 4

Reset to default
16

--headless is not argument for chromedriver but for Chrome. --headless Run chrome in headless mode, i.e., without a UI or display server dependencies. ChromeDriver is a separate executable that WebDriver uses to control Chrome and Webdriver is a a collection of language specific bindings to drive a browser.

I am able to run in headless mode with this set of options. I hope this will help: 

from bs4 import BeautifulSoup, NavigableString from selenium.webdriver.chrome.options import Options from selenium import webdriver import requests import re options = Options() options.add_argument('--headless') options.add_argument('--no-sandbox') options.add_argument('--disable-gpu') browser = webdriver.Chrome(chrome_options=options) # see edit for recent code change. browser.implicitly_wait(20)

Update 12 Aug 2019:

old : browser = webdriver.Chrome(chrome_options=options)

new : browser = webdriver.Chrome(options=options)

Improve this answer
Sign up to request clarification or add additional context in comments.

4 Comments

But the executable has no headless option... what does the line options.add_argument('--headless') do?
As @ewwink sugeested this is for chrome --headless Run chrome in headless mode, i.e., without a UI or display server dependencies. ChromeDriver is a separate executable that WebDriver uses to control Chrome. Webdriver is a a collection of language specific bindings to drive a browser
so let me get this straight... chromedriver.exe itself calls Chrome.exe at some point. The idea is to have it call the driver with an additional argument ?
Thanks, that's working well. yeap 'chrome_options' was deprecated so must use 'options'.
5

Try

options.headless=True

The following is how I set up my headless chrome 

options = webdriver.ChromeOptions() options.headless=True options.add_argument('window-size=1920x1080') prefs = { "download.default_directory": r"C:\FilePath\Download", "download.prompt_for_download": False, "download.directory_upgrade": True} options.add_experimental_option('prefs', prefs) chromedriver = (r"C:\Filepath\chromedriver.exe")
Improve this answer

Comments

1

--headless is not argument for chromedriver but Chrome, you can see more arguments or Command Line Switches for chrome here

Improve this answer

Comments

1

As many answers correctly state, headless is an option to chrome and not for chromedriver.

Besides all of that:

  • there is --headless=new since chrome 109, which will use the very same chrome. Before that chrome and headless chrome was two different built executable (both were installed by chrome installers and packages)

  • some answer recommends to use options.headless=True. Do not. See Headless is Going Away!

Improve this answer

2 Comments

So to spell it out, do you mean options.add_argument('--headless=new')?
exactly. and actually this is the answer. In general we can try pass any Chrome's argument to Chrome using chromedriver's add_argument. In practice this means, that Chrome's documentation is relevant in many cases when we want to control Chrome

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.