Ad

Using PhantomJS For Headless Browser In Selenium - Python

- 1 answer

I'm currently using Python-Selenium to run some tests. It's supposed to log in the webpage, input username and password, and do some other things. When I execute it using the Firefox browser, it works fine, however when I use PhantonJS I'm getting the following error:

2016-01-29 16:18:29 - ERROR - An exception occurred Message: {"errorMessage":"Unable to find element with id 'user_email'","request":{"headers":{"Accept":"application/json","Accept-Encoding":"identity","Connection":"close","Content-Length":"91","Content-Type":"application/json;charset=UTF-8","Host":"127.0.0.1:57257","User-Agent":"Python-urllib/2.7"},"httpVersion":"1.1","method":"POST","post":"{\"using\": \"id\", \"sessionId\": \"fcd54080-c6e6-11e5-94bd-27954be890c7\", \"value\": \"user_email\"}","url":"/element","urlParsed":{"anchor":"","query":"","file":"element","directory":"/","path":"/element","relative":"/element","port":"","host":"","password":"","user":"","userInfo":"","authority":"","protocol":"","source":"/element","queryKey":{},"chunks":["element"]},"urlOriginal":"/session/fcd54080-c6e6-11e5-94bd-27954be890c7/element"}}
Screenshot: available via screen
Traceback (most recent call last):
  File "webcrawler.py", line 105, in login
    email = self.singleton.driver.find_element_by_id("user_email")
  File "/Library/Python/2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 234, in find_element_by_id
    return self.find_element(by=By.ID, value=id_)
  File "/Library/Python/2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 712, in find_element
    {'using': by, 'value': value})['value']
  File "/Library/Python/2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 201, in execute
    self.error_handler.check_response(response)
  File "/Library/Python/2.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 181, in check_response
    raise exception_class(message, screen, stacktrace)
NoSuchElementException: Message: {"errorMessage":"Unable to find element with id 'user_email'","request":{"headers":{"Accept":"application/json","Accept-Encoding":"identity","Connection":"close","Content-Length":"91","Content-Type":"application/json;charset=UTF-8","Host":"127.0.0.1:57257","User-Agent":"Python-urllib/2.7"},"httpVersion":"1.1","method":"POST","post":"{\"using\": \"id\", \"sessionId\": \"fcd54080-c6e6-11e5-94bd-27954be890c7\", \"value\": \"user_email\"}","url":"/element","urlParsed":{"anchor":"","query":"","file":"element","directory":"/","path":"/element","relative":"/element","port":"","host":"","password":"","user":"","userInfo":"","authority":"","protocol":"","source":"/element","queryKey":{},"chunks":["element"]},"urlOriginal":"/session/fcd54080-c6e6-11e5-94bd-27954be890c7/element"}}
Screenshot: available via screen

Debugging the issue a little more, I find the page content is:

DEBUG - <html><head></head><body></body></html>

This is I start the Selenium driver:

def __init__(self, browser="phantomjs"):
        if browser == "phantomjs":
            self.driver = webdriver.PhantomJS()
            self.driver.set_window_size(1120, 550)
        elif browser == "firefox":
            self.driver = webdriver.Firefox()
        elif browser == "chrome":
            self.driver = webdriver.Chrome()
        elif browser == "remote":
            self.driver = webdriver.Remote()
        elif browser == "ie":
            self.driver = webdriver.Ie()
        else:
            raise ValueError("Invalid browser value:  %s"  % browser)

Could someone tell me how to fix that? It works perfectly using Firefox, but I'm going to deploy this in a node in AWS so I need to use some some of headless browser.

Ad

Answer

The option given by @alecxe is valid too. In my particular case, the webpage uses SSL, so the solution is to create the PhantomJS as:

self.driver = webdriver.PhantomJS(service_args=['--ignore-ssl-errors=true', '--ssl-protocol=any'])

Also make sure to set the window size, in order to create a dummy browser:

self.driver.set_window_size(1120, 550)
Ad
source: stackoverflow.com
Ad