Ad

Selenium Is Normal During Debugging, But The Run Click Fails

- 1 answer

I've searched several solutions but it didn't work. That's my code

driver = webdriver.PhantomJS()
driver.get('https://baijia.baidu.com')
wait = WebDriverWait(driver,10)
wait.until(EC.element_to_be_clickable((By.ID, 'getMoreArticles'))).click()
content = driver.page_source
page = open('test.html','wb')
page.write(content)

I've tried to debug the code, It successfully returns the clicked page. when I run the code, it also returns successfully, however it don't returns the clicked page, just the source page.

I tried to search the solutions, take the page down to the bottom:

driver.execute_script("window.scrollTo(0, document.body.scrollHeight)",element)

But it's the same result, only debug successfully.

Thanks

Ad

Answer

It seems, that your button initiate an AJAX request. Driver doesn't wait it finished, because there is no page reload. So you should add explicit wait. Something like that:

expected_number_of_articles = 10 # enter your number
article_locator = (By.CSS_SELECTOR, 'div#article') # enter your locator
wait.until(lambda driver: len(driver.find_elements(*article_locator)) >= expected_number_of_articles)
Ad
source: stackoverflow.com
Ad