Ad

Click Javascript Tab Using Mechanize And Ruby

I am using mechanize, ruby and ruby & rails to scrape this website . And i want to click the "Trabajo En Sala" tab so that I could scrape whatever information in that tab.

I know that mechanize doesn't support javascript, but i read it here how this guy is using mechanize to handle the javascript response. And one thing I noticed, I have more or less the same problem and could probably use the same solution like he did. The reasons being:

1)The tab href is using the same __doPostBack() function

<a id="ctl00_mainPlaceHolder_btnSala" target="_blank" rel="nofollow noreferrer" href="javascript:__doPostBack('ctl00$mainPlaceHolder$btnSala','')">Trabajo en sala</a>

enter image description here

2)When I look at the source code, I could clearly see the form which is related to the javascript __doPostBack function:

enter image description here

So I have read the post of that guy wrote and tried to modified his solution into mine. And this is what I got so far:

require 'mechanize'

task :scraper_test => [:environment] do

    agent = Mechanize.new

    page = agent.get("https://www.camara.cl/camara/diputado_detalle.aspx?prmid=968")

    form = page.form("aspnetForm.add_field!('__EVENTTARGET','')")
    form.add_field!('ctl00$mainPlaceHolder$btnSala','')
    tab = agent.submit(form)

end

p/s: im using rake within rails app to run this.

But when i run it, I got this error:

NoMethodError: undefined method `add_field!' for nil:NilClass

So, can you help me to figure out the right way to do this? Thanks!

Ad

Answer

I just ran this in my console and you're getting this error

NoMethodError: undefined method `add_field!' for nil:NilClass

because this line returns nil

form = page.form("aspnetForm.add_field!('__EVENTTARGET','')")

Change it to this and it will fix that current error.

form = page.form("aspnetForm")
Ad
source: stackoverflow.com
Ad