python - CSS select with beautifulsoup4 doesn't work -


i tried bs4, select method doesn't work.

what's wrong code?

import requests import bs4  def main():     r = requests.get("http://nodejs.org/download/")     soup = bs4.beautifulsoup(r.text)      selector = "div.interior:nth-child(2) > table:nth-child(2) > tbody:nth-child(1) > tr:nth-child(1) > td:nth-child(3) > a:nth-child(1)"     print(soup.select(selector)[0].text)  if __name__ == "__main__":     main()

the answer on page differs viewing in browser parsing bs. have @ r.text , parse there.

the response 

<div class="interior row"> <div id="installers"> <ul> <li> <a href="http://nodejs.org/dist/v0.10.26/node-v0.10.26-x86.msi"> <img alt="" height="50" src="http://nodejs.org/images/platform-icon-win.png" width="45">               windows installer               <small>node-v0.10.26-x86.msi</small> </img></a> </li> <li> <a href="http://nodejs.org/dist/v0.10.26/node-v0.10.26.pkg"> <img alt="" height="50" src="http://nodejs.org/images/platform-icon-osx.png" width="45">               macintosh installer               <small>node-v0.10.26.pkg</small>

so there no table here. hope helps.

edit: code following response:

def main():     r = requests.get("http://nodejs.org/download/")     soup = bs4.beautifulsoup(r.text)     # print r.text     selector = "div.interior"     print(soup.select(selector)[2])

edit 2: try find. mor flexible one.

soup = bs4.beautifulsoup(r.text) print(soup.find("a", text="64-bit"))

edit 3: should work:

def main():     r = requests.get("http://nodejs.org/download/", headers={"content-type":"text", "user-   agent": "mozilla/5.0 (windows nt 6.1; wow64) applewebkit/536.5 (khtml, gecko) chrome/19.0.1084.52 safari/536.5"})     soup = bs4.beautifulsoup(r.text)     print(soup.find("table").tr.td.findnextsibling().a['href'])

Comments

Post a Comment

Popular posts from this blog

android - Get AccessToken using signpost OAuth without opening a browser (Two legged Oauth) -

i using signpost oauth access data magento server. have read various tutorials on same , reach point open browser user can enter credentials. however, per requirement have automate part. hence, user should not browser page. have set-up done on server side ( magento ), hit url , returned call page. same through program in android. i have tried below, consumer = new commonshttpoauthconsumer(key, secret); provider = new commonshttpoauthprovider(oauth_init_url,access_token_url, authorize_url); try { provider.retrieverequesttoken(consumer, oauth.out_of_band); log.d("tokens" , consumer.gettoken() + " -- " + consumer.gettokensecret()); } and request tokens. dont know how bypass next step. tried directly accessing accesstoken (stupid of me) provider.retrieveaccesstoken(consumer, "mycallback://callback"); no luck, ends in oauth.signpost.exception.oauthnotauthorizedexception: authorization failed (server replied 401). ...
Read more

org.mockito.exceptions.misusing.InvalidUseOfMatchersException: mockito -

getting error org.mockito.exceptions.misusing.invaliduseofmatchersexception: invalid use of argument matchers! 1 matchers expected, 2 recorded. exception may occur if matchers combined raw values: //incorrect: somemethod(anyobject(), "raw string"); when using matchers, arguments have provided matchers. example: //correct: somemethod(anyobject(), eq("string matcher")); at when( getprogramservice .callservice(any(getprogramtype.class))) .thenreturn(jaxbresponse); please explain error , possible resolution the code posted looks fine; don't see problem it. code before or above causing exception. key here: invalid use of argument matchers! 1 matchers expected, 2 recorded. any , used above, doesn't return "special kind of null" or "special kind of getprogramtype"; don't exist. instead, returns null , side effect puts matcher on stack. whe...
Read more

google shop client API returns 400 bad request error while adding an item -

i'm working on solution add items in google shopping market java application. following tutorial real mess because of missing classes of laste release, after setting 1.5-beta maven managed make base example typecheck correctly classes. still can make simple task of adding 1 item. instead of posting whole code i'm posting part creates product feed. private product createproduct(string id) { product product = new product(); product.title = "red wool sweater"; product.content = new content( "text", "comfortable , soft, sweater keep warm on " + "cold winter nights. red , blue stripes."); product.appcontrol = new appcontrol(); product.appcontrol.addrequireddestination("productads"); product.externalid = id; product.lang = "it"; product.country = "it"; product.condition = "nuovo"; product.price = new price("eu...
Read more