python - Scraping poorly structured HTML -


i have website want scrape using scrapy has html structure shown @ bottom of post (titled html). want able extract information contained first <td class="small-txt dkgrey-txt rightinfotd">, i.e., 1 contains <span property =""> tag. using following code snippet try , grab data 

listings = selector.css("div.whenwherecontent")          listing in listings:             body in listing.css('td.small-txt.dkgrey-txt.rightinfotd')

however, since there multiple <td> tags same class of td.small-txt.dkgrey-txt.rightinfotd (see admission , tickets data @ bottom of html code), getting duplicate results. how can restrict for loop <td> tag correct data avoid problem?

html

<div class="whenwherecontent">     <table width="100%" cellpadding="0" cellspacing="5">             <tr>             <td class="small-txt medgrey-txt leftlabeltd">                             </td>             <td class="small-txt dkgrey-txt rightinfotd">                 <span property="v:name">                     sound academy                 </span>                 <span property="v:street-address">                 11 polson                 </span>                 <span property="v:locality">                     toronto                 </span>                  <span property="v:postal-code">                 m5a 1a4                 </span>                 <span property="v:tel" style="white-space: nowrap;">                     416-461-3625                 </span>                  info@sound-academy.com                  <a href="http://sound-academy.com" style="font-weight:900">                     <span property="v:url">                         sound-academy.com                     </span>                 </a>             </td><         </tr>          <tr>             <td class="small-txt medgrey-txt leftlabeltd">                 admission             </td>              <td class="small-txt dkgrey-txt rightinfotd">                 $39.50-$55             </td>         </tr>          <tr>             <td class="small-txt medgrey-txt leftlabeltd">                 tickets @             </td>              <td class="small-txt dkgrey-txt rightinfotd">                 ln, rt, ss             </td>         </tr>          <tr>             <td class="small-txt medgrey-txt leftlabeltd">                 when             </td>                                                <td class="rightinfotd">                 <div class="small-txt dkgrey-txt">                     <span property="v:datestart" content="2014-03-24">                         mar&nbsp;24                     </span>                       <span property="v:datestart" content="2014-03-25">                         mar&nbsp;25                     </span>                  </div>             </td>         </tr> </div>

if want restrict td in first tr can use :nth-child() pseudo-class:

listing.css('tr:nth-child(1) td.small-txt.dkgrey-txt.rightinfotd')

or equivalently:

listing.css('tr:first-child td.small-txt.dkgrey-txt.rightinfotd')

css selectors can quite helpful , easier maintain. in cases, xpath may way achieve specific selection. in case, selecting td contains <span property="v:name"> can like

listing.xpath('.//td[ span[ @property="v:name" ] ]')

Comments

Post a Comment

Popular posts from this blog

user interface - How to replace the Python logo in a Tkinter-based Python GUI app? -

is there way change default logo, python logo , appears in window's task bar? notice have replaced default tk logo used appear in application window. i using windows 7 , python 2.6 , developing gui of tkinter. you can using winico tk extension package. winico package can used add system tray icons tk programs. the following sample shows 1 way change runtime application icon. note need provide .ico file suitable sizes of icons in on command line , need use pythonw. not change taskbar icon console when running python script. test extracted winico0.6 package python\tcl\winico0.6 folder package require winico work , ran code below using pythonw winico_test.py path\to\some\ico\file.ico . import sys tkinter import * def main(argv): root = tk() root.update() root.tk.call('package','require','winico') id = root.tk.call('winico','createfrom',argv[1]) root.tk.call('winico','setwindow',root,id...
Read more

objective c - Greedy NSProgressIndicator Allocation -

i'm using bunch of nsprogressindicators @ once (32 exact) , looks when they're animating start allocating memory no one's business greedy_run http://i62.tinypic.com/15rgmso.png i've got reports app slowdown , crash after days of uptime, , looks culprit, it's hard pinpoint bugs take days appear. i comment out call start animating, memory behaves expected. result of multiple nsprogressindicators alloc'ed on same thread? or maybe sandboxing? of constant allocations coming core animation threads, don't know else going on. custom nsview progress indicator lives in @interface myview () @property (nonatomic, strong) nsimageview *imageview; @property (nonatomic, strong) nstextview *numbertxt; @property (nonatomic, strong) nsprogressindicator *progress; @end @implementation myview - (id)initwithframe:(nsrect)framerect { self = [super initwithframe:framerect]; if (self) { // image nsimage *image = [nsimage imagenamed:@"bg.png"]; c...
Read more

how to set an OCR language in Google Drive -

i need upload image google drive including ocr feature. the problem setocrlanguage method. need specify hebrew language , based on iso 639-1 codes should "heb". i'm getting response "code" : 400, "errors" : [ { "domain" : "global", "location" : "ocrlanguage", "locationtype" : "parameter", "message" : "invalid value", "reason" : "invalid" } ], any idea should code or can list of avaliable codes?
Read more