selenium - Update response.body in scrapy(without reload) -


i use scrapy , selenium crawl! site use ajax pagination! actully , url no change , response.body no change! want click selenium (for pagination) , self.driver.page_source , use instead response.body! writed code :

 res = scrapy.http.textresponse(url=self.driver.current_url, body=self.driver.page_source,                                            encoding='utf-8')             print(str(res)) //nothing print!              quote in res.css("#ctl00_contentplaceholder1_grd_dr_dxmaintable > tr.dxgvdatarow_office2003blue"):                 = i+1                 item = dict()                   item['id'] =  int(quote.css("td.dxgv:nth-child(1)::text").extract_first()) 

and no error !

you can replace body of original response in scrapy using response.replace() method:

def parse(self, response):     response = response.replace(body=driver.page_source) 

Comments

Popular posts from this blog

account - Script error login visual studio DefaultLogin_PCore.js -

xcode - CocoaPod Storyboard error: -