幽灵资源网 Design By www.bzswh.com
花瓣图片的加载使用了延迟加载的技术,源代码只能下载20多张图片,修改后基本能下载所有的了,只是速度有点慢,后面再优化下
import urllib, urllib2, re, sys, os,requests
path=r"C:\wqa\beautify"
url = 'http://huaban.com/favorite/beauty'
#http://huaban.com/explore/zhongwenlogo/"User-Agent": "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.71 Safari/537.36"}
count=0
def urlHandle(url):
req = urllib2.Request(url, headers=i_headers)
html = urllib2.urlopen(req).read()
reg = re.compile(r'"pin_id":(\d+),.+"file":{"farm":"farm1", "bucket":"hbimg",.+"key":"(.*",.+"type":"image/(.*"', re.S)
groups = re.findall(reg, html)
return groups
def imgHandle(groups):
if groups:
for att in groups:
pin_id = att[0]
att_url = att[1] + '_fw236'
img_type = att[2]
img_url = 'http://img.hb.aicdn.com/' + att_url
r = requests.get(img_url)
with open(path + att_url + '.' + img_type, 'wb') as fd:
for chunk in r.iter_content():
fd.write(chunk)
groups = urlHandle(url)
imgHandle(groups)
while(groups):
count+=1
print count
pin_id = groups[-1][0]
print pin_id
urltemp = url+'/?max=' + str(pin_id) + '&limit=' + str(20) + '&wfl=1'
print(urltemp)
groups = urlHandle(urltemp)
#print groups
imgHandle(groups)
标签:
python图片爬虫
幽灵资源网 Design By www.bzswh.com
广告合作:本站广告合作请联系QQ:858582 申请时备注:广告合作(否则不回)
免责声明:本站文章均来自网站采集或用户投稿,网站不提供任何软件下载或自行开发的软件! 如有用户或公司发现本站内容信息存在侵权行为,请邮件告知! 858582#qq.com
免责声明:本站文章均来自网站采集或用户投稿,网站不提供任何软件下载或自行开发的软件! 如有用户或公司发现本站内容信息存在侵权行为,请邮件告知! 858582#qq.com
幽灵资源网 Design By www.bzswh.com
暂无评论...