requests爬虫实践之虎扑步行街主干道帖子数据并保存为txt文件（python实现）

1.先将网页代码信息汇总，如下：

地址	数据名称
‘ul’,class_=‘for-list’	所有数据
‘div’,class_=‘titlelink box’	题目
‘div’,class_=‘titlelink box’> a[‘href’]	题目链接
‘span’,class_=‘endauthor’	最后回复者
‘div’,class_=‘author box’ >a	作者名字
‘div’,class_=‘author box’ > a[‘href’]	作者名字链接
‘div’,class_=‘endreply box’ >a	最后回复时间
‘div’,class_=‘author box’ >contents[5]	帖子创建时间
‘span’,class_=‘ansour box’	回复数和浏览数

注：
constants=['\n', <a class="aulink" href="https://my.hupu.com/223289078965017" target="_blank">颜歆陈</a>, '\n', <br/>, '\n', <a style="color:#808080;cursor: initial; ">2019-09-06</a>, '\n']
因为我们要的是 <a style="color:#808080;cursor: initial; ">2019-09-06</a>这串代码，所以取constents[5]。
2.源码

    import requests#调用头文件from bs4 import BeautifulSoupimport datetimedef get_page(link):headers={'user-agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.131 Safari/537.36'}r=requests.get(link,headers=headers)html=r.contenthtml=html.decode('utf-8')soup=BeautifulSoup(html,'lxml')return soupdef get_data(post_list):#定义抓取数据函数data_list=[]for post in post_list:title_name=post.find('div',class_='titlelink box').a.text.strip()#抓取题目 title_link='https://bbs.hupu.com'+post.find('div',class_='titlelink box').a['href']#抓取题目链接#因为链接在div.titlelink box名下的a.href中         author=post.find('div',class_='author box').a.text.strip()#抓取坐作者名字 author_link=post.find('div',class_='author box').a['href']#名字链接 start_time = post.find('div',class_='author box').contents[5].text.strip()#帖子创建时间 reply_view=post.find('span',class_='ansour box').text.strip()#回复数和浏览数reply=reply_view.split('/')[0].strip()#因为格式为‘回复／浏览数‘，因此使用split对字符串进行分割，并取其第0元素 view=reply_view.split('/')[1].strip()reply_time=post.find('div',class_='endreply box').a.text.strip()#最后回复时间reply_people=post.find('span',class_='endauthor').text.strip()#最后回复者 if ':' in reply_time:#如果有具体时间，那么就说明是在今天所发的贴子 date_time=str(datetime.date.today())+' '+reply_time#今天的日期年月和最后回复的具体时间date_time=datetime.datetime.strptime(date_time,'%Y-%m-%d %H:%M')#按照具体时间的格式来约束表示时间 else:#否则说明时间不是在今天，帖子仅显示年月日   date_time=datetime.datetime.strptime('2019-'+reply_time,'%Y-%m-%d').date()#2019－8－16data_list.append([title_name,title_link,author,author_link,start_time,reply,view,reply_people,date_time])#将后期的个体加入到列表中的最后 return data_listlink='https://bbs.hupu.com/bxj'#网址soup=get_page(link)post_all=soup.find('ul',class_='for-list')#包含所有数据post_list=post_all.find_all('li')#找到所有的帖子的开端data_list=get_data(post_list)for each in data_list:print(each)

注：若是想将运行结果保存为txt，只需在最后一个for循环的print后加上这段代码即可：

    with open('D:\HUPU.txt','a+')as f:f.writelines(str(each))f.write('\n')f.close()

3.运行结果：

['[招聘]2020虎扑校园招聘正式启动', 'https://bbs.hupu.com/29289329.html', '颜歆陈', 'https://my.hupu.com/223289078965017', '2019-09-06', '0', '787143', '颜歆陈', datetime.date(2019, 9, 6)]
['在《权力的游戏中》，如果你是罗柏史塔克你会怎么做?', 'https://bbs.hupu.com/29318502.html', '最初的夢想kzk', 'https://my.hupu.com/117163277131717', '2019-09-07', '38', '30429', '最初的夢想kzk', datetime.datetime(2019, 9, 8, 0, 50)]
['巴厘岛，真是令人有所失望', 'https://bbs.hupu.com/29311179.html', '奶油香蕉', 'https://my.hupu.com/80426953985446', '2019-09-07', '149', '118466', '起点是零', datetime.datetime(2019, 9, 8, 0, 50)]
['作为西北人，让我告诉你什么是正宗牛肉面！', 'https://bbs.hupu.com/29307440.html', '用户1810241125', 'https://my.hupu.com/164009692916253', '2019-09-07', '382', '783504', '马泽尔法克尔', datetime.datetime(2019, 9, 8, 0, 50)]
['请问有jrs有百度文库的会员吗？？？', 'https://bbs.hupu.com/29303140.html', '单手持球破紧紧逼', 'https://my.hupu.com/253840640250371', '2019-09-06', '141', '81508', '不忘初心的bboy', datetime.datetime(2019, 9, 8, 0, 50)]
['是不是人生就一定会有遗憾', 'https://bbs.hupu.com/29318911.html', 'Meronica', 'https://my.hupu.com/189252899224917', '2019-09-08', '5', '839', '一只大王', datetime.datetime(2019, 9, 8, 0, 50)]
['我坦白，我馋她的身子。', 'https://bbs.hupu.com/29314341.html', '三只小团团', 'https://my.hupu.com/199718694183378', '2019-09-07', '121', '117671', '澳洲袋鼠砸舢板', datetime.datetime(2019, 9, 8, 0, 50)]
['有个女生总是偷偷看我，我该怎么办', 'https://bbs.hupu.com/29314290.html', '万径人踪灭2', 'https://my.hupu.com/198879975892399', '2019-09-07', '221', '489149', 'All丶Might', datetime.datetime(2019, 9, 8, 0, 50)]
['大家觉得这个贼的智商适合做点啥？你们给点建议', 'https://bbs.hupu.com/29319013.html', '飞行员李丑萌', 'https://my.hupu.com/204957243107702', '2019-09-08', '1', '466', '虎扑最帅小林', datetime.datetime(2019, 9, 8, 0, 50)]
['有可能吗~', 'https://bbs.hupu.com/29318974.html', 'LBJ总教练', 'https://my.hupu.com/258352875598311', '2019-09-08', '2', '1143', '假情真钞换贞操', datetime.datetime(2019, 9, 8, 0, 50)]
['这双aj为什么没人炒？？', 'https://bbs.hupu.com/29299809.html', '吃葡萄吐瓜皮', 'https://my.hupu.com/70552830151364', '2019-09-06', '113', '127540', 'lsx0310', datetime.datetime(2019, 9, 8, 0, 50)]
['为什么我发现现在创业越来越难？', 'https://bbs.hupu.com/29318848.html', 'gvuhfv', 'https://my.hupu.com/15549252320327', '2019-09-08', '8', '1691', '吃菜养心', datetime.datetime(2019, 9, 8, 0, 50)]
['新郎比91年的新娘大44岁，这波什么水平？', 'https://bbs.hupu.com/29313717.html', '2科比4布莱8恩特', 'https://my.hupu.com/68503429326033', '2019-09-07', '207', '278181', '卡忙kevin', datetime.datetime(2019, 9, 8, 0, 50)]
['有没有什么保护脚踝好点的球鞋，jrs们求推荐', 'https://bbs.hupu.com/29313035.html', '一生科比2408', 'https://my.hupu.com/140173851944277', '2019-09-07', '19', '12616', 'Acooling', datetime.datetime(2019, 9, 8, 0, 50)]
['1700一年的宿舍，什么水平？', 'https://bbs.hupu.com/29316921.html', '老王家的羊', 'https://my.hupu.com/196122336972355', '2019-09-07', '317', '422814', '单车都几快窝', datetime.datetime(2019, 9, 8, 0, 50)]
['秋招结束，拿到街薪，作为女生满足了', 'https://bbs.hupu.com/29315004.html', '齐天小小小圣', 'https://my.hupu.com/278463345418505', '2019-09-07', '309', '658509', '在座的各位都是S_B', datetime.datetime(2019, 9, 8, 0, 50)]
['小区有人违建可以举报不？', 'https://bbs.hupu.com/29309066.html', '不吃猪肉会爆炸', 'https://my.hupu.com/264893487532558', '2019-09-07', '126', '118354', '没得名字吧', datetime.datetime(2019, 9, 8, 0, 50)]
['女孩模仿“易拉罐制爆米花”不幸离世,其父欲状告博主和平台 zt', 'https://bbs.hupu.com/29307424.html', '睡了我上铺', 'https://my.hupu.com/201067859476784', '2019-09-07', '210', '523004', '暮幽萧极寂', datetime.datetime(2019, 9, 8, 0, 50)]
['感觉现在谈恋爱好随便啊', 'https://bbs.hupu.com/29306372.html', '完美主义777', 'https://my.hupu.com/183371680299245', '2019-09-07', '43', '87346', '华夏时光', datetime.datetime(2019, 9, 8, 0, 50)]
['zt 我国男子女子三大球世界排名、亚洲排名一览表', 'https://bbs.hupu.com/29302638.html', '今今妈妈', 'https://my.hupu.com/266753461217201', '2019-09-06', '96', '93980', '你三婶摸男人丶', datetime.datetime(2019, 9, 8, 0, 50)]
['秋招终于可以结束了', 'https://bbs.hupu.com/29316417.html', '路口里弗斯', 'https://my.hupu.com/113928686066086', '2019-09-07', '50', '22817', '勇士小太阳', datetime.datetime(2019, 9, 8, 0, 50)]
['照片真的能看出来 满眼对一个人的爱', 'https://bbs.hupu.com/29316503.html', '就是你的轩轩', 'https://my.hupu.com/4628094406313', '2019-09-07', '101', '106666', '东河tom', datetime.datetime(2019, 9, 8, 0, 50)]
['广西清吧驻唱一个月6000正常吗？', 'https://bbs.hupu.com/29311678.html', '四根皮带', 'https://my.hupu.com/148677819286631', '2019-09-07', '38', '25249', '5咱们屯里的人5', datetime.datetime(2019, 9, 8, 0, 50)]
['哈哈哈，xswl', 'https://bbs.hupu.com/29318943.html', 'wwwwwww丶', 'https://my.hupu.com/242847236797794', '2019-09-08', '1', '222', '气焰极其嚣张', datetime.datetime(2019, 9, 8, 0, 50)]
['农村同学找了个深圳的女朋友，现在已经在深圳定居了', 'https://bbs.hupu.com/29311319.html', '三观最正的jrs', 'https://my.hupu.com/80683868062645', '2019-09-07', '34', '35587', '鹿角巷尾右转楼梯等', datetime.datetime(2019, 9, 8, 0, 50)]
['老实说，虽无感，但90后男性歌手中唱功领头应该是华晨宇没疑问吧', 'https://bbs.hupu.com/29310720.html', '我麦怒肛大威少', 'https://my.hupu.com/70157555907888', '2019-09-07', '120', '55389', '运尸的艺术', datetime.datetime(2019, 9, 8, 0, 50)]
['谁是中国体坛第一美女？', 'https://bbs.hupu.com/29312771.html', '罗李他', 'https://my.hupu.com/214830196548512', '2019-09-07', '213', '180995', 'DariusLEE', datetime.datetime(2019, 9, 8, 0, 50)]
['终于轮到我了，这波什么水平？彻底引起你们的舒适。', 'https://bbs.hupu.com/29309703.html', '搞死唐马儒', 'https://my.hupu.com/146050821901926', '2019-09-07', '31', '42002', 'afjejd', datetime.datetime(2019, 9, 8, 0, 50)]
['2019青岛大学毕业典礼，一首高音版《起风了》点燃全场', 'https://bbs.hupu.com/29289056.html', '冰帝迹部', 'https://my.hupu.com/35437063005932', '2019-09-06', '119', '79068', '执着成偏执', datetime.datetime(2019, 9, 8, 0, 50)]
['一个月前被分 原来是被绿了', 'https://bbs.hupu.com/29318421.html', '库里抱一下', 'https://my.hupu.com/214104650131660', '2019-09-07', '13', '4806', 'Claire1128', datetime.datetime(2019, 9, 8, 0, 50)]
['NBA2k20手机版太容易犯规了吧', 'https://bbs.hupu.com/29319033.html', '怕把加热后', 'https://my.hupu.com/75048731488423', '2019-09-08', '0', '0', '怕把加热后', datetime.datetime(2019, 9, 8, 0, 50)]
['搬进出租屋，身上起了湿疹，发现了这种虫子不知道有关系吗？', 'https://bbs.hupu.com/29304154.html', '天地斗', 'https://my.hupu.com/22522186662747', '2019-09-06', '80', '58983', '三井瘦子', datetime.datetime(2019, 9, 8, 0, 50)]
['Bloody的用法和fxxking差不多吗？', 'https://bbs.hupu.com/29318641.html', '脑子被驴踢了的小脑斧', 'https://my.hupu.com/3009255043936', '2019-09-08', '14', '6586', 'zhl73pro', datetime.datetime(2019, 9, 8, 0, 50)]
['Jrs，觉得倪妮怎么样？算大美人么？', 'https://bbs.hupu.com/29314916.html', 'cloudchen0123', 'https://my.hupu.com/200486834484663', '2019-09-07', '10', '5116', '刚毅66', datetime.datetime(2019, 9, 8, 0, 50)]
['鼻子太丑了', 'https://bbs.hupu.com/29319031.html', '梅花发', 'https://my.hupu.com/23357877173323', '2019-09-08', '0', '0', '梅花发', datetime.datetime(2019, 9, 8, 0, 50)]
['安吉丽娜朱莉身材什么水平？', 'https://bbs.hupu.com/29314893.html', '蕾蕾521', 'https://my.hupu.com/272027095736030', '2019-09-07', '35', '50356', '叫我艾科麦', datetime.datetime(2019, 9, 8, 0, 50)]
['我是不是被绿了？？', 'https://bbs.hupu.com/29308283.html', '黑子之篮球', 'https://my.hupu.com/113290820546011', '2019-09-07', '30', '83489', '步行街小魔童', datetime.datetime(2019, 9, 8, 0, 50)]
['老虎狮子放开来打，谁赢？', 'https://bbs.hupu.com/29317979.html', '有龙哲林', 'https://my.hupu.com/49259091285114', '2019-09-07', '14', '12255', '麦麦K', datetime.datetime(2019, 9, 8, 0, 50)]
['世界杯女主播这个是哪位？', 'https://bbs.hupu.com/29315573.html', 'mxjkfjdjd', 'https://my.hupu.com/129700207656981', '2019-09-07', '42', '73707', 'zyxxi', datetime.datetime(2019, 9, 8, 0, 50)]
['被发律师函了 家人们看看怎么办！', 'https://bbs.hupu.com/29297415.html', 'ascia', 'https://my.hupu.com/268464913791130', '2019-09-06', '15', '25230', '刚劲dei唷', datetime.datetime(2019, 9, 8, 0, 50)]
['撩妹高手们来支支招，我这有没有戏，没戏我就不坚持了。', 'https://bbs.hupu.com/29318758.html', '暴打牛魔王', 'https://my.hupu.com/119560984188301', '2019-09-08', '35', '10272', '一场漂泊', datetime.datetime(2019, 9, 8, 0, 50)]
['来品一品男友的价值观，他让我发虎扑问jr他对不对', 'https://bbs.hupu.com/29317556.html', '请以你的名字召唤我', 'https://my.hupu.com/120549958772569', '2019-09-07', '87', '84024', '算术代数几何', datetime.datetime(2019, 9, 8, 0, 50)]
['糖糖，我想你了', 'https://bbs.hupu.com/29318868.html', 'TScarlett', 'https://my.hupu.com/131292845844878', '2019-09-08', '6', '1392', '北卡悠', datetime.datetime(2019, 9, 8, 0, 50)]
['应届生！   分手、工作不顺心   求家人开解', 'https://bbs.hupu.com/29318927.html', 'wjose', 'https://my.hupu.com/163416712014467', '2019-09-08', '1', '826', '洋葱头1111', datetime.datetime(2019, 9, 8, 0, 50)]
['想找这样一个对象，是不是有点要求过高了？', 'https://bbs.hupu.com/29317647.html', 'AndrewCh', 'https://my.hupu.com/18021804379410', '2019-09-07', '19', '10654', '颓靡又传奇', datetime.datetime(2019, 9, 8, 0, 50)]
['妹子约我十一去武功山徒步，要带点什么', 'https://bbs.hupu.com/29311165.html', '虎尾巴上的兰花', 'https://my.hupu.com/238918255931399', '2019-09-07', '221', '399821', '茂名欧文', datetime.datetime(2019, 9, 8, 0, 50)]
['秦始皇在统一天下前的称号是什么？', 'https://bbs.hupu.com/29318835.html', 'AstralP', 'https://my.hupu.com/265768977544143', '2019-09-08', '3', '872', 'AstralP', datetime.datetime(2019, 9, 8, 0, 50)]
['河南马戏团出逃老虎被麻醉后捉回。村民：捉虎现场那是人山人海 zt', 'https://bbs.hupu.com/29311551.html', '詹姆斯洛特', 'https://my.hupu.com/167768960565670', '2019-09-07', '158', '589181', 'Dendro', datetime.datetime(2019, 9, 8, 0, 50)]
['请在历史上选出君，相，将三套班子，尽量考虑化学反应，如何尽可能延续一个王朝？', 'https://bbs.hupu.com/29292145.html', '用户1563424099', 'https://my.hupu.com/5619319600300', '2019-09-06', '227', '571690', '湖人精神股东', datetime.datetime(2019, 9, 8, 0, 50)]
['现在这社会低学历人士除了进工厂还有更好的选择吗？', 'https://bbs.hupu.com/29305746.html', '我是你耶耶', 'https://my.hupu.com/267111445771755', '2019-09-07', '649', '1107482', '荷包里的双人床', datetime.datetime(2019, 9, 8, 0, 50)]
['外企女HR，能给打几分？', 'https://bbs.hupu.com/29318863.html', '银色弹卡', 'https://my.hupu.com/72752967309321', '2019-09-08', '7', '594', '银色弹卡', datetime.datetime(2019, 9, 8, 0, 50)]
['为何完全没有战绩的赵云会评价如此之高？', 'https://bbs.hupu.com/29318676.html', '狙击手邓肯', 'https://my.hupu.com/94966370078092', '2019-09-08', '47', '21973', '狙击手邓肯', datetime.datetime(2019, 9, 8, 0, 50)]
['在古代大规模战役中，一个统帅的个人能力能起到多大的决定性作用？', 'https://bbs.hupu.com/29308490.html', '朱棣', 'https://my.hupu.com/214957620707760', '2019-09-07', '65', '44756', 'PeterZP', datetime.datetime(2019, 9, 8, 0, 50)]
['为什么手机版2k20没有球迷?', 'https://bbs.hupu.com/29289899.html', 'stage30', 'https://my.hupu.com/265450635579099', '2019-09-06', '139', '90159', 'NBA名宿莱昂纳德', datetime.datetime(2019, 9, 8, 0, 50)]
['我妈宁愿花二百万现金买理财，也不愿意买房投资，该怎么改变老年人的想法？', 'https://bbs.hupu.com/29313322.html', '惜潮', 'https://my.hupu.com/278333795603598', '2019-09-07', '274', '367227', 'Srdpty', datetime.datetime(2019, 9, 8, 0, 50)]
['不吹不黑，尊龙杨洋严屹宽古天乐，谁颜值最高？', 'https://bbs.hupu.com/29302417.html', '运尸的艺术', 'https://my.hupu.com/74744679431734', '2019-09-06', '90', '67816', '曼小猪', datetime.datetime(2019, 9, 8, 0, 50)]
['在抖音看到的美国小哥哥，好羡慕(/≧≦/)，为什么他们可以这么帅', 'https://bbs.hupu.com/29315797.html', '那時年少啊', 'https://my.hupu.com/141931968195440', '2019-09-07', '17', '10123', 'BlackRussian', datetime.datetime(2019, 9, 8, 0, 50)]
['闲鱼碰上憨批怎么办？', 'https://bbs.hupu.com/29313270.html', '克瑞斯炮丶', 'https://my.hupu.com/2088025057683', '2019-09-07', '458', '700928', '河东尼克杨', datetime.datetime(2019, 9, 8, 0, 50)]
['长腿的文艺大叔的征婚帖', 'https://bbs.hupu.com/29318750.html', '爱和自由', 'https://my.hupu.com/81857571008087', '2019-09-08', '10', '5905', '爱和自由', datetime.datetime(2019, 9, 8, 0, 50)]
['感觉武汉不宜居啊', 'https://bbs.hupu.com/29314596.html', '禁欲未满', 'https://my.hupu.com/47140837591629', '2019-09-07', '155', '111407', '贴吧黑子撕马', datetime.datetime(2019, 9, 8, 0, 50)]
['孩子快一岁了，媳妇儿看不起我要离婚！', 'https://bbs.hupu.com/29318626.html', '强哥哥哥', 'https://my.hupu.com/259571342383986', '2019-09-07', '36', '71294', '赢得慢', datetime.datetime(2019, 9, 8, 0, 49)]
['第一次吃人均900左右的餐厅，有钱人的生活这么有震撼力吗？', 'https://bbs.hupu.com/29314350.html', '我们一不一样ne', 'https://my.hupu.com/63206717720605', '2019-09-07', '424', '718388', 'NO.1大胖胖', datetime.datetime(2019, 9, 8, 0, 49)]
['仙逆，求魔，凡人修仙传哪部书更好看', 'https://bbs.hupu.com/29142407.html', 'TASF', 'https://my.hupu.com/143838067155896', '2019-08-27', '4', '439', 'MNJR祝君好', datetime.datetime(2019, 9, 8, 0, 49)]
['又见捕鼠神器，我竟然无聊到看完了 zt', 'https://bbs.hupu.com/29307764.html', '辗转难眠叫', 'https://my.hupu.com/36802551729664', '2019-09-07', '202', '667449', 'I胖锤', datetime.datetime(2019, 9, 8, 0, 49)]
['儿子的幼儿园，觉得都挺正常的，没有梓化。', 'https://bbs.hupu.com/29223209.html', 'BosRondo', 'https://my.hupu.com/262078426283073', '2019-09-02', '347', '285086', '哈姆詹斯登', datetime.datetime(2019, 9, 8, 0, 49)]
['晚上没看中国队比赛，看了下回放，发现这一球，在还剩1.7秒时，下面计时间的停了一下', 'https://bbs.hupu.com/29305314.html', '小政真', 'https://my.hupu.com/23853017318030', '2019-09-07', '15', '42911', '小政真', datetime.datetime(2019, 9, 8, 0, 49)]
['在广州毕业生找电商工作', 'https://bbs.hupu.com/29318967.html', '只是有我', 'https://my.hupu.com/26417658639412', '2019-09-08', '2', '579', '庄庄大吉吧', datetime.datetime(2019, 9, 8, 0, 49)]
['我下次一定会带头盔的', 'https://bbs.hupu.com/29317980.html', 'Styles', 'https://my.hupu.com/266610719718665', '2019-09-07', '17', '18832', '禽兽啊', datetime.datetime(2019, 9, 8, 0, 49)]
['蔡徐坤《没有意外》live，什么水平', 'https://bbs.hupu.com/29316614.html', '古朴一霸', 'https://my.hupu.com/66618448176010', '2019-09-07', '73', '65695', '盖世太保485', datetime.datetime(2019, 9, 8, 0, 49)]
['应届大专毕业生失业三个月了', 'https://bbs.hupu.com/29313312.html', '德怀恩韦德666', 'https://my.hupu.com/154201786020661', '2019-09-07', '65', '39988', '好我是球迷', datetime.datetime(2019, 9, 8, 0, 49)]
['看着认真工作的艾露猫，真的很治癒！ zt', 'https://bbs.hupu.com/29312315.html', '杀掉比尔', 'https://my.hupu.com/174042810189695', '2019-09-07', '5', '3718', 'whycry', datetime.datetime(2019, 9, 8, 0, 49)]
['《中餐厅》这段，怀疑林大厨是假戏真做', 'https://bbs.hupu.com/29191812.html', '滋滋滋麦子', 'https://my.hupu.com/266254906683463', '2019-08-31', '567', '1451043', 'kaggi', datetime.datetime(2019, 9, 8, 0, 49)]
['秋招被家门口一家国企录取了什么水平', 'https://bbs.hupu.com/29316340.html', 'sean_fan', 'https://my.hupu.com/145411095333586', '2019-09-07', '54', '62066', 'Gy', datetime.datetime(2019, 9, 8, 0, 49)]
['如果以亚洲人的身体，像欧洲球员那样的训练强度，到底能不能练出肌肉棒子？', 'https://bbs.hupu.com/29273430.html', '唠嗑复出当教练', 'https://my.hupu.com/60650640966472', '2019-09-05', '232', '127126', '少年锦时24', datetime.datetime(2019, 9, 8, 0, 49)]
['女朋友发给我这个，求问街上英语大佬，怎么怼回去', 'https://bbs.hupu.com/29305094.html', '富儿子他爹', 'https://my.hupu.com/239693609830832', '2019-09-07', '34', '56833', '阿姆斯特朗回旋加速喷', datetime.datetime(2019, 9, 8, 0, 49)]
['在酒吧一个女的用手握了一下我胳膊，感觉有些刺痛', 'https://bbs.hupu.com/29318836.html', 'Yesigner', 'https://my.hupu.com/50854059037796', '2019-09-08', '19', '6073', 'Yesigner', datetime.datetime(2019, 9, 8, 0, 49)]
['你们初中的前三名现在过得怎么样了?', 'https://bbs.hupu.com/29295265.html', '异世李风行', 'https://my.hupu.com/230056427470015', '2019-09-06', '448', '270205', '成为传奇者', datetime.datetime(2019, 9, 8, 0, 49)]
['昨天去看男篮比赛遇上小楠，真人真的好漂亮！', 'https://bbs.hupu.com/29308312.html', '我就是哈技师粉丝', 'https://my.hupu.com/186855993089966', '2019-09-07', '303', '598136', '谭浩然', datetime.datetime(2019, 9, 8, 0, 49)]
['jrs遇到这种情况应该怎么办', 'https://bbs.hupu.com/29318988.html', '吃赫糖', 'https://my.hupu.com/70760043137972', '2019-09-08', '1', '142', '假情真钞换贞操', datetime.datetime(2019, 9, 8, 0, 49)]
['不懂就问，嚼槟榔有啥用', 'https://bbs.hupu.com/29257891.html', '太阳的雨Kobe', 'https://my.hupu.com/106978615508593', '2019-09-04', '5', '2902', '腳下抹油', datetime.datetime(2019, 9, 8, 0, 49)]
['遗传精神病真的没救么', 'https://bbs.hupu.com/29318934.html', 'eifl', 'https://my.hupu.com/191310645212606', '2019-09-08', '1', '2234', '宠妻狂魔11', datetime.datetime(2019, 9, 8, 0, 49)]
['李连杰的黄飞鸿真的是清秀正气，充满英雄气概！', 'https://bbs.hupu.com/29311705.html', '列夫托尔斯群', 'https://my.hupu.com/109744153240189', '2019-09-07', '26', '22309', '任天堂流泪2006', datetime.datetime(2019, 9, 8, 0, 49)]
['30岁男人', 'https://bbs.hupu.com/29311571.html', '人称sao哥', 'https://my.hupu.com/17981093065077', '2019-09-07', '11', '7464', '373284607', datetime.datetime(2019, 9, 8, 0, 49)]
['有没有钓鱼老手 清道夫到底能不能吃？', 'https://bbs.hupu.com/29317824.html', '时久番', 'https://my.hupu.com/187620856352235', '2019-09-07', '56', '43443', '情圣太阳神l', datetime.datetime(2019, 9, 8, 0, 49)]
['男友月薪一万送礼一千……后续来了……', 'https://bbs.hupu.com/29315844.html', '云南白药创口贴', 'https://my.hupu.com/201658183143156', '2019-09-07', '165', '163222', '湖人精神股东', datetime.datetime(2019, 9, 8, 0, 49)]
['在世界历史中，拿破仑算是什么水平的将领？', 'https://bbs.hupu.com/29313822.html', 'w13955386654', 'https://my.hupu.com/200183457205826', '2019-09-07', '228', '480559', '漢壽亭侯關雲長', datetime.datetime(2019, 9, 8, 0, 49)]
['深夜了，为什么你还没睡？', 'https://bbs.hupu.com/29319023.html', '我是小萝卜头', 'https://my.hupu.com/202524139764982', '2019-09-08', '0', '13', '我是小萝卜头', datetime.datetime(2019, 9, 8, 0, 49)]
['就这样，一直跑下去，到四十岁！', 'https://bbs.hupu.com/29319021.html', '迷鹿婚礼电影高老师', 'https://my.hupu.com/185836661826646', '2019-09-08', '0', '3', '迷鹿婚礼电影高老师', datetime.datetime(2019, 9, 8, 0, 49)]
['在闲鱼卖卡被骗，求助JRS分析', 'https://bbs.hupu.com/29318874.html', '我爱绿军', 'https://my.hupu.com/70580954544089', '2019-09-08', '4', '5440', '我爱绿军', datetime.datetime(2019, 9, 8, 0, 49)]
['老实讲，假如你是东哥，会娶章泽天吗？', 'https://bbs.hupu.com/29316513.html', '飞行员李丑萌', 'https://my.hupu.com/204957243107702', '2019-09-07', '15', '7595', '安守RUSSELL', datetime.datetime(2019, 9, 8, 0, 48)]
['夜里12点还在打王者的舍友，大嚷大叫的是怎么想的', 'https://bbs.hupu.com/29318705.html', '永远爱LBL', 'https://my.hupu.com/253131478263680', '2019-09-08', '9', '3863', '最爱李梦林', datetime.datetime(2019, 9, 8, 0, 48)]
['白人妹子喝水少饮食高热量那她们口臭严重吗？', 'https://bbs.hupu.com/29314434.html', '侏儒症患者', 'https://my.hupu.com/138102463774864', '2019-09-07', '36', '37149', 'CTWVC', datetime.datetime(2019, 9, 8, 0, 48)]
['懂猫咪的老哥看下，这猫藓严重吗', 'https://bbs.hupu.com/29288959.html', '找个阳光可爱的女朋友', 'https://my.hupu.com/17793737806940', '2019-09-06', '9', '4144', '小柴柴呀', datetime.datetime(2019, 9, 8, 0, 48)]
['我女儿这个看似简单的动作，有多少人能做到呢', 'https://bbs.hupu.com/29285860.html', '大河马大', 'https://my.hupu.com/130071011920094', '2019-09-05', '172', '411227', '跃一', datetime.datetime(2019, 9, 8, 0, 48)]
['吃霸王餐逃跑摔伤向店主索赔4万 zt', 'https://bbs.hupu.com/29314814.html', '我服了有', 'https://my.hupu.com/249779574187168', '2019-09-07', '19', '19164', 'TonyStork', datetime.datetime(2019, 9, 8, 0, 48)]
['拍一次小牙片等于吃10根香蕉？关于牙科X光辐射剂量知多少', 'https://bbs.hupu.com/29318452.html', 'Ngailung', 'https://my.hupu.com/76429834450939', '2019-09-07', '3', '1487', 'embfour', datetime.datetime(2019, 9, 8, 0, 48)]
['暴雪，第二天早上才想起，狗还栓在外面 zt', 'https://bbs.hupu.com/29308134.html', '保罗我科', 'https://my.hupu.com/272078736149864', '2019-09-07', '147', '435130', '2324Wang', datetime.datetime(2019, 9, 8, 0, 48)]
['你觉得cpa会计中哪一章节难度最大？', 'https://bbs.hupu.com/29318769.html', '詹姆赵四', 'https://my.hupu.com/16825422681943', '2019-09-08', '4', '1269', '黄金大白菜', datetime.datetime(2019, 9, 8, 0, 48)]
['各位相亲了几次才成功？', 'https://bbs.hupu.com/29315695.html', '开始起飞了', 'https://my.hupu.com/26979084275218', '2019-09-07', '21', '14086', '一只大王', datetime.datetime(2019, 9, 8, 0, 48)]
['女警上厕所忘记关对讲机 结果被全组听到 尴尬了', 'https://bbs.hupu.com/29306412.html', 'GD泫雅我男女神', 'https://my.hupu.com/118459397194781', '2019-09-07', '137', '586186', 'moonlight2010', datetime.datetime(2019, 9, 8, 0, 48)]
['讲真这段子真发生在我身上，我可能会翻脸啊', 'https://bbs.hupu.com/29314900.html', '峨眉教主张无忌', 'https://my.hupu.com/87703412554578', '2019-09-07', '23', '15798', '硬刚虎扑不怕封', datetime.datetime(2019, 9, 8, 0, 48)]
['我有戏吗', 'https://bbs.hupu.com/29318992.html', '情书小老弟老铁', 'https://my.hupu.com/231402752507156', '2019-09-08', '1', '271', '假情真钞换贞操', datetime.datetime(2019, 9, 8, 0, 48)]
['说说最近无限循环的一首歌', 'https://bbs.hupu.com/29318649.html', '科詹威时代', 'https://my.hupu.com/232299748569546', '2019-09-08', '20', '8511', '御剑飞行官', datetime.datetime(2019, 9, 8, 0, 48)]
['有男人时常约自己的女朋友蹦迪喝酒等夜生活', 'https://bbs.hupu.com/29318774.html', '白驹过隙时光荏苒', 'https://my.hupu.com/274192206822671', '2019-09-08', '8', '3037', '白驹过隙时光荏苒', datetime.datetime(2019, 9, 8, 0, 48)]
['穷，可以让一个男人卑微到什么地步', 'https://bbs.hupu.com/29278182.html', '炜纬玮伟韦', 'https://my.hupu.com/59839486706433', '2019-09-05', '730', '929615', '有梦想的样子', datetime.datetime(2019, 9, 8, 0, 48)]
['一个东北人关于野狼Disco的一些看法', 'https://bbs.hupu.com/29318887.html', '跑道上的柠檬', 'https://my.hupu.com/85875337126450', '2019-09-08', '10', '5872', '欧浩成', datetime.datetime(2019, 9, 8, 0, 48)]
['CBA第一帅杨鸣与泳坛第一帅宁泽涛谁颜值更高', 'https://bbs.hupu.com/29313629.html', '亚洲顶级后卫郭艾伦', 'https://my.hupu.com/165756463000785', '2019-09-07', '335', '499949', '独行侠总冠军', datetime.datetime(2019, 9, 8, 0, 48)]
['野球场打球问题', 'https://bbs.hupu.com/29318232.html', '熹熹哈哈哈哈哈哈哈哈哈', 'https://my.hupu.com/77838916740846', '2019-09-07', '4', '5354', '肥皂侠', datetime.datetime(2019, 9, 8, 0, 48)]
['备受争议的霸榜歌手，花粥！你给几分？！？', 'https://bbs.hupu.com/29273656.html', '保护我方林大俊', 'https://my.hupu.com/17176955021810', '2019-09-05', '330', '473776', '晴明SAMA', datetime.datetime(2019, 9, 8, 0, 48)]
['自己喜欢的和喜欢自己的，选哪个？', 'https://bbs.hupu.com/29318764.html', '原来名字能起这么长', 'https://my.hupu.com/278594050298536', '2019-09-08', '12', '5882', 'Mark丶z', datetime.datetime(2019, 9, 8, 0, 48)]
['问一下，网文作家烽火戏诸侯是什么水平', 'https://bbs.hupu.com/29315571.html', '种花师兄', 'https://my.hupu.com/14444892858673', '2019-09-07', '134', '76006', '小二斗鱼', datetime.datetime(2019, 9, 8, 0, 48)]
['下个月去三亚了，麻烦各位吴彦祖看下我定的酒店如何', 'https://bbs.hupu.com/29314593.html', 'FinalBell', 'https://my.hupu.com/151624689841154', '2019-09-07', '15', '6421', '李白说粤语', datetime.datetime(2019, 9, 8, 0, 48)]
['终于轮到我了，前一段刚刚从朝鲜旅游回来，大家有什么想问的没有。', 'https://bbs.hupu.com/29312249.html', '年华恍然', 'https://my.hupu.com/18638529839326', '2019-09-07', '4', '2020', '校队首发中锋老虎', datetime.datetime(2019, 9, 8, 0, 48)]
['女朋友一定要和学长单独吃饭 多图', 'https://bbs.hupu.com/29310850.html', '楼外七楼', 'https://my.hupu.com/264562258687835', '2019-09-07', '450', '698934', '我我她', datetime.datetime(2019, 9, 8, 0, 48)]

requests爬虫实践之虎扑步行街主干道帖子数据并保存为txt文件（python实现）相关推荐

Python网络爬虫爬取虎扑步行街爆照区话题
Python网络爬虫爬取虎扑步行街爆照区话题作者:郜科科最近的任务挺多的,但是心情很烦躁,想做一些自己喜欢的东西,前些时候感觉Python爬虫很好玩,就自学了一下.本人比较喜欢逛街--虎扑步行街, ...
python网络爬虫-爬取虎扑步行街数据
前言由于虎扑页面的限制,因为访问虎扑步行街的第11个页面就需要用户进行登录,鉴于此时技术还没有学全,只能爬取1到10的页面. 抓取什么数据帖子名称帖子链接发帖人发帖人链接发帖时间帖子回复 ...
python爬虫爬取虎扑步行街爆照区图片
开始动手的时候毫无头绪,主要借鉴了以下帖子: https://blog.csdn.net/sinat_28797501/article/details/70195443 但是后来发现代码中还是有一些不 ...
python爬虫之虎扑步行街主题帖
前言 python爬虫的盛行让数据变得不在是哪么的难以获取.现在呢,我们可以根据我们的需求去寻找我们需要的数据,我们下来就利用python来写一个虎扑步行街主题帖的基本信息,主要包括:帖子主题(tit ...
爬取虎扑步行街——秋名山论美美女壁纸图片
最近学了学爬虫,由于平时笔者经常刷虎扑,于是决定实战一下,对虎扑上的美女图片进行爬取,特地来分享一下经验. 这次主要使用了request和BeautifulSoup两个库,使用urllib库对图片进行 ...
MongoDB爬虫实践：虎扑论坛
实践目的: 本实践项的目的是获取虎扑步行街论坛上所有帖子的数据,网站地址如下: https://bbs.hupu.com/bxj 实践代码: import requests from bs4 impo ...
仿造虎扑步行街首页。
原网站 https://bbs.hupu.com/all-gambia 我非常喜欢虎扑,所以我打算仿造一个虎扑论坛,这是仿造的第一个网页,也是虎扑步行街的首页,我仿造这个页面并对这个页面进行了优化,比 ...
BeautifulSoup爬取虎扑步行街信息到Mysql
人生苦短,我用python 今天给大家爬取虎扑步行街的详细信息效果图如下: 首先根据虎扑步行街链接获取前十个页面链接 for i in range(1, 10):link = "https ...
爬取虎扑步行街论坛数据保存到MySQL、MongoDB
本文内容围绕获取虎扑步行街论坛上所有帖子的数据开展的,爬取的内容包括帖子名称.帖子链接.作者.作者链接.创建时间.回复数.浏览数.最后回复用户和最后回复时间,将爬取的数据分别保存在MongoDB和My ...

requests爬虫实践之虎扑步行街主干道帖子数据并保存为txt文件（python实现）

requests爬虫实践之虎扑步行街主干道帖子数据并保存为txt文件（python实现）相关推荐

最新文章

热门文章