Python爬取麻辣烫所有视频

# !/usr/bin/env python
# -*-coding:utf-8-*-
# date :2021/4/16 12:57
# author:Sabo
import os
import requests
from bs4 import BeautifulSoupsavePath = 'F:/麻辣烫耙耳朵'
root = 'http://show.sctv.com/mlt/index'
urlTail = '.shtml'
headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.104 Safari/537.36','Connection': 'close'}def getRootUrls(printFlag):originUrl = ''rootUrls = []for i in range(0, 10):if i is not 0:originUrl = root + '_' + i.__str__() + urlTailelse:originUrl = root + urlTailrootUrls.append(originUrl)if printFlag is True:print(rootUrls)return rootUrlsdef getLinksPerRootUrl(rootUrl, printFlag):response = requests.get(url = rootUrl,  header = headers)if response.status_code != 200:print('Get response error!')return ''else:response.encoding = 'utf-8'txt = response.textmainLink = BeautifulSoup(txt, 'html.parser')childLinks = []mainLink_txt = mainLink.find_all('div', attrs={"class": "txt"})for i in range(0, mainLink_txt.__len__()):link = mainLink_txt[i].find_next('a')href = link.get('href')childLinks.append(href)if printFlag == True:print(childLinks)return childLinksdef catUrl(catFlag, signalLinks):root = 'http://show.sctv.com/mlt'result = []for index in range(0, signalLinks.__len__()):if catFlag == 0:result.append(root+signalLinks[index][1:])else:result.append(root + catFlag.__str__() + signalLinks[index][1:])return resultdef urlTitles(rootUrl):titles = []response = requests.get(url=rootUrl, header = headers)if response.status_code != 200:print('Get titles error!')return ''response.encoding ='utf-8'txt = response.textmainPage = BeautifulSoup(txt, 'html.parser')nameLinks = mainPage.find_all('div', attrs={'class':'name'})for index in range(0, nameLinks.__len__()):titles.append(nameLinks[index].text)return titlesdef download(savePath, titles, links):for index in range(0, titles.__len__()):commond = 'you-get -o {0} -O {1} "{2}"'.format(savePath, titles[index], links[index])print(commond)os.system(commond)def main():RootUrls = getRootUrls(printFlag = False)catFlag = 0for RootUrl in RootUrls:links = getLinksPerRootUrl(rootUrl=RootUrl, printFlag=False)titles = urlTitles(rootUrl=RootUrl)dstUrls = catUrl(catFlag = catFlag, signalLinks=links)download(savePath=savePath, titles=titles, links=dstUrls)if __name__ == '__main__':main()

Python爬取麻辣烫所有视频相关推荐

Python爬取麻辣烫首页视频并保存到本地
code: # !/usr/bin/env python # -*-coding:utf-8-*- # date :2021/3/10 22:23 # author:Sabo import reque ...
python爬取b站评论_学习笔记(1):写了个python爬取B站视频评论的程序
学习笔记(1):写了个python爬取B站视频评论的程序 import requests import json import os table='fZodR9XQDSUm21yCkr6zBqiveY ...
python爬取bilibili弹幕_用Python爬取B站视频弹幕
原标题:用Python爬取B站视频弹幕 via:菜J学Python 众所周知,弹幕,即在网络上观看视频时弹出的评论性字幕.不知道大家看视频的时候会不会点开弹幕,于我而言,弹幕是视频内容的良好补充,是一 ...
python爬取抖音用户数据_使用python爬取抖音视频列表信息
如果看到特别感兴趣的抖音vlogger的视频,想全部dump下来,如何操作呢?下面介绍介绍如何使用python导出特定用户所有视频信息抓包分析 Chrome Deveploer Tools Chro ...
python爬b站评论_学习笔记(1):写了个python爬取B站视频评论的程序
学习笔记(1):写了个python爬取B站视频评论的程序 import requests import json import os table='fZodR9XQDSUm21yCkr6zBqiveY ...
使用python爬取最右视频
最近学习了python,在此记录一下用python爬取最右视频首先安装mitmproxy,安装方式:pip install mitmproxy 因为我使用的是Windows系统,无法使用mitmpr ...
python爬取m3u8格式视频
python爬取m3u8格式视频 m3u8原理脚本环境爬取步骤步骤的具体实现 python细节处理说明参考代码 m3u8原理当我们在网页播放视频时,网页向服务器发起一个以.m3u8结尾的连接 ...
python爬取腾讯视频会员V力值
python爬取腾讯视频会员V力值练练手,只需要将cookies改成自己的便即可运行. from bs4 import BeautifulSoup import requests import r ...
python爬取腾讯视频弹幕_用Python爬取腾讯视频弹幕
原标题:用Python爬取腾讯视频弹幕 via:菜J学Python 1.网页分析本文以爬取<脱口秀大会第3季>最后一期视频弹幕为例,首先通过以下步骤找到存放弹幕的真实url. 通过删减 ...

Python爬取麻辣烫所有视频

Python爬取麻辣烫所有视频相关推荐

最新文章

热门文章