【Python爬虫】爬取网易云评论

爬取网易云音乐最新评论

import requests
import time
import json
import datetime
import xlwt
headers = {'Host': 'music.163.com','User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36'
}
def get_comments(songId,page):url = 'http://music.163.com/api/v1/resource/comments/R_SO_4_{0}?limit=100&offset={1}'.format(songId,str(page)) #字符串格式化response = requests.get(url=url, headers=headers)result = json.loads(response.text)temp = []for item in result['comments']:data = {}# 评论者iddata['userId'] = item['user']['userId']# 评论者昵称data['nickname'] = item['user']['nickname']# 评论内容data['content'] = item['content'].replace("\n",",")# 评论发表时间timeArray = time.localtime(item['time'] / 1000)date = time.strftime("%Y-%m-%d %H:%M:%S", timeArray)data['date'] = date# 点赞数data['likedCount'] = item['likedCount']temp.append(data)return temp
if __name__ == '__main__':list = ["评论者id","评论者昵称","评论内容","评论发表时间","点赞数"]workbook = xlwt.Workbook(encoding="utf-8")worksheet = workbook.add_sheet("sheet1",cell_overwrite_ok=True)  # 创建工作表for index, item in enumerate(list):worksheet.write(0, index, item)for i in range(0,1101,100):temps = get_comments("1330348068", i)count = i + 1for row_index,temp in enumerate(temps):for col_index, item in enumerate(temp.values()):worksheet.write(count+row_index, col_index, item)workbook.save('起风了评论表.xls')

爬取网易云音乐热评

import requests
import time
import json
import xlwt
headers = {'Host': 'music.163.com','User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36'
}
def get_comments(songId):url = 'http://music.163.com/api/v1/resource/comments/R_SO_4_{0}?limit=100&offset=0'.format(songId) #字符串格式化response = requests.get(url=url, headers=headers)result = json.loads(response.text)temp = []for item in result['hotComments']:data = {}# 评论者iddata['userId'] = item['user']['userId']# 评论者昵称data['nickname'] = item['user']['nickname']# 评论内容data['content'] = item['content'].replace("\n",",")# 评论发表时间timeArray = time.localtime(item['time'] / 1000)date = time.strftime("%Y-%m-%d %H:%M:%S", timeArray)data['date'] = date# 点赞数data['likedCount'] = item['likedCount']temp.append(data)return temp
if __name__ == '__main__':list = ["评论者id","评论者昵称","评论内容","评论发表时间","点赞数"]workbook = xlwt.Workbook(encoding="utf-8")worksheet = workbook.add_sheet("sheet1",cell_overwrite_ok=True)  # 创建工作表for index, item in enumerate(list):worksheet.write(0, index, item)for row_index,temp in enumerate(get_comments("566436427")):for col_index, item in enumerate(temp.values()):worksheet.write(row_index+1, col_index, item)workbook.save('起风了热评表.xls')

jxl的使用总结（java操作excel）

界面

网易云音乐（Cloudmusic）API

from origin ‘http://localhost:8080’ has been blocked by CORS policy:
遇到了跨域问题

解决vue-cli项目开发中跨域问题

proxyTable: {'/api': {target: 'https://music.163.com', // 目标接口的域名secure: false,                   // 如果是https,就要加上changeOrigin: true,             // 是否跨域pathRewrite: {'^api': ''               //重写接口}// 如果不写pathRewrite,则实际会访问地址为https://music.163.com/api/...// 如果写了pathRewrite，则实际会访问的地址为https://music.163.com/..}}

展示:

涉及知识：vue + element-ui + axios + vuex + router

<template>
<div style="width: 90%;margin: 0 auto"><el-row style="padding: 20px;"><el-col :span="7"><el-input v-model="songName" placeholder="请输入歌曲名称"></el-input></el-col><el-col :span="6"><el-button type="primary" icon="el-icon-search" @click="searchSongByName" >搜索</el-button></el-col></el-row><!--表格带边框、绑定的数据、设置标签居中、设置文字内容居中--><el-table stripe border:data="tableData":header-cell-style="{'text-align':'center'}":cell-style="{'text-align':'center'}"style="width: 80%;"><el-table-column prop="name" label="歌曲名称" width="180"></el-table-column><el-table-column prop="songer" label="歌手" width="180"></el-table-column><el-table-column prop="time" label="时间"></el-table-column><el-table-column label="操作"><template slot-scope="scope"><el-button type="text" @click="queryComment(scope.row.id,'hotComments')">查看热评</el-button><el-button type="text" @click="queryComment(scope.row.id,'comments')">查看最新评论</el-button></template></el-table-column></el-table>
</div>
</template><script>
export default {name: '',data () {return {songName: '起风了',tableData: []}},methods: {searchSongByName () {this.$axios.get('https://api.imjad.cn/cloudmusic/', {params: {type: 'search',search_type: 1,s: this.songName}}).then(res => {this.tableData = []for (let i = 0; i < res.data.result.songs.length; i++) {let song = {}song.id = res.data.result.songs[i].idsong.name = res.data.result.songs[i].namesong.songer = res.data.result.songs[i].ar[0].namesong.time = this.secTotime(res.data.result.songs[i].dt / 1000)this.tableData.push(song)}}).catch(err => {console.log(err)})},// 将秒转化为时分秒，由于一首歌时间几乎不会超过小时，所以去掉hoursecTotime (secs) {var timeif (secs > -1) {var min = Math.floor(secs / 60) % 60var sec = secs % 60}if (min < 10) {time = '0'}time += min + ':'if (sec < 10) {time += '0'}time += sec.toFixed(0) // 不要小数点后面的return time},queryComment (songId, commentType) {this.$router.push('/songComments') // 实现路由跳转this.$store.commit('setSongIdAndCommentType', {songId: songId, commentType: commentType})}},created () {this.searchSongByName()}
}
</script><style scoped></style>

<template>
<div style="width: 100%;margin: 0 auto"><el-row style="padding: 20px;"><el-col :span="6"><el-button type="success" plain icon="el-icon-back" @click="back">返回</el-button></el-col><el-col :span="6"><el-button type="primary" icon="el-icon-search" @click="queryCommentByIdAndType" style="float: right">搜索</el-button></el-col></el-row><!--表格带边框、绑定的数据、设置标签居中、设置文字内容居中--><el-table stripe border:data="tableData":header-cell-style="{'text-align':'center'}":cell-style="{'text-align':'center'}"style="width: 100%;"><el-table-columnprop="userId"label="评论者id"width="180"></el-table-column><el-table-columnprop="nickname"label="评论者昵称"width="180"></el-table-column><el-table-columnprop="content"label="评论内容"width="300"></el-table-column><el-table-columnprop="date"label="评论发表时间"></el-table-column><el-table-columnprop="likedCount"label="点赞数"></el-table-column></el-table><el-pagination@size-change="handleSizeChange"@current-change="handleCurrentPageChange":page-sizes="[10,20,30,40,50]"backgroundlayout="total, sizes, prev, pager, next, jumper":total="1000"><!--显示总共有多少条数据--></el-pagination>
</div>
</template><script>
export default {name: '',data () {return {tableData: [],currentPage: 1, // 初始页pageSize: 10 // 每页的数据条数}},methods: {queryCommentByIdAndType () {this.$axios.get('/api/v1/resource/comments/R_SO_4_' + this.$store.state.songId, {params: {limit: this.pageSize,offset: (this.currentPage - 1) * this.pageSize}}).then(res => {// 解决res.data.小数点后面跟变量var comments = res.data[this.$store.state.commentType]this.tableData = []for (let i = 0; i < comments.length; i++) {let comment = {}comment.userId = comments[i].user.userIdcomment.nickname = comments[i].user.nicknamecomment.content = comments[i].contentlet date = new Date(comments[i].time)comment.date = this.formatDate(date, 'yyyy-MM-dd hh:mm:ss')comment.likedCount = comments[i].likedCountthis.tableData.push(comment)}}).catch(err => {console.log(err)})},back () {this.$router.push('/searchSong') // 实现路由跳转},padLeftZero (str) {return ('00' + str).substr(str.length)},// 将时间戳转换为标准时间formatDate (date, fmt) {let o = {'M+': date.getMonth() + 1, // 月份'd+': date.getDate(), // 日'h+': date.getHours(), // 小时'm+': date.getMinutes(), // 分's+': date.getSeconds(), // 秒'q+': Math.floor((date.getMonth() + 3) / 3), // 季度'S': date.getMilliseconds() // 毫秒}if (/(y+)/.test(fmt)) { // 年份fmt = fmt.replace(RegExp.$1, (date.getFullYear() + '').substr(4 - RegExp.$1.length))}for (let k in o) {if (new RegExp('(' + k + ')').test(fmt)) {let str = o[k] + ''fmt = fmt.replace(RegExp.$1, (RegExp.$1.length === 1) ? str : this.padLeftZero(str))}}return fmt},// 设置每页显示的条数handleSizeChange (size) {this.pageSize = size},// 设置当前页为第currentPage页handleCurrentPageChange (currentPage) {this.currentPage = currentPage}},// 跳转该页面之后，就会执行该函数mounted () {this.queryCommentByIdAndType()},// 监听函数watch: {currentPage () {console.log('Hello,World')this.queryCommentByIdAndType()},pageSize () {this.queryCommentByIdAndType()}}
}
</script><style scoped></style>

【Python爬虫】爬取网易云评论相关推荐

python爬虫爬取网易云音乐歌曲_Python网易云音乐爬虫进阶篇
image.png 年前写过一篇爬网易云音乐评论的文章,爬不了多久又回被封,所以爬下来那么点根本做不了什么分析,后面就再改了下,加入了多线程,一次性爬一个歌手最热门50首歌曲的评论,算是进阶版了- 思 ...
python网易云_用python爬虫爬取网易云音乐
标签: 使用python爬虫爬取网易云音乐需要使用的模块只需要requests模块和os模块即可开始工作先去网易云音乐网页版找一下你想要听的歌曲点击进去.按键盘F12打开网页调试工具,点击Ne ...
python爬虫----爬取网易云音乐
使用python爬虫爬取网易云音乐目录使用python爬虫爬取网易云音乐需要使用的模块开始工作运行结果需要使用的模块只需要requests模块和os模块即可开始工作先去网易云音乐网页 ...
python+execjs爬取网易云评论
python+execjs爬取网易云评论分析网站 JS分析 execjs解密js 运行结果代码分析网站首先打开网易云首页,随便点一首歌曲进入到评论区. 接着按F12进入开发者工具,重新刷新页面 ...
Python爬虫—爬取网易云音乐【热歌榜】歌曲的精彩评论（写入txt文本文件或者MySQL数据库）
最近在学Python爬虫,看了Blibili爬取网易云音乐评论的视频,视频中是将一首歌的评论存入json文件,我在此代码的基础上扩展了三点: 1.爬取热歌榜200首歌曲的精彩评论: ...
python爬虫爬取网易云音乐下载_Python爬虫实践-网易云音乐！没有版权又如何！照样爬取！...
1.前言最近,网易的音乐很多听不到了,刚好也看到很多教程,跟进学习了一下,也集大全了吧,本来想优化一下的,但是发现问题还是有点复杂,最后另辟捷径,提供了简单的方法啊! 本文主要参考 python编写 ...
python爬虫-爬取网易云音乐歌曲评论
本文借鉴了@平胸小仙女的知乎回复 https://www.zhihu.com/question/36081767 以及@lyrichu的博客 https://www.cnblogs.com/lyric ...
python逆向爬取网易云评论进行情感分析！网易评论才是高手
好久不见,各位小伙伴们!嗐,春节真滴快啊!祝大家新年快乐! 书山有路勤为径,学海无涯苦作舟!又得开始愉快滴学习了! 小夜斗今天给大家伙分享一期干货,芜湖起飞! JS逆向网易云爬取评论并利用snownp ...
【原来python还可以这么玩】python逆向爬取网易云评论进行情感分析
遥遥微光,与我同行好久不见,各位小伙伴们!嗐,春节真滴快啊!祝大家新年快乐! 书山有路勤为径,学海无涯苦作舟!又得开始愉快滴学习了! 小夜斗今天给大家伙分享一期干货,芜湖起飞! JS逆向网易云爬取评 ...
python爬虫爬取网易云音乐歌曲_如何用爬虫获取网易云音乐歌单中的歌曲？
--------------------------------- 泻药,以我抓取了307835首网易云音乐的歌单歌曲的经验,讲一下这个问题. 喜欢用Github的可以直接看我的项目源码,代码简单.具 ...

【Python爬虫】爬取网易云评论

爬取网易云音乐最新评论

爬取网易云音乐热评

界面

【Python爬虫】爬取网易云评论相关推荐

最新文章

热门文章