socket接受数据的三种方式
原位置:http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/408859
Description:
An issue with socket.recv is how to know when you are done receiving data. A TCP stream guarantees the bytes will not arrive out of order or be sent more than once. But you do not know the size of the data that will be sent to you. 100 bytes could be sent as group of 10 bytes or maybe in one shot. Ultimately, this means you have to use a loop in some fashion until you know it is done.
The basic recv returns an empty string when the socket is disconnected.
From that you can build a simple loop that will work as long as the sender manages to disconnect the socket at the appropriate time. However, there could be situations where a local error will mask as a clean shutdown or maybe a close() is never called.
Three very basic methods are shown below that try to fix that problem. They use either a time-based, end marker, or size of payload method. Since you cannot be sure just what you are going to receive, you have to be careful that you get enough of a message to determine the size of payload or end marker.
I updated the recv_size method to allocate data in larger chunks if it gets a large stream of data, which can increase performance.
代码如下:
import socket,struct,sys,time
Port=2222 #assume a socket disconnect (data returned is empty string) means all data was #done being sent. def recv_basic(the_socket):total_data=[]while True:data = the_socket.recv(8192)if not data: breaktotal_data.append(data)return ''.join(total_data)def recv_timeout(the_socket,timeout=2):the_socket.setblocking(0)total_data=[];data='';begin=time.time()while 1:#if you got some data, then break after wait secif total_data and time.time()-begin>timeout:break#if you got no data at all, wait a little longerelif time.time()-begin>timeout*2:breaktry:data=the_socket.recv(8192)if data:total_data.append(data)begin=time.time()else:time.sleep(0.1)except:passreturn ''.join(total_data)End='something useable as an end marker' def recv_end(the_socket):total_data=[];data=''while True:data=the_socket.recv(8192)if End in data:total_data.append(data[:data.find(End)])breaktotal_data.append(data)if len(total_data)>1:#check if end_of_data was splitlast_pair=total_data[-2]+total_data[-1]if End in last_pair:total_data[-2]=last_pair[:last_pair.find(End)]total_data.pop()breakreturn ''.join(total_data)def recv_size(the_socket):#data length is packed into 4 bytestotal_len=0;total_data=[];size=sys.maxintsize_data=sock_data='';recv_size=8192while total_len<size:sock_data=the_socket.recv(recv_size)if not total_data:if len(sock_data)>4:size_data+=sock_datasize=struct.unpack('>i', size_data[:4])[0]recv_size=sizeif recv_size>524288:recv_size=524288total_data.append(size_data[4:])else:size_data+=sock_dataelse:total_data.append(sock_data)total_len=sum([len(i) for i in total_data ])return ''.join(total_data)############## def start_server(recv_type=''):sock=socket.socket(socket.AF_INET,socket.SOCK_STREAM)sock.bind(('',Port))sock.listen(5)print 'started on',Portwhile True:newsock,address=sock.accept()print 'connected'if recv_type=='size': result=recv_size(newsock)elif recv_type=='end': result=recv_end(newsock)elif recv_type=='timeout': result=recv_timeout(newsock)else: result=newsock.recv(8192) print 'got',resultif __name__=='__main__':#start_server()#start_server(recv_type='size')#start_server(recv_type='timeout')start_server(recv_type='end')def send_size(data):sock=socket.socket(socket.AF_INET,socket.SOCK_STREAM)sock.connect(('localhost',Port))sock.sendall(struct.pack('>i', len(data))+data)sock.close()def send_end(data):sock=socket.socket(socket.AF_INET,socket.SOCK_STREAM)sock.connect(('localhost',Port))sock.sendall(data+End)sock.close()
Discussion:
I employ a trivial server, to keep this as simple as possible. Just uncomment the type of receiving server you want to use to see the recv type chosen in action.
The recv_timeout function, which uses non-blocking sockets, will continue trying to get data as long as the client manages to even send a single byte. This is useful for moving data which you know very little about (like encrypted data), so cannot check for completion in a sane way.
The recv_end function tacks on an end marker understood by both the client and the server. One problem with this is that data cannot look like the marker.
The recv_size function looks for the size of the payload to be prepended to the data. I use a fixed length, 4 bytes. So then I know to always look for that. It is packed to allow a terse representation that can fit sys.maxint. This avoids the problem of data looking like the marker, however, even if it means a lot of data, you are limited w/the payload being the maximum number that can be packed. An interesting advantage of this method is that it can allocate data in larger chunks since it knows this size of the data ahead of time. For large streams of data, I saw it increase performace by 10 times.
To test this, in a another process, try using the functions that match with the server type.
send_size(data) #for recv_size
send_end(data) #for recv_end
sock.sendall(data) #for timeout or simple recv(8192)
#do not forget to close if you do a raw sendall
sock.close()
socket接受数据的三种方式相关推荐
- python socket接收图像 数据_python中socket接受数据的三种方法 | 学步园
原位置:http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/408859 Description: An issue with socket ...
- pythonsocket数据对接_python中socket接受数据的三种方法
原位置:http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/408859 Description: An issue with socket ...
- discard connection丢失数据_python kafka 生产者发送数据的三种方式
python kafka 生产者发送数据的三种方式 发送方式 同步发送 发送数据耗时最长 有发送数据的状态,不会丢失数据,数据可靠性高 以同步的方式发送消息时,一条一条的发送,对每条消息返回的结果判断 ...
- layui根据条件显示列_templet渲染layui表格数据的三种方式
layui前端框架是我一直在使用,也很好用. 今天记录一下,templet渲染layui表格数据的三种方式. 第一种:直接渲染(对于表格数据样式要求不高) 直接在动态表格字段声明,添加templet属 ...
- ios网络学习------4 UIWebView的加载本地数据的三种方式
ios网络学习------4 UIWebView的加载本地数据的三种方式 分类: IOS2014-06-27 12:56 959人阅读 评论(0) 收藏 举报 UIWebView是IOS内置的浏览器, ...
- python读取图像数据流_浅谈TensorFlow中读取图像数据的三种方式
本文面对三种常常遇到的情况,总结三种读取数据的方式,分别用于处理单张图片.大量图片,和TFRecorder读取方式.并且还补充了功能相近的tf函数. 1.处理单张图片 我们训练完模型之后,常常要用图片 ...
- android sqlite使用之模糊查询数据库数据的三种方式
android sqlite使用之模糊查询数据库数据的三种方式 android应用开发中常常需要记录一下数据,而在查询的时候如何实现模糊查询呢?很少有文章来做这样的介绍,所以这里简单的介绍下三种sql ...
- layui 表格内容写temple函数_templet渲染layui表格数据的三种方式
layui前端框架是我一直在使用,也很好用. 今天记录一下,templet渲染layui表格数据的三种方式. 第一种:直接渲染(对于表格数据样式要求不高) 直接在动态表格字段声明,添加templet属 ...
- mysql数据库删除数据的三种方式:
mysql数据库删除数据的三种方式: delete from table where 直接删除表中的某一行数据,并且同时将该行的删除操作作为事务记录在日志中保存以便进行进行回滚操作.所以delete相 ...
最新文章
- java queue 线程安全_java并发编程之线程安全方法
- 微信小程序与uniapp中 px与rpx 相互转换
- switch 没有default_「java面试」switch究竟怎么用,不要让它吃灰了!
- 2.设计模式中状态模式(对象的行为模式)(Python实现)
- 持续20年,一场威胁Linux存亡的诉讼终结束
- JVM架构和GC垃圾回收机制--面试
- AMESim软件包 百度云下载
- 对XSCJ数据库进行数据查询 -- 学生成绩管理系统
- ibms智能化系统集成服务器,智能化集成管理系统(IBMS)
- 网络分析工具——WireShark的使用(超详细)
- 李梦恬爸爸分享之第十周课程资源
- 镜像电流源特点_镜像电流源电路.ppt
- 3、python 视频转图片及图片转视频、裁剪、创建图片压缩成视频
- 亲测UEFI启动模式的电脑安装Win10和Ubuntu双系统
- Unit Test and Integration Test
- Red Hat 认证工程师(RHCE)
- svhost占用内存过高
- input框不能输入的几种方式
- 使用FTP协议的服务,包括匿名用户验证访问,用户验证访问,虚拟用户验证访问等
- Mybatis动态传入order by