urllib 库学习笔记

未分类

urllib 库

1、urllib 库模块分类

• request
• parse
• erro
• robotparser

2、urlopen() 函数

通过urlib.request.urlopen()函数方法来构建http请求

urllib.request.urlopen(url, data=None, [timeout,]  cafile=None, capath=None, cadefault=False, context=None)

（1）参数1: url

from urllib import request
url = 'http://httpbin.org/get'
response = request.urlopen(url)
print(response.read().decode('utf-8'))

（2）参数2：data

from urllib import parse,request
url = 'http://httpbin.org/post'
form_data = {'key':'value'}
data = bytes(parse.urlencode(form_data),encoding='utf-8')
response = request.urlopen(url,data)
response_text = response.read().decode('utf-8')
print(response_text)

（3）参数3：timeout

from urllib import request
url = 'http://httpbin.org/get'
response = request.urlopen(url, timeout = 1)
print(response.read().decode('utf-8'))

from urllib import request, error
import socket
url = 'http://httpbin.org/get'
try:
    response = request.urlopen(url, timeout = 0.1)
except error.URLError as e:
    if isinstance(e.reason, socket.timeout):
        print("Time Out ...")

3、response 响应类

（1）响应类型

from urllib import request
url = 'http://httpbin.org/get'
response = request.urlopen(url)
print(type(response))

<class 'http.client.HTTPResponse'>

（2）状态码、响应头

from urllib import request
url = 'http://httpbin.org/get'
response = request.urlopen(url)
print(response.status)
print(response.getheaders())
#print(response.getheaders['Server'])

200
[('Connection', 'close'), ('Server', 'gunicorn/19.9.0'), ('Date', 'Fri, 11 Jan 2019 15:30:00 GMT'), ('Content-Type', 'application/json'), ('Content-Length', '233'), ('Access-Control-Allow-Origin', '*'), ('Access-Control-Allow-Credentials', 'true'), ('Via', '1.1 vegur')]

（3）响应实体内容

from urllib import request
url = 'http://httpbin.org/ip'
response = request.urlopen(url)
print(response.read().decode('utf-8'))

{
  "origin": "60.166.83.51"
}

4、 urllib.request.Request() 类