requests-html库

python库

from requests_html import HTMLSession

基本使用

session = HTMLSession()
r = session.get('https://python.org/')

功能函数

requests_html.user_agent

requests_html.user_agent(style=None) 

Returns an apparently legit user-agent, if not requested one of a specific style. Defaults to a Chrome-style User-Agent.

main class

HTML, Element, HTMLSession

HTML类

表示一个HTML文本，等待被parse。

class requests_html.HTML(*, session: Union[_ForwardRef('HTTPSession'), _ForwardRef('AsyncHTMLSession')] = None, url: str = 'https://example.org/', html: Union[str, bytes], default_encoding: str = 'utf-8') 

• url: The URL from which the HTML originated, used for absolute_links
• html : HTML from which to base the parsing upon (optional).
• default_encoding – Which encoding to default to.

1. 属性

• absolute_links： All found links on page, in absolute form

• base_url: The base URL for the page.

• encoding: The encoding string to be used, extracted from the HTML and HTMLResponse headers.
• full_text: The full text content (including links) of the Element or HTML
• html: Unicode representation of the HTML content
• links: All found links on page, in as–is form.
• lxml: lxml representation of the Element or HTML.
• pq: PyQuery representation of the Element or HTML.
• raw_html: Bytes representation of the HTML content.
• text: The text content of the Element or HTML.

2. 方法

find()

给定一个CSS Selector，

find(selector: str = '*', *, containing: Union[str, typing.List[str]] = None, clean: bool = False, first: bool = False, _encoding: str = None) → Union[typing.List[_ForwardRef('Element')], _ForwardRef('Element')]

• selector: CSS Selector to use.
• clean: Whether or not to sanitize the found HTML of and tags.
• containing : If specified, only return elements that contain the provided text.
• first: Whether or not to return just the first result.
• _encoding: The encoding format.

reader()

search()

search_all()

xpath()

类 Element

表示一个HTML的element。

class requests_html.Element(*, element, url: str, default_encoding: str = None)

• element : The element from which to base the parsing upon.
• url :The URL from which the HTML originated, used for absolute_links.
• default_encoding: Which encoding to default to.

1. 属性

2. 方法

类 HTMLSession

class requests_html.HTMLSession(mock_browser=True)

方法

request()

Makes an HTTP Request, with mocked User–Agent headers. Returns a class:HTTPResponse .

request(*args, **kwargs)

send()

• send(request, **kwargs)
• 返回值： requests.Response

send(request, **kwargs)

get()

• Sends a GET request. Returns Response object.
• 返回类型： requests.Response

get(url, **kwargs)

• url:
• **kwargs: Optional arguments that request takes.

head()

• Sends a HEAD request. Returns Response object.
• 返回类型： requests.Response

head(url, **kwargs)

• url:
• **kwargs: Optional arguments that request takes.

post()

• Sends a POST request. Returns Response object.
• 返回类型： requests.Response

post(url, data=None, json=None, **kwargs)

• url:
• **kwargs: Optional arguments that request takes.

put()

• Sends a PUT request. Returns Response object.
• 返回类型： requests.Response

put(url, data=None, **kwargs)

• url:
• **kwargs: Optional arguments that request takes.

options()

• 发出一个OPTIONS request， 返回一个Response对象。
• 返回类型： requests.Response

options(url, **kwargs)

• url:
• **kwargs: Optional arguments that request takes.

patch()

• 发出一个PATCH request， 返回一个Response对象。
• 返回类型： requests.Response

patch(url, data=None, **kwargs)

• url:
• **kwargs: Optional arguments that request takes.

delete()

• 发出一个DELETE request， 返回一个Response对象。
• 返回类型： requests.Response

delete(url, **kwargs)

• url:
• **kwargs: Optional arguments that request takes.

close()

If a browser was created close it first.