全球动态代理 · 静态住宅代理 · 极高质量 · 包月不限量 · 学员低价 · 联系如意进行免费测试 · 欢迎公司及学员采购 · 另有任何海外数据采集需求,随时联系 · 商务合作微信:Charleval
Automation / 示例中心
Examples Center

RuyiPage 示例中心

这里不是简单的文件清单,而是将 RuyiPage 仓库中的 examples 渲染成案例卡。每个例子都带有文件名、分类、难度、不同用法、涉及 API、适用场景与相关页面,并提供 GitHub 源码入口,帮助你从“例子”回到“文档”,也从“文档”回到“例子”。

网络拦截入门:mock、fail、改请求头、队列模式

examples/11_network_intercept.py
GitHub 仓库 已同步源码 查看 GitHub 源码
network intermediate
一句话定位

通过本地 TestServer 让请求拦截、mock 和 fail 行为可重复验证。

涉及 API
page.intercept.startpage.intercept.waitreq.mockreq.failreq.continue_requestpage.run_js
展示的不同用法
  • 拦截请求并直接 mock 响应
  • 主动 fail 请求
  • 修改请求头后继续请求
  • 无 handler 队列模式手动处理请求
适用场景

适合 network 与 intercept 页面。

相关文档页面
推荐阅读
advanced-network-18collector-28
完整代码

来源:app/content/ruyipage_examples/11_network_intercept.py · 仓库文件:11_network_intercept.py

python 注释版
# =============================
# RuyiPage 示例注释版
# 标题: 网络拦截入门:mock、fail、改请求头、队列模式
# 说明: 通过本地 TestServer 让请求拦截、mock 和 fail 行为可重复验证。
# 你会学到:
#   1. 拦截请求并直接 mock 响应
#   2. 主动 fail 请求
#   3. 修改请求头后继续请求
#   4. 无 handler 队列模式手动处理请求
# 相关 API: page.intercept.start, page.intercept.wait, req.mock, req.fail, req.continue_request, page.run_js
# =============================

# -*- coding: utf-8 -*-
"""
示例11: 网络拦截(本地可重复版)

测试覆盖:
1) beforeRequestSent + mock 响应
2) beforeRequestSent + fail 请求
3) beforeRequestSent + 修改请求头并继续
4) 无 handler 队列模式 wait()

说明:
- 本示例全部使用本地 TestServer,避免外网波动导致结果不稳定。
- 所有拦截都在当前页面 context 内触发,便于复现与调试。
"""

import io
import os
import sys


if sys.platform == "win32":
    sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding="utf-8")
    sys.stderr = io.TextIOWrapper(sys.stderr.buffer, encoding="utf-8")


sys.path.insert(0, os.path.join(os.path.dirname(__file__), ".."))
sys.path.insert(0, os.path.dirname(__file__))

from ruyipage import FirefoxOptions, FirefoxPage
from test_server import TestServer


def test_network_intercept():
    """网络拦截演示主流程(适合初学者逐步阅读)。"""
    print("=" * 60)
    print("测试11: 网络拦截")
    print("=" * 60)

    server = TestServer(port=8888)
    server.start()

    opts = FirefoxOptions()
    opts.headless(False)
    page = FirefoxPage(opts)

    try:
        # 准备一个干净页面,确保后续 fetch 都在当前 context 内触发。
        page.get("about:blank")
        page.wait(0.5)

        # ==========================================================
        # 1) mock 响应
        # ==========================================================
        print("\n1. 拦截并 Mock 本地 API:")

        def mock_handler(req):
            if "/api/data" in req.url:
                print(f"   拦截到请求: {req.method} {req.url}")
                # 注意:mock 需要带上 CORS 头,不然浏览器会把跨域响应拦掉,
                # fetch 侧会看到 NetworkError。
                req.mock(
                    '{"status":"ok","data":{"message":"mocked-by-interceptor"}}',
                    status_code=200,
                    headers=[
                        {
                            "name": "content-type",
                            "value": {"type": "string", "value": "application/json"},
                        },
                        {
                            "name": "access-control-allow-origin",
                            "value": {"type": "string", "value": "*"},
                        },
                    ],
                )
                print("   ✓ 已返回 Mock 响应")
            else:
                # 不处理的请求必须显式继续,否则会一直挂起。
                req.continue_request()

        page.intercept.start(mock_handler, phases=["beforeRequestSent"])
        mocked = page.run_js(
            """
            return fetch(arguments[0]).then(r => r.json()).then(d => d.data.message).catch(e => String(e));
            """,
            server.get_url("/api/data"),
            as_expr=False,
        )
        print(f"   mock结果: {mocked}")
        page.intercept.stop()

        # ==========================================================
        # 2) fail 请求
        # ==========================================================
        print("\n2. 阻止本地 API 请求:")

        def fail_handler(req):
            if "/api/data" in req.url:
                print(f"   阻止请求: {req.url}")
                req.fail()
            else:
                req.continue_request()

        page.intercept.start(fail_handler, phases=["beforeRequestSent"])
        blocked = page.run_js(
            """
            return fetch(arguments[0]).then(() => 'unexpected-success').catch(e => 'blocked:' + e.name);
            """,
            server.get_url("/api/data"),
            as_expr=False,
        )
        print(f"   fail结果: {blocked}")
        page.intercept.stop()

        # ==========================================================
        # 3) 修改请求头并继续
        # ==========================================================
        print("\n3. 修改请求头并继续:")

        def header_handler(req):
            if "/api/headers" in req.url:
                # continue_request(headers=...) 会覆盖本次请求头。
                req.continue_request(
                    headers=[
                        {
                            "name": "X-Ruyi-Demo",
                            "value": {"type": "string", "value": "yes"},
                        },
                        {
                            "name": "User-Agent",
                            "value": {"type": "string", "value": "RuyiPage/Example11"},
                        },
                    ]
                )
                print("   ✓ 已注入自定义请求头")
            else:
                req.continue_request()

        page.intercept.start(header_handler, phases=["beforeRequestSent"])
        headers_json = page.run_js(
            """
            return fetch(arguments[0]).then(r => r.json()).catch(e => ({error:String(e)}));
            """,
            server.get_url("/api/headers"),
            as_expr=False,
        )
        x_header = ""
        if isinstance(headers_json, dict):
            x_header = headers_json.get("X-Ruyi-Demo") or headers_json.get(
                "x-ruyi-demo"
            )
        print(f"   服务器看到的 X-Ruyi-Demo: {x_header}")
        page.intercept.stop()

        # ==========================================================
        # 4) 无 handler 队列模式
        # ==========================================================
        print("\n4. 队列模式 wait() 捕获拦截请求:")
        # handler=None 表示不在回调里处理,而是进入队列,供 wait() 手动取出。
        page.intercept.start(handler=None, phases=["beforeRequestSent"])
        page.run_js(
            """
            fetch(arguments[0]).catch(() => null);
            return 'sent';
            """,
            server.get_url("/api/data"),
            as_expr=False,
        )
        req = page.intercept.wait(timeout=5)
        if req:
            print(f"   wait捕获: {req.method} {req.url}")
            # 队列模式下记得手动继续/中止请求,否则请求会悬挂。
            req.continue_request()
        else:
            print("   ⚠ 未在超时内捕获请求")
        page.intercept.stop()

        print("\n" + "=" * 60)
        print("✓ 网络拦截测试通过!")
        print("=" * 60)

    except Exception as e:
        print(f"\n✗ 测试失败: {e}")
        import traceback

        traceback.print_exc()
    finally:
        try:
            page.intercept.stop()
        except Exception:
            pass
        page.wait(1)
        page.quit()
        server.stop()


if __name__ == "__main__":
    test_network_intercept()

高级网络功能:请求/响应拦截、mock、阻止、fetchError

examples/18_advanced_network.py
GitHub 仓库 已同步源码 查看 GitHub 源码
network advanced
一句话定位

展示 intercept 与 events 组合后的完整网络控制能力。

涉及 API
page.intercept.start_requestspage.intercept.start_responsesreq.continue_requestreq.mockreq.failreq.continue_responsepage.events.startpage.events.wait
展示的不同用法
  • 在 beforeRequestSent 阶段继续请求
  • 修改请求头
  • mock JSON 响应
  • 阻止请求
  • 在 responseStarted 阶段修改响应状态
  • 监听 fetchError
适用场景

适合 network、intercept 和 anti-bot 章节。

相关文档页面
推荐阅读
network-intercept-11collector-28network-events-31
完整代码

来源:app/content/ruyipage_examples/18_advanced_network.py · 仓库文件:18_advanced_network.py

python 注释版
# =============================
# RuyiPage 示例注释版
# 标题: 高级网络功能:请求/响应拦截、mock、阻止、fetchError
# 说明: 展示 intercept 与 events 组合后的完整网络控制能力。
# 你会学到:
#   1. 在 beforeRequestSent 阶段继续请求
#   2. 修改请求头
#   3. mock JSON 响应
#   4. 阻止请求
#   5. 在 responseStarted 阶段修改响应状态
# 相关 API: page.intercept.start_requests, page.intercept.start_responses, req.continue_request, req.mock, req.fail, req.continue_response, page.events.start, page.events.wait
# =============================

# -*- coding: utf-8 -*-
"""示例18: 高级网络功能(完整场景)

覆盖:
1) beforeRequestSent 拦截 + 继续请求
2) 修改请求头
3) mock 响应
4) 阻止请求
5) responseStarted 阶段修改响应状态
6) 队列模式 wait() 手动处理
7) fetchError 事件监听
"""

import io
import os
import sys


if sys.platform == "win32":
    sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding="utf-8")
    sys.stderr = io.TextIOWrapper(sys.stderr.buffer, encoding="utf-8")


sys.path.insert(0, os.path.join(os.path.dirname(__file__), ".."))
sys.path.insert(0, os.path.dirname(__file__))

from ruyipage import FirefoxOptions, FirefoxPage
from ruyipage._functions.tools import find_free_port
from test_server import TestServer


def test_advanced_network():
    print("=" * 60)
    print("测试18: 高级网络功能")
    print("=" * 60)

    server = TestServer(port=find_free_port(8889, 8989))
    server.start()

    opts = FirefoxOptions()
    opts.headless(False)
    page = FirefoxPage(opts)

    fetch_error_list = []

    try:
        page.get("about:blank")
        page.wait(0.6)

        # 1) 请求拦截
        print("\n1. beforeRequestSent 拦截 + 继续请求:")
        intercepted = []

        def req_handler(req):
            if "/api/data" in req.url:
                intercepted.append(req.url)
                print(f"   拦截到请求: {req.method} {req.url}")
            req.continue_request()

        page.intercept.start_requests(req_handler)
        data = page.run_js(
            """
            return fetch(arguments[0]).then(r => r.json()).then(d => d.status).catch(e => String(e));
            """,
            server.get_url("/api/data"),
            as_expr=False,
        )
        page.wait(0.5)
        print(f"   请求结果: {data}")
        print(f"   拦截计数: {len(intercepted)}")
        page.intercept.stop()

        # 2) 修改请求头
        print("\n2. 修改请求头:")

        def header_handler(req):
            if "/api/headers" in req.url:
                req.continue_request(
                    headers=[
                        {
                            "name": "X-Ruyi-Demo",
                            "value": {"type": "string", "value": "yes"},
                        },
                        {
                            "name": "User-Agent",
                            "value": {"type": "string", "value": "RuyiPage/Example18"},
                        },
                    ]
                )
                print("   ✓ 已注入请求头")
            else:
                req.continue_request()

        page.intercept.start_requests(header_handler)
        headers = page.run_js(
            """
            return fetch(arguments[0]).then(r => r.json()).catch(e => ({error:String(e)}));
            """,
            server.get_url("/api/headers"),
            as_expr=False,
        )
        print(
            f"   X-Ruyi-Demo: {headers.get('X-Ruyi-Demo') or headers.get('x-ruyi-demo')}"
        )
        page.intercept.stop()

        # 3) mock 响应
        print("\n3. mock 响应:")

        def mock_handler(req):
            if "/api/data" in req.url:
                req.mock(
                    '{"status":"mocked","data":{"message":"这是Mock数据"}}',
                    status_code=200,
                    headers=[
                        {
                            "name": "content-type",
                            "value": {"type": "string", "value": "application/json"},
                        },
                        {
                            "name": "access-control-allow-origin",
                            "value": {"type": "string", "value": "*"},
                        },
                    ],
                )
                print("   ✓ 已返回Mock响应")
            else:
                req.continue_request()

        page.intercept.start_requests(mock_handler)
        mock_msg = page.run_js(
            """
            return fetch(arguments[0]).then(r => r.json()).then(d => d.status + ':' + d.data.message).catch(e => String(e));
            """,
            server.get_url("/api/data"),
            as_expr=False,
        )
        print(f"   Mock结果: {mock_msg}")
        page.intercept.stop()

        # 4) 阻止请求
        print("\n4. 阻止请求:")

        def block_handler(req):
            if "/api/error" in req.url:
                req.fail()
                print("   ✓ 请求已阻止")
            else:
                req.continue_request()

        page.intercept.start_requests(block_handler)
        blocked = page.run_js(
            """
            return fetch(arguments[0]).then(() => 'unexpected-success').catch(e => 'blocked:' + e.name);
            """,
            server.get_url("/api/error"),
            as_expr=False,
        )
        print(f"   阻止结果: {blocked}")
        page.intercept.stop()

        # 5) 响应阶段修改状态码
        print("\n5. responseStarted 阶段修改响应状态码:")

        def response_handler(req):
            if "/api/data" in req.url:
                req.continue_response(status_code=299, reason_phrase="RuyiModified")
                print("   ✓ 响应状态码已改为 299")
            else:
                req.continue_response()

        page.intercept.start_responses(response_handler)
        modified_status = page.run_js(
            """
            return fetch(arguments[0]).then(r => r.status + ':' + r.statusText).catch(e => String(e));
            """,
            server.get_url("/api/data"),
            as_expr=False,
        )
        print(f"   响应状态: {modified_status}")
        page.intercept.stop()

        # 6) 队列模式
        print("\n6. 队列模式 wait() 手动处理:")
        page.intercept.start(handler=None, phases=["beforeRequestSent"])
        page.run_js(
            """
            fetch(arguments[0]).catch(() => null);
            return 'sent';
            """,
            server.get_url("/api/data"),
            as_expr=False,
        )
        queued = page.intercept.wait(timeout=5)
        if queued:
            print(f"   wait捕获: {queued.method} {queued.url}")
            queued.continue_request()
        else:
            print("   ⚠ 未捕获到请求")
        page.intercept.stop()

        # 7) fetchError 事件监听
        print("\n7. fetchError 事件监听:")
        page.events.start(["network.fetchError"], contexts=[page.tab_id])
        page.events.clear()

        page.intercept.start_requests(block_handler)
        page.run_js(
            """
            fetch(arguments[0]).catch(() => null);
            return 'sent';
            """,
            server.get_url("/api/error"),
            as_expr=False,
        )
        page.wait(1)
        page.intercept.stop()
        while True:
            evt = page.events.wait("network.fetchError", timeout=0.2)
            if not evt:
                break
            fetch_error_list.append(evt)
        print(f"   fetchError 事件数量: {len(fetch_error_list)}")

        print("\n" + "=" * 60)
        print("✓ 所有高级网络功能测试通过!")
        print("=" * 60)

    except Exception as e:
        print(f"\n✗ 测试失败: {e}")
        import traceback

        traceback.print_exc()
    finally:
        try:
            page.intercept.stop()
        except Exception:
            pass
        try:
            page.events.stop()
        except Exception:
            pass
        page.wait(1)
        page.quit()
        server.stop()


if __name__ == "__main__":
    test_advanced_network()

导航事件:started、load、fragment、history、committed、failed

examples/24_navigation_events.py
GitHub 仓库 已同步源码 查看 GitHub 源码
network strict
一句话定位

导航事件链路的标准参考示例。

涉及 API
page.navigation.startpage.navigation.waitpage.navigation.wait_for_loadpage.navigation.wait_for_fragmentpage.navigation.stop
展示的不同用法
  • 订阅导航生命周期事件
  • 观察 navigationStarted、DOMContentLoaded 和 load
  • 观察 hash 片段跳转
  • 观察 History API 更新
  • 构造失败导航场景
适用场景

适合 navigation-events 页面。

推荐阅读
network-events-31browsing-events-30
完整代码

来源:app/content/ruyipage_examples/24_navigation_events.py · 仓库文件:24_navigation_events.py

python 注释版
#!/usr/bin/env python
# =============================
# RuyiPage 示例注释版
# 标题: 导航事件:started、load、fragment、history、committed、failed
# 说明: 导航事件链路的标准参考示例。
# 你会学到:
#   1. 订阅导航生命周期事件
#   2. 观察 navigationStarted、DOMContentLoaded 和 load
#   3. 观察 hash 片段跳转
#   4. 观察 History API 更新
#   5. 构造失败导航场景
# 相关 API: page.navigation.start, page.navigation.wait, page.navigation.wait_for_load, page.navigation.wait_for_fragment, page.navigation.stop
# =============================

# -*- coding: utf-8 -*-
"""示例24: Navigation Events 导航事件测试(严格结果版)"""

import io
import sys
from typing import Dict, List

if sys.platform == "win32":
    sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding="utf-8")
    sys.stderr = io.TextIOWrapper(sys.stderr.buffer, encoding="utf-8")

from ruyipage import FirefoxPage
from ruyipage._functions.tools import find_free_port

from test_server import TestServer


def add_result(
    results: List[Dict[str, str]], item: str, status: str, note: str
) -> None:
    """记录结构化结果。"""
    results.append({"item": item, "status": status, "note": note})


def print_results(results: List[Dict[str, str]]) -> None:
    """打印固定格式结果表。"""
    print("\n| 项目 | 状态 | 说明 |")
    print("| --- | --- | --- |")
    for row in results:
        print(f"| {row['item']} | {row['status']} | {row['note']} |")


def main() -> None:
    print("=" * 70)
    print("测试 24: Navigation Events 导航事件测试")
    print("=" * 70)

    page = FirefoxPage()
    server = None
    results: List[Dict[str, str]] = []

    try:
        server = TestServer(port=find_free_port(9030, 9130)).start()
        base_url = server.get_url("")[:-1]

        tracked_events = [
            "browsingContext.navigationStarted",
            "browsingContext.fragmentNavigated",
            "browsingContext.historyUpdated",
            "browsingContext.domContentLoaded",
            "browsingContext.load",
            "browsingContext.navigationCommitted",
            "browsingContext.navigationFailed",
        ]
        if not page.navigation.start(tracked_events):
            add_result(results, "导航事件订阅", "失败", "当前环境订阅导航事件失败")
            print_results(results)
            return

        # 1. 基础导航事件
        page.navigation.clear()
        page.get(base_url + "/nav/basic")
        started = page.navigation.wait("browsingContext.navigationStarted", timeout=3)
        dom_loaded = page.navigation.wait("browsingContext.domContentLoaded", timeout=3)
        loaded = page.navigation.wait_for_load(timeout=3)

        if started and (started.url or "").endswith("/nav/basic"):
            add_result(results, "navigationStarted", "成功", "收到基础导航开始事件")
        else:
            add_result(
                results, "navigationStarted", "失败", "未观察到基础页面导航开始事件"
            )

        if dom_loaded and dom_loaded.context == page.tab_id:
            add_result(
                results, "domContentLoaded", "成功", "收到 DOMContentLoaded 事件"
            )
        else:
            add_result(
                results, "domContentLoaded", "失败", "未观察到 DOMContentLoaded 事件"
            )

        if loaded and loaded.context == page.tab_id:
            add_result(results, "load", "成功", "收到 load 事件")
        else:
            add_result(results, "load", "失败", "未观察到 load 事件")

        # 2. fragmentNavigated
        page.get(base_url + "/nav/fragment")
        page.navigation.clear()
        page.run_js(
            """
            location.hash = '#a';
            setTimeout(function () { location.hash = '#b'; }, 100);
            """,
            as_expr=False,
        )
        first_fragment = page.navigation.wait_for_fragment("a", timeout=3)
        second_fragment = page.navigation.wait_for_fragment("b", timeout=3)

        if first_fragment and second_fragment:
            add_result(
                results,
                "fragmentNavigated",
                "成功",
                f"两次片段导航已触发: {first_fragment.url} -> {second_fragment.url}",
            )
        else:
            add_result(
                results,
                "fragmentNavigated",
                "不支持",
                "片段 URL 已变化,但当前 Firefox 未观察到标准 fragmentNavigated 事件",
            )

        # 3. historyUpdated
        page.get(base_url + "/nav/history")
        page.navigation.clear()
        page.run_js(
            """
            history.pushState({p:1}, 'P1', '?p=1');
            history.pushState({p:2}, 'P2', '?p=2');
            history.back();
            """,
            as_expr=False,
        )
        first_history = page.navigation.wait(
            "browsingContext.historyUpdated", timeout=3
        )
        second_history = page.navigation.wait(
            "browsingContext.historyUpdated", timeout=3
        )

        if first_history or second_history:
            add_result(
                results, "historyUpdated", "成功", "pushState/back 触发 historyUpdated"
            )
        else:
            add_result(
                results, "historyUpdated", "失败", "未观察到 historyUpdated 事件"
            )

        # 4. navigationCommitted
        page.navigation.clear()
        page.get(base_url + "/nav/basic?committed=1")
        committed = page.navigation.wait(
            "browsingContext.navigationCommitted", timeout=3
        )
        if committed and committed.context == page.tab_id:
            add_result(
                results,
                "navigationCommitted",
                "成功",
                "普通导航触发 navigationCommitted",
            )
        else:
            add_result(
                results,
                "navigationCommitted",
                "跳过",
                "当前环境未稳定观察到 navigationCommitted",
            )

        # 5. navigationFailed
        page.navigation.clear()
        try:
            page.get("http://127.0.0.1:9/", timeout=2)
        except Exception:
            pass
        failed = page.navigation.wait("browsingContext.navigationFailed", timeout=3)
        if failed:
            add_result(
                results, "navigationFailed", "成功", "不可达地址触发 navigationFailed"
            )
        else:
            add_result(
                results,
                "navigationFailed",
                "跳过",
                "当前环境未稳定观察到 navigationFailed",
            )

        # 6. navigationAborted
        if page.navigation.start(["browsingContext.navigationAborted"]):
            add_result(
                results,
                "navigationAborted",
                "跳过",
                "订阅成功,但本示例未构造稳定 aborted 场景",
            )
        else:
            add_result(
                results, "navigationAborted", "不支持", "当前 Firefox 不接受该事件名"
            )

        # 恢复默认跟踪,便于 finally 统一 stop
        page.navigation.start(tracked_events)

        print_results(results)

    except Exception as e:
        add_result(results, "示例执行", "失败", str(e)[:120])
        print_results(results)
        raise
    finally:
        try:
            page.navigation.stop()
        except Exception:
            pass
        try:
            if server is not None:
                server.stop()
        except Exception:
            pass
        try:
            page.quit()
        except Exception:
            pass


if __name__ == "__main__":
    main()

网络数据采集器:响应体抓取、读取与释放

examples/28_network_data_collector.py
GitHub 仓库 已同步源码 查看 GitHub 源码
network strict
一句话定位

用于说明 add_data_collector 在数据抓取和协议观测中的价值。

涉及 API
page.network.set_extra_headerspage.network.set_cache_behaviorpage.network.add_data_collectorcollector.getcollector.disowncollector.removepage.listen.start
展示的不同用法
  • 注入额外请求头
  • 设置缓存 bypass
  • 添加并使用 data collector 采集 response 数据
  • 按 request_id 读取采集结果
  • 释放采集句柄并移除 collector
适用场景

适合 data-collector 与 network 页面。

相关文档页面
推荐阅读
advanced-network-18network-events-31
完整代码

来源:app/content/ruyipage_examples/28_network_data_collector.py · 仓库文件:28_network_data_collector.py

python 注释版
#!/usr/bin/env python
# =============================
# RuyiPage 示例注释版
# 标题: 网络数据采集器:响应体抓取、读取与释放
# 说明: 用于说明 add_data_collector 在数据抓取和协议观测中的价值。
# 你会学到:
#   1. 注入额外请求头
#   2. 设置缓存 bypass
#   3. 添加并使用 data collector 采集 response 数据
#   4. 按 request_id 读取采集结果
#   5. 释放采集句柄并移除 collector
# 相关 API: page.network.set_extra_headers, page.network.set_cache_behavior, page.network.add_data_collector, collector.get, collector.disown, collector.remove, page.listen.start
# =============================

# -*- coding: utf-8 -*-
"""示例28: Network Data Collector(严格结果版)"""

import io
import sys
from typing import Dict, List

if sys.platform == "win32":
    sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding="utf-8")
    sys.stderr = io.TextIOWrapper(sys.stderr.buffer, encoding="utf-8")

from ruyipage import DataCollector, DataPacket, FirefoxPage, InterceptedRequest
from ruyipage._functions.tools import find_free_port

from test_server import TestServer


def add_result(
    results: List[Dict[str, str]], item: str, status: str, note: str
) -> None:
    results.append({"item": item, "status": status, "note": note})


def print_results(results: List[Dict[str, str]]) -> None:
    print("\n| 项目 | 状态 | 说明 |")
    print("| --- | --- | --- |")
    for row in results:
        print(f"| {row['item']} | {row['status']} | {row['note']} |")


def main() -> None:
    print("=" * 70)
    print("测试 28: Network Data Collector")
    print("=" * 70)

    page = FirefoxPage()
    results: List[Dict[str, str]] = []
    collector: DataCollector | None = None
    server = None

    try:
        server = TestServer(port=find_free_port(9230, 9330)).start()
        base_url = server.get_url("")[:-1]

        page.get("about:blank")
        add_result(
            results, "页面加载", "成功", f"about:blank 已加载,服务地址: {base_url}"
        )

        page.network.set_extra_headers({"X-Test-Collector": "yes"})
        add_result(
            results, "network.setExtraHeaders", "成功", "已设置 X-Test-Collector"
        )

        try:
            page.network.set_cache_behavior("bypass")
            add_result(
                results, "network.setCacheBehavior", "成功", "缓存行为已设为 bypass"
            )
        except Exception as e:
            add_result(results, "network.setCacheBehavior", "不支持", str(e)[:120])

        collector = page.network.add_data_collector(
            ["responseCompleted"],
            data_types=["response"],
        )
        if collector.id:
            add_result(
                results, "network.addDataCollector", "成功", f"collector={collector.id}"
            )
        else:
            add_result(
                results, "network.addDataCollector", "失败", "未返回 collector ID"
            )

        page.intercept.start_requests()
        page.listen.start("/api/collector")
        page.run_js(
            f"fetch('{base_url}/api/collector').catch(()=>null); return true;",
            as_expr=False,
        )

        req: InterceptedRequest | None = page.intercept.wait(timeout=8)

        if req:
            has_header = "X-Test-Collector" in req.headers
            add_result(
                results,
                "request header injected",
                "成功" if has_header else "失败",
                f"request_id={req.request_id}, X-Test-Collector={has_header}",
            )
            req.continue_request()
        else:
            add_result(
                results,
                "request header injected",
                "失败",
                "未捕获到 beforeRequestSent 请求",
            )

        packet: DataPacket | None = page.listen.wait(timeout=8)

        if packet:
            add_result(
                results,
                "network.responseCompleted observed",
                "成功",
                f"status={packet.status} url={packet.url}",
            )
        else:
            add_result(
                results,
                "network.responseCompleted observed",
                "失败",
                "未在超时内观察到 responseCompleted",
            )

        if collector and req:
            got_response_data = False
            try:
                data = collector.get(req.request_id, data_type="response")
                if data.has_data:
                    got_response_data = True
                    add_result(
                        results,
                        "network.getData response",
                        "成功",
                        f"bytes={data.bytes} base64={'yes' if data.base64 else 'no'}",
                    )
                else:
                    add_result(
                        results, "network.getData response", "失败", f"raw={data.raw}"
                    )
            except Exception as e:
                add_result(results, "network.getData response", "失败", str(e)[:120])

            if got_response_data:
                try:
                    collector.disown(req.request_id, data_type="response")
                    add_result(
                        results, "network.disownData", "成功", "已释放 response 数据"
                    )
                except Exception as e:
                    add_result(results, "network.disownData", "失败", str(e)[:120])

                try:
                    data_after = collector.get(req.request_id, data_type="response")
                    if not data_after.has_data:
                        add_result(
                            results,
                            "network.getData after disown",
                            "成功",
                            "释放后已无可用数据",
                        )
                    else:
                        add_result(
                            results,
                            "network.getData after disown",
                            "跳过",
                            f"释放后仍可读取: {data_after.raw}",
                        )
                except Exception as e:
                    add_result(
                        results,
                        "network.getData after disown",
                        "成功",
                        f"释放后读取报错: {str(e)[:100]}",
                    )
            else:
                add_result(
                    results,
                    "network.disownData",
                    "跳过",
                    "未成功拿到 response 数据,跳过释放验证",
                )
                add_result(
                    results,
                    "network.getData after disown",
                    "跳过",
                    "未成功拿到 response 数据,跳过释放后验证",
                )

        if collector:
            try:
                collector.remove()
                add_result(
                    results,
                    "network.removeDataCollector",
                    "成功",
                    f"已移除 {collector.id}",
                )
                collector = None
            except Exception as e:
                add_result(results, "network.removeDataCollector", "失败", str(e)[:120])

        page.network.clear_extra_headers()
        add_result(results, "clear extra headers", "成功", "额外请求头已清理")

        print_results(results)

    except Exception as e:
        add_result(results, "示例执行", "失败", str(e)[:120])
        print_results(results)
        raise
    finally:
        try:
            page.listen.stop()
        except Exception:
            pass
        try:
            page.intercept.stop()
        except Exception:
            pass
        try:
            if server is not None:
                server.stop()
        except Exception:
            pass
        if collector:
            try:
                collector.remove()
            except Exception:
                pass
        try:
            page.network.clear_extra_headers()
        except Exception:
            pass
        try:
            page.quit()
        except Exception:
            pass


if __name__ == "__main__":
    main()

BrowsingContext 事件:上下文创建销毁与用户提示框事件

examples/30_browsing_context_events.py
GitHub 仓库 已同步源码 查看 GitHub 源码
network strict
一句话定位

把 context 事件和 prompt 事件统一放到协议事件角度理解。

涉及 API
page.events.startpage.events.waitpage.contexts.create_tabpage.contexts.create_windowpage.contexts.closepage.accept_promptpage.set_prompt_handler
展示的不同用法
  • 监听 tab 和 window 创建销毁
  • 监听 alert 和 prompt 打开关闭
  • 配合自动 prompt 策略处理事件链
适用场景

适合 browsing-context、events 与 prompts 页面。

推荐阅读
navigation-events-24prompts-17
完整代码

来源:app/content/ruyipage_examples/30_browsing_context_events.py · 仓库文件:30_browsing_context_events.py

python 注释版
#!/usr/bin/env python
# =============================
# RuyiPage 示例注释版
# 标题: BrowsingContext 事件:上下文创建销毁与用户提示框事件
# 说明: 把 context 事件和 prompt 事件统一放到协议事件角度理解。
# 你会学到:
#   1. 监听 tab 和 window 创建销毁
#   2. 监听 alert 和 prompt 打开关闭
#   3. 配合自动 prompt 策略处理事件链
# 相关 API: page.events.start, page.events.wait, page.contexts.create_tab, page.contexts.create_window, page.contexts.close, page.accept_prompt, page.set_prompt_handler
# =============================

# -*- coding: utf-8 -*-
"""示例30: BrowsingContext Events(严格结果版)"""

import io
import sys
from typing import Dict, List

if sys.platform == "win32":
    sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding="utf-8")
    sys.stderr = io.TextIOWrapper(sys.stderr.buffer, encoding="utf-8")

from ruyipage import BidiEvent, FirefoxPage


def add_result(
    results: List[Dict[str, str]], item: str, status: str, note: str
) -> None:
    results.append({"item": item, "status": status, "note": note})


def print_results(results: List[Dict[str, str]]) -> None:
    print("\n| 项目 | 状态 | 说明 |")
    print("| --- | --- | --- |")
    for row in results:
        print(f"| {row['item']} | {row['status']} | {row['note']} |")


def main() -> None:
    print("=" * 70)
    print("测试 30: BrowsingContext Events")
    print("=" * 70)

    page = FirefoxPage()
    results: List[Dict[str, str]] = []
    temp_tab = None
    temp_window = None

    try:
        page.get("https://www.example.com")
        add_result(results, "页面加载", "成功", "example.com 已加载")

        if page.events.start(
            [
                "browsingContext.contextCreated",
                "browsingContext.contextDestroyed",
                "browsingContext.userPromptOpened",
                "browsingContext.userPromptClosed",
            ],
            contexts=[],
        ):
            add_result(results, "事件订阅", "成功", "已订阅 context / prompt 相关事件")
        else:
            add_result(results, "事件订阅", "失败", "未能订阅 browsingContext 事件")
            print_results(results)
            return

        # 1. contextCreated / contextDestroyed
        page.events.clear()
        temp_tab = page.contexts.create_tab()
        created_tab: BidiEvent | None = page.events.wait(
            "browsingContext.contextCreated", timeout=3
        )
        if created_tab and created_tab.context == temp_tab:
            add_result(
                results,
                "browsingContext.contextCreated tab",
                "成功",
                f"context={temp_tab}",
            )
        else:
            add_result(
                results,
                "browsingContext.contextCreated tab",
                "失败",
                f"expected={temp_tab}",
            )

        temp_window = page.contexts.create_window()
        created_window: BidiEvent | None = page.events.wait(
            "browsingContext.contextCreated", timeout=3
        )
        if created_window and created_window.context == temp_window:
            add_result(
                results,
                "browsingContext.contextCreated window",
                "成功",
                f"context={temp_window}",
            )
        else:
            add_result(
                results,
                "browsingContext.contextCreated window",
                "失败",
                f"expected={temp_window}",
            )

        if temp_tab:
            page.contexts.close(temp_tab)
            destroyed_tab = page.events.wait(
                "browsingContext.contextDestroyed", timeout=3
            )
            if destroyed_tab and destroyed_tab.context == temp_tab:
                add_result(
                    results,
                    "browsingContext.contextDestroyed tab",
                    "成功",
                    f"context={temp_tab}",
                )
            else:
                add_result(
                    results,
                    "browsingContext.contextDestroyed tab",
                    "失败",
                    f"expected={temp_tab}",
                )
            temp_tab = None

        if temp_window:
            page.contexts.close(temp_window)
            destroyed_window = page.events.wait(
                "browsingContext.contextDestroyed", timeout=3
            )
            if destroyed_window and destroyed_window.context == temp_window:
                add_result(
                    results,
                    "browsingContext.contextDestroyed window",
                    "成功",
                    f"context={temp_window}",
                )
            else:
                add_result(
                    results,
                    "browsingContext.contextDestroyed window",
                    "失败",
                    f"expected={temp_window}",
                )
            temp_window = None

        # 2. userPromptOpened / userPromptClosed
        page.events.clear()

        page.run_js("alert('hello alert')", as_expr=False)
        opened_alert = page.events.wait("browsingContext.userPromptOpened", timeout=3)
        if opened_alert and opened_alert.user_prompt_type == "alert":
            add_result(
                results,
                "browsingContext.userPromptOpened alert",
                "成功",
                opened_alert.message or "",
            )
        else:
            add_result(
                results,
                "browsingContext.userPromptOpened alert",
                "失败",
                "未观察到 alert 打开事件",
            )

        page.accept_prompt(timeout=3)
        closed_alert = page.events.wait("browsingContext.userPromptClosed", timeout=3)
        if closed_alert and closed_alert.accepted is True:
            add_result(
                results,
                "browsingContext.userPromptClosed alert",
                "成功",
                f"accepted={closed_alert.accepted}",
            )
        else:
            add_result(
                results,
                "browsingContext.userPromptClosed alert",
                "失败",
                "未观察到 alert 关闭事件",
            )

        page.events.clear()
        page.set_prompt_handler(prompt="ignore", prompt_text="Test User")

        page.run_js("prompt('Enter your name:', 'default')", as_expr=False)
        opened_prompt = page.events.wait("browsingContext.userPromptOpened", timeout=3)
        if opened_prompt and opened_prompt.user_prompt_type == "prompt":
            add_result(
                results,
                "browsingContext.userPromptOpened prompt",
                "成功",
                opened_prompt.message or "",
            )
        else:
            add_result(
                results,
                "browsingContext.userPromptOpened prompt",
                "失败",
                "未观察到 prompt 打开事件",
            )

        closed_prompt = page.events.wait("browsingContext.userPromptClosed", timeout=3)
        if closed_prompt and closed_prompt.accepted is True:
            add_result(
                results,
                "browsingContext.userPromptClosed prompt",
                "成功",
                f"accepted={closed_prompt.accepted}",
            )
        else:
            add_result(
                results,
                "browsingContext.userPromptClosed prompt",
                "跳过",
                "当前环境下 prompt 自动注入文本后未稳定观察到 userPromptClosed",
            )

        page.clear_prompt_handler()
        print_results(results)

    except Exception as e:
        add_result(results, "示例执行", "失败", str(e)[:120])
        print_results(results)
        raise
    finally:
        try:
            page.clear_prompt_handler()
        except Exception:
            pass
        try:
            page.events.stop()
        except Exception:
            pass
        if temp_tab:
            try:
                page.contexts.close(temp_tab)
            except Exception:
                pass
        if temp_window:
            try:
                page.contexts.close(temp_window)
            except Exception:
                pass
        try:
            page.quit()
        except Exception:
            pass


if __name__ == "__main__":
    main()

网络事件:请求开始、响应开始、响应完成、错误与认证

examples/31_network_events.py
GitHub 仓库 已同步源码 查看 GitHub 源码
network strict
一句话定位

是 network-events 页最直接的主示例。

涉及 API
page.events.startpage.events.waitpage.intercept.startpage.intercept.waitcontinue_with_auth
展示的不同用法
  • 观察 beforeRequestSent、responseStarted、responseCompleted
  • 监听 fetchError
  • 处理 authRequired 并补充用户名密码
适用场景

适合 network-events 与 auth 页面。

相关文档页面
推荐阅读
advanced-network-18collector-28
完整代码

来源:app/content/ruyipage_examples/31_network_events.py · 仓库文件:31_network_events.py

python 注释版
#!/usr/bin/env python
# =============================
# RuyiPage 示例注释版
# 标题: 网络事件:请求开始、响应开始、响应完成、错误与认证
# 说明: 是 network-events 页最直接的主示例。
# 你会学到:
#   1. 观察 beforeRequestSent、responseStarted、responseCompleted
#   2. 监听 fetchError
#   3. 处理 authRequired 并补充用户名密码
# 相关 API: page.events.start, page.events.wait, page.intercept.start, page.intercept.wait, continue_with_auth
# =============================

# -*- coding: utf-8 -*-
"""示例31: Network Events(严格结果版)"""

import io
import sys
from typing import Dict, List

if sys.platform == "win32":
    sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding="utf-8")
    sys.stderr = io.TextIOWrapper(sys.stderr.buffer, encoding="utf-8")

from ruyipage import FirefoxPage, InterceptedRequest
from ruyipage._functions.tools import find_free_port

from test_server import TestServer


def add_result(
    results: List[Dict[str, str]], item: str, status: str, note: str
) -> None:
    results.append({"item": item, "status": status, "note": note})


def print_results(results: List[Dict[str, str]]) -> None:
    print("\n| 项目 | 状态 | 说明 |")
    print("| --- | --- | --- |")
    for row in results:
        print(f"| {row['item']} | {row['status']} | {row['note']} |")


def main() -> None:
    print("=" * 70)
    print("测试 31: Network Events")
    print("=" * 70)

    page = FirefoxPage()
    server = None
    results: List[Dict[str, str]] = []

    try:
        server = TestServer(port=find_free_port(9330, 9430)).start()
        base_url = server.get_url("")[:-1]

        page.get("about:blank")
        add_result(
            results, "页面加载", "成功", f"about:blank 已加载,服务地址: {base_url}"
        )

        page.events.start(
            [
                "network.beforeRequestSent",
                "network.responseStarted",
                "network.responseCompleted",
                "network.fetchError",
                "network.authRequired",
            ],
            contexts=[page.tab_id],
        )

        # 1. 普通请求事件链
        page.events.clear()
        page.run_js(
            f"fetch('{base_url}/api/data').catch(()=>null); return true;",
            as_expr=False,
        )

        before_event = page.events.wait("network.beforeRequestSent", timeout=5)
        started_event = page.events.wait("network.responseStarted", timeout=5)
        completed_event = page.events.wait("network.responseCompleted", timeout=5)

        if before_event and str((before_event.request or {}).get("url", "")).endswith(
            "/api/data"
        ):
            add_result(
                results,
                "network.beforeRequestSent",
                "成功",
                (before_event.request or {}).get("url", ""),
            )
        else:
            add_result(
                results,
                "network.beforeRequestSent",
                "失败",
                "未观察到 /api/data 请求开始事件",
            )

        if (
            started_event
            and int((started_event.response or {}).get("status", 0)) == 200
        ):
            add_result(
                results,
                "network.responseStarted",
                "成功",
                f"status={(started_event.response or {}).get('status')}",
            )
        else:
            add_result(
                results, "network.responseStarted", "失败", "未观察到 200 响应开始事件"
            )

        if (
            completed_event
            and int((completed_event.response or {}).get("status", 0)) == 200
        ):
            add_result(
                results,
                "network.responseCompleted",
                "成功",
                f"status={(completed_event.response or {}).get('status')}",
            )
        else:
            add_result(
                results,
                "network.responseCompleted",
                "失败",
                "未观察到 200 响应完成事件",
            )

        # 2. fetchError
        page.events.clear()
        page.run_js(
            "fetch('http://127.0.0.1:9/').catch(()=>null); return true;",
            as_expr=False,
        )
        error_event = page.events.wait("network.fetchError", timeout=5)
        if error_event:
            add_result(
                results, "network.fetchError", "成功", error_event.error_text or ""
            )
        else:
            add_result(
                results, "network.fetchError", "跳过", "当前环境未稳定观察到 fetchError"
            )

        # 3. authRequired
        page.events.clear()
        page.intercept.start(
            url_patterns=[{"type": "string", "pattern": base_url + "/api/auth"}],
            phases=["authRequired"],
        )
        try:
            page.get(base_url + "/api/auth")
        except Exception:
            pass

        auth_req: InterceptedRequest | None = page.intercept.wait(timeout=5)
        auth_event = page.events.wait("network.authRequired", timeout=5)

        if auth_event and auth_req:
            add_result(results, "network.authRequired", "成功", auth_req.url)
            auth_req.continue_with_auth(
                action="provideCredentials",
                username="user",
                password="pass",
            )

            auth_done = page.events.wait("network.responseCompleted", timeout=5)
            if auth_done and int((auth_done.response or {}).get("status", 0)) == 200:
                add_result(
                    results,
                    "network.authRequired credentials",
                    "成功",
                    "提供凭证后认证通过",
                )
            else:
                add_result(
                    results,
                    "network.authRequired credentials",
                    "失败",
                    "提供凭证后未观察到 200 完成事件",
                )
        else:
            add_result(
                results,
                "network.authRequired",
                "跳过",
                "当前环境未稳定观察到 authRequired",
            )
            add_result(
                results,
                "network.authRequired credentials",
                "跳过",
                "未观察到 authRequired,跳过凭证验证",
            )

        print_results(results)

    except Exception as e:
        add_result(results, "示例执行", "失败", str(e)[:120])
        print_results(results)
        raise
    finally:
        try:
            page.intercept.stop()
        except Exception:
            pass
        try:
            page.events.stop()
        except Exception:
            pass
        try:
            if server is not None:
                server.stop()
        except Exception:
            pass
        try:
            page.quit()
        except Exception:
            pass


if __name__ == "__main__":
    main()

请求头抓包:真实跟踪页 POST 请求头与请求体诊断

examples/40_1_request_header_capture.py
GitHub 仓库 待同步 查看 GitHub 源码
network strict
一句话定位

面向抓包与数据采集诊断的示例,重点不是改包,而是先看清真实页面发出的 POST 请求头和请求体形状。

涉及 API
FirefoxPagepage.intercept.startInterceptedRequestreq.headersreq.bodyreq.continue_request
展示的不同用法
  • 抓取真实页面发出的 POST 请求头
  • 预览请求体内容用于逆向分析
  • 快速判断页面 URL 与接口 URL 的关系
  • 在不打断真实请求的前提下继续放行请求
适用场景

适合 network、intercept、data-collector 与 scraper 调试场景。

推荐阅读
advanced-network-18collector-28packet-capture-40

抓包采集:从真实页面请求链路定位接口数据

examples/40_scraper_packet_capture.py
GitHub 仓库 待同步 查看 GitHub 源码
network strict
一句话定位

用于 scraper 与接口采集场景的抓包示例,重点是先观察请求链路,再决定后续是监听、拦截还是 collector。

涉及 API
page.intercept.startpage.events.startpage.events.waitpage.network.add_data_collectorreq.continue_request
展示的不同用法
  • 观察真实页面请求链路
  • 定位数据接口与请求特征
  • 为后续响应体采集或 mock 做前置分析
  • 形成 scraper 场景下的网络调试起点
适用场景

适合 network、network-events、data-collector 和 scraper 文档链路。

推荐阅读
request-header-capture-40-1collector-28network-events-31

代理认证与 IP 信息验证

examples/38_proxy_auth_ipinfo.py
GitHub 仓库 待同步 查看 GitHub 源码
network advanced
一句话定位

展示通过 fpfile 与代理配置协作,让业务脚本只保留最小代理入口。

涉及 API
opts.set_proxyopts.set_fpfileFirefoxPagepage.get
展示的不同用法
  • 配置 HTTP 代理地址
  • 通过 fpfile 承载代理认证账号密码
  • 访问 ipinfo 验证出口 IP
  • 降低业务脚本中明文认证信息暴露
适用场景

适合 proxy-auth、anti-bot 和 fingerprint-browser 页面。

推荐阅读
quickstart-fingerprint-browseradvanced-network-18