知识卡片

问题卡 · 4460

动态链接需要渲染后提取

理解等级:中级

只下载静态 HTML 无法得到由 JavaScript 在运行时生成的链接。对此可以先进行动态渲染再解析;这增加处理成本,适用于确有必要发现动态内容的范围,不应对所有页面无差别开启。

这张卡片看不懂?

已有反馈 · 0 / 10 条

    还没有人提交反馈。

    在整书笔记中的位置

    原书出处 · 1 处

    • 网络爬虫 · 查看原书依据

      Step 4 - Wrap up In this chapter, we first discussed the characteristics of a good crawler: scalability, politeness, extensibility, and robustness. Then, we proposed a design and discussed key components. Building a scalable web crawler is not a trivial task because the web is enormously large and full of traps. Even though we have covered many topics, we still miss many relevant talking points:

    参考来源

    - 位置:《System Design Interview – An insiders guide, Second Edition》第09章《网络爬虫》(来源项目 book-sources;源文件:extracted/System Design Interview – An insiders guide, Second Edition/_epub-src/text/part0023_split_004.html) - 结论依据:原文在该位置给出所述对象、机制或比较,支持本卡的解释;原书未展开的实现不作为已证明结论,算式与一致性边界在正文明确限定。 - 原始内容:Step 4 - Wrap up In this chapter, we first discussed the characteristics of a good crawler: scalability, politeness, extensibility, and robustness. Then, we proposed a design and discussed key components. Building a scalable web crawler is not a trivial task because the web is enormously large and full of traps. Even though we have covered many topics, we still miss many relevant talking points: