WebSPHINX - 可定制的Java Web爬虫
WebSPHINX是一个Java类包和Web爬虫的交互式开发环境。Web爬虫(也叫作机器人或蜘蛛)是可以自动浏览与处理Web页面的程序。WebSPHINX由两部分组成：爬虫工作平台和WebSPHINX类包。
- Multithreaded Web page retrieval in a simple application framework
- An object model that explicitly represents pages and links
- Support for reusable page content classifiers
- Tolerant HTML parsing
- Support for the robot exclusion standard
- Pattern matching, including regular expressions, Unix shell wildcards, and HTML tag expressions. Regular expressions are provided by the Apache jakarta-regexp regular expression library.
- Common HTML transformations , such as concatenating pages , saving pages to disk, and renaming links
本文由用户 jopen 自行上传分享，仅供网友学习交流。所有权归原作者，若您的权利被侵害，请联系管理员。
转载本站原创文章，请注明出处，并保留原始链接、图片水印。
本站是一个以用户分享为主的开源技术平台，欢迎各类分享！