PHPCrawl 0.81

Developed by Uwe Hunfeld, phpcrawl , listed in Search Engine
Create your own niche community site! Free PHP Social Networking Software
download now
Download Now
From PHPKode | 371.5 KB
license type
Visit Website
The page visits: 2537

PHPCrawl is a PHP framework searching the Web. It can be used in writing search crawlers (spiders) that mine Web pages for various information. PHPCrawl acquires information it was configured to fetch and passes it to more powerful apps for further processing.

Features of PHPCrawl:
- Filters for URL and Content-Type data
- Define ways to handle cookies
- Define ways to handle robots.txt files
- Limit its activity in various ways
- Multi-processing modes

Requirements:
- PHP 5 or Higher
- PHP with OpenSSL support

What's New in This Version:
Fixed bugs:
- Links that are partially urlencoded and partially not get rebuild/encoded correctly now.
- Removed a unnecessary debug var_dump() from PHPCrawlerRobotsTxtParser.class.php
- Server-name-indication in TLS/SSL works correctly now.
- "base-href"-tags in websites get interpreted correctly now again.

Tags: cookie crawl filter robot spider
License type: GNU General Public License
Date added: 2 years, 6 months 29 days ago | Last updated: 7 months 8 days ago

More popular Search Engine

100% Free PHP Newsletter Script! Download now!

Listing Files

PHPCrawl_080
libs
UrlCache
ProcessCommunication
List All Files