PROGRAMS BY cwlr.sf.net

Very simple single-thread crawler for web (http) written in perl. Supports links following rules and collections to grab information of visited pages (regexp based).