For a research project, we are systematically downloading web-pages indexed by DMOZ. IN principle each webpage should be downloaded once, but there might be multiple pages downloaded from one site.
Schlobach KS
Last modified: Tue Nov 4 15:06:50 CET 2008