PROGRAMS BY lionheart

This robots.txt analysis tool can easily verify the content of "robots.txt" of an arbitrary web server (file that specifies behavior on the server for the robot of the search engine etc.) from online.