Submit Software
  1. Home
  2. Windows
  3. Other
  4. RoboGen ROBOTS.TXT Editor
  5. Download

RoboGen ROBOTS.TXT Editor 1.52

License: Free Trial ‎File size: 1.52 MB
‎Users Rating: 5.0/5 - ‎1 ‎votes

ABOUT RoboGen ROBOTS.TXT Editor

RoboGen is a visual editor for Robot Exclusion Files. It allows the user to quickly and easily create the ROBOTS.TXT files required to instruct web search engines which parts of a web site are not to be indexed and made searchable by the general web public. RoboGen does this by providing the user a way to log onto his FTP server and then select the documents and directories which are not to be made searchable. RoboGen maintains a database of over 180 search engine user-agents, which are selectable from a drop down menu.

Related Programs

  • Viegas Rubim Free

    Viegas Rubim is a clone system livecd based on Ubuntu. It aims to be a powerful clone-rescue system for x86 x64 archs. Using the bittorrent to data transfer, accelerating the clone process.

  • ftp_lookup Free

    ftp_lookup is an indexer for ftp-servers on a LAN- / demo-scene-party.It does index the servers submitted by the user. It does not scan thenetwork on it's own.

  • Rizone MD5 Checksum Calculator Free

    MD5 Checksum Calculator was designed to be a small freeware utility for calculating the MD5 (Message Digest number 5) checksum values of specified files.The algorithm takes as input a message of arbitrary length and produce

  • Windows Controller Free Trial

    Windows Controller lets you to use keyboard shortcut key (the hotkey, e.g. Ctrl+Alt+K ) to execute a set of commands which can set the active window's position and size to custom values; move or resize each edge of the active

  • SpamArrow Free Trial

    SpamArrow is a simple and stable spam blocker to effectively filter out any unwanted spam emails with near zero false positives. It uses the famous SpamAssassin engine and the Bayesian filtering to automatically classify inco

  • HP 200LX connectivity tools Free

    There are several connectivity tools to transfer data between the HP 200LX PDA and Unix machines. This project is an attempt to integrate these tools.

  • Process And Port Analyzer Free

    A Process to port mapper, Active TCP and UDP connection viewer, Find out Which processes are using which ports

  • jTomtom Free

    jTomtom is et Java GPS updater. With jTomtom you can update the QuickFix et Radar POI in your Tomtom GPS

  • REN Free

    REN is a file renamer, to rename files. Various tests have proved its effectiveness in easy and quick renaming. REN helps home users and technicians who deal with a mixture of documents, sequential versions of text files, mus

  • Linux-DAQ-PCI Free

    The Linux-DAQ-PCI project develops and maintains a family ofLinux device drivers for PCI based data acquisition systems.Incremental package release naming convention is as follows:R0.x(Alpha), R1.x(beta), R2.x(production),