how to make a good robots.txt file

Discussion in 'Web Development and Programming' started by kieras, Jun 16, 2011.

  1. kieras

    kieras Regular Member

    Joined:
    Jun 16, 2011
    Messages:
    1
    Likes Received:
    0
    hello friends

    (link removed -- new users, please read forum rules & FAQ)

    User-agent: *
    Disallow: *.js
    Disallow: search.php
    Disallow: /includes/
    Disallow: /install/
    Disallow: /customavatars/

    Maybe someone of you explain to me why in this tutorial to create a robots.txt
    is
    Is such a manner without the slash of Use is correct?


    Please explain why the tutorial was used with vBSEO Disallow: without a slash for the search
     
  2. Cerberus

    Cerberus Admin Talk Staff

    Joined:
    May 3, 2009
    Messages:
    1,031
    Likes Received:
    500

Share This Page