Loading ...
Sorry, an error occurred while loading the content.
 

robots TXT file - Google

Expand Messages
  • Gordon Stewart
    Hello. this will take me a few weeks for google to update its index / directory etc I m wondering if this :- user-agent: googlebot disallow: *.txt will that
    Message 1 of 3 , Apr 3, 2010
      Hello.

      this will take me a few weeks for google to update its index / directory etc

      I'm wondering if this :-


      user-agent: googlebot
      disallow: *.txt


      will that stop / remove all 'txt' files from being indexed by Google ?

      (somehow, the txt / source files are being found / located & indexed
      by Google, & i only want the true URL / PHP scripts from being
      indexed. )

      I'll try & locate how google is finding the source files - but
      wondering about the above - Is it correct ?

      --
      G
      Freecycle Auckland :-
      http://www.freecycle.org/
    • Gordon Stewart
      ... Why does it always happen to me ? - Hours of hunting & no luck, as soon as I ask the question, I find the answer. its actually this :- User-agent:
      Message 2 of 3 , Apr 3, 2010
        On Sun, Apr 4, 2010 at 11:17 AM, Gordon Stewart <gordonisnz@...> wrote:
        > I'm wondering if this :-
        >
        >
        > user-agent: googlebot
        > disallow: *.txt
        >
        > will that stop / remove all 'txt' files from being indexed by Google ?

        Why does it always happen to me ? - Hours of hunting & no luck, as
        soon as I ask the question, I find the answer.

        its actually this :-


        User-agent: Googlebot
        Disallow: /*.txt$

        source: http://tools.seobook.com/robots-txt/

        --
        G
        Freecycle Auckland :-
        http://www.freecycle.org/
      • Bob
        ... Why does it always happen to me ? - Hours of hunting & no luck, as soon as I ask the question, I find the answer. its actually this :- User-agent:
        Message 3 of 3 , Apr 3, 2010
          ----- Original Message -----
          On Sun, Apr 4, 2010 at 11:17 AM, Gordon Stewart wrote:
          > I'm wondering if this :-
          >
          > user-agent: googlebot
          > disallow: *.txt
          >
          > will that stop / remove all 'txt' files from being indexed by Google ?

          Why does it always happen to me ? - Hours of hunting & no luck, as
          soon as I ask the question, I find the answer.

          its actually this :-

          User-agent: Googlebot
          Disallow: /*.txt$

          source: http://tools.seobook.com/robots-txt/


          Hi Gordon,
          You and me both.
          It happens.
          Regards, Bob E.
        Your message has been successfully submitted and would be delivered to recipients shortly.