1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

A Members Only Section: Can Spiders Crawl it?

Discussion in 'Content Management' started by Litwriter21, Mar 13, 2008.

0
  1. #1
    What if the website has a members only section, can the spiders crawl that type of information, since they cannot log in?
     
    Litwriter21, Mar 13, 2008 IP
  2. Pixelrage

    Pixelrage Peon

    Messages:
    5,083
    Likes Received:
    128
    Best Answers:
    0
    Trophy Points:
    0
    #2
    It's possible - be sure to block it in your robots.txt:

     
    Pixelrage, Mar 13, 2008 IP
  3. Litwriter21

    Litwriter21 Peon

    Messages:
    19
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #3
    Thanks, for the reply.


    Currently we don't have a robots text. LOL


    For the sake of my non web developing brain, how would blocking access allow access? I'm soooo confused. :)
     
    Litwriter21, Mar 16, 2008 IP
  4. Tearabite

    Tearabite Prominent Member

    Messages:
    4,629
    Likes Received:
    429
    Best Answers:
    0
    Trophy Points:
    300
    #4
    my site has several members-only/premium areas, when the bots hit it, all they get is 'access denied'..
     
    Tearabite, Mar 16, 2008 IP
  5. Dan Schulz

    Dan Schulz Peon

    Messages:
    6,032
    Likes Received:
    436
    Best Answers:
    0
    Trophy Points:
    0
    #5
    No, it's not possible. If a person has to register/log in in order to access content, a search engine spider cannot access the content since they would need to register/log in, which they are incapable of doing.
     
    Dan Schulz, Mar 17, 2008 IP