1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

superfeedr bot to block or not to block?

Discussion in 'Web Hosting' started by Jeffr2014, Mar 26, 2015.

  1. #1
    I was analyzing traffic logs and noticed that my sites get a lot of connections from superfeedr.com bots. I know that superfeedr crawls RSS feeds but I am getting concerned when I see a new connection every few minutes from different IP addresses...

    After a bit of browsing I learned that superfeedr is associated with IFTTT (at least that's what superfeedr's Twitter guy claims). I do have IFTTT trigger rules for RSS updates enabled, so this can explain this unwanted attention.

    Anyway, I am wondering if allowing these superfeedr bots is a good idea... we had few cases in the past when the new content was stolen from our sites before it got indexed by G. This is a worst case scenario - you spend time and/or money preparing new posts and then this content gets stolen and in some cases indexed prior, so you end up being treated by G as plagiarist. Apparently, Superfeedr could be a perfect mechanism for stealing content...

    Should just I treat it similarly to semalt and others referral spam bots and block using .htaccess and WordFence plugin?

    Any feedback/experience with these superfeedr bots is appreciated.
     
    Jeffr2014, Mar 26, 2015 IP
  2. mikebvm

    mikebvm Active Member

    Messages:
    75
    Likes Received:
    5
    Best Answers:
    0
    Trophy Points:
    98
    #2
    It's safe to block it. If they're all coming from a specific IP range that would probably be the easiest way to block it. What sorts of IPs are you seeing?
     
    mikebvm, Mar 26, 2015 IP
  3. Jeffr2014

    Jeffr2014 Active Member

    Messages:
    254
    Likes Received:
    18
    Best Answers:
    0
    Trophy Points:
    55
    #3
    I didn't check loacations but, but according to one of the logs, there is at least 10 different IPs for this bot just this morning. They are not from the same subnet/range, so it is difficult to block by IP, will have to use pattern match if decide to block it (still unsure).
     
    Jeffr2014, Mar 26, 2015 IP