Who is majestic 12 bot

Sign up to join this community. The best answers are voted up and rise to the top. Stack Overflow for Teams — Collaborate and share knowledge with a private group. Create a free Team What is Teams? Learn more. How to prevent majestic 12 from indexing a site Ask Question. Asked 11 years, 6 months ago. Active 11 years, 6 months ago. Viewed 5k times. We experience a lot of traffic and server load on a web server. All I can find out is majestic12 accessing pages all the time.

I wonder how I can prevent majestic12 from indexing the site Do they respect any robots. Improve this question. Add a comment. Active Oldest Votes. There should be no issue with blocking it like so:. That does appear weird.

It could be malicious users hiding their hack attempts as the bot, which does happen. I ended up blocking the bot. That didn't help, it went and added 2, more attempts that day then I blocked it in Cloudflare. The crazy part is the site it was trying to index only has 30 static pages on it. Not sure why it needed to access it 11, times in two days. If you only put the bot you want to block in the robots. Remember robots. I you really want to block bad robots mj12bot from majestic is only an example, please note that if you block mj12bot he also come from anonymous Ip Address you need to block them directly in the.

Blocking them in the. There is the script that you can customize with other bot or Ip address if you wan't to optimize server's ressources. Sign up to join this community. The best answers are voted up and rise to the top.

Stack Overflow for Teams — Collaborate and share knowledge with a private group. Create a free Team What is Teams?

Learn more. Trying to determine if bot crawling my site is malicious - MJ12bot Ask Question. Asked 1 year, 6 months ago. Learn more. Do I really have to block MJ12Bot as the prevailing visitor on my site? Ask Question. Asked 5 years, 5 months ago. Active 3 years, 6 months ago. Viewed 30k times. Improve this question.

I'm looking at your log in the question and it doesn't show any one computer spamming your server. MJ12Bot appeared only once. Mike, can you elaborate, please? Mike, so, how did MJ12Bot appeared only once? Show 2 more comments. Active Oldest Votes. Improve this answer. Sjoerd Linders Sjoerd Linders 1 1 silver badge 2 2 bronze badges. If this is ignored, it may follow that an imposter is just using the MJ12 user agent string? Add a comment. Greg Lindahl Greg Lindahl 2 2 silver badges 5 5 bronze badges.

You can easily slow down bot by adding the following to your robots. MrWhite MrWhite Crawl-Delay is not an answer, because with only a couple of pages on the site, it will have zero effect on the total number of requests per month. Only two pages!?