Ahrefs Pte. Ltd. They spider our website with brutal force, taking it down almost daily Singapore

 Our website, wetcrow..com, is hosted on servers at Hostek in St. Louis Missouri.

Lately, a company called AHrefss, began spidering our website, much like Google and others do.

AHerefs.com has been causing issues all over the Internet, by their spider robots working with brute force that they end up taking many of the websites offline for an hour or so, while they spider each page on our website.

We have contacted them several times over the last few months, plus followed their instructions to remove us from the list of websites tht they spider.

We researched the Internet and found articles on how to disable their spidering of our site and implemented those changes.

THeir robots do not obey the robots.txt file in the root folder of our domain.

They keep spidering our site, despite, our requests to stop, plus following their instructions to stop spidering us.

We posted on our terms and conditions that any spidering of our site is illegal for all sites, excpet Google, Yahoo, and Bing.

We also stated that anyone not obeying our robots.txt file agrees to pay us $5,000 per access to our site, per log entry.

We ask all Internet Marketers to boycott using AHrefs.com for tracking backlinks, due to their failure to obey standards other sites use in spdering websites.