Department of Homeland Security Scraping Sites?
Recently I noticed that one of the web servers for my business, Admo.net, had a higher than normal load. Normally this means that someone has exploited a PHP script and is mass-mailing everyone on the ‘net. This time, however, it looks like there was an IP address 18.104.22.168, from a OrgName DHS/BSL002A. This netblock belonged to "dhs.gov”. I quickly wondered what this was and pulled up the site.
Apparently the Department of Homeland Security has been scraping a harmless gaming community’s forums for some type of information. The alarming thing was that the requests came in at 5-10 request per second. Why must they scrape the sites? If they do, why can’t they do it at a more comfortable rate? This is crazy!