Page 18 of 19

Re: Webmaster's Blog

Posted: Thu Mar 26, 2020 5:05 pm
by Mr-Paulio
This is a funny one? Any ideas?
Screenshot_20200326-170347_Chrome.jpg

Re: Webmaster's Blog

Posted: Thu Mar 26, 2020 6:49 pm
by ClanFever
Does it go away after a cached refresh?

Re: Webmaster's Blog

Posted: Thu Mar 26, 2020 9:50 pm
by mknz
Interesting. Probably an out of date spam black list. I'll take a look

Re: Webmaster's Blog

Posted: Thu Mar 26, 2020 10:32 pm
by Mr-Paulio
ClanFever wrote:Does it go away after a cached refresh?
Dunno, same thing on all my devices on multiple topics when on WiFi. Worked fine on phone data. Only seems to affect this website.

Re: Webmaster's Blog

Posted: Thu Mar 26, 2020 11:43 pm
by mknz
Mr-Paulio wrote:
ClanFever wrote:Does it go away after a cached refresh?
Dunno, same thing on all my devices on multiple topics when on WiFi. Worked fine on phone data. Only seems to affect this website.
Should be fixed now

Re: Webmaster's Blog

Posted: Fri Mar 27, 2020 2:41 am
by mknz
I've texted and emailed the owner of mr2.co.nz becuase we're in conflict for mr2.nz , we'll see what happens

Re: Webmaster's Blog

Posted: Fri Mar 27, 2020 3:18 am
by mknz
Rolled out a minor forum update

Re: Webmaster's Blog

Posted: Fri Mar 27, 2020 8:04 am
by GDII
Looks a bit nicer and things have been fixed. ie the MR2OCNZ logo image is actually in place now. Thanks!

Re: Webmaster's Blog

Posted: Fri Mar 27, 2020 9:47 am
by ClanFever
Yeah I love the refinements, they look quite smart

Re: Webmaster's Blog

Posted: Fri Mar 27, 2020 10:33 am
by Curly
Should the format of the poll be changed? Folk may have voted previously and since changed their mind but unable to register that change of mind.

Re: Webmaster's Blog

Posted: Fri Mar 27, 2020 12:28 pm
by mknz
Curly wrote:
Fri Mar 27, 2020 10:33 am
Should the format of the poll be changed? Folk may have voted previously and since changed their mind but unable to register that change of mind.
The poll is at least old enough that I voted for it :P

Re: Webmaster's Blog

Posted: Fri Mar 27, 2020 2:45 pm
by Curly
Me too. Many years ago

Re: Webmaster's Blog

Posted: Sun May 03, 2020 12:33 pm
by mknz
Went down from 6am till now (12:30). Looks like something hammered the server. Will investigate. I woke up and saw an email that a healthcheck had failed

Re: Webmaster's Blog

Posted: Sun May 03, 2020 1:10 pm
by mknz
As suspected it was a search engine crawler; specifically Bing (msnbot) and Huawei's beta search engine (AspiegelBot) and Ahrefs (AhrefsBot) and SEMrush (SemrushBot) and finally the least intrusive of all Google (Googlebot). So it's quite understandable why the poor DB crashed (looks like no data loss, just application got killed by the kernel).

The frustrating thing is a lot of these bots kept going despite the 503s for quite a while.

I'll limit Bing and Google's crawl rate. I'll block the rest, they have no need to index us

Re: Webmaster's Blog

Posted: Sun May 03, 2020 2:39 pm
by mknz
So in typical Google fashion, they've automated crawl control and don't give you anything. I'm not that worried about it since it was by far the least intrusive.

I've turned bing's rate way down and to the morning when we have fewer users.

I've given ahrefs, Huawei, semrush, and others the finger since they were hammering and we don't need their services.

I'm going to set up something that monitors the DB, and tries to start it again