L0st S0ul
 member, 173 posts
Wed 22 Mar 2017
at 04:44
Daily Site outage
How come at 4pm EST does the site go down every day.  Sometimes it is for 5mins and other times it is up to 30 mins or like last night over an hour?
JohnB
 member, 2033 posts
 Demigod of the Stunties
Wed 22 Mar 2017
at 07:59
Daily Site outage
There is an earlier thread on this ...

link to a message in this forum

In it jase says ...

jase:
... Netdepot, our current provider, have looked after us well but I have heard reports of how the business isn't what it once was due to it being sold.  ...


He is researching new suppliers.
jase
 admin, 3545 posts
 Cogito, ergo procuro.
 Carpe stultus!
Thu 23 Mar 2017
at 00:46
Daily Site outage
Something has been smashing the server at around 20:10 GMT every day, and yesterday it caused the server to run out of memory and pagefile (so it then locked up).  8pm GMT is 4am my time, so I don't see the down alert for a several hours.

Anyway I did a bunch of cleaning up yesterday which seemed to avoid the issue last night, hopefully everyone else saw the same.  I'll also lock down the firewall later today and see if any other spring cleaning can be done.
ericca
 member, 117 posts
 Just a chillin
 having fun
Thu 23 Mar 2017
at 01:07
Daily Site outage
I just want to thank you for all your work and those of your admins and mods.
L0st S0ul
 member, 176 posts
Thu 23 Mar 2017
at 06:52
Daily Site outage
Thanks Jase, yes last night was a lot better.
jase
 admin, 3546 posts
 Cogito, ergo procuro.
 Carpe stultus!
Fri 24 Mar 2017
at 01:40
Daily Site outage
Always a pleasure.

Still need to fix the server (well, get a new one) but it seems more stable at the moment after my spring clean.  Real test will be after about a week of running when resource allocation etc settles down, right now it's still on a high after a nice refreshing reboot.  As mentioned I also tightened up the firewall (about 16 hours ago) to increase security.  That's certainly stopped the attempts to log into the server from exotic locations.

Server is monitored 24x7 so I do get alerts if it's down, but keep in mind I'm GMT+8 so I might be fast asleep if there is another issue.
byzantinex
 member, 77 posts
Fri 24 Mar 2017
at 13:52
Daily Site outage
In reply to jase (msg # 6):

Jase, did you see my other post about SiteGround [link: https://www.siteground.com/ind...873a86f6cf8b1e249076] having Cloud services that automatically scale and move between physical servers to make downtime almost nonexistent?

I've got two really big clients (I'm a web dev) on these and I'm REALLY happy with them.
jase
 admin, 3547 posts
 Cogito, ergo procuro.
 Carpe stultus!
Sat 25 Mar 2017
at 05:41
Daily Site outage
I did thanks.  It's an interesting option but I've still got concerns around processing power, plus we'd need a bit more disk (currently using 77GB) than most of their options.  Then there's the fact it's fully managed, which could cause issues with our customisations.  Certainly on the list though.
jase
 admin, 3548 posts
 Cogito, ergo procuro.
 Carpe stultus!
Sun 2 Apr 2017
at 04:03
Daily Site outage
FYI;

I've been keeping a close eye on the server and noticed it suffered poor performance about two days ago.

Through a careful process of elimination in combination of checking the obvious I've tracked down what the issue was -- the backups.  We replicate all of RPoL's data to another location but the 3 million threads on record mean we've got a lot of files to backup, and the backup application was running out of memory trying to keep track of everything (as well as compress it on the fly to minimise bandwidth).

The backups have now been modified to do everything in much smaller parts (10th the size it used to), so hopefully that'll solve the memory utilisation issue.

Anecdotal evidence seems to indicate this (particular) issue has been resolved.  Fingers crossed.
byzantinex
 member, 79 posts
Sun 2 Apr 2017
at 04:58
Daily Site outage
In reply to jase (msg # 9):

Thank you sir!

Holler back at me on rmail about my request to help! :D
Evil Empryss
 member, 1541 posts
 Insert witty and
 appropriate quote here
Thu 11 May 2017
at 20:26
Daily Site outage
Looks like the problem has popped back up again.
GammaBear
 member, 766 posts
 Gaymer
Thu 11 May 2017
at 21:18
Daily Site outage
Yeah, I've noticed that, too. Seems to have been hitting the same time; around 4pm Eastern (US). Doesn't last long, though. I usually just close the tab, busy myself for about a half hour and it'll come up.
pdboddy
 member, 533 posts
Mon 15 May 2017
at 00:39
Daily Site outage
The site seems to be suffering from some sort of hiccup, it's taking ages for pages to come up.  As of 2030 EST.
Evil Empryss
 member, 1547 posts
 Insert witty and
 appropriate quote here
Fri 26 May 2017
at 10:25
Daily Site outage
Any word on fixing the daily site outage?
silverelf
 member, 221 posts
Sat 27 May 2017
at 23:13
Daily Site outage
Its still happening for me too, sometimes I can work around 4 other times its around 6 est for me when it dips.