Vanguard News Network
VNN Media
VNN Digital Library
VNN Reader Mail
VNN Broadcasts

Old January 18th, 2008 #1
Alex Linder
Administrator
 
Join Date: Nov 2003
Posts: 45,756
Blog Entries: 34
Default Technical Question Related to VNNF Performance

The forum is back up, but timing out too often.

We are trying to establish the reason for this.

The apache webserver software is spiking in memory usage, specifically in the
privvmpages.

If you have an idea how to solve this, let us know.

Thanks.
 
Old January 18th, 2008 #2
PC}{Magician
Semper Fi
 
PC}{Magician's Avatar
 
Join Date: Oct 2006
Location: Florida
Posts: 401
Default

There is so much I don't know about your system but what is happening to me, having to reload/refresh is a common event when the MaxClient setting is not configured correctly. This is a common setting and I'm sure your people have done this but just double checking......

Quote:
The single biggest hardware issue affecting webserver performance is RAM. A webserver should never ever have to swap, as swapping increases the latency of each request beyond a point that users consider "fast enough". This causes users to hit stop and reload, further increasing the load. You can, and should, control the MaxClients setting so that your server does not spawn so many children it starts swapping. This procedure for doing this is simple: determine the size of your average Apache process, by looking at your process list via a tool such as top, and divide this into your total available memory, leaving some room for other processes.
 
Old January 18th, 2008 #3
Richard H.
Banned
 
Join Date: Jul 2007
Location: Right Here
Posts: 1,798
Default Has Anybody Lost Reps

I've lost about 400 rep points and am missing who did them since 27 December; they were origionally up to date as of Jan 7th or 8th when the forum went down...what happened?
 
Old January 18th, 2008 #4
AlbionMP...
Senior Member
 
Join Date: Mar 2007
Posts: 1,463
Default

I will have a guess, that the webserver is trafficking each posters set of PM's (Private Messages) for every client request.

Try to find out whether you can toggle the sendPM setting within VB (as its most likely the problem is with the setup of the V-Bulletin Software), otherwise you will have to look at the Apache Server.

Last edited by AlbionMP; January 18th, 2008 at 06:53 PM.
 
Old January 18th, 2008 #5
OTPTT
Banned
 
Join Date: Nov 2006
Posts: 9,896
Default

I'll post some things I find online. Maybe this will be helpful.

Quote:
you should first see if there is an abnormal reason for it being high (like perhaps a runaway process). But if not, we can raise this limit by 10-20% for you.

As long as you are not running out of privvmpages. If you are hitting that too, you are exceeding your total memory allocation, and agin you will need to try identify what using it, or upgrade plans in this case.

...

privvmpages is roughly the RAM available in 4K blocks (its more complicated than that, but thats a decent analogy). If you get close to the soft limit, the power panel will make it yellow or orange to show you. When you exceed that it will go red, and then black if you hit the hard limit.

No failures occur until you hit the hard limit.

You can see these on the commandline with

cat /proc/user_beancounters

it shows if you have hit a limit and how many times, but not when it happened like the power panel shows you. Link.
 
Old January 18th, 2008 #6
OTPTT
Banned
 
Join Date: Nov 2006
Posts: 9,896
Default

Quote:
Originally Posted by Richard H. View Post
I've lost about 400 rep points and am missing who did them since 27 December; they were origionally up to date as of Jan 7th or 8th when the forum went down...what happened?
This thread is not for such posts Richard. We've all lost rep points. Please stay out of this thread unless you have information specifically related to the question that Alex asked.
 
Old January 18th, 2008 #7
AlbionMP...
Senior Member
 
Join Date: Mar 2007
Posts: 1,463
Default

Quote:
Originally Posted by Alex Linder View Post
The apache webserver software is spiking in memory usage, specifically in the privvmpages.
Have a look at the Private Messages settings in the V-Bulletin Software.
 
Old January 18th, 2008 #8
PC}{Magician
Semper Fi
 
PC}{Magician's Avatar
 
Join Date: Oct 2006
Location: Florida
Posts: 401
Default

Yes, there is a lot of good info on the World Wide Web....

Without knowing more about the system it's hard to pinpoint which tweak you need.

Varg should be able to find something out there that will fix it, I have confidence in him. Heck, he's almost as smart as me......
 
Old January 18th, 2008 #9
Jett Rink
Banned
 
Join Date: Aug 2007
Location: Squaresville
Posts: 7,502
Default

Quote:
Originally Posted by Alex Linder View Post
The forum is back up, but timing out too often.

We are trying to establish the reason for this.

The apache webserver software is spiking in memory usage, specifically in the
privvmpages.

If you have an idea how to solve this, let us know.

Thanks.

You need to use a PHP Cache so that the server does not need to recompile the same script every time.

Install this:

http://pecl.php.net/package/APC
 
Old January 18th, 2008 #10
OTPTT
Banned
 
Join Date: Nov 2006
Posts: 9,896
Default

Not sure if this is helpful but maybe there's some to be gleaned here. These are from support forums.

Quote:
If Apache, do you properly have a maximum number of concurrent Apache children processes defined? If your maximum allowed is too large, it's very possible for this number to increase depending upon a large number of factors (including usage of your physical hardware node by other VPSs on that node), resulting in all of your memory being consumed by Apache processes.
...

It only takes a few seconds for Apache to spin up to the maximum number of processes --- unless you're monitoring the memory usage second-by-second, you may not even see this.

And again, if you're seeing failcnts increase, then you *are* using all of your memory at the point of the failure --- failcnts are incremented *only* by conditions within your individual VPS.

And also, again, this event may not be a result of peaking of *your* sites. Remember, that VPSs are inherently shared. If, for some reason, another user of your physical node has a drastic load over a short period of time, it will affect the operation of your VPS. In this situation, even though your own site usage may be low, your Apache processes can spike because the overall system response may be slow resulting in spinning up of additional Apache children to handle requests. So don't assume that the problem is a direct result of the environment within your individual VPS --- it may be a result of other VPSs on the same physical node.

But what you should be sure of, however, is that the maximum number of children * the average memory usage of each of your apache children isn't greater than the memory you have available on average. There's no reason to configure apache into a situation where you can have more children than can fit in your available memory (especially at VPSLink, because you have no burst and no swap).

....

It only takes a few seconds for Apache to spin up to the maximum number of processes --- unless you're monitoring the memory usage second-by-second, you may not even see this.

And again, if you're seeing failcnts increase, then you *are* using all of your memory at the point of the failure --- failcnts are incremented *only* by conditions within your individual VPS.

And also, again, this event may not be a result of peaking of *your* sites. Remember, that VPSs are inherently shared. If, for some reason, another user of your physical node has a drastic load over a short period of time, it will affect the operation of your VPS. In this situation, even though your own site usage may be low, your Apache processes can spike because the overall system response may be slow resulting in spinning up of additional Apache children to handle requests. So don't assume that the problem is a direct result of the environment within your individual VPS --- it may be a result of other VPSs on the same physical node.

But what you should be sure of, however, is that the maximum number of children * the average memory usage of each of your apache children isn't greater than the memory you have available on average. There's no reason to configure apache into a situation where you can have more children than can fit in your available memory (especially at VPSLink, because you have no burst and no swap).

Thanks for sharing that, this is the main reason I posted this as I have a feeling that there is something that is effecting my memory availability or something else that is not on my side of things.

I have a few more logging options enabled now so I will keep an eye on it...
...

I made some coonfig changes tonight and now my memory usage has been cut in half and there are no more slow queries at all so far in the last hour. Before this newest config change there were a bunch in just 5 minutes.


I'm not convinced this will fix this issue, but it's a start in the right direction I think.

Uptime: 4697 Threads: 2 Questions: 104215 Slow queries: 0 Opens: 909 Flush tables: 1 Open tables: 365 Queries per second avg: 22.188
mysqladmin Ver 8.41 Distrib 5.0.46, for redhat-linux-gnu on i686

Link.

Last edited by OTPTT; January 18th, 2008 at 07:10 PM.
 
Old January 18th, 2008 #11
Jett Rink
Banned
 
Join Date: Aug 2007
Location: Squaresville
Posts: 7,502
Default

Most likely you do not have MySQL Cache enabled. By default its off.

http://www.cyberciti.biz/tips/enable...rformance.html
 
Old January 18th, 2008 #12
Jett Rink
Banned
 
Join Date: Aug 2007
Location: Squaresville
Posts: 7,502
Default

Do you have innodb running, if so it will suck up a ton of memory.
 
Old January 18th, 2008 #13
varg
...
 
Join Date: Apr 2004
Posts: 9,741
Default

no innodb is disabled..

we used a php cache program -- eAccelerator.. it didnt help the problem we're having though..

Have changed values of MaxClients / etc in Apache..

Apache isnt even the thing using all the memory.. Me and Dietrich just stopped both apache and mysqld .. with just apache running the memory was fine, but when starting mysqld it ate it all....

I think it has to do with VNNF's database being 1gig in size and us only have 256mb ram with 500 burstable
 
Old January 18th, 2008 #14
yankee jane
Mene, Mene, Tekel Upharsin
 
yankee jane's Avatar
 
Join Date: Sep 2007
Posts: 3,588
Default

Glad to know its not me - 98% of the time when I try to go to a different page I get Proxy Error and have to keep hitting refresh to get the page to come up and that works about 70% of the time ...
 
Old January 18th, 2008 #15
Jett Rink
Banned
 
Join Date: Aug 2007
Location: Squaresville
Posts: 7,502
Default

Yep, 256MB is waaaaaaaaaaaaaaaaaaaayyyyyy too small. With 100+ people connected you are going to need 768MB minimum.
 
Old January 18th, 2008 #16
OTPTT
Banned
 
Join Date: Nov 2006
Posts: 9,896
Default

Quote:
Originally Posted by varg View Post
I think it has to do with VNNF's database being 1gig in size and us only have 256mb ram with 500 burstable
What kind of machine are you guys running? I have a couple sticks of pc100 memory I'm not using.
 
Old January 18th, 2008 #17
Jett Rink
Banned
 
Join Date: Aug 2007
Location: Squaresville
Posts: 7,502
Default

I think they should ditch the old database and see if a new fresh board would run on this VPS at 256 megs.
 
Old January 18th, 2008 #18
Subrosa
Banned
 
Join Date: Dec 2003
Location: Burning Down Indian Country
Posts: 6,185
Default

I think there should be a fund raiser on VNNF to get new equipment. I was under the assumption that server space was being rented, but it appears someone is running a whitebox clone somewhere?
 
Old January 18th, 2008 #19
varg
...
 
Join Date: Apr 2004
Posts: 9,741
Default

A fundraiser should definitely be held for Alex again.. like the last one..

cause after the last server gave us the boot, Alex has had to pay more for hosting

Hating the jewz isnt cheap people
 
Old January 19th, 2008 #20
Leshrac
Mad Science
 
Leshrac's Avatar
 
Join Date: Jan 2007
Location: Belgium
Posts: 2,686
Default

Yup, if the load is too much for the burst to handle, it fails :/

Nothing can solve that, except more burst/less load
__________________
Thinking... Please wait.
 
Reply

Share


Thread
Display Modes


All times are GMT -5. The time now is 09:03 AM.
Page generated in 0.49412 seconds.