While trying to find the perfect balance to make my sites run well out of my small VPS I noticed that my CPU usage was spiking. Unlike memory issues the Virtuozzo Power Panel didn’t issue a QoS alert for CPU overages. Apparently when you use too much CPU you just don’t get cycles for a while. Due to the spiky nature of CPU usage you don’t see the problem unless you catch it near the top of a spike.
I realized that my backup processes were helping to spike the CPU. I have a cron job to a mysql dump, and I have a remote machine regularly ssh/rsync in to copy files off. ssh and rsync use quite a bit of CPU. I should’ve realized that would happen, but “duhhhh”. I changed all my nonessential scripts to “nice” the commands. rsync was a bit tricky to get “nice”d, but I found the answers via Googling. What’s interesting is that Virtuozzo doesn’t seem to count the nice’d processes against the VPS’s CPU usage. They used spare host cycles apparently, and there are tons of spare host cycles. I think I actually sped my backups up with “nice”.
It’s tempting to try to run services nice’d, because I seem to get more “spare” cpu cycles than I get normal ones, but I have a feeling that will cause one problem or another down the line.
hi, I assume you are talking about DRUPAL here.
I do not understand all of your statements thouhgth I have a very similar problem to this one:
what do you mean by “to nice services”, please?
how can you switch off backup processes, on a hosted environemnt?
and, please, what changes did you do to your scripts, to be easier on CPU.
thanks a lot.
dave
Hello, and thanks for being my first commenter! I’m sorry I didn’t see your reply earlier. I’m not used to having replies here and hadn’t checked in ages.
I am running Drupal, but the processes I’m speaking about here aren’t specific to Drupal.
I am not using a hosted environment but running my own Virtual Private Server which for practical purposes gives me my own linux server to configure as I want. But perhaps some of my experience may be useful for you.
“nice” services: Each running program or process has a priority level; higher priorities can choke out lower priority processes. The command line program to run a program with an altered priority is “nice”. So if I am using a command like “scp -r /var/www user@remotehost:/home/backups/www” which uses the cpu to compress and encrypt the data I can give it a lower priority by instead using “nice scp -r /var/www user@remotehost:/home/backups/www”. Now if the web server process needs cpu it will get priority over the cpu-intensive but not-in-a-hurry backup process.
Now, in my particular case I am using rsync over ssh. My home box initiates and controls the session and pulls the data from the remote web server. Here are a couple of articles explaining this:
http://www.jdmz.net/ssh/
http://www.linux.com/article.pl?sid=04/11/04/0346256
I’ll let those do the heavy explaining and add here that if you run the local rsync as “nice” it doesn’t automatically make the server-side process “nice”. I made a script called “nice-rsync” and modified the “validate-rsync” script from the articles as shown below.
“nice-rsync” just runs rsync “nice”d and passes on the original parameters. This is what reduces the backup load on my server, but if you’re using a different backup it will probably be simpler.
The altered “validate-rsync” just adds my “nice-rsync” script as an allowed executable to run under the ssh shell.
nice-rsync:
#!/bin/shexec nice -n 19 /usr/bin/rsync $*
validate-rsync:
#!/bin/shcase "$SSH_ORIGINAL_COMMAND" in
*\&*)
echo "Rejected"
;;
*\(*)
echo "Rejected"
;;
*\{*)
echo "Rejected"
;;
*\;*)
echo "Rejected"
;;
*\< *)
echo "Rejected"
;;
*\`*)
echo "Rejected"
;;
rsync\ --server*)
$SSH_ORIGINAL_COMMAND
;;
/home/backup/nice-rsync\ --server*)
$SSH_ORIGINAL_COMMAND
;;
*)
echo "Rejected"
;;
esac
If you’re used to a hosted environment and not the Linux command line I may have left you needlessly confused in my previous reply, but maybe somebody will get something out of it.
I’ll try to keep this one simpler.
First, there are two things you should be backing up for your Drupal site: the Drupal and user files and the Drupal database.
In my case I am using command line scripts to dump the database to a backup file and then my home machine grabs the backup dump remotely. In your case you may have a Panel to manage the database. I’ve toyed with cPanel on an associate’s server but haven’t used it for backups, so I can’t help you with specifics there. Just be aware you probably have to worry about the database backups in addition to the file backups.
For file backups, using scp or rsync may be doable for you even in a hosted environment. If you are initiating the backup from your server’s command line, then just add “nice” to the beginning of the command and your backups won’t take cpu cycles from your web server. If your backup machine is initiating the request then you have to go through gyrations like in my previous comment to make sure the backup process on the web server side is “nice”.
Okay, I’m so excited to get a response I just keep replying to you.
I connected to my associate’s cPanel and clicked the backup icon. I have the option to download the home directory backup and a MySQL database backup. You’d want both of those for a Drupal site.
However I don’t see any way to affect cpu usage during backups using cPanel.
Are your cpu problems happening only when you’re doing backups, or always? I’m starting to wonder if your problem isn’t with backups, but I don’t have enough info.