Convert's max execution time

[redna379]

Hello everyone.
I know this is not strictly an Imagemagick related question, but hopefully someone can help or suggest some hint anyhow.
And sorry in advance for my english, i am Italian

I created a PHP application to generate images, using convert via PHP exec() function.
OS is a linux box, Fedora core 6.
Sometimes it may happen that heavy paramenters are sent to it, requesting a very big and huge process.
It happened that, using "top" i found a convert process running from more than 5 hours!
The result of that was a whole server slowdown.

Considering that after such amount of time the requesting user would surely not be there waiting for the result, i wanted to limit convert max execution time, IE "kill convert if it is running from more than 5 mins".

I tried using shell_exec to run the command and get the pid, then using ps to check if the script is still running, but this silly solution is even worst, as it calls ps hundreds of times.

Do you have any hints on how i may run convert setting a max execution time for it?
Every way would be fine: controlling it via PHP, or using some linux program to launch convert itself.

The dream would probably be a small program in C being something like:
time_controlled_exec(cmd_to_execute,max_exec_time)
But i wasn't able to find any, and i'm not able to write one by myself.

Any hint will be very appreciated, thanks in advance!

[anthony]

Launch both a sleep and the convert in the background and get both their process ID's. You then just 'wait' for a signal that one of those processes has finished.

When the wait returns (a sub-process finished) kill the other and you can do another wait for the other child process, to give it a final cleanup.

None of this is IM specific. I have done it in scripts involving name service calls, which can lock up for hours if there are network faults. My experiments are details in my shell script notes...
http://www.cit.gu.edu.au/~anthony/info/ ... ript.hints
And do a search for "Command timeout"

If you manage to improve the above, or find a better simpler solution, then PLEASE let me know! This is a major problem I have needed to deal with many times over many many years.

[redna379]

Yes, what you described is really similar to what i did.
The only difference is mine is done in PHP only, so i had to exec 'ps' to check if the process was still running (PHP has not much system processes control).
With PHP, i saw the function proc_open, proc_get_status and proc_terminate, but proc_get_status (the most interestin one) is only for PHP5... not good for my software distrib.

My problem there is the server is already superloaded (peaks of 80/100 convert's execution at a time), so i will have to find an as light as possible way: i will surely let you know if i will find it.

Thanks for the hints!

[anthony]

Using 'ps' to check on a process is a VERY SLOW method.

The better method is to send the process a signal '0'. This is ling a 'ping' it does not actually signal the process just report is a process is present (and not a 'zombie') If the return from the kill 0 is Success, then the process is still running.

The life of processes in the UNIX environment is wierd. Parent
processes normally out live their children, and in fact processes who
die become zombies until either their parents reconise they are in
fact dead, or they themselves die. -- Treaty on UNIX Process Handling

Of course if you are just checking is the process is running, for the purpose of killing it, why not just kill it. It will fail if the process is already dead, and succeed if the process isn't dead

[redna379]

Maybe i found a better way ot do the trick.

First of all, let me show you the kind of app i am developing, so that you can better understand what my issue is:
http://www.realtimedesigner.com/

you can see it working here, for example (works BEST with FF):
http://designer.realtimedesigner.com/stickerz/

As you can see, you can do texts, shapes, images, apply effects to them, etc...
All the images you will see in designer are created with imagemagick on the fly.
So i really have tons of small generations: my purpose is not check the process to kill it, but kill it if it exhausted a given max execution time. Of course, if it will finish in time, i want the result immediately.

I couldn't use signal sending, as in PHP sending a signal to a process and reading its report would be really a pain.
So i had an idea: in Linux, the /proc filesystem is very fast, as it's stored in memory.
What if i place a pointer to the PID's status file to check what's going on?
And so i did. Here's the function i wrote so far.
It's PHP, but i think it can easily be ported in any other language.

Code: Select all

function TNG_execute($command,$timeout=6,$sleep=0.2) {
	$PID = trim(shell_exec("$command > /dev/null & echo $!"));
	$proc_file='/proc/'.$PID.'/stat';
	$cur = 0;
	$stillrunning=1;
	$return_value=true;
	
	$handle = fopen($proc_file, "r");
	if ($handle) {
		while ($cur<$timeout) {
			list($proc_pid, $proc_name, $proc_status) = fscanf($handle, "%d %s %c");
			// Status is stored in $proc_status, current runtime in $cur
			
			if (!file_exists($proc_file) || $proc_status!='R') {
				// No more running, exit this loop!
				$stillrunning=0;
				break;
			}
			$cur += $sleep;
			usleep(floor($sleep*1000000));
			rewind($handle);
		}
		fclose($handle);
	}
	
	if ($stillrunning==1 && file_exists($proc_file)) {
		// Time exhausted KILLING!!
		$return_value=false;
		exec("kill $PID");
	}
	
	return $return_value;
}

You may simply want to use it this way:

Code: Select all

$convertexecutable="/usr/bin/convert";
$sourcefile='file.jpg';
$destfile='file.png';
$command="$convertexecutable $sourcefile $destfile";
if (TNG_execute($command)) {
	echo "i did it in time!";
	// do what you want with the file...
} else {
	echo "sorry, time exhausted...";
	if (file_exists($destfile)) unlink($destfile);
}

As you can see, the main idea here is to use an fopen handle to avoid doing loop processes.
The handle will also read file changes, which is why i used

Code: Select all

if (!file_exists($proc_file) || $proc_status!='R') {

This code is not yet uploaded in the links i provided above, but on another testserver.
Anyhow, i feel this may be a good way to do a light time controlled execution under PHP.

If you' want, please let me know your thoughts!

[anthony]

Yes but /proc is different for different OS's, and different kernels. The exec() command however should return the exit status of the kill command, and if that is 0 the process was running. That works for ALL UNIX systems, even very horrible old ones.

[redna379]

That's true, but with PHP the only way i know of to check the returned signal is exec or some other execution function.
As i want to terminate the loop immediately in case the process finishes to run before timeout, i would need to do an exec every 0.2 seconds, more or less.
But everything called via exec function will be a new executed process: doing a kill to check status every 0.2 seconds, or even something more, results in tons of "kill" calls. Which is exactly the kind of problem that slowed down everything with my "ps" tests.

I do know kill is faster, but consider some numbers:
- expect an average, for each image creation, of 3 convert calls and 2 secs of overall exec time
(around 0.2 secs the 1st, 1.2 sec the 2nd and 0.6 secs the 3rd)
- expect around 50 images per second, or even more, to be requested by various designer users

This would mean an average of 75 convert calls per second.
I should keep the lowest average exec time as reference for the sleep in the loop, this is why i used 0.2
With kill check done via exec, i would call kill 375 times per second!!
Well, this would really KILL the whole server, in case of high loads (peaks of 80/100 images per second, as i mentioned).

This is true only for PHP, probably. C has internal commands to check processes, i guess.
And this is why i feel the very advantage of using proc as no "375 extra processes" and no disk job.

About /proc being different... you sure for PID?
As far as i know, the /proc/PID/stat file is quite standard: Linux, FreeBSD, Solaris, and AIX all has it.
Am i wrong? (which is always REALLY possible, btw)

[Marcel71]

May i bump this message?

Is there already a good alternative to kill long Imagemagick process?
(Running on Ubuntu and php with exec())

Regards,
Marcel

[magick]

See http://www.imagemagick.org/script/resou ... #configure and reference policy.xml. On our system, we use:

• <policy domain="resource" name="time" value="120"/>

[Marcel71]

*policy.xml (not .php )

Thanks that did the trick...