Export Speed with ImageMagick

Questions and postings pertaining to the usage of ImageMagick regardless of the interface. This includes the command-line utilities, as well as the C and C++ APIs. Usage questions are like "How do I use ImageMagick to create drop shadows?".
Post Reply
User avatar
Seth
Posts: 14
Joined: 2019-05-11T04:32:10-07:00
Authentication code: 1152
Location: Belgium
Contact:

Export Speed with ImageMagick

Post by Seth » 2019-06-20T07:44:54-07:00

I have a repository on GitLab dedicated to emotes, it uses CI to take the original emotes and masks and exports them all in various colors and sizes.

May I ask based on the setup does anyone have a solution that may speed up the build time for these emotes however, even if just by seconds?
The file that actually exports all the images in all sizes and colors:
https://gitlab.com/Elypia/elypia-emotes ... s/build.sh

The original size of emotes are 512px (some are 500px, and some old emotes which are in the process of being redone are 128px).

Currently the repo has 60 emotes, so it does 900 exports in roughly 2.5 minutes. (512, 256, 128)
In the past it was 60 emotes with with more sizes, 3,000 exports in around 4 minutes. (512, 258, 128, 112, 72, 64, 56, 36, 28, 18)

Any input would be handy, whether it's ImageMagick related or not, I just hoping to reduce the overall wait between commit and getting the emotes.
Fun is what matters!

User avatar
fmw42
Posts: 25761
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: Export Speed with ImageMagick

Post by fmw42 » 2019-06-20T07:58:29-07:00

You can speed it up by writing all sizes from one image in one command line rather than re-reading the input for each size. See https://imagemagick.org/Usage/files/#write

User avatar
magick
Site Admin
Posts: 11098
Joined: 2003-05-31T11:32:55-07:00

Re: Export Speed with ImageMagick

Post by magick » 2019-06-20T09:50:24-07:00

By default, ImageMagick supports HDRI which requires floating-point pixel components. If you build ImageMagick without HDRI support (add --disable-HDRI to your configure script command-line), you get unsigned pixels and will likely result in a substantial speed-up.

User avatar
fmw42
Posts: 25761
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: Export Speed with ImageMagick

Post by fmw42 » 2019-06-20T10:56:45-07:00

You could also build a Q8. That might help also.

User avatar
Seth
Posts: 14
Joined: 2019-05-11T04:32:10-07:00
Authentication code: 1152
Location: Belgium
Contact:

Re: Export Speed with ImageMagick

Post by Seth » 2019-06-20T14:48:48-07:00

fmw42 wrote:
2019-06-20T07:58:29-07:00
You can speed it up by writing all sizes from one image in one command line rather than re-reading the input for each size. See https://imagemagick.org/Usage/files/#write
Thank you for the advice, I managed to get this working with sizes only earlier making a huge difference from 2.5 minutes to 1.5 minutes already.
I'm currently trying to make it work colors as well now but am having a slight problem, may I get your advice?

The below is what the script looks like now:

Code: Select all

#!/bin/bash
if [ -d output/ ]; then rm -rf output/*; fi

# Append here for colors (Color | Prefix | Hue)
declare -A colors
colors[0,0]="red";    colors[0,1]="";  colors[0,2]="100,100,100";
colors[1,0]="blue";   colors[1,1]="b"; colors[1,2]="100,100,0";
colors[2,0]="green";  colors[2,1]="g"; colors[2,2]="100,100,166";
colors[3,0]="pink";   colors[3,1]="p"; colors[3,2]="100,100,66.6";
colors[4,0]="yellow"; colors[4,1]="y"; colors[4,2]="100,115,115";

for c in 0 1 2 3 4
do
    export_arg="${export_arg} \( +clone -modulate ${colors[${c},2]} +clip-mask"

    # When adding a new resolution, just append the output size here.
    for size in 512 258 128 112 72 64 56 36 28 18
    do
        mkdir -p output/${colors[${c},0]}/${size}px/
        export_arg="${export_arg} \( +clone -resize ${size}X${size} -write \"output/${colors[${c},0]}/${size}px/${colors[${c},1]}[EMOTE]\" +delete \)"
    done

    export_arg="${export_arg} +delete \)"
done

echo ${export_arg}

for file in emotes/*
do
    title=$(identify -format "%t" ${file})
    emote="${title}.png"
    mask="masks/${title}Mask.png"

    if [ -f ${mask} ]; then
        mask_arg="-clip-mask ${mask}"
    else
        unset mask_arg
    fi

    echo "Exporting: ${file}"

    emote_export_arg=${export_arg//\[EMOTE\]/${emote}}
    echo "magick ${file} -filter Catrom ${mask_arg} ${emote_export_arg} null:"
    magick ${file} -filter Catrom ${mask_arg} ${emote_export_arg} null:
done

Code: Select all

magick emotes/pandaWut.png -filter Catrom   \( +clone -modulate 100,100,100 +clip-mask \( +clone -resize 512X512 -write "output/red/512px/pandaWut.png" +delete \) \( +clone -resize 258X258 -write "output/red/258px/pandaWut.png" +delete \) \( +clone -resize 128X128 -write "output/red/128px/pandaWut.png" +delete \) \( +clone -resize 112X112 -write "output/red/112px/pandaWut.png" +delete \) \( +clone -resize 72X72 -write "output/red/72px/pandaWut.png" +delete \) \( +clone -resize 64X64 -write "output/red/64px/pandaWut.png" +delete \) \( +clone -resize 56X56 -write "output/red/56px/pandaWut.png" +delete \) \( +clone -resize 36X36 -write "output/red/36px/pandaWut.png" +delete \) \( +clone -resize 28X28 -write "output/red/28px/pandaWut.png" +delete \) \( +clone -resize 18X18 -write "output/red/18px/pandaWut.png" +delete \) +delete \) \( +clone -modulate 100,100,0 +clip-mask \( +clone -resize 512X512 -write "output/blue/512px/bpandaWut.png" +delete \) \( +clone -resize 258X258 -write "output/blue/258px/bpandaWut.png" +delete \) \( +clone -resize 128X128 -write "output/blue/128px/bpandaWut.png" +delete \) \( +clone -resize 112X112 -write "output/blue/112px/bpandaWut.png" +delete \) \( +clone -resize 72X72 -write "output/blue/72px/bpandaWut.png" +delete \) \( +clone -resize 64X64 -write "output/blue/64px/bpandaWut.png" +delete \) \( +clone -resize 56X56 -write "output/blue/56px/bpandaWut.png" +delete \) \( +clone -resize 36X36 -write "output/blue/36px/bpandaWut.png" +delete \) \( +clone -resize 28X28 -write "output/blue/28px/bpandaWut.png" +delete \) \( +clone -resize 18X18 -write "output/blue/18px/bpandaWut.png" +delete \) +delete \) \( +clone -modulate 100,100,166 +clip-mask \( +clone -resize 512X512 -write "output/green/512px/gpandaWut.png" +delete \) \( +clone -resize 258X258 -write "output/green/258px/gpandaWut.png" +delete \) \( +clone -resize 128X128 -write "output/green/128px/gpandaWut.png" +delete \) \( +clone -resize 112X112 -write "output/green/112px/gpandaWut.png" +delete \) \( +clone -resize 72X72 -write "output/green/72px/gpandaWut.png" +delete \) \( +clone -resize 64X64 -write "output/green/64px/gpandaWut.png" +delete \) \( +clone -resize 56X56 -write "output/green/56px/gpandaWut.png" +delete \) \( +clone -resize 36X36 -write "output/green/36px/gpandaWut.png" +delete \) \( +clone -resize 28X28 -write "output/green/28px/gpandaWut.png" +delete \) \( +clone -resize 18X18 -write "output/green/18px/gpandaWut.png" +delete \) +delete \) \( +clone -modulate 100,100,66.6 +clip-mask \( +clone -resize 512X512 -write "output/pink/512px/ppandaWut.png" +delete \) \( +clone -resize 258X258 -write "output/pink/258px/ppandaWut.png" +delete \) \( +clone -resize 128X128 -write "output/pink/128px/ppandaWut.png" +delete \) \( +clone -resize 112X112 -write "output/pink/112px/ppandaWut.png" +delete \) \( +clone -resize 72X72 -write "output/pink/72px/ppandaWut.png" +delete \) \( +clone -resize 64X64 -write "output/pink/64px/ppandaWut.png" +delete \) \( +clone -resize 56X56 -write "output/pink/56px/ppandaWut.png" +delete \) \( +clone -resize 36X36 -write "output/pink/36px/ppandaWut.png" +delete \) \( +clone -resize 28X28 -write "output/pink/28px/ppandaWut.png" +delete \) \( +clone -resize 18X18 -write "output/pink/18px/ppandaWut.png" +delete \) +delete \) \( +clone -modulate 100,115,115 +clip-mask \( +clone -resize 512X512 -write "output/yellow/512px/ypandaWut.png" +delete \) \( +clone -resize 258X258 -write "output/yellow/258px/ypandaWut.png" +delete \) \( +clone -resize 128X128 -write "output/yellow/128px/ypandaWut.png" +delete \) \( +clone -resize 112X112 -write "output/yellow/112px/ypandaWut.png" +delete \) \( +clone -resize 72X72 -write "output/yellow/72px/ypandaWut.png" +delete \) \( +clone -resize 64X64 -write "output/yellow/64px/ypandaWut.png" +delete \) \( +clone -resize 56X56 -write "output/yellow/56px/ypandaWut.png" +delete \) \( +clone -resize 36X36 -write "output/yellow/36px/ypandaWut.png" +delete \) \( +clone -resize 28X28 -write "output/yellow/28px/ypandaWut.png" +delete \) \( +clone -resize 18X18 -write "output/yellow/18px/ypandaWut.png" +delete \) +delete \) null:
Quite long, but this is the actual command being performed per emote.

However it keeps returning the error:

Code: Select all

magick: unable to open image '\(': No such file or directory @ error/blob.c/OpenBlob/3497.
magick: no decode delegate for this image format `' @ error/constitute.c/ReadImage/556.
However I'm unsure what exactly the syntax I should be going for is regarding nested clones.
Fun is what matters!

snibgo
Posts: 12432
Joined: 2010-01-23T23:01:33-07:00
Authentication code: 1151
Location: England, UK

Re: Export Speed with ImageMagick

Post by snibgo » 2019-06-20T15:32:58-07:00

Seth wrote:magick: unable to open image '\(':
"magick" should not see the backslash. In normal use it would be stripped by the bash interpreter, but not when it is inside a variable.

I suggest you read how and when bash processes escapes, and expands variables. I think the problem is that you escape the parentheses when setting export_arg, but shouldn't.
snibgo's IM pages: im.snibgo.com

User avatar
Seth
Posts: 14
Joined: 2019-05-11T04:32:10-07:00
Authentication code: 1152
Location: Belgium
Contact:

Re: Export Speed with ImageMagick

Post by Seth » 2019-06-20T15:58:25-07:00

snibgo wrote:
2019-06-20T15:32:58-07:00
Seth wrote:magick: unable to open image '\(':
"magick" should not see the backslash. In normal use it would be stripped by the bash interpreter, but not when it is inside a variable.

I suggest you read how and when bash processes escapes, and expands variables. I think the problem is that you escape the parentheses when setting export_arg, but shouldn't.
Thank you very much! You're right, that was the problem.
It's been addressed now and works perfectly!
Fun is what matters!

Post Reply