ImageMagick reading Mars Express HRSC image data

Questions and postings pertaining to the development of ImageMagick, feature enhancements, and ImageMagick internals. ImageMagick source code and algorithms are discussed here. Usage questions which are too arcane for the normal user list should also be posted here.
User avatar
magick
Site Admin
Posts: 11018
Joined: 2003-05-31T11:32:55-07:00

Post by magick » 2006-05-13T08:26:36-07:00

You could try:
  • convert vicar:H1293_0000_ND3.IMG H1293_0000_ND3.jp2
If that fails, post a URL to the image so we can download it and investigate the problem.

User avatar
magick
Site Admin
Posts: 11018
Joined: 2003-05-31T11:32:55-07:00

Post by magick » 2006-05-13T10:14:06-07:00

JPEG does have a size limit of 65500 pixels. Not sure about JP2. PNG seems to have a 65535 pixel limit. You could try TIFF with -compress zip. ImageMagick can process images of practically any size as long as you have enough temporary disk space. You can specify where ImageMagick puts its temporary files with the MAGICK_TMPDIR environment variable.

If you post a URL to your image we will try it here and see if we can convert the image without complaint.

User avatar
magick
Site Admin
Posts: 11018
Joined: 2003-05-31T11:32:55-07:00

Post by magick » 2006-05-13T12:53:40-07:00

We converted your image with ImageMagick 6.2.7-6, the current release, under Fedora Core 5 and on a Windows XP 2 system. It was slow to convert but finished without complaint:
  • convert vicar:H1293_0000_ND3.IMG -compress zip image.miff

User avatar
magick
Site Admin
Posts: 11018
Joined: 2003-05-31T11:32:55-07:00

Post by magick » 2006-05-14T09:11:09-07:00

Add -debug cache right after convert on your command line. It will tell you where the pixel cache is being created. Make sure it is being created on a disk with plenty of free space.

User avatar
magick
Site Admin
Posts: 11018
Joined: 2003-05-31T11:32:55-07:00

Post by magick » 2006-05-14T15:26:28-07:00

We're at the point of diminishing returns here. ImageMagick can convert your image to any format supported that also supports greater than 65535 pixels (as you know JPEG and PNG do not support greater than 65535 pixels). If your i: drive has 16GB of free space or more, we're clueless why the conversion is failing for you. We did the conversion on a 2.2GHZ Intel chip with XP2 with 512MB of memory and a c: drive that has 55GB of free space. The conversion worked without complaint using the latest ImageMagick release, 6.2.7-6.
Last edited by magick on 2006-05-15T07:21:38-07:00, edited 1 time in total.

User avatar
magick
Site Admin
Posts: 11018
Joined: 2003-05-31T11:32:55-07:00

Post by magick » 2006-05-16T09:57:55-07:00

Someone mentioned that a FAT filesystem cannot support files greated than 4GB. Make sure you set the path for temporary files to a NTFS filesystem rather than FAT.

User avatar
magick
Site Admin
Posts: 11018
Joined: 2003-05-31T11:32:55-07:00

Post by magick » 2006-05-22T13:40:08-07:00

Grab ImageMagick 6.2.7-7 and try this command: convert logo: -crop 100x100 +repage logo-%d.miff[/list] Do you get 30 some slices of the image? If so, try the command with your image but adjust the crop to 4416x4034.

In regards to the temporary files, see http://magick.imagemagick.org/script/architecture.php.

User avatar
magick
Site Admin
Posts: 11018
Joined: 2003-05-31T11:32:55-07:00

Post by magick » 2006-05-22T18:17:41-07:00

Try something like
  • convert logo: \
    \( -clone 0 -crop 100x100+0+100 +repage -write 1.jpg -delete 0 \) \
    \( -clone 0 -crop 100x100+0+200 +repage -write 2.jpg -delete 0 \) \
    \( -clone 0 -crop 100x100+0+300 +repage -write 3.jpg -delete 0 \) \
    null:

User avatar
magick
Site Admin
Posts: 11018
Joined: 2003-05-31T11:32:55-07:00

Post by magick » 2006-05-23T07:42:55-07:00

To create a cache file, use the MPC format:
  • convert vicar:H1597_0001_ND3.IMG H1597_0001_ND3.mpc
You can now extract portions of the MPC in a most efficient manner. However, your H1597_0001_ND3.cache file will be quite large.

User avatar
magick
Site Admin
Posts: 11018
Joined: 2003-05-31T11:32:55-07:00

Post by magick » 2006-05-23T18:05:27-07:00

Yet another patch is required. Its in ImageMagick 6.2.7-8 available tommorrow. The -extract option is a setting so it carries forward and affects the input image as well as the output image. To reset the extract geometry before the output image, you must use +extract:
  • convert -extract 8832x16135+0+16136 H1597_0001_ND3.mpc +extract H1597_0001_ND3.l2.s1.tif
Unfortunately, a line patch is required to make this work correctly.

User avatar
magick
Site Admin
Posts: 11018
Joined: 2003-05-31T11:32:55-07:00

Post by magick » 2006-05-24T08:39:07-07:00

A line patch is patch to the source distribution that is a single line of code.

User avatar
magick
Site Admin
Posts: 11018
Joined: 2003-05-31T11:32:55-07:00

Post by magick » 2006-05-28T09:22:33-07:00

Try this instead:
  • convert -monitor H1597_0001_ND3.mpc -crop 8832x16136+0+0 +repage H1597_0001_ND3.l1.s1.tif
    convert -monitor H1597_0001_ND3.mpc -crop8832x16135+0+16136 +repage H1597_0001_ND3.l2.s1.tif

User avatar
magick
Site Admin
Posts: 11018
Joined: 2003-05-31T11:32:55-07:00

Post by magick » 2006-05-29T11:21:42-07:00

We process multi-gigabyte images with ImageMagick quite frequently but we have a terabyte of temporary disk space. In your case it would be best to read and write a scanline of the image at a time to minimize resource usage.

User avatar
magick
Site Admin
Posts: 11018
Joined: 2003-05-31T11:32:55-07:00

Post by magick » 2006-05-29T11:34:47-07:00

Sure, but as you know it first decodes the entire image requiring what you consider excessive resource usage. Instead you would need to code up a program yourself.

You might have better luck with the NetPBM programs fitstopnm and pnmtotiff.

If you have any multi-gigabyte images we could download, we would like to verify ImageMagick works properly with your huge images although we suspect it will.

User avatar
magick
Site Admin
Posts: 11018
Joined: 2003-05-31T11:32:55-07:00

Post by magick » 2006-05-30T08:07:21-07:00

Ok, works for us and it only took 3 1/2 hours and 13GB of temporary space:
  • setenv MAGICK_TMPDIR .
    time identify H1337_0000_ND3.IMG
    H1337_0000_ND3.IMG VICAR 10905x113316 10905x113316+0+0 PseudoClass 256c 1.1509gb 138.700u 46:48

    time convert H1337_0000_ND3.IMG -crop 10905x16188 +repage -compress zip H1337_0000_ND3-%d.tif
    367.330u 401.733s 3:29:45.80 6.1% 0+0k 0+0io 702485pf+0w

    -rw------- 1 magick magick 54818590 May 30 03:51 H1337_0000_ND3-0.tif
    -rw------- 1 magick magick 54617858 May 30 03:55 H1337_0000_ND3-1.tif
    -rw------- 1 magick magick 54954838 May 30 03:59 H1337_0000_ND3-2.tif
    -rw------- 1 magick magick 54745032 May 30 04:04 H1337_0000_ND3-3.tif
    -rw------- 1 magick magick 55180960 May 30 04:10 H1337_0000_ND3-4.tif
    -rw------- 1 magick magick 55827498 May 30 04:16 H1337_0000_ND3-5.tif
    -rw------- 1 magick magick 56515944 May 30 04:23 H1337_0000_ND3-6.tif

Post Reply