Writing Black&White 1bit/pixel image file in C, _-FAST-_?

Questions and postings pertaining to the usage of ImageMagick regardless of the interface. This includes the command-line utilities, as well as the C and C++ APIs. Usage questions are like "How do I use ImageMagick to create drop shadows?".
Post Reply
BrianP007
Posts: 49
Joined: 2013-12-13T09:54:14-07:00
Authentication code: 6789

Writing Black&White 1bit/pixel image file in C, _-FAST-_?

Post by BrianP007 » 2016-02-17T12:11:06-07:00

I need to covert ~36 million bits in C to a "standard" png/jpg/gif/mng/??? file format FAST.
Would appreciate recommendation on suitable C library.
One goal is to create a movie with 1025 frames: Pure image and then 1024 with a single bin masked

DETAILS:
=======
libgd is used to create 1 bit/pixel lossless black&white (actually BLACK&TRANSPARENT) masks.

o Profiling shows 43% of program time is spent writing 3 in-memory, gdi images to disk
o >open binary file (PingFilePtr) for output
o gdImagePng(gdi, pfp); fflush(pfp); fclose(pfp); destroy gdi
o 3 gdi->PNG files, ~20kb each

9% of program time to read 217 MB, 36MPix raw rgb to SSD so disk is not the bottleneck
Output file size has near zero importance, data are already <2% of RAW image size and highly compressible.

Will be making many thousands to create movies, probably with FFMPEG
Feeding a raw, bit stream to FFMPEG library function linked in to the C program would be the Holy Grail for the video!
Video will show some full-color image with various masks obscuring parts of the image on successive frames

REQUIREMENTS: <br>
==============
  • C language, Linux/64, currently GCC (may try Intel), (without Byzantine dependencies :)
    Speed is Paramount, file size is of little/no concern
    MUST support transparency for mask creation; black -> masked, rest (most) is Clear
    SIMD/SSE/SMP aware, processor is Skylake 6700K
    Format should be widely readable (FFMPEG, photoshop, Perl Data Language, Firefox/chrome)
    Should have very low loss, but not necessarily lossless
    Short of downloading/building/performance testing umpteen different packages, recommendations or horror stories would be of great interest.
  • JPEG-8/9? libjpeg-turbo? TurboJPEG? JPEG 2000?
    GIF? MNG? PNG?
    BNG (Better Net Graphics by Fabrice Bellard look great vs JPEG but only supports 8-14 bits/channel)
    WebP - optimized for size, ~4x processing time. Not suitable
    ImageMagick? GraphicsMagick? OpenCV? OpenGL? GIMP?

snibgo
Posts: 12001
Joined: 2010-01-23T23:01:33-07:00
Authentication code: 1151
Location: England, UK

Re: Writing Black&White 1bit/pixel image file in C, _-FAST-_?

Post by snibgo » 2016-02-17T12:32:07-07:00

Writing to disk is the usual bottleneck, so smaller files are faster than larger files, even if the CPU effort is greater. I've never tested GIF for speed. The fastest portable format is usually JPG, with a low quality setting, eg 10.

Or get a SSD disk.

ffmpeg happily takes input from a pipe. If your o/s does piping without using disk, that could work well.

In other words: don't worry about the C library. Worry about avoiding disk. Only then, worry about the C. The gains you'll get from the quickest library are trivial compared to disk versus no-disk.
snibgo's IM pages: im.snibgo.com

BrianP007
Posts: 49
Joined: 2013-12-13T09:54:14-07:00
Authentication code: 6789

Re: Writing Black&White 1bit/pixel image file in C, _-FAST-_?

Post by BrianP007 » 2016-02-17T13:21:26-07:00

>> Writing to disk is the usual bottleneck
There is no disk. The SSD is M.2, PCI-E 3.0 x4 SSD. All memory, all the time. 1,500,000,000 BYTES/second write speed

My log indicates I am spending 43% time ENCODING 4.5 MBytes in libGD and writing 60 Kilo-BYTES to SSD.
That's 4 times LONGER than reading a 217MB .RAW from the same disk. It's NOT the DISK.

>> worry about the C.
OK!

snibgo
Posts: 12001
Joined: 2010-01-23T23:01:33-07:00
Authentication code: 1151
Location: England, UK

Re: Writing Black&White 1bit/pixel image file in C, _-FAST-_?

Post by snibgo » 2016-02-17T13:29:32-07:00

Ah, sorry, I missed that. Sorry, I don't have comparison timings for different libraries.
snibgo's IM pages: im.snibgo.com

Post Reply