Re: [Jmagick] Testing JMagick

>
> I have been through this a number of times before, having written the
>  test suites for ImageMagick, GraphicsMagick, and Magick++.  When
>  comparing output images against a reference image, you really need to
>  do a statisical comparison rather than absolute.

Yes. If you look in MagickTesttools line 87 you wil se that that's
exactly what is being done:
(http://jmagick.svn.sourceforge.net/viewvc/jmagick/test/magicktest/MagickTesttools.java?view=markup)

		String cmd = "compare -metric MSE "+file1+" "+file2+"
tempImageDifference.jpg";


So 'compare -metric MSE.correctFile.jpg actualFile.jpg' is being used
to determine the difference.

I have used this technique the last year and my experience is this:
Usually differences are under 1 on the test image and images supposed
to be identical
but I have set a "fudge factor" margin  of 20 for the maximal
statistical derivation.

When images differ becaurse of algorithmic errors, they usually do
this with aprox 100  or more (for example input pics.jpg and
sharpened.jpg differs by 77, convolved.jpg by 70 and quantized.png by
943).


> Otherwise
>  differences between Q8/Q16/Q32 will cause test failure.

I dont think so, for the above reasons. But all has been done with Q8
only, I still needs someone to try on Q16.


> Formats like
>  JPEG are lossy so they require a large "fudge factor".

Yes. Therefore the standard "fudge factor" margin is set to 20
(usually a margin of 2 is really enough).


> I think that
>  reference images should be generated using a Q8 build since they
>  represent the maximum variation from the "one true image" and the
>  files are likely to be smaller.

I agree and that's also how Ive done it.


>  Dealing with fonts is a major problem
>  since each version of FreeType produces different results, and glyph
>  hinting (FreeType compile-time options) can make a huge difference in
>  the output.  There are similar problems with PDF and Postscript.

I agree and that's also how Ive done it:
Some tests are dependent on the fonts installed on the system and will
thus probably fail. These have been '_failMightBeOk' in their name (to
not upset you).



-- 
Jacob Nordfalk
http://javabog.dk
Skype: jacobnordfalk
_______________________________________________
JMagick mailing list
[JMagick@masked-domain]
http://www.yeo.id.au/mailman/listinfo/jmagick
Please ensure that [JMagick@masked-domain] is a recipient when replying.

Other messages in this topic