FPGA Central

Hi friends !

Iam trying to project the performance of our video processing algorith
objectively in terms of PSNR and SSIM. We are dealing with YV12 videos. Ia
using the MSU Video Quality measurement tool to compute the PSNR and SSI
values. Here they give the options to compute the values for Y-compone
alone (and average for all the Y frames), U componet alone (and average fo
all the U frames),V componet alone (and average for all the V frames.

My question here is which value should I take into consideration (Y, U o
V?? or the average of Y,U,V values??) for comparing with unprocessed video



to summarize,

what will be correct PSNR for a YUV video - Y component PSNR, U componen
PSNR or V component PSNR or the average of the three?

Pls give ur valuable inputs..

Thanks in advance,
Praveen

On Apr 3, 6:25*am, "praveen.264" <kpk_...@yahoo.co.in> wrote:
> Hi friends !
>
> Iam trying to project the performance of our video processing algorithm
> objectively in terms of PSNR and SSIM. We are dealing with YV12 videos. Iam
> using the MSU Video Quality measurement tool to compute the PSNR and SSIM
> values. Here they give the options to compute the values for Y-componet
> alone (and average for all the Y frames), U componet alone (and average for
> all the U frames),V componet alone (and average for all the V frames.
>
> My question here is which value should I take into consideration (Y, U or
> V?? or the average of Y,U,V values??) for comparing with unprocessed video?
>
> to summarize,
>
> what will be correct PSNR for a YUV video - *Y component PSNR, U component
> PSNR or V component PSNR or the average of the three?
>
> Pls give ur valuable inputs..
>
> Thanks in advance,
> Praveen

There is no one answer that is "correct". I have seen much literature
in the video compression community where the Y PSNR is the primary
metric, and the others are ignored, or assumed to follow fairly
closely. If you wanted conservative results, you could use max of
Y,U,V. But there's certainly no reason you can't average them either.
If on the other hand you're try to compare against some other
published literature, you should do exactly what they do to make the
comparison more fair.

- Kenn

>On Apr 3, 6:25=A0am, "praveen.264" <kpk_...@yahoo.co.in> wrote:
>> Hi friends !
>>
>> Iam trying to project the performance of our video processin
algorithm
>> objectively in terms of PSNR and SSIM. We are dealing with YV12 videos
I=
>am
>> using the MSU Video Quality measurement tool to compute the PSNR an
SSIM
>> values. Here they give the options to compute the values fo
Y-componet
>> alone (and average for all the Y frames), U componet alone (and averag
f=
>or
>> all the U frames),V componet alone (and average for all the V frames.
>>
>> My question here is which value should I take into consideration (Y,
or
>> V?? or the average of Y,U,V values??) for comparing with unprocesse
vide=
>o?
>>
>> to summarize,
>>
>> what will be correct PSNR for a YUV video - =A0Y component PSNR,
compon=
>ent
>> PSNR or V component PSNR or the average of the three?
>>
>> Pls give ur valuable inputs..
>>
>> Thanks in advance,
>> Praveen
>
>There is no one answer that is "correct". I have seen much literature
>in the video compression community where the Y PSNR is the primary
>metric, and the others are ignored, or assumed to follow fairly
>closely. If you wanted conservative results, you could use max of
>Y,U,V. But there's certainly no reason you can't average them either.
>If on the other hand you're try to compare against some other
>published literature, you should do exactly what they do to make the
>comparison more fair.
>
> - Kenn
>

Thanks Kenn for the reply. I have been going through some literatures fo
reference. I could only find the term "Avg. PSNR" or "MSSIM" but no wher
clearly mentioned whether it is of Y, U ,V or avg of YUV. Can u pls let m
know what the standard practice is ? I am a newbie to this stuff.

Thanks

Praveen

>On Apr 3, 6:25=A0am, "praveen.264" <kpk_...@yahoo.co.in> wrote:
>> Hi friends !
>>
>> Iam trying to project the performance of our video processin
algorithm
>> objectively in terms of PSNR and SSIM. We are dealing with YV12 videos
I=
>am
>> using the MSU Video Quality measurement tool to compute the PSNR an
SSIM
>> values. Here they give the options to compute the values fo
Y-componet
>> alone (and average for all the Y frames), U componet alone (and averag
f=
>or
>> all the U frames),V componet alone (and average for all the V frames.
>>
>> My question here is which value should I take into consideration (Y,
or
>> V?? or the average of Y,U,V values??) for comparing with unprocesse
vide=
>o?
>>
>> to summarize,
>>
>> what will be correct PSNR for a YUV video - =A0Y component PSNR,
compon=
>ent
>> PSNR or V component PSNR or the average of the three?
>>
>> Pls give ur valuable inputs..
>>
>> Thanks in advance,
>> Praveen
>
>There is no one answer that is "correct". I have seen much literature
>in the video compression community where the Y PSNR is the primary
>metric, and the others are ignored, or assumed to follow fairly
>closely. If you wanted conservative results, you could use max of
>Y,U,V. But there's certainly no reason you can't average them either.
>If on the other hand you're try to compare against some other
>published literature, you should do exactly what they do to make the
>comparison more fair.
>
> - Kenn
>

Hi Kenn!

I was caught in this dilemma because of the following situation:

Our processing is mainly concentrated on Y component than U and
components. So, obviously as the U and V components are not much affected
the PSNR or SSIM for U and V will be very high compared to the Y component
Consider the follwoing example:

I have to compare our encoder with an existing encoder (MPEG4 or H.26
etc..) which gave the following PSNR values (compared with original file
for Y, U, and V respectively: 49, 53, 54 => Avg of YUV = 52 db (only thi
Avg value being shown in the literature without the individual values)

let the values obtained by our encoder (compared with original file) be
51, 80, 80=> Avg. of YUV = 70.33

now, going by the Avg., can we claim a whopping difference of 18.33 d
while the actual improvement is 2 db for Y-component alone?

On Apr 3, 12:13*pm, "praveen.264" <kpk_...@yahoo.co.in> wrote:
> >On Apr 3, 6:25=A0am, "praveen.264" <kpk_...@yahoo.co.in> wrote:
> >> Hi friends !
>
> >> Iam trying to project the performance of our video processing
> algorithm
> >> objectively in terms of PSNR and SSIM. We are dealing with YV12 videos..
> I=
> >am
> >> using the MSU Video Quality measurement tool to compute the PSNR and
> SSIM
> >> values. Here they give the options to compute the values for
> Y-componet
> >> alone (and average for all the Y frames), U componet alone (and average
> f=
> >or
> >> all the U frames),V componet alone (and average for all the V frames.
>
> >> My question here is which value should I take into consideration (Y, U
> or
> >> V?? or the average of Y,U,V values??) for comparing with unprocessed
> vide=
> >o?
>
> >> to summarize,
>
> >> what will be correct PSNR for a YUV video - =A0Y component PSNR, U
> compon=
> >ent
> >> PSNR or V component PSNR or the average of the three?
>
> >> Pls give ur valuable inputs..
>
> >> Thanks in advance,
> >> Praveen
>
> >There is no one answer that is "correct". I have seen much literature
> >in the video compression community where the Y PSNR is the primary
> >metric, and the others are ignored, or assumed to follow fairly
> >closely. If you wanted conservative results, you could use max of
> >Y,U,V. But there's certainly no reason you can't average them either.
> >If on the other hand you're try to compare against some other
> >published literature, you should do exactly what they do to make the
> >comparison more fair.
>
> > - Kenn
>
> Hi Kenn!
>
> I was caught in this dilemma because of the following situation:
>
> Our processing is mainly concentrated on Y component than U and V
> components. So, obviously as the U and V components are not much affected,
> the PSNR or SSIM for U and V will be very high compared to the Y component.
> Consider the follwoing example:
>
> I have to compare our encoder with an existing encoder (MPEG4 or H.264
> etc..) which gave the following PSNR values (compared with original file)
> for Y, U, and V respectively: 49, 53, 54 => Avg of YUV = 52 db (only this
> Avg value being shown in the literature without the individual values)
>
> let the values obtained by our encoder (compared with original file) be :
> 51, 80, 80=> Avg. of YUV = 70.33
>
> now, going by the Avg., can we claim a whopping difference of 18.33 db
> while the actual improvement is 2 db for Y-component alone?

That's why you *can* average the PSNRs if you want to but it doesn't
necessarily give you what you think it gives you. PSNR is in a log
scale - what you're usually worried about is some measure of visible
distortion. If the Y component looks like garbage, at 20 dB, the whole
picture will still look like garbage even if U and V are clean.
That's what I meant by taking the maximum... although maximum
distortion is really minimum PSNR after you turn it into logarithmic
form.

Everyone who does image metrics that I've seen runs into this problem
but I have yet to see the one, simple, right answer that is
universally accepted and implemented by everyone.

One possibility I'd propose is to treat the YUV as points in a 3-D
vector space, find the norm of the 3-D difference, then take the PSNR
as the normalized log of that sum. But that only works cleanly if you
have co-sited 4:4:4 pixels to begin with, and will undoubtedly still
cause arguments.

And as far as SSIM and derivatives, there seem to be as many different
implementations that give slightly different results as there are
papers. It seems to be more of a family of metrics than a single
one... for that reason I'm leery of doing comparisons to external
results based purely on one scalar number called "SSIM".

- Kenn

>>On Apr 3, 6:25=A0am, "praveen.264" <kpk_...@yahoo.co.in> wrote:
>>> Hi friends !
>>>
>>> Iam trying to project the performance of our video processing
>algorithm
>>> objectively in terms of PSNR and SSIM. We are dealing with YV1
videos.
>I=
>>am
>>> using the MSU Video Quality measurement tool to compute the PSNR and
>SSIM
>>> values. Here they give the options to compute the values for
>Y-componet
>>> alone (and average for all the Y frames), U componet alone (an
average
>f=
>>or
>>> all the U frames),V componet alone (and average for all the V frames.
>>>
>>> My question here is which value should I take into consideration (Y
U
>or
>>> V?? or the average of Y,U,V values??) for comparing with unprocessed
>vide=
>>o?
>>>
>>> to summarize,
>>>
>>> what will be correct PSNR for a YUV video - =A0Y component PSNR, U
>compon=
>>ent
>>> PSNR or V component PSNR or the average of the three?
>>>
>>> Pls give ur valuable inputs..
>>>
>>> Thanks in advance,
>>> Praveen
>>
>>There is no one answer that is "correct". I have seen much literature
>>in the video compression community where the Y PSNR is the primary
>>metric, and the others are ignored, or assumed to follow fairly
>>closely. If you wanted conservative results, you could use max of
>>Y,U,V. But there's certainly no reason you can't average them either.
>>If on the other hand you're try to compare against some other
>>published literature, you should do exactly what they do to make the
>>comparison more fair.
>>
>> - Kenn
>>
>
>Thanks Kenn for the reply. I have been going through some literature
for
>reference. I could only find the term "Avg. PSNR" or "MSSIM" but n
where
>clearly mentioned whether it is of Y, U ,V or avg of YUV. Can u pls le
me
>know what the standard practice is ? I am a newbie to this stuff.
>
>Thanks
>
>Praveen
>
>
>

I am doing some reading on SSIM for my DSP course and here is a good star
on SSIM and also the code to calculate.

This is the web page of the originator (to my knowledge) for thi
algorithm

http://www.ece.uwaterloo.ca/~z70wang/research/ssim/

and if you scroll through you will find the link to download hi
algorithm

http://www.ece.uwaterloo.ca/~z70wang...ch/ssim/ssim.m

[SSIM, qMap] = ssim(original_image, comparison_image );

he also has a non reference algorithm that works pretty nice. Its no
perfect though

good luck!