FPGA Central - World's 1st FPGA / CPLD Portal

FPGA Central

World's 1st FPGA Portal

 

Go Back   FPGA Groups > NewsGroup > DSP

DSP comp.dsp newsgroup, mailing list

Reply
 
LinkBack Thread Tools Display Modes
  #1 (permalink)  
Old 08-16-2003, 06:41 PM
walala
Guest
 
Posts: n/a
Default question about fixed-integer precision in digital signal processing?

Dear all,

I am studying 2D DCT for an image. The 2D DCT can be summarized as computing
Y=T*X*T', where X is the input matrix, T is the 2D DCT basis matrix, Y is
the output.

X is integer 0-255. T is real-valued, 0 - 1.

In order to convert real-valued T to integers, I multiply T with 256(left
shift), and round it to integer, and use these new integer values in my
computing.

Now Y will be larger than its correct value by a factor of 256*256, so I
will divide it by 65536(right shift the result by 16 bits to get correct
result).

My question is, is my approach correct?

How many bits I need to allocate for internal matrix computation and the
output?

Is there any better approach to save some datawidth?

In general, are there any resources to study for dealing with
real-to-integer conversion?

Thanks a lot,

-Walala


Reply With Quote
  #2 (permalink)  
Old 08-18-2003, 05:32 PM
walala
Guest
 
Posts: n/a
Default Re: question about fixed-integer precision in digital signal processing?

Hi Vic,

> by prepared T you convert to int as T'=(int)(T+0.5).


T is a matrix with element in [0, 1]... so do you mean T'=round(T*2^b),
where 2^b is a scaling factor? But it is unclear to me how to choose this
scaling factor optimally?

> Y1=(T*X+1)>>1.


The above expression is quite unfamiliar to me... why +1 then >>1?

> Y=Y1*T will have 15 bit exactitude, scaler 128 in mind.


If you choose b=7, scaling 128, then the final result will be scaling
128*128=16384(14 bits), right?

Sorry I may need more help on this, can you explain to me a little more in
details?

Thanks a lot,

-Walala


Reply With Quote
  #3 (permalink)  
Old 08-19-2003, 04:56 PM
Vic
Guest
 
Posts: n/a
Default Re: question about fixed-integer precision in digital signal processing?

Hi Walala,
i answered fastly but with errors as see now.
> > by prepared T you convert to int as T'=(int)(T+0.5).

sorry my negligance: T'=(int)(T>>8+0.5) .

> > Y1=(T*X+1)>>1.

> The above expression is quite unfamiliar to me... why +1 then >>1?
> '+1' is round up for fixed number by /2. There is because my CCS bad work with unsigned numbers, for next round up use bit 14.


> > Y=Y1*T will have 15 bit exactitude, scaler 128 in mind.

Must be Y=(Y1*T+0x4000)>>8;

> If you choose b=7, scaling 128, then the final result will be scaling
> 128*128=16384(14 bits), right?

As you see, b=8 for T & b=7 for result.
Sorry my bad hurry.

Cheers
Reply With Quote
  #4 (permalink)  
Old 08-19-2003, 05:03 PM
Vic
Guest
 
Posts: n/a
Default Re: question about fixed-integer precision in digital signal processing?

Again bad hurry: shift rezult only >>8, thus Y=(T*X*T+0x100)>>9
Shame for me
Reply With Quote
  #5 (permalink)  
Old 08-20-2003, 01:33 PM
Vic
Guest
 
Posts: n/a
Default Re: question about fixed-integer precision in digital signal processing?

Walala & all have read above: i am sorry & never shall write after birthday's wine.
Reply With Quote
Reply

Bookmarks

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Linkedin Group for DSP - Digital Signal Processing Vikram FPGA 1 07-22-2008 12:36 AM
Precision Synthesis verilog netlist black box question [email protected] FPGA 0 06-18-2008 03:32 PM
Re: precision errors. microblaze vs matlab single precision... huh? chriskoh FPGA 5 09-13-2007 05:18 AM
precision errors. microblaze vs matlab single precision... huh? chriskoh FPGA 0 09-12-2007 12:27 PM
which graduate schools and PhD programs are good for VLSI Digital Signal Processing & IC Design? walala DSP 0 08-16-2003 06:32 PM


All times are GMT +1. The time now is 03:30 AM.


Powered by vBulletin® Version 3.8.0
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.
Search Engine Friendly URLs by vBSEO 3.2.0
Copyright 2008 @ FPGA Central. All rights reserved