FPGA Central

I am hoping someone can send me in the right direction for this signa
processing question, or even supply a simple answer. What I have is a
elementary problem in signal processing, but am unfamiliar with how t
approach the problem correctly.

I have a two part question that has been bothering me for some years, an
only recently have I been told that the answer falls within the field o
signal processing. hence I've joined the group. I am hoping you migh
supply me with the two part answer; the answer may simply be to take th
mean of each segment as the best estimate, which is what I'm guessing, bu
first let met state the question.

THE QUESTION:
I have a "desired signal" of fixed length I am interested in. It can b
broken into M linear component pieces. S = {S1, S2, S3, ...SM}
Next, I have N samples of observations of this desired signal. Let us cal
each of them Xi. Each can be decomposed into M linear segments as well.
An example of a desired signal might be the shape of the "average Friday
trading day of a stock index made by 15 minute bars spaced evenl
throughout the day.
I may wish to find what the "average Friday" trading day looks like, th
desired signal, and may have 30,40, 50 etc. (N=number of observations) pas
Fridays I can use to derive this chart.

My FIRST question is how to derive the signal S that looks like the mos
expected, so-to-speak, signal pattern shape. Not necessarily the averag
pattern but "most likely" based on common repetiveness.
Do I simply average all the observations for each piecewise segment, or i
it better to compute a linear combination of observations to derive th
desired signal, or is there yet another mathematical technique possibl
(and what is it, such as maximum likelihood) that would produce the desire
signal? If the goal is to derive the most probable expected signal shape
which might mean eliminating observations that look very different tha
others that are looking very similar to one another, how should I do that
Second, what measure can I produce that would tell me how close the variou
observed signals in the observation set look to one another?

In other words, I can think of two scenarios that might answer th
question, but I don't know if either is best:
1) Take the average of all samples to produce the best estimate fo
section S1 of the desired signal
S1 = ( x1,1 + x2,1 + x3,1 + ... xn,1 )/n for segment 1
S2 = ( x1,2 + x2,2 + x3,2 + ... xn,2 )/n for segment 2
...
SM = ( x1,m + x2,m + x3,m + ... xn,m )/n for segment m

2) Take a linear combination of the samples to produce the best estimat
for the sections S1...SM of the signal. Coefficients a,b,c are the sam
for all equations.
S1 = a* x1,1 + b* x2,1 + c* x3,1 + ... k* xn,1
S2 = a* x1,2 + b* x2,2 + c* x3,2 + ... k* xn,2
...
SM = a* x1,m + b*x2,m + c* x3,m + ... k* xn,m

Is there another method that would provide a better estimate of the signa
shape S that would get rid of outlier observations that really do not loo
like all the others? What methodology would derive the most probabl
looking pattern for this Friday, for instance, of stock market tradin
given independent observations of previous Fridays or observations of othe
data sets expected to be most like Friday's signal pattern?

QUESTION #2
The second question is the following: what measure can I produce tha
would tell me how close the observed signals look like one another? I
other words, is there a measure of fitness I can use if I take som
mathematical formula to compute a desired signal, and that fitness measur
(such as a sum of squared errors) would tell me how close the observation
look like one another?

I can restate the question another way. Given various subsets o
observations, is there some measure I can compute for each group that tell
me the individuals in one set look more like each other than th
individuals in another set? That way I can tell if one set is superior i
predicting the signal shape S since most of its members look more closel
like each other than any other set I can come up with.

For instance, let us take the intraday shape of trading for a stock index
once again to make matters clear. Let's assume that I have various
independent groups of data that I believe may look like this coming
Friday's shape, which is what I want to predict. One group of estimates of
the day's trading pattern might be 30 Fridays. If this Friday is a
pre-holiday, perhaps I have 20 other pre-holidays I can put in a group and
use the answer from #1 to derive an expected signal. Or perhaps Friday is a
Federal reserve announcement day, and I have 40 such instances that can be
put in a group. Or perhaps I have 60 days where the trading bar pattern
shape on Thursday did something special, and since that's what happened so
I expect a similar follow-up of that set for Friday.

Let's say I have multiple different subsets of days I can use to
guess/estimate what tomorrow's trading day will look like, the desired
signal S. If in looking at these various groups I find one where the group
members looked more alike each other than for other groups, that might be
the desired signal prediction for the day in question. It would then be a
question of trying different classification schemes to see which ones came
up with closest fits of member shapes to one another, so-to-speak. My
question is, what one or two mathematical measures can I compute that will
tell me how close the members within a set most looked like one another
(which might mean how closely the derived signal curve best fit the
observations)? Is that possible?

I realize this is not an typical DSP question, and perhaps is too basic,
but it has bothered me for years searching for a way to even describe the
question, and only recently did I find that this falls within the field of
signal processing. Would someone be so kind enough as to provide a possible
answer.

wbodri wrote:
> I am hoping someone can send me in the right direction for this signal
> processing question, or even supply a simple answer. What I have is an
> elementary problem in signal processing, but am unfamiliar with how to
> approach the problem correctly.
>
> I have a two part question that has been bothering me for some years, and
> only recently have I been told that the answer falls within the field of
> signal processing. hence I've joined the group. I am hoping you might
> supply me with the two part answer; the answer may simply be to take the
> mean of each segment as the best estimate, which is what I'm guessing, but
> first let met state the question.
>
> THE QUESTION:
> I have a "desired signal" of fixed length I am interested in. It can be
> broken into M linear component pieces. S = {S1, S2, S3, ...SM}
> Next, I have N samples of observations of this desired signal. Let us call
> each of them Xi. Each can be decomposed into M linear segments as well.
> An example of a desired signal might be the shape of the "average Friday"
> trading day of a stock index made by 15 minute bars spaced evenly
> throughout the day.
> I may wish to find what the "average Friday" trading day looks like, the
> desired signal, and may have 30,40, 50 etc. (N=number of observations) past
> Fridays I can use to derive this chart.
>
> My FIRST question is how to derive the signal S that looks like the most
> expected, so-to-speak, signal pattern shape. Not necessarily the average
> pattern but "most likely" based on common repetiveness.
> Do I simply average all the observations for each piecewise segment, or is
> it better to compute a linear combination of observations to derive the
> desired signal, or is there yet another mathematical technique possible
> (and what is it, such as maximum likelihood) that would produce the desired
> signal? If the goal is to derive the most probable expected signal shape,
> which might mean eliminating observations that look very different than
> others that are looking very similar to one another, how should I do that?
> Second, what measure can I produce that would tell me how close the various
> observed signals in the observation set look to one another?
>
> In other words, I can think of two scenarios that might answer the
> question, but I don't know if either is best:
> 1) Take the average of all samples to produce the best estimate for
> section S1 of the desired signal
> S1 = ( x1,1 + x2,1 + x3,1 + ... xn,1 )/n for segment 1
> S2 = ( x1,2 + x2,2 + x3,2 + ... xn,2 )/n for segment 2
> ..
> SM = ( x1,m + x2,m + x3,m + ... xn,m )/n for segment m
>
> 2) Take a linear combination of the samples to produce the best estimate
> for the sections S1...SM of the signal. Coefficients a,b,c are the same
> for all equations.
> S1 = a* x1,1 + b* x2,1 + c* x3,1 + ... k* xn,1
> S2 = a* x1,2 + b* x2,2 + c* x3,2 + ... k* xn,2
> ..
> SM = a* x1,m + b*x2,m + c* x3,m + ... k* xn,m
>
> Is there another method that would provide a better estimate of the signal
> shape S that would get rid of outlier observations that really do not look
> like all the others? What methodology would derive the most probable
> looking pattern for this Friday, for instance, of stock market trading
> given independent observations of previous Fridays or observations of other
> data sets expected to be most like Friday's signal pattern?
>
> QUESTION #2
> The second question is the following: what measure can I produce that
> would tell me how close the observed signals look like one another? In
> other words, is there a measure of fitness I can use if I take some
> mathematical formula to compute a desired signal, and that fitness measure
> (such as a sum of squared errors) would tell me how close the observations
> look like one another?
>
> I can restate the question another way. Given various subsets of
> observations, is there some measure I can compute for each group that tells
> me the individuals in one set look more like each other than the
> individuals in another set? That way I can tell if one set is superior in
> predicting the signal shape S since most of its members look more closely
> like each other than any other set I can come up with.
>
> For instance, let us take the intraday shape of trading for a stock index
> once again to make matters clear. Let's assume that I have various
> independent groups of data that I believe may look like this coming
> Friday's shape, which is what I want to predict. One group of estimates of
> the day's trading pattern might be 30 Fridays. If this Friday is a
> pre-holiday, perhaps I have 20 other pre-holidays I can put in a group and
> use the answer from #1 to derive an expected signal. Or perhaps Friday is a
> Federal reserve announcement day, and I have 40 such instances that can be
> put in a group. Or perhaps I have 60 days where the trading bar pattern
> shape on Thursday did something special, and since that's what happened so
> I expect a similar follow-up of that set for Friday.
>
> Let's say I have multiple different subsets of days I can use to
> guess/estimate what tomorrow's trading day will look like, the desired
> signal S. If in looking at these various groups I find one where the group
> members looked more alike each other than for other groups, that might be
> the desired signal prediction for the day in question. It would then be a
> question of trying different classification schemes to see which ones came
> up with closest fits of member shapes to one another, so-to-speak. My
> question is, what one or two mathematical measures can I compute that will
> tell me how close the members within a set most looked like one another
> (which might mean how closely the derived signal curve best fit the
> observations)? Is that possible?
>
> I realize this is not an typical DSP question, and perhaps is too basic,
> but it has bothered me for years searching for a way to even describe the
> question, and only recently did I find that this falls within the field of
> signal processing. Would someone be so kind enough as to provide a possible
> answer.
>
Question #1:

Yes, there are various ways that you could take a number of recorded
days of trading and try to find a 'typical' or 'average' day. Some sort
of least-squares best fit on a basis set of functions may be your best
bet (i.e. the Fourier transform uses a basis set of sinusoids -- you may
find that a transform using a different basis set may work better).

Question #2:

If you treat question #1 as an optimization problem, where you're trying
to find the best set of parameters to weigh basis functions, or
otherwise optimizing an estimate, then just use the cost function from
your optimization process.

Question #0:

No, you didn't ask it, but you should have.

If you're _really_ trying to predict stock prices keep in mind that it's
not a field where past behavior is a good predictor of future behavior.
The stock market does what the stock market does, and the whiz kids
are all kids because after they write their books on how to win big on
the stock market it inevitably does something unpredictable. You may be
able to figure out what _was_ a typical day in the past, you may even be
able to figure out what _might_ be a typical day next Friday, but the
next big financial snafu (or volcanic eruption, or hurricane) is going
to blow your 'quality of fit' measure into the toilet.

Question #3:

You didn't ask this one either.

If you're doing this as an academic exercise and someone else is paying
for it -- go for it!

Be careful with staking too much of your own money and reputation on
being able to predict the stock market, though.

--

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

Do you need to implement control loops in software?
"Applied Control Theory for Embedded Systems" gives you just what it says.
See details at http://www.wescottdesign.com/actfes/actfes.html

Tim Wescott <[email protected]> writes:
> [...several stock market analysis suggestions snipped...]

Tim,

Have you ever studied Ito calculus? If so, is it just as unsuccessful
at handling stock market predictions as any other tool?
--
% Randy Yates % "Maybe one day I'll feel her cold embrace,
%% Fuquay-Varina, NC % and kiss her interface,
%%% 919-577-9882 % til then, I'll leave her alone."
%%%% <[email protected]> % 'Yours Truly, 2095', *Time*, ELO
http://www.digitalsignallabs.com

On 19 Mai, 12:49, Randy Yates <ya...@ieee.org> wrote:
> Tim Wescott <t...@seemywebsite.com> writes:
> > [...several stock market analysis suggestions snipped...]
>
> Tim,
>
> Have you ever studied Ito calculus? If so, is it just as unsuccessful
> at handling stock market predictions as any other tool?

Yesterday I saw a newspaper article stating that oil prices
are expected to average $140/barrel in the second half of
2008. The reason for this rise in expected prices (and no, I don't
know what the previous prediction was) is the recent
earthquakes in China over last week or so.

Would your friend Ito have been able to predict that sort
of thing to happen based on the information available two
weeks ago?

Rune

Rune Allnor <[email protected]> writes:
> [...]
> Would your friend Ito have been able to predict that sort
> of thing to happen based on the information available two
> weeks ago?

Do you follow the yellow brick road? Do you believe in
Unicorns and Centaurs? Is Harry Potter your friend?
--
% Randy Yates % "Ticket to the moon, flight leaves here today
%% Fuquay-Varina, NC % from Satellite 2"
%%% 919-577-9882 % 'Ticket To The Moon'
%%%% <[email protected]> % *Time*, Electric Light Orchestra
http://www.digitalsignallabs.com

Randy Yates wrote:
> Rune Allnor <[email protected]> writes:
>> [...]
>> Would your friend Ito have been able to predict that sort
>> of thing to happen based on the information available two
>> weeks ago?
>
> Do you follow the yellow brick road? Do you believe in
> Unicorns and Centaurs? Is Harry Potter your friend?

Lots of toads knew about the quake a couple of days before it happened.
Maybe those were friends on Harry Potter. They sure seem smarter than
humans.

Steve

Steve Underwood <[email protected]> writes:

> Randy Yates wrote:
>> Rune Allnor <[email protected]> writes:
>>> [...]
>>> Would your friend Ito have been able to predict that sort
>>> of thing to happen based on the information available two
>>> weeks ago?
>>
>> Do you follow the yellow brick road? Do you believe in
>> Unicorns and Centaurs? Is Harry Potter your friend?
>
> Lots of toads knew about the quake a couple of days before it
> happened. Maybe those were friends on Harry Potter. They sure seem
> smarter than humans.

It seems I remember reading somewhere about precursory subsonic waves -
perhaps the toads are sensitive to those?

However, I believe Harry Potter was fluent in snake.
--
% Randy Yates % "She has an IQ of 1001, she has a jumpsuit
%% Fuquay-Varina, NC % on, and she's also a telephone."
%%% 919-577-9882 %
%%%% <[email protected]> % 'Yours Truly, 2095', *Time*, ELO
http://www.digitalsignallabs.com

On 19 Mai, 14:32, Randy Yates <ya...@ieee.org> wrote:
> Rune Allnor <all...@tele.ntnu.no> writes:
> > [...]
> > Would your friend Ito have been able to predict that sort
> > of thing to happen based on the information available two
> > weeks ago?
>
> Do you follow the yellow brick road? Do you believe in
> Unicorns and Centaurs? Is Harry Potter your friend?

My very simple point was that some arithmetics that
can not predict major stuff like earthquakes or wars,
can not possibly be used to predict stock market data,
for the very simple reason that earthquakes and wars
happen to influence the stock market.

I know it comes as a surprise to lots of people, but
sme very basic understanding of mere trivial aspects
of the 'data domain' (the processes that generate the
data in question) goes a very long way to determine
what can be done and what can not be done.

Rune

On May 19, 10:38 am, Rune Allnor <all...@tele.ntnu.no> wrote:
> On 19 Mai, 14:32, Randy Yates <ya...@ieee.org> wrote:
>
> > Rune Allnor <all...@tele.ntnu.no> writes:
> > > [...]
> > > Would your friend Ito have been able to predict that sort
> > > of thing to happen based on the information available two
> > > weeks ago?
>
> > Do you follow the yellow brick road? Do you believe in
> > Unicorns and Centaurs? Is Harry Potter your friend?
>
> My very simple point was that some arithmetics that
> can not predict major stuff like earthquakes or wars,
> can not possibly be used to predict stock market data,
> for the very simple reason that earthquakes and wars
> happen to influence the stock market.
>
> I know it comes as a surprise to lots of people, but
> sme very basic understanding of mere trivial aspects
> of the 'data domain' (the processes that generate the
> data in question) goes a very long way to determine
> what can be done and what can not be done.
>
> Rune

An aspect of stock-market analysis using sampled data that puzzles me
is aliasing. How do the pundits apply anti-alias filtering to the raw
data before they sample it?

Daily closing prices, daily river-height measurements, daily high-low
temperature measurements .. these are all informative, but they do not
suffice as fodder for DSP methods that yield daily data. What is the
crossover between statistical and DSP manipulation?

Jerry

On 19 Mai, 16:58, Jerry Avins <j...@ieee.org> wrote:
> On May 19, 10:38 am, Rune Allnor <all...@tele.ntnu.no> wrote:
>
>
>
>
>
> > On 19 Mai, 14:32, Randy Yates <ya...@ieee.org> wrote:
>
> > > Rune Allnor <all...@tele.ntnu.no> writes:
> > > > [...]
> > > > Would your friend Ito have been able to predict that sort
> > > > of thing to happen based on the information available two
> > > > weeks ago?
>
> > > Do you follow the yellow brick road? Do you believe in
> > > Unicorns and Centaurs? Is Harry Potter your friend?
>
> > My very simple point was that some arithmetics that
> > can not predict major stuff like earthquakes or wars,
> > can not possibly be used to predict stock market data,
> > for the very simple reason that earthquakes and wars
> > happen to influence the stock market.
>
> > I know it comes as a surprise to lots of people, but
> > sme very basic understanding of mere trivial aspects
> > of the 'data domain' (the processes that generate the
> > data in question) goes a very long way to determine
> > what can be done and what can not be done.
>
> > Rune
>
> An aspect of stock-market analysis using sampled data that puzzles me
> is aliasing. How do the pundits apply anti-alias filtering to the raw
> data before they sample it?

Well stock market data are discrete time, not continuous. Even if
there
are huge amounts of transactions going on, the number is finite. So
stock market data ader discrete in nature. No anti-alias filter
required.

> Daily closing prices, daily river-height measurements, daily high-low
> temperature measurements .. these are all informative, but they do not
> suffice as fodder for DSP methods that yield daily data. What is the
> crossover between statistical and DSP manipulation?

I have found that 'econometrics' seems to be an important application
for fancy data analysis methods. The book on Kalman filters (State
space
analysis of time series) use economical data for case studies.

Rune

Rune Allnor <[email protected]> writes:

> On 19 Mai, 14:32, Randy Yates <ya...@ieee.org> wrote:
>> Rune Allnor <all...@tele.ntnu.no> writes:
>> > [...]
>> > Would your friend Ito have been able to predict that sort
>> > of thing to happen based on the information available two
>> > weeks ago?
>>
>> Do you follow the yellow brick road? Do you believe in
>> Unicorns and Centaurs? Is Harry Potter your friend?
>
> My very simple point was that some arithmetics that
> can not predict major stuff like earthquakes or wars,
> can not possibly be used to predict stock market data,
> for the very simple reason that earthquakes and wars
> happen to influence the stock market.

There is nothing between perfect prediction and having
no knowledge?
--
% Randy Yates % "She tells me that she likes me very much,
%% Fuquay-Varina, NC % but when I try to touch, she makes it
%%% 919-577-9882 % all too clear."
%%%% <[email protected]> % 'Yours Truly, 2095', *Time*, ELO
http://www.digitalsignallabs.com

On Mon, 19 May 2008 07:38:44 -0700 (PDT), Rune Allnor
<[email protected]> wrote:

>On 19 Mai, 14:32, Randy Yates <ya...@ieee.org> wrote:
>> Rune Allnor <all...@tele.ntnu.no> writes:
>> > [...]
>> > Would your friend Ito have been able to predict that sort
>> > of thing to happen based on the information available two
>> > weeks ago?
>>
>> Do you follow the yellow brick road? Do you believe in
>> Unicorns and Centaurs? Is Harry Potter your friend?
>
>My very simple point was that some arithmetics that
>can not predict major stuff like earthquakes or wars,
>can not possibly be used to predict stock market data,
>for the very simple reason that earthquakes and wars
>happen to influence the stock market.
>
>I know it comes as a surprise to lots of people, but
>sme very basic understanding of mere trivial aspects
>of the 'data domain' (the processes that generate the
>data in question) goes a very long way to determine
>what can be done and what can not be done.
>
>Rune

FWIW, IMHO, that's one of the things that Andor's excellent blog on
negative group delay showed:

http://www.dsprelated.com/showarticle/54.php

specficially, that predictive systems are still causal. Even if the
underlying process is bandlimited enough to lend itself well to
prediction, a sudden event can come along and remove all the
predictability.

Since this doesn't seem to work on the stock market it suggests that
events like earthquakes and board indictments effect the price quickly
enough that trying to predict future prices on technical data alone
can't be all that reliable. The amount of volatility in stock prices
over the last several years even suggests that events that are really
unrelated (but affect the price due to paranoia or whatever reason)
make reliable price prediction even less feasible.

Eric Jacobsen
Minister of Algorithms
Abineau Communications
http://www.ericjacobsen.org

Blog: http://www.dsprelated.com/blogs-1/hf/Eric_Jacobsen.php

On 19 Mai, 17:49, Randy Yates <ya...@ieee.org> wrote:
> Rune Allnor <all...@tele.ntnu.no> writes:
> > On 19 Mai, 14:32, Randy Yates <ya...@ieee.org> wrote:
> >> Rune Allnor <all...@tele.ntnu.no> writes:
> >> > [...]
> >> > Would your friend Ito have been able to predict that sort
> >> > of thing to happen based on the information available two
> >> > weeks ago?
>
> >> Do you follow the yellow brick road? Do you believe in
> >> Unicorns and Centaurs? Is Harry Potter your friend?
>
> > My very simple point was that some arithmetics that
> > can not predict major stuff like earthquakes or wars,
> > can not possibly be used to predict stock market data,
> > for the very simple reason that earthquakes and wars
> > happen to influence the stock market.
>
> There is nothing between perfect prediction and having
> no knowledge?

Yes and no.

General statements like 'there will be a boom', 'there
will be a recession', 'some company stocks will rise'
or 'some company stocks will fall' are obviously withing
that realm, but such statements are tautologies and
hardly useful when trying to play the market.

The question is whether it is possible to come up with
precise statements like which stocks will rise and
which will fall, when the movements will take place
and so on. Those factors are governed by unrelated
events as well as personal and psychological factors
among the people who trade the stocks.

Hardly stuff that can be easily explained by maths.

Rune

On May 19, 11:03 am, Rune Allnor <all...@tele.ntnu.no> wrote:
> On 19 Mai, 16:58, Jerry Avins <j...@ieee.org> wrote:

...

> > An aspect of stock-market analysis using sampled data that puzzles me
> > is aliasing. How do the pundits apply anti-alias filtering to the raw
> > data before they sample it?
>
> Well stock market data are discrete time, not continuous. Even if
> there
> are huge amounts of transactions going on, the number is finite. So
> stock market data ader discrete in nature. No anti-alias filter
> required.

Transactions are discrete, but closing proves are a vert small subset
of all the transactions. Does that change anything?

> > Daily closing prices, daily river-height measurements, daily high-low
> > temperature measurements .. these are all informative, but they do not
> > suffice as fodder for DSP methods that yield daily data. What is the
> > crossover between statistical and DSP manipulation?
>
> I have found that 'econometrics' seems to be an important application
> for fancy data analysis methods. The book on Kalman filters (State
> space
> analysis of time series) use economical data for case studies.

So with econometrics, results are valid ip to Fs, rather than Fs/2? Is
there a theorem about thst?

Jerry

On May 19, 12:03 pm, Jerry Avins <j...@ieee.org> wrote:
> On May 19, 11:03 am, Rune Allnor <all...@tele.ntnu.no> wrote:
>
> > On 19 Mai, 16:58, Jerry Avins <j...@ieee.org> wrote:
>
> ...
>
> > > An aspect of stock-market analysis using sampled data that puzzles me
> > > is aliasing. How do the pundits apply anti-alias filtering to the raw
> > > data before they sample it?
>
> > Well stock market data are discrete time, not continuous. Even if
> > there
> > are huge amounts of transactions going on, the number is finite. So
> > stock market data ader discrete in nature. No anti-alias filter
> > required.
>
> Transactions are discrete, but closing proves are a vert small subset
> of all the transactions. Does that change anything?
>
> > > Daily closing prices, daily river-height measurements, daily high-low
> > > temperature measurements .. these are all informative, but they do not
> > > suffice as fodder for DSP methods that yield daily data. What is the
> > > crossover between statistical and DSP manipulation?
>
> > I have found that 'econometrics' seems to be an important application
> > for fancy data analysis methods. The book on Kalman filters (State
> > space
> > analysis of time series) use economical data for case studies.
>

> So with econometrics, results are valid ip to Fs, rather than Fs/2? Is
> there a theorem about thst?
>
> Jerry

I don't recall any claim of validity. Only that the absence of a
continuous domain preceding a sampling to a discrete domain prevents a
requirement for anti-aliasing in a continuous domain. There may well
be reasons to be cautious about interpreting such data, reasons that
are often ignored.

Dale B. Dalrymple
http://dbdimages.com

wbodri wrote:
>
> I may wish to find what the "average Friday" trading day looks like, the
> desired signal, and may have 30,40, 50 etc. (N=number of observations) past
> Fridays I can use to derive this chart.
>

He made multiple references to "shape" of data.
His only reference to stock market was to *one* of *FIVE* days in a
_TYPICAL_ trading week.

You folks kept reading things into his statements while apparently
ignoring his question.

Since you are so Wall Street oriented.
How do you know that the market is a perfect random sequence without
even analyzing the data. Maybe there is an an intrinsic market behavior
of Fridays which reappears after impulses of earthquakes or tsunamis
have passed.

He asked a math question, and got epistemology instead.
I know the feeling well.

Rant fades

On May 19, 4:59 pm, Richard Owlett <rowl...@atlascomm.net> wrote:
> wbodri wrote:
>

> > ...

>
> He made multiple references to "shape" of data.
> His only reference to stock market was to *one* of *FIVE* days in a
> _TYPICAL_ trading week.

Actually, he made market references in paragraphs 3, 7, 10, and 11.

>
> You folks kept reading things into his statements while apparently
> ignoring his question.

The question received a technical response within two hours. If the OP
has not found that response useful or applicable, I hope he will
provide a clarification. The OP's language was rather fuzzy and the
way we usually work around that is to take a shot, as Tom did, and
rely on the OP to give us feedback on whether his intent was
understood or not.

>
> Since you are so Wall Street oriented.
> How do you know that the market is a perfect random sequence without
> even analyzing the data. Maybe there is an an intrinsic market behavior
> of Fridays which reappears after impulses of earthquakes or tsunamis
> have passed.
>

See, you are doing it, too.

> He asked a math question, and got epistemology instead.

He asked a math question and got an answer, until the OP chooses to
reply about whether he thinks he has been understood or not in that
answer, we'll pass the time.

> I know the feeling well.

If the OP chooses not to help us understand his question better,
perhaps you do.

> Rant fades

Dale B. Dalrymple

On 20 Mai, 03:44, dbd <d...@ieee.org> wrote:
> On May 19, 4:59 pm, Richard Owlett <rowl...@atlascomm.net> wrote:
> > You folks kept reading things into his statements while apparently
> > ignoring his question.
>
> The question received a technical response within two hours.
>
> > He asked a math question, and got epistemology instead.
>
> He asked a math question and got an answer,

Since my posts might contain the offending views:

Tim Wescott's reply is, as far as I am concerned, the
best answer the OP can expect to his questions. I don't
know if there exists such a thing as a FAQ on DSP applied
to stock market data; Tim's post ought to be included in
such a FAQ.

My post on the connection between technicalities and the
nature of the data was not aimed at the OP but at Randy,
who asked if a different technical approach would solve
the fundamental problems pointed out by Tim.

Rune

On May 19, 10:20 pm, Rune Allnor <all...@tele.ntnu.no> wrote:

...

> Since my posts might contain the offending views:
>
> Tim Wescott's reply is, as far as I am concerned, the
> best answer the OP can expect to his questions. I don't
> know if there exists such a thing as a FAQ on DSP applied
> to stock market data; Tim's post ought to be included in
> such a FAQ.
>
> My post on the connection between technicalities and the
> nature of the data was not aimed at the OP but at Randy,
> who asked if a different technical approach would solve
> the fundamental problems pointed out by Tim.

I think Randy's tougue was making a big bulge in his cheek.

Jerry

wbodri wrote:
(snip)

> THE QUESTION:
> I have a "desired signal" of fixed length I am interested in. It can be
> broken into M linear component pieces. S = {S1, S2, S3, ...SM}
> Next, I have N samples of observations of this desired signal. Let us call
> each of them Xi. Each can be decomposed into M linear segments as well.
> An example of a desired signal might be the shape of the "average Friday"
> trading day of a stock index made by 15 minute bars spaced evenly
> throughout the day.
> I may wish to find what the "average Friday" trading day looks like, the
> desired signal, and may have 30,40, 50 etc. (N=number of observations) past
> Fridays I can use to derive this chart.

My first question in answering this is, what part of the signal
are you NOT interested in? It might be that you want the shape
of the change during the day, but independent of the actual value.

If, for example, it goes up from 10 to 12 the shape is the same
as going from 100 to 120 though the values are different.
In that case, scale each by dividing by the geometric mean.
(Arithmetic mean might be close enough. I note that stock
trends are often graphed as semilog (log in Y axis) form.

So, you might divide by the geometric mean of each day and
then do either geometric or arithmetic mean between different
fridays of the corresponding observations.

-- glen

On 20 Mai, 06:21, Jerry Avins <j...@ieee.org> wrote:
> On May 19, 10:20 pm, Rune Allnor <all...@tele.ntnu.no> wrote:
>
> * ...

> > My post on the connection between technicalities and the
> > nature of the data was not aimed at the OP but at Randy,
> > who asked if a different technical approach would solve
> > the fundamental problems pointed out by Tim.
>
> I think Randy's tougue was making a big bulge in his cheek.

I'm merely using an old-style text interface to comp.dsp,
so I wouldn't know (or even suspect) if he did.

Rune

Jerry Avins <[email protected]> writes:

> On May 19, 10:20 pm, Rune Allnor <all...@tele.ntnu.no> wrote:
>
> ...
>
>> Since my posts might contain the offending views:
>>
>> Tim Wescott's reply is, as far as I am concerned, the
>> best answer the OP can expect to his questions. I don't
>> know if there exists such a thing as a FAQ on DSP applied
>> to stock market data; Tim's post ought to be included in
>> such a FAQ.
>>
>> My post on the connection between technicalities and the
>> nature of the data was not aimed at the OP but at Randy,
>> who asked if a different technical approach would solve
>> the fundamental problems pointed out by Tim.
>
> I think Randy's tougue was making a big bulge in his cheek.

It was not.

Maybe I phrased the question poorly. Of course no algorithm can predict
the future. However, as I intimated in one of my responses to Rune, I
believe (purely as a "gut" feeling) that you can glean some information
from stock prices using mathematical techniques. I was curious if Tim
(or anyone else here) had any experience using Ito calculus, which
I have only heard of and have always thought of as some mysterious, cool
math, as such a mathematical technique and whether or not it was
actually better than other more common DSP techniques (e.g., spectral
analysis, weighted averaging, etc.).
--
% Randy Yates % "How's life on earth?
%% Fuquay-Varina, NC % ... What is it worth?"
%%% 919-577-9882 % 'Mission (A World Record)',
%%%% <[email protected]> % *A New World Record*, ELO
http://www.digitalsignallabs.com

Randy Yates wrote:
> Steve Underwood <[email protected]> writes:
>
>> Randy Yates wrote:
>>> Rune Allnor <[email protected]> writes:
>>>> [...]
>>>> Would your friend Ito have been able to predict that sort
>>>> of thing to happen based on the information available two
>>>> weeks ago?
>>> Do you follow the yellow brick road? Do you believe in
>>> Unicorns and Centaurs? Is Harry Potter your friend?
>> Lots of toads knew about the quake a couple of days before it
>> happened. Maybe those were friends on Harry Potter. They sure seem
>> smarter than humans.
>
> It seems I remember reading somewhere about precursory subsonic waves -
> perhaps the toads are sensitive to those?

Apparently seismic monitoring stations are not :-\

> However, I believe Harry Potter was fluent in snake.

He's training to be a politician, I guess.

Steve

Rune Allnor wrote:

(snip)

> Well stock market data are discrete time, not continuous.
> Even if there are huge amounts of transactions going on,
> the number is finite. So stock market data ader discrete
> in nature. No anti-alias filter required.

The price of a stock is a continuous function of time
with a discontiguous derivative. Assuming time is
a continuous variable (there are still some quantum
mechanics questions here), a stock price can change at
any time.

-- glen

Randy Yates wrote:
> Tim Wescott <[email protected]> writes:
>> [...several stock market analysis suggestions snipped...]
>
> Tim,
>
> Have you ever studied Ito calculus? If so, is it just as unsuccessful
> at handling stock market predictions as any other tool?

(sorry about taking so long to answer the question)

No, I haven't. I'm just an engineer, grown cynical in mid-life (but
based on my cynicism, I doubt that Ito calculus is any better).

I probably put too much weight on how problematic it can be to predict
the stock market from past behavior. It is, but if your goal is to
glean information about broader issues from the stock market, or just to
tune up some automatic trading software, then it makes sense to analyze
trends.

My (not too well informed -- I'm an engineer, not a financier)
impression is that instead of looking for blind trends to guide your
trading you should look at companies' worth, and by stock in the worthy
ones.

--

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

Do you need to implement control loops in software?
"Applied Control Theory for Embedded Systems" gives you just what it says.
See details at http://www.wescottdesign.com/actfes/actfes.html