lec 3 mn theory integration-i statistics

7/30/2019 Lec 3 MN Theory Integration-i Statistics

1/60


2/60


3/60


4/60


5/60


6/60


7/60


8/60


9/60


10/60


11/60


12/60


13/60


14/60


15/60


16/60


17/60


18/60


19/60


20/60


21/60


22/60


23/60


24/60


25/60


26/60


27/60


28/60


29/60


30/60


31/60


32/60


33/60


34/60


35/60


36/60


37/60


38/60


39/60


40/60


41/60


42/60


43/60


44/60

44

Now we can define the integral.

=

=n

iii Ad

1

)(

Since positive measurable functions can be written as theincreasing pointwise limit of simple functions, we define theintegral as

=

d fd f

sup

+ = d f d f fd

Simple Functions: =

=n

i Ai i

1

1

Positive Functions:

For a general measurable function write+ = f f f


45/60

45

This picture says everything!!!!

A

f

This is a set! and theintegral is measuringthe size of this set!

Integrals are likemeasures! Theymeasure the size of aset. We just describethat set by afunction.

Therefore, integralsshould satisfy the

properties of measures.

A

fd How should I think about


46/60

46

Measures and integrals are different descriptions of of the same concept. Namely, size.

Therefore, they should satisfy the same properties!!

This leads us to another important principle...

Lebesgue defined the integral so that this would betrue!


47/60

47

Left Continuous Left Continuous

Bdd Right Cont. Bdd Right Cont.

etc...

Measures are: Integrals are:

A Ai )()(lim A Ai = f f i = fd d f ilim

A Ai


48/60

48

The L p Spaces

A function is in L p(,F, ) if =< fgd g f ,

1 p

E l f N d


49/60

49

Examples of Normed spacesLp spaces

Lp={f:C| (|f|pd )1/p


50/60

50

Examples of Normed spacesLp spaces (Cont.)

The space is very very large.The space is not only important for Kernel methods

but also for the development of Fucntional Analysis aswell as Foundation of Theory of Statistics

When X is finite, , the counting measure and p=2,we have our well-known, Rk

Checking of Existence of Mean of Cauchy Distribution:


51/60

51


Let X be a standard Cauchy variate withprobability density function,


52/60

52

Checking of Existence of Mean of Cauchy Distribution:Lebesgue Integral Way

2

2 2

0 _

2 20

1( ) 1

1 1 11 1

1 1 1

1 1

x xf xdx dx x

x dx x dx x x

x dx x dx x x

+

+

= +

= + +

= + +

Now,

)

2 20 1

2[1, ), 1

1 1

1nn

x x dx dx

x x x

dx x

== +

>+ +

=+

U

put

dz xdx

z x

=

=+

2

1 2

x n+1 1+(n+1) 2

z n 1+n 2( )2

2

1 ( 1)

112

21

1 log2

1 1 ( 1)log2 1

n

x n

n

z

n

n

+ +

+=

=

= + +=

+



53/60

53


Since 22

2

2

21)1(1

log1

)1(1log

n

n

n

n

+++>

+++

And

)(

1

21

1)1(1

loglim

02

2

2

X E

dx x

xn

nn

+

=+

++

does not

existdoes not exist

Therefore mean of the Cauchy distribution doesnot exist



54/60

54

g yRiemann Integral Way

Method (i):We knowthat,

( )2

2

221

142

2

Butlim1

1lim l og

21 1

limlog2 1 41

2( )

a

a a

a

a a

a

x dx

x

z

a

a

EX

+

+

+ =

+=+

=

does notexist.

Since is an oddfunction

21 x

x

+2l im 0;

1

a

aa

x dx x

=+



55/60

55

g yRiemann Integral Way

Method (ii):

20 1

( )

x dx x

EX

+

does not exist byquotient test

does notexist

Therefore, mean of the Cauchy distribution doesnot exist.

2 22

2 2

1 lim 1 01 1 1 x

x x x x

x x

x

+ = = =

+ +


56/60

56

Measure-Probability Measure-CumulativeDistribution Function

Bounded measure Probabilitymeasure

CumulativeDistribution

Function


57/60

57

Broad Categories of Probability Measures

ProbabilityMeasures

Discrete P(A)=1,#(A)=finite or

Continuous

P{x}=0 for all x

AbsolutelyContinous w.r.t.

Lebesgue measureNon A.C.

OnE

u cl i

d e an

s am pl e

s p a c e


58/60

58

A random variable is a measurable function.

)( X

The expectation of a random variable is its integral:

= XdP X E )(A density function is the Radon-Nikodym derivative wrtLebesgue measure:

dxdP f X =

== dx x xf XdP X E X )()(

Expectation

=xi p

i

Counting measure p=dP/d


59/60

59

Expectation

Change of variables. Let f be measurable from (, F , v)to (,) and g be Borel on (,). Then

i.e. , if either integral exists, then so does the other, and thetwo are the same.

Note that integration domains are indicated on bothsides of 1. This result extends the change of variableformula for Reimann integrals, i.e. ,

(1)


60/60

In statistics we will talk about expectations with respectto different measures.

P = XdP X E P )(Q = XdQ X E Q )(

)()( X E dQ X dQdQdP

X XdP X E Q P

==== where =

dQdP

or dQdP =

And write expectations in terms of the different measures:

Expectation

lec 3 mn theory integration-i statistics

Documents