Bài giảng Chapter 2: Finite sample properties of the ols estimator

Chia sẻ: Codon_09 Codon_09 | Ngày: | Loại File: PDF | Số trang:0

Thêm vào BST

Báo xấu

40
lượt xem 2
download

Download Vui lòng tải xuống để xem tài liệu đầy đủ

Mời các bạn cùng tìm hiểu unbiased; linearity; efficiency; gauss - markov theorem;... được trình bày cụ thể trong "Bài giảng Chapter 2: Finite sample properties of the ols estimator". Hy vọng tài liệu là nguồn thông tin hữu ích cho quá trình học tập và nghiên cứu của các bạn.

Chủ đề:

Bình luận(0) Đăng nhập để gửi bình luận!

Lưu

Nội dung Text: Bài giảng Chapter 2: Finite sample properties of the ols estimator

Advanced Econometrics Chapter 2: Finite Sample Properties Of The OLS Estimator Chapter 2 FINITE SAMPLE PROPERTIES OF THE OLS ESTIMATOR Y = X.β + ε with ε ~ N [0, σ 2 I ] • rank(X) = k non-stochastic. ε random → Y random. • βˆ = ( X ′X ) −1 X ′Y ; βˆ is a statistics on a sample, βˆ is random because Y is random. Being random: - βˆ has a probability distribution, called the sampling distribution. - Repeatedly draw all possible random sample of size n calculate " βˆ " each time. Let explore some statistical properties of the OLS estimator βˆ & build up its sampling distribution. I. UNBIASED: βˆ = ( X ′X ) −1 X ′Y = ( X ′X ) −1 X ′( Xβ + ε ) = ( ′X ) −1 X ′X β + ( X ′X ) −1 X ′ε X  I = β + ( X ′X ) −1 X ′ε E( βˆ ) = E[ β + ( X ′X ) −1 X ′ε ] Nam T. Hoang University of New England - Australia 1 University of Economics - HCMC - Vietnam
Advanced Econometrics Chapter 2: Finite Sample Properties Of The OLS Estimator = β + E[( X ′X ) −1 X ′ε ] = β + ( X ′X ) −1 X ′ E (ε ) = β 0 ⇒ E ( βˆ ) = β βˆ is an estimator of β, it is a function of the random sample (the element of Y). Note: we talk about the sample → that means we talk about Y only. Because X is a constant - fix matrix. "Repeatedly draw all possible random samples of size n → draw Y". The least squares estimator is unbiased for β (E(ε) = 0, X is non-stochastic). → VarCov( βˆ ) = E[( βˆ − (β E))( β −  ˆ ˆ E ( βˆ ))' ]  βˆ − β = ( X ′X ) −1 X ′ε β β VarCov( βˆ ) = E [( βˆ − β )( βˆ − β )' ] = E[( X ′X ) −1 X ′ε )(( X ′X ) −1 X ′ε )' ] = E [( X ′X ) −1 X ′εε ' X ( X ′X ) −1 ] = ( X ′X ) −1 X ′E (εε ' ) X ( X ′X ) −1 = ( X ′X ) −1 X ′σ ε2 X ( X ′X ) −1 = σ ε2 ( X ′X ) −1 X ′X ( X ′X ) −1  I = σ ε2 ( X ′X ) −1 So: VarCov( βˆ ) = σ ε2 ( X ′X ) −1 For the model: ~ ~ ~ Yi = βˆ2 X i 2 + βˆ3 X i 3 + ei  βˆ2  βˆ =    βˆ3   ∑ X i23 ∑ X ~X  ~ ~ ~ 1 σ ε ( X ′X ) 2 −1 = σ ε  − X~ X~ 2 i2 i3  ∑ i 2 i 3 ∑ X  ∑ X~ 2 i2 2 i2 ~ X i23 − (∑ X~ i2 ~ X i3 ) 2 Nam T. Hoang University of New England - Australia 2 University of Economics - HCMC - Vietnam
Advanced Econometrics Chapter 2: Finite Sample Properties Of The OLS Estimator  βˆ  = VarCov  2   βˆ   3 σ ε2 ∑ X i23 ~ → Var ( βˆ ) = ∑X ~2 ~2 i2 X i3 − (∑ X~ i2 ~ X i3 ) 2 σ ε2 / ∑ X i22 ~ ∑( X = ~ ~ 2 i2 X i3 ) 1− n2 ∑ X i 2 ∑ X i23 ~2 ~ nn   2 r23 sample correlation between X i 2 ; X i 3 σ ε2 → Var ( βˆ ) = ∑X ~2 i2 (1 − r232 ) determined by: i. σ ε2 ↑ → Var ( βˆ ) ↑ ii. r232 ↑ → Var ( βˆ ) ↑ ∑X ~2 iii. Variation in Xi2 i2 ↑ → Var ( βˆ ) ↓ iv. n sample size ↑ → Var ( βˆ ) ↓ VarCov ( βˆ ) = σ ε2 ( X ′X ) −1 → we don't know σ ε2 → need an estimator for σ ε2 . e' e Define: σˆ ε2 = n−k n: observations. k: number of estimators. e' e = ∑ ei2 = sum of squares. • Show σˆ ε2 is an unbiased estimator. e = Mε → e'e = ε'M'Mε=ε'Mε • Note: trace of a square matrix. Nam T. Hoang University of New England - Australia 3 University of Economics - HCMC - Vietnam
Advanced Econometrics Chapter 2: Finite Sample Properties Of The OLS Estimator n A is the sum of its principal diagonal elements (= n ×n ∑a i =1 ii ). Rules: A, B nxn matrix tr(A+B) = tr(A) + tr(B) tr(A.B) = tr(B.A) tr(λA) = λtr(A) Trace is a linear operation → sum of certain elements. E ( e' e ) = E (ε ' Mε ) = E[tr (ε ' Mε )] = E[tr (εε ' M )] = trE (ε ' Mε ) = tr[σ ε2 . I .M )] = σ ε2 tr ( M ) = σ ε2 [tr ( I n ) − tr ( X ( X ' X ) −1 X ' )] = σ ε2 [n − tr ( X ( X ' X ) −1 X ')] = σ ε2 ( n − k )    I k ×k E ( e' e) σ ε2 ( n − k ) And: = = σ ε2 n−k n−k So: E (σˆ ε2 ) = σ ε2 → σˆ ε2 is an unbiased estimator of σ ε2 . II. LINEARITY: Any estimator that is a linear function of the random sample data is called a linear estimator. Yi: random sample data. β  = ( ˆ X′X ) −1 X ′Y =   A . Y k ×1 A k × n n ×1 where A is non-random: Nam T. Hoang University of New England - Australia 4 University of Economics - HCMC - Vietnam
Advanced Econometrics Chapter 2: Finite Sample Properties Of The OLS Estimator  βˆ1   a11 a12  a1n  Y1     βˆ2  a a 22  a 2 n  Y2   21     =               βˆk  a k 1 X k2  a kn  Yn    → βˆ1 = a11Y1 + a12Y2 + ... + a1nYk 1 → βˆ , OLS estimator is linear and unbiased for β. Because βˆ is a linear function of Y and Y is a linear function of ε, → if ε is normal then βˆ is normal. So the sampling distribution of the OLS estimator of β is: βˆ ~ N[β, σ ε2 ( X ′X ) −1 ] III. EFFICIENCY: Suppose we have 2 unbiased estimators, θˆ1 ; θˆ2 for θ . Then we say θˆ1 is more efficient than θˆ2 if Var (θˆ1 ) ≤ Var (θˆ2 ) . If θˆ1 ; θˆ2 are vectors unbiased estimators of θ , then θˆ1 is more efficient than θˆ2 if   k ×1 k ×1 k ×1 ∆ = [V (θˆ1 ) − V (θˆ2 )] is positive semi-definite. IV. GAUSS - MARKOV THEOREM: "Under the assumptions of the classical regression model, the least squares estimators of β, βˆ = ( X ′X ) −1 X ′Y are the best linear unbiased estimators". (BLUE). Linear: in Y Best: Best for any alternative linear on unbiased estimators. Var ( βˆ j ) ≤ Var (b j ) ∀j . Proof: Let b is any other linear estimator of β: Nam T. Hoang University of New England - Australia 5 University of Economics - HCMC - Vietnam
Advanced Econometrics Chapter 2: Finite Sample Properties Of The OLS Estimator b =  A . Y k ×1 k × n n ×1 Unbiased: E(b) = β E(b) = E(AY) =E(AXβ + Aε) E(b) = AXβ + 0 = AXβ = β → AX =I Let A = (X'X)-1X' + C where C is any non-stochastic (k×n) matrix. I = AX = [( X ' X ) −1 X '+C ] X = ( X') −1 X X X + CX = CX = 0 ' I b = AY = [( X ' X ) −1 X '+C ][ Xβ + ε ] = ( X') −1 X X X β + ( X ' X ) −1 X ' ε + CXβ + Cε ' I = β + ( X ' X ) −1 X ' ε + Cε VarCov(b) = E[(b − β )(b − β )' ] = E{[( X ' X ) −1 X ' ε + Cε ][( X ' X ) −1 X ' ε + Cε ]' } = E[( X ' X ) −1 X ' (εε ' ) X ( X ' X ) −1 + ( X ' X ) −1 (εε ' )C '+Cεε ' X ( X ' X ) −1 + Cεε ' C ' ] = σ ε2 ( X') −1 X X X ( X ' X ) −1 + σ ε2 ( X ' X ) −1 X ' C '+σ ε2 CX ( X ' X ) −1 + σ ε2 CC ' ' I = σ ε2 ( X ' X ) −1 + σ ε2 CC '  VarCov ( βˆ ) The jth diagonal element: n Var (b j ) = Var ( βˆ j ) + σ ε2 ∑ c 2ji ≥ Var ( βˆ j ) ∀j = 1, k i =1 → Var (b j ) ≥ Var ( βˆ j ) ∀j = 1, k → βˆ j is the best linear unbiased estimator (BLUE). → βˆ j is efficient estimator (smallest variance). Nam T. Hoang University of New England - Australia 6 University of Economics - HCMC - Vietnam
Advanced Econometrics Chapter 2: Finite Sample Properties Of The OLS Estimator V. REVIEW: STATISTICAL INFERENCE: 1. Linear function of normal random variables are also normal: u ~ N( µ , Σ ) n ×1 n ×1 n × n →  =  Z P u is normally distributed. m ×1 m × n n ×1 E ( Z ) = E ( Pu ) = PE (u ) = Pµ VarCov( Z ) = E [( Z − E ( Z ))( Z − E ( Z ))' ] = E[( Pu − Pµ )( Pu − Pµ )' ] = P E[( u µ − u −µ )( )' ]P' = PΣP'  Σ Then Z ~ N ( Pµ , PΣP' ) 2. Chi-squared distribution: If Z ~ N (0, I ) then Z'Z has the Chi-squared distribution with r degree of freedom r×1 or Z ' Z ~ χ [2r ] Z'Z r: number of these independent standard normal variables in the sum of squares: Theorem: If Z ~ N (0, I ) and A is idempotent with rank equal to r, then: r×1 n ×n i. Z ' AZ ~ χ [2r ] ii. r = tr ( A) = rank ( A) 3. Eigenvalue - eigenvector problem: For a square matrix A , we can find n pairs of (λ j , c j ) such that: n ×n 1×1 n ×1 A c j = (λ j c j ) j = 1,2, ... , n n ×n n ×1 1×1 n ×1 Nam T. Hoang University of New England - Australia 7 University of Economics - HCMC - Vietnam
Advanced Econometrics Chapter 2: Finite Sample Properties Of The OLS Estimator n normalizing: c j ' c j = 1 ( ∑ c 2j = 1) j =1 The eigenvectors are orthogonal to each other: ci ' c j = 0 (∀i ≠ j ) so c = [c1, c2, ..., cn] is an orthogonal matrix: c' c = I ( c ' = c −1 ) Eigenvalue - eigenvector problem: A c j = (λ j c j ) j = 1,2, ... , n n ×n n ×1 1×1 n ×1 c1 j    c2 j cj'cj = 1 ci ' c j = 0 (∀i ≠ j ) cj =       cnj  Let: C = [c1 c2  cn ] ⇒ c' c = I n ×n n ×n → c' = c-1: orthogonal matrix: AC = A[c1 c2  cn ] = [ Ac1 Ac2  Acn ] = [c1λ1 c 2 λ2  c n λn ] λ1 0  0  0 λ  0 AC = [c1 c2  cn ]  2  = CΛ       00  λn    Λ where Λ is a diagonal matrix: C ' AC = C ' CΛ = Λ and also Rank ( A) = Rank ( Λ ) = number of no-zero of λj's. Note: C' AC = Λ → C ' −1 C ' ACC −1 = (C ' ) −1 ΛC −1 = CΛC ' Remember: A = CΛC ' and C' AC = Λ ; C'C = I, C' = C-1 Theorem: Let A be an idempotent matrix with rank = r and let Z ~ N (0, I ) then: r×1 Z ' AZ ~ χ [2r ] and rank ( A) = tr ( A) Nam T. Hoang University of New England - Australia 8 University of Economics - HCMC - Vietnam
Advanced Econometrics Chapter 2: Finite Sample Properties Of The OLS Estimator Proof: C' AC = Λ , Z ~ N (0, I ) r×1 For A idempotent, λj = 0 or 1 Because: AC j = C j λ j → AAC j = AC j λ j = C j λ2j So: C j λ2j = C j λ j → C j (λ2j − λ j ) = 0 → C j λ j (λ j − 1) = 0 → λ j = 0 or λ j = 1 1 0  0 0 0 1  0 0   Write: C' AC = Λ =         0 0  1 0 0 0  0 0 There must be r nonzero elements of Λ , because rank ( A) = r = rank ( Λ ) = tr ( Λ ) since all diagonal elements are 0 or 1. (Rule: tr(A.B) = tr(B.A)) Also tr ( Λ ) = tr ( ACC ' ) = tr ( A) so rank ( A) = tr ( A) = r u = C ' , Z ) Z ~ N (0, I ) n×1 n ×1 n × n n ×1 E (uu' ) = E (C ' ZZ ' C ) = C '  E ( ' )C = C ' C = I ZZ I Contruct quadratic form: n u' Λu = Z ' C (C ' AC )C ' Z = Z ' AZ = ∑ ui2 ~ χ [2r ] i =1 So if Z ~ N (0, I ) and A is idempotent with rank equal to r, then n ×n Z ' AZ ~ χ [2r ] Z ' AZ Extension: So if Z ~ N (0, σ 2 I ) , then ~ χ [2r ] σ 2 4. Other distribution: Let Z be N(0,I) and let W be χ [r2 ] and let Z and W be independently distributed, then: Nam T. Hoang University of New England - Australia 9 University of Economics - HCMC - Vietnam
Advanced Econometrics Chapter 2: Finite Sample Properties Of The OLS Estimator Z ~ t[ r ] W r has the t-distribution with r degree of freedom. Let W be χ [r2 ] and let v be χ [2s ] and W and v be independently distributed, then: W r ~ Fsr v s has the F-distribution with r (numerator) and s (denominator) degree of freedom. VI. TESTING HYPOTHESIS ON INDIVIDUAL COEFFICIENT: Y = X.β + ε with ε ~ N [0, σ 2 I ] • Recall: βˆ ~ N[β, σ ε2 ( X ′X ) −1 ] So βˆ j ~ N[βj, σ ε2 [( X ′X ) −1 ]ij ] βˆ j − β j → ~ N [0,1] σ 2 ( X ' X ) −jj1 but σ2, so this can't be used directly for constructing test or confidence intervals. e' e = ε ' M ' Mε = ε ' Mε , M is idempotent with with rank(M) = its trace = n-k. ε ~ N [0, σ 2 I ] → ε / σ ~ N [0, I ] ( n ×1) e' e ε ' Mε ⇒ = ~ χ [2n − k ] σ 2 σ2 βˆ j − β j σ 2 ( X ' X ) −jj1 So follow theorem: ~ tn −k e' e σ2 (n − k ) βˆ j − β j ⇔ ~ tn −k e' e ( X ' X ) −jj1 n  − k σˆ 2 Nam T. Hoang University of New England - Australia 10 University of Economics - HCMC - Vietnam
Advanced Econometrics Chapter 2: Finite Sample Properties Of The OLS Estimator βˆ j − β j ⇔ ~ tn −k σˆ 2 ( X ' X ) −jj1 σˆ 2 ( X ' X ) −jj1 = σˆ β2ˆ = standard error of βˆ j . j βˆ j − β j Finally: ~ tn −k σˆ β2ˆ j This basic result enables us to test hypothesis about elements of β and to construct confidence intervals for them (note that we need the assumption of normality of ε's). EX: yˆ i = 1.4 + 0.2 xi 2 + 0.6 xi 3 ( 0.7 ) 0.05 (1.4 ) H0: β2 = 0 H1: β2 > 0 βˆ j − β j 0.2 − 0 t= = =4 SE ( βˆi ) 0.05 tα (5%) = 1.74 d.o.f = n-k =17. tα (1%) = 2.567 t > tα → reject H0. EX: H0: β1 = 1.5 H1: β2 ≠ 1.5 ( or ≥ 1.5 or ≤ 1.5) βˆ j − β j 1.4 − 1.5 t= = = −0.1429 d.o.f = n-k =17. SE ( βˆi ) 0.7 2.5% 2.5% Nam T. Hoang University of New England - Australia 11 University of Economics - HCMC - Vietnam
Advanced Econometrics Chapter 2: Finite Sample Properties Of The OLS Estimator t < tα / 2 ⇒ cannot reject H0 at 5%. VII.CONFIDENCE INTERVALS: βˆi − β i Recall: ti = ~ tn −k SE ( βˆi ) so Pr[ −tα / 2 ≤ ti ≤ −tα / 2 ] = 1 − α βˆi − β i Pr[ −tα / 2 ≤ ≤ − tα / 2 ] = 1 − α SE ( βˆi ) Pr[ βˆi − tα / 2 SE ( βˆi ) ≤ β i ≤ βˆi + tα / 2 SE ( βˆi )] = 1 − α • If we were to take a sample of size "n", construct this repeat many times then 100(1-α)% of such intervals would cover the true value of βi • If we construct the interval once, there is no guarantee that the internal will cover the true βi]. • Type of errors: size & power of tests. Type I: Reject H0 when it is true. Type II: Accept H0 when it is false. Assume: Prob(type I error) = α Prob(type II error) = β If sample size is fixed: α↓ ⇒ β↑ call α: significant level or size of the test. → Fix α and try to design the test so to minimize β. • Definition: The power of a test is 1- β. Power = 1 - Pr(accept H0/H0 false) = Pr(reject H0/H0 false) Nam T. Hoang University of New England - Australia 12 University of Economics - HCMC - Vietnam
Advanced Econometrics Chapter 2: Finite Sample Properties Of The OLS Estimator • A test is "uniformly most powerful" if its power exceeds that of any other test (for the same choice of α) over all possible alternative hypothesis. • A test is "consistent" if its power → 1 as n →∞ for any false hypothesis. • A test is unbiased of its power never falls below α. VIII. FAMILY OF F-TEST: For general linear restrictions, unrestricted model (U-model), original model. H0: some restrictions on β . These define the restricted model (R-model): k ×1 ( ESS R − ESSU ) / r r Fdfu = ESSU ) / dfu ESSR = error sum of squares from R-model: e′R e R ESSU = error sum of squares from U-model: eU′ eU r: number of restrictions in H0. dfu: degree of freedom in U-model = n-k. ESSU eU′ eU ε ′Mε = = σ 2 σ 2 σ2 ε′ ε = M ~ χ [2n − k ] σ σ  ESS R  σ 2 ~ χ [2n − ( k − r )] ESS R ESSU  → − ~ χ [2r ]  ESSU σ 2 σ2  σ 2 ~ χ [2n − k ] ( ESS R − ESSU ) / σ 2 r ( ESS R − ESSU ) / r = ESSU ) /(n − k )σ 2 ESSU ) /(n − k ) ( ESS R − ESSU ) / r → ~ Fnr− k ESSU ) /(n − k ) Nam T. Hoang University of New England - Australia 13 University of Economics - HCMC - Vietnam
Advanced Econometrics Chapter 2: Finite Sample Properties Of The OLS Estimator Case 1: Join significant of all slopes: β  1 β =  1 H0: β = 0 → r = k −1  β 2  k −1 2 k ×1 ( k −1) ×1 U-model: Y = X β +ε → ESSU =e'e dfu = n-k k ×1 R-model: Yi = β1 + ε i → βˆ1 + Y → Yi = Y + ei n ESS R = ∑ (Yi − Y ) 2 i =1 n ( ∑ (Yi − Y ) 2 − e' e) /(k − 1) R 2 /(k − 1) → Fnk−−k1 = i =1 = e' e /(n − k ) (1 − R 2 ) /(n − k ) Case 2: k −r β  β =  1 H0: β 2 = 0 k ×1 β 2  r r ×1 r ×1 U-model: Y = Xβ + ε → ESSU = eU′ eU R-model: Y = X β +ε → ESSU = e′R e R ( k − r ) ×1 n ESS R = ∑ (Yi − Y ) 2 i =1 ( ESS R − ESSU ) / r → Fnr− k = ESSU ) /(n − k ) EX: Translog of production function: log Y = β1 + β 2 log K + β 3 log L + β 4 (log K ) 2 / 2 + β 5 (log L) 2 / 2 + β 6 (log K log L) + ε H 0 : β 4 = β 5 = β 6 = 0 Cobb-Douglas restrictions. n = 27 ESSU = 0.67993 r=3 ESSR = 0.85163 n - k = 21 Nam T. Hoang University of New England - Australia 14 University of Economics - HCMC - Vietnam
Advanced Econometrics Chapter 2: Finite Sample Properties Of The OLS Estimator → Fnr− k = 1.768 . Critical value: F213 ,5% = 3.1 → Fnr− k < Critical value ⇒ So do not reject H0 and conclude that are consistent with the Cobb-Douglas model. Case 3: General restrictions.  β1  R β =C β =  β 2  r × k k ×1 r ×1  β 2  Restrictions: β2 + β3 = 1 r ×1 r ×1 r ×1 → [0 1 1]β = 1 ( r = 1)   R If restrictions: β 2 + β 3 = 1  ( r = 2) β1 = 0 0 1 1  1  →   β =  1 0 0  0 Jarque - Beta statistics: H0: εi are normally distributed. H1: εi are not normally distributed. JB ~ χ 22 JB = SK2 +(Kur)2 Reject H0 for large JB. Reject H0 if JB >7 (critical) or if p-value < 0.05 Nam T. Hoang University of New England - Australia 15 University of Economics - HCMC - Vietnam