Functional Data Analysis

Functional Data Analysis

Sample path regularity — a key concept

• Data: (randomly) discrete time noisy measurements of the curves

• Recover the curves

  A nonparametric regression problem where optimality depends on the curve regularity;

  see, e.g., (Tsybakov, 2009).

• Estimate the mean and the covariance

  The optimal rates depend on the curve regularity;

  see, e.g., (Cai and Yuan, 2010), (Cai and Yuan, 2011), (Cai and Yuan, 2016).

• Fit usual predictive models (linear...)

  The optimal convergence rates depend on the predictor curve regularity;

  see, e.g., (Hall and Horowitz, 2007).

Sample path regularity

• Let $O_{\star }$ be a neighborhood of $t$.

• For $H_t\in (0,1)$, $L_t>0$, $u,v\in O_{\star }$ assume that the stochastic process $X$ satisfies the condition:

$$E(X_u-X_v{)}^2\asymp L_t^2|v-u{|}^{2H_t}.$$

• $H_t$ is called the local regularity of the process $X$ on $O_{\star }$.

• Our parameter $H_t$ is related to Hurst exponent.

• The definition extends to smoother sample paths using the derivatives of $X_u$ and $X_v$ instead.

Local regularity vs. eigenvalue decrease rate

• The (local) regularity is related to the decrease rate of the eigenvalues of the covariance operator of the process.

• Under some conditions, if

$${\lambda }_j\sim j^{-\nu },j\ge 1,$$

for some $\nu >1$, then

$$2(H+\delta )=\nu -1$$

when the sample paths admit derivative up to order $\delta$ and $H$ is the regularity of the derivatives of order $\delta$.

The value of `$\nu$` is usually supposed given!

Observed data

• Let $X^{\left(1\right)},\dots ,X^{\left(N\right)}$ be an independent sample of a random process $X=\left(X\right(t):t\in [0,1\left]\right)$ with continuous trajectories.

• For each $1\le n\le N$

  $M_n$ is a random positive integer.

  $T_m^{\left(n\right)}\in [0,1],1\le m\le M_n$ be the (random) observation times, design points, for the curve $X^{\left(n\right)}$.

• The observations are $(Y_m^{\left(n\right)},T_m^{\left(n\right)})$, $1\le m\le M_n,1\le n\le N$, where

  $$Y_m^{\left(n\right)}=X^{\left(n\right)}\left(T_m^{\left(n\right)}\right)+\sigma (T_m^{\left(n\right)},X^{\left(n\right)}(T_m^{\left(n\right)}\left)\right)e_m^{\left(n\right)}.$$

• $e_m^{\left(n\right)}$ are independent copies of a standardized error term.

Estimation

• For $s,t\in O_{\star }$, let

$$\theta (s,t)=E\left[\right(X_t-X_s{)}^2]\approx L^2|t-s{|}^{2H_{t_0}}.$$

• Let $t_1$ and $t_3$ be such that $[t_1,t_3]\subset O_{\star }$, and denote $t_2$ the middle point of $[t_1,t_3]$.

• A natural proxy of $H_{t_0}$ is given by

$$\frac{\log \left(\theta \right(t_1,t_3\left)\right)-\log \left(\theta \right(t_1,t_2\left)\right)}{2\log 2},\text{if}{t}_3-t_1\text{is small.}$$

• An estimator of $H_{t_0}$ is given by

$$\frac{\log \left(\hat{\theta }\right(t_1,t_3\left)\right)-\log \left(\hat{\theta }\right(t_1,t_2\left)\right)}{2\log 2},\text{if}{t}_3-t_1\text{is small.}$$ where, given a nonparametric estimator $\widetilde{X_t}$ of $X_t$, $$\hat{\theta }(s,t)=\frac{1}{N}\sum _{n=1}^N{({\tilde{X}}_t^{\left(n\right)}-{\tilde{X}}_s^{\left(n\right)})}^2.$$

Estimator of the mean function

• For any $t\in T$, let

$$w_n(t;h)=1\text{if}\sum _{m=1}^{M_n}1\left\{\right|T_m^{\left(n\right)}-t\le h\left|\right\}\ge k_0,n\in \{1,\dots ,N\}.$$

• Then,

$$W_N(t,h)=\sum _{n=1}^N{w}_n(t,h).$$

• The estimator of the mean function is

$${\hat{\mu }}_N(t,h)=\frac{1}{W_N(t,h)}\sum _{n=1}^N{w}_n(t,h){\hat{X}}^{\left(n\right)}\left(t\right),t\in T.$$

• An adaptive optimal bandwidth is

$${\hat{h}}_{\mu }^{\star }=C_{\mu }(Nm{)}^{-1/(1+2{\hat{H}}_t)}.$$

Results for mean estimation

Estimator of the covariance function

• For any $s\ne t$, let

$$W_N(s,t,h)=\sum _{n=1}^N{w}_n(s,h)w_n(t,h).$$

• The estimator of the covariance function is, for $|s-t|>\delta$,

$${\hat{\Gamma }}_N(s,t,h)=\frac{1}{W_N(s,t,h)}\sum _{n=1}^N{w}_n(s,h){\hat{X}}^{\left(n\right)}\left(s\right)w_n(t,h){\hat{X}}^{\left(n\right)}\left(t\right)-{\hat{\mu }}^{\star }\left(s\right){\hat{\mu }}^{\star }\left(t\right).$$

• An adaptive optimal bandwidth is

$${\hat{h}}_{\Gamma }^{\star }=C_{\Gamma }(Nm{)}^{-1/(1+2min({\hat{H}}_s,{\hat{H}}_t\left)\right)}.$$

Results for covariance estimation

Takeaway ideas

• The available data in FDA are usually noisy measurements at a discrete, possibly random design points.

• The usual FDA methods require the reconstruction of the curves.

• The optimal curve recovery depends on the purpose, but in most cases depends on the regularity of the sample paths.

• We formalize the concept of local regularity of the process, propose a first simple estimator for it and mean and covariance functions.

• The paper concerning the estimation of the regularity is available at

  DOI: 10.1214/22-EJS1997

• A preprint of the paper for the estimation of the mean and covariance is available at

  https://arxiv.org/abs/2108.06507

Thank you for your attention!

References

Cai, T. T. and M. Yuan (2010). “Nonparametric Covariance Function Estimation for Functional and Longitudinal Data”. In: University of Pennsylvania and Georgia institute of technology, p. 36.

Cai, T. T. and M. Yuan (2011). “Optimal estimation of the mean function based on discretely sampled functional data: Phase transition”. EN. In: Annals of Statistics 39.5, pp. 2330–2355. ISSN: 0090-5364, 2168-8966. (Visited on Dec. 31, 2020).

Cai, T. T. and M. Yuan (2016). “Minimax and Adaptive Estimation of Covariance Operator for Random Variables Observed on a Lattice Graph”. In: Journal of the American Statistical Association 111.513, pp. 253–265.

Hall, P. and J. L. Horowitz (2007). “Methodology and convergence rates for functional linear regression”. EN. In: The Annals of Statistics 35.1, pp. 70–91. (Visited on Nov. 29, 2018).

Ramsay, J. and B. W. Silverman (2005). Functional Data Analysis. 2nd ed. Springer Series in Statistics. New York: Springer-Verlag.

Tsybakov, A. B. (2009). Introduction to Nonparametric Estimation. En. Springer Series in Statistics. New York, NY: Springer New York. DOI: 10.1007/b13794. (Visited on Jan. 22, 2020).