Multivariate functional data

Some notations

• Observation space:

$$H=\underset{P\text{terms}}{\underset{}{L^2\left(T_1\right)\times \cdots \times L^2\left(T_P\right)}}.$$

• Inner product in $H$:

$$⟨⟨f,g⟩⟩=\sum _{p=1}^P{\int }_{T_p}{f}^{\left(p\right)}\left(t_p\right)g^{\left(p\right)}\left(t_p\right)d{t}_p.$$

• For $N$ realizations of a process $X$, we note

  • the mean function $\mu$,

  • the covariance operator $\Gamma$, with covariance kernel $C$,

  • the Gram (inner-product) matrix $M$.

• Each feature of each observation is sampled on a regular grid of $M_p$ points.

Cloud of individuals

  

Cloud of individuals

• Let ${\pi }_n,n\in \{1,\dots ,N\}$ be a weight on each observation such that ${\sum }_n{\pi }_n=1$.

• Distance between observations

$$d^2(M_f,M_g)=⟨⟨f-g,f-g⟩⟩,f,g\in H.$$

• Inertia of the cloud $C_N$ using $d$

$$\sum _{n=1}^N{\pi }_n{d}^2(M_n,G_{\mu })=\frac{1}{2}\sum _{n=1}^N\sum _{m=1}^N{\pi }_n{\pi }_m{d}^2(M_n,M_m)=\sum _{p=1}^P{\int }_{T_p}\text{Var}{X}^{\left(p\right)}\left(t_p\right)d{t}_p.$$

• Another distance between observations

$$d_{\Gamma }^2(M_f,M_g)=⟨⟨f-g,\Gamma (f-g)⟩⟩,f,g\in H.$$

• Inertia of the cloud $C_N$ using $d_{\Gamma }$

$$\sum _{n=1}^N{\pi }_n{d}_{\Gamma }^2(M_n,G_{\mu })=\frac{1}{2}\sum _{n=1}^N\sum _{m=1}^N{\pi }_n{\pi }_m{d}_{\Gamma }^2(M_n,M_m)=\sum _{p=1}^P{\int }_{T_p}\left|\right|\left|C_{p\cdot }\right(t_p,\cdot \left)\right||{|}^{2}d{t}_p.$$

Cloud of features

  

Cloud of features

• Distance between features

$$d^2(M_f,M_g)=\sum _{n=1}^N{\pi }_n⟨⟨X_n-\mu ,f-g⟩{⟩}^2,f,g\in H.$$

• Inertia of the cloud $C_P$

$$\sum _{n=1}^N{\pi }_n{d}^2(M_n,G_{\mu })=\frac{1}{2}\sum _{n=1}^N\sum _{m=1}^N{\pi }_n{\pi }_m{d}_{\Gamma }^2(M_n,M_m)=\sum _{p=1}^P{\int }_{T_p}\left|\right|\left|C_{p\cdot }\right(t_p,\cdot \left)\right||{|}^{2}d{t}_p.$$

• Correlation coefficient

$$\cos {\theta }_{fg}=\frac{{\sum }_{n=1}^N{\pi }_n⟨⟨X_n-\mu ,f⟩⟩⟨⟨X_n-\mu ,g⟩⟩}{{({\sum }_{n=1}^N{\pi }_n⟨⟨X_n-\mu ,f⟩{⟩}^2)}^{1/2}{({\sum }_{n=1}^N{\pi }_n⟨⟨X_n-\mu ,g⟩{⟩}^2)}^{1/2}}=\frac{⟨⟨f,\Gamma g⟩⟩}{⟨⟨f,\Gamma f⟩⟩⟨⟨g,\Gamma g⟩⟩}.$$

Duality diagram

MFPCA

• Consider the matrix $M$ of size $N\times N$ with entries

$$M_{ij}=\sqrt{{\pi }_i{\pi }_j}⟨⟨X_i-\mu ,X_j-\mu ⟩⟩,i,j\in 1,\dots N.$$

• Eigenvalues of $\Gamma$ and $M$ are related by

$${\lambda }_k=l_k,k=1,2,\dots$$

• Eigenvectors of $\Gamma$ and $M$ are related by

$${\varphi }_k\left(t\right)=\frac{1}{\sqrt{N{l}_k}}\sum _{n=1}^N{v}_{nk}\left\{X_n\right(t)-\mu (t\left)\right\},k=1,2,\dots$$

• Scores are given by

$$c_{nk}=\sqrt{N{l}_k}{v}_{nk},n=1,\dots ,N,k=1,2,\dots$$

Computational complexity

• Assume $M^a={\sum }_{p=1}^P{M}_p^a$ and $K={\sum }_{p=1}^P{K}_p$.

• Using the diagonalization of the covariance operator (Happ and Greven (2018))

$$O(\underset{\begin{array}{c}\text{Univariate covariance}\end{array}\begin{array}{c}\text{decomposition}\end{array}}{\underset{}{N{M}^2+M^3+N\sum _{p=1}^P{M}_p{K}_p}}+\underset{\begin{array}{c}\text{Univariate scores}\end{array}\begin{array}{c}\text{decomposition}\end{array}}{\underset{}{N{K}^2+K^3}}+\underset{\begin{array}{c}\text{Multivariate eigencomponents}\end{array}\begin{array}{c}\text{and scores estimation}\end{array}}{\underset{}{K\sum _{p=1}^P{M}_p{K}_p+N{K}^2}}).$$

• Using the diagonalization of the inner product matrix

$$O(\underset{\begin{array}{c}\text{Gram matrix}\end{array}\begin{array}{c}\text{decomposition}\end{array}}{\underset{}{N^2{M}^1+N^3}}+\underset{\begin{array}{c}\text{Multivariate eigencomponents}\end{array}\begin{array}{c}\text{and scores estimation}\end{array}}{\underset{}{KPN+KN}}).$$

• Note that, here, the smoothing part is not considered into the computational complexity.

Simulation of multivariate functional data

Simulation of multivariate functional data

Simulation of images data

Simulation of images data

Takeaway ideas

• We gave a geometric interpretation of the duality between rows and columns of a functional data matrix.

• We provided relationships between the eigenelements of the covariance operator and the ones of the Gram matrix.

• When to use the covariance operator?

  • Only one-dimensional curves.

  • For sparse to relatively dense functional data.

• When to use the Gram matrix?

  • For two-dimensional (or higher dimensional) functional data (images).

  • For ultra-dense functional data.

• The paper is available on arXiv: arXiv:2306.12949

Thank you for your attention!

References

De la Cruz, O. and S. Holmes (2011). “The Duality Diagram in Data Analysis: Examples of Modern Applications”. In: The annals of applied statistics 5.4, pp. 2266–2277. ISSN: 1932-6157.

Happ, C. and S. Greven (2018). “Multivariate Functional Principal Component Analysis for Data Observed on Different (Dimensional) Domains”. In: Journal of the American Statistical Association 113.522, pp. 649-659. DOI: 10.1080/01621459.2016.1273115.