Functional additive model

In statistics, a functional additive model (FAM) can be viewed as an extension of a generalized functional linear model where the linearity assumption between the response (scalar or functional) and the functional linear predictor is replaced by an additivity assumption.

In these models, functional predictors ( X {\displaystyle X} ) are paired with responses ( Y {\displaystyle Y} ) that can be either scalar or functional. The response can follow a continuous or discrete distribution and this distribution may be in the exponential family. In the latter case, there would be a canonical link that connects predictors and responses. Functional predictors (or responses) can be viewed as random trajectories generated by a square-integrable stochastic process. Using functional principal component analysis and the Karhunen-Loève expansion, these processes can be equivalently expressed as a countable sequence of their functional principal component scores (FPCs) and eigenfunctions. In the FAM^[1] the responses (scalar or functional) conditional on the predictor functions are modeled as function of the functional principal component scores of the predictor function in an additive structure. This model can be categorized as a Frequency Additive Model since it is additive in the predictor FPC scores.

The Continuously Additive Model (CAM)^[2] assumes additivity in the time domain. The functional predictors are assumed to be smooth across the time domain since the times contained in an interval domain are an uncountable set, an unrestricted time-additive model is not feasible. This motivates to approximate sums of additive functions by integrals so that the traditional vector additive model be replaced by a smooth additive surface. CAM can handle generalized responses paired with multiple functional predictors.

The Functional Generalized Additive Model (FGAM)^[3] is an extension of generalized additive model with a scalar response and a functional predictor. This model can also deal with multiple functional predictors. The CAM and the FGAM are essentially equivalent apart from implementation details and therefore can be covered under one description. They can be categorized as Time-Additive Models.

Functional Additive Model for scalar and functional responses respectively, are given by

E ( Y ∣ X ) = μ Y + ∑ k = 1 ∞ f k ( ξ k ) {\displaystyle E(Y\mid X)=\mu _{Y}+\sum _{k=1}^{\infty }f_{k}(\xi _{k})}

E ( Y ( t ) ∣ X ) = μ Y ( t ) + ∑ k = 1 ∞ ∑ m = 1 ∞ f k m ( ξ k ) ψ m ( t ) , {\displaystyle E(Y(t)\mid X)=\mu _{Y}(t)+\sum _{k=1}^{\infty }\sum _{m=1}^{\infty }f_{km}(\xi _{k})\psi _{m}(t),}

where ξ k {\displaystyle \xi _{k}} and ζ m {\displaystyle \zeta _{m}} are FPC scores of the processes X {\displaystyle X} and Y {\displaystyle Y} respectively, ϕ k {\displaystyle \phi _{k}} and ψ m {\displaystyle \psi _{m}} are the eigenfunctions of processes X {\displaystyle X} and Y {\displaystyle Y} respectively, and f k {\displaystyle f_{k}} and f k m {\displaystyle f_{km}} are arbitrary smooth functions.

To ensure identifiability one may require, E f k ( ξ k ) = 0 , k = 1 , 2 , … E f k m ( ξ k ) = 0 , k = 1 , 2 , … m = 1 , 2 , … {\displaystyle Ef_{k}(\xi _{k})=0,\quad k=1,2,\ldots Ef_{km}(\xi _{k})=0,k=1,2,\ldots m=1,2,\ldots }

The above model is considered under the assumption that the true FPC scores ξ k {\displaystyle \xi _{k}} for predictor processes are known. In general, estimation in the generalized additive model requires backfitting algorithm or smooth backfitting to account for the dependencies between predictors. Now FPCs are always uncorrelated and if the predictor processes are assumed to be gaussian then the FPCs are independent. Then

E ( Y − μ Y | ξ k ) = E { E ( Y − μ Y | X ) | ξ k } = E { ∑ j = 1 ∞ f j ( ξ j ) | ξ k } = f k ( ξ k ) , {\displaystyle E(Y-\mu _{Y}|\xi _{k})=E\{E(Y-\mu _{Y}|X)|\xi _{k}\}=E\{\sum _{j=1}^{\infty }f_{j}(\xi _{j})|\xi _{k}\}=f_{k}(\xi _{k}),}

similarly for functional responses

E ( ζ m | ξ k ) = f k m ( ξ k ) , {\displaystyle E(\zeta _{m}|\xi _{k})=f_{km}(\xi _{k}),}

This simplifies the estimation and requires only one-dimensional smoothing of responses against individual predictor scores and will yield consistent estimates of f j . {\displaystyle f_{j}.} In data analysis one needs to estimate ξ k {\displaystyle \xi _{k}} before proceeding to infer the functions f k {\displaystyle f_{k}} and f k m {\displaystyle f_{km}} , so there are errors in the predictors. functional principal component analysis generates estimates ξ k ^ {\displaystyle {\hat {\xi _{k}}}} of ξ k {\displaystyle \xi _{k}} for individual predictor trajectories along with estimates for eigenfunctions, eigenvalues, mean functions and covariance functions. Different smoothing methods can be applied to the data { ξ ^ i k , Y i } i = 1 , . . . , n {\displaystyle \{{\hat {\xi }}_{ik},Y_{i}\}_{i=1,...,n}} and { ξ ^ i k , ζ ^ i m } i = 1 , . . . , n {\displaystyle \{{\hat {\xi }}_{ik},{\hat {\zeta }}_{im}\}_{i=1,...,n}} to estimate f k {\displaystyle f_{k}} and f k m {\displaystyle f_{km}} respectively.

The fitted Functional Additive Model for scalar response is given by

E ^ ( Y | X ) = Y ¯ + ∑ k = 1 K f k ^ ( ξ k ) , {\displaystyle {\hat {E}}(Y|X)={\bar {Y}}+\sum _{k=1}^{K}{\hat {f_{k}}}(\xi _{k}),} and the fitted Functional Additive Model for functional responses is by

E ^ ( Y ( t ) | X ) = μ ^ Y ( t ) + ∑ m = 1 M ∑ k = 1 K f ^ k m ( ξ k ) ψ ^ m ( t ) , t ∈ T {\displaystyle {\hat {E}}(Y(t)|X)={\hat {\mu }}_{Y}(t)+\sum _{m=1}^{M}\sum _{k=1}^{K}{\hat {f}}_{km}(\xi _{k}){\hat {\psi }}_{m}(t),t\in {T}}

Note: The truncation points K {\displaystyle K} and M {\displaystyle M} need to be chosen data-adaptively. Possible methods include pseudo-AIC, fraction of variance explained or minimization of prediction error or cross-validation.^{[citation needed]}

For the case of multiple functional predictors with a scalar response, the Functional Additive Model can be extended by fitting a functional regression which is additive in the FPCs of each of the predictor processes X j , j = 1 , . . . , d {\displaystyle X_{j},j=1,...,d} . The model considered here is Additive Functional Score Model (AFSM) given by

E ( Y | X 1 , X 2 , . . . , X d ) = ∑ j = 1 d ∑ k f j k ( ξ j k ) {\displaystyle E(Y|X_{1},X_{2},...,X_{d})=\sum _{j=1}^{d}\sum _{k}f_{jk}(\xi _{jk})}

In case of multiple predictors the FPCs of different predictors are in general correlated and a smooth backfitting^[4] technique has been developed to obtain consistent estimates of the component functions f j k {\displaystyle f_{jk}} when the predictors are observed with errors having unknown distribution.

Since the number of time points on an interval domain is uncountable, an unrestricted time-additive model E ( Y | X ) = ∑ { t ∈ [ 0 , T ] } f t ( X ( t ) ) {\displaystyle E(Y|X)=\sum _{\{t\in [0,T]\}}f_{t}(X(t))} is not feasible. Thus a sequence of time-additive models is considered on an increasingly dense finite time grid t 1 , t 2 , . . . , t m {\displaystyle t_{1},t_{2},...,t_{m}} in T {\displaystyle T} leading to

E ( Y | X ( t 1 ) , . . . , X ( t m ) ) = E ( Y ) + ∑ j = 1 m f j ( X t j ) {\displaystyle E(Y|X(t_{1}),...,X(t_{m}))=E(Y)+\sum _{j=1}^{m}f_{j}(X_{t_{j}})} where f j ( ⋅ ) = g ( t j , ⋅ ) {\displaystyle f_{j}(\cdot )=g(t_{j},\cdot )} for a smooth bivariate function g {\displaystyle g} with E ( { g ( t j , X ( t j ) } ) = 0 {\displaystyle E(\{g(t_{j},X(t_{j})\})=0} (to ensure identifiability). In the limit m → ∞ {\displaystyle m\rightarrow \infty } this becomes the continuously additive model

E ( Y | X ) = E ( Y ) + lim m → ∞ 1 m ∑ j = 1 m g { t j , X ( t j ) } = E ( Y ) + ∫ T g { t , X ( t ) } d t . {\displaystyle E(Y|X)=E(Y)+\lim _{m\to \infty }{\frac {1}{m}}\sum _{j=1}^{m}g\{t_{j},X(t_{j})\}=E(Y)+\int _{T}g\{t,X(t)\}dt.}

For g { t , X ( t ) } = β ( t ) { X ( t ) − E X ( t ) } {\displaystyle g\{t,X(t)\}=\beta (t)\{X(t)-EX(t)\}} the model reduces to generalized functional linear model

For non-Gaussian predictor process, g { t , X ( t ) } = β ( t ) [ ζ { X ( t ) } − E ζ { X ( t ) } , {\displaystyle g\{t,X(t)\}=\beta (t)[\zeta \{X(t)\}-E\zeta \{X(t)\},} where ζ {\displaystyle \zeta } is a smooth transformation of X ( t ) {\displaystyle X(t)} reduces CAM to a Functional Transformation model.

This model has also been introduced with a different notation under the name Functional Generalized Additive Model (FGAM). Adding a link function h {\displaystyle h} to the mean-response and applying a probability transformation G t {\displaystyle G_{t}} to X ( t ) {\displaystyle X(t)} yields the FGAM given by

h ( E ( Y | X ) = θ 0 + ∫ T F [ G t { X i ( t ) } , t ] d t , {\displaystyle h(E(Y|X)=\theta _{0}+\int _{T}F[G_{t}\{X_{i}(t)\},t]dt,}

where θ 0 {\displaystyle \theta _{0}} is the intercept.
Note: For estimation and implementation see^[2][3]