Billy Ian's Short Leisure-time Wander

into learning, investment, intelligence and beyond

[Notes on Mathematics for ESL] Chapter 5: Basis Expansions and Regularization

| Comments

5.4 Smoothing Splines

Derivation of Equation (5.12)

Equal the derivative of Equation (5.11) as zero, we get

Put the terms related to $\theta$ on one side and the others on the other side, we get

Multiply the inverse of $N^TN+\lambda\Omega_N$ on both sides completes the derivation of Equation (5.12)

Explanations on Equation (5.17) and (5.18)

It’s a little confusing to get Equation (5.18) directly from Equation (5.17) and its original form Equation (5.11). In order to give a clear explanation, here we give the proof of the equation,

which are the different terms between Equation (5.11) and Equation (5.18).

We know that

following Equation (5.14) in the book.

From Equation (5.17), we can get

Plug the above two equation into the right side of the equation remains to be proved, we get

which completes the proof.