On Regression

In this article, I will try to demonstrate why regression works at all (particularly with different regularizations) and how regularization should be controlled to have convergent answers. I will operate in the fixed design setting, i.e. when bounds are assumed on the input as opposed to assuming the random distribution from whence it came (the random design setting).