To improve the performance of Linear Regression, you can consider feature engineering, regularization, and handling outliers.

• Perform feature engineering to create new features that capture important i...read more

Regularization promotes the use of simpler models, this can be achieved by having fewer parameters (works best with neural networks) or by simply reducing the weights (works better for Linear Regression).

L1 and L2 regularization are two methods that penalize larger weights by adding a parameter to the loss function. The importance/weightage of this can be controlled with a hyperparameter. L1 introduces the term |w| into the loss function and penalizes large weights linearly, L2 introduces the term |w|² thereby penalizing weights quadratically.

We can also make use of Early Stopping, which is stopping the training when the validation accuracy stagnates/stops improving. A validation set is a separate subset of data, which is used to judge the models training performance, and providing a measure of the models accuracy during training.