Introduction
There is no doubt that machine learning has become one of the trending technologies, and it is used widely within various applications.
Due to frequent use of machine learning within various applications, people are moving towards it.
Regression analysis is one of the primitive or introductory type of machine learning algorithms. And because most of the people start their machine learning career with Supervised Learning, it is regression analysis which is learnt within the initial phase through most of the practitioners.
Now, let’s try to understand what actually regression analysis is.
What is Regression Analysis?
Regression Analysis can be understood as the set of statistical processes or approaches to estimate relationships between dependent variables and independent variables (one or more). In layman terms, it is used to understand how the small changes within independent variables affect the dependent variables.
In regression algorithms, the targeted variable (predicted values) is a continuous numeric variable, for example sales and income of an entity, house prices and test scores of the students.
It falls under supervised machine learning type. There is an another one of the main types of supervised machine learning, known as Classification Algorithms. Many times, Logistic regression is considered as one of the classification techniques but here we are listing it within regression algorithms.
Mainly, there are three major uses of regression analysis are determination of:
- Strength of prediction
- Forecasting effects
- Trend Forecasting
Types of Regression Algorithms
There are mainly 6 types of regression algorithms:
- Linear Regression
- Multiple Regression
- Polynomial Regression
- Ridge Regression
- Lasso Regression
- Elastic Net Regression
We will learn each of them one by one.
So, let’s get started!
Linear Regression
Linear Regression: Linear regression algorithms are used when there are only one dependent variable and one independent variable, which are denoted by y and x respectively. These algorithms examine two factors:
- How closely are x and y related?
It gives a number between -1 to 1, which indicates the correlation between both of the variables.
Where,
0 indicates no relation
1 indicates positive correlation
-1 indicates negative correlation
- Prediction
When we know about x and y, and the model is used to predict the results for unknown x variables. It is done by fitting a linear relationship and which is represented as
y = mx + c
x = independent variable
y = dependent variable
m = weightage
c = intercept
Multiple Linear Regression
Multiple Linear Regression- It is a statistical technique used to predict the outcome or dependent variable which depends on more than one independent variable. In layman terms, linear regression algorithm only runs when there is only one independent variable. But what if there are more than one independent variable. And here multiple linear regression algorithm introduces itself.
Polynomial Regression
Linear regression algorithm works only for straight line functions (which has only one degree) while in case of polynomial regression the independent variable contains polynomial functions or equations. And that’s why, Polynomial regression algorithm is applied when we do not get a linear function or representation.
Although polynomial regression algorithm is used to fit the non-linear dataset. It is widely used in curvilinear form of datasets.
Ridge Regression
Ridge Regression- It is used when the system faces a problem of multicollinearity. The goal behind adopting this technique is to reduce the variance or identifying a new line which has some bias with respect to the training data through adding a degree of bias to the regression estimates. In brief, these are used to reduce the standard errors.
Where bias is referred as error due to erroneous or over simplistic or primitive assumptions in the learning algorithm.
- Model underfitting your data
- Making it hard to have high predictive accuracy
And variance refers to error due to too much complexity in learning algorithm that you are using.
Lasso Regression
Lasso Regression- Lasso regression is similar to ridge regression but it also includes feature selection. It sets the coefficient value for the features which do not help in decision making very low or potentially.
LASSO (Least Absolute Shrinkage Selector Operator) Regression is used to reduce the number of dependent variables. It is broadly used in regression analysis to complete both tasks (regularization and variable selection).
Elastic Net Regression
Elastic Net Regression – Elastic net regression is the combination of Lasso and Ridge regression. When we are not able to select between Lasso and Ridge regression algorithms then we used to adopt ElasticNet Regression as the hybrid of both of the techniques. Elastic-net regression is mainly used when the number of predictors become greater than the number of observations.
Powered by Froala Editor
