Essentials of Machine Learning Algorithms | EduGrad

Data Science, Machine Learning, Deep Learning, and Artificial Intelligence are some of the popular buzzwords in the analytics Eco space. These technologies were there in the past as well but the recent hype is due to the large volumes of structured, and unstructured data that is getting generated and the massive computational capacity that modern computers possess.

Machine Learning is the study of training machine with historical data to build predictive models for the unknown datasets. Most companies these days are accepting ML in their architecture to speed up their workflow and automate tasks which needed repetitive human intervention. There are several pre-programmed algorithms which are used to build such predictive models and solve either classification, regression, or clustering problems.

The state-of-the-art Machine Learning algorithms could be classified into –

  • Supervised Machine Learning – The dataset used in Supervised Learning is labeled which means for each row there is a target variable given. The model is trained with the supervised training set and then tested on the unknown data. Linear Regression, Logistic Regression, etc., are some Supervised Machine Learning algorithm.
  • Un-Supervised Learning – Unlike supervised learning, Un-Supervised Machine Learning algorithm, the dataset is unlabelled and needs to be grouped together based on the similarity among the data points. K-Means clustering, Apriori are some of the algorithms used for clustering the data points into different groups.
  • Reinforcement Learning – A special type of Machine Learning where the model learns from past actions and it is rewarded for every correct move and penalized for any wrong move taken. Google’s AlphaGo is an example of a Reinforcement Learning application.

Now that we understood what Machine Learning is, we would now look into the essentials of the Machine Learning algorithm in Python.

1. Linear Regression: 

In a supervised learning problem, the target variable could be numeric or discrete in nature. Linear regression is one of the first algorithms one should master which takes into account the linear relationship between the independent variables and the continuous dependent variable.

The Univariate Linear Regression is represented by y = a*x + b while the multivariate linear regression is represented by y = a + x(1)b(1) + x(2)b(2) +….+ x(n)b(n).

The goal in Linear Regression Machine Learning algorithm is to reduce the cost function to its global minimum with the technique known as Gradient Descent where the value of the coefficient is updated after each iteration until it converges.

Essentials of Machine Learning Algorithms - Linear Regression

Essentials of Machine Learning Algorithms - Linear regression Algorithm | EduGrad

2. Logistic Regression:

Unlike Linear Regression, the target variable in Logistic Regression Machine Learning algorithm is discrete in nature which could be binary, multinomial or ordinal. In Binary Classification problem, the output is either 0/1, True/False, and so on. The activation function used here is known as the Sigmoid function which is the log of odds in favor.

Essentials of Machine Learning Algorithms - Sigmoid function | EduGrad

The Sigmoid function takes a lot of time to compute and hence for multiclass problems, the softmax function is used.

Essentials of Machine Learning Algorithms - Softmax function | EduGrad

The ROC curve is often the go-to metric to evaluate a classification model.

Essentials of ML Algorithms - Comparing ROC curves for classification model | EduGrad

The Python code for Logistic Regression –

Essentials of ML Algorithm - Python code for logistic regression | EduGrad

Essentials of Machine Learning Algorithms - Logistic Regression code | EduGrad

3. Support Vector Machines:

The Support Vector Machines are used in classification problems where a hyperplane separates two classes. The vectors used to decide the optimal position of the hyperplane are known as the support vectors and it ensures maximum separation from the hyperplane to the classes.

Essentials of Machine Learning Algorithms - Support Vector Machine | EduGrad

It’s important to tune the parameters such as the kernel, gamma, etc., in SVM. Based on the data the kernel could be linear or polynomial. Additionally, the regularization value or the value of C should be optimal to prevent overfitting and underfitting.

Essentials of ML Algorithms - Support Vector Machine code | EduGrad

Essentials of ML Algorithms - Support Vector Machine code 2 | EduGrad

4. Naïve Bayes:

Naïve Bayes algorithm works on both classification and regression problems. It works on the principle of Bayes Theorem which is the like hood of an event occurring considering some condition is true.

Essentials of ML Algorithms - Naive Bayes Algorithm | EduGrad

The algorithm is called Naïve because it believes the presence of one feature is independent of another and considers all features to be equally important in predicting the outcome.

Essentials of ML Algorithms - Naive Bayes code | EduGrad

5. K-Nearest Neighbors:

KNN or K-Nearest Neighbors classifies each data point based on the mode of the k Neighbors. The value of k is usually kept as an odd number to prevent any conflict. In case of continued value output, the value is the mean of the nearest Neighbors while for discrete output the value is the mode of the nearest Neighbors.

Essentials of ML Algorithms - KNN Algorithm | EduGrad

The Python code for KNN –

Essentials of ML Algorithm - Python code for KNN | EduGrad

Essentials of ML Algorithms - KNN code | EduGrad

6. Decision Tree:

One of the simplest CART algorithms, Decision Tree is interpretable and is not affected by the presence of outliers, or missing values in the data. The root node is chosen based on the feature which carries the maximum information and this iterative process continues in the child nodes as well.

Essentials of ML Algorithms - Decision Tree | EduGrad

The splitting is stopped when the tree has reached its maximum depth or all instances has been classified. Decision Trees are prone to overfitting and hence it’s required to set constraints at each step or prune the tree.

Essential of ML Algorithms - Decision tree code 1 | EduGrad

Essentials of ML Algorithms - Decision Tree code 2 | EduGrad

Essentials of ML Algorithms - Decision Tree code 3 | EduGrad

7. Random Forest:

Random Forest is a bagging model which reduces the variance in a model. In Random Forest, the data is sampled into many small datasets which could be defined as a parameter. Then on each sampled data, the Decision Tree algorithm is applied and the final output is either the mean of all the outputs or the mode of a class.

Random Forest reduces overfitting and could be used as a dimensionality reduction techniques as well. However, it is not interpretable.

Essentials of ML Algorithms - Python code for Random Forest | EduGrad

Essentials of ML Algorithms - Random Forest Algorithm | EduGrad

8. K-Means Clustering:

An unsupervised learning algorithm where the data needs to be clustered into k groups in such a way that within a cluster, the distance is minimized and is maximum between the two clusters.

Essentials of ML Algorithm - K Means clustering Algorithm | EduGrad

The elbow method is used to choose the number of clusters maintaining the maximum variance in the data. Once k is defined, the centroids are initialized and adjusted repeatedly until all the points in a cluster are closest to the centroid.

Essentials of ML Algorithm - Python code for K Means clustering | EduGrad

Essentials of ML Algorithms - K Means clustering | EduGrad

Conclusion –

The advancement in the field of Machine Learning is endless and several new techniques and algorithms are coming out every now and then to simplify the predictive modeling tasks. This article consisted of the intuition behind some of the basic ML algorithms and their implementations in Python.

We hope you enjoy the blog post. You should try our Regression Analysis Quiz

Explore our Data science courses – 

Learn Data Analytics using Python | EduGradLearn web scraping using Python | EduGrad

Learn Python for Data science | EduGrad Learn Presentation skills for Data scientists | EduGrad

Our Popular Data science tutorials –

Learn Regression Analysis in 2 min | EduGrad Learn Natural Language Processing tutorial | EduGrad


Please enter your comment!
Please enter your name here