Stock Price Prediction is arguably the difficult task one could face. The uncertainty that surrounds it makes it nearly impossible to estimate the price with utmost accuracy. Moreover, there are so many factors like trends, seasonality, etc., that needs to be considered while predicting the stock price.
As this article encompasses the use of Machine Learning and Deep Learning to predict stock prices, we would first provide a brief intuition of both these terms.
Machine Learning is a study of training machines to learn patterns from old data and make predictions with the new one. The computer is trained first with historical data which could be labeled or unlabelled based on the problem statement and once it performs well on the training data, it is evaluated on the test data set. Often the metrics used for prediction could be misleading and hence it is necessary to define the KPI and the metrics of evaluation beforehand keeping the business objective in mind.
With the advancement of technology and the huge amounts of unique data that is getting generated from a variety of sources, it is imperative that modern systems are well equipped to deal with such volumes data. Deep Learning is a branch of Machine Learning which deals with neural networks that is similar to the neurons in our brain. The more data you feed on a neural network, the better it is trained and the more accurate predictions you get. However, you should be aware of using regularization in case the neural network overfits. The default is having one layer of the hidden layer along with the input and the output layers but you could also define more layers keeping the number of units in each layer same.
In this article, we would cover Stock Price Prediction using Machine Learning algorithms like Linear Regression and then transit into Stock Price Prediction using Deep Learning techniques like LSTM or Long Short Term Memory network built on the Recursive Neural Network (RNN) architecture. All the codes covered in the blog are written in Python. The description of the implementation of Stock Price Prediction algorithms is provided.
Problem Statement for Stock Price Prediction Project –
The dataset used for this stock price prediction project is downloaded from here. It consists of S&P 500 companies’ data and the one we have used is of Google Finance.
Prediction of Stock Price with Machine Learning
Below are the algorithms and the techniques used to predict stock price in Python.
- We have created a function first to get the historical stock price data of the company
- Once the data is received, we load it into a CSV file for further processing
- Once the data is collected and loaded, it needs to be pre-processed. The necessary Python libraries are imported and the first five rows of the data are displayed
- A couple of columns like Date and High are removed
- The data is visualized to look for any underlying relationship
- As seen from the data, there are high range values which often results in the model giving more importance to the higher number and thus giving a poor prediction. Scaling the data would ensure that it is limited within a specific range and there is no bias in the data while training the model. Thus the data is normalized with the MinMaxScaler which scales each value within the range 0 to 1.
- Visualizing the data further
- We would save the Pre-processed data for later use
- Now, we would start building the model using the Linear Regression algorithm. The libraries are imported and the pre-processed data is loaded
Stock price prediction using Linear Regression –
- The data is split into train and test set and the Linear Regressor model is trained on the training data
- Once the model is trained, it is evaluated on the test set
- The Predicted against the Actual Values are visualized
- The accuracy is measured
- The LSTM model is used below to predict the stock price
- Similarly, the dataset is split into train and test set
- The Deep Learning model using the Long Short Term Memory network is built
- The model is trained and then predicted on the test set
- The prediction is visualized against the actual data points and its accuracy is measured
It is clearly observed that the LSTM model has outperformed the Linear Regression model and has significantly reduced the cost function as well. The model could be tuned further by adding dropout values, changing the LSTM layers, adding more units in the layers, increasing the number of epochs, and so on.
Stock Price prediction is an application of Time Series forecasting which is one of the hardest and intriguing aspects of Data Science. This blog covered how both machine learning and deep learning could be used to predict stock prices which may be daunting as it might seem but with the right technique it could be accomplished.