Data Science Learnings

How many times have you got baffled with the beginner part in Data Analysis? I am sure many times as I faced the same. So in this post, I will guide you as how to do Exploratory Data Analysis (EDA) with Python. EDA is one of the basic steps involved in Data Analysis. Before you begin with the analysis part, understanding data is very important. You should be familier with certain terminologies of the type of data on which you are working. For instance, you are working with any financial data let’s say stocks and thus, you should be familier with terminologies like open, close, high, low, volume and so on. Alright, we now know understanding data is important so let’s move ahead with the analysis part! Why Data Analysis? To get inferences from the data and to know future possible outcomes based on historical values of any data. Continuing the above example, with analysis of certain stock, let’s say I analyzed performance of a particular stock from year 2000 to 2020. I will find certain t

How many times have you tried to predict the sales, the weather or the stocks with good accuracy? Often times we may find the good Neural model yet we fail to tune it's hyper-parameters. Here's where deep learning model, PROPHET will be useful much for beginners in developing predictive models. Let's begin!

Analysis of the data before running into predictions is very important. Understand a few rows and a few columns is very nominal task and we can easily examine the data. However, with a little larger data, suppose 10,000 rows with 50 columns, we really need to do analysis of the data so that we can come to know which factors are going to affect our prediction. Data Analysis with Python is a bit tedious task as we have to prepare the data i.e. cleaning, pre-processing and normalization. We use Seaborn and Matplotlib for our data visualization. But before plotting the graphs, we need to know which columns are inter-related. For that, we need a co-relation matrix which we can create using Python. However, PPS matrix is more better than co-relation matrix. Fig1: Co-relation Matrix of Covid19 dataset Fig2: PPS Matrix of Covid19 dataset It is always a tedious task when we code for Data Analysis. So, we have certain tools available in the market for it like Power BI, Tableau, etc. I have done

In supervised learning, we train the machine using data which is well labelled.   It means some data is already tagged with the correct answer. Types of Supervised Machine Learning Algorithms: Regression (Linear and Multi-linear) Logistic Regression Classification Naïve Bayes Classifiers Decision Trees Support Vector Machine (SVM) Challenges in Supervised machine learning: Irrelevant input feature present training data could give inaccurate or wrong results. Data preparation and pre-processing is always a challenge. Accuracy suffers when impossible, unlikely, and incomplete values have been inputted as training data. If the concerned expert is not available, then the other approach is "brute-force." It means you need to think that the right features to train the machine on. It could be inaccurate. Advantages of Supervised Learning: Supervised learning allows you to collect data or produce a data output from the previous experience. Helps you to optimize performance criteria u

In the previous blog, we had discussed brief about What is Machine Learning? In this blog, we are going to learn about the types of ML.  ML is broadly classified into four types: Supervised Learning Unsupervised Learning Semi-supervised Learning Reinforcement Learning 1. Supervised Learning Supervised learning is where there are input variables, say X and there are corresponding output variables, say Y. We use a particular algorithm to map a function from input(X) to output(Y). Mathematically, Y=f(X). Majority of the ML models use this type of learning to feed itself and learn. The goal of supervised learning is to approximate the said function so well that whenever we enter any new input, it's output is accurately predicted. Here, we can say that there is a teacher who guides the model if it generates incorrect results and hence, the machine will keep on learning until it performs to desired results. Supervised Learning can be further classified into: Classification : Here, the ou

Arthur Samuel, firstly coined the term "Machine Learning". He defined the term as, "Field of study that gives computers the capability to learn without being explicitly programmed." Explaining in layman terms, Machine learning means improving the process of learning for computers which is based on it's experiences to do a certain task without further guidance through programs. In other words, we can say that machine learns through initial program and feeds itself the data which obtained from the experiences while executing a particular task. Let's take an example to understand this. A father and a baby went to a park to make the baby learn how to walk. Initially, the father hold the hands of his baby so that the baby can walk without tripping. As the baby can now stand on it's own legs, the father did not hold hands of the baby, thus the baby kept going on and  tripped as stone hit the toes. The baby stood up and learned not to walk over stones. The next