Introduction
Machine learning and statistics have always been closely related to each other. This led to an argument about whether it was different from machine learning or formed a part of machine learning. Several Machine learning courses specify statistics as one of the perquisites for machine learning.
Hence, we need to develop an understanding of the fact if statistics relate to machine learning and if it does, how?
Individuals working in the field of machine learning concentrate on the task of model building and the result interpretation from the model that was constructed while the statisticians perform the same task but under the cover of a mathematician concentrating more on the mathematical theory involved in the machine learning task concentrating more on the explanation of the predictions made by the machine learning model. So, we can say that in spite of the differences between statistics and machine learning, we need to learn statistics in machine learning.
Statistics and machine learning
Both statistics and machine learning are related to data. Although they work with the data in their way, some requirements are needed by both and hence they form a close relationship with each other. Given below is a step by step analysis as to how statistics relate to machine learning.
Data preprocessing requires statistics
To proceed with the machine learning task, cleaning of data is a mandatory step. This process involves tasks such as identifying missing values, normalization of the values, identifying the outliers, etc. These operations call for statistical concepts such as distributions, mean, median, mode etc.
Model construction and statistics
After the data has been cleaned, the next step is to build a model with that data. A hypothesis test might be needed for model construction which calls for good statistical concepts.
Statistics in evaluation
Model evaluation requires tasks such as validation techniques to be performed so that the accuracy and model performance increases. These validation techniques are easily understood by the statisticians but a bit difficult for the machine learners to interpret as it involves mathematical concepts.
Presenting the model
After the successful construction and evaluation of the model, the model is presented to the general public. The interpretation of results requires a good understanding of concepts such as confidence interval, quantification, an average of the predicted results based on outputs produced and so on.
Other than the above-mentioned steps some additional concepts must be adhered to while working with machine learning. Some of these concepts are listed below:
- Gaussian distribution – It is often represented by a bell-shaped curve. The bell-shaped curve plays a very important role while normalising the data as a normalised data is supposed to lie at the point where the bell-shaped curve is divided into two equal parts.
- Correlation- It can be either positive, negative or neutral. A positive correlation indicates that the values change in the same manner(positive causes positive and negative leads to negative). A negative correlation indicates values change oppositely while neural suggests no relationship. This concept is of great importance to the analysts while identifying the tendencies in the data.
- Hypothesis- An assumption might be done for the elementary predictive analysis in machine learning that requires a good understanding of the hypothesis.
- Probability – Probability plays an important role in predicting the possible class values in classification tasks and hence forms an important part in machine learning.
Conclusion
Statistics is of huge importance to machine learning, especially in the analysis field. It is one of the key concepts for data visualization and pattern recognition. It is widely used in regression and classification and helps in establishing a relationship between data points. Hence, statistics and machine learning go hand in hand.
