Answer to Question #294799 in Python for DYuthi

Question

preprocess the data and by giving a proper rationale behind the steps you would follow. The two datasets should be merged before preprocessing. first task in your new job, to check the quality of the data for the next step of analysis. Following are the additional description

a)    The data set contains six variables - Date,Open,Close,HIgh,Low,volume

                                      

the stock market opens at 9:15 hours and closes at 15:30 hours. Each stock is defined by an opening and a closing price which are the prices it opens and closes with. Within the operating hours, the stock price touches a maximum and minimum which are the highest and lowest prices achieved by the stock in the working hours of the stock market. You have access to ten years of monthly stock price data . On some days when there is no trading.Furthermore, your manager also claims that the model prediction is too bad since data is polluted. 

Answer 1

To work with the dataset, you need to do the following steps:

- Data cleaning is the process to remove incorrect data, incomplete data and inaccurate data from the datasets, and it also replaces the missing values.

- Data integration. The process of combining multiple sources into a single dataset. The Data integration process is one of the main components in data management.

- Data reduction. This process helps in the reduction of the volume of the data which makes the analysis easier yet produces the same or almost the same result. This reduction also helps to reduce storage space. There are some of the techniques in data reduction are Dimensionality reduction, Numerosity reduction, Data compression.

- Data Transformation. The change made in the format or the structure of the data is called data transformation. This step can be simple or complex based on the requirements.