# Data Science Handbook

## A Practical Approach

1. Edition November 2022

480 Pages, Hardcover*Wiley & Sons Ltd*

**978-1-119-85733-4**

DATA SCIENCE HANDBOOK

This desk reference handbook gives a hands-on experience on various algorithms and popular techniques used in real-time in data science to all researchers working in various domains.

Data Science is one of the leading research-driven areas in the modern era. It is having a critical role in healthcare, engineering, education, mechatronics, and medical robotics. Building models and working with data is not value-neutral. We choose the problems with which we work, make assumptions in these models, and decide on metrics and algorithms for the problems. The data scientist identifies the problem which can be solved with data and expert tools of modeling and coding.

The book starts with introductory concepts in data science like data munging, data preparation, and transforming data. Chapter 2 discusses data visualization, drawing various plots and histograms. Chapter 3 covers mathematics and statistics for data science. Chapter 4 mainly focuses on machine learning algorithms in data science. Chapter 5 comprises of outlier analysis and DBSCAN algorithm. Chapter 6 focuses on clustering. Chapter 7 discusses network analysis. Chapter 8 mainly focuses on regression and naive-bayes classifier. Chapter 9 covers web-based data visualizations with Plotly. Chapter 10 discusses web scraping.

The book concludes with a section discussing 19 projects on various subjects in data science.

Audience

The handbook will be used by graduate students up to research scholars in computer science and electrical engineering as well as industry professionals in a range of industries such as healthcare.

Preface xiii

1 Data Munging Basics

1 Introduction 1

1.1 Filtering and Selecting Data 6

1.2 Treating Missing Values 11

1.3 Removing Duplicates 14

1.4 Concatenating and Transforming Data 16

1.5 Grouping and Data Aggregation 20

References 20

2 Data Visualization 23

2.1 Creating Standard Plots (Line, Bar, Pie) 26

2.2 Defining Elements of a Plot 30

2.3 Plot Formatting 33

2.4 Creating Labels and Annotations 38

2.5 Creating Visualizations from Time Series Data 42

2.6 Constructing Histograms, Box Plots, and Scatter Plots 44

References 54

3 Basic Math and Statistics 57

3.1 Linear Algebra 57

3.2 Calculus 58

3.2.1 Differential Calculus 58

3.2.2 Integral Calculus 58

3.3 Inferential Statistics 60

3.3.1 Central Limit Theorem 60

3.3.2 Hypothesis Testing 60

3.3.3 ANOVA 60

3.3.4 Qualitative Data Analysis 60

3.4 Using NumPy to Perform Arithmetic Operations on Data 61

3.5 Generating Summary Statistics Using Pandas and Scipy 64

3.6 Summarizing Categorical Data Using Pandas 68

3.7 Starting with Parametric Methods in Pandas and Scipy 84

3.8 Delving Into Non-Parametric Methods Using Pandas and Scipy 87

3.9 Transforming Dataset Distributions 91

References 94

4 Introduction to Machine Learning 97

4.1 Introduction to Machine Learning 97

4.2 Types of Machine Learning Algorithms 101

4.3 Explanatory Factor Analysis 114

4.4 Principal Component Analysis (PCA) 115

References 121

5 Outlier Analysis 123

5.1 Extreme Value Analysis Using Univariate Methods 123

5.2 Multivariate Analysis for Outlier Detection 125

5.3 DBSCan Clustering to Identify Outliers 127

References 133

6 Cluster Analysis 135

6.1 K-Means Algorithm 135

6.2 Hierarchial Methods 141

6.3 Instance-Based Learning w/ k-Nearest Neighbor 149

References 156

7 Network Analysis with NetworkX 157

7.1 Working with Graph Objects 159

7.2 Simulating a Social Network (ie; Directed Network Analysis) 163

7.3 Analyzing a Social Network 169

References 171

8 Basic Algorithmic Learning 173

8.1 Linear Regression 173

8.2 Logistic Regression 183

8.3 Naive Bayes Classifiers 189

References 195

9 Web-Based Data Visualizations with Plotly 197

9.1 Collaborative Aanalytics 197

9.2 Basic Charts 208

9.3 Statistical Charts 212

9.4 Plotly Maps 216

References 219

10 Web Scraping with Beautiful Soup 221

10.1 The BeautifulSoup Object 224

10.2 Exploring NavigableString Objects 228

10.3 Data Parsing 230

10.4 Web Scraping 233

10.5 Ensemble Models with Random Forests 235

References 254

Data Science Projects 257

11 Covid19 Detection and Prediction 259

Bibliography 275

12 Leaf Disease Detection 277

Bibliography 283

13 Brain Tumor Detection with Data Science 285

Bibliography 295

14 Color Detection with Python 297

Bibliography 300

15 Detecting Parkinson's Disease 301

Bibliography 302

16 Sentiment Analysis 303

Bibliography 306

17 Road Lane Line Detection 307

Bibliography 315

18 Fake News Detection 317

Bibliography 318

19 Speech Emotion Recognition 319

Bibliography 322

20 Gender and Age Detection with Data Science 323

Bibliography 339

21 Diabetic Retinopathy 341

Bibliography 350

22 Driver Drowsiness Detection in Python 351

Bibliography 356

23 Chatbot Using Python 357

Bibliography 363

24 Handwritten Digit Recognition Project 365

Bibliography 368

25 Image Caption Generator Project in Python 369

Bibliography 379

26 Credit Card Fraud Detection Project 381

Bibliography 391

27 Movie Recommendation System 393

Bibliography 411

28 Customer Segmentation 413

Bibliography 431

29 Breast Cancer Classification 433

Bibliography 443

30 Traffic Signs Recognition 445

Bibliography 453