Source

Adapted from my own handwritten and hand-typed notes from Year 1 to Year 3 of my EEE degree

The running example of the data science part is the Kaggle Pokemon with stats dataset (Alberto Barradas). It is structured, mixed data: 800 rows with numeric and categorical columns.

Variables

ColumnMeaningType
#ID of each Pokemon (1 to 721)index
Namenamestring (neither numeric nor factor)
Type 1basic type, sets weakness and resistancecategorical, one of 18
Type 2second type for dual type Pokemon, else NaNcategorical, 18 or NaN
Totalsum of all stats, a guide to overall strengthnumeric
HPhit points, damage it can take before faintingnumeric
Attackbase modifier for normal attacksnumeric
Defensebase resistance to normal attacksnumeric
Sp. Atkbase modifier for special attacksnumeric
Sp. Defbase resistance to special attacksnumeric
Speeddecides which Pokemon attacks firstnumeric
Generationgeneration 1 to 6categorical
Legendaryrare and powerful or notcategorical (True or False)

Loading with pandas

import pandas as pd
 
pkmndata = pd.read_csv('pokemonData.csv')
pkmndata.head()          # first 5 rows
pkmndata.shape           # (800, 13)
pkmndata.info()          # column names, dtypes, missing values
pkmndata['Type 1'].value_counts()      # counts of each category
hp = pd.DataFrame(pkmndata['HP'])      # one variable as a DataFrame

Questions to Ask

Exploratory analysis begins with:

  • what variables are in the data
  • how to characterise each variable on its own
  • how to find relations between variables

Characterising one variable at a time starts with Univariate Statistics.