Source
Adapted from my own handwritten and hand-typed notes from Year 1 to Year 3 of my EEE degree
The running example of the data science part is the Kaggle Pokemon with stats dataset (Alberto Barradas). It is structured, mixed data: 800 rows with numeric and categorical columns.
Variables
| Column | Meaning | Type |
|---|---|---|
# | ID of each Pokemon (1 to 721) | index |
Name | name | string (neither numeric nor factor) |
Type 1 | basic type, sets weakness and resistance | categorical, one of 18 |
Type 2 | second type for dual type Pokemon, else NaN | categorical, 18 or NaN |
Total | sum of all stats, a guide to overall strength | numeric |
HP | hit points, damage it can take before fainting | numeric |
Attack | base modifier for normal attacks | numeric |
Defense | base resistance to normal attacks | numeric |
Sp. Atk | base modifier for special attacks | numeric |
Sp. Def | base resistance to special attacks | numeric |
Speed | decides which Pokemon attacks first | numeric |
Generation | generation 1 to 6 | categorical |
Legendary | rare and powerful or not | categorical (True or False) |
Loading with pandas
import pandas as pd
pkmndata = pd.read_csv('pokemonData.csv')
pkmndata.head() # first 5 rows
pkmndata.shape # (800, 13)
pkmndata.info() # column names, dtypes, missing values
pkmndata['Type 1'].value_counts() # counts of each category
hp = pd.DataFrame(pkmndata['HP']) # one variable as a DataFrameQuestions to Ask
Exploratory analysis begins with:
- what variables are in the data
- how to characterise each variable on its own
- how to find relations between variables
Characterising one variable at a time starts with Univariate Statistics.