{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Palmer Penguins: leakage-safe classification\n",
    "Use all 344 observations to predict `species`. Preserve the final test set until the workflow is selected. Record every exclusion and keep claims within this dataset's population and collection context."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "from sklearn.compose import ColumnTransformer\n",
    "from sklearn.dummy import DummyClassifier\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn.impute import SimpleImputer\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.metrics import classification_report, confusion_matrix\n",
    "from sklearn.model_selection import StratifiedKFold, cross_validate, train_test_split\n",
    "from sklearn.pipeline import Pipeline\n",
    "from sklearn.preprocessing import OneHotEncoder, StandardScaler\n",
    "RANDOM_STATE = 42"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "DATA_URL = 'https://intuitiveai.dev/data/penguins.csv'\n",
    "penguins = pd.read_csv(DATA_URL)\n",
    "display(penguins.head())\n",
    "print('shape:', penguins.shape)\n",
    "print('duplicates:', penguins.duplicated().sum())\n",
    "display(pd.DataFrame({'dtype': penguins.dtypes.astype(str), 'missing': penguins.isna().sum(), 'unique': penguins.nunique()}))\n",
    "display(penguins['species'].value_counts())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prediction contract\n",
    "Write: one row, target, prediction time, intended user, intended use, non-goals, feature availability, error costs, and why species classification is a teaching task rather than a deployment claim."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "target = 'species'\n",
    "features = ['island', 'bill_length_mm', 'bill_depth_mm', 'flipper_length_mm', 'body_mass_g', 'sex']\n",
    "X = penguins[features]\n",
    "y = penguins[target]\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.20, stratify=y, random_state=RANDOM_STATE)\n",
    "print(X_train.shape, X_test.shape, y_train.value_counts(normalize=True).round(3).to_dict())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "numeric = ['bill_length_mm', 'bill_depth_mm', 'flipper_length_mm', 'body_mass_g']\n",
    "categorical = ['island', 'sex']\n",
    "preprocess = ColumnTransformer([\n",
    "    ('numeric', Pipeline([('impute', SimpleImputer(strategy='median')), ('scale', StandardScaler())]), numeric),\n",
    "    ('categorical', Pipeline([('impute', SimpleImputer(strategy='most_frequent')), ('encode', OneHotEncoder(handle_unknown='ignore'))]), categorical),\n",
    "])\n",
    "models = {\n",
    "    'dummy': DummyClassifier(strategy='most_frequent'),\n",
    "    'logistic': LogisticRegression(max_iter=2000),\n",
    "    'forest': RandomForestClassifier(n_estimators=300, min_samples_leaf=3, random_state=RANDOM_STATE),\n",
    "}\n",
    "cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=RANDOM_STATE)\n",
    "rows = []\n",
    "for name, model in models.items():\n",
    "    pipeline = Pipeline([('preprocess', preprocess), ('model', model)])\n",
    "    result = cross_validate(pipeline, X_train, y_train, cv=cv, scoring=['accuracy', 'f1_macro'])\n",
    "    for fold, (accuracy, f1) in enumerate(zip(result['test_accuracy'], result['test_f1_macro']), 1):\n",
    "        rows.append({'model': name, 'fold': fold, 'accuracy': accuracy, 'f1_macro': f1})\n",
    "cv_results = pd.DataFrame(rows)\n",
    "display(cv_results)\n",
    "display(cv_results.groupby('model')[['accuracy', 'f1_macro']].agg(['mean', 'std']))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Freeze the choice\n",
    "Choose one candidate from cross-validation evidence, complexity, and the teaching objective. State the choice before running the next cell."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "selected = Pipeline([('preprocess', preprocess), ('model', LogisticRegression(max_iter=2000))])\n",
    "selected.fit(X_train, y_train)\n",
    "predictions = selected.predict(X_test)\n",
    "print(classification_report(y_test, predictions))\n",
    "print(confusion_matrix(y_test, predictions, labels=selected.classes_))\n",
    "errors = X_test.assign(actual=y_test, predicted=predictions).query('actual != predicted')\n",
    "display(errors)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Error analysis and model card\n",
    "1. Inspect each error and performance by island and sex where sample size supports it.\n",
    "2. Separate measurement ambiguity, sparse slices, and model limitations.\n",
    "3. Write intended use, non-goals, data provenance, metrics, limitations, ethical considerations, monitoring, and a next experiment.\n",
    "4. Export this notebook or a report. Do not claim production readiness or biological generalization beyond the evidence."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {"display_name": "Python 3", "language": "python", "name": "python3"},
  "language_info": {"name": "python", "version": "3.11"}
 },
 "nbformat": 4,
 "nbformat_minor": 5
}