|
909 | 909 | "cell_type": "markdown", |
910 | 910 | "metadata": {}, |
911 | 911 | "source": [ |
912 | | - "## 📏 How Do We Evaluate Classification Models?\n", |
913 | | - "\n", |
914 | | - "We evaluate classification models using various metrics to understand their performance. Common evaluation metrics include accuracy, precision, recall, F1-score and Mathew's correlation coefficient (MCC) based on the confusion matrix.\n", |
915 | | - "\n", |
916 | | - "### Confusion Matrix:\n", |
917 | | - "- A table summarizing true positives, false positives, true negatives, and false negatives.\n", |
918 | | - "\n", |
919 | | - "| | Predicted Positive | Predicted Negative |\n", |
920 | | - "|----------------|---------------------|---------------------|\n", |
921 | | - "| **Actual Positive** | True Positive (TP) | False Negative (FN) |\n", |
922 | | - "| **Actual Negative** | False Positive (FP) | True Negative (TN) |\n", |
923 | | - "\n", |
924 | | - "**True Positive (TP)**: Correctly predicted positive samples.\n", |
925 | | - "\n", |
926 | | - "**False Positive (FP)**: Incorrectly predicted positive samples.\n", |
927 | | - "\n", |
928 | | - "**True Negative (TN)**: Correctly predicted negative samples.\n", |
929 | | - "\n", |
930 | | - "**False Negative (FN)**: Incorrectly predicted negative samples.\n", |
931 | | - "\n", |
932 | | - "### 1️⃣ Accuracy:\n", |
933 | | - "- Percentage of correctly predicted samples.\n", |
934 | | - "- Formula: Accuracy = \\(\\frac{TP + TN}{TP + TN + FP + FN}\\)\n", |
935 | | - "- Limitation: Can be misleading for imbalanced datasets.\n", |
936 | | - "\n", |
937 | | - "---\n", |
938 | | - "\n", |
939 | | - "### 2️⃣ Precision:\n", |
940 | | - "- Of all the predicted positives, how many are truly positive?\n", |
941 | | - "- Formula:\n", |
942 | | - " \\[\n", |
943 | | - " \\text{Precision} = \\frac{\\text{True Positives}}{\\text{True Positives} + \\text{False Positives}}\n", |
944 | | - " \\]\n", |
945 | | - "\n", |
946 | | - "---\n", |
947 | | - "\n", |
948 | | - "### 3️⃣ Recall (Sensitivity):\n", |
949 | | - "- Of all the actual positives, how many were correctly predicted?\n", |
950 | | - "- Formula:\n", |
951 | | - " \\[\n", |
952 | | - " \\text{Recall} = \\frac{\\text{True Positives}}{\\text{True Positives} + \\text{False Negatives}}\n", |
953 | | - " \\]\n", |
954 | | - "\n", |
955 | | - "---\n", |
956 | | - "\n", |
957 | | - "### 4️⃣ F1-Score:\n", |
958 | | - "- Harmonic mean of precision and recall. Useful when you want to balance both.\n", |
959 | | - "- Formula:\n", |
960 | | - " \\[\n", |
961 | | - " \\text{F1-Score} = 2 \\times \\frac{\\text{Precision} \\times \\text{Recall}}{\\text{Precision} + \\text{Recall}}\n", |
962 | | - " \\]\n", |
963 | | - "\n", |
964 | | - "---\n", |
965 | | - "\n", |
966 | | - "### 5️⃣ Confusion Matrix:\n", |
967 | | - "- A table summarizing true positives, false positives, true negatives, and false negatives.\n", |
968 | | - "\n", |
969 | | - "| | Predicted Positive | Predicted Negative |\n", |
970 | | - "|----------------|---------------------|---------------------|\n", |
971 | | - "| **Actual Positive** | True Positive (TP) | False Negative (FN) |\n", |
972 | | - "| **Actual Negative** | False Positive (FP) | True Negative (TN) |\n" |
| 912 | + "## In-class Activity:\n", |
| 913 | + "- Implement another classification algorithm using a different dataset.\n", |
| 914 | + "- Use scikit-learn to choose dataset and new classification algorithm.\n", |
| 915 | + "- Datasets: [Toy datasets](https://scikit-learn.org/1.5/datasets/toy_dataset.html)\n", |
| 916 | + "- Algorithms: [Scikit-learn algorithms](https://scikit-learn.org/stable/supervised_learning.html#supervised-learning)" |
973 | 917 | ] |
974 | 918 | } |
975 | 919 | ], |
|
989 | 933 | "name": "python", |
990 | 934 | "nbconvert_exporter": "python", |
991 | 935 | "pygments_lexer": "ipython3", |
992 | | - "version": "3.11.5" |
| 936 | + "version": "3.11.10" |
993 | 937 | } |
994 | 938 | }, |
995 | 939 | "nbformat": 4, |
|
0 commit comments