Week Introduction
This week focuses on evaluating AI and ML models using various metrics. Students will learn about different evaluation metrics and their applications, performing a comparative analysis on a dataset selected by the student.
Week Objectives
At the end of this week students will be able to:
· Describe different evaluation metrics for ML models. (CO2)
· Apply various evaluation metrics to a classification problem. (CO3)
· Compare the effectiveness of different evaluation metrics. (CO4)
Online Lecture
The instructor will post the video recorded from this week's synchronous session here.
Below is an outline of the items for which you will be responsible throughout the week.
Reading with a due date
DUE: Early in the week
12 Important Model Evaluation Metrics for Machine Learning Everyone Should Know (Updated 2023)
https:
www.analyticsvidhya.com
log/2019/08/11-important-model-evaluation-e
or-metrics/Links to an external site.
Evaluating the Quality of Machine Learning Explanations: A Survey on Methods and Metrics
https:
www.mdpi.com/ XXXXXXXXXX/10/5/593/pdfLinks to an external site.
Model Evaluation Techniques in Machine Learning (python example at end)
https:
medium.com/@fatmanurkutlu1/model-evaluation-techniques-in-machine-learning-8cd88deb8655Links to an external site.
Lecture with a due date
DUE: Early in the week
How to evaluate ML models | Evaluation metrics for machine learning
https:
www.youtube.com/watch?v=LbX4X71-TFILinks to an external site.
Metrics and evaluation of machine learning models
https:
www.youtube.com/watch?v=u1-m_hsF7DELinks to an external site.
Evaluating Machine Learning Models
https:
www.youtube.com/watch?v=FeKSQy5t_TILinks to an external site.
Evaluating Classification and Regression Machine Learning Models
https:
www.youtube.com/watch?v=pyl6fO4C7h4Links to an external site.
Additional Resources (reading + videos)
No Due date
No additional resources.
Submit your completed written assignment by Day 7 of this week. For detailed instructions on completing this assignment, see the associated course page.
Week 6 Assignment – DUE: Day 7
Week 6 Assignment
Instructions:
Conduct a comparative analysis of different evaluation metrics for a classification problem. Use a dataset and demonstrate the application of at least three metrics. Submit your code AND a separate 1-page report on your approach and results. Use Jupyter notebooks for coding.
Your essay should be at least 2-3 pages in length, not including cover sheet and reference page, and fully explore all of the following items described above. Include at least 2 outside citations and use proper APA formatting.
This assignment is worth 100 points towards the maximum 1460 points you can earn in class.