SPS Learning Analytics Initiative

Project 1: Discussion Board Analytics

This project is part of the University wide Experimental Teaching and Learning Analytics group organized by Northwestern Information Technology.

Goals

  • Use Natural Language Processing tools to identify discussion themes
  • Develop student and faculty facing reports of discussion activity
  • Use data reports on discussion participation to improve course design

Findings

Natural Language Processing – Presto Discussion Analytics

The Presto Canvas Discussion Analytics LTI, developed by Northwestern Information Technology, integrates with Canvas and performs keyword extraction for keywords, places, and people mentioned in the discussion as well as sentiment analysis. It also collects statistics on participation by user and time, as well as performs a reading level analysis of the text.

Presto was deployed in the Fall 2015 courses in the Master of Science in Global Health program: MSGH 405, 408, 417, 427, and 452. A summary of the findings appears below.

405 408 417 427 452 Overall Average
Average Posts Per Student 38.96 60.73 35.42 13.04 50.53 39.736
Average Word Count Per Post 241.75 76.59 365.38 114.76 216.13 202.922
Average Reading Level 12.167 10.309 12.052 11.15 11.629 11.4614

Glossary Terms

Prior to the first term of the MSGH program, MSGH faculty collaborated to create a glossary of terms they wanted students to become familiar with throughout their time in the MSGH program. The following tables show which terms from the glossary were used within each course’s discussions.

Most-Used Glossary Terms – Overall

Term Mentions
Nutrition 451
Public Health 300
Control 263
Mortality 262
Obesity 168
Prevalence 156
Infectious Disease 137
Burden of Disease 126
Epidemic 117
Incidence 117

Unused Glossary Terms

Ecological Amplifier
Genotypic Differentiation
Gross Happiness Indicators
Leveraging of Resources
Vaccine Preventable Diseases

 Most Used Glossary Terms by Course

405 408 417 427 452
Mortality 138 Control 72 Control 85 Nutrition 21 Nutrition 243
Public Health 126 Incidence 69 Public Health 82 Public Health 18 Obesity 69
Nutrition 97 Prevalence 64 Nutrition 62 Partnerships 18 Partnerships 51
Control 77 Epidemiology 63 Mortality 42 Mortality 17 Public Health 48
Infectious Disease 77 Epidemic 48 Burden of Disease 36 Infectious Disease 9 Mortality 32
Burden of Disease 77 Outbreak 46 Social Determinants of Health 32 Control 6 Control 23
Obesity 54 Pandemic 38 Chronic Disease 32 Immunization 6 Diabetes 20
DALYs 53 Mortality 33 Diabetes 31 Body Mass Index (BMI) 5 Prevalence 18
Prevalence 42 Distribution 29 Prevalence 29 Neglected Tropical Disease 5 Epidemiology 18
Morbidity 38 Nutrition 28 Indicators 28 5 Words With 3 Indicators 17

Count of Countries Mentioned

Foundations of Global Health (MSGH 405) is an overview course and provides an introduction to worldwide health issues; the map below shows the frequency of mentions of various countries on the course discussion board.

Organizations Mentioned

The Presto Discussion Analytics tool pulls the names of organizations from the discussion. The following organizations were mentioned most frequently in MSGH 405.

Organization Mentions
World Health Organization 397
United Nations 99
Lancet 94
NAMI 35
CDC 22
New York Times 22

Next steps

  • Use Presto to see which course resources and authors students are citing most in the discussions.
  • Collaborate with faculty and learning designers to create useful discussion analytics reports on request.