Improving STEM Education via NLP, Visualization, and Mobile Interfaces

The degree and quality of interaction between students and instructors are critical factors for students’ engagement, retention, and learning outcomes across domains (National Research Council, 2012). Although many suggestions and innovations have been proposed, interactive engagement is still very limited between students and instructors. This is especially true for the introductory Science, Technology, Engineering and Math (STEM) courses at the undergraduate level since these courses are generally taught in lecture halls due to a large number of students enrolled (Mervis, 2013). The larger the class size, the less likely instructors are to employ best teaching practices that foster robust learning such as timely feedback and interactive learning activities (Cuseo, 2007). Recent developments in educational technology (i.e. Massive Open Online Courses, blended learning environments) and financial troubles in universities (i.e. budget cuts by states) make it safe to predict that the class size problem will only get worse both in traditional face-to-face and online classes. So how can we modify the passive nature of lectures and increase the interaction while actively involving both students and instructors in the learning process in these circumstances?

In order to address this problem, we present CourseMIRROR (Mobile In-situ Reflections and Review with Optimized Rubrics), a system that integrates Natural Language Processing (NLP) with a mobile application that prompts students to reflect as well as provide immediate and continuous feedback to instructors about the difficulties that their students encounter. By enhancing the student reflection and instructor feedback cycle with technological tools, this project will incorporate three lines of research: 1) role of students’ reflection and instructor’s feedback on students’ retention and learning outcomes; 2) the effectiveness and reliability of NLP to summarize written responses in a meaningful way; and 3) the value and design of mobile technologies to improve retention and learning in STEM domains.

Classroom Adoptions

  • CS2001, University of Pittsburgh.
  • CS2610, University of Pittsburgh.
  • PHYS0175, University of Pittsburgh.
  • IE256, Bogazici University.
  • IE312, Bogazici University.
  • CS0401, University of Pittsburgh.
  • CS1635, University of Pittsburgh.
  • IE256, Bogazici University.
  • MATH125, Thiel College.
  • CS2610, University of Pittsburgh.
  • ENGR132, Purdue University.
  • PSY0422, University of Pittsburgh.


  • Heo, D., Anwar, S., & Menekse, M. (2018). The relationship between engineering students’ achievement goals, reflection behaviors, and learning outcomes. International Journal of Engineering Education, 34(5), 1634-1643 (pdf)
  • Menekse, M., Anwar, S., & Purzer, S. (2018). Self-Efficacy and Mobile Learning Technologies: A Case Study of CourseMIRROR. C. B. Hodges (ed.), Self-Efficacy in Instructional Technology Contexts,  Springer Nature Switzerland AG 2018.
  • Anwar, S., Menekse, M., Heo, D., & Kim, D. (2018). Work-in-Progress: Students’ reflection quality and effective team membership. In Proceedings of the 2018 ASEE Annual Conference, Salt Lake City, Utah. (pdf)
  • Heo, D., Anwar, S., & Menekse, M. (2017). How do engineering students’ achievement goals influence their reflection behaviors and learning outcomes? In Proceedings of the 2017 ASEE Annual Conference, Columbus, Ohio. (pdf)
  • Fan, X., Luo, W., Menekse, M., Litman, D., & Wang, J. (2017). Scaling reflection prompts in large classrooms via mobile interfaces and natural language processing. In Proceedings of 22nd ACM Conference on Intelligent User Interfaces (IUI 2017), Limassol, Cyprus. (pdf)
  • Luo, W., Liu, F., & Litman, D. (2016). An improved phrase-based approach to annotating and summarizing student course responses. In Proceedings of the 26th International Conference on Computational Linguistics (COLING), pp. 53-63, Osaka, Japan. (pdf)
  • Luo, W., & Litman, D. J. (2016). Determining the quality of a student reflective response. In Proceedings 29th International FLAIRS Conference, pp. 226-231, Key Largo, FL. (Best Student Paper Award Nominee) (pdf )
  • Luo, W., Liu, F., Liu, Z., & Litman, D. (2016). Automatic summarization of student course feedback. In Proceedings Conference of the North American Chapter of the Association for Computational Linguistics – Human Language Technologies (NAACL HLT), pp. 80-85, San Diego, CA. (short paper) (pdf )
  • Fan, X., Luo, W., Menekse, M., Litman, D., & Wang, J. (2015). CourseMIRROR: Enhancing large classroom instructor-student interactions via mobile interfaces and natural language processing. Works-In-Progress, In Proceedings of ACM Conference on Human Factors in Computing Systems (CHI 2015), 1473-1478, Seoul, Korea. (extended abstract) (pdf )
  • Luo, W., Fan, X., Menekse, M., Wang, J., & Litman, D. J. (2015). Enhancing instructor-student and student-student interactions with mobile interfaces and summarization. In Proceedings NAACL HLT Companion, 16-20, Denver, Colorado. (demo) (pdf )
  • Luo, W., & Litman, D. J. (2015). Summarizing student responses to reflection prompts. In Proceedings of Empirical Methods in Natural Language Processing (EMNLP ) pp. 1955–1960, Lisbon, Portugal (short paper). (pdf)


  • Fan, X. (2017). Scalable teaching and learning via intelligent user interfaces, (Doctoral Dissertation). University of Pittsburgh. (pdf)
  • Luo, W. (2017) Automatic summarization for student reflective responses, (Doctoral Dissertation). University of Pittsburgh. (pdf )