User:Matthew

From REU@MU
Revision as of 02:07, 2 June 2023 by Matthew (Talk | contribs)

Jump to: navigation, search

Tuesday, May 30

  • met new people
  • heard some people talk
  • ate some panera
  • heard more talking
  • Started to get up to speed with TA Bot project.
  • In particular, I read a paper (``Experiences with TA-Bot in CS1") highlighting experiences with using TA Bot for Marquette CS classes, and read a survey that was given out to assess the effectiveness of the TA Bot.
    • starting to understand the leveling system: it only concerns test cases, and the levels represent difficulty of the tests (higher levels corner/edge cases for example)
    • big emphasis on the TBS system to encourage students to start work early, but many did not like it
    • how are the learning outcomes affected? how do we measure good learning outcomes
  • Also, got started with looking at the database of submission scores from TA Bot, and am beginning to look at Python libraries that will help me manipulate this data.
    • using base code as a reference
    • pandas library: I understand the very basics of series and data frames
    • need to understand sorting/grouping/splitting

May 31, 2023

  • heard some talking
  • getting comfortable with `pandas` and getting pertinent parts of the TA Bot submissions database
  • brainstorming ideas for comparisons/visualizations we want between TBS and non TBS semesters to assess positive/negative student outcomes
    • right now, we focus on the effects on linter errors: how much linter errors go down using TBS vs no TBS, and if students correct linter errors even after attaining 100% on an assignment
  • made a graph comparing the average reduction in linter errors from a student's first submission to his last per assignment with TBS vs. no TBS
    • a clear correlation in assignments 1-5 that showed that TBS had a higher reduction in linter errors
    • assignments 6-10 are not so clear. Brylow: either students aren't making as many errors or they are just not correcting them
  • some other data gathered, needing visualizations
    • students submit far fewer times on average using TBS for a given assignment
    • students tend to resubmit more often after reaching 100% without TBS though the numbers are both low
    • we also studied the number of linter error reductions after reaching 100% w//w/o TBS, but the data does not make entirely clear any overarching trends (that might also help explain the 1-5/6-10 disparity)

1 June 2023

  • refactored visualization code
  • made visualizations of data from two more semesters
    • those semesters did not use TBS
    • improvements not very obvious, but the two new semesters did not use the same projects, so other factors may be at play
  • thought of idea for new visualizations
    • looking at percent change of linter errors reduced instead of just the number reduced
    • instead of comparing to a student's first submission (may be a test or a mess, which is unreliable) look at submissions beyond a certain scoring threshold (like 70%)
  • meeting with Dr. Islam
  • read the following papers studying failure rates of introductory CS courses:
    • ``My Program is Correct But it Doesn’t Run: A Preliminary Investigation of Novice Programmers’ Problems"
    • ``Failure Rates in Introductory Programming Revisited"
    • ``Pass Rates in Introductory Programming and in other STEM Disciplines"
    • ``Failure Rates in Introductory Programming — 12 Years Later"