User:Matthew

Tuesday, May 30

• met new people
• heard some people talk
• ate some panera
• heard more talking
• Started to get up to speed with TA Bot project.
• In particular, I read a paper (``Experiences with TA-Bot in CS1") highlighting experiences with using TA Bot for Marquette CS classes, and read a survey that was given out to assess the effectiveness of the TA Bot.
  • starting to understand the leveling system: it only concerns test cases, and the levels represent difficulty of the tests (higher levels corner/edge cases for example)
  • big emphasis on the TBS system to encourage students to start work early, but many did not like it
  • how are the learning outcomes affected? how do we measure good learning outcomes
• Also, got started with looking at the database of submission scores from TA Bot, and am beginning to look at Python libraries that will help me manipulate this data.
  • using base code as a reference
  • pandas library: I understand the very basics of series and data frames
  • need to understand sorting/grouping/splitting

May 31, 2023

• heard some talking
• getting comfortable with `pandas` and getting pertinent parts of the TA Bot submissions database
• brainstorming ideas for comparisons/visualizations we want between TBS and non TBS semesters to assess positive/negative student outcomes
  • right now, we focus on the effects on linter errors: how much linter errors go down using TBS vs no TBS, and if students correct linter errors even after attaining 100% on an assignment
• made a graph comparing the average reduction in linter errors from a student's first submission to his last per assignment with TBS vs. no TBS
  • a clear correlation in assignments 1-5 that showed that TBS had a higher reduction in linter errors
  • assignments 6-10 are not so clear. Brylow: either students aren't making as many errors or they are just not correcting them
• some other data gathered, needing visualizations
  • students submit far fewer times on average using TBS for a given assignment
  • students tend to resubmit more often after reaching 100% without TBS though the numbers are both low
  • we also studied the number of linter error reductions after reaching 100% w//w/o TBS, but the data does not make entirely clear any overarching trends (that might also help explain the 1-5/6-10 disparity)

1 June 2023

• refactored visualization code
• made visualizations of data from two more semesters
  • those semesters did not use TBS
  • improvements not very obvious, but the two new semesters did not use the same projects, so other factors may be at play
• thought of idea for new visualizations
  • looking at percent change of linter errors reduced instead of just the number reduced
  • instead of comparing to a student's first submission (may be a test or a mess, which is unreliable) look at submissions beyond a certain scoring threshold (like 70%)
• meeting with Dr. Islam
• read the following papers studying failure rates of introductory CS courses:
  • ``My Program is Correct But it Doesn’t Run: A Preliminary Investigation of Novice Programmers’ Problems"
  • ``Failure Rates in Introductory Programming Revisited"
  • ``Pass Rates in Introductory Programming and in other STEM Disciplines"
  • ``Failure Rates in Introductory Programming — 12 Years Later"