User:Matthew

Tuesday, May 30

met new people
heard some people talk
ate some panera
heard more talking
Started to get up to speed with TA Bot project.
In particular, I read a paper (``Experiences with TA-Bot in CS1") highlighting experiences with using TA Bot for Marquette CS classes, and read a survey that was given out to assess the effectiveness of the TA Bot.
- starting to understand the leveling system: it only concerns test cases, and the levels represent difficulty of the tests (higher levels corner/edge cases for example)
- big emphasis on the TBS system to encourage students to start work early, but many did not like it
- how are the learning outcomes affected? how do we measure good learning outcomes
Also, got started with looking at the database of submission scores from TA Bot, and am beginning to look at Python libraries that will help me manipulate this data.
- using base code as a reference
- pandas library: I understand the very basics of series and data frames
- need to understand sorting/grouping/splitting

heard some talking
getting comfortable with `pandas` and getting pertinent parts of the TA Bot submissions database
brainstorming ideas for comparisons/visualizations we want between TBS and non TBS semesters to assess positive/negative student outcomes
- right now, we focus on the effects on linter errors: how much linter errors go down using TBS vs no TBS, and if students correct linter errors even after attaining 100% on an assignment
made a graph comparing the average reduction in linter errors from a student's first submission to his last per assignment with TBS vs. no TBS
- a clear correlation in assignments 1-5 that showed that TBS had a higher reduction in linter errors
- assignments 6-10 are not so clear. Brylow: either students aren't making as many errors or they are just not correcting them
some other data gathered, needing visualizations
- students submit far fewer times on average using TBS for a given assignment
- students tend to resubmit more often after reaching 100% without TBS though the numbers are both low
- we also studied the number of linter error reductions after reaching 100% w//w/o TBS, but the data does not make entirely clear any overarching trends (that might also help explain the 1-5/6-10 disparity)

refactored visualization code
made visualizations of data from two more semesters
- those semesters did not use TBS
- improvements not very obvious, but the two new semesters did not use the same projects, so other factors may be at play
thought of idea for new visualizations
- looking at percent change of linter errors reduced instead of just the number reduced
- instead of comparing to a student's first submission (may be a test or a mess, which is unreliable) look at submissions beyond a certain scoring threshold (like 70%)
meeting with Dr. Islam
read the following papers studying failure rates of introductory CS courses:
- ``My Program is Correct But it Doesn’t Run: A Preliminary Investigation of Novice Programmers’ Problems"
- ``Failure Rates in Introductory Programming Revisited"
- ``Pass Rates in Introductory Programming and in other STEM Disciplines"
- ``Failure Rates in Introductory Programming — 12 Years Later"

created new visualizations comparing reductions in pylint errors between submissions that score 70% or more, and submissions that are passing
- clear data that suggests TBS is helping reduce more pylint errors
- further work needs to be studied on the later assignments: is TBS helping students to create fewer linter errors in the later semesters (so that they wouldn't have many to fix)
made pie graphs representing percentage of students with passing submissions who resubmitted
- total is about 20%. need further analysis on who these people are
- comparing number of linter errors with people who submitted only once vs multiple times
RCR training