https://reu.cs.mu.edu/api.php?action=feedcontributions&user=Crepaci&feedformat=atomREU@MU - User contributions [en]2024-03-19T06:01:30ZUser contributionsMediaWiki 1.23.13https://reu.cs.mu.edu/index.php/File:Slide17.pngFile:Slide17.png2020-08-07T19:51:26Z<p>Crepaci: </p>
<hr />
<div></div>Crepacihttps://reu.cs.mu.edu/index.php/Developing_Ethical_Algorithms_for_Placement_Stability_in_the_Foster_Care_SystemDeveloping Ethical Algorithms for Placement Stability in the Foster Care System2020-08-07T19:49:46Z<p>Crepaci: /* Final Presentation: */</p>
<hr />
<div>Student: [[User:Crepaci|Charlie Repaci]]<br />
<br />
Mentor: [https://www.shionguha.net/ Dr. Shion Guha] and [https://saxena.io/ Devansh Saxena]<br />
<br />
==Description:==<br />
The goal of this project is to use a human-centered approach grounded in current social science theory and frameworks to add context to and further develop existing placement stability and risk assessment models that are used to aid overworked social workers in making, explaining, and standardizing their decisions.<br />
<br />
==Goals:==<br />
Quoted from the [https://reu.cs.mu.edu/index.php/Ethical_Algorithms_for_Placement_Stability_in_the_Foster_Care project page:]<br />
<br />
<blockquote>''This project aims to collaborate with the WI Department of Children and Families and SaintA (a private non-profit organization that actually provides foster care services in WI) by utilizing important, useful and contextual caseworkers judgment that are recorded as detailed case notes but never actually used to add norms, values, and context to existing algorithms that determine placement stability. Topic modeling will be used to extract latent themes from such text and incrementally added to existing placement stability models to test improvements in outcomes.''<br />
* ''Perform a literature review of topic modeling usage in the social science domain.''<br />
* ''Understand latent and human context from caseworker notes in Wisconsin foster care system using the data provided from the mentor.''<br />
* ''Review algorithmic biases, fairness, and transparency in the foster care system.''<br />
* ''Evaluate the latent themes from text and incrementally add to existing placement stability models to test improvement in outcomes.''</blockquote><br />
<br />
==Schedule:==<br />
{| class="wikitable"<br />
|-<br />
|'''Week'''<br />
|'''Description'''<br />
|-<br />
|Week 1<br />
|<br />
* Orientation<br />
* Data Science Bootcamp<br />
* Good Research Practices Lecture<br />
* Begin literature review -- algorithms in social work<br />
|-<br />
|Week 2<br />
|<br />
* Responsible Conduct of Research Training<br />
* CITI certification<br />
* Technical writing workshop<br />
* Continue literature review -- ethics and algorithms<br />
|-<br />
|Week 3<br />
|<br />
* Set up wiki<br />
* Begin reviewing data<br />
* Continue literature review -- bias in algorithms and policy<br />
|-<br />
|Week 4<br />
|<br />
* Import data<br />
* Research presentation by guest lecturer Dr. Walt Bialkowski<br />
* Presentation of work done in weeks 3 and 4 to mentors<br />
|-<br />
|Week 5<br />
|<br />
* Research Presentations lecture<br />
* Data exploration and visualization<br />
* Technical presentations workshop and informal presentations<br />
|-<br />
|Week 6<br />
|<br />
* Data Ethics by guest lecturer Dr. Michael Zimmer<br />
* Making Research Posters lecture<br />
* Continued exploratory work and visualizations to highlight important factors<br />
* Continue literature review -- bias in algorithm development<br />
|-<br />
|Week 7<br />
|<br />
* Looking more deeply into Phase I<br />
* Begin WARM algorithm audit<br />
* Continue literature review -- gender diversity and algorithms<br />
|-<br />
|Week 8<br />
|<br />
* WARM algorithm audit<br />
* Continue literature review -- gender diversity and algorithms<br />
* Graduate Schools discussion by Dr. Brylow<br />
|-<br />
|Week 9<br />
|<br />
* Industry guest panel (Northwestern Mutual Data Science Institute)<br />
* Begin the final paper<br />
* Start to create project poster<br />
* WARM algorithm audit<br />
|-<br />
|Week 10<br />
|<br />
* Prepare and give the oral presentation<br />
* Present project to other REU sites and see their research in return<br />
* Finish and submit the final paper<br />
|}<br />
<br />
==Project Poster:==<br />
<br />
[[File:Project Poster.png|center|1000px]]<br />
<br />
==Final Presentation:==<br />
<br />
{| class="wikitable"<br />
|-<br />
|'''Slide'''<br />
|'''Transcription'''<br />
|-<br />
|[[File:Slide1.png|500px]]<br />
|<br />
Hello, my name is Charlie Repaci, and I am a Senior at Simmons University. My project is titled Developing Ethical Algorithms for Placement Stability in the Foster Care System, which I worked on under the guidance of doctoral student Devansh Saxena and Dr. Shion Guha.<br />
|-<br />
|[[File:slide2.png|500px]]<br />
|<br />
In this presentation, I’ll go through the work I have done the last 10 weeks, starting with an introduction and background, then moving on to objectives, methodology, results, discussion, and plans for future work.<br />
|-<br />
|[[File:slide3.png|500px]]<br />
|<br />
Social work is a high-stress and mostly thankless job. Social workers are understaffed, underfunded, and overworked. It’s due to this high stress with little reward that there is such a high job turnover rate. <br />
<br />
This high turnover rate also influences the cases that are being investigated, and the care that each receives can be disjointed. A social worker newly assigned to an active case then has to do extra work of familiarizing themself with it, which can also make the process take longer. Children can be moved from a foster home not because there is a difficulty in the home, but because there is no longer a social worker in the area to check in on them. For greater placement stability and better outcomes for children in the foster care system, social workers need more support.<br />
<br />
It is for this reason that algorithms were first introduced into the field: to aid social workers in making, explaining, and standardizing their decisions.<br />
|-<br />
|[[File:slide4.png|500px]]<br />
|<br />
Though the technical definition is any process with a well defined sequence of steps, what algorithms actually are in practice is widely debated and varies depending on the field and context. Algorithms in social work are often not of a strictly computational nature; they start out as psychometric or communimetric assessment matrices that have evolved to become guidelines on data collection which are then used to make decisions about future cases. They are becoming increasingly entrenched in various social programs, and while they were originally introduced for the purpose of enhancing and rehabilitating existing systems, they are often used without due consideration of the ethical concerns their application raises or the framework on which social fields usually rely. <br />
<br />
Family and community stakeholders who comprise the bulk of Child Protective Services (CPS) inquiries are not placated by the inclusion of algorithms, and mistrust them just as much as they mistrust the rest of the system, in part because of the perception that they are unknowable, and in part because they have not shown that they decrease bias in the system as they were meant to. Additionally, social workers are often suspicious of the new technology and tools that are intended to aid them. This is for many reasons, including that these algorithms were developed without users’ inputs and so don’t adequately address their needs, and often contradict their theoretical assessments. <br />
<br />
In addition, algorithms are often misappropriated for purposes outside their original scope, and don’t take well to the changes. For example, the Child and Adolescent Needs and Strengths (CANS) model, originally created to assess the needs of a child, is being used to calculate compensation for caretakers so that when a child is doing better than they were at their previous assessment due to the caretakers’ efforts, compensation is lowered.<br />
|-<br />
|[[File:slide5.png|500px]]<br />
|<br />
The perception of algorithms as unbiased paragons of decision making with god-like insight has contributed to their use without due process and ethical consideration. Algorithms are not implemented in a vacuum. Their input can be biased in a multitude of ways including the way the data was collected, the exclusion, intentional or not, of other data, the designer’s bias and other such issues, which in turn can reproduce and in some cases may even worsen the same biases that they are implemented with the intention of exterminating. This remains, however, unrecognized by many policy and decision makers. <br />
<br />
Systemic, regular, analysis of algorithms in social work (and many other fields for that matter) to ensure performance after their implementation are nonexistent. There is no standardized framework for testing for ethics in algorithms in the design process and after implementation, and a formalized process for third party inquiries would be ideal, however, this measure is not in place either.<br />
<br />
Moreover, algorithms tend to struggle in high-consequence fields like child welfare, and this is inherent in the way they make decisions. While a social worker is capable of refining their decision criteria before making decisions on a complex case, an algorithm can only update its decision boundary in response to an incorrect classification, which has deep ethical implications. You can see in the figure above the difference in how a human would make a decision and how an algorithm makes a decision. Because of this, the design of an algorithm should include the ability to identify edge cases that require greater human attention and at a policy level, there should be room for repeals and recourse.<br />
<br />
Overall, it is clear there is a need for assessment, development, and accountability of algorithms in child welfare systems.<br />
|-<br />
|[[File:slide6.png|500px]]<br />
|<br />
This project was an audit of the Washington Assessment of Risk Matrix (WARM), which was used to aid in coding the risk each child faced in the case at intake and at the conclusion of the investigation in planning future accommodations. The purpose of this was to compare the results of the algorithm to the coded narrative elements in the case notes that a social worker writes and contrast not only the risk differences and outcomes but also the subjects of interest and what’s being measured.<br />
|-<br />
|[[File:slide7.png|500px]]<br />
|<br />
Although we started the project with four related data sets, we focused on just one due to the time constraints of the project. The Phase I dataset came from another study of case notes which aimed to determine factors associated with the unsubstantiation of CPS referrals. In this dataset, the individuals recorded were children with a CPS referral. Data was gathered through the review of administrative records. 3,000 CPS referrals were randomly selected from one year of records (a total of 7,701 referrals). Cases excluded from the review and narrative coding included those with "limited access, information only referrals, risk tag pending, licensing, third party perpetrators, a sibling as the perpetrator, duplicate referrals, and referrals where there was no identifiable victim”. The study then coded the case notes of 2,000 referrals, the final number of records. My mentors provided this dataset from the National Data Archive on Child Abuse and Neglect (NDACAN).<br />
|-<br />
|[[File:slide8.png|500px]]<br />
|<br />
The above graph shows risk associated with a case based on the referral compared to the outcome of the case. Risk Tag is the output total risk of the WARM, and the finding is the conclusion of whether or not the social worker determined that past abuse had occurred and the child was at risk of it happening again. Founded cases go on to other social services to address their specific needs and lessen risk to the child. <br />
<br />
These are clearly mostly at the moderate through high levels of risk. Additionally, cases that are founded in general are associated with higher risk. One reason that the risks assigned at the referral may be so high is that low risk cases have a lower depth of investigation and attention. If a referrer wants the case to be treated to a higher standard of investigation they must ensure that the reported risk is greater.<br />
|-<br />
|[[File:slide9.png|500px]]<br />
|<br />
This graph shows the risk assessed at the summary of the investigation compared to the finding of the referral. In comparison to the previous graph, there are far fewer moderate through high risk cases. Again, founded cases are in general associated with higher risks. It is interesting to note that there are cases with ratings of no risk that are founded and cases with moderate through high risk that are unfounded.<br />
|-<br />
|[[File:slide10.png|500px]]<br />
|<br />
You don’t have to read all this, but I wanted to make sure it was all in here. Categories that are mentioned specifically in the narrative in addition to being scored in the WARM are either requiring further elaboration of the score (for example, the social worker may feel that the WARM category is too restrictive), or they are the pieces that they feel are essential to the case, more so than the other axes that the WARM measures. The many narrative categories that go unmentioned by the WARM but that social workers feel are important to mention in the case notes were numerous, and mostly focused on inconclusive evidence, abuse types not accounted for by the model, specifics about how the referral was made, and other family conflicts that could endanger the child.<br />
|-<br />
|[[File:slide11.png|500px]]<br />
|<br />
I’m not going to show you all of them, but here are a few of the graphs of the variables that were both in narrative coding and the WARM. Most cases with medical evidence of sexual abuse are founded, and those that are founded in general have higher levels of risk.<br />
|-<br />
|[[File:slide12.png|500px]]<br />
|<br />
This graph of medical negligence against risk shows a similar pattern, with more founded cases than not, and higher risk associated with founded cases.<br />
|-<br />
|[[File:slide13.png|500px]]<br />
|<br />
This graph of medical evidence of injury shows that same pattern of finding and risk. You can also note, however, that many cases with explicit injury mentioned are marked no risk even if they are founded. This is most likely to be a failing of the algorithm or but it could also be an example of a caseworker determining that while abuse happened there was no risk of it happening again.<br />
|-<br />
|[[File:slide14.png|500px]]<br />
|<br />
Subjects that were coded from the narratives and are also used in the WARM assessment were chosen to investigate accuracy of the computed risk. The level of risk that is acceptable before becoming a false positive is an ethical question and varies from one application to another and also depends on the way the social workers are instructed to collect and input the information. Cases with a low or moderately low assigned risk also have a low standard of investigation; that is, CPS reviews prior involvement and collateral contacts to determine if further investigation should occur. Risk levels of “Moderate” or higher demand a high standard of investigation, including a review of prior CPS interaction, collateral contacts, interviews, and other assessments. Based on this, the false positive threshold was chosen as anything categorized as “Moderate” or higher in risk.<br />
<br />
In this section, we’re more interested in false negatives than false positives, as a false negative is a disregard of the explicit risk, however the false positive may be falsely positive because of another category of risk that the case is high in that brings the overall risk score up.<br />
|-<br />
|[[File:slide15.png|500px]]<br />
|<br />
These are the categories that the narrative notes and the WARM had in common with each other. Using the narrative rates as the “true” presence of risk in the case, we can test the WARM accuracy by variable. Risk at intake has a very low false negative rate with 0 for all categories and a high false positive rate while risk at the summary of the investigation has more accurate rates, but this is expected in risk at intake, because, if you remember, the risk distribution at intake, nearly all cases are rated “Moderate” or higher, and in these categories specifically, there were no cases rated lower than “Moderate”. The risk at summary has more usual false negatives and positives.<br />
<br />
In terms of findings, those which had higher percentages involved hard or medical evidence of some kind. Factors that were not directly involved in abuse, such as developmental delay, difficult behavior in the child, and economic stress or hardship, had lower percentages of findings and were associated with lower risks than some of the other variables.<br />
|-<br />
|[[File:slide16.png|500px]]<br />
|<br />
The main limitation of this work is that the WARM was replaced by the Structured Decision Making model and thus these findings cannot be extended directly to modern cases. The SDM uses more machine learning techniques than the WARM and addresses some of the shortcomings, but still has many of the same issues as its predecessor including racial disproportionality and bias in other areas. Notably, however, it also includes a discretionary override for the caseworker to increase the risk with specified justification if they feel the algorithm has judged risk of future harm to be too low.<br />
|-<br />
|[[File:slide17.png|500px]]<br />
|<br />
The algorithm audit performed gives incomplete insight into investigating the narrative factors that influence social workers’ decisions and whether or not they are included in current placement stability and risk assessment models and future work would continue the research. Caseworker notes from the Wisconsin Department of Children and Families and from SaintA, a private foster care service, would be analyzed with topic modeling to extract latent themes. Open source placement stability and risk assessment models such as the CANS and SDM models, which have been adopted and modified in many states, would be used also. This would allow experimentation with adding narrative themes to existing placement models to test improvement in outcomes.<br />
|-<br />
|[[File:slide18.png|500px]]<br />
|<br />
<br />
|-<br />
|[[File:slide19.png|500px]]<br />
|<br />
I’d like to thank my mentors for all their help on this project, the NSF for funding it, and Doctors Brylow and Madiraju for hosting and putting this all together. I had a lot of fun on the project and really enjoyed my summer. I’ll take any questions at this time.<br />
|}</div>Crepacihttps://reu.cs.mu.edu/index.php/File:Slide19.pngFile:Slide19.png2020-08-07T19:49:07Z<p>Crepaci: </p>
<hr />
<div></div>Crepacihttps://reu.cs.mu.edu/index.php/File:Slide18.pngFile:Slide18.png2020-08-07T19:48:55Z<p>Crepaci: </p>
<hr />
<div></div>Crepacihttps://reu.cs.mu.edu/index.php/File:Slide16.pngFile:Slide16.png2020-08-07T19:48:41Z<p>Crepaci: </p>
<hr />
<div></div>Crepacihttps://reu.cs.mu.edu/index.php/File:Slide15.pngFile:Slide15.png2020-08-07T19:48:26Z<p>Crepaci: </p>
<hr />
<div></div>Crepacihttps://reu.cs.mu.edu/index.php/File:Slide14.pngFile:Slide14.png2020-08-07T19:48:11Z<p>Crepaci: </p>
<hr />
<div></div>Crepacihttps://reu.cs.mu.edu/index.php/File:Slide13.pngFile:Slide13.png2020-08-07T19:47:55Z<p>Crepaci: </p>
<hr />
<div></div>Crepacihttps://reu.cs.mu.edu/index.php/File:Slide12.pngFile:Slide12.png2020-08-07T19:47:41Z<p>Crepaci: </p>
<hr />
<div></div>Crepacihttps://reu.cs.mu.edu/index.php/File:Slide11.pngFile:Slide11.png2020-08-07T19:47:19Z<p>Crepaci: </p>
<hr />
<div></div>Crepacihttps://reu.cs.mu.edu/index.php/File:Slide10.pngFile:Slide10.png2020-08-07T19:46:54Z<p>Crepaci: </p>
<hr />
<div></div>Crepacihttps://reu.cs.mu.edu/index.php/File:Slide9.pngFile:Slide9.png2020-08-07T19:46:37Z<p>Crepaci: </p>
<hr />
<div></div>Crepacihttps://reu.cs.mu.edu/index.php/File:Slide8.pngFile:Slide8.png2020-08-07T19:46:24Z<p>Crepaci: </p>
<hr />
<div></div>Crepacihttps://reu.cs.mu.edu/index.php/File:Slide7.pngFile:Slide7.png2020-08-07T19:45:56Z<p>Crepaci: </p>
<hr />
<div></div>Crepacihttps://reu.cs.mu.edu/index.php/File:Slide6.pngFile:Slide6.png2020-08-07T19:45:41Z<p>Crepaci: </p>
<hr />
<div></div>Crepacihttps://reu.cs.mu.edu/index.php/File:Slide5.pngFile:Slide5.png2020-08-07T19:45:24Z<p>Crepaci: </p>
<hr />
<div></div>Crepacihttps://reu.cs.mu.edu/index.php/File:Slide4.pngFile:Slide4.png2020-08-07T19:44:58Z<p>Crepaci: </p>
<hr />
<div></div>Crepacihttps://reu.cs.mu.edu/index.php/File:Slide3.pngFile:Slide3.png2020-08-07T19:44:40Z<p>Crepaci: </p>
<hr />
<div></div>Crepacihttps://reu.cs.mu.edu/index.php/File:Slide2.pngFile:Slide2.png2020-08-07T19:43:33Z<p>Crepaci: </p>
<hr />
<div></div>Crepacihttps://reu.cs.mu.edu/index.php/File:Slide1.pngFile:Slide1.png2020-08-07T19:43:01Z<p>Crepaci: </p>
<hr />
<div></div>Crepacihttps://reu.cs.mu.edu/index.php/Developing_Ethical_Algorithms_for_Placement_Stability_in_the_Foster_Care_SystemDeveloping Ethical Algorithms for Placement Stability in the Foster Care System2020-08-07T19:37:00Z<p>Crepaci: </p>
<hr />
<div>Student: [[User:Crepaci|Charlie Repaci]]<br />
<br />
Mentor: [https://www.shionguha.net/ Dr. Shion Guha] and [https://saxena.io/ Devansh Saxena]<br />
<br />
==Description:==<br />
The goal of this project is to use a human-centered approach grounded in current social science theory and frameworks to add context to and further develop existing placement stability and risk assessment models that are used to aid overworked social workers in making, explaining, and standardizing their decisions.<br />
<br />
==Goals:==<br />
Quoted from the [https://reu.cs.mu.edu/index.php/Ethical_Algorithms_for_Placement_Stability_in_the_Foster_Care project page:]<br />
<br />
<blockquote>''This project aims to collaborate with the WI Department of Children and Families and SaintA (a private non-profit organization that actually provides foster care services in WI) by utilizing important, useful and contextual caseworkers judgment that are recorded as detailed case notes but never actually used to add norms, values, and context to existing algorithms that determine placement stability. Topic modeling will be used to extract latent themes from such text and incrementally added to existing placement stability models to test improvements in outcomes.''<br />
* ''Perform a literature review of topic modeling usage in the social science domain.''<br />
* ''Understand latent and human context from caseworker notes in Wisconsin foster care system using the data provided from the mentor.''<br />
* ''Review algorithmic biases, fairness, and transparency in the foster care system.''<br />
* ''Evaluate the latent themes from text and incrementally add to existing placement stability models to test improvement in outcomes.''</blockquote><br />
<br />
==Schedule:==<br />
{| class="wikitable"<br />
|-<br />
|'''Week'''<br />
|'''Description'''<br />
|-<br />
|Week 1<br />
|<br />
* Orientation<br />
* Data Science Bootcamp<br />
* Good Research Practices Lecture<br />
* Begin literature review -- algorithms in social work<br />
|-<br />
|Week 2<br />
|<br />
* Responsible Conduct of Research Training<br />
* CITI certification<br />
* Technical writing workshop<br />
* Continue literature review -- ethics and algorithms<br />
|-<br />
|Week 3<br />
|<br />
* Set up wiki<br />
* Begin reviewing data<br />
* Continue literature review -- bias in algorithms and policy<br />
|-<br />
|Week 4<br />
|<br />
* Import data<br />
* Research presentation by guest lecturer Dr. Walt Bialkowski<br />
* Presentation of work done in weeks 3 and 4 to mentors<br />
|-<br />
|Week 5<br />
|<br />
* Research Presentations lecture<br />
* Data exploration and visualization<br />
* Technical presentations workshop and informal presentations<br />
|-<br />
|Week 6<br />
|<br />
* Data Ethics by guest lecturer Dr. Michael Zimmer<br />
* Making Research Posters lecture<br />
* Continued exploratory work and visualizations to highlight important factors<br />
* Continue literature review -- bias in algorithm development<br />
|-<br />
|Week 7<br />
|<br />
* Looking more deeply into Phase I<br />
* Begin WARM algorithm audit<br />
* Continue literature review -- gender diversity and algorithms<br />
|-<br />
|Week 8<br />
|<br />
* WARM algorithm audit<br />
* Continue literature review -- gender diversity and algorithms<br />
* Graduate Schools discussion by Dr. Brylow<br />
|-<br />
|Week 9<br />
|<br />
* Industry guest panel (Northwestern Mutual Data Science Institute)<br />
* Begin the final paper<br />
* Start to create project poster<br />
* WARM algorithm audit<br />
|-<br />
|Week 10<br />
|<br />
* Prepare and give the oral presentation<br />
* Present project to other REU sites and see their research in return<br />
* Finish and submit the final paper<br />
|}<br />
<br />
==Project Poster:==<br />
<br />
[[File:Project Poster.png|center|1000px]]<br />
<br />
==Final Presentation:==<br />
<br />
{| class="wikitable"<br />
|-<br />
|'''Slide'''<br />
|'''Transcription'''<br />
|-<br />
|[[File:slide1.png|500px]]<br />
|<br />
Hello, my name is Charlie Repaci, and I am a Senior at Simmons University. My project is titled Developing Ethical Algorithms for Placement Stability in the Foster Care System, which I worked on under the guidance of doctoral student Devansh Saxena and Dr. Shion Guha.<br />
|-<br />
|[[File:slide2.png|500px]]<br />
|<br />
In this presentation, I’ll go through the work I have done the last 10 weeks, starting with an introduction and background, then moving on to objectives, methodology, results, discussion, and plans for future work.<br />
|-<br />
|[[File:slide3.png|500px]]<br />
|<br />
Social work is a high-stress and mostly thankless job. Social workers are understaffed, underfunded, and overworked. It’s due to this high stress with little reward that there is such a high job turnover rate. <br />
<br />
This high turnover rate also influences the cases that are being investigated, and the care that each receives can be disjointed. A social worker newly assigned to an active case then has to do extra work of familiarizing themself with it, which can also make the process take longer. Children can be moved from a foster home not because there is a difficulty in the home, but because there is no longer a social worker in the area to check in on them. For greater placement stability and better outcomes for children in the foster care system, social workers need more support.<br />
<br />
It is for this reason that algorithms were first introduced into the field: to aid social workers in making, explaining, and standardizing their decisions.<br />
|-<br />
|[[File:slide4.png|500px]]<br />
|<br />
Though the technical definition is any process with a well defined sequence of steps, what algorithms actually are in practice is widely debated and varies depending on the field and context. Algorithms in social work are often not of a strictly computational nature; they start out as psychometric or communimetric assessment matrices that have evolved to become guidelines on data collection which are then used to make decisions about future cases. They are becoming increasingly entrenched in various social programs, and while they were originally introduced for the purpose of enhancing and rehabilitating existing systems, they are often used without due consideration of the ethical concerns their application raises or the framework on which social fields usually rely. <br />
<br />
Family and community stakeholders who comprise the bulk of Child Protective Services (CPS) inquiries are not placated by the inclusion of algorithms, and mistrust them just as much as they mistrust the rest of the system, in part because of the perception that they are unknowable, and in part because they have not shown that they decrease bias in the system as they were meant to. Additionally, social workers are often suspicious of the new technology and tools that are intended to aid them. This is for many reasons, including that these algorithms were developed without users’ inputs and so don’t adequately address their needs, and often contradict their theoretical assessments. <br />
<br />
In addition, algorithms are often misappropriated for purposes outside their original scope, and don’t take well to the changes. For example, the Child and Adolescent Needs and Strengths (CANS) model, originally created to assess the needs of a child, is being used to calculate compensation for caretakers so that when a child is doing better than they were at their previous assessment due to the caretakers’ efforts, compensation is lowered.<br />
|-<br />
|[[File:slide5.png|500px]]<br />
|<br />
The perception of algorithms as unbiased paragons of decision making with god-like insight has contributed to their use without due process and ethical consideration. Algorithms are not implemented in a vacuum. Their input can be biased in a multitude of ways including the way the data was collected, the exclusion, intentional or not, of other data, the designer’s bias and other such issues, which in turn can reproduce and in some cases may even worsen the same biases that they are implemented with the intention of exterminating. This remains, however, unrecognized by many policy and decision makers. <br />
<br />
Systemic, regular, analysis of algorithms in social work (and many other fields for that matter) to ensure performance after their implementation are nonexistent. There is no standardized framework for testing for ethics in algorithms in the design process and after implementation, and a formalized process for third party inquiries would be ideal, however, this measure is not in place either.<br />
<br />
Moreover, algorithms tend to struggle in high-consequence fields like child welfare, and this is inherent in the way they make decisions. While a social worker is capable of refining their decision criteria before making decisions on a complex case, an algorithm can only update its decision boundary in response to an incorrect classification, which has deep ethical implications. You can see in the figure above the difference in how a human would make a decision and how an algorithm makes a decision. Because of this, the design of an algorithm should include the ability to identify edge cases that require greater human attention and at a policy level, there should be room for repeals and recourse.<br />
<br />
Overall, it is clear there is a need for assessment, development, and accountability of algorithms in child welfare systems.<br />
|-<br />
|[[File:slide6.png|500px]]<br />
|<br />
This project was an audit of the Washington Assessment of Risk Matrix (WARM), which was used to aid in coding the risk each child faced in the case at intake and at the conclusion of the investigation in planning future accommodations. The purpose of this was to compare the results of the algorithm to the coded narrative elements in the case notes that a social worker writes and contrast not only the risk differences and outcomes but also the subjects of interest and what’s being measured.<br />
|-<br />
|[[File:slide7.png|500px]]<br />
|<br />
Although we started the project with four related data sets, we focused on just one due to the time constraints of the project. The Phase I dataset came from another study of case notes which aimed to determine factors associated with the unsubstantiation of CPS referrals. In this dataset, the individuals recorded were children with a CPS referral. Data was gathered through the review of administrative records. 3,000 CPS referrals were randomly selected from one year of records (a total of 7,701 referrals). Cases excluded from the review and narrative coding included those with "limited access, information only referrals, risk tag pending, licensing, third party perpetrators, a sibling as the perpetrator, duplicate referrals, and referrals where there was no identifiable victim”. The study then coded the case notes of 2,000 referrals, the final number of records. My mentors provided this dataset from the National Data Archive on Child Abuse and Neglect (NDACAN).<br />
|-<br />
|[[File:slide8.png|500px]]<br />
|<br />
The above graph shows risk associated with a case based on the referral compared to the outcome of the case. Risk Tag is the output total risk of the WARM, and the finding is the conclusion of whether or not the social worker determined that past abuse had occurred and the child was at risk of it happening again. Founded cases go on to other social services to address their specific needs and lessen risk to the child. <br />
<br />
These are clearly mostly at the moderate through high levels of risk. Additionally, cases that are founded in general are associated with higher risk. One reason that the risks assigned at the referral may be so high is that low risk cases have a lower depth of investigation and attention. If a referrer wants the case to be treated to a higher standard of investigation they must ensure that the reported risk is greater.<br />
|-<br />
|[[File:slide9.png|500px]]<br />
|<br />
This graph shows the risk assessed at the summary of the investigation compared to the finding of the referral. In comparison to the previous graph, there are far fewer moderate through high risk cases. Again, founded cases are in general associated with higher risks. It is interesting to note that there are cases with ratings of no risk that are founded and cases with moderate through high risk that are unfounded.<br />
|-<br />
|[[File:slide10.png|500px]]<br />
|<br />
You don’t have to read all this, but I wanted to make sure it was all in here. Categories that are mentioned specifically in the narrative in addition to being scored in the WARM are either requiring further elaboration of the score (for example, the social worker may feel that the WARM category is too restrictive), or they are the pieces that they feel are essential to the case, more so than the other axes that the WARM measures. The many narrative categories that go unmentioned by the WARM but that social workers feel are important to mention in the case notes were numerous, and mostly focused on inconclusive evidence, abuse types not accounted for by the model, specifics about how the referral was made, and other family conflicts that could endanger the child.<br />
|-<br />
|[[File:slide11.png|500px]]<br />
|<br />
I’m not going to show you all of them, but here are a few of the graphs of the variables that were both in narrative coding and the WARM. Most cases with medical evidence of sexual abuse are founded, and those that are founded in general have higher levels of risk.<br />
|-<br />
|[[File:slide12.png|500px]]<br />
|<br />
This graph of medical negligence against risk shows a similar pattern, with more founded cases than not, and higher risk associated with founded cases.<br />
|-<br />
|[[File:slide13.png|500px]]<br />
|<br />
This graph of medical evidence of injury shows that same pattern of finding and risk. You can also note, however, that many cases with explicit injury mentioned are marked no risk even if they are founded. This is most likely to be a failing of the algorithm or but it could also be an example of a caseworker determining that while abuse happened there was no risk of it happening again.<br />
|-<br />
|[[File:slide14.png|500px]]<br />
|<br />
Subjects that were coded from the narratives and are also used in the WARM assessment were chosen to investigate accuracy of the computed risk. The level of risk that is acceptable before becoming a false positive is an ethical question and varies from one application to another and also depends on the way the social workers are instructed to collect and input the information. Cases with a low or moderately low assigned risk also have a low standard of investigation; that is, CPS reviews prior involvement and collateral contacts to determine if further investigation should occur. Risk levels of “Moderate” or higher demand a high standard of investigation, including a review of prior CPS interaction, collateral contacts, interviews, and other assessments. Based on this, the false positive threshold was chosen as anything categorized as “Moderate” or higher in risk.<br />
<br />
In this section, we’re more interested in false negatives than false positives, as a false negative is a disregard of the explicit risk, however the false positive may be falsely positive because of another category of risk that the case is high in that brings the overall risk score up.<br />
|-<br />
|[[File:slide15.png|500px]]<br />
|<br />
These are the categories that the narrative notes and the WARM had in common with each other. Using the narrative rates as the “true” presence of risk in the case, we can test the WARM accuracy by variable. Risk at intake has a very low false negative rate with 0 for all categories and a high false positive rate while risk at the summary of the investigation has more accurate rates, but this is expected in risk at intake, because, if you remember, the risk distribution at intake, nearly all cases are rated “Moderate” or higher, and in these categories specifically, there were no cases rated lower than “Moderate”. The risk at summary has more usual false negatives and positives.<br />
<br />
In terms of findings, those which had higher percentages involved hard or medical evidence of some kind. Factors that were not directly involved in abuse, such as developmental delay, difficult behavior in the child, and economic stress or hardship, had lower percentages of findings and were associated with lower risks than some of the other variables.<br />
|-<br />
|[[File:slide16.png|500px]]<br />
|<br />
The main limitation of this work is that the WARM was replaced by the Structured Decision Making model and thus these findings cannot be extended directly to modern cases. The SDM uses more machine learning techniques than the WARM and addresses some of the shortcomings, but still has many of the same issues as its predecessor including racial disproportionality and bias in other areas. Notably, however, it also includes a discretionary override for the caseworker to increase the risk with specified justification if they feel the algorithm has judged risk of future harm to be too low.<br />
|-<br />
|[[File:slide17.png|500px]]<br />
|<br />
The algorithm audit performed gives incomplete insight into investigating the narrative factors that influence social workers’ decisions and whether or not they are included in current placement stability and risk assessment models and future work would continue the research. Caseworker notes from the Wisconsin Department of Children and Families and from SaintA, a private foster care service, would be analyzed with topic modeling to extract latent themes. Open source placement stability and risk assessment models such as the CANS and SDM models, which have been adopted and modified in many states, would be used also. This would allow experimentation with adding narrative themes to existing placement models to test improvement in outcomes.<br />
|-<br />
|[[File:slide18.png|500px]]<br />
|<br />
<br />
|-<br />
|[[File:slide19.png|500px]]<br />
|<br />
I’d like to thank my mentors for all their help on this project, the NSF for funding it, and Doctors Brylow and Madiraju for hosting and putting this all together. I had a lot of fun on the project and really enjoyed my summer. I’ll take any questions at this time.<br />
|}</div>Crepacihttps://reu.cs.mu.edu/index.php/Developing_Ethical_Algorithms_for_Placement_Stability_in_the_Foster_Care_SystemDeveloping Ethical Algorithms for Placement Stability in the Foster Care System2020-08-05T17:35:13Z<p>Crepaci: /* Project Poster: */</p>
<hr />
<div>Student: [[User:Crepaci|Charlie Repaci]]<br />
<br />
Mentor: [https://www.shionguha.net/ Dr. Shion Guha] and [https://saxena.io/ Devansh Saxena]<br />
<br />
==Description:==<br />
The goal of this project is to use a human-centered approach grounded in current social science theory and frameworks to add context to and further develop existing placement stability and risk assessment models that are used to aid overworked social workers in making, explaining, and standardizing their decisions.<br />
<br />
==Goals:==<br />
Quoted from the [https://reu.cs.mu.edu/index.php/Ethical_Algorithms_for_Placement_Stability_in_the_Foster_Care project page:]<br />
<br />
<blockquote>''This project aims to collaborate with the WI Department of Children and Families and SaintA (a private non-profit organization that actually provides foster care services in WI) by utilizing important, useful and contextual caseworkers judgment that are recorded as detailed case notes but never actually used to add norms, values, and context to existing algorithms that determine placement stability. Topic modeling will be used to extract latent themes from such text and incrementally added to existing placement stability models to test improvements in outcomes.''<br />
* ''Perform a literature review of topic modeling usage in the social science domain.''<br />
* ''Understand latent and human context from caseworker notes in Wisconsin foster care system using the data provided from the mentor.''<br />
* ''Review algorithmic biases, fairness, and transparency in the foster care system.''<br />
* ''Evaluate the latent themes from text and incrementally add to existing placement stability models to test improvement in outcomes.''</blockquote><br />
<br />
==Schedule:==<br />
{| class="wikitable"<br />
|-<br />
|'''Week'''<br />
|'''Description'''<br />
|-<br />
|Week 1<br />
|<br />
* Orientation<br />
* Data Science Bootcamp<br />
* Good Research Practices Lecture<br />
* Begin literature review -- algorithms in social work<br />
|-<br />
|Week 2<br />
|<br />
* Responsible Conduct of Research Training<br />
* CITI certification<br />
* Technical writing workshop<br />
* Continue literature review -- ethics and algorithms<br />
|-<br />
|Week 3<br />
|<br />
* Set up wiki<br />
* Begin reviewing data<br />
* Continue literature review -- bias in algorithms and policy<br />
|-<br />
|Week 4<br />
|<br />
* Import data<br />
* Research presentation by guest lecturer Dr. Walt Bialkowski<br />
* Presentation of work done in weeks 3 and 4 to mentors<br />
|-<br />
|Week 5<br />
|<br />
* Research Presentations lecture<br />
* Data exploration and visualization<br />
* Technical presentations workshop and informal presentations<br />
|-<br />
|Week 6<br />
|<br />
* Data Ethics by guest lecturer Dr. Michael Zimmer<br />
* Making Research Posters lecture<br />
* Continued exploratory work and visualizations to highlight important factors<br />
* Continue literature review -- bias in algorithm development<br />
|-<br />
|Week 7<br />
|<br />
* Looking more deeply into Phase I<br />
* Begin WARM algorithm audit<br />
* Continue literature review -- gender diversity and algorithms<br />
|-<br />
|Week 8<br />
|<br />
* WARM algorithm audit<br />
* Continue literature review -- gender diversity and algorithms<br />
* Graduate Schools discussion by Dr. Brylow<br />
|-<br />
|Week 9<br />
|<br />
* Industry guest panel (Northwestern Mutual Data Science Institute)<br />
* Begin the final paper<br />
* Start to create project poster<br />
* WARM algorithm audit<br />
|-<br />
|Week 10<br />
|<br />
* Prepare and give the oral presentation<br />
* Present project to other REU sites and see their research in return<br />
* Finish and submit the final paper<br />
|}<br />
<br />
==Project Poster:==<br />
<br />
[[File:Project Poster.png|center|1000px]]</div>Crepacihttps://reu.cs.mu.edu/index.php/Developing_Ethical_Algorithms_for_Placement_Stability_in_the_Foster_Care_SystemDeveloping Ethical Algorithms for Placement Stability in the Foster Care System2020-08-05T17:34:57Z<p>Crepaci: /* Project Poster: */</p>
<hr />
<div>Student: [[User:Crepaci|Charlie Repaci]]<br />
<br />
Mentor: [https://www.shionguha.net/ Dr. Shion Guha] and [https://saxena.io/ Devansh Saxena]<br />
<br />
==Description:==<br />
The goal of this project is to use a human-centered approach grounded in current social science theory and frameworks to add context to and further develop existing placement stability and risk assessment models that are used to aid overworked social workers in making, explaining, and standardizing their decisions.<br />
<br />
==Goals:==<br />
Quoted from the [https://reu.cs.mu.edu/index.php/Ethical_Algorithms_for_Placement_Stability_in_the_Foster_Care project page:]<br />
<br />
<blockquote>''This project aims to collaborate with the WI Department of Children and Families and SaintA (a private non-profit organization that actually provides foster care services in WI) by utilizing important, useful and contextual caseworkers judgment that are recorded as detailed case notes but never actually used to add norms, values, and context to existing algorithms that determine placement stability. Topic modeling will be used to extract latent themes from such text and incrementally added to existing placement stability models to test improvements in outcomes.''<br />
* ''Perform a literature review of topic modeling usage in the social science domain.''<br />
* ''Understand latent and human context from caseworker notes in Wisconsin foster care system using the data provided from the mentor.''<br />
* ''Review algorithmic biases, fairness, and transparency in the foster care system.''<br />
* ''Evaluate the latent themes from text and incrementally add to existing placement stability models to test improvement in outcomes.''</blockquote><br />
<br />
==Schedule:==<br />
{| class="wikitable"<br />
|-<br />
|'''Week'''<br />
|'''Description'''<br />
|-<br />
|Week 1<br />
|<br />
* Orientation<br />
* Data Science Bootcamp<br />
* Good Research Practices Lecture<br />
* Begin literature review -- algorithms in social work<br />
|-<br />
|Week 2<br />
|<br />
* Responsible Conduct of Research Training<br />
* CITI certification<br />
* Technical writing workshop<br />
* Continue literature review -- ethics and algorithms<br />
|-<br />
|Week 3<br />
|<br />
* Set up wiki<br />
* Begin reviewing data<br />
* Continue literature review -- bias in algorithms and policy<br />
|-<br />
|Week 4<br />
|<br />
* Import data<br />
* Research presentation by guest lecturer Dr. Walt Bialkowski<br />
* Presentation of work done in weeks 3 and 4 to mentors<br />
|-<br />
|Week 5<br />
|<br />
* Research Presentations lecture<br />
* Data exploration and visualization<br />
* Technical presentations workshop and informal presentations<br />
|-<br />
|Week 6<br />
|<br />
* Data Ethics by guest lecturer Dr. Michael Zimmer<br />
* Making Research Posters lecture<br />
* Continued exploratory work and visualizations to highlight important factors<br />
* Continue literature review -- bias in algorithm development<br />
|-<br />
|Week 7<br />
|<br />
* Looking more deeply into Phase I<br />
* Begin WARM algorithm audit<br />
* Continue literature review -- gender diversity and algorithms<br />
|-<br />
|Week 8<br />
|<br />
* WARM algorithm audit<br />
* Continue literature review -- gender diversity and algorithms<br />
* Graduate Schools discussion by Dr. Brylow<br />
|-<br />
|Week 9<br />
|<br />
* Industry guest panel (Northwestern Mutual Data Science Institute)<br />
* Begin the final paper<br />
* Start to create project poster<br />
* WARM algorithm audit<br />
|-<br />
|Week 10<br />
|<br />
* Prepare and give the oral presentation<br />
* Present project to other REU sites and see their research in return<br />
* Finish and submit the final paper<br />
|}<br />
<br />
==Project Poster:==<br />
<br />
[[File:Project Poster.png|center|1500px]]</div>Crepacihttps://reu.cs.mu.edu/index.php/Developing_Ethical_Algorithms_for_Placement_Stability_in_the_Foster_Care_SystemDeveloping Ethical Algorithms for Placement Stability in the Foster Care System2020-08-05T17:34:23Z<p>Crepaci: /* Project Poster: */</p>
<hr />
<div>Student: [[User:Crepaci|Charlie Repaci]]<br />
<br />
Mentor: [https://www.shionguha.net/ Dr. Shion Guha] and [https://saxena.io/ Devansh Saxena]<br />
<br />
==Description:==<br />
The goal of this project is to use a human-centered approach grounded in current social science theory and frameworks to add context to and further develop existing placement stability and risk assessment models that are used to aid overworked social workers in making, explaining, and standardizing their decisions.<br />
<br />
==Goals:==<br />
Quoted from the [https://reu.cs.mu.edu/index.php/Ethical_Algorithms_for_Placement_Stability_in_the_Foster_Care project page:]<br />
<br />
<blockquote>''This project aims to collaborate with the WI Department of Children and Families and SaintA (a private non-profit organization that actually provides foster care services in WI) by utilizing important, useful and contextual caseworkers judgment that are recorded as detailed case notes but never actually used to add norms, values, and context to existing algorithms that determine placement stability. Topic modeling will be used to extract latent themes from such text and incrementally added to existing placement stability models to test improvements in outcomes.''<br />
* ''Perform a literature review of topic modeling usage in the social science domain.''<br />
* ''Understand latent and human context from caseworker notes in Wisconsin foster care system using the data provided from the mentor.''<br />
* ''Review algorithmic biases, fairness, and transparency in the foster care system.''<br />
* ''Evaluate the latent themes from text and incrementally add to existing placement stability models to test improvement in outcomes.''</blockquote><br />
<br />
==Schedule:==<br />
{| class="wikitable"<br />
|-<br />
|'''Week'''<br />
|'''Description'''<br />
|-<br />
|Week 1<br />
|<br />
* Orientation<br />
* Data Science Bootcamp<br />
* Good Research Practices Lecture<br />
* Begin literature review -- algorithms in social work<br />
|-<br />
|Week 2<br />
|<br />
* Responsible Conduct of Research Training<br />
* CITI certification<br />
* Technical writing workshop<br />
* Continue literature review -- ethics and algorithms<br />
|-<br />
|Week 3<br />
|<br />
* Set up wiki<br />
* Begin reviewing data<br />
* Continue literature review -- bias in algorithms and policy<br />
|-<br />
|Week 4<br />
|<br />
* Import data<br />
* Research presentation by guest lecturer Dr. Walt Bialkowski<br />
* Presentation of work done in weeks 3 and 4 to mentors<br />
|-<br />
|Week 5<br />
|<br />
* Research Presentations lecture<br />
* Data exploration and visualization<br />
* Technical presentations workshop and informal presentations<br />
|-<br />
|Week 6<br />
|<br />
* Data Ethics by guest lecturer Dr. Michael Zimmer<br />
* Making Research Posters lecture<br />
* Continued exploratory work and visualizations to highlight important factors<br />
* Continue literature review -- bias in algorithm development<br />
|-<br />
|Week 7<br />
|<br />
* Looking more deeply into Phase I<br />
* Begin WARM algorithm audit<br />
* Continue literature review -- gender diversity and algorithms<br />
|-<br />
|Week 8<br />
|<br />
* WARM algorithm audit<br />
* Continue literature review -- gender diversity and algorithms<br />
* Graduate Schools discussion by Dr. Brylow<br />
|-<br />
|Week 9<br />
|<br />
* Industry guest panel (Northwestern Mutual Data Science Institute)<br />
* Begin the final paper<br />
* Start to create project poster<br />
* WARM algorithm audit<br />
|-<br />
|Week 10<br />
|<br />
* Prepare and give the oral presentation<br />
* Present project to other REU sites and see their research in return<br />
* Finish and submit the final paper<br />
|}<br />
<br />
==Project Poster:==<br />
<br />
[[File:Project Poster.png|center|1700px]]</div>Crepacihttps://reu.cs.mu.edu/index.php/Developing_Ethical_Algorithms_for_Placement_Stability_in_the_Foster_Care_SystemDeveloping Ethical Algorithms for Placement Stability in the Foster Care System2020-08-05T17:34:00Z<p>Crepaci: /* Project Poster: */</p>
<hr />
<div>Student: [[User:Crepaci|Charlie Repaci]]<br />
<br />
Mentor: [https://www.shionguha.net/ Dr. Shion Guha] and [https://saxena.io/ Devansh Saxena]<br />
<br />
==Description:==<br />
The goal of this project is to use a human-centered approach grounded in current social science theory and frameworks to add context to and further develop existing placement stability and risk assessment models that are used to aid overworked social workers in making, explaining, and standardizing their decisions.<br />
<br />
==Goals:==<br />
Quoted from the [https://reu.cs.mu.edu/index.php/Ethical_Algorithms_for_Placement_Stability_in_the_Foster_Care project page:]<br />
<br />
<blockquote>''This project aims to collaborate with the WI Department of Children and Families and SaintA (a private non-profit organization that actually provides foster care services in WI) by utilizing important, useful and contextual caseworkers judgment that are recorded as detailed case notes but never actually used to add norms, values, and context to existing algorithms that determine placement stability. Topic modeling will be used to extract latent themes from such text and incrementally added to existing placement stability models to test improvements in outcomes.''<br />
* ''Perform a literature review of topic modeling usage in the social science domain.''<br />
* ''Understand latent and human context from caseworker notes in Wisconsin foster care system using the data provided from the mentor.''<br />
* ''Review algorithmic biases, fairness, and transparency in the foster care system.''<br />
* ''Evaluate the latent themes from text and incrementally add to existing placement stability models to test improvement in outcomes.''</blockquote><br />
<br />
==Schedule:==<br />
{| class="wikitable"<br />
|-<br />
|'''Week'''<br />
|'''Description'''<br />
|-<br />
|Week 1<br />
|<br />
* Orientation<br />
* Data Science Bootcamp<br />
* Good Research Practices Lecture<br />
* Begin literature review -- algorithms in social work<br />
|-<br />
|Week 2<br />
|<br />
* Responsible Conduct of Research Training<br />
* CITI certification<br />
* Technical writing workshop<br />
* Continue literature review -- ethics and algorithms<br />
|-<br />
|Week 3<br />
|<br />
* Set up wiki<br />
* Begin reviewing data<br />
* Continue literature review -- bias in algorithms and policy<br />
|-<br />
|Week 4<br />
|<br />
* Import data<br />
* Research presentation by guest lecturer Dr. Walt Bialkowski<br />
* Presentation of work done in weeks 3 and 4 to mentors<br />
|-<br />
|Week 5<br />
|<br />
* Research Presentations lecture<br />
* Data exploration and visualization<br />
* Technical presentations workshop and informal presentations<br />
|-<br />
|Week 6<br />
|<br />
* Data Ethics by guest lecturer Dr. Michael Zimmer<br />
* Making Research Posters lecture<br />
* Continued exploratory work and visualizations to highlight important factors<br />
* Continue literature review -- bias in algorithm development<br />
|-<br />
|Week 7<br />
|<br />
* Looking more deeply into Phase I<br />
* Begin WARM algorithm audit<br />
* Continue literature review -- gender diversity and algorithms<br />
|-<br />
|Week 8<br />
|<br />
* WARM algorithm audit<br />
* Continue literature review -- gender diversity and algorithms<br />
* Graduate Schools discussion by Dr. Brylow<br />
|-<br />
|Week 9<br />
|<br />
* Industry guest panel (Northwestern Mutual Data Science Institute)<br />
* Begin the final paper<br />
* Start to create project poster<br />
* WARM algorithm audit<br />
|-<br />
|Week 10<br />
|<br />
* Prepare and give the oral presentation<br />
* Present project to other REU sites and see their research in return<br />
* Finish and submit the final paper<br />
|}<br />
<br />
==Project Poster:==<br />
<br />
[[File:Project Poster.png|center|468px]]</div>Crepacihttps://reu.cs.mu.edu/index.php/File:Project_Poster.pngFile:Project Poster.png2020-08-05T17:30:36Z<p>Crepaci: </p>
<hr />
<div></div>Crepacihttps://reu.cs.mu.edu/index.php/Developing_Ethical_Algorithms_for_Placement_Stability_in_the_Foster_Care_SystemDeveloping Ethical Algorithms for Placement Stability in the Foster Care System2020-08-05T17:29:59Z<p>Crepaci: </p>
<hr />
<div>Student: [[User:Crepaci|Charlie Repaci]]<br />
<br />
Mentor: [https://www.shionguha.net/ Dr. Shion Guha] and [https://saxena.io/ Devansh Saxena]<br />
<br />
==Description:==<br />
The goal of this project is to use a human-centered approach grounded in current social science theory and frameworks to add context to and further develop existing placement stability and risk assessment models that are used to aid overworked social workers in making, explaining, and standardizing their decisions.<br />
<br />
==Goals:==<br />
Quoted from the [https://reu.cs.mu.edu/index.php/Ethical_Algorithms_for_Placement_Stability_in_the_Foster_Care project page:]<br />
<br />
<blockquote>''This project aims to collaborate with the WI Department of Children and Families and SaintA (a private non-profit organization that actually provides foster care services in WI) by utilizing important, useful and contextual caseworkers judgment that are recorded as detailed case notes but never actually used to add norms, values, and context to existing algorithms that determine placement stability. Topic modeling will be used to extract latent themes from such text and incrementally added to existing placement stability models to test improvements in outcomes.''<br />
* ''Perform a literature review of topic modeling usage in the social science domain.''<br />
* ''Understand latent and human context from caseworker notes in Wisconsin foster care system using the data provided from the mentor.''<br />
* ''Review algorithmic biases, fairness, and transparency in the foster care system.''<br />
* ''Evaluate the latent themes from text and incrementally add to existing placement stability models to test improvement in outcomes.''</blockquote><br />
<br />
==Schedule:==<br />
{| class="wikitable"<br />
|-<br />
|'''Week'''<br />
|'''Description'''<br />
|-<br />
|Week 1<br />
|<br />
* Orientation<br />
* Data Science Bootcamp<br />
* Good Research Practices Lecture<br />
* Begin literature review -- algorithms in social work<br />
|-<br />
|Week 2<br />
|<br />
* Responsible Conduct of Research Training<br />
* CITI certification<br />
* Technical writing workshop<br />
* Continue literature review -- ethics and algorithms<br />
|-<br />
|Week 3<br />
|<br />
* Set up wiki<br />
* Begin reviewing data<br />
* Continue literature review -- bias in algorithms and policy<br />
|-<br />
|Week 4<br />
|<br />
* Import data<br />
* Research presentation by guest lecturer Dr. Walt Bialkowski<br />
* Presentation of work done in weeks 3 and 4 to mentors<br />
|-<br />
|Week 5<br />
|<br />
* Research Presentations lecture<br />
* Data exploration and visualization<br />
* Technical presentations workshop and informal presentations<br />
|-<br />
|Week 6<br />
|<br />
* Data Ethics by guest lecturer Dr. Michael Zimmer<br />
* Making Research Posters lecture<br />
* Continued exploratory work and visualizations to highlight important factors<br />
* Continue literature review -- bias in algorithm development<br />
|-<br />
|Week 7<br />
|<br />
* Looking more deeply into Phase I<br />
* Begin WARM algorithm audit<br />
* Continue literature review -- gender diversity and algorithms<br />
|-<br />
|Week 8<br />
|<br />
* WARM algorithm audit<br />
* Continue literature review -- gender diversity and algorithms<br />
* Graduate Schools discussion by Dr. Brylow<br />
|-<br />
|Week 9<br />
|<br />
* Industry guest panel (Northwestern Mutual Data Science Institute)<br />
* Begin the final paper<br />
* Start to create project poster<br />
* WARM algorithm audit<br />
|-<br />
|Week 10<br />
|<br />
* Prepare and give the oral presentation<br />
* Present project to other REU sites and see their research in return<br />
* Finish and submit the final paper<br />
|}<br />
<br />
==Project Poster:==<br />
[[File:Project Poster.png]]</div>Crepacihttps://reu.cs.mu.edu/index.php/User:CrepaciUser:Crepaci2020-08-01T21:40:54Z<p>Crepaci: /* Work Log */</p>
<hr />
<div><br />
== About Me ==<br />
<br />
I'm [https://www.linkedin.com/in/charlie-repaci-07b723179/ Charlie Repaci], a senior at [https://www.simmons.edu Simmons University] studying [https://www.simmons.edu/undergraduate/academics/majors-minors/data-science-and-analytics Data Science], with a special interest in Biochemistry and Sociology. This summer I am working with [https://www.shionguha.net/ Dr. Shion Guha] and doctoral candidate [https://www.saxena.io/ Devansh Saxena] on [[Developing Ethical Algorithms for Placement Stability in the Foster Care System]].<br />
<br />
== Work Log ==<br />
<br />
===Week 1===<br />
June 1 to June 7<br />
# Orientation<br />
#* Introduction to other mentors, mentees, and REU heads Dr. Praveen Madiraju and Dr. Dennis Brylow<br />
#* Review of REU calendar and expectations<br />
# Data Science Bootcamp (talk by Dr. Madiraju)<br />
#* Introduction and basics of data analysis with python (Anaconda and Jypyter Notebook; pandas, numpy, matplotlib, seaborn, scipy)<br />
#** Read in data<br />
#** Pre-processing<br />
#** Modeling<br />
#** Data visualization<br />
# Good Research Practices (talk by Dr. Brylow)<br />
# Literature Review<br />
#* [https://dl.acm.org/doi/abs/10.1145/3313831.3376229 A Human-Centered Review of Algorithms used within the U.S. Child Welfare System]<br />
#* [https://dl.acm.org/doi/abs/10.1145/3290605.3300271 Toward Algorithmic Accountability in Public Services: A Qualitative Study of Affected Community Perspectives on Algorithmic Decision-making in Child Welfare Services]<br />
#* [https://dl.acm.org/doi/abs/10.1145/3323994.3369888 Child Welfare System: Interaction of Policy, Practice, and Algorithms]<br />
#* (Supplementary) [https://dl.acm.org/doi/abs/10.1145/3290605.3300497 Risk vs. Restriction: The Tension between Providing a Sense of Normalcy and Keeping Foster Teens Safe Online]<br />
<br />
===Week 2===<br />
June 8 to June 14<br />
# Responsible Conduct of Research Training (talk by Dr. Brylow)<br />
#* Ethical treatment of data<br />
#* Authorship, credit, plagiarism<br />
#* Human participants<br />
#* Intellectual property<br />
#* Conflicts of interest and professional standards<br />
# Worked on getting Citi certification for all three RCR sessions<br />
# Technical Writing Workshop (talk by Dr. Brylow and Dr. Madiraju)<br />
#* What the sections of a technical paper are<br />
#* What the publication process is like<br />
#* General tips<br />
# Literature review<br />
#* [http://dx.doi.org/10.2139/ssrn.2245322 Governing Algorithms: A Provocation Piece]<br />
#* [https://academiccommons.columbia.edu/doi/10.7916/D8ZK5TW2 Algorithmic Accountability Reporting: On the Investigation of Black Boxes]<br />
#* [http://www.tandfonline.com/doi/full/10.1080/1369118X.2016.1154087#abstract Thinking Critically About and Researching Algorithms]<br />
#* (Supplementary) [http://culturedigitally.org/wp-content/uploads/2016/07/Gillespie-2016-Algorithm-Digital-Keywords-Peters-ed.pdf Algorithm in Digital Keywords: a Vocabulary of Information, Society, and Culture]<br />
# Meeting with mentors<br />
#* Github created for the project<br />
#* Questions and discussion of the literature reviewed<br />
#* Planned work for the next two weeks<br />
<br />
===Week 3===<br />
June 15 to June 21<br />
# Set up Wiki entrees for myself and my project<br />
# Meeting with all REU interns to discuss our projects so far and any problems we have run into<br />
# Literature review<br />
#* [https://dl.acm.org/doi/abs/10.1145/3290605.3300760 Street-Level Algorithms: A Theory at the Gaps Between Policy and Decisions]<br />
#* [https://journals.sagepub.com/doi/abs/10.1177/2053951717738104 Algorithms as culture: Some tactics for the ethnography of algorithmic systems]<br />
#* (Supplementary) [https://journals.sagepub.com/doi/full/10.1177/2053951717751552 Algorithms as fetish: Faith and possibility in algorithmic work]<br />
# Meeting with Devansh to discuss the first dataset<br />
#* Understand variables and components -- reading the documentation that comes with datasets<br />
#* Import dataset into R and common problems<br />
<br />
===Week 4===<br />
June 22 to June 28<br />
<br />
# Importing data<br />
#* Need SAS to run a program included with the data files that creates .sas7bdat and .sas7bcat files from the .dat files that were given to us<br />
#* Some are not importing correctly for yet unknown reasons<br />
# Student Check-In (by Dr. Brylow and Dr. Madiraju)<br />
# Research Presentation (talk by guest lecturer Dr. Walter Bialkowski)<br />
#* Blood donation and potential risk of lower bone density to donors due to prolonged and repeated exposure to the anticoagulant citrate added to the blood (and then returned to the donor) during the donation process<br />
#* Study 1: Data analysis of Scandinavian blood donor data<br />
#** Concluded that there was no association between blood donation and the number of bone fractures donors had later in life<br />
#** Limitations included differences in blood donation policy, process, and popularity of varying donation types between Scandanavia and the United States<br />
#* Study 2: Longitudinal study of blood donors<br />
#** Concluded that current guidelines were enough to protect adult male donors between the ages of 20 and 65<br />
#** Limited in that conclusion can not be extrapolated to women or to men outside that age range<br />
# Presentation on work done in weeks 3 and 4 to mentors (readings, problems with data)<br />
<br />
===Week 5===<br />
June 29 to July 5<br />
<br />
# Research Presentations (talk by Dr. Brylow)<br />
#* Format and sections<br />
#** Very similar to the paper itself: introduction, background (know your audience), description of your work, results, conclusion<br />
#** Figure out 2-3 points you want your audience to take away from the talk and centralize your material around those<br />
#* General tips<br />
#** 1 slide per minute rule of thumb<br />
#** People sometimes bring extra slides in anticipation of questions<br />
#** Avoid distractions and keep it simple: avoid full sentences, use (and cite) diagrams, use simple color schemes (light text on a dark background reads well from afar)<br />
# Presented work done so far to other mentors and students<br />
#* You can find my slide deck with presentation notes [https://docs.google.com/presentation/d/1CUA1-6la1RL5SZk0vk4eeMWW7DA-WO-SVQSlE5SfKbQ/edit?usp=sharing here]<br />
# Began data exploration of the Phase I dataset (from project 107: Factors that Influence the Decision Not to Substantiate a CPS Referral)<br />
<br />
===Week 6===<br />
July 6 to July 12<br />
<br />
# Data Ethics Lecture (talk by Dr. Michael Zimmer)<br />
#* Empiricist epistemology and criticisms<br />
#** The idea that big data captures everything, so there is no need for theory or models, no need to worry about biased values, and no need to consult domain-specific experts<br />
#** Hidden biases in both the collection and analysis stages present considerable risks<br />
#* Rich, identifiable data from multiple sources on the same person (ex: different apps on phone)<br />
#* Questionable consent<br />
#** Clicking through the Terms of Use without reading it<br />
#** "Public" data used without identification efforts<br />
#* Reproducibility vs deidentification<br />
# More exploratory work and visualizations to highlight factors that would be interesting to look at from a technical perspective and their implications for any predictive systems<br />
# Making Research Posters (talk by Dr. Brylow)<br />
#* Usually posters display work that isn't so far along as to be published yet<br />
#* Stylistic tips to improve readability but also conserve space and convey the topic effectively from across the room<br />
#* Talk to each person for less than three minutes as this is only a peek into your work<br />
# Meeting with Mentors<br />
#* Focus on risk assessment in Phase I of the 107 data set. How is it measured? What is "normal" risk? Was it generated using a human or an algorithm? How well does it perform?<br />
#* Goals for the next two weeks<br />
#** Make a small multiple for all the factors in the set against risk assessment and write a few lines of analysis for each<br />
#** Start attempting to answer the questions posed above<br />
# Literature review<br />
#* [https://dgergle.soc.northwestern.edu//resources/pn3458-diazA.pdf Addressing Age-Related Bias in Sentiment Analysis]<br />
#* [https://www.researchgate.net/publication/257560404_Bias_in_algorithmic_filtering_and_personalization Bias in algorithmic filtering and personalization]<br />
#* Towards a Feminist HCI Methodology: Social Science, Feminism, and HCI by Shaowen Bardzell and Jeffrey Bardzell<br />
<br />
===Week 7===<br />
July 13 to July 19<br />
<br />
# Phase I deeper variable exploration / Start algorithm audit<br />
#* Spreadsheet created for ease of sorting by source and type<br />
#* Determined and highlighted various measures of risk and their source (human vs modeled)<br />
#* Continued to read up on Phase I methods of collection <br />
#* Continued to create small multiples of the 664 variables<br />
# Literature Review<br />
#* [https://doi.org/10.1145/3274357 The misgendering machines: Trans/HCI implications of automatic gender recognition]<br />
#* [https://reallifemag.com/counting-the-countless/ Counting the Countless: Why data science is a profound threat for queer people]<br />
#* [https://ainowinstitute.org/discriminatingsystems.pdf Discriminating systems: Gender, race and power in AI]<br />
#* [https://doi.org/10.1145/3274424 Safe spaces and safe places: Unpacking technology-mediated experiences of safety and harm with transgender people]<br />
#* [https://doi.org/10.1145/3290607.3311750 Queer(ing) HCI: Moving forward in theory and practice]<br />
<br />
===Week 8===<br />
July 20 to July 26<br />
<br />
# WARM (Washington Assessment of Risk Matrix) factors vs Risk Tag graphs<br />
# Graduate Schools - Discussion with Dr. Brylow<br />
#* Types of programs and funding sources<br />
#* Important parts of the application<br />
#* How to write a personal statement<br />
#* Selecting your school<br />
<br />
===Week 9===<br />
July 27 to August 2<br />
<br />
===Week 10===<br />
August 3 to August 9</div>Crepacihttps://reu.cs.mu.edu/index.php/Developing_Ethical_Algorithms_for_Placement_Stability_in_the_Foster_Care_SystemDeveloping Ethical Algorithms for Placement Stability in the Foster Care System2020-08-01T21:31:36Z<p>Crepaci: /* Schedule: */</p>
<hr />
<div>Student: [[User:Crepaci|Charlie Repaci]]<br />
<br />
Mentor: [https://www.shionguha.net/ Dr. Shion Guha] and [https://saxena.io/ Devansh Saxena]<br />
<br />
==Description:==<br />
The goal of this project is to use a human-centered approach grounded in current social science theory and frameworks to add context to and further develop existing placement stability and risk assessment models that are used to aid overworked social workers in making, explaining, and standardizing their decisions.<br />
<br />
==Goals:==<br />
Quoted from the [https://reu.cs.mu.edu/index.php/Ethical_Algorithms_for_Placement_Stability_in_the_Foster_Care project page:]<br />
<br />
<blockquote>''This project aims to collaborate with the WI Department of Children and Families and SaintA (a private non-profit organization that actually provides foster care services in WI) by utilizing important, useful and contextual caseworkers judgment that are recorded as detailed case notes but never actually used to add norms, values, and context to existing algorithms that determine placement stability. Topic modeling will be used to extract latent themes from such text and incrementally added to existing placement stability models to test improvements in outcomes.''<br />
* ''Perform a literature review of topic modeling usage in the social science domain.''<br />
* ''Understand latent and human context from caseworker notes in Wisconsin foster care system using the data provided from the mentor.''<br />
* ''Review algorithmic biases, fairness, and transparency in the foster care system.''<br />
* ''Evaluate the latent themes from text and incrementally add to existing placement stability models to test improvement in outcomes.''</blockquote><br />
<br />
==Schedule:==<br />
{| class="wikitable"<br />
|-<br />
|'''Week'''<br />
|'''Description'''<br />
|-<br />
|Week 1<br />
|<br />
* Orientation<br />
* Data Science Bootcamp<br />
* Good Research Practices Lecture<br />
* Begin literature review -- algorithms in social work<br />
|-<br />
|Week 2<br />
|<br />
* Responsible Conduct of Research Training<br />
* CITI certification<br />
* Technical writing workshop<br />
* Continue literature review -- ethics and algorithms<br />
|-<br />
|Week 3<br />
|<br />
* Set up wiki<br />
* Begin reviewing data<br />
* Continue literature review -- bias in algorithms and policy<br />
|-<br />
|Week 4<br />
|<br />
* Import data<br />
* Research presentation by guest lecturer Dr. Walt Bialkowski<br />
* Presentation of work done in weeks 3 and 4 to mentors<br />
|-<br />
|Week 5<br />
|<br />
* Research Presentations lecture<br />
* Data exploration and visualization<br />
* Technical presentations workshop and informal presentations<br />
|-<br />
|Week 6<br />
|<br />
* Data Ethics by guest lecturer Dr. Michael Zimmer<br />
* Making Research Posters lecture<br />
* Continued exploratory work and visualizations to highlight important factors<br />
* Continue literature review -- bias in algorithm development<br />
|-<br />
|Week 7<br />
|<br />
* Looking more deeply into Phase I<br />
* Begin WARM algorithm audit<br />
* Continue literature review -- gender diversity and algorithms<br />
|-<br />
|Week 8<br />
|<br />
* WARM algorithm audit<br />
* Continue literature review -- gender diversity and algorithms<br />
* Graduate Schools discussion by Dr. Brylow<br />
|-<br />
|Week 9<br />
|<br />
* Industry guest panel (Northwestern Mutual Data Science Institute)<br />
* Begin the final paper<br />
* Start to create project poster<br />
* WARM algorithm audit<br />
|-<br />
|Week 10<br />
|<br />
* Prepare and give the oral presentation<br />
* Present project to other REU sites and see their research in return<br />
* Finish and submit the final paper<br />
|}</div>Crepacihttps://reu.cs.mu.edu/index.php/Developing_Ethical_Algorithms_for_Placement_Stability_in_the_Foster_Care_SystemDeveloping Ethical Algorithms for Placement Stability in the Foster Care System2020-07-27T17:54:05Z<p>Crepaci: /* Schedule: */</p>
<hr />
<div>Student: [[User:Crepaci|Charlie Repaci]]<br />
<br />
Mentor: [https://www.shionguha.net/ Dr. Shion Guha] and [https://saxena.io/ Devansh Saxena]<br />
<br />
==Description:==<br />
The goal of this project is to use a human-centered approach grounded in current social science theory and frameworks to add context to and further develop existing placement stability and risk assessment models that are used to aid overworked social workers in making, explaining, and standardizing their decisions.<br />
<br />
==Goals:==<br />
Quoted from the [https://reu.cs.mu.edu/index.php/Ethical_Algorithms_for_Placement_Stability_in_the_Foster_Care project page:]<br />
<br />
<blockquote>''This project aims to collaborate with the WI Department of Children and Families and SaintA (a private non-profit organization that actually provides foster care services in WI) by utilizing important, useful and contextual caseworkers judgment that are recorded as detailed case notes but never actually used to add norms, values, and context to existing algorithms that determine placement stability. Topic modeling will be used to extract latent themes from such text and incrementally added to existing placement stability models to test improvements in outcomes.''<br />
* ''Perform a literature review of topic modeling usage in the social science domain.''<br />
* ''Understand latent and human context from caseworker notes in Wisconsin foster care system using the data provided from the mentor.''<br />
* ''Review algorithmic biases, fairness, and transparency in the foster care system.''<br />
* ''Evaluate the latent themes from text and incrementally add to existing placement stability models to test improvement in outcomes.''</blockquote><br />
<br />
==Schedule:==<br />
{| class="wikitable"<br />
|-<br />
|'''Week'''<br />
|'''Description'''<br />
|-<br />
|Week 1<br />
|<br />
* Orientation<br />
* Data Science Bootcamp<br />
* Good Research Practices Lecture<br />
* Begin literature review -- algorithms in social work<br />
|-<br />
|Week 2<br />
|<br />
* Responsible Conduct of Research Training<br />
* CITI certification<br />
* Technical writing workshop<br />
* Continue literature review -- ethics and algorithms<br />
|-<br />
|Week 3<br />
|<br />
* Set up wiki<br />
* Begin reviewing data<br />
* Continue literature review -- bias in algorithms and policy<br />
|-<br />
|Week 4<br />
|<br />
* Import data<br />
* Research presentation by guest lecturer Dr. Walt Bialkowski<br />
* Presentation of work done in weeks 3 and 4 to mentors<br />
|-<br />
|Week 5<br />
|<br />
* Research Presentations lecture<br />
* Data exploration and visualization<br />
* Technical presentations workshop and informal presentations<br />
|-<br />
|Week 6<br />
|<br />
* Data Ethics by guest lecturer Dr. Michael Zimmer<br />
* Making Research Posters lecture<br />
* Continued exploratory work and visualizations to highlight important factors<br />
* Continue literature review -- bias in algorithm development<br />
|-<br />
|Week 7<br />
|<br />
* Looking more deeply into Phase I<br />
* Exploratory visualizations<br />
* Continue literature review -- gender diversity and algorithms<br />
|-<br />
|Week 8<br />
|<br />
* Exploratory visualizations<br />
* Continue literature review -- gender diversity and algorithms<br />
* Graduate Schools discussion by Dr. Brylow<br />
|-<br />
|Week 9<br />
|<br />
* Industry guest panel (Northwestern Mutual Data Science Institute)<br />
* Begin the final paper<br />
* Start to create project poster<br />
* Exploratory visualizations<br />
|-<br />
|Week 10<br />
|<br />
* Prepare and give the oral presentation<br />
* TBD: Present project to other REU sites and see their research in return<br />
* Finish and submit the final paper<br />
|}</div>Crepacihttps://reu.cs.mu.edu/index.php/User:CrepaciUser:Crepaci2020-07-27T17:46:53Z<p>Crepaci: /* Week 8 */</p>
<hr />
<div><br />
== About Me ==<br />
<br />
I'm [https://www.linkedin.com/in/charlie-repaci-07b723179/ Charlie Repaci], a senior at [https://www.simmons.edu Simmons University] studying [https://www.simmons.edu/undergraduate/academics/majors-minors/data-science-and-analytics Data Science], with a special interest in Biochemistry and Sociology. This summer I am working with [https://www.shionguha.net/ Dr. Shion Guha] and doctoral candidate [https://www.saxena.io/ Devansh Saxena] on [[Developing Ethical Algorithms for Placement Stability in the Foster Care System]].<br />
<br />
== Work Log ==<br />
<br />
===Week 1===<br />
June 1 to June 7<br />
# Orientation<br />
#* Introduction to other mentors, mentees, and REU heads Dr. Praveen Madiraju and Dr. Dennis Brylow<br />
#* Review of REU calendar and expectations<br />
# Data Science Bootcamp (talk by Dr. Madiraju)<br />
#* Introduction and basics of data analysis with python (Anaconda and Jypyter Notebook; pandas, numpy, matplotlib, seaborn, scipy)<br />
#** Read in data<br />
#** Pre-processing<br />
#** Modeling<br />
#** Data visualization<br />
# Good Research Practices (talk by Dr. Brylow)<br />
# Literature Review<br />
#* [https://dl.acm.org/doi/abs/10.1145/3313831.3376229 A Human-Centered Review of Algorithms used within the U.S. Child Welfare System]<br />
#* [https://dl.acm.org/doi/abs/10.1145/3290605.3300271 Toward Algorithmic Accountability in Public Services: A Qualitative Study of Affected Community Perspectives on Algorithmic Decision-making in Child Welfare Services]<br />
#* [https://dl.acm.org/doi/abs/10.1145/3323994.3369888 Child Welfare System: Interaction of Policy, Practice, and Algorithms]<br />
#* (Supplementary) [https://dl.acm.org/doi/abs/10.1145/3290605.3300497 Risk vs. Restriction: The Tension between Providing a Sense of Normalcy and Keeping Foster Teens Safe Online]<br />
<br />
===Week 2===<br />
June 8 to June 14<br />
# Responsible Conduct of Research Training (talk by Dr. Brylow)<br />
#* Ethical treatment of data<br />
#* Authorship, credit, plagiarism<br />
#* Human participants<br />
#* Intellectual property<br />
#* Conflicts of interest and professional standards<br />
# Worked on getting Citi certification for all three RCR sessions<br />
# Technical Writing Workshop (talk by Dr. Brylow and Dr. Madiraju)<br />
#* What the sections of a technical paper are<br />
#* What the publication process is like<br />
#* General tips<br />
# Literature review<br />
#* [http://dx.doi.org/10.2139/ssrn.2245322 Governing Algorithms: A Provocation Piece]<br />
#* [https://academiccommons.columbia.edu/doi/10.7916/D8ZK5TW2 Algorithmic Accountability Reporting: On the Investigation of Black Boxes]<br />
#* [http://www.tandfonline.com/doi/full/10.1080/1369118X.2016.1154087#abstract Thinking Critically About and Researching Algorithms]<br />
#* (Supplementary) [http://culturedigitally.org/wp-content/uploads/2016/07/Gillespie-2016-Algorithm-Digital-Keywords-Peters-ed.pdf Algorithm in Digital Keywords: a Vocabulary of Information, Society, and Culture]<br />
# Meeting with mentors<br />
#* Github created for the project<br />
#* Questions and discussion of the literature reviewed<br />
#* Planned work for the next two weeks<br />
<br />
===Week 3===<br />
June 15 to June 21<br />
# Set up Wiki entrees for myself and my project<br />
# Meeting with all REU interns to discuss our projects so far and any problems we have run into<br />
# Literature review<br />
#* [https://dl.acm.org/doi/abs/10.1145/3290605.3300760 Street-Level Algorithms: A Theory at the Gaps Between Policy and Decisions]<br />
#* [https://journals.sagepub.com/doi/abs/10.1177/2053951717738104 Algorithms as culture: Some tactics for the ethnography of algorithmic systems]<br />
#* (Supplementary) [https://journals.sagepub.com/doi/full/10.1177/2053951717751552 Algorithms as fetish: Faith and possibility in algorithmic work]<br />
# Meeting with Devansh to discuss the first dataset<br />
#* Understand variables and components -- reading the documentation that comes with datasets<br />
#* Import dataset into R and common problems<br />
<br />
===Week 4===<br />
June 22 to June 28<br />
<br />
# Importing data<br />
#* Need SAS to run a program included with the data files that creates .sas7bdat and .sas7bcat files from the .dat files that were given to us<br />
#* Some are not importing correctly for yet unknown reasons<br />
# Student Check-In (by Dr. Brylow and Dr. Madiraju)<br />
# Research Presentation (talk by guest lecturer Dr. Walter Bialkowski)<br />
#* Blood donation and potential risk of lower bone density to donors due to prolonged and repeated exposure to the anticoagulant citrate added to the blood (and then returned to the donor) during the donation process<br />
#* Study 1: Data analysis of Scandinavian blood donor data<br />
#** Concluded that there was no association between blood donation and the number of bone fractures donors had later in life<br />
#** Limitations included differences in blood donation policy, process, and popularity of varying donation types between Scandanavia and the United States<br />
#* Study 2: Longitudinal study of blood donors<br />
#** Concluded that current guidelines were enough to protect adult male donors between the ages of 20 and 65<br />
#** Limited in that conclusion can not be extrapolated to women or to men outside that age range<br />
# Presentation on work done in weeks 3 and 4 to mentors (readings, problems with data)<br />
<br />
===Week 5===<br />
June 29 to July 5<br />
<br />
# Research Presentations (talk by Dr. Brylow)<br />
#* Format and sections<br />
#** Very similar to the paper itself: introduction, background (know your audience), description of your work, results, conclusion<br />
#** Figure out 2-3 points you want your audience to take away from the talk and centralize your material around those<br />
#* General tips<br />
#** 1 slide per minute rule of thumb<br />
#** People sometimes bring extra slides in anticipation of questions<br />
#** Avoid distractions and keep it simple: avoid full sentences, use (and cite) diagrams, use simple color schemes (light text on a dark background reads well from afar)<br />
# Presented work done so far to other mentors and students<br />
#* You can find my slide deck with presentation notes [https://docs.google.com/presentation/d/1CUA1-6la1RL5SZk0vk4eeMWW7DA-WO-SVQSlE5SfKbQ/edit?usp=sharing here]<br />
# Began data exploration of the Phase I dataset (from project 107: Factors that Influence the Decision Not to Substantiate a CPS Referral)<br />
<br />
===Week 6===<br />
July 6 to July 12<br />
<br />
# Data Ethics Lecture (talk by Dr. Michael Zimmer)<br />
#* Empiricist epistemology and criticisms<br />
#** The idea that big data captures everything, so there is no need for theory or models, no need to worry about biased values, and no need to consult domain-specific experts<br />
#** Hidden biases in both the collection and analysis stages present considerable risks<br />
#* Rich, identifiable data from multiple sources on the same person (ex: different apps on phone)<br />
#* Questionable consent<br />
#** Clicking through the Terms of Use without reading it<br />
#** "Public" data used without identification efforts<br />
#* Reproducibility vs deidentification<br />
# More exploratory work and visualizations to highlight factors that would be interesting to look at from a technical perspective and their implications for any predictive systems<br />
# Making Research Posters (talk by Dr. Brylow)<br />
#* Usually posters display work that isn't so far along as to be published yet<br />
#* Stylistic tips to improve readability but also conserve space and convey the topic effectively from across the room<br />
#* Talk to each person for less than three minutes as this is only a peek into your work<br />
# Meeting with Mentors<br />
#* Focus on risk assessment in Phase I of the 107 data set. How is it measured? What is "normal" risk? Was it generated using a human or an algorithm? How well does it perform?<br />
#* Goals for the next two weeks<br />
#** Make a small multiple for all the factors in the set against risk assessment and write a few lines of analysis for each<br />
#** Start attempting to answer the questions posed above<br />
# Literature review<br />
#* [https://dgergle.soc.northwestern.edu//resources/pn3458-diazA.pdf Addressing Age-Related Bias in Sentiment Analysis]<br />
#* [https://www.researchgate.net/publication/257560404_Bias_in_algorithmic_filtering_and_personalization Bias in algorithmic filtering and personalization]<br />
#* Towards a Feminist HCI Methodology: Social Science, Feminism, and HCI by Shaowen Bardzell and Jeffrey Bardzell<br />
<br />
===Week 7===<br />
July 13 to July 19<br />
<br />
# Phase I deeper variable exploration<br />
#* Spreadsheet created for ease of sorting by source and type<br />
#* Determined and highlighted various measures of risk and their source (human vs modeled)<br />
#* Continued to read up on Phase I methods of collection <br />
#* Continued to create small multiples of the 664 variables<br />
# Literature Review<br />
#* [https://doi.org/10.1145/3274357 The misgendering machines: Trans/HCI implications of automatic gender recognition]<br />
#* [https://reallifemag.com/counting-the-countless/ Counting the Countless: Why data science is a profound threat for queer people]<br />
#* [https://ainowinstitute.org/discriminatingsystems.pdf Discriminating systems: Gender, race and power in AI]<br />
#* [https://doi.org/10.1145/3274424 Safe spaces and safe places: Unpacking technology-mediated experiences of safety and harm with transgender people]<br />
#* [https://doi.org/10.1145/3290607.3311750 Queer(ing) HCI: Moving forward in theory and practice]<br />
<br />
===Week 8===<br />
July 20 to July 26<br />
<br />
# WARM (Washington Assessment of Risk Matrix) factors vs Risk Tag graphs<br />
# Graduate Schools - Discussion with Dr. Brylow<br />
#* Types of programs and funding sources<br />
#* Important parts of the application<br />
#* How to write a personal statement<br />
#* Selecting your school<br />
<br />
===Week 9===<br />
July 27 to August 2<br />
<br />
===Week 10===<br />
August 3 to August 9</div>Crepacihttps://reu.cs.mu.edu/index.php/User:CrepaciUser:Crepaci2020-07-27T17:46:02Z<p>Crepaci: /* Week 8 */</p>
<hr />
<div><br />
== About Me ==<br />
<br />
I'm [https://www.linkedin.com/in/charlie-repaci-07b723179/ Charlie Repaci], a senior at [https://www.simmons.edu Simmons University] studying [https://www.simmons.edu/undergraduate/academics/majors-minors/data-science-and-analytics Data Science], with a special interest in Biochemistry and Sociology. This summer I am working with [https://www.shionguha.net/ Dr. Shion Guha] and doctoral candidate [https://www.saxena.io/ Devansh Saxena] on [[Developing Ethical Algorithms for Placement Stability in the Foster Care System]].<br />
<br />
== Work Log ==<br />
<br />
===Week 1===<br />
June 1 to June 7<br />
# Orientation<br />
#* Introduction to other mentors, mentees, and REU heads Dr. Praveen Madiraju and Dr. Dennis Brylow<br />
#* Review of REU calendar and expectations<br />
# Data Science Bootcamp (talk by Dr. Madiraju)<br />
#* Introduction and basics of data analysis with python (Anaconda and Jypyter Notebook; pandas, numpy, matplotlib, seaborn, scipy)<br />
#** Read in data<br />
#** Pre-processing<br />
#** Modeling<br />
#** Data visualization<br />
# Good Research Practices (talk by Dr. Brylow)<br />
# Literature Review<br />
#* [https://dl.acm.org/doi/abs/10.1145/3313831.3376229 A Human-Centered Review of Algorithms used within the U.S. Child Welfare System]<br />
#* [https://dl.acm.org/doi/abs/10.1145/3290605.3300271 Toward Algorithmic Accountability in Public Services: A Qualitative Study of Affected Community Perspectives on Algorithmic Decision-making in Child Welfare Services]<br />
#* [https://dl.acm.org/doi/abs/10.1145/3323994.3369888 Child Welfare System: Interaction of Policy, Practice, and Algorithms]<br />
#* (Supplementary) [https://dl.acm.org/doi/abs/10.1145/3290605.3300497 Risk vs. Restriction: The Tension between Providing a Sense of Normalcy and Keeping Foster Teens Safe Online]<br />
<br />
===Week 2===<br />
June 8 to June 14<br />
# Responsible Conduct of Research Training (talk by Dr. Brylow)<br />
#* Ethical treatment of data<br />
#* Authorship, credit, plagiarism<br />
#* Human participants<br />
#* Intellectual property<br />
#* Conflicts of interest and professional standards<br />
# Worked on getting Citi certification for all three RCR sessions<br />
# Technical Writing Workshop (talk by Dr. Brylow and Dr. Madiraju)<br />
#* What the sections of a technical paper are<br />
#* What the publication process is like<br />
#* General tips<br />
# Literature review<br />
#* [http://dx.doi.org/10.2139/ssrn.2245322 Governing Algorithms: A Provocation Piece]<br />
#* [https://academiccommons.columbia.edu/doi/10.7916/D8ZK5TW2 Algorithmic Accountability Reporting: On the Investigation of Black Boxes]<br />
#* [http://www.tandfonline.com/doi/full/10.1080/1369118X.2016.1154087#abstract Thinking Critically About and Researching Algorithms]<br />
#* (Supplementary) [http://culturedigitally.org/wp-content/uploads/2016/07/Gillespie-2016-Algorithm-Digital-Keywords-Peters-ed.pdf Algorithm in Digital Keywords: a Vocabulary of Information, Society, and Culture]<br />
# Meeting with mentors<br />
#* Github created for the project<br />
#* Questions and discussion of the literature reviewed<br />
#* Planned work for the next two weeks<br />
<br />
===Week 3===<br />
June 15 to June 21<br />
# Set up Wiki entrees for myself and my project<br />
# Meeting with all REU interns to discuss our projects so far and any problems we have run into<br />
# Literature review<br />
#* [https://dl.acm.org/doi/abs/10.1145/3290605.3300760 Street-Level Algorithms: A Theory at the Gaps Between Policy and Decisions]<br />
#* [https://journals.sagepub.com/doi/abs/10.1177/2053951717738104 Algorithms as culture: Some tactics for the ethnography of algorithmic systems]<br />
#* (Supplementary) [https://journals.sagepub.com/doi/full/10.1177/2053951717751552 Algorithms as fetish: Faith and possibility in algorithmic work]<br />
# Meeting with Devansh to discuss the first dataset<br />
#* Understand variables and components -- reading the documentation that comes with datasets<br />
#* Import dataset into R and common problems<br />
<br />
===Week 4===<br />
June 22 to June 28<br />
<br />
# Importing data<br />
#* Need SAS to run a program included with the data files that creates .sas7bdat and .sas7bcat files from the .dat files that were given to us<br />
#* Some are not importing correctly for yet unknown reasons<br />
# Student Check-In (by Dr. Brylow and Dr. Madiraju)<br />
# Research Presentation (talk by guest lecturer Dr. Walter Bialkowski)<br />
#* Blood donation and potential risk of lower bone density to donors due to prolonged and repeated exposure to the anticoagulant citrate added to the blood (and then returned to the donor) during the donation process<br />
#* Study 1: Data analysis of Scandinavian blood donor data<br />
#** Concluded that there was no association between blood donation and the number of bone fractures donors had later in life<br />
#** Limitations included differences in blood donation policy, process, and popularity of varying donation types between Scandanavia and the United States<br />
#* Study 2: Longitudinal study of blood donors<br />
#** Concluded that current guidelines were enough to protect adult male donors between the ages of 20 and 65<br />
#** Limited in that conclusion can not be extrapolated to women or to men outside that age range<br />
# Presentation on work done in weeks 3 and 4 to mentors (readings, problems with data)<br />
<br />
===Week 5===<br />
June 29 to July 5<br />
<br />
# Research Presentations (talk by Dr. Brylow)<br />
#* Format and sections<br />
#** Very similar to the paper itself: introduction, background (know your audience), description of your work, results, conclusion<br />
#** Figure out 2-3 points you want your audience to take away from the talk and centralize your material around those<br />
#* General tips<br />
#** 1 slide per minute rule of thumb<br />
#** People sometimes bring extra slides in anticipation of questions<br />
#** Avoid distractions and keep it simple: avoid full sentences, use (and cite) diagrams, use simple color schemes (light text on a dark background reads well from afar)<br />
# Presented work done so far to other mentors and students<br />
#* You can find my slide deck with presentation notes [https://docs.google.com/presentation/d/1CUA1-6la1RL5SZk0vk4eeMWW7DA-WO-SVQSlE5SfKbQ/edit?usp=sharing here]<br />
# Began data exploration of the Phase I dataset (from project 107: Factors that Influence the Decision Not to Substantiate a CPS Referral)<br />
<br />
===Week 6===<br />
July 6 to July 12<br />
<br />
# Data Ethics Lecture (talk by Dr. Michael Zimmer)<br />
#* Empiricist epistemology and criticisms<br />
#** The idea that big data captures everything, so there is no need for theory or models, no need to worry about biased values, and no need to consult domain-specific experts<br />
#** Hidden biases in both the collection and analysis stages present considerable risks<br />
#* Rich, identifiable data from multiple sources on the same person (ex: different apps on phone)<br />
#* Questionable consent<br />
#** Clicking through the Terms of Use without reading it<br />
#** "Public" data used without identification efforts<br />
#* Reproducibility vs deidentification<br />
# More exploratory work and visualizations to highlight factors that would be interesting to look at from a technical perspective and their implications for any predictive systems<br />
# Making Research Posters (talk by Dr. Brylow)<br />
#* Usually posters display work that isn't so far along as to be published yet<br />
#* Stylistic tips to improve readability but also conserve space and convey the topic effectively from across the room<br />
#* Talk to each person for less than three minutes as this is only a peek into your work<br />
# Meeting with Mentors<br />
#* Focus on risk assessment in Phase I of the 107 data set. How is it measured? What is "normal" risk? Was it generated using a human or an algorithm? How well does it perform?<br />
#* Goals for the next two weeks<br />
#** Make a small multiple for all the factors in the set against risk assessment and write a few lines of analysis for each<br />
#** Start attempting to answer the questions posed above<br />
# Literature review<br />
#* [https://dgergle.soc.northwestern.edu//resources/pn3458-diazA.pdf Addressing Age-Related Bias in Sentiment Analysis]<br />
#* [https://www.researchgate.net/publication/257560404_Bias_in_algorithmic_filtering_and_personalization Bias in algorithmic filtering and personalization]<br />
#* Towards a Feminist HCI Methodology: Social Science, Feminism, and HCI by Shaowen Bardzell and Jeffrey Bardzell<br />
<br />
===Week 7===<br />
July 13 to July 19<br />
<br />
# Phase I deeper variable exploration<br />
#* Spreadsheet created for ease of sorting by source and type<br />
#* Determined and highlighted various measures of risk and their source (human vs modeled)<br />
#* Continued to read up on Phase I methods of collection <br />
#* Continued to create small multiples of the 664 variables<br />
# Literature Review<br />
#* [https://doi.org/10.1145/3274357 The misgendering machines: Trans/HCI implications of automatic gender recognition]<br />
#* [https://reallifemag.com/counting-the-countless/ Counting the Countless: Why data science is a profound threat for queer people]<br />
#* [https://ainowinstitute.org/discriminatingsystems.pdf Discriminating systems: Gender, race and power in AI]<br />
#* [https://doi.org/10.1145/3274424 Safe spaces and safe places: Unpacking technology-mediated experiences of safety and harm with transgender people]<br />
#* [https://doi.org/10.1145/3290607.3311750 Queer(ing) HCI: Moving forward in theory and practice]<br />
<br />
===Week 8===<br />
July 20 to July 26<br />
<br />
# WARM risk factors vs Risk Tag graphs<br />
# Graduate Schools - Discussion with Dr. Brylow<br />
#* Types of programs and funding sources<br />
#* Important parts of the application<br />
#* How to write a personal statement<br />
#* Selecting your school<br />
<br />
===Week 9===<br />
July 27 to August 2<br />
<br />
===Week 10===<br />
August 3 to August 9</div>Crepacihttps://reu.cs.mu.edu/index.php/Developing_Ethical_Algorithms_for_Placement_Stability_in_the_Foster_Care_SystemDeveloping Ethical Algorithms for Placement Stability in the Foster Care System2020-07-21T17:44:00Z<p>Crepaci: /* Schedule: */</p>
<hr />
<div>Student: [[User:Crepaci|Charlie Repaci]]<br />
<br />
Mentor: [https://www.shionguha.net/ Dr. Shion Guha] and [https://saxena.io/ Devansh Saxena]<br />
<br />
==Description:==<br />
The goal of this project is to use a human-centered approach grounded in current social science theory and frameworks to add context to and further develop existing placement stability and risk assessment models that are used to aid overworked social workers in making, explaining, and standardizing their decisions.<br />
<br />
==Goals:==<br />
Quoted from the [https://reu.cs.mu.edu/index.php/Ethical_Algorithms_for_Placement_Stability_in_the_Foster_Care project page:]<br />
<br />
<blockquote>''This project aims to collaborate with the WI Department of Children and Families and SaintA (a private non-profit organization that actually provides foster care services in WI) by utilizing important, useful and contextual caseworkers judgment that are recorded as detailed case notes but never actually used to add norms, values, and context to existing algorithms that determine placement stability. Topic modeling will be used to extract latent themes from such text and incrementally added to existing placement stability models to test improvements in outcomes.''<br />
* ''Perform a literature review of topic modeling usage in the social science domain.''<br />
* ''Understand latent and human context from caseworker notes in Wisconsin foster care system using the data provided from the mentor.''<br />
* ''Review algorithmic biases, fairness, and transparency in the foster care system.''<br />
* ''Evaluate the latent themes from text and incrementally add to existing placement stability models to test improvement in outcomes.''</blockquote><br />
<br />
==Schedule:==<br />
{| class="wikitable"<br />
|-<br />
|'''Week'''<br />
|'''Description'''<br />
|-<br />
|Week 1<br />
|<br />
* Orientation<br />
* Data Science Bootcamp<br />
* Good Research Practices Lecture<br />
* Begin literature review -- algorithms in social work<br />
|-<br />
|Week 2<br />
|<br />
* Responsible Conduct of Research Training<br />
* CITI certification<br />
* Technical writing workshop<br />
* Continue literature review -- ethics and algorithms<br />
|-<br />
|Week 3<br />
|<br />
* Set up wiki<br />
* Begin reviewing data<br />
* Continue literature review -- bias in algorithms and policy<br />
|-<br />
|Week 4<br />
|<br />
* Import data<br />
* Research presentation by guest lecturer Dr. Walt Bialkowski<br />
* Presentation of work done in weeks 3 and 4 to mentors<br />
|-<br />
|Week 5<br />
|<br />
* Research Presentations lecture<br />
* Data exploration and visualization<br />
* Technical presentations workshop and informal presentations<br />
|-<br />
|Week 6<br />
|<br />
* Data Ethics by guest lecturer Dr. Michael Zimmer<br />
* Making Research Posters lecture<br />
* Continued exploratory work and visualizations to highlight important factors<br />
* Continue literature review -- bias in algorithm development<br />
|-<br />
|Week 7<br />
|<br />
* Looking more deeply into Phase I<br />
* Exploratory visualizations<br />
* Continue literature review -- gender diversity and algorithms<br />
|-<br />
|Week 8<br />
|<br />
* Exploratory visualizations<br />
* Continue literature review -- gender diversity and algorithms<br />
* TBD<br />
|-<br />
|Week 9<br />
|<br />
* Industry guest panel (Northwestern Mutual Data Science Institute)<br />
* Begin the final paper<br />
* Start to create project poster<br />
* Consider future work<br />
|-<br />
|Week 10<br />
|<br />
* Prepare and give the oral presentation<br />
* TBD: Present project to other REU sites and see their research in return<br />
* Finish and submit the final paper<br />
|}</div>Crepacihttps://reu.cs.mu.edu/index.php/User:CrepaciUser:Crepaci2020-07-21T16:46:13Z<p>Crepaci: /* Work Log */</p>
<hr />
<div><br />
== About Me ==<br />
<br />
I'm [https://www.linkedin.com/in/charlie-repaci-07b723179/ Charlie Repaci], a senior at [https://www.simmons.edu Simmons University] studying [https://www.simmons.edu/undergraduate/academics/majors-minors/data-science-and-analytics Data Science], with a special interest in Biochemistry and Sociology. This summer I am working with [https://www.shionguha.net/ Dr. Shion Guha] and doctoral candidate [https://www.saxena.io/ Devansh Saxena] on [[Developing Ethical Algorithms for Placement Stability in the Foster Care System]].<br />
<br />
== Work Log ==<br />
<br />
===Week 1===<br />
June 1 to June 7<br />
# Orientation<br />
#* Introduction to other mentors, mentees, and REU heads Dr. Praveen Madiraju and Dr. Dennis Brylow<br />
#* Review of REU calendar and expectations<br />
# Data Science Bootcamp (talk by Dr. Madiraju)<br />
#* Introduction and basics of data analysis with python (Anaconda and Jypyter Notebook; pandas, numpy, matplotlib, seaborn, scipy)<br />
#** Read in data<br />
#** Pre-processing<br />
#** Modeling<br />
#** Data visualization<br />
# Good Research Practices (talk by Dr. Brylow)<br />
# Literature Review<br />
#* [https://dl.acm.org/doi/abs/10.1145/3313831.3376229 A Human-Centered Review of Algorithms used within the U.S. Child Welfare System]<br />
#* [https://dl.acm.org/doi/abs/10.1145/3290605.3300271 Toward Algorithmic Accountability in Public Services: A Qualitative Study of Affected Community Perspectives on Algorithmic Decision-making in Child Welfare Services]<br />
#* [https://dl.acm.org/doi/abs/10.1145/3323994.3369888 Child Welfare System: Interaction of Policy, Practice, and Algorithms]<br />
#* (Supplementary) [https://dl.acm.org/doi/abs/10.1145/3290605.3300497 Risk vs. Restriction: The Tension between Providing a Sense of Normalcy and Keeping Foster Teens Safe Online]<br />
<br />
===Week 2===<br />
June 8 to June 14<br />
# Responsible Conduct of Research Training (talk by Dr. Brylow)<br />
#* Ethical treatment of data<br />
#* Authorship, credit, plagiarism<br />
#* Human participants<br />
#* Intellectual property<br />
#* Conflicts of interest and professional standards<br />
# Worked on getting Citi certification for all three RCR sessions<br />
# Technical Writing Workshop (talk by Dr. Brylow and Dr. Madiraju)<br />
#* What the sections of a technical paper are<br />
#* What the publication process is like<br />
#* General tips<br />
# Literature review<br />
#* [http://dx.doi.org/10.2139/ssrn.2245322 Governing Algorithms: A Provocation Piece]<br />
#* [https://academiccommons.columbia.edu/doi/10.7916/D8ZK5TW2 Algorithmic Accountability Reporting: On the Investigation of Black Boxes]<br />
#* [http://www.tandfonline.com/doi/full/10.1080/1369118X.2016.1154087#abstract Thinking Critically About and Researching Algorithms]<br />
#* (Supplementary) [http://culturedigitally.org/wp-content/uploads/2016/07/Gillespie-2016-Algorithm-Digital-Keywords-Peters-ed.pdf Algorithm in Digital Keywords: a Vocabulary of Information, Society, and Culture]<br />
# Meeting with mentors<br />
#* Github created for the project<br />
#* Questions and discussion of the literature reviewed<br />
#* Planned work for the next two weeks<br />
<br />
===Week 3===<br />
June 15 to June 21<br />
# Set up Wiki entrees for myself and my project<br />
# Meeting with all REU interns to discuss our projects so far and any problems we have run into<br />
# Literature review<br />
#* [https://dl.acm.org/doi/abs/10.1145/3290605.3300760 Street-Level Algorithms: A Theory at the Gaps Between Policy and Decisions]<br />
#* [https://journals.sagepub.com/doi/abs/10.1177/2053951717738104 Algorithms as culture: Some tactics for the ethnography of algorithmic systems]<br />
#* (Supplementary) [https://journals.sagepub.com/doi/full/10.1177/2053951717751552 Algorithms as fetish: Faith and possibility in algorithmic work]<br />
# Meeting with Devansh to discuss the first dataset<br />
#* Understand variables and components -- reading the documentation that comes with datasets<br />
#* Import dataset into R and common problems<br />
<br />
===Week 4===<br />
June 22 to June 28<br />
<br />
# Importing data<br />
#* Need SAS to run a program included with the data files that creates .sas7bdat and .sas7bcat files from the .dat files that were given to us<br />
#* Some are not importing correctly for yet unknown reasons<br />
# Student Check-In (by Dr. Brylow and Dr. Madiraju)<br />
# Research Presentation (talk by guest lecturer Dr. Walter Bialkowski)<br />
#* Blood donation and potential risk of lower bone density to donors due to prolonged and repeated exposure to the anticoagulant citrate added to the blood (and then returned to the donor) during the donation process<br />
#* Study 1: Data analysis of Scandinavian blood donor data<br />
#** Concluded that there was no association between blood donation and the number of bone fractures donors had later in life<br />
#** Limitations included differences in blood donation policy, process, and popularity of varying donation types between Scandanavia and the United States<br />
#* Study 2: Longitudinal study of blood donors<br />
#** Concluded that current guidelines were enough to protect adult male donors between the ages of 20 and 65<br />
#** Limited in that conclusion can not be extrapolated to women or to men outside that age range<br />
# Presentation on work done in weeks 3 and 4 to mentors (readings, problems with data)<br />
<br />
===Week 5===<br />
June 29 to July 5<br />
<br />
# Research Presentations (talk by Dr. Brylow)<br />
#* Format and sections<br />
#** Very similar to the paper itself: introduction, background (know your audience), description of your work, results, conclusion<br />
#** Figure out 2-3 points you want your audience to take away from the talk and centralize your material around those<br />
#* General tips<br />
#** 1 slide per minute rule of thumb<br />
#** People sometimes bring extra slides in anticipation of questions<br />
#** Avoid distractions and keep it simple: avoid full sentences, use (and cite) diagrams, use simple color schemes (light text on a dark background reads well from afar)<br />
# Presented work done so far to other mentors and students<br />
#* You can find my slide deck with presentation notes [https://docs.google.com/presentation/d/1CUA1-6la1RL5SZk0vk4eeMWW7DA-WO-SVQSlE5SfKbQ/edit?usp=sharing here]<br />
# Began data exploration of the Phase I dataset (from project 107: Factors that Influence the Decision Not to Substantiate a CPS Referral)<br />
<br />
===Week 6===<br />
July 6 to July 12<br />
<br />
# Data Ethics Lecture (talk by Dr. Michael Zimmer)<br />
#* Empiricist epistemology and criticisms<br />
#** The idea that big data captures everything, so there is no need for theory or models, no need to worry about biased values, and no need to consult domain-specific experts<br />
#** Hidden biases in both the collection and analysis stages present considerable risks<br />
#* Rich, identifiable data from multiple sources on the same person (ex: different apps on phone)<br />
#* Questionable consent<br />
#** Clicking through the Terms of Use without reading it<br />
#** "Public" data used without identification efforts<br />
#* Reproducibility vs deidentification<br />
# More exploratory work and visualizations to highlight factors that would be interesting to look at from a technical perspective and their implications for any predictive systems<br />
# Making Research Posters (talk by Dr. Brylow)<br />
#* Usually posters display work that isn't so far along as to be published yet<br />
#* Stylistic tips to improve readability but also conserve space and convey the topic effectively from across the room<br />
#* Talk to each person for less than three minutes as this is only a peek into your work<br />
# Meeting with Mentors<br />
#* Focus on risk assessment in Phase I of the 107 data set. How is it measured? What is "normal" risk? Was it generated using a human or an algorithm? How well does it perform?<br />
#* Goals for the next two weeks<br />
#** Make a small multiple for all the factors in the set against risk assessment and write a few lines of analysis for each<br />
#** Start attempting to answer the questions posed above<br />
# Literature review<br />
#* [https://dgergle.soc.northwestern.edu//resources/pn3458-diazA.pdf Addressing Age-Related Bias in Sentiment Analysis]<br />
#* [https://www.researchgate.net/publication/257560404_Bias_in_algorithmic_filtering_and_personalization Bias in algorithmic filtering and personalization]<br />
#* Towards a Feminist HCI Methodology: Social Science, Feminism, and HCI by Shaowen Bardzell and Jeffrey Bardzell<br />
<br />
===Week 7===<br />
July 13 to July 19<br />
<br />
# Phase I deeper variable exploration<br />
#* Spreadsheet created for ease of sorting by source and type<br />
#* Determined and highlighted various measures of risk and their source (human vs modeled)<br />
#* Continued to read up on Phase I methods of collection <br />
#* Continued to create small multiples of the 664 variables<br />
# Literature Review<br />
#* [https://doi.org/10.1145/3274357 The misgendering machines: Trans/HCI implications of automatic gender recognition]<br />
#* [https://reallifemag.com/counting-the-countless/ Counting the Countless: Why data science is a profound threat for queer people]<br />
#* [https://ainowinstitute.org/discriminatingsystems.pdf Discriminating systems: Gender, race and power in AI]<br />
#* [https://doi.org/10.1145/3274424 Safe spaces and safe places: Unpacking technology-mediated experiences of safety and harm with transgender people]<br />
#* [https://doi.org/10.1145/3290607.3311750 Queer(ing) HCI: Moving forward in theory and practice]<br />
<br />
===Week 8===<br />
July 20 to July 26<br />
<br />
===Week 9===<br />
July 27 to August 2<br />
<br />
===Week 10===<br />
August 3 to August 9</div>Crepacihttps://reu.cs.mu.edu/index.php/User:CrepaciUser:Crepaci2020-07-21T16:39:18Z<p>Crepaci: /* Week 7 */</p>
<hr />
<div><br />
== About Me ==<br />
<br />
I'm [https://www.linkedin.com/in/charlie-repaci-07b723179/ Charlie Repaci], a senior at [https://www.simmons.edu Simmons University] studying [https://www.simmons.edu/undergraduate/academics/majors-minors/data-science-and-analytics Data Science], with a special interest in Biochemistry and Sociology. This summer I am working with [https://www.shionguha.net/ Dr. Shion Guha] and doctoral candidate [https://www.saxena.io/ Devansh Saxena] on [[Developing Ethical Algorithms for Placement Stability in the Foster Care System]].<br />
<br />
== Work Log ==<br />
<br />
===Week 1===<br />
June 1 to June 7<br />
# Orientation<br />
#* Introduction to other mentors, mentees, and REU heads Dr. Praveen Madiraju and Dr. Dennis Brylow<br />
#* Review of REU calendar and expectations<br />
# Data Science Bootcamp (talk by Dr. Madiraju)<br />
#* Introduction and basics of data analysis with python (Anaconda and Jypyter Notebook; pandas, numpy, matplotlib, seaborn, scipy)<br />
#** Read in data<br />
#** Pre-processing<br />
#** Modeling<br />
#** Data visualization<br />
# Good Research Practices (talk by Dr. Brylow)<br />
# Literature Review<br />
#* [https://dl.acm.org/doi/abs/10.1145/3313831.3376229 A Human-Centered Review of Algorithms used within the U.S. Child Welfare System]<br />
#* [https://dl.acm.org/doi/abs/10.1145/3290605.3300271 Toward Algorithmic Accountability in Public Services: A Qualitative Study of Affected Community Perspectives on Algorithmic Decision-making in Child Welfare Services]<br />
#* [https://dl.acm.org/doi/abs/10.1145/3323994.3369888 Child Welfare System: Interaction of Policy, Practice, and Algorithms]<br />
#* (Supplementary) [https://dl.acm.org/doi/abs/10.1145/3290605.3300497 Risk vs. Restriction: The Tension between Providing a Sense of Normalcy and Keeping Foster Teens Safe Online]<br />
<br />
===Week 2===<br />
June 8 to June 14<br />
# Responsible Conduct of Research Training (talk by Dr. Brylow)<br />
#* Ethical treatment of data<br />
#* Authorship, credit, plagiarism<br />
#* Human participants<br />
#* Intellectual property<br />
#* Conflicts of interest and professional standards<br />
# Worked on getting Citi certification for all three RCR sessions<br />
# Technical Writing Workshop (talk by Dr. Brylow and Dr. Madiraju)<br />
#* What the sections of a technical paper are<br />
#* What the publication process is like<br />
#* General tips<br />
# Literature review<br />
#* [http://dx.doi.org/10.2139/ssrn.2245322 Governing Algorithms: A Provocation Piece]<br />
#* [https://academiccommons.columbia.edu/doi/10.7916/D8ZK5TW2 Algorithmic Accountability Reporting: On the Investigation of Black Boxes]<br />
#* [http://www.tandfonline.com/doi/full/10.1080/1369118X.2016.1154087#abstract Thinking Critically About and Researching Algorithms]<br />
#* (Supplementary) [http://culturedigitally.org/wp-content/uploads/2016/07/Gillespie-2016-Algorithm-Digital-Keywords-Peters-ed.pdf Algorithm in Digital Keywords: a Vocabulary of Information, Society, and Culture]<br />
# Meeting with mentors<br />
#* Github created for the project<br />
#* Questions and discussion of the literature reviewed<br />
#* Planned work for the next two weeks<br />
<br />
===Week 3===<br />
June 15 to June 21<br />
# Set up Wiki entrees for myself and my project<br />
# Meeting with all REU interns to discuss our projects so far and any problems we have run into<br />
# Literature review<br />
#* [https://dl.acm.org/doi/abs/10.1145/3290605.3300760 Street-Level Algorithms: A Theory at the Gaps Between Policy and Decisions]<br />
#* [https://journals.sagepub.com/doi/abs/10.1177/2053951717738104 Algorithms as culture: Some tactics for the ethnography of algorithmic systems]<br />
#* (Supplementary) [https://journals.sagepub.com/doi/full/10.1177/2053951717751552 Algorithms as fetish: Faith and possibility in algorithmic work]<br />
# Meeting with Devansh to discuss the first dataset<br />
#* Understand variables and components -- reading the documentation that comes with datasets<br />
#* Import dataset into R and common problems<br />
<br />
===Week 4===<br />
June 22 to June 28<br />
<br />
# Importing data<br />
#* Need SAS to run a program included with the data files that creates .sas7bdat and .sas7bcat files from the .dat files that were given to us<br />
#* Some are not importing correctly for yet unknown reasons<br />
# Student Check-In (by Dr. Brylow and Dr. Madiraju)<br />
# Research Presentation (talk by guest lecturer Dr. Walter Bialkowski)<br />
#* Blood donation and potential risk of lower bone density to donors due to prolonged and repeated exposure to the anticoagulant citrate added to the blood (and then returned to the donor) during the donation process<br />
#* Study 1: Data analysis of Scandinavian blood donor data<br />
#** Concluded that there was no association between blood donation and the number of bone fractures donors had later in life<br />
#** Limitations included differences in blood donation policy, process, and popularity of varying donation types between Scandanavia and the United States<br />
#* Study 2: Longitudinal study of blood donors<br />
#** Concluded that current guidelines were enough to protect adult male donors between the ages of 20 and 65<br />
#** Limited in that conclusion can not be extrapolated to women or to men outside that age range<br />
# Presentation on work done in weeks 3 and 4 to mentors (readings, problems with data)<br />
<br />
===Week 5===<br />
June 29 to July 5<br />
<br />
# Research Presentations (talk by Dr. Brylow)<br />
#* Format and sections<br />
#** Very similar to the paper itself: introduction, background (know your audience), description of your work, results, conclusion<br />
#** Figure out 2-3 points you want your audience to take away from the talk and centralize your material around those<br />
#* General tips<br />
#** 1 slide per minute rule of thumb<br />
#** People sometimes bring extra slides in anticipation of questions<br />
#** Avoid distractions and keep it simple: avoid full sentences, use (and cite) diagrams, use simple color schemes (light text on a dark background reads well from afar)<br />
# Presented work done so far to other mentors and students<br />
#* You can find my slide deck with presentation notes [https://docs.google.com/presentation/d/1CUA1-6la1RL5SZk0vk4eeMWW7DA-WO-SVQSlE5SfKbQ/edit?usp=sharing here]<br />
# Began data exploration of the Phase I dataset (from project 107: Factors that Influence the Decision Not to Substantiate a CPS Referral)<br />
<br />
===Week 6===<br />
July 6 to July 12<br />
<br />
# Data Ethics Lecture (talk by Dr. Michael Zimmer)<br />
#* Empiricist epistemology and criticisms<br />
#** The idea that big data captures everything, so there is no need for theory or models, no need to worry about biased values, and no need to consult domain-specific experts<br />
#** Hidden biases in both the collection and analysis stages present considerable risks<br />
#* Rich, identifiable data from multiple sources on the same person (ex: different apps on phone)<br />
#* Questionable consent<br />
#** Clicking through the Terms of Use without reading it<br />
#** "Public" data used without identification efforts<br />
#* Reproducibility vs deidentification<br />
# More exploratory work and visualizations to highlight factors that would be interesting to look at from a technical perspective and their implications for any predictive systems<br />
# Making Research Posters (talk by Dr. Brylow)<br />
#* Usually posters display work that isn't so far along as to be published yet<br />
#* Stylistic tips to improve readability but also conserve space and convey the topic effectively from across the room<br />
#* Talk to each person for less than three minutes as this is only a peek into your work<br />
# Meeting with Mentors<br />
#* Focus on risk assessment in Phase I of the 107 data set. How is it measured? What is "normal" risk? Was it generated using a human or an algorithm? How well does it perform?<br />
#* Goals for the next two weeks<br />
#** Make a small multiple for all the factors in the set against risk assessment and write a few lines of analysis for each<br />
#** Start attempting to answer the questions posed above<br />
# Literature review<br />
#* [https://dgergle.soc.northwestern.edu//resources/pn3458-diazA.pdf Addressing Age-Related Bias in Sentiment Analysis]<br />
#* [https://www.researchgate.net/publication/257560404_Bias_in_algorithmic_filtering_and_personalization Bias in algorithmic filtering and personalization]<br />
#* Towards a Feminist HCI Methodology: Social Science, Feminism, and HCI by Shaowen Bardzell and Jeffrey Bardzell<br />
<br />
===Week 7===<br />
July 13 to July 19<br />
<br />
# Phase I deeper variable exploration<br />
#* Spreadsheet created for ease of sorting by source and type<br />
#* Determined and highlighted various measures of risk and their source (human vs modeled)<br />
#* Continued to read up on Phase I methods of collection <br />
#* Continued to create small multiples of the 664 variables<br />
# Literature Review<br />
<br />
===Week 8===<br />
July 20 to July 26<br />
<br />
===Week 9===<br />
July 27 to August 2<br />
<br />
===Week 10===<br />
August 3 to August 9</div>Crepacihttps://reu.cs.mu.edu/index.php/Developing_Ethical_Algorithms_for_Placement_Stability_in_the_Foster_Care_SystemDeveloping Ethical Algorithms for Placement Stability in the Foster Care System2020-07-13T14:55:52Z<p>Crepaci: /* Schedule: */</p>
<hr />
<div>Student: [[User:Crepaci|Charlie Repaci]]<br />
<br />
Mentor: [https://www.shionguha.net/ Dr. Shion Guha] and [https://saxena.io/ Devansh Saxena]<br />
<br />
==Description:==<br />
The goal of this project is to use a human-centered approach grounded in current social science theory and frameworks to add context to and further develop existing placement stability and risk assessment models that are used to aid overworked social workers in making, explaining, and standardizing their decisions.<br />
<br />
==Goals:==<br />
Quoted from the [https://reu.cs.mu.edu/index.php/Ethical_Algorithms_for_Placement_Stability_in_the_Foster_Care project page:]<br />
<br />
<blockquote>''This project aims to collaborate with the WI Department of Children and Families and SaintA (a private non-profit organization that actually provides foster care services in WI) by utilizing important, useful and contextual caseworkers judgment that are recorded as detailed case notes but never actually used to add norms, values, and context to existing algorithms that determine placement stability. Topic modeling will be used to extract latent themes from such text and incrementally added to existing placement stability models to test improvements in outcomes.''<br />
* ''Perform a literature review of topic modeling usage in the social science domain.''<br />
* ''Understand latent and human context from caseworker notes in Wisconsin foster care system using the data provided from the mentor.''<br />
* ''Review algorithmic biases, fairness, and transparency in the foster care system.''<br />
* ''Evaluate the latent themes from text and incrementally add to existing placement stability models to test improvement in outcomes.''</blockquote><br />
<br />
==Schedule:==<br />
{| class="wikitable"<br />
|-<br />
|'''Week'''<br />
|'''Description'''<br />
|-<br />
|Week 1<br />
|<br />
* Orientation<br />
* Data Science Bootcamp<br />
* Good Research Practices Lecture<br />
* Begin literature review -- algorithms in social work<br />
|-<br />
|Week 2<br />
|<br />
* Responsible Conduct of Research Training<br />
* CITI certification<br />
* Technical writing workshop<br />
* Continue literature review -- ethics and algorithms<br />
|-<br />
|Week 3<br />
|<br />
* Set up wiki<br />
* Begin reviewing data<br />
* Continue literature review -- bias in algorithms and policy<br />
|-<br />
|Week 4<br />
|<br />
* Import data<br />
* Research presentation by guest lecturer Dr. Walt Bialkowski<br />
* Presentation of work done in weeks 3 and 4 to mentors<br />
|-<br />
|Week 5<br />
|<br />
* Research Presentations lecture<br />
* Data exploration and visualization<br />
* Technical presentations workshop and informal presentations<br />
|-<br />
|Week 6<br />
|<br />
* Data Ethics by guest lecturer Dr. Michael Zimmer<br />
* Making Research Posters lecture<br />
* Continued Exploratory work and visualizations to highlight important factors<br />
* Continue literature review -- bias in algorithm development<br />
|-<br />
|Week 7<br />
|<br />
* Data analysis and model development<br />
* TBD<br />
|-<br />
|Week 8<br />
|<br />
* Data analysis and model development<br />
* TBD<br />
|-<br />
|Week 9<br />
|<br />
* Industry guest panel (Northwestern Mutual Data Science Institute)<br />
* Begin the final paper<br />
* Start to create project poster<br />
* Consider future work<br />
|-<br />
|Week 10<br />
|<br />
* Prepare and give the oral presentation<br />
* TBD: Present project to other REU sites and see their research in return<br />
* Finish and submit the final paper<br />
|}</div>Crepacihttps://reu.cs.mu.edu/index.php/User:CrepaciUser:Crepaci2020-07-13T14:39:12Z<p>Crepaci: /* Week 6 */</p>
<hr />
<div><br />
== About Me ==<br />
<br />
I'm [https://www.linkedin.com/in/charlie-repaci-07b723179/ Charlie Repaci], a senior at [https://www.simmons.edu Simmons University] studying [https://www.simmons.edu/undergraduate/academics/majors-minors/data-science-and-analytics Data Science], with a special interest in Biochemistry and Sociology. This summer I am working with [https://www.shionguha.net/ Dr. Shion Guha] and doctoral candidate [https://www.saxena.io/ Devansh Saxena] on [[Developing Ethical Algorithms for Placement Stability in the Foster Care System]].<br />
<br />
== Work Log ==<br />
<br />
===Week 1===<br />
June 1 to June 7<br />
# Orientation<br />
#* Introduction to other mentors, mentees, and REU heads Dr. Praveen Madiraju and Dr. Dennis Brylow<br />
#* Review of REU calendar and expectations<br />
# Data Science Bootcamp (talk by Dr. Madiraju)<br />
#* Introduction and basics of data analysis with python (Anaconda and Jypyter Notebook; pandas, numpy, matplotlib, seaborn, scipy)<br />
#** Read in data<br />
#** Pre-processing<br />
#** Modeling<br />
#** Data visualization<br />
# Good Research Practices (talk by Dr. Brylow)<br />
# Literature Review<br />
#* [https://dl.acm.org/doi/abs/10.1145/3313831.3376229 A Human-Centered Review of Algorithms used within the U.S. Child Welfare System]<br />
#* [https://dl.acm.org/doi/abs/10.1145/3290605.3300271 Toward Algorithmic Accountability in Public Services: A Qualitative Study of Affected Community Perspectives on Algorithmic Decision-making in Child Welfare Services]<br />
#* [https://dl.acm.org/doi/abs/10.1145/3323994.3369888 Child Welfare System: Interaction of Policy, Practice, and Algorithms]<br />
#* (Supplementary) [https://dl.acm.org/doi/abs/10.1145/3290605.3300497 Risk vs. Restriction: The Tension between Providing a Sense of Normalcy and Keeping Foster Teens Safe Online]<br />
<br />
===Week 2===<br />
June 8 to June 14<br />
# Responsible Conduct of Research Training (talk by Dr. Brylow)<br />
#* Ethical treatment of data<br />
#* Authorship, credit, plagiarism<br />
#* Human participants<br />
#* Intellectual property<br />
#* Conflicts of interest and professional standards<br />
# Worked on getting Citi certification for all three RCR sessions<br />
# Technical Writing Workshop (talk by Dr. Brylow and Dr. Madiraju)<br />
#* What the sections of a technical paper are<br />
#* What the publication process is like<br />
#* General tips<br />
# Literature review<br />
#* [http://dx.doi.org/10.2139/ssrn.2245322 Governing Algorithms: A Provocation Piece]<br />
#* [https://academiccommons.columbia.edu/doi/10.7916/D8ZK5TW2 Algorithmic Accountability Reporting: On the Investigation of Black Boxes]<br />
#* [http://www.tandfonline.com/doi/full/10.1080/1369118X.2016.1154087#abstract Thinking Critically About and Researching Algorithms]<br />
#* (Supplementary) [http://culturedigitally.org/wp-content/uploads/2016/07/Gillespie-2016-Algorithm-Digital-Keywords-Peters-ed.pdf Algorithm in Digital Keywords: a Vocabulary of Information, Society, and Culture]<br />
# Meeting with mentors<br />
#* Github created for the project<br />
#* Questions and discussion of the literature reviewed<br />
#* Planned work for the next two weeks<br />
<br />
===Week 3===<br />
June 15 to June 21<br />
# Set up Wiki entrees for myself and my project<br />
# Meeting with all REU interns to discuss our projects so far and any problems we have run into<br />
# Literature review<br />
#* [https://dl.acm.org/doi/abs/10.1145/3290605.3300760 Street-Level Algorithms: A Theory at the Gaps Between Policy and Decisions]<br />
#* [https://journals.sagepub.com/doi/abs/10.1177/2053951717738104 Algorithms as culture: Some tactics for the ethnography of algorithmic systems]<br />
#* (Supplementary) [https://journals.sagepub.com/doi/full/10.1177/2053951717751552 Algorithms as fetish: Faith and possibility in algorithmic work]<br />
# Meeting with Devansh to discuss the first dataset<br />
#* Understand variables and components -- reading the documentation that comes with datasets<br />
#* Import dataset into R and common problems<br />
<br />
===Week 4===<br />
June 22 to June 28<br />
<br />
# Importing data<br />
#* Need SAS to run a program included with the data files that creates .sas7bdat and .sas7bcat files from the .dat files that were given to us<br />
#* Some are not importing correctly for yet unknown reasons<br />
# Student Check-In (by Dr. Brylow and Dr. Madiraju)<br />
# Research Presentation (talk by guest lecturer Dr. Walter Bialkowski)<br />
#* Blood donation and potential risk of lower bone density to donors due to prolonged and repeated exposure to the anticoagulant citrate added to the blood (and then returned to the donor) during the donation process<br />
#* Study 1: Data analysis of Scandinavian blood donor data<br />
#** Concluded that there was no association between blood donation and the number of bone fractures donors had later in life<br />
#** Limitations included differences in blood donation policy, process, and popularity of varying donation types between Scandanavia and the United States<br />
#* Study 2: Longitudinal study of blood donors<br />
#** Concluded that current guidelines were enough to protect adult male donors between the ages of 20 and 65<br />
#** Limited in that conclusion can not be extrapolated to women or to men outside that age range<br />
# Presentation on work done in weeks 3 and 4 to mentors (readings, problems with data)<br />
<br />
===Week 5===<br />
June 29 to July 5<br />
<br />
# Research Presentations (talk by Dr. Brylow)<br />
#* Format and sections<br />
#** Very similar to the paper itself: introduction, background (know your audience), description of your work, results, conclusion<br />
#** Figure out 2-3 points you want your audience to take away from the talk and centralize your material around those<br />
#* General tips<br />
#** 1 slide per minute rule of thumb<br />
#** People sometimes bring extra slides in anticipation of questions<br />
#** Avoid distractions and keep it simple: avoid full sentences, use (and cite) diagrams, use simple color schemes (light text on a dark background reads well from afar)<br />
# Presented work done so far to other mentors and students<br />
#* You can find my slide deck with presentation notes [https://docs.google.com/presentation/d/1CUA1-6la1RL5SZk0vk4eeMWW7DA-WO-SVQSlE5SfKbQ/edit?usp=sharing here]<br />
# Began data exploration of the Phase I dataset (from project 107: Factors that Influence the Decision Not to Substantiate a CPS Referral)<br />
<br />
===Week 6===<br />
July 6 to July 12<br />
<br />
# Data Ethics Lecture (talk by Dr. Michael Zimmer)<br />
#* Empiricist epistemology and criticisms<br />
#** The idea that big data captures everything, so there is no need for theory or models, no need to worry about biased values, and no need to consult domain-specific experts<br />
#** Hidden biases in both the collection and analysis stages present considerable risks<br />
#* Rich, identifiable data from multiple sources on the same person (ex: different apps on phone)<br />
#* Questionable consent<br />
#** Clicking through the Terms of Use without reading it<br />
#** "Public" data used without identification efforts<br />
#* Reproducibility vs deidentification<br />
# More exploratory work and visualizations to highlight factors that would be interesting to look at from a technical perspective and their implications for any predictive systems<br />
# Making Research Posters (talk by Dr. Brylow)<br />
#* Usually posters display work that isn't so far along as to be published yet<br />
#* Stylistic tips to improve readability but also conserve space and convey the topic effectively from across the room<br />
#* Talk to each person for less than three minutes as this is only a peek into your work<br />
# Meeting with Mentors<br />
#* Focus on risk assessment in Phase I of the 107 data set. How is it measured? What is "normal" risk? Was it generated using a human or an algorithm? How well does it perform?<br />
#* Goals for the next two weeks<br />
#** Make a small multiple for all the factors in the set against risk assessment and write a few lines of analysis for each<br />
#** Start attempting to answer the questions posed above<br />
# Literature review<br />
#* [https://dgergle.soc.northwestern.edu//resources/pn3458-diazA.pdf Addressing Age-Related Bias in Sentiment Analysis]<br />
#* [https://www.researchgate.net/publication/257560404_Bias_in_algorithmic_filtering_and_personalization Bias in algorithmic filtering and personalization]<br />
#* Towards a Feminist HCI Methodology: Social Science, Feminism, and HCI by Shaowen Bardzell and Jeffrey Bardzell<br />
<br />
===Week 7===<br />
July 13 to July 19<br />
<br />
===Week 8===<br />
July 20 to July 26<br />
<br />
===Week 9===<br />
July 27 to August 2<br />
<br />
===Week 10===<br />
August 3 to August 9</div>Crepacihttps://reu.cs.mu.edu/index.php/User:CrepaciUser:Crepaci2020-07-13T14:38:43Z<p>Crepaci: /* Work Log */</p>
<hr />
<div><br />
== About Me ==<br />
<br />
I'm [https://www.linkedin.com/in/charlie-repaci-07b723179/ Charlie Repaci], a senior at [https://www.simmons.edu Simmons University] studying [https://www.simmons.edu/undergraduate/academics/majors-minors/data-science-and-analytics Data Science], with a special interest in Biochemistry and Sociology. This summer I am working with [https://www.shionguha.net/ Dr. Shion Guha] and doctoral candidate [https://www.saxena.io/ Devansh Saxena] on [[Developing Ethical Algorithms for Placement Stability in the Foster Care System]].<br />
<br />
== Work Log ==<br />
<br />
===Week 1===<br />
June 1 to June 7<br />
# Orientation<br />
#* Introduction to other mentors, mentees, and REU heads Dr. Praveen Madiraju and Dr. Dennis Brylow<br />
#* Review of REU calendar and expectations<br />
# Data Science Bootcamp (talk by Dr. Madiraju)<br />
#* Introduction and basics of data analysis with python (Anaconda and Jypyter Notebook; pandas, numpy, matplotlib, seaborn, scipy)<br />
#** Read in data<br />
#** Pre-processing<br />
#** Modeling<br />
#** Data visualization<br />
# Good Research Practices (talk by Dr. Brylow)<br />
# Literature Review<br />
#* [https://dl.acm.org/doi/abs/10.1145/3313831.3376229 A Human-Centered Review of Algorithms used within the U.S. Child Welfare System]<br />
#* [https://dl.acm.org/doi/abs/10.1145/3290605.3300271 Toward Algorithmic Accountability in Public Services: A Qualitative Study of Affected Community Perspectives on Algorithmic Decision-making in Child Welfare Services]<br />
#* [https://dl.acm.org/doi/abs/10.1145/3323994.3369888 Child Welfare System: Interaction of Policy, Practice, and Algorithms]<br />
#* (Supplementary) [https://dl.acm.org/doi/abs/10.1145/3290605.3300497 Risk vs. Restriction: The Tension between Providing a Sense of Normalcy and Keeping Foster Teens Safe Online]<br />
<br />
===Week 2===<br />
June 8 to June 14<br />
# Responsible Conduct of Research Training (talk by Dr. Brylow)<br />
#* Ethical treatment of data<br />
#* Authorship, credit, plagiarism<br />
#* Human participants<br />
#* Intellectual property<br />
#* Conflicts of interest and professional standards<br />
# Worked on getting Citi certification for all three RCR sessions<br />
# Technical Writing Workshop (talk by Dr. Brylow and Dr. Madiraju)<br />
#* What the sections of a technical paper are<br />
#* What the publication process is like<br />
#* General tips<br />
# Literature review<br />
#* [http://dx.doi.org/10.2139/ssrn.2245322 Governing Algorithms: A Provocation Piece]<br />
#* [https://academiccommons.columbia.edu/doi/10.7916/D8ZK5TW2 Algorithmic Accountability Reporting: On the Investigation of Black Boxes]<br />
#* [http://www.tandfonline.com/doi/full/10.1080/1369118X.2016.1154087#abstract Thinking Critically About and Researching Algorithms]<br />
#* (Supplementary) [http://culturedigitally.org/wp-content/uploads/2016/07/Gillespie-2016-Algorithm-Digital-Keywords-Peters-ed.pdf Algorithm in Digital Keywords: a Vocabulary of Information, Society, and Culture]<br />
# Meeting with mentors<br />
#* Github created for the project<br />
#* Questions and discussion of the literature reviewed<br />
#* Planned work for the next two weeks<br />
<br />
===Week 3===<br />
June 15 to June 21<br />
# Set up Wiki entrees for myself and my project<br />
# Meeting with all REU interns to discuss our projects so far and any problems we have run into<br />
# Literature review<br />
#* [https://dl.acm.org/doi/abs/10.1145/3290605.3300760 Street-Level Algorithms: A Theory at the Gaps Between Policy and Decisions]<br />
#* [https://journals.sagepub.com/doi/abs/10.1177/2053951717738104 Algorithms as culture: Some tactics for the ethnography of algorithmic systems]<br />
#* (Supplementary) [https://journals.sagepub.com/doi/full/10.1177/2053951717751552 Algorithms as fetish: Faith and possibility in algorithmic work]<br />
# Meeting with Devansh to discuss the first dataset<br />
#* Understand variables and components -- reading the documentation that comes with datasets<br />
#* Import dataset into R and common problems<br />
<br />
===Week 4===<br />
June 22 to June 28<br />
<br />
# Importing data<br />
#* Need SAS to run a program included with the data files that creates .sas7bdat and .sas7bcat files from the .dat files that were given to us<br />
#* Some are not importing correctly for yet unknown reasons<br />
# Student Check-In (by Dr. Brylow and Dr. Madiraju)<br />
# Research Presentation (talk by guest lecturer Dr. Walter Bialkowski)<br />
#* Blood donation and potential risk of lower bone density to donors due to prolonged and repeated exposure to the anticoagulant citrate added to the blood (and then returned to the donor) during the donation process<br />
#* Study 1: Data analysis of Scandinavian blood donor data<br />
#** Concluded that there was no association between blood donation and the number of bone fractures donors had later in life<br />
#** Limitations included differences in blood donation policy, process, and popularity of varying donation types between Scandanavia and the United States<br />
#* Study 2: Longitudinal study of blood donors<br />
#** Concluded that current guidelines were enough to protect adult male donors between the ages of 20 and 65<br />
#** Limited in that conclusion can not be extrapolated to women or to men outside that age range<br />
# Presentation on work done in weeks 3 and 4 to mentors (readings, problems with data)<br />
<br />
===Week 5===<br />
June 29 to July 5<br />
<br />
# Research Presentations (talk by Dr. Brylow)<br />
#* Format and sections<br />
#** Very similar to the paper itself: introduction, background (know your audience), description of your work, results, conclusion<br />
#** Figure out 2-3 points you want your audience to take away from the talk and centralize your material around those<br />
#* General tips<br />
#** 1 slide per minute rule of thumb<br />
#** People sometimes bring extra slides in anticipation of questions<br />
#** Avoid distractions and keep it simple: avoid full sentences, use (and cite) diagrams, use simple color schemes (light text on a dark background reads well from afar)<br />
# Presented work done so far to other mentors and students<br />
#* You can find my slide deck with presentation notes [https://docs.google.com/presentation/d/1CUA1-6la1RL5SZk0vk4eeMWW7DA-WO-SVQSlE5SfKbQ/edit?usp=sharing here]<br />
# Began data exploration of the Phase I dataset (from project 107: Factors that Influence the Decision Not to Substantiate a CPS Referral)<br />
<br />
===Week 6===<br />
July 6 to July 12<br />
<br />
# Data Ethics Lecture (talk by Dr. Michael Zimmer)<br />
#* Empiricist epistemology and criticisms<br />
#** The idea that big data captures everything, so there is no need for theory or models, no need to worry about biased values, and no need to consult domain-specific experts<br />
#** Hidden biases in both the collection and analysis stages present considerable risks<br />
#* Rich, identifiable data from multiple sources on the same person (ex: different apps on phone)<br />
#* Questionable consent<br />
#** Clicking through the Terms of Use without reading it<br />
#** "Public" data used without identification efforts<br />
#* Reproducibility vs deidentification<br />
# More exploratory work and visualizations to highlight factors that would be interesting to look at from a technical perspective and their implications for any predictive systems<br />
# Making Research Posters (talk by Dr. Brylow)<br />
#* Usually posters display work that isn't so far along as to be published yet<br />
#* Stylistic tips to improve readability but also conserve space and convey the topic effectively from across the room<br />
#* Talk to each person for less than three minutes as this is only a peek into your work<br />
# Meeting with Mentors<br />
#* Focus on risk assessment in Phase I of the 107 data set. How is it measured? What is "normal" risk? Was it generated using a human or an algorithm? How well does it perform?<br />
#* Goals for the next two weeks<br />
#** Make a small multiple for all the factors in the set against risk assessment and write a few lines of analysis for each<br />
#** Start attempting to answer the questions posed above<br />
# Literature review<br />
#* [https://dgergle.soc.northwestern.edu//resources/pn3458-diazA.pdf Addressing Age-Related Bias in Sentiment Analysis]<br />
#* [https://www.researchgate.net/publication/257560404_Bias_in_algorithmic_filtering_and_personalization Bias in algorithmic filtering and personalization]<br />
#* [Towards a Feminist HCI Methodology: Social Science, Feminism, and HCI by Shaowen Bardzell and Jeffrey Bardzell]<br />
<br />
===Week 7===<br />
July 13 to July 19<br />
<br />
===Week 8===<br />
July 20 to July 26<br />
<br />
===Week 9===<br />
July 27 to August 2<br />
<br />
===Week 10===<br />
August 3 to August 9</div>Crepacihttps://reu.cs.mu.edu/index.php/User:CrepaciUser:Crepaci2020-07-13T14:32:58Z<p>Crepaci: /* Week 6 */</p>
<hr />
<div><br />
== About Me ==<br />
<br />
I'm [https://www.linkedin.com/in/charlie-repaci-07b723179/ Charlie Repaci], a senior at [https://www.simmons.edu Simmons University] studying [https://www.simmons.edu/undergraduate/academics/majors-minors/data-science-and-analytics Data Science], with a special interest in Biochemistry and Sociology. This summer I am working with [https://www.shionguha.net/ Dr. Shion Guha] and doctoral candidate [https://www.saxena.io/ Devansh Saxena] on [[Developing Ethical Algorithms for Placement Stability in the Foster Care System]].<br />
<br />
== Work Log ==<br />
<br />
===Week 1===<br />
June 1 to June 7<br />
# Orientation<br />
#* Introduction to other mentors, mentees, and REU heads Dr. Praveen Madiraju and Dr. Dennis Brylow<br />
#* Review of REU calendar and expectations<br />
# Data Science Bootcamp (talk by Dr. Madiraju)<br />
#* Introduction and basics of data analysis with python (Anaconda and Jypyter Notebook; pandas, numpy, matplotlib, seaborn, scipy)<br />
#** Read in data<br />
#** Pre-processing<br />
#** Modeling<br />
#** Data visualization<br />
# Good Research Practices (talk by Dr. Brylow)<br />
# Literature Review<br />
#* [https://dl.acm.org/doi/abs/10.1145/3313831.3376229 A Human-Centered Review of Algorithms used within the U.S. Child Welfare System]<br />
#* [https://dl.acm.org/doi/abs/10.1145/3290605.3300271 Toward Algorithmic Accountability in Public Services: A Qualitative Study of Affected Community Perspectives on Algorithmic Decision-making in Child Welfare Services]<br />
#* [https://dl.acm.org/doi/abs/10.1145/3323994.3369888 Child Welfare System: Interaction of Policy, Practice, and Algorithms]<br />
#* (Supplementary) [https://dl.acm.org/doi/abs/10.1145/3290605.3300497 Risk vs. Restriction: The Tension between Providing a Sense of Normalcy and Keeping Foster Teens Safe Online]<br />
<br />
===Week 2===<br />
June 8 to June 14<br />
# Responsible Conduct of Research Training (talk by Dr. Brylow)<br />
#* Ethical treatment of data<br />
#* Authorship, credit, plagiarism<br />
#* Human participants<br />
#* Intellectual property<br />
#* Conflicts of interest and professional standards<br />
# Worked on getting Citi certification for all three RCR sessions<br />
# Technical Writing Workshop (talk by Dr. Brylow and Dr. Madiraju)<br />
#* What the sections of a technical paper are<br />
#* What the publication process is like<br />
#* General tips<br />
# Literature review<br />
#* [http://dx.doi.org/10.2139/ssrn.2245322 Governing Algorithms: A Provocation Piece]<br />
#* [https://academiccommons.columbia.edu/doi/10.7916/D8ZK5TW2 Algorithmic Accountability Reporting: On the Investigation of Black Boxes]<br />
#* [http://www.tandfonline.com/doi/full/10.1080/1369118X.2016.1154087#abstract Thinking Critically About and Researching Algorithms]<br />
#* (Supplementary) [http://culturedigitally.org/wp-content/uploads/2016/07/Gillespie-2016-Algorithm-Digital-Keywords-Peters-ed.pdf Algorithm in Digital Keywords: a Vocabulary of Information, Society, and Culture]<br />
# Meeting with mentors<br />
#* Github created for the project<br />
#* Questions and discussion of the literature reviewed<br />
#* Planned work for the next two weeks<br />
<br />
===Week 3===<br />
June 15 to June 21<br />
# Set up Wiki entrees for myself and my project<br />
# Meeting with all REU interns to discuss our projects so far and any problems we have run into<br />
# Literature review<br />
#* [https://dl.acm.org/doi/abs/10.1145/3290605.3300760 Street-Level Algorithms: A Theory at the Gaps Between Policy and Decisions]<br />
#* [https://journals.sagepub.com/doi/abs/10.1177/2053951717738104 Algorithms as culture: Some tactics for the ethnography of algorithmic systems]<br />
#* (Supplementary) [https://journals.sagepub.com/doi/full/10.1177/2053951717751552 Algorithms as fetish: Faith and possibility in algorithmic work]<br />
# Meeting with Devansh to discuss the first dataset<br />
#* Understand variables and components -- reading the documentation that comes with datasets<br />
#* Import dataset into R and common problems<br />
<br />
===Week 4===<br />
June 22 to June 28<br />
<br />
# Importing data<br />
#* Need SAS to run a program included with the data files that creates .sas7bdat and .sas7bcat files from the .dat files that were given to us<br />
#* Some are not importing correctly for yet unknown reasons<br />
# Student Check-In (by Dr. Brylow and Dr. Madiraju)<br />
# Research Presentation (talk by guest lecturer Dr. Walter Bialkowski)<br />
#* Blood donation and potential risk of lower bone density to donors due to prolonged and repeated exposure to the anticoagulant citrate added to the blood (and then returned to the donor) during the donation process<br />
#* Study 1: Data analysis of Scandinavian blood donor data<br />
#** Concluded that there was no association between blood donation and the number of bone fractures donors had later in life<br />
#** Limitations included differences in blood donation policy, process, and popularity of varying donation types between Scandanavia and the United States<br />
#* Study 2: Longitudinal study of blood donors<br />
#** Concluded that current guidelines were enough to protect adult male donors between the ages of 20 and 65<br />
#** Limited in that conclusion can not be extrapolated to women or to men outside that age range<br />
# Presentation on work done in weeks 3 and 4 to mentors (readings, problems with data)<br />
<br />
===Week 5===<br />
June 29 to July 5<br />
<br />
# Research Presentations (talk by Dr. Brylow)<br />
#* Format and sections<br />
#** Very similar to the paper itself: introduction, background (know your audience), description of your work, results, conclusion<br />
#** Figure out 2-3 points you want your audience to take away from the talk and centralize your material around those<br />
#* General tips<br />
#** 1 slide per minute rule of thumb<br />
#** People sometimes bring extra slides in anticipation of questions<br />
#** Avoid distractions and keep it simple: avoid full sentences, use (and cite) diagrams, use simple color schemes (light text on a dark background reads well from afar)<br />
# Presented work done so far to other mentors and students<br />
#* You can find my slide deck with presentation notes [https://docs.google.com/presentation/d/1CUA1-6la1RL5SZk0vk4eeMWW7DA-WO-SVQSlE5SfKbQ/edit?usp=sharing here]<br />
# Began data exploration of the Phase I dataset (from project 107: Factors that Influence the Decision Not to Substantiate a CPS Referral)<br />
<br />
===Week 6===<br />
July 6 to July 12<br />
<br />
# Data Ethics Lecture (talk by Dr. Michael Zimmer)<br />
#* Empiricist epistemology and criticisms<br />
#** The idea that big data captures everything, so there is no need for theory or models, no need to worry about biased values, and no need to consult domain-specific experts<br />
#** Hidden biases in both the collection and analysis stages present considerable risks<br />
#* Rich, identifiable data from multiple sources on the same person (ex: different apps on phone)<br />
#* Questionable consent<br />
#** Clicking through the Terms of Use without reading it<br />
#** "Public" data used without identification efforts<br />
#* Reproducibility vs deidentification<br />
# More exploratory work and visualizations to highlight factors that would be interesting to look at from a technical perspective and their implications for any predictive systems<br />
# Making Research Posters (talk by Dr. Brylow)<br />
#* Usually posters display work that isn't so far along as to be published yet<br />
#* Stylistic tips to improve readability but also conserve space and convey the topic effectively from across the room<br />
#* Talk to each person for less than three minutes as this is only a peek into your work<br />
# Meeting with Mentors<br />
#* Focus on risk assessment in Phase I of the 107 data set. How is it measured? What is "normal" risk? Was it generated using a human or an algorithm? How well does it perform?<br />
#* Goals for the next two weeks<br />
#** Make a small multiple for all the factors in the set against risk assessment and write a few lines of analysis for each<br />
#** Start attempting to answer the questions posed above<br />
# Literature Review<br />
#*<br />
<br />
===Week 7===<br />
July 13 to July 19<br />
<br />
===Week 8===<br />
July 20 to July 26<br />
<br />
===Week 9===<br />
July 27 to August 2<br />
<br />
===Week 10===<br />
August 3 to August 9</div>Crepacihttps://reu.cs.mu.edu/index.php/User:CrepaciUser:Crepaci2020-07-13T14:32:00Z<p>Crepaci: /* Week 6 */</p>
<hr />
<div><br />
== About Me ==<br />
<br />
I'm [https://www.linkedin.com/in/charlie-repaci-07b723179/ Charlie Repaci], a senior at [https://www.simmons.edu Simmons University] studying [https://www.simmons.edu/undergraduate/academics/majors-minors/data-science-and-analytics Data Science], with a special interest in Biochemistry and Sociology. This summer I am working with [https://www.shionguha.net/ Dr. Shion Guha] and doctoral candidate [https://www.saxena.io/ Devansh Saxena] on [[Developing Ethical Algorithms for Placement Stability in the Foster Care System]].<br />
<br />
== Work Log ==<br />
<br />
===Week 1===<br />
June 1 to June 7<br />
# Orientation<br />
#* Introduction to other mentors, mentees, and REU heads Dr. Praveen Madiraju and Dr. Dennis Brylow<br />
#* Review of REU calendar and expectations<br />
# Data Science Bootcamp (talk by Dr. Madiraju)<br />
#* Introduction and basics of data analysis with python (Anaconda and Jypyter Notebook; pandas, numpy, matplotlib, seaborn, scipy)<br />
#** Read in data<br />
#** Pre-processing<br />
#** Modeling<br />
#** Data visualization<br />
# Good Research Practices (talk by Dr. Brylow)<br />
# Literature Review<br />
#* [https://dl.acm.org/doi/abs/10.1145/3313831.3376229 A Human-Centered Review of Algorithms used within the U.S. Child Welfare System]<br />
#* [https://dl.acm.org/doi/abs/10.1145/3290605.3300271 Toward Algorithmic Accountability in Public Services: A Qualitative Study of Affected Community Perspectives on Algorithmic Decision-making in Child Welfare Services]<br />
#* [https://dl.acm.org/doi/abs/10.1145/3323994.3369888 Child Welfare System: Interaction of Policy, Practice, and Algorithms]<br />
#* (Supplementary) [https://dl.acm.org/doi/abs/10.1145/3290605.3300497 Risk vs. Restriction: The Tension between Providing a Sense of Normalcy and Keeping Foster Teens Safe Online]<br />
<br />
===Week 2===<br />
June 8 to June 14<br />
# Responsible Conduct of Research Training (talk by Dr. Brylow)<br />
#* Ethical treatment of data<br />
#* Authorship, credit, plagiarism<br />
#* Human participants<br />
#* Intellectual property<br />
#* Conflicts of interest and professional standards<br />
# Worked on getting Citi certification for all three RCR sessions<br />
# Technical Writing Workshop (talk by Dr. Brylow and Dr. Madiraju)<br />
#* What the sections of a technical paper are<br />
#* What the publication process is like<br />
#* General tips<br />
# Literature review<br />
#* [http://dx.doi.org/10.2139/ssrn.2245322 Governing Algorithms: A Provocation Piece]<br />
#* [https://academiccommons.columbia.edu/doi/10.7916/D8ZK5TW2 Algorithmic Accountability Reporting: On the Investigation of Black Boxes]<br />
#* [http://www.tandfonline.com/doi/full/10.1080/1369118X.2016.1154087#abstract Thinking Critically About and Researching Algorithms]<br />
#* (Supplementary) [http://culturedigitally.org/wp-content/uploads/2016/07/Gillespie-2016-Algorithm-Digital-Keywords-Peters-ed.pdf Algorithm in Digital Keywords: a Vocabulary of Information, Society, and Culture]<br />
# Meeting with mentors<br />
#* Github created for the project<br />
#* Questions and discussion of the literature reviewed<br />
#* Planned work for the next two weeks<br />
<br />
===Week 3===<br />
June 15 to June 21<br />
# Set up Wiki entrees for myself and my project<br />
# Meeting with all REU interns to discuss our projects so far and any problems we have run into<br />
# Literature review<br />
#* [https://dl.acm.org/doi/abs/10.1145/3290605.3300760 Street-Level Algorithms: A Theory at the Gaps Between Policy and Decisions]<br />
#* [https://journals.sagepub.com/doi/abs/10.1177/2053951717738104 Algorithms as culture: Some tactics for the ethnography of algorithmic systems]<br />
#* (Supplementary) [https://journals.sagepub.com/doi/full/10.1177/2053951717751552 Algorithms as fetish: Faith and possibility in algorithmic work]<br />
# Meeting with Devansh to discuss the first dataset<br />
#* Understand variables and components -- reading the documentation that comes with datasets<br />
#* Import dataset into R and common problems<br />
<br />
===Week 4===<br />
June 22 to June 28<br />
<br />
# Importing data<br />
#* Need SAS to run a program included with the data files that creates .sas7bdat and .sas7bcat files from the .dat files that were given to us<br />
#* Some are not importing correctly for yet unknown reasons<br />
# Student Check-In (by Dr. Brylow and Dr. Madiraju)<br />
# Research Presentation (talk by guest lecturer Dr. Walter Bialkowski)<br />
#* Blood donation and potential risk of lower bone density to donors due to prolonged and repeated exposure to the anticoagulant citrate added to the blood (and then returned to the donor) during the donation process<br />
#* Study 1: Data analysis of Scandinavian blood donor data<br />
#** Concluded that there was no association between blood donation and the number of bone fractures donors had later in life<br />
#** Limitations included differences in blood donation policy, process, and popularity of varying donation types between Scandanavia and the United States<br />
#* Study 2: Longitudinal study of blood donors<br />
#** Concluded that current guidelines were enough to protect adult male donors between the ages of 20 and 65<br />
#** Limited in that conclusion can not be extrapolated to women or to men outside that age range<br />
# Presentation on work done in weeks 3 and 4 to mentors (readings, problems with data)<br />
<br />
===Week 5===<br />
June 29 to July 5<br />
<br />
# Research Presentations (talk by Dr. Brylow)<br />
#* Format and sections<br />
#** Very similar to the paper itself: introduction, background (know your audience), description of your work, results, conclusion<br />
#** Figure out 2-3 points you want your audience to take away from the talk and centralize your material around those<br />
#* General tips<br />
#** 1 slide per minute rule of thumb<br />
#** People sometimes bring extra slides in anticipation of questions<br />
#** Avoid distractions and keep it simple: avoid full sentences, use (and cite) diagrams, use simple color schemes (light text on a dark background reads well from afar)<br />
# Presented work done so far to other mentors and students<br />
#* You can find my slide deck with presentation notes [https://docs.google.com/presentation/d/1CUA1-6la1RL5SZk0vk4eeMWW7DA-WO-SVQSlE5SfKbQ/edit?usp=sharing here]<br />
# Began data exploration of the Phase I dataset (from project 107: Factors that Influence the Decision Not to Substantiate a CPS Referral)<br />
<br />
===Week 6===<br />
July 6 to July 12<br />
<br />
# Data Ethics Lecture (talk by Dr. Michael Zimmer)<br />
#* Empiricist epistemology and criticisms<br />
#** The idea that big data captures everything, so there is no need for theory or models, no need to worry about biased values, and no need to consult domain-specific experts<br />
#** Hidden biases in both the collection and analysis stages present considerable risks<br />
#* Rich, identifiable data from multiple sources on the same person (ex: different apps on phone)<br />
#* Questionable consent<br />
#** Clicking through the Terms of Use without reading it<br />
#** "Public" data used without identification efforts<br />
#* Reproducibility vs deidentification<br />
# More exploratory work and visualizations to highlight factors that would be interesting to look at from a technical perspective and their implications for any predictive systems<br />
# Making Research Posters (talk by Dr. Brylow)<br />
#* Usually posters display work that isn't so far along as to be published yet<br />
#* Stylistic tips to improve readability but also conserve space and convey the topic effectively from across the room<br />
#* Talk to each person for less than three minutes as this is only a peek into your work<br />
# Meeting with Mentors<br />
#* Focus on risk assessment in Phase I of the 107 data set. How is it measured? What is "normal" risk? Was it generated using a human or an algorithm? How well does it perform?<br />
#* Goals for the next two weeks<br />
#** Make a small multiple for all the factors in the set against risk assessment and write a few lines of analysis for each<br />
#** Start attempting to answer the questions posed above<br />
<br />
===Week 7===<br />
July 13 to July 19<br />
<br />
===Week 8===<br />
July 20 to July 26<br />
<br />
===Week 9===<br />
July 27 to August 2<br />
<br />
===Week 10===<br />
August 3 to August 9</div>Crepacihttps://reu.cs.mu.edu/index.php/Developing_Ethical_Algorithms_for_Placement_Stability_in_the_Foster_Care_SystemDeveloping Ethical Algorithms for Placement Stability in the Foster Care System2020-07-06T18:02:00Z<p>Crepaci: /* Schedule: */</p>
<hr />
<div>Student: [[User:Crepaci|Charlie Repaci]]<br />
<br />
Mentor: [https://www.shionguha.net/ Dr. Shion Guha] and [https://saxena.io/ Devansh Saxena]<br />
<br />
==Description:==<br />
The goal of this project is to use a human-centered approach grounded in current social science theory and frameworks to add context to and further develop existing placement stability and risk assessment models that are used to aid overworked social workers in making, explaining, and standardizing their decisions.<br />
<br />
==Goals:==<br />
Quoted from the [https://reu.cs.mu.edu/index.php/Ethical_Algorithms_for_Placement_Stability_in_the_Foster_Care project page:]<br />
<br />
<blockquote>''This project aims to collaborate with the WI Department of Children and Families and SaintA (a private non-profit organization that actually provides foster care services in WI) by utilizing important, useful and contextual caseworkers judgment that are recorded as detailed case notes but never actually used to add norms, values, and context to existing algorithms that determine placement stability. Topic modeling will be used to extract latent themes from such text and incrementally added to existing placement stability models to test improvements in outcomes.''<br />
* ''Perform a literature review of topic modeling usage in the social science domain.''<br />
* ''Understand latent and human context from caseworker notes in Wisconsin foster care system using the data provided from the mentor.''<br />
* ''Review algorithmic biases, fairness, and transparency in the foster care system.''<br />
* ''Evaluate the latent themes from text and incrementally add to existing placement stability models to test improvement in outcomes.''</blockquote><br />
<br />
==Schedule:==<br />
{| class="wikitable"<br />
|-<br />
|'''Week'''<br />
|'''Description'''<br />
|-<br />
|Week 1<br />
|<br />
* Orientation<br />
* Data Science Bootcamp<br />
* Good Research Practices Lecture<br />
* Begin literature review -- algorithms in social work<br />
|-<br />
|Week 2<br />
|<br />
* Responsible Conduct of Research Training<br />
* CITI certification<br />
* Technical writing workshop<br />
* Continue literature review -- ethics and algorithms<br />
|-<br />
|Week 3<br />
|<br />
* Set up wiki<br />
* Begin reviewing data<br />
* Continue literature review -- bias in algorithms and policy<br />
|-<br />
|Week 4<br />
|<br />
* Import data<br />
* Research presentation by guest lecturer Dr. Walt Bialkowski<br />
* Presentation of work done in weeks 3 and 4 to mentors<br />
|-<br />
|Week 5<br />
|<br />
* Data exploration and visualization<br />
* Technical presentations workshop and informal presentations<br />
|-<br />
|Week 6<br />
|<br />
* Data analysis and model development<br />
* Data ethics special lecture<br />
* Research posters workshop<br />
* TBD<br />
|-<br />
|Week 7<br />
|<br />
* Data analysis and model development<br />
* TBD<br />
|-<br />
|Week 8<br />
|<br />
* Data analysis and model development<br />
* TBD<br />
|-<br />
|Week 9<br />
|<br />
* Industry guest panel (Northwestern Mutual Data Science Institute)<br />
* Begin the final paper<br />
* Start to create project poster<br />
* Consider future work<br />
|-<br />
|Week 10<br />
|<br />
* Prepare and give the oral presentation<br />
* TBD: Present project to other REU sites and see their research in return<br />
* Finish and submit the final paper<br />
|}</div>Crepacihttps://reu.cs.mu.edu/index.php/User:CrepaciUser:Crepaci2020-07-06T17:58:21Z<p>Crepaci: /* Week 5 */</p>
<hr />
<div><br />
== About Me ==<br />
<br />
I'm [https://www.linkedin.com/in/charlie-repaci-07b723179/ Charlie Repaci], a senior at [https://www.simmons.edu Simmons University] studying [https://www.simmons.edu/undergraduate/academics/majors-minors/data-science-and-analytics Data Science], with a special interest in Biochemistry and Sociology. This summer I am working with [https://www.shionguha.net/ Dr. Shion Guha] and doctoral candidate [https://www.saxena.io/ Devansh Saxena] on [[Developing Ethical Algorithms for Placement Stability in the Foster Care System]].<br />
<br />
== Work Log ==<br />
<br />
===Week 1===<br />
June 1 to June 7<br />
# Orientation<br />
#* Introduction to other mentors, mentees, and REU heads Dr. Praveen Madiraju and Dr. Dennis Brylow<br />
#* Review of REU calendar and expectations<br />
# Data Science Bootcamp (talk by Dr. Madiraju)<br />
#* Introduction and basics of data analysis with python (Anaconda and Jypyter Notebook; pandas, numpy, matplotlib, seaborn, scipy)<br />
#** Read in data<br />
#** Pre-processing<br />
#** Modeling<br />
#** Data visualization<br />
# Good Research Practices (talk by Dr. Brylow)<br />
# Literature Review<br />
#* [https://dl.acm.org/doi/abs/10.1145/3313831.3376229 A Human-Centered Review of Algorithms used within the U.S. Child Welfare System]<br />
#* [https://dl.acm.org/doi/abs/10.1145/3290605.3300271 Toward Algorithmic Accountability in Public Services: A Qualitative Study of Affected Community Perspectives on Algorithmic Decision-making in Child Welfare Services]<br />
#* [https://dl.acm.org/doi/abs/10.1145/3323994.3369888 Child Welfare System: Interaction of Policy, Practice, and Algorithms]<br />
#* (Supplementary) [https://dl.acm.org/doi/abs/10.1145/3290605.3300497 Risk vs. Restriction: The Tension between Providing a Sense of Normalcy and Keeping Foster Teens Safe Online]<br />
<br />
===Week 2===<br />
June 8 to June 14<br />
# Responsible Conduct of Research Training (talk by Dr. Brylow)<br />
#* Ethical treatment of data<br />
#* Authorship, credit, plagiarism<br />
#* Human participants<br />
#* Intellectual property<br />
#* Conflicts of interest and professional standards<br />
# Worked on getting Citi certification for all three RCR sessions<br />
# Technical Writing Workshop (talk by Dr. Brylow and Dr. Madiraju)<br />
#* What the sections of a technical paper are<br />
#* What the publication process is like<br />
#* General tips<br />
# Literature review<br />
#* [http://dx.doi.org/10.2139/ssrn.2245322 Governing Algorithms: A Provocation Piece]<br />
#* [https://academiccommons.columbia.edu/doi/10.7916/D8ZK5TW2 Algorithmic Accountability Reporting: On the Investigation of Black Boxes]<br />
#* [http://www.tandfonline.com/doi/full/10.1080/1369118X.2016.1154087#abstract Thinking Critically About and Researching Algorithms]<br />
#* (Supplementary) [http://culturedigitally.org/wp-content/uploads/2016/07/Gillespie-2016-Algorithm-Digital-Keywords-Peters-ed.pdf Algorithm in Digital Keywords: a Vocabulary of Information, Society, and Culture]<br />
# Meeting with mentors<br />
#* Github created for the project<br />
#* Questions and discussion of the literature reviewed<br />
#* Planned work for the next two weeks<br />
<br />
===Week 3===<br />
June 15 to June 21<br />
# Set up Wiki entrees for myself and my project<br />
# Meeting with all REU interns to discuss our projects so far and any problems we have run into<br />
# Literature review<br />
#* [https://dl.acm.org/doi/abs/10.1145/3290605.3300760 Street-Level Algorithms: A Theory at the Gaps Between Policy and Decisions]<br />
#* [https://journals.sagepub.com/doi/abs/10.1177/2053951717738104 Algorithms as culture: Some tactics for the ethnography of algorithmic systems]<br />
#* (Supplementary) [https://journals.sagepub.com/doi/full/10.1177/2053951717751552 Algorithms as fetish: Faith and possibility in algorithmic work]<br />
# Meeting with Devansh to discuss the first dataset<br />
#* Understand variables and components -- reading the documentation that comes with datasets<br />
#* Import dataset into R and common problems<br />
<br />
===Week 4===<br />
June 22 to June 28<br />
<br />
# Importing data<br />
#* Need SAS to run a program included with the data files that creates .sas7bdat and .sas7bcat files from the .dat files that were given to us<br />
#* Some are not importing correctly for yet unknown reasons<br />
# Student Check-In (by Dr. Brylow and Dr. Madiraju)<br />
# Research Presentation (talk by guest lecturer Dr. Walter Bialkowski)<br />
#* Blood donation and potential risk of lower bone density to donors due to prolonged and repeated exposure to the anticoagulant citrate added to the blood (and then returned to the donor) during the donation process<br />
#* Study 1: Data analysis of Scandinavian blood donor data<br />
#** Concluded that there was no association between blood donation and the number of bone fractures donors had later in life<br />
#** Limitations included differences in blood donation policy, process, and popularity of varying donation types between Scandanavia and the United States<br />
#* Study 2: Longitudinal study of blood donors<br />
#** Concluded that current guidelines were enough to protect adult male donors between the ages of 20 and 65<br />
#** Limited in that conclusion can not be extrapolated to women or to men outside that age range<br />
# Presentation on work done in weeks 3 and 4 to mentors (readings, problems with data)<br />
<br />
===Week 5===<br />
June 29 to July 5<br />
<br />
# Research Presentations (talk by Dr. Brylow)<br />
#* Format and sections<br />
#** Very similar to the paper itself: introduction, background (know your audience), description of your work, results, conclusion<br />
#** Figure out 2-3 points you want your audience to take away from the talk and centralize your material around those<br />
#* General tips<br />
#** 1 slide per minute rule of thumb<br />
#** People sometimes bring extra slides in anticipation of questions<br />
#** Avoid distractions and keep it simple: avoid full sentences, use (and cite) diagrams, use simple color schemes (light text on a dark background reads well from afar)<br />
# Presented work done so far to other mentors and students<br />
#* You can find my slide deck with presentation notes [https://docs.google.com/presentation/d/1CUA1-6la1RL5SZk0vk4eeMWW7DA-WO-SVQSlE5SfKbQ/edit?usp=sharing here]<br />
# Began data exploration of the Phase I dataset (from project 107: Factors that Influence the Decision Not to Substantiate a CPS Referral)<br />
<br />
===Week 6===<br />
July 6 to July 12<br />
<br />
===Week 7===<br />
July 13 to July 19<br />
<br />
===Week 8===<br />
July 20 to July 26<br />
<br />
===Week 9===<br />
July 27 to August 2<br />
<br />
===Week 10===<br />
August 3 to August 9</div>Crepacihttps://reu.cs.mu.edu/index.php/Developing_Ethical_Algorithms_for_Placement_Stability_in_the_Foster_Care_SystemDeveloping Ethical Algorithms for Placement Stability in the Foster Care System2020-06-29T17:40:11Z<p>Crepaci: /* Schedule: */</p>
<hr />
<div>Student: [[User:Crepaci|Charlie Repaci]]<br />
<br />
Mentor: [https://www.shionguha.net/ Dr. Shion Guha] and [https://saxena.io/ Devansh Saxena]<br />
<br />
==Description:==<br />
The goal of this project is to use a human-centered approach grounded in current social science theory and frameworks to add context to and further develop existing placement stability and risk assessment models that are used to aid overworked social workers in making, explaining, and standardizing their decisions.<br />
<br />
==Goals:==<br />
Quoted from the [https://reu.cs.mu.edu/index.php/Ethical_Algorithms_for_Placement_Stability_in_the_Foster_Care project page:]<br />
<br />
<blockquote>''This project aims to collaborate with the WI Department of Children and Families and SaintA (a private non-profit organization that actually provides foster care services in WI) by utilizing important, useful and contextual caseworkers judgment that are recorded as detailed case notes but never actually used to add norms, values, and context to existing algorithms that determine placement stability. Topic modeling will be used to extract latent themes from such text and incrementally added to existing placement stability models to test improvements in outcomes.''<br />
* ''Perform a literature review of topic modeling usage in the social science domain.''<br />
* ''Understand latent and human context from caseworker notes in Wisconsin foster care system using the data provided from the mentor.''<br />
* ''Review algorithmic biases, fairness, and transparency in the foster care system.''<br />
* ''Evaluate the latent themes from text and incrementally add to existing placement stability models to test improvement in outcomes.''</blockquote><br />
<br />
==Schedule:==<br />
{| class="wikitable"<br />
|-<br />
|'''Week'''<br />
|'''Description'''<br />
|-<br />
|Week 1<br />
|<br />
* Orientation<br />
* Data Science Bootcamp<br />
* Good Research Practices Lecture<br />
* Begin literature review -- algorithms in social work<br />
|-<br />
|Week 2<br />
|<br />
* Responsible Conduct of Research Training<br />
* CITI certification<br />
* Technical writing workshop<br />
* Continue literature review -- ethics and algorithms<br />
|-<br />
|Week 3<br />
|<br />
* Set up wiki<br />
* Begin reviewing data<br />
* Continue literature review -- bias in algorithms and policy<br />
|-<br />
|Week 4<br />
|<br />
* Import data<br />
* Research presentation by guest lecturer Dr. Walt Bialkowski<br />
* Presentation of work done in weeks 3 and 4 to mentors<br />
|-<br />
|Week 5<br />
|<br />
* Data exploration and visualization<br />
* Technical presentations workshop and informal presentations<br />
* TBD<br />
|-<br />
|Week 6<br />
|<br />
* Data analysis and model development<br />
* Data ethics special lecture<br />
* Research posters workshop<br />
* TBD<br />
|-<br />
|Week 7<br />
|<br />
* Data analysis and model development<br />
* TBD<br />
|-<br />
|Week 8<br />
|<br />
* Data analysis and model development<br />
* TBD<br />
|-<br />
|Week 9<br />
|<br />
* Industry guest panel (Northwestern Mutual Data Science Institute)<br />
* Begin the final paper<br />
* Start to create project poster<br />
* Consider future work<br />
|-<br />
|Week 10<br />
|<br />
* Prepare and give the oral presentation<br />
* TBD: Present project to other REU sites and see their research in return<br />
* Finish and submit the final paper<br />
|}</div>Crepacihttps://reu.cs.mu.edu/index.php/User:CrepaciUser:Crepaci2020-06-29T17:04:36Z<p>Crepaci: /* Week 4 */</p>
<hr />
<div><br />
== About Me ==<br />
<br />
I'm [https://www.linkedin.com/in/charlie-repaci-07b723179/ Charlie Repaci], a senior at [https://www.simmons.edu Simmons University] studying [https://www.simmons.edu/undergraduate/academics/majors-minors/data-science-and-analytics Data Science], with a special interest in Biochemistry and Sociology. This summer I am working with [https://www.shionguha.net/ Dr. Shion Guha] and doctoral candidate [https://www.saxena.io/ Devansh Saxena] on [[Developing Ethical Algorithms for Placement Stability in the Foster Care System]].<br />
<br />
== Work Log ==<br />
<br />
===Week 1===<br />
June 1 to June 7<br />
# Orientation<br />
#* Introduction to other mentors, mentees, and REU heads Dr. Praveen Madiraju and Dr. Dennis Brylow<br />
#* Review of REU calendar and expectations<br />
# Data Science Bootcamp (talk by Dr. Madiraju)<br />
#* Introduction and basics of data analysis with python (Anaconda and Jypyter Notebook; pandas, numpy, matplotlib, seaborn, scipy)<br />
#** Read in data<br />
#** Pre-processing<br />
#** Modeling<br />
#** Data visualization<br />
# Good Research Practices (talk by Dr. Brylow)<br />
# Literature Review<br />
#* [https://dl.acm.org/doi/abs/10.1145/3313831.3376229 A Human-Centered Review of Algorithms used within the U.S. Child Welfare System]<br />
#* [https://dl.acm.org/doi/abs/10.1145/3290605.3300271 Toward Algorithmic Accountability in Public Services: A Qualitative Study of Affected Community Perspectives on Algorithmic Decision-making in Child Welfare Services]<br />
#* [https://dl.acm.org/doi/abs/10.1145/3323994.3369888 Child Welfare System: Interaction of Policy, Practice, and Algorithms]<br />
#* (Supplementary) [https://dl.acm.org/doi/abs/10.1145/3290605.3300497 Risk vs. Restriction: The Tension between Providing a Sense of Normalcy and Keeping Foster Teens Safe Online]<br />
<br />
===Week 2===<br />
June 8 to June 14<br />
# Responsible Conduct of Research Training (talk by Dr. Brylow)<br />
#* Ethical treatment of data<br />
#* Authorship, credit, plagiarism<br />
#* Human participants<br />
#* Intellectual property<br />
#* Conflicts of interest and professional standards<br />
# Worked on getting Citi certification for all three RCR sessions<br />
# Technical Writing Workshop (talk by Dr. Brylow and Dr. Madiraju)<br />
#* What the sections of a technical paper are<br />
#* What the publication process is like<br />
#* General tips<br />
# Literature review<br />
#* [http://dx.doi.org/10.2139/ssrn.2245322 Governing Algorithms: A Provocation Piece]<br />
#* [https://academiccommons.columbia.edu/doi/10.7916/D8ZK5TW2 Algorithmic Accountability Reporting: On the Investigation of Black Boxes]<br />
#* [http://www.tandfonline.com/doi/full/10.1080/1369118X.2016.1154087#abstract Thinking Critically About and Researching Algorithms]<br />
#* (Supplementary) [http://culturedigitally.org/wp-content/uploads/2016/07/Gillespie-2016-Algorithm-Digital-Keywords-Peters-ed.pdf Algorithm in Digital Keywords: a Vocabulary of Information, Society, and Culture]<br />
# Meeting with mentors<br />
#* Github created for the project<br />
#* Questions and discussion of the literature reviewed<br />
#* Planned work for the next two weeks<br />
<br />
===Week 3===<br />
June 15 to June 21<br />
# Set up Wiki entrees for myself and my project<br />
# Meeting with all REU interns to discuss our projects so far and any problems we have run into<br />
# Literature review<br />
#* [https://dl.acm.org/doi/abs/10.1145/3290605.3300760 Street-Level Algorithms: A Theory at the Gaps Between Policy and Decisions]<br />
#* [https://journals.sagepub.com/doi/abs/10.1177/2053951717738104 Algorithms as culture: Some tactics for the ethnography of algorithmic systems]<br />
#* (Supplementary) [https://journals.sagepub.com/doi/full/10.1177/2053951717751552 Algorithms as fetish: Faith and possibility in algorithmic work]<br />
# Meeting with Devansh to discuss the first dataset<br />
#* Understand variables and components -- reading the documentation that comes with datasets<br />
#* Import dataset into R and common problems<br />
<br />
===Week 4===<br />
June 22 to June 28<br />
<br />
# Importing data<br />
#* Need SAS to run a program included with the data files that creates .sas7bdat and .sas7bcat files from the .dat files that were given to us<br />
#* Some are not importing correctly for yet unknown reasons<br />
# Student Check-In (by Dr. Brylow and Dr. Madiraju)<br />
# Research Presentation (talk by guest lecturer Dr. Walter Bialkowski)<br />
#* Blood donation and potential risk of lower bone density to donors due to prolonged and repeated exposure to the anticoagulant citrate added to the blood (and then returned to the donor) during the donation process<br />
#* Study 1: Data analysis of Scandinavian blood donor data<br />
#** Concluded that there was no association between blood donation and the number of bone fractures donors had later in life<br />
#** Limitations included differences in blood donation policy, process, and popularity of varying donation types between Scandanavia and the United States<br />
#* Study 2: Longitudinal study of blood donors<br />
#** Concluded that current guidelines were enough to protect adult male donors between the ages of 20 and 65<br />
#** Limited in that conclusion can not be extrapolated to women or to men outside that age range<br />
# Presentation on work done in weeks 3 and 4 to mentors (readings, problems with data)<br />
<br />
===Week 5===<br />
June 29 to July 5<br />
<br />
===Week 6===<br />
July 6 to July 12<br />
<br />
===Week 7===<br />
July 13 to July 19<br />
<br />
===Week 8===<br />
July 20 to July 26<br />
<br />
===Week 9===<br />
July 27 to August 2<br />
<br />
===Week 10===<br />
August 3 to August 9</div>Crepacihttps://reu.cs.mu.edu/index.php/User:CrepaciUser:Crepaci2020-06-29T17:03:37Z<p>Crepaci: /* Week 4 */</p>
<hr />
<div><br />
== About Me ==<br />
<br />
I'm [https://www.linkedin.com/in/charlie-repaci-07b723179/ Charlie Repaci], a senior at [https://www.simmons.edu Simmons University] studying [https://www.simmons.edu/undergraduate/academics/majors-minors/data-science-and-analytics Data Science], with a special interest in Biochemistry and Sociology. This summer I am working with [https://www.shionguha.net/ Dr. Shion Guha] and doctoral candidate [https://www.saxena.io/ Devansh Saxena] on [[Developing Ethical Algorithms for Placement Stability in the Foster Care System]].<br />
<br />
== Work Log ==<br />
<br />
===Week 1===<br />
June 1 to June 7<br />
# Orientation<br />
#* Introduction to other mentors, mentees, and REU heads Dr. Praveen Madiraju and Dr. Dennis Brylow<br />
#* Review of REU calendar and expectations<br />
# Data Science Bootcamp (talk by Dr. Madiraju)<br />
#* Introduction and basics of data analysis with python (Anaconda and Jypyter Notebook; pandas, numpy, matplotlib, seaborn, scipy)<br />
#** Read in data<br />
#** Pre-processing<br />
#** Modeling<br />
#** Data visualization<br />
# Good Research Practices (talk by Dr. Brylow)<br />
# Literature Review<br />
#* [https://dl.acm.org/doi/abs/10.1145/3313831.3376229 A Human-Centered Review of Algorithms used within the U.S. Child Welfare System]<br />
#* [https://dl.acm.org/doi/abs/10.1145/3290605.3300271 Toward Algorithmic Accountability in Public Services: A Qualitative Study of Affected Community Perspectives on Algorithmic Decision-making in Child Welfare Services]<br />
#* [https://dl.acm.org/doi/abs/10.1145/3323994.3369888 Child Welfare System: Interaction of Policy, Practice, and Algorithms]<br />
#* (Supplementary) [https://dl.acm.org/doi/abs/10.1145/3290605.3300497 Risk vs. Restriction: The Tension between Providing a Sense of Normalcy and Keeping Foster Teens Safe Online]<br />
<br />
===Week 2===<br />
June 8 to June 14<br />
# Responsible Conduct of Research Training (talk by Dr. Brylow)<br />
#* Ethical treatment of data<br />
#* Authorship, credit, plagiarism<br />
#* Human participants<br />
#* Intellectual property<br />
#* Conflicts of interest and professional standards<br />
# Worked on getting Citi certification for all three RCR sessions<br />
# Technical Writing Workshop (talk by Dr. Brylow and Dr. Madiraju)<br />
#* What the sections of a technical paper are<br />
#* What the publication process is like<br />
#* General tips<br />
# Literature review<br />
#* [http://dx.doi.org/10.2139/ssrn.2245322 Governing Algorithms: A Provocation Piece]<br />
#* [https://academiccommons.columbia.edu/doi/10.7916/D8ZK5TW2 Algorithmic Accountability Reporting: On the Investigation of Black Boxes]<br />
#* [http://www.tandfonline.com/doi/full/10.1080/1369118X.2016.1154087#abstract Thinking Critically About and Researching Algorithms]<br />
#* (Supplementary) [http://culturedigitally.org/wp-content/uploads/2016/07/Gillespie-2016-Algorithm-Digital-Keywords-Peters-ed.pdf Algorithm in Digital Keywords: a Vocabulary of Information, Society, and Culture]<br />
# Meeting with mentors<br />
#* Github created for the project<br />
#* Questions and discussion of the literature reviewed<br />
#* Planned work for the next two weeks<br />
<br />
===Week 3===<br />
June 15 to June 21<br />
# Set up Wiki entrees for myself and my project<br />
# Meeting with all REU interns to discuss our projects so far and any problems we have run into<br />
# Literature review<br />
#* [https://dl.acm.org/doi/abs/10.1145/3290605.3300760 Street-Level Algorithms: A Theory at the Gaps Between Policy and Decisions]<br />
#* [https://journals.sagepub.com/doi/abs/10.1177/2053951717738104 Algorithms as culture: Some tactics for the ethnography of algorithmic systems]<br />
#* (Supplementary) [https://journals.sagepub.com/doi/full/10.1177/2053951717751552 Algorithms as fetish: Faith and possibility in algorithmic work]<br />
# Meeting with Devansh to discuss the first dataset<br />
#* Understand variables and components -- reading the documentation that comes with datasets<br />
#* Import dataset into R and common problems<br />
<br />
===Week 4===<br />
June 22 to June 28<br />
<br />
# Importing data<br />
#* Need SAS to run a program included with the data files that creates .sas7bdat and .sas7bcat files from the .dat files that were given to us<br />
#* Some are not importing correctly for yet unknown reasons<br />
# Student Check-In (by Dr. Brylow and Dr. Madiraju)<br />
# Research Presentation (talk by guest lecturer Dr. Walter Bialkowski)<br />
#* Blood donation and potential risk of lower bone density to donors due to prolonged and repeated exposure to the anticoagulant citrate added to the blood (and then returned to the donor) during the donation process<br />
#* Study 1: Data analysis of Scandinavian blood donor data<br />
#** Concluded that there was no association between blood donation and the number of bone fractures donors had later in life<br />
#** Limitations included differences in blood donation policy, process, and popularity of varying donation types between Scandanavia and the United States<br />
#* Study 2: Longitudinal study of blood donors<br />
#** Concluded that current guidelines were enough to protect adult male donors between the ages of 20 and 65<br />
#** Limited in that conclusion can not be extrapolated to women or men outside that age range<br />
# Presentation on work done in weeks 3 and 4 to mentors (readings, problems with data)<br />
<br />
===Week 5===<br />
June 29 to July 5<br />
<br />
===Week 6===<br />
July 6 to July 12<br />
<br />
===Week 7===<br />
July 13 to July 19<br />
<br />
===Week 8===<br />
July 20 to July 26<br />
<br />
===Week 9===<br />
July 27 to August 2<br />
<br />
===Week 10===<br />
August 3 to August 9</div>Crepacihttps://reu.cs.mu.edu/index.php/User:CrepaciUser:Crepaci2020-06-29T17:03:16Z<p>Crepaci: /* Week 4 */</p>
<hr />
<div><br />
== About Me ==<br />
<br />
I'm [https://www.linkedin.com/in/charlie-repaci-07b723179/ Charlie Repaci], a senior at [https://www.simmons.edu Simmons University] studying [https://www.simmons.edu/undergraduate/academics/majors-minors/data-science-and-analytics Data Science], with a special interest in Biochemistry and Sociology. This summer I am working with [https://www.shionguha.net/ Dr. Shion Guha] and doctoral candidate [https://www.saxena.io/ Devansh Saxena] on [[Developing Ethical Algorithms for Placement Stability in the Foster Care System]].<br />
<br />
== Work Log ==<br />
<br />
===Week 1===<br />
June 1 to June 7<br />
# Orientation<br />
#* Introduction to other mentors, mentees, and REU heads Dr. Praveen Madiraju and Dr. Dennis Brylow<br />
#* Review of REU calendar and expectations<br />
# Data Science Bootcamp (talk by Dr. Madiraju)<br />
#* Introduction and basics of data analysis with python (Anaconda and Jypyter Notebook; pandas, numpy, matplotlib, seaborn, scipy)<br />
#** Read in data<br />
#** Pre-processing<br />
#** Modeling<br />
#** Data visualization<br />
# Good Research Practices (talk by Dr. Brylow)<br />
# Literature Review<br />
#* [https://dl.acm.org/doi/abs/10.1145/3313831.3376229 A Human-Centered Review of Algorithms used within the U.S. Child Welfare System]<br />
#* [https://dl.acm.org/doi/abs/10.1145/3290605.3300271 Toward Algorithmic Accountability in Public Services: A Qualitative Study of Affected Community Perspectives on Algorithmic Decision-making in Child Welfare Services]<br />
#* [https://dl.acm.org/doi/abs/10.1145/3323994.3369888 Child Welfare System: Interaction of Policy, Practice, and Algorithms]<br />
#* (Supplementary) [https://dl.acm.org/doi/abs/10.1145/3290605.3300497 Risk vs. Restriction: The Tension between Providing a Sense of Normalcy and Keeping Foster Teens Safe Online]<br />
<br />
===Week 2===<br />
June 8 to June 14<br />
# Responsible Conduct of Research Training (talk by Dr. Brylow)<br />
#* Ethical treatment of data<br />
#* Authorship, credit, plagiarism<br />
#* Human participants<br />
#* Intellectual property<br />
#* Conflicts of interest and professional standards<br />
# Worked on getting Citi certification for all three RCR sessions<br />
# Technical Writing Workshop (talk by Dr. Brylow and Dr. Madiraju)<br />
#* What the sections of a technical paper are<br />
#* What the publication process is like<br />
#* General tips<br />
# Literature review<br />
#* [http://dx.doi.org/10.2139/ssrn.2245322 Governing Algorithms: A Provocation Piece]<br />
#* [https://academiccommons.columbia.edu/doi/10.7916/D8ZK5TW2 Algorithmic Accountability Reporting: On the Investigation of Black Boxes]<br />
#* [http://www.tandfonline.com/doi/full/10.1080/1369118X.2016.1154087#abstract Thinking Critically About and Researching Algorithms]<br />
#* (Supplementary) [http://culturedigitally.org/wp-content/uploads/2016/07/Gillespie-2016-Algorithm-Digital-Keywords-Peters-ed.pdf Algorithm in Digital Keywords: a Vocabulary of Information, Society, and Culture]<br />
# Meeting with mentors<br />
#* Github created for the project<br />
#* Questions and discussion of the literature reviewed<br />
#* Planned work for the next two weeks<br />
<br />
===Week 3===<br />
June 15 to June 21<br />
# Set up Wiki entrees for myself and my project<br />
# Meeting with all REU interns to discuss our projects so far and any problems we have run into<br />
# Literature review<br />
#* [https://dl.acm.org/doi/abs/10.1145/3290605.3300760 Street-Level Algorithms: A Theory at the Gaps Between Policy and Decisions]<br />
#* [https://journals.sagepub.com/doi/abs/10.1177/2053951717738104 Algorithms as culture: Some tactics for the ethnography of algorithmic systems]<br />
#* (Supplementary) [https://journals.sagepub.com/doi/full/10.1177/2053951717751552 Algorithms as fetish: Faith and possibility in algorithmic work]<br />
# Meeting with Devansh to discuss the first dataset<br />
#* Understand variables and components -- reading the documentation that comes with datasets<br />
#* Import dataset into R and common problems<br />
<br />
===Week 4===<br />
June 22 to June 28<br />
<br />
# Importing data<br />
#* Need SAS to run a program included with the data files that creates .sas7bdat and .sas7bcat files from the .dat files that were given to us<br />
#* Some are not importing correctly for yet unknown reasons<br />
# Student Check-In (by Dr. Brylow and Dr. Madiraju)<br />
# Research Presentation (talk by guest lecturer Dr. Walter Bialkowski)<br />
#* Blood donation and potential risk of lower bone density to donors due to prolonged and repeated exposure to the anticoagulant citrate added to the blood (and then returned to the donor) during the donation process<br />
#* Study 1: Data analysis of Scandinavian blood donor data<br />
#** Concluded that there was no association between blood donation and the number of bone fractures donors had later in life<br />
#** Limitations included differences in blood donation policy, process, and popularity of varying donation types between Scandanavia and the United States<br />
#* Study 2: Longitudinal study of blood donors<br />
#** Concluded that current guidelines were enough to protect adult male donors between the ages of 20 and 65<br />
#** Limited in that conclusion can not be extrapolated to women or men outside that age range<br />
#* Presentation on work done in weeks 3 and 4 to mentors (readings, problems with data)<br />
<br />
===Week 5===<br />
June 29 to July 5<br />
<br />
===Week 6===<br />
July 6 to July 12<br />
<br />
===Week 7===<br />
July 13 to July 19<br />
<br />
===Week 8===<br />
July 20 to July 26<br />
<br />
===Week 9===<br />
July 27 to August 2<br />
<br />
===Week 10===<br />
August 3 to August 9</div>Crepacihttps://reu.cs.mu.edu/index.php/User:CrepaciUser:Crepaci2020-06-29T17:02:31Z<p>Crepaci: /* Week 4 */</p>
<hr />
<div><br />
== About Me ==<br />
<br />
I'm [https://www.linkedin.com/in/charlie-repaci-07b723179/ Charlie Repaci], a senior at [https://www.simmons.edu Simmons University] studying [https://www.simmons.edu/undergraduate/academics/majors-minors/data-science-and-analytics Data Science], with a special interest in Biochemistry and Sociology. This summer I am working with [https://www.shionguha.net/ Dr. Shion Guha] and doctoral candidate [https://www.saxena.io/ Devansh Saxena] on [[Developing Ethical Algorithms for Placement Stability in the Foster Care System]].<br />
<br />
== Work Log ==<br />
<br />
===Week 1===<br />
June 1 to June 7<br />
# Orientation<br />
#* Introduction to other mentors, mentees, and REU heads Dr. Praveen Madiraju and Dr. Dennis Brylow<br />
#* Review of REU calendar and expectations<br />
# Data Science Bootcamp (talk by Dr. Madiraju)<br />
#* Introduction and basics of data analysis with python (Anaconda and Jypyter Notebook; pandas, numpy, matplotlib, seaborn, scipy)<br />
#** Read in data<br />
#** Pre-processing<br />
#** Modeling<br />
#** Data visualization<br />
# Good Research Practices (talk by Dr. Brylow)<br />
# Literature Review<br />
#* [https://dl.acm.org/doi/abs/10.1145/3313831.3376229 A Human-Centered Review of Algorithms used within the U.S. Child Welfare System]<br />
#* [https://dl.acm.org/doi/abs/10.1145/3290605.3300271 Toward Algorithmic Accountability in Public Services: A Qualitative Study of Affected Community Perspectives on Algorithmic Decision-making in Child Welfare Services]<br />
#* [https://dl.acm.org/doi/abs/10.1145/3323994.3369888 Child Welfare System: Interaction of Policy, Practice, and Algorithms]<br />
#* (Supplementary) [https://dl.acm.org/doi/abs/10.1145/3290605.3300497 Risk vs. Restriction: The Tension between Providing a Sense of Normalcy and Keeping Foster Teens Safe Online]<br />
<br />
===Week 2===<br />
June 8 to June 14<br />
# Responsible Conduct of Research Training (talk by Dr. Brylow)<br />
#* Ethical treatment of data<br />
#* Authorship, credit, plagiarism<br />
#* Human participants<br />
#* Intellectual property<br />
#* Conflicts of interest and professional standards<br />
# Worked on getting Citi certification for all three RCR sessions<br />
# Technical Writing Workshop (talk by Dr. Brylow and Dr. Madiraju)<br />
#* What the sections of a technical paper are<br />
#* What the publication process is like<br />
#* General tips<br />
# Literature review<br />
#* [http://dx.doi.org/10.2139/ssrn.2245322 Governing Algorithms: A Provocation Piece]<br />
#* [https://academiccommons.columbia.edu/doi/10.7916/D8ZK5TW2 Algorithmic Accountability Reporting: On the Investigation of Black Boxes]<br />
#* [http://www.tandfonline.com/doi/full/10.1080/1369118X.2016.1154087#abstract Thinking Critically About and Researching Algorithms]<br />
#* (Supplementary) [http://culturedigitally.org/wp-content/uploads/2016/07/Gillespie-2016-Algorithm-Digital-Keywords-Peters-ed.pdf Algorithm in Digital Keywords: a Vocabulary of Information, Society, and Culture]<br />
# Meeting with mentors<br />
#* Github created for the project<br />
#* Questions and discussion of the literature reviewed<br />
#* Planned work for the next two weeks<br />
<br />
===Week 3===<br />
June 15 to June 21<br />
# Set up Wiki entrees for myself and my project<br />
# Meeting with all REU interns to discuss our projects so far and any problems we have run into<br />
# Literature review<br />
#* [https://dl.acm.org/doi/abs/10.1145/3290605.3300760 Street-Level Algorithms: A Theory at the Gaps Between Policy and Decisions]<br />
#* [https://journals.sagepub.com/doi/abs/10.1177/2053951717738104 Algorithms as culture: Some tactics for the ethnography of algorithmic systems]<br />
#* (Supplementary) [https://journals.sagepub.com/doi/full/10.1177/2053951717751552 Algorithms as fetish: Faith and possibility in algorithmic work]<br />
# Meeting with Devansh to discuss the first dataset<br />
#* Understand variables and components -- reading the documentation that comes with datasets<br />
#* Import dataset into R and common problems<br />
<br />
===Week 4===<br />
June 22 to June 28<br />
<br />
# Importing data<br />
#* Need SAS to run a program included with the data files that creates .sas7bdat and .sas7bcat files from the .dat files that were given to us<br />
#* Some are not importing correctly for yet unknown reasons<br />
# Student Check-In (by Dr. Brylow and Dr. Madiraju)<br />
# Research Presentation (talk by guest lecturer Dr. Walter Bialkowski)<br />
#* Blood donation and potential risk of lower bone density to donors due to prolonged and repeated exposure to the anticoagulant citrate added to the blood (and then returned to the donor) during the donation process<br />
#* 6 years of research and 2 studies conducted<br />
#** Data analysis of Scandinavian blood donor data<br />
#*** Concluded that there was no association between blood donation and the number of bone fractures donors had later in life<br />
#*** Limitations included differences in blood donation policy, process, and popularity of varying donation types between Scandanavia and the United States<br />
#** Longitudinal study of blood donors<br />
#*** Concluded that current guidelines were enough to protect adult male donors between the ages of 20 and 65<br />
#*** Limited in that conclusion can not be extrapolated to women or men outside that age range<br />
#* Presentation on work done in weeks 3 and 4 to mentors (readings, problems with data)<br />
<br />
===Week 5===<br />
June 29 to July 5<br />
<br />
===Week 6===<br />
July 6 to July 12<br />
<br />
===Week 7===<br />
July 13 to July 19<br />
<br />
===Week 8===<br />
July 20 to July 26<br />
<br />
===Week 9===<br />
July 27 to August 2<br />
<br />
===Week 10===<br />
August 3 to August 9</div>Crepacihttps://reu.cs.mu.edu/index.php/Developing_Ethical_Algorithms_for_Placement_Stability_in_the_Foster_Care_SystemDeveloping Ethical Algorithms for Placement Stability in the Foster Care System2020-06-23T22:18:51Z<p>Crepaci: /* Schedule: */</p>
<hr />
<div>Student: [[User:Crepaci|Charlie Repaci]]<br />
<br />
Mentor: [https://www.shionguha.net/ Dr. Shion Guha] and [https://saxena.io/ Devansh Saxena]<br />
<br />
==Description:==<br />
The goal of this project is to use a human-centered approach grounded in current social science theory and frameworks to add context to and further develop existing placement stability and risk assessment models that are used to aid overworked social workers in making, explaining, and standardizing their decisions.<br />
<br />
==Goals:==<br />
Quoted from the [https://reu.cs.mu.edu/index.php/Ethical_Algorithms_for_Placement_Stability_in_the_Foster_Care project page:]<br />
<br />
<blockquote>''This project aims to collaborate with the WI Department of Children and Families and SaintA (a private non-profit organization that actually provides foster care services in WI) by utilizing important, useful and contextual caseworkers judgment that are recorded as detailed case notes but never actually used to add norms, values, and context to existing algorithms that determine placement stability. Topic modeling will be used to extract latent themes from such text and incrementally added to existing placement stability models to test improvements in outcomes.''<br />
* ''Perform a literature review of topic modeling usage in the social science domain.''<br />
* ''Understand latent and human context from caseworker notes in Wisconsin foster care system using the data provided from the mentor.''<br />
* ''Review algorithmic biases, fairness, and transparency in the foster care system.''<br />
* ''Evaluate the latent themes from text and incrementally add to existing placement stability models to test improvement in outcomes.''</blockquote><br />
<br />
==Schedule:==<br />
{| class="wikitable"<br />
|-<br />
|'''Week'''<br />
|'''Description'''<br />
|-<br />
|Week 1<br />
|<br />
* Orientation<br />
* Data Science Bootcamp<br />
* Begin literature review -- algorithms in social work<br />
|-<br />
|Week 2<br />
|<br />
* Continue literature review -- ethics and algorithms<br />
* Ethics training and CITI certification<br />
* Technical writing workshop<br />
|-<br />
|Week 3<br />
|<br />
* Continue literature review -- bias in algorithms and policy<br />
* Begin reviewing data<br />
|-<br />
|Week 4<br />
|<br />
* Exploratory data visualization<br />
* Develop more specific questions and a plan of analysis<br />
|-<br />
|Week 5<br />
|<br />
* Data analysis and model development<br />
* Technical presentations workshop and informal presentations<br />
* TBD<br />
|-<br />
|Week 6<br />
|<br />
* Data analysis and model development<br />
* Data ethics special lecture<br />
* Research posters workshop<br />
* TBD<br />
|-<br />
|Week 7<br />
|<br />
* Data analysis and model development<br />
* TBD<br />
|-<br />
|Week 8<br />
|<br />
* Data analysis and model development<br />
* TBD<br />
|-<br />
|Week 9<br />
|<br />
* Industry guest panel (Northwestern Mutual Data Science Institute)<br />
* Begin the final paper<br />
* Start to create project poster<br />
* Consider future work<br />
|-<br />
|Week 10<br />
|<br />
* Prepare and give the oral presentation<br />
* TBD: Present project to other REU sites and see their research in return<br />
* Finish and submit the final paper<br />
|}</div>Crepacihttps://reu.cs.mu.edu/index.php/User:CrepaciUser:Crepaci2020-06-23T22:13:39Z<p>Crepaci: /* Work Log */</p>
<hr />
<div><br />
== About Me ==<br />
<br />
I'm [https://www.linkedin.com/in/charlie-repaci-07b723179/ Charlie Repaci], a senior at [https://www.simmons.edu Simmons University] studying [https://www.simmons.edu/undergraduate/academics/majors-minors/data-science-and-analytics Data Science], with a special interest in Biochemistry and Sociology. This summer I am working with [https://www.shionguha.net/ Dr. Shion Guha] and doctoral candidate [https://www.saxena.io/ Devansh Saxena] on [[Developing Ethical Algorithms for Placement Stability in the Foster Care System]].<br />
<br />
== Work Log ==<br />
<br />
===Week 1===<br />
June 1 to June 7<br />
# Orientation<br />
#* Introduction to other mentors, mentees, and REU heads Dr. Praveen Madiraju and Dr. Dennis Brylow<br />
#* Review of REU calendar and expectations<br />
# Data Science Bootcamp (talk by Dr. Madiraju)<br />
#* Introduction and basics of data analysis with python (Anaconda and Jypyter Notebook; pandas, numpy, matplotlib, seaborn, scipy)<br />
#** Read in data<br />
#** Pre-processing<br />
#** Modeling<br />
#** Data visualization<br />
# Good Research Practices (talk by Dr. Brylow)<br />
# Literature Review<br />
#* [https://dl.acm.org/doi/abs/10.1145/3313831.3376229 A Human-Centered Review of Algorithms used within the U.S. Child Welfare System]<br />
#* [https://dl.acm.org/doi/abs/10.1145/3290605.3300271 Toward Algorithmic Accountability in Public Services: A Qualitative Study of Affected Community Perspectives on Algorithmic Decision-making in Child Welfare Services]<br />
#* [https://dl.acm.org/doi/abs/10.1145/3323994.3369888 Child Welfare System: Interaction of Policy, Practice, and Algorithms]<br />
#* (Supplementary) [https://dl.acm.org/doi/abs/10.1145/3290605.3300497 Risk vs. Restriction: The Tension between Providing a Sense of Normalcy and Keeping Foster Teens Safe Online]<br />
<br />
===Week 2===<br />
June 8 to June 14<br />
# Responsible Conduct of Research Training (talk by Dr. Brylow)<br />
#* Ethical treatment of data<br />
#* Authorship, credit, plagiarism<br />
#* Human participants<br />
#* Intellectual property<br />
#* Conflicts of interest and professional standards<br />
# Worked on getting Citi certification for all three RCR sessions<br />
# Technical Writing Workshop (talk by Dr. Brylow and Dr. Madiraju)<br />
#* What the sections of a technical paper are<br />
#* What the publication process is like<br />
#* General tips<br />
# Literature review<br />
#* [http://dx.doi.org/10.2139/ssrn.2245322 Governing Algorithms: A Provocation Piece]<br />
#* [https://academiccommons.columbia.edu/doi/10.7916/D8ZK5TW2 Algorithmic Accountability Reporting: On the Investigation of Black Boxes]<br />
#* [http://www.tandfonline.com/doi/full/10.1080/1369118X.2016.1154087#abstract Thinking Critically About and Researching Algorithms]<br />
#* (Supplementary) [http://culturedigitally.org/wp-content/uploads/2016/07/Gillespie-2016-Algorithm-Digital-Keywords-Peters-ed.pdf Algorithm in Digital Keywords: a Vocabulary of Information, Society, and Culture]<br />
# Meeting with mentors<br />
#* Github created for the project<br />
#* Questions and discussion of the literature reviewed<br />
#* Planned work for the next two weeks<br />
<br />
===Week 3===<br />
June 15 to June 21<br />
# Set up Wiki entrees for myself and my project<br />
# Meeting with all REU interns to discuss our projects so far and any problems we have run into<br />
# Literature review<br />
#* [https://dl.acm.org/doi/abs/10.1145/3290605.3300760 Street-Level Algorithms: A Theory at the Gaps Between Policy and Decisions]<br />
#* [https://journals.sagepub.com/doi/abs/10.1177/2053951717738104 Algorithms as culture: Some tactics for the ethnography of algorithmic systems]<br />
#* (Supplementary) [https://journals.sagepub.com/doi/full/10.1177/2053951717751552 Algorithms as fetish: Faith and possibility in algorithmic work]<br />
# Meeting with Devansh to discuss the first dataset<br />
#* Understand variables and components -- reading the documentation that comes with datasets<br />
#* Import dataset into R and common problems<br />
<br />
===Week 4===<br />
June 22 to June 28<br />
<br />
===Week 5===<br />
June 29 to July 5<br />
<br />
===Week 6===<br />
July 6 to July 12<br />
<br />
===Week 7===<br />
July 13 to July 19<br />
<br />
===Week 8===<br />
July 20 to July 26<br />
<br />
===Week 9===<br />
July 27 to August 2<br />
<br />
===Week 10===<br />
August 3 to August 9</div>Crepacihttps://reu.cs.mu.edu/index.php/Developing_Ethical_Algorithms_for_Placement_Stability_in_the_Foster_Care_SystemDeveloping Ethical Algorithms for Placement Stability in the Foster Care System2020-06-15T22:10:29Z<p>Crepaci: /* Goals: */</p>
<hr />
<div>Student: [[User:Crepaci|Charlie Repaci]]<br />
<br />
Mentor: [https://www.shionguha.net/ Dr. Shion Guha] and [https://saxena.io/ Devansh Saxena]<br />
<br />
==Description:==<br />
The goal of this project is to use a human-centered approach grounded in current social science theory and frameworks to add context to and further develop existing placement stability and risk assessment models that are used to aid overworked social workers in making, explaining, and standardizing their decisions.<br />
<br />
==Goals:==<br />
Quoted from the [https://reu.cs.mu.edu/index.php/Ethical_Algorithms_for_Placement_Stability_in_the_Foster_Care project page:]<br />
<br />
<blockquote>''This project aims to collaborate with the WI Department of Children and Families and SaintA (a private non-profit organization that actually provides foster care services in WI) by utilizing important, useful and contextual caseworkers judgment that are recorded as detailed case notes but never actually used to add norms, values, and context to existing algorithms that determine placement stability. Topic modeling will be used to extract latent themes from such text and incrementally added to existing placement stability models to test improvements in outcomes.''<br />
* ''Perform a literature review of topic modeling usage in the social science domain.''<br />
* ''Understand latent and human context from caseworker notes in Wisconsin foster care system using the data provided from the mentor.''<br />
* ''Review algorithmic biases, fairness, and transparency in the foster care system.''<br />
* ''Evaluate the latent themes from text and incrementally add to existing placement stability models to test improvement in outcomes.''</blockquote><br />
<br />
==Schedule:==<br />
{| class="wikitable"<br />
|-<br />
|'''Week'''<br />
|'''Description'''<br />
|-<br />
|Week 1<br />
|<br />
* Orientation<br />
* Data Science Bootcamp<br />
* Begin literature review -- algorithms in social work<br />
|-<br />
|Week 2<br />
|<br />
* Continue literature review -- ethics and algorithms<br />
* Ethics training and CITI certification<br />
* Technical writing workshop<br />
|-<br />
|Week 3<br />
|<br />
* Continue literature review -- bias in algorithms and policy<br />
* Begin reviewing data<br />
* Develop more specific questions and a plan of analysis<br />
|-<br />
|Week 4<br />
|<br />
* Exploratory data visualization<br />
* TBD<br />
|-<br />
|Week 5<br />
|<br />
* Data analysis and model development<br />
* Technical presentations workshop and informal presentations<br />
* TBD<br />
|-<br />
|Week 6<br />
|<br />
* Data analysis and model development<br />
* Data ethics special lecture<br />
* Research posters workshop<br />
* TBD<br />
|-<br />
|Week 7<br />
|<br />
* Data analysis and model development<br />
* TBD<br />
|-<br />
|Week 8<br />
|<br />
* Data analysis and model development<br />
* TBD<br />
|-<br />
|Week 9<br />
|<br />
* Industry guest panel (Northwestern Mutual Data Science Institute)<br />
* Begin the final paper<br />
* Start to create project poster<br />
* Consider future work<br />
|-<br />
|Week 10<br />
|<br />
* Prepare and give the oral presentation<br />
* TBD: Present project to other REU sites and see their research in return<br />
* Finish and submit the final paper<br />
|}</div>Crepaci