AI tool has real implications for families in the child welfare system

For family law attorney Robin Frank, defending parents at one of their lowest points — when they’re at risk of losing their kids — has never been easy.

The job is never easy, but in the past she knew what she was up against when she took on child protection in family court. Now she’s worried she’s fighting something she can’t see: an opaque algorithm whose statistical calculations help social workers decide which families should be examined in the first place.

“A lot of people don’t know it’s even used,” Frank said. “Families should have the right to have all the information in their records.”

From all over Oregon to Los Angeles to Colorado, as child welfare agencies use or consider tools similar to those in Allegheny County, Pennsylvania, an Associated Press review identified a number of concerns about the technology, including questions about its reliability and the potential to reinforce racial inequalities in the child welfare system. Related issues have already torpedoed some jurisdictions’ plans to use predictive modeling, such as the tool dropped notably by the state of Illinois.

Case work supervisor Jessie Schemm looks at the first screen of software used by employees making phone calls at an intake interview screening center for the Allegheny County Children and Youth Services, in Penn Hills, Pennsylvania.  algorithmic tool uses data to support agency workers in protecting children from neglect.  The nuanced term can include everything from inadequate housing to poor hygiene.

According to new research from a Carnegie Mellon University team obtained exclusively by AP, in its early years of use, Allegheny’s algorithm showed a pattern of flagging a disproportionate number of black children for a “mandatory” study of neglect, compared to white children. The independent researchers, who received data from the county, also found that social workers disagreed with the risk scores the algorithm produced about a third of the time.

Leave a Comment