The department told Bloomberg that the investigation will both look into any legal wrongdoing as well guarantee that every customer is treated equally. Any algorithmic bias (including unintentional bias) “violates New York law,” department superintendent Linda Lacewell said.
Goldman maintained in a statement that credit decisions were based solely on “creditworthiness” and not qualities like gender or ethnicity, although it didn’t explain why a woman with a stronger credit score received a much lower limit.
It’s not certain how long the investigation will take, and there’s no guarantee Goldman will be asked to make adjustments. However, the incident underscores concerns that bias in algorithms is creating serious disadvantages for some groups, such as denying adequate medical coverage. It also clouds an otherwise strong debut for Apple Card — both Apple and Goldman have claimed this was the “most successful launch” of a credit card in US history. Whether or not that’s true, Hansson’s incident suggests there may still be flaws to address.
The @AppleCard is such a fucking sexist program. My wife and I filed joint tax returns, live in a community-property state, and have been married for a long time. Yet Apple’s black box algorithm thinks I deserve 20x the credit limit she does. No appeals work.
— DHH (@dhh) November 7, 2019
social experiment by Livio Acerbo #greengroundit #engadget https://www.engadget.com/2019/11/09/new-york-investigates-apple-card-credit-limit-sexism/