Jackson soon discovered that Amazon suspended his account because a Black delivery driver who’d come to his house the previous day had reported hearing racist remarks from his video doorbell. In a brief email sent to Jackson at 3 a.m., the company explained how it unilaterally placed all of his linked devices and services on hold as it commenced an internal investigation.

The accusations baffled Jackson. He and his family are Black. When he reviewed the doorbell’s footage, he saw that nobody was home at the time of the delivery. At a loss for what could have prompted the accusation of racism, he suspected the driver had misinterpreted the doorbell’s automated response: “Excuse me, can I help you?”

  • tweeks
    link
    fedilink
    English
    arrow-up
    3
    ·
    1 year ago

    In this case it’s the “suspect” of racism, in which I think we all agree should not directly lock you out of your account, but perhaps give you a warning.

    But what if it’s suspected illegal actions or content? Like them catching the home server being part of a DDoS attack, or overhearing signs of child abuse / identifying possible child pornography content or noticing illicit gun ownership. Their AI will determine that there’s a 97% chance of that being the case.

    I wonder if that would change things.

    If such a system is not allowed to block usage, it will probably at least inform the local police.Your home setup will in most cases eventually act as a panopticon.