Amazon boosted poison to suicidal users in recommendations

[Supersonic 49,376]

 

[AURAOFSHEISSE 6,517]

this is sick, and the fact that those purchases were just adding to Jeff's billions

[TimisaMonster 31,073]

This is disturbing af...

[gumzy3000 7,953]

Sounds like something a real life villain would do 

[HEARTSTOP 3,160]

It is hardly Amazon's fault that people buy items for different purposes than those intended. If I buy a knife and kill someone with it, Amazon has nothing to do with it. The suggested items are based on what other people buy, it is not a big mastermind plan to push products that can be used together to kill people.

However, there's a HUGE problem when they purposefully remove reviews warning people of the danger. This makes me think so many other products out there only have reviews that they deem beneficial to their business. They should have investigated the situation and applied the appropriate solutions, instead of hiding it until the media got hold of it. So...let them ROT

[TheSlash 14,539]

I can hardly see this to be Amazon's fault in this particular issue. The recommendations are based on an algorithm which just analyzes other users' purchase behavior and what they look up after seeing that particular product.

I agree that Amazon's practices are questionable to say the least, but they don't actively promote these things.

That thing with removed reviews is pure evil, tho.

[Supersonic 49,376]

9 hours ago, HEARTSTOP said:

It is hardly Amazon's fault that people buy items for different purposes than those intended. If I buy a knife and kill someone with it, Amazon has nothing to do with it. The suggested items are based on what other people buy, it is not a big mastermind plan to push products that can be used together to kill people.

However, there's a HUGE problem when they purposefully remove reviews warning people of the danger. This makes me think so many other products out there only have reviews that they deem beneficial to their business. They should have investigated the situation and applied the appropriate solutions, instead of hiding it until the media got hold of it. So...let them ROT

 

8 hours ago, TheSlash said:

I can hardly see this to be Amazon's fault in this particular issue. The recommendations are based on an algorithm which just analyzes other users' purchase behavior and what they look up after seeing that particular product.

I agree that Amazon's practices are questionable to say the least, but they don't actively promote these things.

That thing with removed reviews is pure evil, tho.

...it is Amazon's choice to employ algorithm's, they are aware of its shortcomings (like any other social media platform is aware how their algorithms spiral misinformation, suicidal ideation, depression etc. out of control). We have enough proof of the higher ups on Facebook, Instagram, YouTube & TikTok knowing that their algorithms are causing tangible harm to society, but are willing to accept that because it farms revenue and engagement. Amazon is no stranger to this. 

There is no benefit of the doubt left to give, we're in 2022 and aware of how these algorithms harm real life people. If these companies don't change them, knowing that they are causing harm, they actively WANT to cause harm. 

[Sepsami 21,074]

is it really Amazon's fault? I know we love to hate billion dollar companies and rightfully so most of the time, but isn't this just what those algorithms are made for?

[Dojo 19,823]

it's what AI do, in the end is just a bunch of machines making decisions based on consumer tendencies, unless actual humans were involved.

[DelusionalGaga 29,734]

I'm always shocked at how many people lack sympathy for other humans and will rather continue to defend global corporations, who's only goal is to make more money. You can't blame everything on the algorithm as if it's just a computer making "beep-boop", when these companies have whole departments dedicated to machine learning, who absolutely could restrict and change the algorithm.

To anyone interested, heres a link to an article talking about the testimony of the Facebook whistleblower Frances Haugen, who testified about how the Facebook/Instagram algorithm exploits mental health issues to create more user engagement on their platforms:

https://www.technologyreview.com/2021/10/05/1036519/facebook-whistleblower-frances-haugen-algorithms/

some excerpts: 

How does Facebook’s content ranking relate to teen mental health?

One of the more shocking revelations from the Journal’s Facebook Files was Instagram’s internal research, which found that its platform is worsening mental health among teenage girls. “Thirty-two percent of teen girls said that when they felt bad about their bodies, Instagram made them feel worse,” researchers wrote in a slide presentation from March 2020.

Haugen connects this phenomenon to engagement-based ranking systems as well, which she told the Senate today “is causing teenagers to be exposed to more anorexia content.”

“If Instagram is such a positive force, have we seen a golden age of teenage mental health in the last 10 years? No, we have seen escalating rates of suicide and depression amongst teenagers,” she continued. “There’s a broad swath of research that supports the idea that the usage of social media amplifies the risk of these mental health harms.”

In my own reporting, I heard from a former AI researcher who also saw this effect extend to Facebook.

The researcher’s team…found that users with a tendency to post or engage with melancholy content—a possible sign of depression—could easily spiral into consuming increasingly negative material that risked further worsening their mental health.

But as with Haugen, the researcher found that leadership wasn’t interested in making fundamental algorithmic changes.

The team proposed tweaking the content-ranking models for these users to stop maximizing engagement alone, so they would be shown less of the depressing stuff. “The question for leadership was: Should we be optimizing for engagement if you find that somebody is in a vulnerable state of mind?” he remembers.

But anything that reduced engagement, even for reasons such as not exacerbating someone’s depression, led to a lot of hemming and hawing among leadership. With their performance reviews and salaries tied to the successful completion of projects, employees quickly learned to drop those that received pushback and continue working on those dictated from the top down….

That former employee, meanwhile, no longer lets his daughter use Facebook.

[CooterFritter 777]

I definitely don’t think this is a hardcore blame Amazon thing…this is a learning opportunity for Amazon and AI to improve their algorithm so they aren’t recommending harmful products to people.

They could also use the AI to counteract that and recommend help/resources to people searching things like this. 

[HEARTSTOP 3,160]

7 hours ago, CooterFritter said:

I definitely don’t think this is a hardcore blame Amazon thing…this is a learning opportunity for Amazon and AI to improve their algorithm so they aren’t recommending harmful products to people.

They could also use the AI to counteract that and recommend help/resources to people searching things like this. 

Exactly.

9 hours ago, Supersonic said:

 

...it is Amazon's choice to employ algorithm's, they are aware of its shortcomings (like any other social media platform is aware how their algorithms spiral misinformation, suicidal ideation, depression etc. out of control). We have enough proof of the higher ups on Facebook, Instagram, YouTube & TikTok knowing that their algorithms are causing tangible harm to society, but are willing to accept that because it farms revenue and engagement. Amazon is no stranger to this. 

There is no benefit of the doubt left to give, we're in 2022 and aware of how these algorithms harm real life people. If these companies don't change them, knowing that they are causing harm, they actively WANT to cause harm. 

Using an algorithm based on user behavior just as a concept is not what should be at stake here. There's millions of examples these AI algorithms have helped society, as long as their users are responsible enough to apply exceptions when needed (even with AI, computers still can't predict and implement these exceptions on their own, since they're rooted in such complex human behaviors that are not understandable by the AI - for example, a machine would never consider harming itself because it's very unlikely someone would program that behavior).

Any and all algorithms CAN and WILL be abused by both the company and the users. What Amazon did wrong was purposefully hiding and ignoring concerns presented in the reviews, and therefore should be punished for that. They should also implement some kind of "exceptions" to the code to make it so these concerns can be diminished.