okay so, here's the thing. neural networks are cute and all but if you ever build a cyclical system that goes (collect data) -> (ML algorithm) -> (take actions that influence the original data pool) then what you are building is a very dangerous feedback loop

"I aggregated a bunch of public domain fiction to make a poetry generator": oh neat! sounds fun
"I collect data of which posts people like on my platform and then use that to inform an AI algorithm that serves posts users might like": very very bad oh god

Show thread

"I made a thing that turns your photo into an anime girl": lol cute
"We use a network trained on past successul candidates' resumes to do prefiltering on future applicants": what are you DOING

Show thread
@clarity that unfortunately does not become self aware, either
@clarity most people think the trees were a bad idea. imo consciousness was a mistake and we should have never have gone multicellular either, this is the hill I will die on imo.
@clarity oh I was making a joke because consciousness is also a feedback loop of sorts :)
@clarity I have listened to too much douglas adams so I stole the format

@alexandria oh, yeah. it is!! but it's not a positive feedback loop, it's a complicated one with lots of different things in place to keep it from (usually) just amplifying in a single direction forever

@clarity I stand by the idea that cognitive biases are, in fact, essential for consciousness to function in the way that it does

@clarity that's the default though for data driven decision making, is it not? Not dangerous per se, but seems to me dangerous if unaware (which also seems a default)

@ton @clarity specifically calling it "the output of an algorithm" rather than "humans making a best guess using data and heuristics" is laundering bias and deferring accountability, is the point maybe.

@jplebreton @ton if you have a "humans monitor and interpret algo output" step in between "ML algorithm" and "take actions" that's a very different thing, because now your feedback loop is regulated instead of running autonomously

I am also skeptical of "data-driven" anything but that's an unrelated topic

Sign in to participate in the conversation

The social network of the future: No ads, no corporate surveillance, ethical design, and decentralization! Own your data with Mastodon!