Advocates: take survivors of abuse seriously.
Society: Let’s have computers tell us what to do!I mean I guess the risk of repeated murder-suicide is pretty low…
In the late 1970s (I was a kid) the computer is always right was a common sarcastic parody of all the people who actually believed it.
We’d discoverin the 1980s it was possible to have missing data, insufficient data or erroneous data.
It’s a sentiment at least as old as the first things that we now call computers.
On two occasions I have been asked, “Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?” … I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question.
—Charles Babbage
As if humans can magically make correct decisions with incorrect information lmao. So true.
The algorithm:
isSafe = random();
if isSafe >.5 println (“everything is fine\n”);
per the article, it’s rather better than that.
I admit to having grossly oversimplified things. Sorry.
I thought that would have been obvious to everyone already.
I remember years ago when they said the value of our lives would be determined by a panel of people.
Now its by a machine.
Oh, it’s far worse than that… the value of our lives have been determined by the (so-called) “free market” for a very long time now.
The machine is simply going to streamline the process.
The algorithm itself is just a big “whatever”. The key issue here is that some assumptive piece of shit decided to conclude, based on partial information, that those women would be safe in the future.
Why do we live in a dystopian hellscape
People are complicated creatures that can’t be easily fit into niche categories despite our brains need to do so.
Culture.
Capitalism.
The way to use these kinds of systems is to have the judge came to an independent decision, then, after that’s keyed in, the AI spits out theirs and whichever predicts more danger is then acted on.
Relatedly, the way you have an AI select people and companies to get spot-checked by tax investigators is not to show investigators the AI scores, but mix in AI suspicions among a stream of randomly selected people.
Relatedly, the way you have AI involved in medical diagnoses is not to tell the human doctor results, but suggest additional tests to be made. The “have you ruled out lupus” approach.
And from what I’ve heard the medical profession actually got that right from the very beginning. They know what priming and bias is. Law enforcement? I fear we’ll have to ELI5 them the basics for the next five hundred years.
But that doesn’t save money and the only reason the capitalists want AI is saving money
I don’t think there’s any AI involved. The article mentions nothing of the sort, it’s at least 8 years old (according to the article) and the input is 35 yes/no questions, so it’s probably just some points assigned for the answers and maybe some simple arithmetic.
Sounds like an expert system then (just judging by the age) which was AI before the whole machine learning craze, in any case you need to take the same kind of care when integrating them into whatever real-world structures there are.
Medicine used them with quite some success problem being they take a long time to develop because humans need to input expert knowledge, and then they get outdated quite quickly.
Back to the system though: 35 questions is not enough for these kinds of questions. And that’s not an issue of number of questions, but things like body language and tone of voice not being included.
so it’s probably just some points assigned for the answers and maybe some simple arithmetic.
Why yes, that’s all that machine learning is, a bunch of statistics :)
so it’s probably just some points assigned for the answers and maybe some simple arithmetic.Why yes, that’s all that machine learning is, a bunch of statistics :)
I know, but that’s not what I meant. I mean literally something as simple and mundane as assigning points per answer and evaluating the final score:
// Pseudo code risk = 0 if (Q1 == true) { risk += 20 } if (Q2 == true) { risk += 10 } // etc... // Maybe throw in a bit of if (Q28 == true) { if (Q22 == true and Q23 == true) { risk *= 1.5 } else { risk += 10 } } // And finally, evaluate the risk: if (risk < 10) { return "negligible" } else if (risk >= 10 and risk < 40) { return "low risk" } // etc... You get the picture.And yes, I know I can just write
if (Q1) {, but I wanted to make it a bit more accessible for non-programmers.The article gives absolutely no reason for us to assume it’s anything more than that, and I apparently missed the part of the article that mentioned that the system had been in use since 2007. I know we had machine learning too back then, but looking at the project description here: https://eucpn.org/sites/default/files/document/files/Buena practica VIOGEN_0.pdf it looks more like they looked at a bunch of cases (2159) and came up with the 35 questions and a scoring system not unlike what I just described above.
Edit: I managed to find this, which has apparently been taken down since (but thanks to archive.org it’s still available): https://web.archive.org/web/20240227072357/https://eticasfoundation.org/gender/the-external-audit-of-the-viogen-system/
VioGén’s algorithm uses classical statistical models to perform a risk evaluation based on the weighted sum of all the responses according to pre-set weights for each variable. It is designed as a recommendation system but, even though the police officers are able to increase the automatically assigned risk score, they maintain it in 95% of the cases.
… which incidentally matches what the article says (that police maintain the VioGen risk score in 95% of the cases).
It looks like the minority report future didn’t consider poor coding or AI “hallucinations”
Despite this article, I’m still not convinced that the algorithms aren’t better. The policy states that people need to use their best judgement and can override the algorithms. The article argues that the algorithms are being over relied on. The article mentions in passing, however, that the statistics were worse before the algorithm was introduced.
The point of the matter is, best judgement can be shitty. Your average cop has no idea what questions to ask without a list and how important they are per research. Some suggestions are too continue using the tool but use things like psychologists to administer it. The only way you could reasonably have a psych on call for every police station is to make it a remote interview, which frankly doesn’t seem better to me.
In the end, the unstated problem is resources and how best to utilize them to prevent the violence. I’m sure Spain’s policy could be improved but shoring it up with an algorithm is a good practice.
Can we stop having AI do… anything?
Algorithms aren’t AI. They’re standardization measures in cases like this. Hell you don’t even need computers for many of them. We use tons in healthcare to classify risk, decide on treatment options, and even decide on how much medication to give.
The computer response should be treated as just an indication and in all cases a human needs to decide to override that
Otherwise we’ll all become useless pieces of a simulation
I went to the bank to ask a loan and then it got rejected because the computer said I didn’t met the parameters by just 40 euro. Ah ok, I told the clerk, just lower the amount that I’m asking or spread it over a longer period. No, because after the quote is done and I signed the authorization for the algorithm to perform credit score, it can’t do it again in 3 months. What?? Call a supervisor and let them override it, 40 euro is so minimal that it’s not that big issue. No, impossible. So that means each single employee in the bank is just an interface to the computer and can be fired at will?
Minority Don’t Report
Cracked me up thanks!
Pedantic Mathematician here.
If it failed, then it was a heuristic, rather than an algorithm.
Clearly, that’s the most important thing about this post.
You’re welcome.
Pretty much anything trying to predict human behavior is a heuristic; people using them as if they’ve got some kind of certainty is a problem.
Yes, exactly.
Why not both? A bad algorithm based on bad heuristics? There are many many algorithms that fail at what they’re supposed to do.
As a non-condescending “mathematician”, I’m happy to help.
I have no issues with using ML to predict outcomes. It’s going to be wrong sometimes, so will humans. The system just needs review and input from humans understanding the model.
The police accepted the software’s judgment and Ms. Hemid went home with no further protection.
This is what happens when you rely on your Nintendos, instead of using your damn brains.
Even when given the best and most sophisticated tools and equipment available, police will manage to fuck things up at every opportunity because they’re utterly incompetent.
And that’s why I’m against ALL such things.
Not because they can’t be done right and you can’t teach people to use them.
But because there’s a slippery slope of human nature where people want to offload the burden of decision to a machine, an oracle, a die, a set of bird intestines. The genie is out and they will do that again and again, but in a professional organization, like police, one can make a decision of creating fewer opportunities for such catastrophes.
The rule is that people shouldn’t use machines above their brains, as one other commenter says, and they should only use this in a logical OR with their own judgment made earlier, as another commenter says, but the problem is in human nature and I’d rather not introduce this particular point of failure to police, politics, anything juridical and military.
What is that?
It’s from movie Idiocracy from hospital scene. Initial diagnosis.
Here’s this part of the scene: https://youtu.be/LXzJR7K0wK0
It’s 2505 and the average man from 2005 is now by far the smartest man in the world.
It’s a Doctor’s diagnostic desk from the film, “Idiocracy”
And that’s why I’m against ALL such things.
Absolutely, ACAB
Cops are still necessary. It’s giving humans a machine to blame any failure upon is a very bad thing.
I personally think these "AI"s are supported by governments. There’s been a lot of talk 10-15 years ago how many government official’s functions can be replaced by AI (without quotes), since these functions do not require agenda and are not even too fuzzy, but require semantic understanding. So "AI"s (with quotes) are being used like a vaccine, so that the wide mass of humans would hate the guts of the very idea, having experienced them (EDIT: and wouldn’t want actual semantic reasoning systems). Why - because people working in governments love power and hate transparency, they also hate the idea of being replaced with machines.
Or maybe it’s a conspiracy theory and they all really believe in accelerationism.
some political groups engage in mismanagement on purpose to make people dislike the government, that’s hardly a conspiracy, but it’s a little weird to think they’re propping up the misuse of LLMs rather than that being a natural consequence of stupid capitalism
But the system seems to be better than police officers. Which is entirely believable. Humans have all kinds of biases that make the decisions we make far less than desirable.
Per the article, it has decreased the risk of repeated violence and, according to an expert, its the best systen we have. Why would you want to go back to a worse system? This is using our brains in an attempt to overcoming our biases.
