This is tough. If it was just a sicko who generated the images for himself locally… that is the definition of a victimless crime, no? And it might actually dissuade him from seeking out real CSAM…
BUT, iirc he was actually distributing the material, and even contacted minors, so… yeah he definitely needed to be arrested.
But, I’m still torn on the first scenario…
What is the AI trained on?
Image-generating AI is capable of generating images that are not like anything that was in its training set.
In that case probably the strongest argument is that if it were legal, many people would get off charges of real CSAM because the prosecuter can’t prove that it wasn’t AI generated.
Better a dozen innocent men go to prison than one guilty man go free?
In this case if they know it’s illegal, then they knowingly broke the law? Things are still illegal even if you don’t agree with it.
Most (many?) Western countries also ban cartoon underage content, what’s the justification for that?
You suggested a situation where “many people would get off charges of real CSAM because the prosecuter can’t prove that it wasn’t AI generated.” That implies that in that situation AI-generated CSAM is legal. If it’s not legal then what does it matter if it’s AI-generated or not?
That’s not quite what I was getting at over the course of the comment thread.
It one scenario, AI material is legal. Those with real CSAM use the defense that it’s actually AI and you can’t prove otherwise. In this scenario, no innocent men are going to prison, and most guilty men aren’t either.
The second scenario we make AI material illegal. Now the ones with real CSAM go to prison, and many people with AI material do too because it’s illegal and they broke the law.
To be honest, if it prevents that one guilty man from carrying out such high degrees of abuse to a dozen children, I can’t say I’d say no.
I want to stress that this isn’t sensationalist grandstanding like wanting to ban rock music or video games or spying on all digital communication in the name of protecting the children. It’s just the pragmatic approach towards preventing CSAM in an age where the “know it when I see it” definition of pornographic material is starting to blur the lines.
Well, your philosophy runs counter to the fundamentals of Western justice systems, then.
AI can compose novel looking things from components it has been trained on - it can’t imagine new concepts. If CSAM is being generated it’s because it was included in it’s training set which is highly suspected as we know the common corpus had CSAM in it: https://cyber.fsi.stanford.edu/news/investigation-finds-ai-image-generation-models-trained-child-abuse
If it has images of construction equipment and houses, it can make images of houses that look like construction equipment. Swap out vocabulary as needed.
Cool, how would it know what a naked young person looks like? Naked adults look significantly different.
It understands young and old.
Is a kid just a 60% reduction by volume of an adult? And these are generative algorithms… nobody really understands how it perceives the world and word relations.
Very, very good point. Depending on the answer, I retract the “victimless” narrative.
But, I’m still torn on the first scenario…
To me it comes down to a single question:
“Does exposure and availability to CSAM for pedophiles correlate with increased or decreased likelihood of harming a child?”
If there’s a reduction effect by providing an outlet for arousal that isn’t actually harming anyone - that sounds like a pretty big win.
If there’s a force multiplier effect where exposure and availability means it’s even more of an obsession and focus such that there’s increased likelihood to harm children, then society should make the AI generated version illegal too.
Hoooooly hell, good luck getting that study going. No ethical concerns there!
How they’ve done it in the past is by tracking the criminal history of people caught with csam, arrested for abuse, or some combination thereof, or by tracking the outcomes of people seeking therapy for pedophilia.
It’s not perfect due to the sample biases, but the results are also quite inconsistent, even amongst similar populations.
I’m willing to bet it’ll differ from person to person, to complicate matters further
I think the general consensus is that availability of CSAM is bad, because it desensitizes and makes harming of actual children more likely. But I must admit that I only remember reading about that and don’t have a scientific source.
It’s interesting your bring this up. Not long ago I was having basically this exact same discussion with my brother. Baring you second point, I honestly don’t know how I feel.
On the one hand - if it’s strictly images for himself and it DOES dissuade seeking out real CSAM (I’m not convinced of this) then I don’t really see the issue.
On the other hand - I feel like it could be a gateway to something more (your second point). Kinda like a drug, right? You need a heavier and heavier hit to keep the same high. Seems like it wouldn’t be a stretch to go from AI generated imagery to actual CSAM.
But yeah, I don’t know. We live in an odd time for sure.
First off, this is obviously a sticky topic. Every conversation is controversial and speculative.
Second, I don’t really see a lot of legitimacy to the “gateway” concept. The vast majority of people use some variety of drug (caffeine, alcohol, nicotine), and that doesn’t really reliably predict “harder” drug use. Lots of people use marijuana and that doesn’t reliably predict hard drug use. Obviously, the people who use heroin and meth have probably used cocaine and ketamine, and weed before that, and alcohol/caffeine/nicotine before that, but that’s not really a “gateway” pipeline so much as paying through finer and finer filters. As far as I know, the concept has fallen pretty heavily out of favor with serious researchers.
In light of that perspective, I think you have to consider the goal. Is your goal to punish people, or to reduce the number and severity of victims? Mine is the latter. Personally, I think this sort of thing peels off many more low-level offenders to low-effort outlets than it emboldens to higher-severity outlets. I think this is ultimately a mental-health problem, and zero-tolerance mandatory reporting (while well-meaning) does more harm than good.
I’d rather that those with these kinds of mental issues have 1. the tools to take the edge off in victimless ways 2. safe spaces to discuss these inclinations without fear of incarceration. I think blockading those avenues yields a net increase the number and severity of victims.
This seems like a net benefit, reducing the overall number and severity of actual victims.
On the other hand - I feel like it could be a gateway to something m
You mean like marijuana and violent video games?
Except in the case of pornography, it’s an open question if viewing it has a net increase or decrease in sexual desire.
With legal pornography, it’s typically correlated with higher sexual desire. This tracks intuitively, since the existence of pornography does not typically seem to line up with a drop in people looking for romantic partners.There’s little reason to believe it works the other way around for people attracted to children.
What’s unknown is if that desire is enough to outweigh the legal consequences they’re aware of, or any social or ethical boundaries present.
Studies have been done, but finding people outside of the legal system who abuse children is exceptionally difficult, even before the ethical obligation to report them to the police would trash the study.
So the studies end up focusing either on people actively seeking treatment for unwanted impulses (less likely to show a correlation), or people engaged with the legal system in some capacity (more likely to show correlation).Holy strawman, Batman! Just because someone uses the term “gateway” doesn’t mean they think that games and weed are going to turn all people and frogs gay and violent.
Thanks for being honest and well-meaning. Sorry you’re getting downvoted, we both said pretty much exactly the same thing! A difficult subject, but important to get right…
Fuck that guy first of all.
What makes me think is, what about all that cartoon porn showing cartoon kids? What about hentai showing younger kids? What’s the difference if all are fake and being distributed online as well?
Not defending him.
I think there’s certainly an argument here. What if the hentai was more lifelike? What if the AI stuff was less realistic? Where’s the line?
At least in the US, courts have been pretty shitty at defining things like “obscenity”. This AI stuff might force them to delineate more clearly.
What if someone draws their own CSAM and they’re terrible at drawing but it’s still recognizable as CSAM?
Ethically is one question, but the law is written such that it’s pretty narrowly covering only photograph-style visual depictions that are virtually indistinguishable from an actual child engaged in explicit conduct in the view of a reasonable person that is also lacking in any other artistic or cultural significance.
Or in short: if it looks like an actual image of actual children being actually explicit, then it’s illegal.Makes sense.
While I think Hentai showing that stuff is disgusting AI is worse because you need to get the training material from somewhere so its far from victimless. Edit: I just learned that it does not have to be in the dataset though there should be regulations that forces the companies to open source the data set.
I find it interesting that the relabeling of CP to CSAM weakens their argument here. “CP generated by AI is still CP” makes sense, but if there’s no abusee, it’s just CSM. Makes me wonder if they would have not rebranded if they knew about the proliferation of AI pornography.
The problem is that it abets the distribution of legitimate CSAM more easily. If a government declares “these types of images are okay if they’re fake”, you’ve given probable deniability to real CSAM distributors who can now claim that the material is AI generated, placing the burden on the legal system to prove it to the contrary. The end result will be a lot of real material flying under the radar because of weak evidence, and continued abuse of children.
Better to just blanket ban the entire concept and save us all the trouble, in my opinion. Back before it was so easy to generate photorealistic images, it was easier to overlook victimless CP because illustrations are easy to tell apart from reality, but times have changed, and so should the laws.
Not necessarily. There’s been a lot of advances in watermarking AI outputs.
As well, there’s the opposite argument.
Right now, pedophile rings have very high price points to access CSAM or require users to upload original CSAM content, adding a significant motivator to actually harm children.
The same way rule 34 artists were very upset with AI being able to create what they were getting commissions to create, AI generated CSAM would be a significant dilution of the market.
Is the average user really going to risk prison, pay a huge amount of money or harm a child with an even greater prison risk when effectively identical material is available for free?
Pretty much overnight the CSAM dark markets would lose the vast majority of their market value and the only remaining offerings would be ones that could demonstrate they weren’t artificial to justify the higher price point, which would undermine the notion of plausible deniability.
Legalization of AI generated CSAM would decimate the existing CSAM markets.
That said, the real question that needs to be answered from a social responsibility perspective is what the net effect of CSAM access by pedophiles has on their proclivity to offend. If there’s a negative effect then it’s an open and shut case that it should be legalized. If it’s a positive effect than we should probably keep it very much illegal, even if that continues to enable dark markets for the real thing.
Not necessarily. There’s been a lot of advances in watermarking AI outputs.
That presumes that the image generation is being done by some corporation or government entity that adds the watermarks to AI outputs and doesn’t add them to non-AI outputs. I’m not thrilled that AI of this sort exists at all, but given that it does, I’d rather not have it controlled by such entities. We’re heading towards a world where we can all run that stuff on our own computers and control the watermarks ourselves. Is that good or bad? Probably bad, but having it under the exclusive control of megacorps has to be even worse.
Is the average user really going to risk prison, pay a huge amount of money or harm a child with an even greater prison risk when effectively identical material is available for free?
Average users aren’t pedophiles and it would appear that yes they would considering he did exactly that. He had access to tools that generated the material for free, which he then used to entice boys.
I agree, just the linguistics are interesting.
placing the burden on the legal system to prove it to the contrary.
That’s how it should be. Everyone is innocent until proven otherwise.
Who’s even in charge of that lol
Have to agree. Because I have no clue what CSAM is. My first glance at the title made me think it was CSPAN (the TV channel)… So CP is better identifier, as of at least recognize the initialism.
If we could stop turning everything, and especially important things, into acronyms and initialisms that’d be great.
A generative AI could not generate CSAM without access to CSAM training data. Abuse was a necessary step in the generation.
Can’t generate Abuse Material without Abuse. Generative AI does not need any indecent training to be able to produce indecent merial.
But it is a nice story to shock and scare many people so i guess the goal is reached.
It is amazing how Lemmy can usually be such a well informed audience but for some reason when it comes to AI people simply refuse to acknowledge that it was trained on CSAM https://cyber.fsi.stanford.edu/news/investigation-finds-ai-image-generation-models-trained-child-abuse
And don’t understand how generative AI combines existing concepts to synthesize images - it doesn’t have the ability to create novel concepts.
AI models don’t resynthesize their training data. They use their training data to determine parameters which enable them to predict a response to an input.
Consider a simple model (too simple to be called AI but really the underlying concepts are very similar) - a linear regression. In linear regression we produce a model which follows a straight line through the “middle” of our training data. We can then use this to predict values outside the range of the original data - albeit will less certainty about the likely error.
In the same way, an LLM can give answers to questions that were never asked in its training data - it’s not taking that data and shuffling it around, it’s synthesising an answer by predicting tokens. Also similarly, it does this less well the further outside the training data you go. Feed them the right gibberish and it doesn’t know how to respond. ChatGPT is very good at dealing with nonsense, but if you’ve ever worked with simpler LLMs you’ll know that typos can throw them off notably… They still respond OK, but things get weirder as they go.
Now it’s certainly true that (at least some) models were trained on CSAM, but it’s also definitely possible that a model that wasn’t could still produce sexual content featuring children. It’s training set need only contain enough disparate elements for it to correctly predict what the prompt is asking for. For example, if the training set contained images of children it will “know” what children look like, and if it contains pornography it will “know” what pornography looks like - conceivably it could mix these two together to produce generated CSAM. It will probably look odd, if I had to guess? Like LLMs struggling with typos, and regression models being unreliable outside their training range, image generation of something totally outside the training set is going to be a bit weird, but it will still work.
None of this is to defend generating AI CSAM, to be clear, just to say that it is possible to generate things that a model hasn’t “seen”.
Not all models use the same training sets, and not all future models would either.
Generating images of humans of different ages doesn’t require having images of that type for humans of all ages.
Like, no one is arguing your link. Some models definitely used training data with that, but your claim that the type of image discussed is “novel” simply isn’t accurate to how these models can combine concepts
it was trained on CSAM
In that case, why haven’t the people who made the AI models been arrested?
Dunno, probably because they didn’t knowingly train it on CSAM - maybe because it’s difficult to prove what actually goes into neural network configuration so it’s unclear how strongly weighted it is… and lastly, maybe because this stuff is so cloaked in obscurity and proprietaryness that nobody is confident how such a case would go.
And don’t understand how generative AI combines existing concepts to synthesize images - it doesn’t have the ability to create novel concepts.
Imagine someone asks you to shoop up some pr0n showing Donald Duck and Darth Vader. You’ve probably never seen that combination in your “training set” (past experience) but it doesn’t exactly take creating novel concepts to fulfill the request. It’s just combining existing ones. Web search on “how stable diffusion works” finds some promising looking articles. I read one a while back and found it understandable. Stable Diffusion was the first of these synthesis programs but the newer ones are just bigger and fancier versions of the same thing.
Of course idk what the big models out there are actually trained on (basically everything they can get, probably not checked too carefully) but just because some combination can be generated in the output doesn’t mean it must have existed in the input. You can test that yourself easily enough, by giving weird and random enough queries.
No, you’re quite right that the combination didn’t need to exist in the input for an output to be generated - this shit is so interesting because you can throw stuff like “A medieval castle but with Iranian architecture with a samurai standing on the ramparts” at it and get something neat out. I’ve leveraged AI image generation for visual D&D references and it’s excellent at combining comprehended concepts… but it can’t innovate a new thing - it excels at mixing things but it isn’t creative or novel. So I don’t disagree with anything you’ve said - but I’d reaffirm that it currently can make CSAM because it’s trained on CSAM and, in my opinion, it would be unable to generate CSAM (at least to the quality level that would decrease demand for CSAM among pedos) without having CSAM in the training set.
it currently can make CSAM because it’s trained on CSAM
That is a non sequitur. I don’t see any reason to believe such a cause and effect relationship. The claim is at least falsifiable in principle though. Remove whatever CSAM found its way into the training set, re-run the training to make a new model, and put the same queries in again. I think you are saying that the new model should be incapable of producing CSAM images, but I’m extremely skeptical, as your medieval castle example shows. If you’re now saying the quality of the images might be subtly different, that’s the no true Scotsman fallacy and I’m not impressed. Synthetic images in general look impressive but not exactly real. So I have no idea how realistic the stuff this person was arrested for was.
Then we should be able to charge AI (the developers moreso) for the same disgusting crime, and shut AI down.
Camera-makers, too. And people who make pencils. Lock the whole lot up, the sickos.
Camera makers and pencil makers (and the users of those devices) aren’t making massive server farms that spy on every drop of information they can get ahold of.
If AI has the means to generate inappropriate material, then that means the developers have allowed it to train from inappropriate material.
Now when that’s the case, well where did the devs get the training data?.. 🤔
If AI has the means to generate inappropriate material, then that means the developers have allowed it to train from inappropriate material.
That’s not how generative AI works. It’s capable of creating images that include novel elements that weren’t in the training set.
Go ahead and ask one to generate a bonkers image description that doesn’t exist in its training data and there’s a good chance it’ll be able to make one for you. The classic example is an “avocado chair”, which an early image generator was able to produce many plausible images of despite only having been trained on images of avocados and chairs. It understood the two general concepts and was able to figure out how to meld them into a common depiction.
Yes, I’ve tried similar silly things. I’ve asked AI to render an image of Mr. Bean hugging Pennywise the clown. And it delivered, something randomly silly looking, but still not far off base.
But when it comes to inappropriate material, well the AI shouldn’t be able to generate any such thing in the first place, unless the developers have allowed it to train from inappropriate sources…
The trainers didn’t train the image generator on images of Mr. Bean hugging Pennywise, and yet it’s able to generate images of Mr. Bean hugging Pennywise. Yet you insist that it can’t generate inappropriate images without having been specifically trained on inappropriate images? Why is that suddenly different?
The trainers taught it what Mr. Bean looks like and what Pennywise looks like - it took those concepts and combined them to create your image. To make CSAM it was, unfortunately, trained on CSAM https://cyber.fsi.stanford.edu/news/investigation-finds-ai-image-generation-models-trained-child-abuse
3,226 suspected images out of 5.8 billion. About 0.00006%. And probably mislabeled to boot, or it would have been caught earlier. I doubt it had any significant impact on the model’s capabilities.
Who is responsible then? Cuz the devs basically gotta let the AI go to town on many websites and documents for any sort of training set.
So you mean to say, you can’t blame the developers, because they just made a tool (one that scrapes data from everywhere possible), can’t blame the tool (don’t mind that AI is scraping all your data), and can’t blame the end users, because some dirty minded people search or post inappropriate things…?
So where’s the blame go?
First, you need to figure out exactly what it is that the “blame” is for.
If the problem is the abuse of children, well, none of that actually happened in this case so there’s no blame to begin with.
If the problem is possession of CSAM, then that’s on the guy who generated them since they didn’t exist at any point before then. The trainers wouldn’t have needed to have any of that in the training set so if you want to blame them you’re going to need to do a completely separate investigation into that, the ability of the AI to generate images like that doesn’t prove anything.
If the problem is the creation of CSAM, then again, it’s the guy who generated them.
If it’s the provision of general-purpose art tools that were later used to create CSAM, then sure, the AI trainers are in trouble. As are the camera makers and the pencil makers, as I mentioned sarcastically in my first comment.
I think that’s a bit of a stretch. If it was being marketed as “make your fantasy, no matter how illegal it is,” then yeah. But just because I use a tool someone else made doesn’t mean they should be held liable.
Check my other comments. My thought was compared to a hammer.
Hammers aren’t trained to act or respond on their own from millions of user inputs.
Image AIs also don’t act or respond on their own. You have to prompt them.
And if I prompted AI for something inappropriate, and it gave me a relevant image, then that means the AI had inappropriate material in it’s training data.
No, you keep repeating this but it remains untrue no matter how many times you say it. An image generator is able to create novel images that are not directly taken from its training data. That’s the whole point of image AIs.
An image generator is able to create novel images that are not directly taken from its training data. That’s the whole point of image AIs.
I just want to clarity that you’ve bought the silicon valley hype for AI but that is very much not the truth. It can create nothing novel - it can merely combine concepts and themes and styles in an incredibly complex manner… but it can never create anything novel.
What it’s able and intended to do is besides the point, if it’s also capable of generating inappropriate material.
Let me spell it more clearly. AI wouldn’t know what a pussy looked like if it was never exposed to that sort of data set. It wouldn’t know other inappropriate things if it wasn’t exposed to that data set either.
Do you see where I’m going with this? AI only knows what people allow it to learn…
You realize that there are perfectly legal photographs of female genitals out there? I’ve heard it’s actually a rather popular photography subject on the Internet.
Do you see where I’m going with this? AI only knows what people allow it to learn…
Yes, but the point here is that the AI doesn’t need to learn from any actually illegal images. You can train it on perfectly legal images of adults in pornographic situations, and also perfectly legal images of children in non-pornographic situations, and then when you ask it to generate child porn it has all the concepts it needs to generate novel images of child porn for you. The fact that it’s capable of that does not in any way imply that the trainers fed it child porn in the training set, or had any intention of it being used in that specific way.
As others have analogized in this thread, if you murder someone with a hammer that doesn’t make the people who manufactured the hammer guilty of anything. Hammers are perfectly legal. It’s how you used it that is illegal.
I learned how to write by reading. The AI did the same, more or less, no?
The AI didn’t learn to draw or generate photos from blind words though…
Oh, it learned from art? Like how human artists learn?
AI hasn’t exactly kicked out a Picasso with a naked young girl missing an ear yet has it?
I sure hope not!
But if it can, then that seriously indicates it must have some bad training data in the system…
I won’t be testing these hypotheses.
It in fact does have bad training data! https://cyber.fsi.stanford.edu/news/investigation-finds-ai-image-generation-models-trained-child-abuse
…no
That’d be like outlawing hammers because someone figured out they make a great murder weapon.
Just because you can use a tool for crime, doesn’t mean that tool was designed/intended for crime.
It would be more like outlawing ivory grand pianos because they require dead elephants to make - the AI models under question here were trained on abuse.
A person (the arrested software engineer from the article) acquired a tool (a copy of Stable Diffusion, available on github) and used it to commit crime (trained it to generate CSAM + used it to generate CSAM).
That has nothing to do with the developer of the AI, and everything to do with the person using it. (hence the arrest…)
I stand by my analogy.
Unfortunately the developer trained it on some CSAM which I think means they’re not free of guilt - we really need to rebuild these models from the ground up to be free of that taint.
Reading that article:
Given it’s public dataset not owned or maintained by the developers of Stable Diffusion; I wouldn’t consider that their fault either.
I think it’s reasonable to expect a dataset like that should have had screening measures to prevent that kind of data being imported in the first place. It shouldn’t be on users (here meaning the devs of Stable Diffusion) of that data to ensure there’s no illegal content within the billions of images in a public dataset.
That’s a different story now that users have been informed of the content within this particular data, but I don’t think it should have been assumed to be their responsibility from the beginning.
Sounds to me it would be more like outlawing grand pianos because of all of the dead elephants - while some people are claiming that it is possible to make a grand piano without killing elephants.
There’s CSAM in the training set[1] used for these models so some elephants have been murdered to make this piano.
3,226 suspected images out of 5.8 billion. About 0.00006%. And probably mislabeled to boot, or it would have been caught earlier. I doubt it had any significant impact on the model’s capabilities.
Sadly that’s what most of the gun laws are designed about. Book banning and anti-abortion both are limiting tools because of what a small minority choose to do with the tool.
AI image generation shouldn’t be considered in obscenity laws. His distribution or pornography to minor should be the issue, because not everyone stuck with that disease should be deprived tools that can be used to keep them away from hurting others.
Using AI images to increase charges should be wrong. A pedophile contacting and distributing pornography to children should be all that it takes to charge a person. This will just setup new precedent that is beyond the scope of the judiciary.
That’s not the point. You don’t train a hammer from millions of user inputs.
You gotta ask, if the AI can produce inappropriate material, then where did the developers get the training data, and what exactly did they train those AI models for?
Do… Do you really think the creators/developers of Stable Diffusion (the AI art tool in question here) trained it on CSAM before distributing it to the public?
Or are you arguing that we should be allowed to do what’s been done in the article? (arrest and charge the individual responsible for training their copy of an AI model to generate CSAM)
One, AI image generators can and will spit out content vastly different than anything in the training dataset (this ofc can be influenced greatly by user input). This can be fed back into the training data to push the model towards the desired outcome. Examples of the desired outcome are not required at all. (IE you don’t have to feed it CSAM to get CSAM, you just have to consistently push it more and more towards that goal)
Two, anyone can host an AI model; it’s not reserved for big corporations and their server farms. You can host your own copy and train it however you’d like on whatever material you’ve got. (that’s literally how Stable Diffusion is used) This kind of explicit material is being created by individuals using AI software they’ve downloaded/purchased/stolen and then trained themselves. They aren’t buying a CSAM generator ready to use off the open market… (nor are they getting this material from publicly operating AI models)
They are acquiring a tool and moulding it into a weapon of their own volition.
Some tools you can just use immediately, others have a setup process first. AI is just a tool, like a hammer. It can be used appropriately, or not. The developer isn’t responsible for how you decide to use it.
Do… Do you really think the creators/developers of Stable Diffusion (the AI art tool in question here) trained it on CSAM before distributing it to the public?
Yes. Because they did (not intentionally though)
3,226 suspected images out of 5.8 billion. About 0.00006%. And probably mislabeled to boot, or it would have been caught earlier. I doubt it had any significant impact on the model’s capabilities.
Then that settles it. It’s whoever allows bad data into the training data.
Can we do guns next?
I’d rather not fart bullets, but thank you for inviting me to the party.
