A new study from researchers at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) suggests that the more data a generative model is trained on, the less likely an AI-generated image is to be traced back to a source image. They call this phenomenon attribution decay.
“If you take away a piece of data and the output of the model doesn’t change, then that piece of data didn’t affect the output,” said Zheng Dai, a former MIT CSAIL researcher and the lead author on the study, published yesterday in Nature Communications. “So it doesn’t make much sense to attribute the output to that piece of data. And if you then do this one at a time for every other piece of data and find that the output doesn’t change for any of them either, then it doesn’t make much sense to attribute the output to any one of them.”
The researchers trained 24 ensembles on datasets ranging from 256 images to more than 160,000, taken from public collections. They say their method involved retraining the model from scratch each time they removed a particular image so that they could observe concrete results; previous studies, they say, relied on approximations to estimate the influence of any one source image. They built an architecture they call a “diffusion ensemble,” with several models, each trained on a different data subset.
“All previous methods were approximate,” said MIT professor David Gifford, a CSAIL principal investigator. “They really could not absolutely show that deleting individual things did not change the output. This paper introduces the first method that is absolute. You’re actually deleting the inputs and deleting all influences of the inputs. This is the first exact method for doing large-scale deletion efficiently and showing that the results don’t change.”
The authors put their ensemble up against 24 conventional diffusion models trained on the same data and found that the images “came out looking about as good by standard measures,” according to press materials.
Gifford believes the results bear on the legal question of whether models’ outputs are merely derivative works, with implications for copyright.
“One way to think about this is that these models are creative,” he said in press materials. “They are not simply copying what they are fed, but creating brand new outputs. If those outputs have nothing to do with any individual piece of training data, that raises questions about fair use, about whether the outputs are themselves copyrightable as novel works, and about how authors get compensated when what comes out of a model isn’t attributable to anything on the internet.”
