TeChNicAlLy it sort of already existed. A diffusion model can only generate an impossibly huge but finite number of images, and the content of these images is determined when the model is created. So when you ‘generate’ images you’re kinda just browsing a catalog
That’s not correct. It allows permutations of concepts it has “identified”. It’s only finite in the sense of being limited by the number of pixels and possible color values.
This is far from what “browsing a catalog” makes it sound like. It’s almost correct if you consider the catalogue as a collection of concepts. But it is literally generating an image based on a prompt that projects those concepts to an image. You can generate something with a combination of concepts, mixed together in a way that were never part of any training set
A racoon with purple fur, wearing sunglasses and a metal viking helmet with ivory horns, the blue planet earth is visible in the starry background
It doesn’t do a perfect job, but I also spent 2 minutes on this.
You missed my point I think. There is a finite number of possible prompts and settings resulting in a finite number of possible images. I wasn’t talking about training sets at all
Why do you think this is the case? When people say “finite”, it’s not obvious what they mean by it. Everything humans have made is finite, anything computers can represent, is finite, mathematically speaking. But, “finite” in mathematics, is as unsuited to use to describe the number of permutations of concepts in prompts, as it is to say that there is only a “finite amount of images” a computer can generate. It’s not a very interesting observation, because of course it is finite in this sense. Not to mention that the limits of this “finite”-ness, is still further bounded by human comprehension.
But, you are saying that this is missing your point… it’s difficult to know what that point is is. Because, if your point is that it is finite, in the mathematical sense, then… yes, you are technically right.
The latent space is discrete
The latent space is ultimately bounded
The latent space can only model a N-dimensional space for N “concepts”
All these three statements are true. The problem is mixing a vernacular use and understanding of finite, with a mathematical one.
The first one is similar to saying there are only a set of “finite images”, because Width * height * channels * (2^color_depth-1) is finite. The second one is also a bit strange to refer to it as such, because of course it will be bounded by the concepts that can exist and can be identified from training sets. The third one is fine. However, because there are a finite number of concepts, which then implies a mathematical finite number of permutations,… suggesting that this is “finite” in terms of being a “a catalogue”, is where you crossed the line between “technically correct” and “practically incorrect”.
Yes, the permutations are finite, but… putting all three together, there are as many permutations as there is resolution in the N-dimensional discretization of the latent space. I.e. similar to “all possible images a computer can output for a given color depth, channel count and size” is “finite”. If you mean to be “technically correct”, then it isn’t an interesting statement. If you allude to a vernacular understanding of finite (i.e. “just a catalogue”), then it simply isn’t correct.
My original comment was about the timing when the image is ‘created’. The ‘finiteness’ was supposed to be the supporting argument, but the other commenter put it in better words: “deterministic” (Even though they disagree on that). I’m not sure why you are so hung up on the word ‘catalogue’, that was just illustrative wording.
If you mean to be “technically correct”, then it isn’t an interesting statement.
My comment startet with the very word “technically”. If you find that uninteresting you’re free to just scroll by
TeChNicAlLy it sort of already existed. A diffusion model can only generate an impossibly huge but finite number of images, and the content of these images is determined when the model is created. So when you ‘generate’ images you’re kinda just browsing a catalog
The “already existed” gives it away. No, this didn’t “sort of already existed”, and there is no technically correct interpretation that can give you the benefit of the doubt either.
content of these images is determined when the model is created
Nope. The diffusion model guides a random noise image. Mathematically, any image can be generated out of pure randomness without any effect by model. See how silly and uninteresting “technical correct” makes things?
So when you ‘generate’ images you’re kinda just browsing a catalog
Nope.
It is OK to admit you were wrong. No need to double down on this. I just wanted to explain why it was wrong.
Besides the point the other commenter already made, I’d like to add that inference isn’t deterministic per model. There are a bunch of sources of inconsistency:
GPU hardware/software can influence the results of floating point operations
Different inference implementations can change the order of operations (and matrix operations aren’t necessarily commutative)
Different RNG implementations can change the space of possible seed images
If you generate with the same prompt and settings you get what I would consider the same image except for tiny variations (they aren’t matching pixel-perfect)
Edit: A piece of paper has a random 3D relief of fibers, so the exact position a printer ink droplet ends up at is also not deterministic, and so no two copies of a physical catalog are identical. But we would still consider them the “same” catalog
If there’s slight variation, it means it’s not the same image.
And that’s skipping over different RNG etc. You can build a machine learning model today and give it to me, tomorrow I can create a new RNG - suddenly the model can produce images it couldn’t ever produce before.
It’s very simple: the possible resulting images aren’t purely determined by the model, as you claimed.
TeChNicAlLy it sort of already existed. A diffusion model can only generate an impossibly huge but finite number of images, and the content of these images is determined when the model is created. So when you ‘generate’ images you’re kinda just browsing a catalog
That’s not correct. It allows permutations of concepts it has “identified”. It’s only finite in the sense of being limited by the number of pixels and possible color values.
This is far from what “browsing a catalog” makes it sound like. It’s almost correct if you consider the catalogue as a collection of concepts. But it is literally generating an image based on a prompt that projects those concepts to an image. You can generate something with a combination of concepts, mixed together in a way that were never part of any training set
It doesn’t do a perfect job, but I also spent 2 minutes on this.
Vikings didn’t have helmets with horns.
Indeed. That’s why I specified “with ivory horns” :P
That goes hard
You missed my point I think. There is a finite number of possible prompts and settings resulting in a finite number of possible images. I wasn’t talking about training sets at all
Why do you think this is the case? When people say “finite”, it’s not obvious what they mean by it. Everything humans have made is finite, anything computers can represent, is finite, mathematically speaking. But, “finite” in mathematics, is as unsuited to use to describe the number of permutations of concepts in prompts, as it is to say that there is only a “finite amount of images” a computer can generate. It’s not a very interesting observation, because of course it is finite in this sense. Not to mention that the limits of this “finite”-ness, is still further bounded by human comprehension.
But, you are saying that this is missing your point… it’s difficult to know what that point is is. Because, if your point is that it is finite, in the mathematical sense, then… yes, you are technically right.
All these three statements are true. The problem is mixing a vernacular use and understanding of finite, with a mathematical one.
The first one is similar to saying there are only a set of “finite images”, because Width * height * channels * (2^color_depth-1) is finite. The second one is also a bit strange to refer to it as such, because of course it will be bounded by the concepts that can exist and can be identified from training sets. The third one is fine. However, because there are a finite number of concepts, which then implies a mathematical finite number of permutations,… suggesting that this is “finite” in terms of being a “a catalogue”, is where you crossed the line between “technically correct” and “practically incorrect”.
Yes, the permutations are finite, but… putting all three together, there are as many permutations as there is resolution in the N-dimensional discretization of the latent space. I.e. similar to “all possible images a computer can output for a given color depth, channel count and size” is “finite”. If you mean to be “technically correct”, then it isn’t an interesting statement. If you allude to a vernacular understanding of finite (i.e. “just a catalogue”), then it simply isn’t correct.
My original comment was about the timing when the image is ‘created’. The ‘finiteness’ was supposed to be the supporting argument, but the other commenter put it in better words: “deterministic” (Even though they disagree on that). I’m not sure why you are so hung up on the word ‘catalogue’, that was just illustrative wording.
My comment startet with the very word “technically”. If you find that uninteresting you’re free to just scroll by
Naah. I call BS. You know what you’re implying.
The “already existed” gives it away. No, this didn’t “sort of already existed”, and there is no technically correct interpretation that can give you the benefit of the doubt either.
Nope. The diffusion model guides a random noise image. Mathematically, any image can be generated out of pure randomness without any effect by model. See how silly and uninteresting “technical correct” makes things?
Nope.
It is OK to admit you were wrong. No need to double down on this. I just wanted to explain why it was wrong.
Besides the point the other commenter already made, I’d like to add that inference isn’t deterministic per model. There are a bunch of sources of inconsistency:
If you generate with the same prompt and settings you get what I would consider the same image except for tiny variations (they aren’t matching pixel-perfect)
Edit: A piece of paper has a random 3D relief of fibers, so the exact position a printer ink droplet ends up at is also not deterministic, and so no two copies of a physical catalog are identical. But we would still consider them the “same” catalog
If there’s slight variation, it means it’s not the same image.
And that’s skipping over different RNG etc. You can build a machine learning model today and give it to me, tomorrow I can create a new RNG - suddenly the model can produce images it couldn’t ever produce before.
It’s very simple: the possible resulting images aren’t purely determined by the model, as you claimed.