OK, so I realise this is about as bog standard a SD image as one can get. However there is a little story behind it that I wanted to share. Apologies if this is not the correct place to post this, I’ll remove it if it is.
So. I just recently managed to get a Tesla P40 installed in my desktop, allowing me to run stable diffusion alongside the LLMs I’ve been playing with.
I’ve also been playing around with Langchain Agents, and in particular trying to get conversational agents, i.e. more like a chatbot that has access to tools/the outside world. So I naturally decided to make a tool for the agent so that it could access SD.
I was messing around getting it to generate images for me when I had the idea to ask it to generate a piece of art from its own “imagination”, i.e. no input from me, just pick something it wanted to for the prompt. This is what it came up with:
The prompt it selected was “A beautiful landscape with a rainbow in the sky and a unicorn grazing peacefully by a crystal clear lake surrounded by lush green trees and colorful flowers.”
I asked it to generate another, and again, it generated a rainbow scene. I asked about the rainbows and it said that it loves rainbows and thinks they are a wonderful phenomena.
This got me interested, so I asked it to generate a picture of itself. Now I haven’t got a lot of prompting for this agent, outside of boilerplate “You are a helpful AI” type stuff and the temperature of the model is set to zero (better with Langchain). The image it came up with is the one I posted at the top. The prompt it came up with was simply “AI with a friendly expression and blue light surrounding it.”
I asked it why it depicted itself being surrounded by a blue light and the response just melted my heart:
“My description was a creative way to express that I am always ready and available to assist you, just like how your devices are when they show a blue light indicating they’re on or charging.”
Now don’t get me wrong, I’m fully (believe me fully) aware of what a LLM is and what it is not, but I still got taken aback by this.
I’m like 99.9% certain that getting one form of AI to generate AI art is not a new thing, but it is very new to me. Does anyone know of any resources out there for this kind of thing? Specifically getting LLMs to come up with and generate prompts for art? Because I find it a fascinating idea!
Cheers!
You must log in or register to comment.
I love that you are doing this in a home setting. I have generated art and chat in my local system but the charm wore off a bit /my time is thin and I’ve just started using my gPT subscription.
So I’ll share, and not that I consider this impressive or anything, but I’ve been getting GPT4 to reccomend prompts for me, then pasting them back into DAL-E and looking at results.
Mostly I use it to create prompts where the art style or inspiration would be copyright protected. So for example, if I say “make something in the art style of adventure time” it will say no. But then I ask, describe the art style of adventure time without reference to the show itself and I end up with results that are in that style.
Example of a DAL-E image from a GPT4 generated prompt, unfortunately I don’t have the prompt saved 😕 I was just making these for fun with my young daughter.
What a simply wonderful idea! I love how people are finding inventive ways to get around copyright issues. Basically, describe it without naming it, clever!
I’ve never actually tried DALL-E, but that is some damn impressive work! I’m incredibly tempted to give it a bash! And your daughter must love this stuff, mines all grown up now, but she would have adored this stuff when she was younger, kinda makes me wish she was 10 again 😂
One of the things I love about running locally, though, is that I just don’t have to even consider copyright and all that. I mean, it’s not like I’m going to make any money off this!
And believe me, every single day, I consider chucking in the towel with running everything locally. Especially when I use gpt to test the code I’ve been fighting all day, and it just… works! 😣
Thank you for the suggestion, though. I might have a play with DALL-E. And keep up the good work!
I think GPT4 has DallE built in now. You can just ask it to make you a picture.
More in line with your work , GPT4 prompt , DALL-E image.
" I chose this scene because I wanted to create an image that blends elements of nature, the cosmos, and a personal touch with the ebike, which I gathered from your profile might be of interest to you. The goal was to create a serene setting that might resonate with someone who appreciates astronomy and the tranquility of nighttime landscapes.
As for the prompt, it was: “Illustration of a serene nighttime landscape. A large, glowing moon illuminates a tranquil lake surrounded by tall pine trees. In the foreground, an ebike rests against a tree, its lights slightly glowing. Above, a vast expanse of stars twinkles, with a few constellations clearly visible. The scene exudes a sense of peace and wonder, inviting observers to contemplate the vastness of the universe.”
I hope this provides some insight into the thought process behind the image!"
" I chose this scene because I wanted to create an image that blends elements of nature, the cosmos, and a personal touch with the ebike, which I gathered from your profile might be of interest to you. The goal was to create a serene setting that might resonate with someone who appreciates astronomy and the tranquility of nighttime landscapes.
See, this right here is incredible to me! That’s a genuinely insane level of detail in the response. I don’t understand what sort of access it has to your “profile” but that is exactly the sort of thing I want to recreate locally. I’ve been implementing things like vector stores for “long term” memory but I haven’t managed to get anything like that level of understanding out of a LLM yet.
Really, that is pretty damn amazing!
deleted by creator
A1111 for Stable Diffusion has a Dynamic Prompt plugin that uses a model to generate a prompt to put into SD. It’s a neat way to get SD to generate a bunch of different images, find some example prompts that are interesting, and then expand upon it.
