I think the inherit issue is the current “AI” is inherently non-deterministic, so it’s impossible to fix these issues totally. You can feed am AI all the data on how to not sound AI, but you need massive amounts of non-AI writing to reinforce that. With AI being so prevalent nowadays you can’t guarantee a dataset nowadays is AI free, so you get the old “garbage in garbage out” problem that AI companies cannot solve. I still think generative AI has it’s place as a tool, I use it for quick and dirty text manipulation, but it’s being applied to every problem we have like it’s a magic silver bullet. I’m ranting at this point and I’m going to stop here.
FWIW, LLMs are deterministic. Usually the commercial front-ends don’t let you set the seed but behind the scenes the only reason the output changes each time it’s that the seed changes. If you set a fixed seed, input X always leads to output Y.
I honestly disagree that it has any use. Being a statistical model with high variance makes it a liability, no matter which task you use it for will produce worse results than a human being and will create new problems that didn’t exist before.
I use it to put together study guides so that, instead of spending a bunch of time typing and formatting, I’m spending it studying. It’s fed directly from my notes and slides and it rarely gets anything wrong (I read through it twice and cross reference with my notes). If anything, I’m usually removing stuff for being unnecessary or rewording things here and there to be.bstter suited to me. What took several hours now takes 30-45 minutes
Don’t take this as a defense of AI, it definitely isn’t. If AI disappeared tomorrow the world would be better off. Formatting study guides are literally the only utility I’ve found in LLMs
Tell that to my 3.9 GPA lol. I’m not learning from slop, I’m using a program to format my notes and slides from my lectures, and then verifying that information before committing it to memory
My deepest apologies, sir. You’re clearly a man of deep intellect and wisdom beyond your years! You obviously know how I should do things and I’d be a buffoon not to follow your sagely demands. How silly was I, a lowly working full time student, to doubt you! I’ll get to grinding out those study materials by hand in what precious little free time I have immediately! Executive dysfunction, ADHD, and depression be damned! My limited study time, already hampered by pre-existing conditions, would obviously be better spent fiddling with margins and bullet points on a word document and flashcard sets! The smug satisfaction and sense of superiority, knowing I painstakingly typed out every letter on those documents will surely yeild me the straight A’s I so desperately desire, but have yet failed to realize! Thank you, thank you, thank you FiniteBanjo, I don’t know where I’d be without you!!!
If you’re running it locally you can set how much variance it has. However, I mostly agree, in that it creates a bunch of trash. This doesn’t mean it has no use though. It’s like the monkeys on a typewriter thought experiment, but the monkey’s output is fairly constrained so it takes much fewer attempts to create what you want. It depends on the complexity of the solution required whether it’ll come up with a good solution in a reasonable amount of tries. If it’s a novel solution, it probably never will, because it’s constrained to solutions it’s seen before.
The high variance is why I only use it for dead simple tasks, e.g. “create and array of US states abbreviations in JavaScript”, otherwise I’m in full agreement with you. If you can’t verify the output is correct the it’s useless.
Wouldn’t that be slower to do, simply because checking it got all states, didn’t repeat any and didn’t make up any would be slower than copying a list from the web and quickly turning that into an array by hand with multiline cursors?
That’s like one web search and then one shell command. You can probably just copy paste a column of a table from wikipedia and then run a simple search/replace in your text editor. Why are you feeding the orphan crushing machine for this?
Also many people laugh at you if you try to say how ai is destroying the environment for no reason. Doesn’t affect them, you go live in a cave you luddite!
The simple stuff it can do, trying to remember how propublica used it, but it was just like sifting through a database and pulling out all mentions of a word.
When you get into giving case law, it’s way too complicated for it and it hallucinates.
I think the best use is “making filler” so like in a game, having some deep background shit that no one looks at, or making a fake advertisement in a cyberpunk type game. Something to fill the world out that reduces the work of real artists if they choose to
Fair enough, I’m trying pretty hard to devil’s advocate the “it has zero use” commentary. I hate the AI hype and LLMs getting shoved down our throats. I try a little to imagine a world where it’s somewhat helpful, cuz that type of tech would’ve been cool if it wasn’t a dystopian nightmare socially.
I mean long and short of it is Fuck AI and especially the people pushing it
Datasets are not the only mechanism to train AI. You can also use reinforcement learning. This requires you to have a good fitness function. In some domains, that is not a problem. For LLMs, however, we do not have such a function. However, we can use a hybrid approach, where we train a model based on a data set and optimizing for fitness functions that address part of what we want (e.g. avoiding em dashes). In practice, this tends to be tricky, as ML tends to be a bit too good at optimizing for fitness functions, and will often do it in ways you don’t want.
This is why if you want to develop a real AI product, you actually need AI engineers who know what they are doing; not prompt engineers who will try and find the magic incantation that makes someone else’s AI do what they want
You have to understand that their public facing product is not the same as the one they allow enterprise or state actors to use.
They benefit from public thinking they have these stupid limitations, gives them more space to curate their product offerings where the real money is made.
I don’t understand how the public thinking these are bad products is an incentive for especially state actors to use them. That seems counterintuitive.
Seeing as OpenAI struggled to make its AI avoid the em dash and still hasn’t entirely managed to do it, I’m not too worried.
i’m fine with LLM contributions to wikipedia as long as they have references and are human validated
it’s actually something that LLMs can potentially do quite well
TBF OpenAI are a bunch of idiots running the world’s largest ponzi scheme. If DeepMind tried it and failed then…
Well I still wouldn’t be surprised, but at least it would be worth citing.
I think the inherit issue is the current “AI” is inherently non-deterministic, so it’s impossible to fix these issues totally. You can feed am AI all the data on how to not sound AI, but you need massive amounts of non-AI writing to reinforce that. With AI being so prevalent nowadays you can’t guarantee a dataset nowadays is AI free, so you get the old “garbage in garbage out” problem that AI companies cannot solve. I still think generative AI has it’s place as a tool, I use it for quick and dirty text manipulation, but it’s being applied to every problem we have like it’s a magic silver bullet. I’m ranting at this point and I’m going to stop here.
FWIW, LLMs are deterministic. Usually the commercial front-ends don’t let you set the seed but behind the scenes the only reason the output changes each time it’s that the seed changes. If you set a fixed seed, input X always leads to output Y.
I honestly disagree that it has any use. Being a statistical model with high variance makes it a liability, no matter which task you use it for will produce worse results than a human being and will create new problems that didn’t exist before.
I use it to put together study guides so that, instead of spending a bunch of time typing and formatting, I’m spending it studying. It’s fed directly from my notes and slides and it rarely gets anything wrong (I read through it twice and cross reference with my notes). If anything, I’m usually removing stuff for being unnecessary or rewording things here and there to be.bstter suited to me. What took several hours now takes 30-45 minutes
Don’t take this as a defense of AI, it definitely isn’t. If AI disappeared tomorrow the world would be better off. Formatting study guides are literally the only utility I’ve found in LLMs
Any amount of using AI for learning is learning from Slop which will make you less informed.
Tell that to my 3.9 GPA lol. I’m not learning from slop, I’m using a program to format my notes and slides from my lectures, and then verifying that information before committing it to memory
Man have I got a solution for you, check out this formatted method to accomplish the same task:
My deepest apologies, sir. You’re clearly a man of deep intellect and wisdom beyond your years! You obviously know how I should do things and I’d be a buffoon not to follow your sagely demands. How silly was I, a lowly working full time student, to doubt you! I’ll get to grinding out those study materials by hand in what precious little free time I have immediately! Executive dysfunction, ADHD, and depression be damned! My limited study time, already hampered by pre-existing conditions, would obviously be better spent fiddling with margins and bullet points on a word document and flashcard sets! The smug satisfaction and sense of superiority, knowing I painstakingly typed out every letter on those documents will surely yeild me the straight A’s I so desperately desire, but have yet failed to realize! Thank you, thank you, thank you FiniteBanjo, I don’t know where I’d be without you!!!
If you’re running it locally you can set how much variance it has. However, I mostly agree, in that it creates a bunch of trash. This doesn’t mean it has no use though. It’s like the monkeys on a typewriter thought experiment, but the monkey’s output is fairly constrained so it takes much fewer attempts to create what you want. It depends on the complexity of the solution required whether it’ll come up with a good solution in a reasonable amount of tries. If it’s a novel solution, it probably never will, because it’s constrained to solutions it’s seen before.
The high variance is why I only use it for dead simple tasks, e.g. “create and array of US states abbreviations in JavaScript”, otherwise I’m in full agreement with you. If you can’t verify the output is correct the it’s useless.
Wouldn’t that be slower to do, simply because checking it got all states, didn’t repeat any and didn’t make up any would be slower than copying a list from the web and quickly turning that into an array by hand with multiline cursors?
Why would you have this use for multi-billion dollar earth scorching torment nexus?
That’s like one web search and then one shell command. You can probably just copy paste a column of a table from wikipedia and then run a simple search/replace in your text editor. Why are you feeding the orphan crushing machine for this?
Because its .01% easier to do this.
Also many people laugh at you if you try to say how ai is destroying the environment for no reason. Doesn’t affect them, you go live in a cave you luddite!
Ai is useful for sorting datasets amd pulling relevent info in some cases, ie propublica has used it for articles.
Obviously simple sorting for them, case law is too complicated for such sifting of data, it was trained on reddit after all.
And when, not if but when, it makes a mistake by pulling hallucinated info or data then it’s going to be your fault, that’s why it’s a liability.
The simple stuff it can do, trying to remember how propublica used it, but it was just like sifting through a database and pulling out all mentions of a word.
When you get into giving case law, it’s way too complicated for it and it hallucinates.
You mean keyword search that has existed since the beginning of time?
Idk there are legitimate uses of it sorting through large data sets that keyword searches do not fulfill.
I think the best use is “making filler” so like in a game, having some deep background shit that no one looks at, or making a fake advertisement in a cyberpunk type game. Something to fill the world out that reduces the work of real artists if they choose to
If you can’t be bothered to write filler then it’s an insult for you to expect others to read it. You’re just wasting people’s time.
I guess the point is for people to not read the filler.
I think of the text that’s too small to read on a computer in the background. It’s nice that it’s slightly more real looking than a copy/paste screen.
Not even close to worth destroying the environment over, but it’s a neat use case to me
Lorem ipsum has been used in typesetting since the 60s. If it’s not meant to be read, it doesn’t matter if it’s lorem ipsum text.
Not trying to dogpile you, I just think even things that seem ‘useful’ for LLMs almost always have preexisting solutions that are decades old.
Fair enough, I’m trying pretty hard to devil’s advocate the “it has zero use” commentary. I hate the AI hype and LLMs getting shoved down our throats. I try a little to imagine a world where it’s somewhat helpful, cuz that type of tech would’ve been cool if it wasn’t a dystopian nightmare socially.
I mean long and short of it is Fuck AI and especially the people pushing it
Datasets are not the only mechanism to train AI. You can also use reinforcement learning. This requires you to have a good fitness function. In some domains, that is not a problem. For LLMs, however, we do not have such a function. However, we can use a hybrid approach, where we train a model based on a data set and optimizing for fitness functions that address part of what we want (e.g. avoiding em dashes). In practice, this tends to be tricky, as ML tends to be a bit too good at optimizing for fitness functions, and will often do it in ways you don’t want. This is why if you want to develop a real AI product, you actually need AI engineers who know what they are doing; not prompt engineers who will try and find the magic incantation that makes someone else’s AI do what they want
We should crowdsource a program to sniff out ai data crawlers, then poison the data they harvest without them knowing, for companies to employ.
You have to understand that their public facing product is not the same as the one they allow enterprise or state actors to use.
They benefit from public thinking they have these stupid limitations, gives them more space to curate their product offerings where the real money is made.
I don’t understand how the public thinking these are bad products is an incentive for especially state actors to use them. That seems counterintuitive.