The study tracked around 800 developers, comparing their output with and without GitHub’s Copilot coding assistant over three-month periods. Surprisingly, when measuring key metrics like pull request cycle time and throughput, Uplevel found no meaningful improvements for those using Copilot.
It’s a glorified autocorrect. Using it for anything else and expecting magic is an interesting idea. I’m not sure what folks are expecting there.
- It suggests variables and function names I was already going to type more accurately. Better, it suggests ones even when I cannot remember the name because i got stumped trying to remember.
- It fills in basic control structures and functions more effectively than the IDE’s completion templates.
- It provides a decent starting point for both comments and documentation (the hard part). Meaning my code actually has comments and documentation. Regularly!
But I don’t ask it to explain things or generate algorithms willy nilly. I don’t expect or try to have it do something that’s not more than simply auto-completion.
I honestly like it, even if I strongly dislike the use of AI elsewhere. It’s working in this area for me.
I got a bridge to sell to anyone who thought AI would help reduce burnout lmao
No really, AI has great uses but I’m in awe anyone thought this was one of them
I basically exclusively use LLMs to explain broad concepts I’m unfamiliar with. a contrived example would be ‘what is a component in angular’ or ‘explain to a c# dev how x,y, and z work in rust’ The answers don’t need to be 100% accurate and they provide a nice general jumping point to get specific information.
Exactly, I’ve found them most useful to either summarize a text you feed it, or do broad ‘google like’ queries. I don’t trust it with anything beyond that
I use it sometimes to ask whether there’s an API for whatever I need to do. But it lies.
I do a lot of scripting for cloud infrastructure deployments and linux/windows basic scripting and the bing chat is great for banging out 5 liners in 1 second that would take me an hour even after multiple decades of being an admin.
Anything more complex it is useless for so it is limited but nice to have.
I’ve tried it for even some boiler plate code a few times. I’ve had to end up rewriting it every time.
It makes mistakes like Junior engineers, but it doesn’t make them in the same way that junior engineers do, which means that as a senior engineer it takes me significantly more effort to review. It also makes mistakes that humans don’t, which is even weirder to catch in review.
Also my experience. It sometimes tries to be smart and gets everything wrong.
I think code shows clearly, that LLMs don’t actually understand what’s written. Often enough you can clearly see it trying to insert a common pattern even though that doesn’t make sense at this point.
The few times I have used AI to help me with coding has mostly been to ask it for examples on how to use a specific feature, then it has been ok for the most part.
I mostly code in PowerShell, HTML and CSS, and Bing Chat helpful when I am stuck on a small issue.
We also recently started testing Copilot Pro 365, the one that can help you make documents or search through company documents and stuff like that.
As a test I asked it to make me a powerpoint presentation about the top ten podcasting microphones to buy.
The result looked great at first glance, but quicly got very generic.
Sure, it did show pictures of some microphones and even spoke about them, but it was just vauge and generic
