i tried out github copilot agent mode (using GPT 4.1 [preview]) today in vscode (sorry folks, i only use vim if i'm editing a single file). here are some thoughts:
it's scary. it will just kind of overwrite things willy nilly, and although you can ‘undo’ its changes it's difficult to (a) know if it's got rid of anything important and (b) have fine-grained undo-ment.
the UI for viewing changes it has made just seems a bit broken sometimes - i found myself in a loop of pressing ‘undo’ but each time it would append an extra line without actually doing anything, so i had to completely abort that iteration - and certainlt isn't intuitive to use (for me)
it does a little bit of hallucination (insert troll face emoji). for me it seemed to like changing one particular closing
)to a}, change one particular snake_case identifier to camelCase in one particular place, and consistently make up the incorrect parameters t one particular function. this of course isn't a massive issue because i actually check to see what it's done, and fix the stupid bits. alas, when it next writes to that file on a subsequent iteration it'll go ahead and overwrite my fixes, even when asked not to.however, on the whole i was impressed with its comprehension of what was going on in a fairly large (~7k LoC) codebase, especially once i gave it clearer instructions on what to do. it was also more manageable if i asked it to explain what changes it would make, then ask for permission before making those changes
using it probably saved me a little bit of time and tediousness, though I'm not sure it was worth it for what I did. It was, however, far superior to just using the copilot chat sidebar (which I tried once before and regretted)
it's also quite good at naming variables, which is nice.
i was very surprised at how few ‘chat credits' it used up. but maybe that's a bug. if it is, don't tell microsoft!
I'm not going to become a vibe coder any time soon, nor do I plan on using copilot often, but I get the sense that this sort of technology may now be becoming useful for helping out with long/tedious/monotonous tasks, under very careful supervision!