Lessons Learned from Vibe-Coding a Configuration Parser

I vibe-coded something the other day. And not something temporary – this is a library I want to use in production environments. The process got me thinking about where the real value – and the real work – of functional code lies.

Years ago, I came up with a syntax for configuring a “pipelined” software process. It was a way to specify “commands” and pass “arguments” to them in plain text.

It looks like this:

doThisThing  -argument:value
thenDoThis
andThenThis  -argument1:value -argument2:value

I know, I know – this is not at all original. It looks a lot like any shell, and that’s clearly what I emulated. (I was doing a fair amount of Powershell at the time, I remember.)

Back then, I wrote a parser for this in C#, for the Denina project. It would take in the text above, and return a configuration object with a clean set of objects to represent the commands and arguments. I could then use that in whatever software process I was managing.

Recently, I had been thinking about a new application for it. I wanted a version of it in JavaScript, along with some enhancements.

…but I just did not want to write this. Parsers are notoriously tedious to write, and it usually devolves to a lot of debugging. Consequently, I was procrastinating.

(You might be looking at the above example and thinking, “But that looks so simple…” Well, read the spec linked below. There’s a lot going on there you can’t see.)

So, I figured this was a good scenario for vibe-coding. This is a pure code problem – there’s no UI in this case. Additionally, this is easily testable – the text turns into an object, and it’s pretty clear if that object is configured the way it’s supposed to be.

I’ve vibe-coded some stuff before, but this was a little different because it wasn’t meant to be disposable, throwaway code. I wanted to keep this for use in production environments.

I decided that I just need to specify very precisely how this was supposed to work, and make sure there was as little room for interpretation as possible.

I decided to write a “spec” for the language. I figured this would be the safest way to explain to AI how it works. This wasn’t meant to be a formal spec (a la PEGN, or whatever), but just a very descriptive explanation of how it works that could double as documentation.

And here it is –

Configuration Language: Specification

That’s 2,000 words and dozens of examples. It took about 90 minutes to write the first draft.

I figured it was in great shape, but just to be sure, I uploaded it to ChatGPT with this prompt:

Read this and let me know if anything is vague, contradictory, or otherwise doesn’t make sense.

It came back with 27 problems.

Some of them were nitpicky, which annoyed me… but that doesn’t matter. If I was asking to AI to write this, then it only mattered that AI understood it. In this case, lack of context was a benefit – AI was coming at it with no prior knowledge, while this idea had been kicking around in my human brain for a decade.

Besides the nitpicking, some other issues were quite valid. Some changes were clarifications to the doc to better explain what was in my head, but others were actual, meaningful decisions I hadn’t thought of, but had to come to a decision on.

For example:

someCommand -argument
someCommand -argument:

Does the trailing colon on an argument matter? I had initially stated in the spec that an argument without a value got a default of “true” (as if to say, “this argument exists”). However, if I end an argument name with a colon as if I’m going to supply a value, but then I don’t… is that the same thing?

No, I decided. If you end with a colon, then that’s an empty string value. I would have never thought to specify this, and having AI bring it up evolved the language and covered an edge case.

I went through this same process dozens of times for other edge cases I hadn’t thought of.

Once I had finished editing that first round of complaints, I told ChatGPT to forget the prior conversation and sent it the text again with the same prompt. I don’t remember what it came back with, but it was damn-near the same number of problems.

I spent an entire morning like this – working on this specification document “alongside” AI. I would edit, it would nit-pick some more, and so on and so on.

I finally got it down to complaining about a single-digit number of really weird problems, and decided an AI could finally take a run at writing it.

I used Claude in Visual Studio Code in agent mode. I told it:

Read this document and create that functionality in a single-file JavaScript module with no external dependencies. Export all classes. Write unit tests to ensure proper implementation.

It took 14 minutes, and I’m happy to report that the results are stellar. It wrote 45 unit tests (all of which passed, after a couple attempts and edits). Performance is exceptional: a fairly complicated chuck of CL parses into an object in about half a millisecond.

I looked through the code a bit. From what I can see, it’s not particularly clever – it’s what you would call a “brute force” parser. It goes line by line and does string spitting to divide everything up. Lots of conditionals and loops. At the risk of sounding defensive, I easily could have written this myself.

I asked for a couple changes – nothing that would affect the spec or the underlying language, but rather just some preferences for the JavaScript implementation. And I changed the name of one construct after some second thoughts. (I had to tell it to unit test the changes. I think it’s up to 49 tests now.)

I’ve been using the resulting code for a week and it’s faded into the background in the sense that it “just works.” Performance is great, and it’s never made a mistake that I’ve detected, so it’s become a lovely black box that I don’t have to care about.

So, what did I learn from this, and what existing notions did it reinforce?

• Vibe-coding is really good when you have a tightly defined use case with objective results that are easily unit-tested.

• It’s helpful to write documentation in advance and use it for context. I basically documented non-existent software, then just told AI to write some actual software to match what I wrote.

• It helps to have very clear demarcation lines around the code. What AI wrote was really modular (I literally told it to write a module). Nothing in here will “leak” into other code. This matches how I use AI to assist when coding: I always ask it for “a pure function with no side-effects or dependencies to do …X.” That keeps AI-generated code tightly confined.

• Did vibe-coding this save me time? …I don’t know. I probably had 4-5 hours total into this. Could I have written the parser in that amount of time with the same performance and quality? …maybe?

• Get second opinions – AI and otherwise. Fight battles with the idea before code gets written. If you assume the code will be fine, what are the other questions that need to be answered?

• It helps to have a clear plan. I’ve spent a decade “with” this language in another project. I knew it worked, so I had an evolved (or so I thought) mental model of what I wanted.

And this last point perhaps gets a little philosophical.

What was the main thread of work in writing a JavaScript parser for my configuration language? Was it writing the actual code? Or was it deciding what problem I was trying to solve, how I wanted it solved, and all the different ways it might go wrong?

Ankur Goyal has borrowed the term “async programming” to describe this:

The shift is subtle but powerful: instead of writing code line by line, we’re learning to describe problems clearly and let tools solve them in the background.

[…] AI isn’t replacing programming, but the most valuable parts of programming are becoming more prominent while routine tasks move to the background.

(And I did, in fact, let AI solve my problem “in the background.” Once Claude started running, I think I set my laptop down and went to get breakfast.)

Honestly, writing the actual code of this parser just wasn’t that important. It really didn’t involve a lot decisions – I even told Claude not to use any dependencies, so there weren’t many questions about architecture (it’s not going to write a parser combinator library from scratch, so brute-force was the only real option).

This reminds me of the book How Big Things Get Done: The Surprising Factors That Determine the Fate of Every Project, from Home Renovations to Space Exploration and Everything In Between. It’s written by a guy who has spent his career studying large-scale construction projects like dams and nuclear power plants.

One of the core ideas of the book is: think slow, move fast. Few projects ever fail from lack of planning. The book encourages lots planning, edge case speculation, proofs-of-concept, and “red team” analysis. That’s where the problems get identified and the core idea and solution starts to emerge and get refined.

And that’s what we’re trying to get to: the core nature of the problem and the solution. We’re trying to solve problems, not write code. The code is just the solution vector. I’m not trying to trivialize it by any stretch, but the real obstacle was not what ended up being just 395 lines of JavaScript – it was the problem definition and the decision of how to solve it.