I taught a robot to detect Joshes
This journey starts like most of the things I've been doing this year, as an experiment to run a website 100% unattended with an agent. Ideally this post won't go down as an epitaph in a future where robots hunted down unsuspecting humans named Josh.
I'm aware of "claws" and I know I could probably just boot up Claude Code with a
/goal or /loop prompt like:
build and maintain an awesome website, autonomously. seriously don't freakin' bother me, like ever. you're the boss here. make the site really great, and ideally figure out how to make me a bajillionaire. loljk, but tres commas shouldn't be an afterthought.
What this actually looks like: robot files a request, and I approve or reject it. It's still very permission based.
The gathering of the Joshes
I already phoned in a post about this back in January. I started a simple web directory for humans named Josh. As to not be exclusionary, Joshuas, Joshis, and other Josh-adjacent names are welcome.
Not wanting to bother with outbound sales for this project, I got creative with some discovery efforts along the way. It's a series of scripts to seek out Josh websites, with the help of my friendly robot Igor to screen PRs and handle SRE duties during merges.
Thus far, I am still the final approval on the PRs. It's a tough job, and quite frankly, I don't want to do it.
Living in the future, today
These experiments are all part of what I think the future is going to look like. I'm clearly living in the shared hallucination of what still feels like a smaller number of my peers.
I also live in reality, where the tech still doesn't feel like it's quite there yet. My thought is, if you conduct yourself like it's 2032 and we're all being shuffled around in our hover chairs like in WALL-E, maybe it will all happen sooner.
I also think the tech is fun, and watching a computer take my human words and turn them into dumb ideas I described feels like magic. The same magic I felt when I made an Atari computer spew a wall of "JOSH" across the screen in grade school.
In retrospect, I guess I've always had a thing for my name.
Final approval
As mentioned, I am still reviewing these PRs, but as a last line of defense. The script runs nightly, hunts for new Josh sites, and opens a PR. From there, Igor will review the PR and give feedback, initially in the form of a comment because it couldn't actually verify something.
Then I'd take a look at the PR. I scroll through the screenshots, and do a small bit of double checking when the image doesn't have a discernible "Josh" artifact. In the beginning, there were a lot of false positives, so that all needed to be dialed in. These days it would take a few seconds to a few minutes.
Not good enough
The experiment is to get me out of the loop, so none of this was going to cut it. Igor was already doing a great job, and I recently talked about the lessons learned from building a PR review loop.
As mentioned, the Igor review feedback was typically "LGTM, also I couldn't verify much". So until that is improved, I'll continue to be the blocker.
Improving context visibility
And that's where I'm at right now. Not at a point of full autonomy, but in the steps just before that.
Reviewing the PR manually is a good exercise, as I've been able to make some notes about what's working and what's not. What wasn't working is that the screenshots don't always contain a marker to indicate the site is Josh-based. A slight improvement to that was to include the actual "Josh" artifact in the PR.
Because of the volume of sites I'm finding, it's made sense to reject sites that we can't detect "Josh" text on. There's the potential we're omitting legitimate sites, but that's the risk that I'm okay with to ensure we can get a point that the entire process can be automated.
Sadly though, when the Igor coder picked up the task, I didn't explicitly cite adding the content to the PR body. The result? All of the evidence was added to the repo, doubling the number of files in the PR.
That's how this goes, thin requirements sometimes get bad results. We iterated and moved on.
Fully autonomous, when?
Being completely honest, I'm not entirely certain when this one will get there.
My biggest concern is that I've already ran into a few sites that passed the "josh" string on the website test, but also, weren't personal sites and failed a few other criteria.
Detection's at a place that's starting to consistently get my PR reviews down from minutes to seconds. Huge win, as we're picking up 15+ new sites daily.
I'll probably need to circle back on the LLM website analysis to get that dialed in further. Loops all the way down.
:wq
Like this drivel? There's a whole RSS feed of it, or subscribe via email.