I ran my agents from my phone for two weeks
The last two weeks of my life have been quite transient. More so than the time I hopped a plane to San Francisco, interviewed for 5 hours, and hopped back on a plane back to Florida the same evening.
The trek started in Texas with a road trip that passed through 11 states, ending up in Rhode Island where my daughter is attending college. The weather was beautiful, but only for a moment as the next leg included a flight down to North Carolina to give Charlotte a test run.
As a newly minted empty nest couple, it's time for a change. Austin was fantastic when we moved there, but since the COVID-19 pandemic and the great migration of folks from California to Austin, the town feels like a shell of what it used to be.
I refer to it as "The Formerly Weird City of Austin" (trademark pending).
I'll talk more about Charlotte in another post, but the early feeling is that it feels more like the Austin we moved to than the one we'll be leaving.
My intentionally limited technical stack
So, this whirlwind trip felt like a good time to try to stay away from the computer as much as possible. That didn't mean I wasn't going to try to eke out some productivity.
My on the go technical stack included my iPhone 17 Pro and the Claude Code iOS app connected to a remote session back at my house for the majority of everything I was able to get knocked out.
Laptop was with me, and it made a brief appearance so I could write a couple of blog posts. There was also a small snafu with my remote session that warranted an actual keyboard (I'll touch on that in a bit), and once more to watch some late night Impractical Jokers, as that tends to be the thing we watch when vacationing.
In addition to my thin tech stack, I did have one north star: don't walk back any of the autonomy we already have in place. I am working towards a system that needs me less and not more.
Only one issue, but it was a doozy
Generally speaking, everything "worked" just as you'd expect.
Talk to the harness to start the game loop, the agent does the rest until the end. That was, until I hopped in to check on a session, and Claude Code said it was time for me to log back into my session.
A remote session. On a computer in my house. On a network that wasn't set up for this level of remote access. Ouch.
I definitely wasn't going to try to futz with an SSH client on my phone and all of that, so the laptop made a brief appearance.
Administration for the Eero mesh network routers is done via an app, and I was fortunate that I did already have Dynamic DNS set up. A few configuration tweaks later I was (albeit slowly) connected to the homelab server that houses my harness and agent.
It took a few tries, but I was eventually able to get the Claude Code session authenticated again, and remote access re-enabled.
After that, I undid some of the configuration changes as I don't like having my home network that open.
What allowed this to work
This isn't an opinion piece on how you can get by without carefully crafted prompts. In fact, this whole experiment generated some new safeguards to help with the fact that I wasn't carefully crafting much of anything.
Everything that got done was vibed in the worst sense of the term. I talked to my remote harness, I gave decent enough direction. We'd volley a few times, but usually just landed on "yeah do that" and hoped for the best.
This was all intentional, and while I love hacking my agent and context library, it's not the most mission critical thing I maintain. If we borked something, we'd recover. None of this is load bearing with customers that would notice if and when my "LGTM" vibes were worse for wear.
Given the nature of how I was driving the harness, trying to write well crafted prompts with a phone keyboard is a pipe dream. The harness as an intermediary between my ideas and the specification that landed in the tickets cut out a lot of my need for a keyboard.
Even if we weren't achieving one-shot status on the tickets.
Fighting with the robots
Having an LLM one-shot a prompt into 100% perfect everything is a goal I think many of us strive for. Meanwhile in reality, it's a nice to have the few times it does happen.
With non-deterministic systems today's one-shot may be tomorrow's nightmare fuel. And vice versa of course.
Not a big deal, but where I was really starting to feel the burden was in how quick tickets were being filed and worked. I like the agent to move quickly, but the harness was a bit trigger happy to file tickets before we were done fleshing out things.
Worse was when the harness would notice something after the agent was already working on it.
That part felt like it would be worth sitting down and reviewing a bunch of the prompts and context files. LLMs being overzealous and quick to act isn't new, but the way it was running background heuristics after filing a ticket isn't how it should be operating.
That part felt like it would need me at the keyboard once we're back home.
Guardrails all the way down
The nice part of running into issues with a harness or an agent, is that you can give feedback, and hopefully get better results. The best case scenario is bolting on more checks and gates to help ensure you identify and fail if and when they are encountered again.
The issue with the tickets being filed too quickly and being picked up before being fully fleshed out did result in a stern talking-to (or three). Ultimately we added a bit of a delay to the agent to not pick up tickets right away.
This delay allows for fresh tickets to get skipped over, rather than worked. Working well-aged tickets means the ticket should be in better shape. Slower to pick up the ticket, but faster overall because we're not wasting time on something that's half-baked.
Even if "code is free" now, I'd prefer to not waste time on it and then throw it away moments after generating it.
The travel game loop
The irony of this trip is that I didn't even bother to bring a retro game console, as I typically do on vacation. Instead, I got to grind on a game loop of daily maintenance tasks, followed by endgame activities around improving my agent and context library.
The constraints of having such a limited setup yielded quite a few different pain points. Each pain point turned into an action item. Each action item fixed. Repeat ad nauseam.
Other improvements included fixing up issues with how the PR review loop worked. This is a part of the system that has a tendency to get bottlenecked by me, which is the thing I've been trying to optimize away from. Turns out there was a directive that was resulting in reviews that were somewhat wishy-washy with a poor assumption that my time was cheap to invoke.
The fix there was to do a better job of classifying the findings, and actioning based on those tangible results. This landed more PRs to me ready for a quick approval, and if not, with specific action items for me to identify.
The context library got quite a bit of love, as I continue to work towards a more robust plugin-like system there. This part has been my biggest concern, and a lot of pre-work has been done to get ready for starting to version context rather than always pulling from the default branch.
5 stars, would do again
Not like every day or anything, but the process was good enough to consider it as a viable way to accomplish trivial work on side projects.
Even as I use agents any and everywhere, I still remain heavy handed when it comes to production systems that have customers that would be impacted by issues.
For my friendly agent, at the current scale of things, it's nice to experiment with how things may end up in the future.
That's assuming we're not on the Judgment Day timeline, of course.
:wq
Like this drivel? There's a whole RSS feed of it, or subscribe via email.