What projects make sense to work on if you believe AGI is coming in <5 years?

Triggered by listening to this podcast. This is a thought experiment, not necessarily something I believe. There is an implicit assumption in the premise that any AGI will both be available to me and steerable by me.

Gwern suggests the following rubric in the podcast:

Is this something I want to do regardless of any future AI, because I enjoy it?

Or is this something where I’m doing only the human part of it and the AGI can later on do it?

Or is this writing down something that is unwritten and thus helping the future AI versions of me?

Thinking through them one by one, and adding one my own:

Things I enjoy

I explored this in this blog post a little while ago. I previously built a simple pairwise ranking app for figuring out my “terminal desires,” and landed on the following:

Curiosity, trolling, and a desire to fulfill my potential/purpose. This is correct, though I would probably change “curiosity” to something more like “astonishment” or “wonder.”

For each these:

• Astonishment/wonder – I spend much of my time reading & learning, and my projects already tend to revolve around this
• Trolling – much to be done here, need to bring the Riley Walz energy I had in college back into my life
• Desire to fulfill my potential/purpose – this is the one I dislike the most about myself, and therefore probably where the most growth is; realistically I think this will be something related to education/children, with a microschool being my current planned manifestation

Doing only the human part

If you take seriously the idea of getting AGI in a few years, you don’t necessarily have to implement stuff and do it yourself. You can sketch out clearly what you want, and why it would be good and how to do it. And then just wait for the better AGI to come along and actually do it then. Unless there’s some really compelling reason to do it right now and pay that cost of your scarce time.

This idea has been at the edge of my awareness – one of the reflections from my Partition Summer comic project was “in an era of LLMs it’s sometimes better to just wait until it’s easy.” Looking back I can count a bunch of times where I tried to do something with LLMs (coding in 2024/early 2025, this comic, etc), stopped because it was annoying, and then when I tried again months later it was wayyy easier. But I realize now that I have been mentally scoping this point only to areas where I think AI will improve – verifiable domains like coding, imagen where there is a lot of energy being expended to improve it, etc.

What if you extend this idea out even further, the thought experiment of AGI being able to implement pretty much any clear proposal you give it? We are kind of there already for digital tasks, albeit with quite a bit of steering required during the implementation.

This points to what Gwern suggests here, things like:

• Thinking carefully and sketching out a clear proposal – what you want, why it would be good, and how to do it. Don’t implement unless there’s a really good reason to do it now
• Focusing on asking precise interesting questions rather than necessarily figuring out the answer. This is what “research taste” and good science already is

I do have a bunch of projects on my list where I suspect writing up proposals rather than implementing is the more valuable use of my time.

But I can think of a couple counterpoints to this view too:

1. For many projects I learn what I want through the process of working on the project. In fact this may be the definition of what makes a project interesting in the first place – it’s unpredictable how it will change through working on it
2. Not really a counter, but IMO this should be superseded by the previous point of doing things because you enjoy. Sometimes I just wanna implement something cause it’s fun to do so (or at least fun to steer an LLM to bring a vision to life)

So perhaps a reasonable takeaway here is – spend more time on upfront thinking; only implement if it’s urgent, fun, or the project will grow creatively through implementation.

This is good advice regardless of AI.

Writing down something that is unwritten to help future AI versions of me

By writing, you are voting on the future of the Shoggoth using one of the few currencies it acknowledges: tokens it has to predict. If you aren’t writing, you are abdicating the future or your role in it…

There are ways to influence the Shoggoth more, but not many. If you don’t already occupy a handful of key roles or work at a frontier lab, your influence rounds off to 0, far more than ever before. If there are values you have which are not expressed yet in text, if there are things you like or want, if they aren’t reflected online, then to the AI they don’t exist. That is dangerously close to won’t exist.

But yes, you are also creating a sort of immortality for yourself personally. You aren’t just creating a persona, you are creating your future self too. What self are you showing the LLMs, and how will they treat you in the future?

This one was kind of new to me. The notion that expressing otherwise unwritten preferences could help future AI versions of you is interesting. I’m not sure I completely buy the idea that you can meaningfully steer the AI in this way, but regardless I already (privately) journal, so a lot of this will be available to any AI system in the future should I choose to share it.

~Anything relational

Hanging out with friends/family, throwing parties, making friends

This is obvious but I can certainly do more of this