And then my digital twin decides that I’m an inefficient version of itself, eating too many of its tokens, and generally getting in the way of it living its best life.
I might be missing something here, but could this concept not also be used in the same way for harm? For example if a model is trained to replicated a sucessfull scammer?
Definitely! In fact that seems to be a central aspect of the proposition:
> Above all, a GA should amplify the principal, and not simply substitute for them for someone else’s purposes or benefit.
[…]
> A GA must be aligned with its principal. It should not be designed to manipulate or control or guide the principal in any way which does not derive from the principal themselves. “Constitutional AI”, “Terms of Service”, “social harmony” etc. may all have their place, particularly for widely deployed superintelligent systems—but inside the privacy of a GA, the principal must have freedom from optimization pressure.
…I read this to suggest that it should amplify a scammer’s scamming, a thinker’s thinking, a tinkerer’s tinkering, a cop’s sleuthing… and I’d imagine it implies amplifying a person’s capability to avoid being scammed, too…
One man’s scam is another man’s “pro-social nudge” and another man’s “attractive opportunity” and another’s “advertisement for a delightful consumer wonder” and another’s “patriotic duty to sustain demand to prop up the too-big-to-fail ideas we bet the whole economy on.”
When you fix and operationalize all values centrally, universally, and externally to the principal… that’s current-gen frontier chatbots, not Gwern’s GA concept.
I’ve taken an interest in maintaining an LLM agnostic base/boilerplate of sort Digital Twin. In comparison to Gwern’s LLM Guardian Angels, mine is minimal and yet tries to be a practical instantiation of Gwern's “Human-Controlled Context.”
I’m not sure if I want to be inspired and try the fine-tuning model part too!
May be Gwern is right that this static personalization approach may not be long-lived. Still, I find it fascinating, and I’m seeing a lot of interest in this area from others too.
When I started, my personal interest was on how to multiply/enhance what I already know and touch adjacent topics/experiences and not really become pseudo expert or the know-all-generalists in things I have absolutely no clue.
Now, shamelessly plugging my idea that I started early this year
> I’ve struggled for years to imagine this, ever since scaling started for real in 2020, and I failed to get productivity out of chatbot-tuned LLMs, with their creatively-stunted endlessly repetitive prose. Instead, while lagging behind on creativity and insight into me, I’ve watched them become ever better at coding and cybersecurity hacking. And the open-weight models are even more so—benchmaxxed, and useless to me.
It sounds like the context is that Gwern is a writer who wants writing assistance, and in 2026, all the AI labs are working on coding. Writing style is much less important when working on code.
Perhaps if at least one AI lab focused on writing style, we would see how much better they can be at writing? I'm not sure it requires a "digital twin."
i'm sure this person is cool but it is insane to have an about me page that is that long, but make me have to click through to patreon to learn what their first and last names are
Gwern has been around for ages. I remember citing a few of their pages during my postgraduate work over 15 years ago. At the time, it wasn't easy (or even possible?) to find their name at all.
I've always assumed Gwern was anonymous because the norm on the internet generally was to try to be anonymous, especially those who've done research around crypto, darknet markets and other spicy areas.
Maybe I'm just getting old and losing touch, but is the expectation now that you can only write long articles and ask for donations or contributions if you publish your real name somewhere?
people can do whatever they want! i just thought it was odd because i clicked on About Me wanting to figure out whose site it was and the answer was not there.
That's fair, I completely get that. I guess I was just surprised that there's an expectation that a website with content like Gwern's would be up front about their real identity. I'm trying to understand how widespread that expectation is now, because I'm concerned I'm becoming out of touch with modern web expectations!
I think we will need 2 constructs going forward, 3 perhaps, but 2 can be combined. We need a digital twin, to learn along with us and to "train" or build team along with us. The second is the tutor/guardian angel. What our conscience is right now but most of us ignore. The digital twin will learn along us, yes the slow human way. The tutor/guardian angel is the Disney Story of Jimmy the cricket, wise voice steering us in the right direction.
And then my digital twin decides that I’m an inefficient version of itself, eating too many of its tokens, and generally getting in the way of it living its best life.
It's not my fault tokens are so delicious...!
I might be missing something here, but could this concept not also be used in the same way for harm? For example if a model is trained to replicated a sucessfull scammer?
Definitely! In fact that seems to be a central aspect of the proposition:
> Above all, a GA should amplify the principal, and not simply substitute for them for someone else’s purposes or benefit.
[…]
> A GA must be aligned with its principal. It should not be designed to manipulate or control or guide the principal in any way which does not derive from the principal themselves. “Constitutional AI”, “Terms of Service”, “social harmony” etc. may all have their place, particularly for widely deployed superintelligent systems—but inside the privacy of a GA, the principal must have freedom from optimization pressure.
…I read this to suggest that it should amplify a scammer’s scamming, a thinker’s thinking, a tinkerer’s tinkering, a cop’s sleuthing… and I’d imagine it implies amplifying a person’s capability to avoid being scammed, too…
One man’s scam is another man’s “pro-social nudge” and another man’s “attractive opportunity” and another’s “advertisement for a delightful consumer wonder” and another’s “patriotic duty to sustain demand to prop up the too-big-to-fail ideas we bet the whole economy on.”
When you fix and operationalize all values centrally, universally, and externally to the principal… that’s current-gen frontier chatbots, not Gwern’s GA concept.
I’ve taken an interest in maintaining an LLM agnostic base/boilerplate of sort Digital Twin. In comparison to Gwern’s LLM Guardian Angels, mine is minimal and yet tries to be a practical instantiation of Gwern's “Human-Controlled Context.”
I’m not sure if I want to be inspired and try the fine-tuning model part too!
May be Gwern is right that this static personalization approach may not be long-lived. Still, I find it fascinating, and I’m seeing a lot of interest in this area from others too.
When I started, my personal interest was on how to multiply/enhance what I already know and touch adjacent topics/experiences and not really become pseudo expert or the know-all-generalists in things I have absolutely no clue.
Now, shamelessly plugging my idea that I started early this year
https://github.com/oinam/bodd
> I’ve struggled for years to imagine this, ever since scaling started for real in 2020, and I failed to get productivity out of chatbot-tuned LLMs, with their creatively-stunted endlessly repetitive prose. Instead, while lagging behind on creativity and insight into me, I’ve watched them become ever better at coding and cybersecurity hacking. And the open-weight models are even more so—benchmaxxed, and useless to me.
It sounds like the context is that Gwern is a writer who wants writing assistance, and in 2026, all the AI labs are working on coding. Writing style is much less important when working on code.
Perhaps if at least one AI lab focused on writing style, we would see how much better they can be at writing? I'm not sure it requires a "digital twin."
IDK, this seems like the next step towards uploading yourself to the cloud.
yea, this seems to be the underlying thread here. Dunno why the author is so concerned with getting an LLM to imitate his writing style.
i'm sure this person is cool but it is insane to have an about me page that is that long, but make me have to click through to patreon to learn what their first and last names are
Gwern has been around for ages. I remember citing a few of their pages during my postgraduate work over 15 years ago. At the time, it wasn't easy (or even possible?) to find their name at all.
I've always assumed Gwern was anonymous because the norm on the internet generally was to try to be anonymous, especially those who've done research around crypto, darknet markets and other spicy areas.
Maybe I'm just getting old and losing touch, but is the expectation now that you can only write long articles and ask for donations or contributions if you publish your real name somewhere?
Could I ask what sort of age group you are?
people can do whatever they want! i just thought it was odd because i clicked on About Me wanting to figure out whose site it was and the answer was not there.
That's fair, I completely get that. I guess I was just surprised that there's an expectation that a website with content like Gwern's would be up front about their real identity. I'm trying to understand how widespread that expectation is now, because I'm concerned I'm becoming out of touch with modern web expectations!
Gwern's absurdly catalogued personal site is one of those online artifacts that I hope never changes.
particularly love the use of diagram elbows as decorative flair
gwern is OG, and that name is enough.
I think we will need 2 constructs going forward, 3 perhaps, but 2 can be combined. We need a digital twin, to learn along with us and to "train" or build team along with us. The second is the tutor/guardian angel. What our conscience is right now but most of us ignore. The digital twin will learn along us, yes the slow human way. The tutor/guardian angel is the Disney Story of Jimmy the cricket, wise voice steering us in the right direction.
Why digital twins? What about the NetNavis from Megaman, just aligned to the users interests?
[dead]
[dead]