> As a last step, the final transformer layers produce numbers called logits. From these logits, the transformer conjures a list of possible words and assigns each a probability. And here is where SynthID diverges. Your friendly transformer rolls the dice and selects the next word from the list for output. But not your transformer with SynthID, as it plays by different rules. With SynthID, the game is rigged. SynthID loads the dice, changing probabilities in subtle ways. And it does it not to win any game but to create a signature of the generated text hidden in plain sight.
This may be a possible feature of current LLMs, but it will lose feasibility when we will achieve deterministic output (as duly). Duly output is the best possible reply, not a "good enough" reply.
This article assumes that Anthropic's watermarks are statistical word choices, not more falsehoods. If Anthropic's advanced models are as good as they say, the method of watermarking might be beyond the understanding of baseline humans. The given explanation might amount to no more than the AI having a gut feel some text is watermarked, because the real reason is not explainable to entities with small working memories, cognitive biases, and 5+/-2 item short term memories.
The article also ignores that Anthropic, as a hard nosed business with fiduciary obligations, has incentives to watermark beyond compliance with mere human laws. Anthropic just might be trying to prevent Model Collapse by weeding out AI generated text. Those of us who depend on AI generation should applaud watermarking. Only Anthropic can prevent Model Collapse.
> As a last step, the final transformer layers produce numbers called logits. From these logits, the transformer conjures a list of possible words and assigns each a probability. And here is where SynthID diverges. Your friendly transformer rolls the dice and selects the next word from the list for output. But not your transformer with SynthID, as it plays by different rules. With SynthID, the game is rigged. SynthID loads the dice, changing probabilities in subtle ways. And it does it not to win any game but to create a signature of the generated text hidden in plain sight.
This may be a possible feature of current LLMs, but it will lose feasibility when we will achieve deterministic output (as duly). Duly output is the best possible reply, not a "good enough" reply.
This article assumes that Anthropic's watermarks are statistical word choices, not more falsehoods. If Anthropic's advanced models are as good as they say, the method of watermarking might be beyond the understanding of baseline humans. The given explanation might amount to no more than the AI having a gut feel some text is watermarked, because the real reason is not explainable to entities with small working memories, cognitive biases, and 5+/-2 item short term memories.
The article also ignores that Anthropic, as a hard nosed business with fiduciary obligations, has incentives to watermark beyond compliance with mere human laws. Anthropic just might be trying to prevent Model Collapse by weeding out AI generated text. Those of us who depend on AI generation should applaud watermarking. Only Anthropic can prevent Model Collapse.
[dead]