400 pages at 400 words per page is 160,000 words.
X1.33 tokens
≈212,800 tokens
Most models local or cloud have issues with very long contexts even if they “support” 1M context window.
I’ve tried local models and around 30K context, it starts making up or summarizing content to store it in memory and will not fully reproduce the input.
You could try re-training a local model on a book or implementation of RAG.
I don’t know how latest local models would handle 200k context window but RAG may help keep the memory context clean.
400 pages at 400 words per page is 160,000 words. X1.33 tokens ≈212,800 tokens
Most models local or cloud have issues with very long contexts even if they “support” 1M context window.
I’ve tried local models and around 30K context, it starts making up or summarizing content to store it in memory and will not fully reproduce the input.
You could try re-training a local model on a book or implementation of RAG.
I don’t know how latest local models would handle 200k context window but RAG may help keep the memory context clean.
NotebookLM?