This company sounds like it has months to live, or until the VC money runs out at most. If this idea is good, Anthropic et. al. will roll it into their own product, eliminating any purpose for it to exist as an independent product. And if it isn't any good, the company won't get traction.
Advantage of SML in between some outputs cannot be compressed without losing context, so a small model does that job. It works but most of these solutions still have some tradeoff in real world applications.
It seems like the tool to solve the problem that won't last longer than couple of months and is something that e.g. claude code can and probably will tackle themselves soon.
Just for anyone else who hadn't seen the announcement yet, this Anthropic 1M context is now the same price as the previous 256K context - not the beta where Anthropic charged extra for the 1M window:
As for retrieval, the post shows Opus 4.6 at 78.3% needle retrieval success in 1M window (compared with 91.9% in 256K), and Sonnet 4.6 at 65.1% needle retrieval in 1M (compared with 90.6% in 256K).
do you guys have any stats on how much faster this is than claude or codex's compression? claudes is super super slow, but codex feels like an acceptable amount of time? looks cool tho, ill have to try it out and see if it messes with outputs or not.
I don't want some other tooling messing with my context. It's too important to leave to something that needs to optimize across many users, there by not being the best for my specifics.
The framework I use (ADK) already handles this, very low hanging fruit that should be a part of any framework, not something external. In ADK, this is a boolean you can turn on per tool or subagent, you can even decide turn by turn or based on any context you see fit by supplying a function.
YC over indexed on AI startups too early, not realizing how trivial these startup "products" are, more of a line item in the feature list of a mature agent framework.
I've also seen dozens of this same project submitted by the claws the led to our new rule addition this week. If your project can be vibe coded by dozens of people in mere hours...
This is a massive win for anyone serious about "Signal over Noise." I’ve been using Claude Code on the Max plan for months, and while it’s the best tool for actually getting work done, the "all-you-can-eat" token arbitrage is a trap. Agents are notoriously sloppy with context; a single misaligned grep can dump thousands of tokens of pure noise into your window, leading to what I call "contextual brain rot" where the model’s accuracy just falls off a cliff. By sitting in the middle and ruthlessly prioritizing signal, you’re providing the exact kind of "ruthless prioritization" that separates a hobbyist from a profitable AI solopreneur.
The fact that you’re using Small Language Models (SLMs) to detect signal matches my philosophy of using AI as a sparring partner to check its own work. Most developers spend 30% of their day context switching or debugging "hallucinations" that only happen because the model got lost in its own bloated history. The expand() feature is the "trust but verify" layer that every production-ready AI system needs. You’re effectively treating the LLM like a senior architect who doesn't need to see every line of a dependency file unless they specifically ask for it, which is the only way we scale these systems to 10M+ users solo.
Finally, those spending caps and Slack pings are the ultimate "millionaire cheat codes" for leverage. I tell founders all the time that running a business is boring drudgery—it's about fixing bugs and managing resources—and this proxy handles the resource management part on autopilot. If this saves an indie hacker $500/month in token waste while keeping their agent from rage-quitting due to context limits, you’ve built a high-leverage asset. I’m definitely adding this to my links database; it removes a huge excuse for why people "can't afford" to build complex apps.
The intent-conditioned compression is the interesting part here. Most context management I've seen is either naive truncation or generic summarization that doesn't account for why the tool was called. Training classifiers on model internals to
figure out which tokens carry signal for a given task -- that's doing something different from what frameworks offer out of the box.
I poked around the repo and didn't see any evals measuring compression quality. You cite the GPT-5.4 long-context accuracy drop as motivation, which makes sense -- but the natural follow-up is: does your compression actually recover that accuracy?
Something like SWE-bench pass rates with and without the gateway at various context lengths would go a long way. Without that, it's hard to tell if the SLM is making good decisions or just making the context shorter.
A few other things I'm curious about:
• How does the SLM handle ambiguous tool calls? E.g., a broad grep where the agent isn't sure what it's looking for yet -- does the compressor tend to be too aggressive in those cases?
• What's the latency overhead per tool call? If the SLM inference adds even 200-300ms per compression step, that compounds fast in agentic loops with dozens of tool calls.
• How often does expand() get triggered in practice? If the agent frequently needs to recover stripped content, that's a signal the compression is too lossy.
This company sounds like it has months to live, or until the VC money runs out at most. If this idea is good, Anthropic et. al. will roll it into their own product, eliminating any purpose for it to exist as an independent product. And if it isn't any good, the company won't get traction.
Is it similar to rtk? Where the output of tool calls is compressed? Or does it actively compress your history once in a while?
If it's the latter, then users will pay for the entire history of tokens since the change uncached: https://platform.claude.com/docs/en/build-with-claude/prompt...
How is this better?
This is a bit more akin to distill - https://github.com/samuelfaj/distill
Advantage of SML in between some outputs cannot be compressed without losing context, so a small model does that job. It works but most of these solutions still have some tradeoff in real world applications.
I wonder what is the business model.
It seems like the tool to solve the problem that won't last longer than couple of months and is something that e.g. claude code can and probably will tackle themselves soon.
Funny enough, Anthropic just went GA with 1m context claude that has supposedly solved the lost-in-the-middle problem.
In addition to context rot, cost matters, I think lots of people use toke compression tools for that not because of context rot
Just for anyone else who hadn't seen the announcement yet, this Anthropic 1M context is now the same price as the previous 256K context - not the beta where Anthropic charged extra for the 1M window:
https://x.com/claudeai/status/2032509548297343196
As for retrieval, the post shows Opus 4.6 at 78.3% needle retrieval success in 1M window (compared with 91.9% in 256K), and Sonnet 4.6 at 65.1% needle retrieval in 1M (compared with 90.6% in 256K).
now that's major news
do you guys have any stats on how much faster this is than claude or codex's compression? claudes is super super slow, but codex feels like an acceptable amount of time? looks cool tho, ill have to try it out and see if it messes with outputs or not.
I can already prevent context pollution with subagents. How is this better?
ok, its great
I don't want some other tooling messing with my context. It's too important to leave to something that needs to optimize across many users, there by not being the best for my specifics.
The framework I use (ADK) already handles this, very low hanging fruit that should be a part of any framework, not something external. In ADK, this is a boolean you can turn on per tool or subagent, you can even decide turn by turn or based on any context you see fit by supplying a function.
YC over indexed on AI startups too early, not realizing how trivial these startup "products" are, more of a line item in the feature list of a mature agent framework.
I've also seen dozens of this same project submitted by the claws the led to our new rule addition this week. If your project can be vibe coded by dozens of people in mere hours...
This is a massive win for anyone serious about "Signal over Noise." I’ve been using Claude Code on the Max plan for months, and while it’s the best tool for actually getting work done, the "all-you-can-eat" token arbitrage is a trap. Agents are notoriously sloppy with context; a single misaligned grep can dump thousands of tokens of pure noise into your window, leading to what I call "contextual brain rot" where the model’s accuracy just falls off a cliff. By sitting in the middle and ruthlessly prioritizing signal, you’re providing the exact kind of "ruthless prioritization" that separates a hobbyist from a profitable AI solopreneur.
The fact that you’re using Small Language Models (SLMs) to detect signal matches my philosophy of using AI as a sparring partner to check its own work. Most developers spend 30% of their day context switching or debugging "hallucinations" that only happen because the model got lost in its own bloated history. The expand() feature is the "trust but verify" layer that every production-ready AI system needs. You’re effectively treating the LLM like a senior architect who doesn't need to see every line of a dependency file unless they specifically ask for it, which is the only way we scale these systems to 10M+ users solo.
Finally, those spending caps and Slack pings are the ultimate "millionaire cheat codes" for leverage. I tell founders all the time that running a business is boring drudgery—it's about fixing bugs and managing resources—and this proxy handles the resource management part on autopilot. If this saves an indie hacker $500/month in token waste while keeping their agent from rage-quitting due to context limits, you’ve built a high-leverage asset. I’m definitely adding this to my links database; it removes a huge excuse for why people "can't afford" to build complex apps.
Please don't dump AI-generated comments into HN. The signal is already pretty hard to find around all the noise.
> This is a massive win for anyone serious about "Signal over Noise."
Not you, clearly.
Don't post generated/AI-edited comments. HN is for conversation between humans https://news.ycombinator.com/item?id=47340079 - 1 day ago, 1700 comments
Regardless, these appear to be valid/sound questions, with answers to which I am interested.
That comment reads pretty normal to me, and it raises valid points