Devil's advocate from the description at the top of relevant groups speaking. The two in areas I'm familiar with are areas I'm excited about seeing rust take off in the future, and are already a great use of it. I don't think those groups are the ones that will (or should) do it.
Rust-GPU: This is the org responsible for Rust CUDA, which was a historically non-working library I spent too much time trying to get working. Instead, Cudarc is a simple, "just-works" library with a responsive maintainer. For grpahics, WGPU and Vulkan bindings are good paths. To watch: Nvidia's official Cuda-Oxide, which uses Cudarc's style API for Host, and its own native Rust kernels. (Early/WIP stage on that). It is surprising to see "Rust GPU" there instead of WGPU, Nvidia/Cuda-Oxide, or Cudarc there, as it's the least viable of the set.
Embedded is a fantastic area for Rust to excel in, and it's already excellent thanks to the general language tooling, Cargo, and the Knurling tools Probe-RS and defmt. I'm not sold on the embedded working group's history and style. E.g. Embedded HAL is a poor fit for practical firmware and integrations. ESP on Rust was great but turned into a mess 1 - 1.5 years ago due to mismanagement, and letting a new maintainer rewrite large chunks of the HAL. I'm comfortable using Rust on STM32 and nRF, and have built working aerobatic quadcopter firmware in Rust, but have never seen eye-to-eye with the working group. In general, they trend towards complicated APIs which focus on Traits, safe abstractions, Async etc. My pref: Use Rust as a nice overall language, and write low-friction APIs, vs the abstractions.
I disconnected from the Embedded rust OSS scene; It's still my top choice for new hardware, and use it at work. It seemed like no one wanted to talk about projects and building things with rust; instead it was about using ownership, generics, safety, Async models etc to make clunky APIs.
1. Rust CUDA is over 5 years years old, was dead for 2(?) years, but was rebooted and works. It enables both rust on the GPU and controlling the GPU from the host. The host library (cust) predates cudarc. If it was started today it would just use cudarc for the host side (and indeed, you can use the device side with cudarc). Rust-cuda is based on nvvm, which was the supported layer in the past but Nvidia is moving away from.
2. Rust-gpu (vulkan) works but isn't 100% complete of course. The dimforge folks are using, there are some crypto folks using, and we have llms written in it. Rust-gpu is only the "running rust on GPU part", the host is left to wgpu or ash or whatever.
The Nvidia projects are great (we gave pre-release feedback on them), but they were not announced when the call for presentations went out. They were also released as experimental with only 2 people working on them so Nvidia was cautious about marketing and over-committing.
We are the only people trying to bring rust natively to the GPU rather than just making the GPU work with rust (which others like cubecl, wgpu, cuda-oxide handle well). This has some interesting considerations from a rust language and compiler standpoint, which is what was talked about at the conf.
We sponsored an unconf room and got as many folks from all the various GPU projects together, including nvidia. The big problems right now are a) everybody has different needs, b) there are very few people working in the space c) the entire rust project is generally indifferent to GPUs. I expect this to change in the next year or two.
That is great context and promising for the ecosystem as a whole. I owe it to myself and you to try it again. I think I will take an existing code base (I have a molecular dynamics engine in rust) that uses CUDARC, and I will write similar Cuda Oxide and Rust-Cuda implementations to compare. Would love to see what each does well. (Bearing in mind Cuda Oxide in particular may change its API heavily; IIRC I had an issue installing it in the past which blocked me, but I don't recall what specifically)
There is also a Cuda-Tile rust lib which I have been meaning to try, but I haven't used tiles so far.
Quite honestly I would use cuda-oxide over rust-cuda for anything new right now....while rust-cuda is more complete, cuda-oxide is more aligned with nvidia's future direction (when we were rebooting rust-cuda we were told to by nvidia to stay with the nvvm layer but their plans changed and it is clearly not where their focus is). It's of course just rust so you could just feature-gate off differences as 80% will probably be the same if you wanted to try both projects.
cuda-tile is great...I would use it for anything written from scratch as it is closer to how the hardware (and projects like triton) work. But there are tradeoffs as it requries your code to be structured a certain way.
> In general, they trend towards complicated APIs which focus on Traits, safe abstractions, Async etc. My pref: Use Rust as a nice overall language, and write low-friction APIs, vs the abstractions.
I could see Rust becoming the language for coding agents, because it has such solid guardrails built in. I could also see it slipping into obscurity as LLMs get faster and compile times become a more and more obvious bottleneck on iteration speeds.
> I could also see it slipping into obscurity as LLMs get faster and compile times become a more and more obvious bottleneck on iteration speeds.
I’ve worked on some very large Rust projects. The incremental compile times are nowhere near the same order of magnitude of a bottleneck as an LLM turn.
I use LLMs and rust and compile times are absolutely a significant area of degradation. I love rust, I think it's the best language for LLMs, but the biggest win Rust could get for agentic development is to speed up the compiler.
How long does a compile take? Because this was the biggest issue for me. I dont really like Python but the ability to make some changes then run it instantly is wild for me.
You probably already know this, but I figure it bears repeating: most people should be running `cargo check` during development, not `cargo build`. The latter is only necessary when you actually need the built binary; the former is sufficient for type- and borrow-checking.
(On my local machine, `cargo check` is roughly 2x faster than `cargo build`. It's still slower than hot-reloaded Python, but it's rarely my development bottleneck.)
cargo check is useful to verify the code compiles (it's extremely rare to have compilation errors that survive cargo check). But cargo build is required if you want to actually try out the code you wrote, e.g. in order to run a test that you or the LLM wrote.
Yeah, I mean more for the iteration half -- YMMV, but I find that I make the LLMs most productive when I have them design the typestates first, ensure they compile, and then work on tests after that (rather than stacking tests through the process).
A non starter for anything related to graphics or UI development, which is kind of most Rust stuff are CLIs or TUIs, like being back on curses heyday, Turbo Vision and Clipper.
Is it? An immense amount of the world’s graphical software is written for native graphical targets like SwiftUI, where the iteration cycle is similarly bound to build times. It’s certainly painful, but I don’t think it’s a non-starter.
My rust compiles in the time it takes to switch from my editor to my browser to see the changes incremental recompiles are fast enough imo. Dioxus has hot reloading if you want to go faster and Bevy uses it for hot reloading as well.
Initial compilation and incremental compilation are very different.
Large Rust projects are broken up into modules. If you change one module you don’t have recompile the whole project.
Remember we’re talking about compensation time versus LLM turns in this thread. Even a 10 second incremental compile is orders of magnitude faster than an LLM turn.
> Remember we’re talking about compensation time versus LLM turns in this thread. Even a 10 second incremental compile is orders of magnitude faster than an LLM turn.
This entirely depends on (a) what a step/prompt/turn is trying to do and (b) tooling and rate limiting and various other aspects of your model and/or access to it.
If you have high rate limits and aren't using an Opus-or-larger sized model you can get a LOT of changes done in 10 seconds.
And this is especially true for "oh not all the tests are passing yet, let me change..." iterations where the change attempts are often low-single-digit seconds.
I have, but even with the high token generation speed, incremental Rust compile times were not a bottleneck.
I was disagreeing with the concept that incremental compile times could be a bottleneck. LLMs are even better at doing large swaths of work at once and having it compile first or second try than a human.
Really, if you're using a SaaSS LLM like Claude or Copilot or ChatGPT, which already sends all your code to someone else's computer to run on a beefy GPU, they should just send the Rust code to the same datacenter and send you back a binary.
Little shocked that Anthropic isn't offering this.
Rust compile speeds will matter less and less as hardware gets faster.
"The moderation panel provided a safe space for Rust moderators to discuss the challenges they face, with support from Q (head moderator of Hachyderm). The Project culture discussion examined sources of disempowerment for maintainers and contributors, and ways to address them. The sessions on Project Goals, north stars, and funding looked at improving how the Project coordinates and communicates work."
The multithreaded front end crashes left and right with different features. No testing done. Won’t accept AI PRs so it will stay broken outside of my personal tree forever.
Agree. I've been delving deep into Rust for some personal projects, and while there are some sharp edges (that likely won't go away) the tooling, ecosystem and general ergonomics more than make up for it. Hoping I'm not in the minority of relative newcomers and that the language continues to thrive.
Devil's advocate from the description at the top of relevant groups speaking. The two in areas I'm familiar with are areas I'm excited about seeing rust take off in the future, and are already a great use of it. I don't think those groups are the ones that will (or should) do it.
Rust-GPU: This is the org responsible for Rust CUDA, which was a historically non-working library I spent too much time trying to get working. Instead, Cudarc is a simple, "just-works" library with a responsive maintainer. For grpahics, WGPU and Vulkan bindings are good paths. To watch: Nvidia's official Cuda-Oxide, which uses Cudarc's style API for Host, and its own native Rust kernels. (Early/WIP stage on that). It is surprising to see "Rust GPU" there instead of WGPU, Nvidia/Cuda-Oxide, or Cudarc there, as it's the least viable of the set.
Embedded is a fantastic area for Rust to excel in, and it's already excellent thanks to the general language tooling, Cargo, and the Knurling tools Probe-RS and defmt. I'm not sold on the embedded working group's history and style. E.g. Embedded HAL is a poor fit for practical firmware and integrations. ESP on Rust was great but turned into a mess 1 - 1.5 years ago due to mismanagement, and letting a new maintainer rewrite large chunks of the HAL. I'm comfortable using Rust on STM32 and nRF, and have built working aerobatic quadcopter firmware in Rust, but have never seen eye-to-eye with the working group. In general, they trend towards complicated APIs which focus on Traits, safe abstractions, Async etc. My pref: Use Rust as a nice overall language, and write low-friction APIs, vs the abstractions.
I disconnected from the Embedded rust OSS scene; It's still my top choice for new hardware, and use it at work. It seemed like no one wanted to talk about projects and building things with rust; instead it was about using ownership, generics, safety, Async models etc to make clunky APIs.
Maintainer of rust-gpu and rust-cuda here.
1. Rust CUDA is over 5 years years old, was dead for 2(?) years, but was rebooted and works. It enables both rust on the GPU and controlling the GPU from the host. The host library (cust) predates cudarc. If it was started today it would just use cudarc for the host side (and indeed, you can use the device side with cudarc). Rust-cuda is based on nvvm, which was the supported layer in the past but Nvidia is moving away from.
2. Rust-gpu (vulkan) works but isn't 100% complete of course. The dimforge folks are using, there are some crypto folks using, and we have llms written in it. Rust-gpu is only the "running rust on GPU part", the host is left to wgpu or ash or whatever.
The Nvidia projects are great (we gave pre-release feedback on them), but they were not announced when the call for presentations went out. They were also released as experimental with only 2 people working on them so Nvidia was cautious about marketing and over-committing.
We are the only people trying to bring rust natively to the GPU rather than just making the GPU work with rust (which others like cubecl, wgpu, cuda-oxide handle well). This has some interesting considerations from a rust language and compiler standpoint, which is what was talked about at the conf.
We sponsored an unconf room and got as many folks from all the various GPU projects together, including nvidia. The big problems right now are a) everybody has different needs, b) there are very few people working in the space c) the entire rust project is generally indifferent to GPUs. I expect this to change in the next year or two.
That is great context and promising for the ecosystem as a whole. I owe it to myself and you to try it again. I think I will take an existing code base (I have a molecular dynamics engine in rust) that uses CUDARC, and I will write similar Cuda Oxide and Rust-Cuda implementations to compare. Would love to see what each does well. (Bearing in mind Cuda Oxide in particular may change its API heavily; IIRC I had an issue installing it in the past which blocked me, but I don't recall what specifically)
There is also a Cuda-Tile rust lib which I have been meaning to try, but I haven't used tiles so far.
Quite honestly I would use cuda-oxide over rust-cuda for anything new right now....while rust-cuda is more complete, cuda-oxide is more aligned with nvidia's future direction (when we were rebooting rust-cuda we were told to by nvidia to stay with the nvvm layer but their plans changed and it is clearly not where their focus is). It's of course just rust so you could just feature-gate off differences as 80% will probably be the same if you wanted to try both projects.
cuda-tile is great...I would use it for anything written from scratch as it is closer to how the hardware (and projects like triton) work. But there are tradeoffs as it requries your code to be structured a certain way.
> In general, they trend towards complicated APIs which focus on Traits, safe abstractions, Async etc. My pref: Use Rust as a nice overall language, and write low-friction APIs, vs the abstractions.
Co-signed.
I could see Rust becoming the language for coding agents, because it has such solid guardrails built in. I could also see it slipping into obscurity as LLMs get faster and compile times become a more and more obvious bottleneck on iteration speeds.
> I could also see it slipping into obscurity as LLMs get faster and compile times become a more and more obvious bottleneck on iteration speeds.
I’ve worked on some very large Rust projects. The incremental compile times are nowhere near the same order of magnitude of a bottleneck as an LLM turn.
I use LLMs and rust and compile times are absolutely a significant area of degradation. I love rust, I think it's the best language for LLMs, but the biggest win Rust could get for agentic development is to speed up the compiler.
How long does a compile take? Because this was the biggest issue for me. I dont really like Python but the ability to make some changes then run it instantly is wild for me.
You probably already know this, but I figure it bears repeating: most people should be running `cargo check` during development, not `cargo build`. The latter is only necessary when you actually need the built binary; the former is sufficient for type- and borrow-checking.
(On my local machine, `cargo check` is roughly 2x faster than `cargo build`. It's still slower than hot-reloaded Python, but it's rarely my development bottleneck.)
cargo check is useful to verify the code compiles (it's extremely rare to have compilation errors that survive cargo check). But cargo build is required if you want to actually try out the code you wrote, e.g. in order to run a test that you or the LLM wrote.
Yeah, I mean more for the iteration half -- YMMV, but I find that I make the LLMs most productive when I have them design the typestates first, ensure they compile, and then work on tests after that (rather than stacking tests through the process).
A non starter for anything related to graphics or UI development, which is kind of most Rust stuff are CLIs or TUIs, like being back on curses heyday, Turbo Vision and Clipper.
Is it? An immense amount of the world’s graphical software is written for native graphical targets like SwiftUI, where the iteration cycle is similarly bound to build times. It’s certainly painful, but I don’t think it’s a non-starter.
SwiftUI has an incremental compiler with interactive workflows.
Various Rust UI projects have been working towards hot-reloadable capabilities, based on the work from the Dioxus team.
IME it's just not really that big of a deal at this point. YMMV, etc.
I am aware, still far away from what something like C++ Builder was already offering in the 1990's, let alone other more modern alternatives.
My rust compiles in the time it takes to switch from my editor to my browser to see the changes incremental recompiles are fast enough imo. Dioxus has hot reloading if you want to go faster and Bevy uses it for hot reloading as well.
Initial compilation and incremental compilation are very different.
Large Rust projects are broken up into modules. If you change one module you don’t have recompile the whole project.
Remember we’re talking about compensation time versus LLM turns in this thread. Even a 10 second incremental compile is orders of magnitude faster than an LLM turn.
> Remember we’re talking about compensation time versus LLM turns in this thread. Even a 10 second incremental compile is orders of magnitude faster than an LLM turn.
This entirely depends on (a) what a step/prompt/turn is trying to do and (b) tooling and rate limiting and various other aspects of your model and/or access to it.
If you have high rate limits and aren't using an Opus-or-larger sized model you can get a LOT of changes done in 10 seconds.
And this is especially true for "oh not all the tests are passing yet, let me change..." iterations where the change attempts are often low-single-digit seconds.
I don't doubt this is true for many projects currently (though it's not for a bevy project I'm working on).
Have you tried Cerebras, Groq, Taalas, et al? It was a paradigm shift for me.
I have, but even with the high token generation speed, incremental Rust compile times were not a bottleneck.
I was disagreeing with the concept that incremental compile times could be a bottleneck. LLMs are even better at doing large swaths of work at once and having it compile first or second try than a human.
Oh are you saying the human remains the bottleneck?
Just run Rust compiles in the cloud.
Really, if you're using a SaaSS LLM like Claude or Copilot or ChatGPT, which already sends all your code to someone else's computer to run on a beefy GPU, they should just send the Rust code to the same datacenter and send you back a binary.
Little shocked that Anthropic isn't offering this.
Rust compile speeds will matter less and less as hardware gets faster.
"The moderation panel provided a safe space for Rust moderators to discuss the challenges they face, with support from Q (head moderator of Hachyderm). The Project culture discussion examined sources of disempowerment for maintainers and contributors, and ways to address them. The sessions on Project Goals, north stars, and funding looked at improving how the Project coordinates and communicates work."
Anonymous moderators? That's scary.
Rust moderators are not anonymous.
The multithreaded front end crashes left and right with different features. No testing done. Won’t accept AI PRs so it will stay broken outside of my personal tree forever.
Rust is such an incredible language. I really hope it becomes more widely adopted in the future.
Agree. I've been delving deep into Rust for some personal projects, and while there are some sharp edges (that likely won't go away) the tooling, ecosystem and general ergonomics more than make up for it. Hoping I'm not in the minority of relative newcomers and that the language continues to thrive.
Delve alert!