It’s more like goroutines or other lightweight concurrency mechanisms. If threads are OS-level concurrency primitives, fibers are scheduled within Ruby itself, which makes them much more efficient than threads.
In fact, I got the following results in HTTP benchmark tests:
Go:
* Latency under stable load: p95 0.25–5.32 ms, p99 2.20–9.92 ms
* Memory: 23–31 MB RSS across HTTP scenarios
Ruby:
* Latency under stable load: p95 1.03–6.45 ms, p99 2.32–8.30 ms
* Memory: 84–295 MB RSS, depending on the scenario
Fibers can also handle WebSockets better because WebSocket workloads involve more I/O waiting.
A typical Falcon setup uses N workers, one thread per worker, and many fibers. Since the fibers are cooperatively scheduled within a single thread, this avoids much of the context-switching overhead associated with OS threads. Multiple workers can still run in parallel across CPU cores.
To dig a bit into why fibers are more efficient than threads.
The OS has a rather opaque view on what happens in a thread. It doesn't know how the memory is used or what memory is efficient. The OS can block a thread when it runs into IO, but it doesn't know or care about what other threads in an application it can or should activate.
Fibers bring the threading into the application layer. While the application can't choose which and when a thread runs, it can make choices about which fibers to run. Further, the application knows intimate details about things like the stack of a given thread. When it goes to park a fiber, it knows just how much memory should be saved off so the fiber can resume and it doesn't have to save off all the memory allocated to a thread's stack. Further, because so many programs are stack based an application can pretty smartly save and reuse segments of the stack which are common amongst fibers. So, for example, if you spin off 1000 fibers at one location in code, the stack for those 1000 fibers will be identical right up until the fiber starts executing.
The main drawback of fibers is they can't implement things like fair scheduling. Applications have few ways to park a currently running fiber to let another one run if, for example, the app wants to make some progress on all the fibers alive. The app has to wait for the fiber to hit some sort of IO point in the code or insert explicit park checks (The JVM actually does this for GC purposes. It creates "safepoints" which application threads make a quick check to see if the JVM wants to start a GC). The OS has more power here, it can simply interrupt the thread and start running something else for a given quanta.
How much load? How many concurrent connections? What machine? Don’t get me wrong, benchmarks like these are useful to help ballpark performance floors, but really only relevant for a given load scenario. They don’t mean a lot without context. Not an attack by the way, just feedback.
I'm unclear on the relevance of a comparison between Go and Ruby here. Ruby is radically slower than Go. It's down there with Python, if not a touch behind [1]. From the outside, I'd expect that it's possible that slight improvements in the context switching time will be dwarfed by the generally slow execution of Ruby itself; that is, if Ruby is going to take 100 microseconds to do something, whether it context switches in 1 microsecond (best-case OS thread cost) or .2 microseconds (best-case goroutine switch) is of somewhat less consequence then it is for a compiled langauge that can complete that task in 2 microseconds. That ratio of 50x is not just something I made up, it's about what you can expect in general. I'd need to see the actual Ruby benchmarks to come to any conclusions as to whether or not I'm right.
The other problem with this sort of benchmark, which is a mistake I also commonly see made by Node developers, is that the Ruby HTTP stack has significant native code in it, like: https://github.com/puma/puma/tree/main/ext/puma_http11 This is a good and proper thing that brings benefits to all involved; it's not like it's "cheating" or anything, it's a real performance benefit. But it does mean when you're benchmarking a simple HTTP server, you're benchmarking Ruby qua Ruby a lot less than you think you are, and so the relevance of such benchmarks to codebases that have actual Ruby in them will be less.
It was mostly HTTP/JSON tests with some sort of validation, basic auth and logging. I think Ruby is used the most for this. I see many companies switching from Ruby to Go or FastAPI for this basic web services stuff, and I have no idea why. Ruby needs to improve its memory management, but the speed is pretty good.
Some programmers find it easier to write concurrent programs that use "light-weight threads", "fibers", "goroutines", "coroutines" or other variants of this idea.
This feature is obviously intended for them and it might enhance their productivity.
Nonetheless, no program that uses a great number of any variant of the "light-weight threads" can ever be as efficient as a thread pool that is dimensioned to have the same number of threads as the number of hardware threads of a SMT CPU, or a slightly greater number of threads than the number of hardware threads of a non-SMT CPU.
For maximum performance, the use of a correctly-sized thread pool remains the best solution, but writing an efficient program that uses it can be significantly more difficult, because good methods of communication and synchronization must be implemented, while the run-time library of a language with "light-weight threads"/"fibers"/etc. already takes care of such problems so the programmer does not need to think about them.
Fiber is a datastructure where the execution context is saved. Eg. registers (including instruction pointer) and stack etc. That is a fiber: data.
You normally use a threadpool, core pinned, to execute these fibers.
Since you jump in userspace, you more or less only have to pay for cache misses.
There are many upsides of designing a program using fibers. The major downside i see is that you can not blindly trust mutex and semaphores any longer - since the fiber can change execution thread while yielding/waiting for condition.
Fiber mode isn’t universally better. CPU-bound jobs get nothing from it, and blocking libraries or C extensions that do not cooperate with Ruby’s fiber scheduler stall the reactor. And that’s fine – you don’t have to pick one.
As Trevor Turk pointed out in the PR discussion, that’s the whole point: separately configured worker pools. Here’s what Chat with Work actually runs in production:
My concern is number of database connections. In the example it's 100 fibers per worker, at that rate you are going to exhaust db connections sooner. Happy to be wrong.
Carmine (which coded this Fibers update) wrote about this in his blog. See the section 'The database connection math'. And no, you won't have one connection per fiber.
The difference is staggering when you compare to threaded mode: it requires 1,320 database connections to run the same benchmark that the fiber mode runs with 60.
Pretty sure that is entirely an unfair comparison. You don't need a connection per thread either. It is common to have web servers handles thousands of requests per second with a connection pool size of 20. As long as the handler threads borrow a connection for only a little time and block waiting for a connection while other threads are doing their thing, it works out fine.
There is absolutely no logical reason why database pool sizing could be a reason for preferring fibers over threads.
I think you want to make sure you don't hold a database connection while waiting on other slow async work (like outgoing HTTP requests). Then you can more feasibly have more workers than pool size. It's just very tricky to do this in Rails...
I don't know Solid Queue or the rails environment, but I expect that not every worker will create it's own connection, there should be a connection pool in-between
I think by default they check out a new connection when obtained a job and then release it. In comparison with some async languages like JS where a connection is only checked when a query is about to be executed
this used to be the case in Rails, but hasn't been for at least a year. Nowadays each framework-controlled action (like record.save) checks out a connection and then returns it back; this was implemented as part of the whole "run rails in fiber-based server" push
As someone who spent 15+ years doing Ruby/Rails, it’s nice to see this land.
That said, these days you’ll pry the BEAM from my cold, dead hands. It’s hard to go back to any other concurrency story.
Ironically, looking into Ruby Fiber led me to BEAM/Elixir. You are right, there is no going back!
Same path & agreed. The amount of things you do not have to care about with BEAM/Elixir compared to Rails is really interesting.
>BEAM/Elixir compared to Rails
Compared to Ruby. Elixir land doesn't have something similar to Rails, and phoenix is not it.
I’m really curious about this as someone that has never tried Elixir/BEAM.
What could I stop caring about?
What's BEAM?
https://en.wikipedia.org/wiki/BEAM_(Erlang_virtual_machine)
So fibers are a lot like threads but they're more scoped to a task that can be paused and resumed, that's kinda cool
It’s more like goroutines or other lightweight concurrency mechanisms. If threads are OS-level concurrency primitives, fibers are scheduled within Ruby itself, which makes them much more efficient than threads.
In fact, I got the following results in HTTP benchmark tests:
Go:
* Latency under stable load: p95 0.25–5.32 ms, p99 2.20–9.92 ms * Memory: 23–31 MB RSS across HTTP scenarios
Ruby:
* Latency under stable load: p95 1.03–6.45 ms, p99 2.32–8.30 ms * Memory: 84–295 MB RSS, depending on the scenario
Fibers can also handle WebSockets better because WebSocket workloads involve more I/O waiting.
A typical Falcon setup uses N workers, one thread per worker, and many fibers. Since the fibers are cooperatively scheduled within a single thread, this avoids much of the context-switching overhead associated with OS threads. Multiple workers can still run in parallel across CPU cores.
To dig a bit into why fibers are more efficient than threads.
The OS has a rather opaque view on what happens in a thread. It doesn't know how the memory is used or what memory is efficient. The OS can block a thread when it runs into IO, but it doesn't know or care about what other threads in an application it can or should activate.
Fibers bring the threading into the application layer. While the application can't choose which and when a thread runs, it can make choices about which fibers to run. Further, the application knows intimate details about things like the stack of a given thread. When it goes to park a fiber, it knows just how much memory should be saved off so the fiber can resume and it doesn't have to save off all the memory allocated to a thread's stack. Further, because so many programs are stack based an application can pretty smartly save and reuse segments of the stack which are common amongst fibers. So, for example, if you spin off 1000 fibers at one location in code, the stack for those 1000 fibers will be identical right up until the fiber starts executing.
The main drawback of fibers is they can't implement things like fair scheduling. Applications have few ways to park a currently running fiber to let another one run if, for example, the app wants to make some progress on all the fibers alive. The app has to wait for the fiber to hit some sort of IO point in the code or insert explicit park checks (The JVM actually does this for GC purposes. It creates "safepoints" which application threads make a quick check to see if the JVM wants to start a GC). The OS has more power here, it can simply interrupt the thread and start running something else for a given quanta.
How much load? How many concurrent connections? What machine? Don’t get me wrong, benchmarks like these are useful to help ballpark performance floors, but really only relevant for a given load scenario. They don’t mean a lot without context. Not an attack by the way, just feedback.
I'm unclear on the relevance of a comparison between Go and Ruby here. Ruby is radically slower than Go. It's down there with Python, if not a touch behind [1]. From the outside, I'd expect that it's possible that slight improvements in the context switching time will be dwarfed by the generally slow execution of Ruby itself; that is, if Ruby is going to take 100 microseconds to do something, whether it context switches in 1 microsecond (best-case OS thread cost) or .2 microseconds (best-case goroutine switch) is of somewhat less consequence then it is for a compiled langauge that can complete that task in 2 microseconds. That ratio of 50x is not just something I made up, it's about what you can expect in general. I'd need to see the actual Ruby benchmarks to come to any conclusions as to whether or not I'm right.
The other problem with this sort of benchmark, which is a mistake I also commonly see made by Node developers, is that the Ruby HTTP stack has significant native code in it, like: https://github.com/puma/puma/tree/main/ext/puma_http11 This is a good and proper thing that brings benefits to all involved; it's not like it's "cheating" or anything, it's a real performance benefit. But it does mean when you're benchmarking a simple HTTP server, you're benchmarking Ruby qua Ruby a lot less than you think you are, and so the relevance of such benchmarks to codebases that have actual Ruby in them will be less.
[1]: https://programming-language-benchmarks.vercel.app/python-vs...
It was mostly HTTP/JSON tests with some sort of validation, basic auth and logging. I think Ruby is used the most for this. I see many companies switching from Ruby to Go or FastAPI for this basic web services stuff, and I have no idea why. Ruby needs to improve its memory management, but the speed is pretty good.
I think the GP just showed that in a particular scenario, Ruby isn’t “radically slower” than Go. So how is the comparison not relevant?
Thanks! That was exactly my point. The ruby community has made many performance improvements since 2.0, including JIT compilation, fibers and ractors.
Some programmers find it easier to write concurrent programs that use "light-weight threads", "fibers", "goroutines", "coroutines" or other variants of this idea.
This feature is obviously intended for them and it might enhance their productivity.
Nonetheless, no program that uses a great number of any variant of the "light-weight threads" can ever be as efficient as a thread pool that is dimensioned to have the same number of threads as the number of hardware threads of a SMT CPU, or a slightly greater number of threads than the number of hardware threads of a non-SMT CPU.
For maximum performance, the use of a correctly-sized thread pool remains the best solution, but writing an efficient program that uses it can be significantly more difficult, because good methods of communication and synchronization must be implemented, while the run-time library of a language with "light-weight threads"/"fibers"/etc. already takes care of such problems so the programmer does not need to think about them.
Some context:
Fiber is a datastructure where the execution context is saved. Eg. registers (including instruction pointer) and stack etc. That is a fiber: data.
You normally use a threadpool, core pinned, to execute these fibers.
Since you jump in userspace, you more or less only have to pay for cache misses.
There are many upsides of designing a program using fibers. The major downside i see is that you can not blindly trust mutex and semaphores any longer - since the fiber can change execution thread while yielding/waiting for condition.
Has anyone played with this and SQLite? I have no data, just a hunch, but I’d think this is a recipe for corruption if you’re doing lots of writes.
PG till the wheels fall off baby
This is a nice update.
Is it possible to either have multiple ractors dispatching jobs with fibres or to set up multiple queues with different strategies?
E.g. one for IO bound and one for CPU bound?
With Sidekiq I’ve had luck having workers running on Truffleruby but generally don’t use it for my main rails apps.
You can, and Carmine (who coded this update) wrote exactly about this on this blog post: https://paolino.me/solid-queue-doesnt-need-a-thread-per-job/
From his article:
One backend, two modes
Fiber mode isn’t universally better. CPU-bound jobs get nothing from it, and blocking libraries or C extensions that do not cooperate with Ruby’s fiber scheduler stall the reactor. And that’s fine – you don’t have to pick one.
As Trevor Turk pointed out in the PR discussion, that’s the whole point: separately configured worker pools. Here’s what Chat with Work actually runs in production:
workers: - queues: [ chat ] fibers: 10 processes: 2 polling_interval: 0.1 - queues: [ turbo ] fibers: 10 processes: 1 polling_interval: 0.05 - queues: [ notifications, default, maintenance ] fibers: 5 processes: 1 polling_interval: 0.2 - queues: [ cpu ] threads: 1 processes: 1
My concern is number of database connections. In the example it's 100 fibers per worker, at that rate you are going to exhaust db connections sooner. Happy to be wrong.
Carmine (which coded this Fibers update) wrote about this in his blog. See the section 'The database connection math'. And no, you won't have one connection per fiber.
The difference is staggering when you compare to threaded mode: it requires 1,320 database connections to run the same benchmark that the fiber mode runs with 60.
https://paolino.me/solid-queue-doesnt-need-a-thread-per-job/
Pretty sure that is entirely an unfair comparison. You don't need a connection per thread either. It is common to have web servers handles thousands of requests per second with a connection pool size of 20. As long as the handler threads borrow a connection for only a little time and block waiting for a connection while other threads are doing their thing, it works out fine.
There is absolutely no logical reason why database pool sizing could be a reason for preferring fibers over threads.
I think you want to make sure you don't hold a database connection while waiting on other slow async work (like outgoing HTTP requests). Then you can more feasibly have more workers than pool size. It's just very tricky to do this in Rails...
I don't know Solid Queue or the rails environment, but I expect that not every worker will create it's own connection, there should be a connection pool in-between
I think by default they check out a new connection when obtained a job and then release it. In comparison with some async languages like JS where a connection is only checked when a query is about to be executed
this used to be the case in Rails, but hasn't been for at least a year. Nowadays each framework-controlled action (like record.save) checks out a connection and then returns it back; this was implemented as part of the whole "run rails in fiber-based server" push