One thing that's lovely about Linux is this kind of analysis is not only possible, but meaningful. These results will get reported back to the graphics software authors and the distribution packagers and the ecosystem will improve. There's no sense with Microsoft that kind of improvement is possible.
I recently switched to Linux after years on Windows desktop, mostly because the KDE Plasma desktop feels snappier than Windows 11. Also the feeling that if something isn't working right I can probably tinker and improve it. It's been really nice. If you haven't tried Linux desktops in awhile give Bazzite a whirl: it's a Fedora customized for gaming. Even if you don't game it's an easy way to get a very functional Linux desktop in no time at all.
Yes of course, but you need to do it via the package manager. You can't "sudo make install". If you build your own software and want to install it system wide, you need to flatpack/brew it or run it in a docker container.
(obviously you can modify the filesystem if you really really really want to).
Bazzite is gaming oriented version of Fedora CoreOs. There are many different versions. I am running bluefin.
I've tried a lot of desktop linux distros, and to be honest, immutable linux feels like the future. Anything you do can simply be rolled back. Break something? Just roll it back.
And if you run something like Bazzite, but want to try out Bluefin-Dx which is developer oriented, then you can rebase your existing installation. If you don't like it, just revert back to Bazzite with a single command.
However, it's desktop oriented. Don't run CoreOs on a server.
The immutable linux is pretty great if you find one that fits you type of dev work. The prebuilt images are powered by a config (so you could build your own) but it's great that you can just follow someones good "config". Switching/updates are suddenly easy.
For example i just need docker for webdev and there is bazzite-dx basically bazzite with docker and few things added. Works pretty great, sometimes when something goes bad i rollback the image and wait for future version.
I use Bazzite for development bc it works like a Mac with sane defaults and a just works approach:
- IDEs are no problem. Editors will "just work" for anything you type into the app store - Bazzite handles the special cases for you and installs them through brew taps or Flatpaks.
- For development it's basically just like a Mac where you also can't install system-level packages: Node, Python etc work through brew / nvm / uv same as on Mac. Development that involves containers will be unchanged from a Mac. For compilers specifically, same as on Mac: Install it through brew, or if you need a Debian or Fedora base you do `distrobox create` and you can apt-install in a transparent podman container.
Bazzite mostly suggests installing stuff via Flatpak or Homebrew. (No, really, brew on Linux lol). You can also layer in things with rpm-ostree but it's clunky. I think the immutable OS makes sense for a consumer but not for a developer.
So I'm using Nobara instead. It's a different Fedora-for-gaming but has most of the same improvements. It is a traditional system, not immutable. CachyOS is also very popular and that gets you an Arch-for-gaming. Just yesterday I learned of PikaOS, a Debian-for-gaming.
The main thing all these gaming-customized systems are doing is getting graphics drivers and proprietary codecs installed for you easily.
Why not for developers? You can install tools from flatpak or in home directory. My workflow includes toolbox containers. Distrobox is also good option.
Many flatpaks aren't actually maintained by the actual developer nor the normal way the package is normally used. You may have bugs that aren't present in the package that the dev isn't aware of or interested in fixing especially if they support a different channel and the bug relates to sandboxing.
There is also a risk that the person may be malicious from the start, sell out, or simply get malware. Given the nature of the ecosystem a malicious release to a previously safe package could propagate incredibly quickly.
Where there are multiple steps for a package to get from developers machine to yours and each is slow enough for malicious behavior to be noticed each step adds friction and decreases the chance of ultimate success. Where all steps are nearly simultaneous your risk multiplies with each step in which a different person has their hands in it and if any of them are malicious or compromised you are screwed.
You can modify the fs using rpm-ostree. It will overlay your changes on top of the immutable image. The recommended way is to create toolbox/distrobox which uses containers to isolate stuff.
Some things do indeed improve, but some other things require a massive effort that no hobbyist or small company can make happen.
This is most obvious in places where a lot of coordination is required, for example in supporting proper color correction throughout all applications, or decent support for advanced printer functions.
There are many incremental changes, but we often get stuck in local minima for years.
Still, I personally like that one can (relatively) easily watch what happens under the hood. It's not entirely clear to me why Windows and MacOS must remain closed source.
This kind of thing is certainly possible under Windows - you can basically patch any kernel API call, replace any COM object instantiation, install filter drivers that intercept any request to and from a device, replace userland DLLs with your own.
It's really scary what you can do, to the point that I often asked myself 'why allow this?' - seeing as hits on certain APIs took me to blackhat forums and articles about writing exploits.
Oh sure, but if I just need to troubleshoot why Minecraft-launcher (a first party app) won't launch ... it didn't give any output; it didn't even exist apparently, but
only because MS were hiding it, had to crack out Process Explorer just to get something to troubleshoot on ... then it turns out the "turnkey" app from their first-party app store, loading the first-party app, on the same company's OS just failed with no indications to the user at all, not even a "this app crashed". Solution was to cut out as much of MS as possible ... it's just infuriating when it doesn't work, which seems to be all-the-time.
More power to Bazzite and Valve, the sooner games app run in other OS the better.
I think it depends who it is making noise. There are some famous Oculus-era stories about John Carmack tracing display latency issues and then writing lengthy screeds to Samsung engineers to get them to give him lower level access.
I bet if someone like him made enough noise, people at MS would pay attention.
Afaik reflex is a bit different - it times the input to frame latency in your game, and tries to start the gameloop as late as possible in the frame so that your input gets sampled at the latest possible time and the frame still makes it to the vsync period, this isn't really a compositor tech.
Why would it need to measure input to frame latency? Surely it just needs to measure how long it takes the game to go from vsync period to frame ready and delay when the game thinks vsync happened to force it to poll later in the cycle?
I switched my daily driver / gaming rig to Fedora a few months back.
Everything seems snappier compared to Windows, but not sure if it’s in my head, and I’ve been very curious about gaming input latency. This helps answer some questions.
I recently switched to hyprland and I’m very interested how that fits in these results. hyprland uses Wayland so I hope the author might revisit now that hyprland is gaining in popularity.
I’ve considered using gamescope to hopefully get in front of some of these concerns, but I’m on nvidia and there is some discussion about it not working well there.
Now the author's got me thinking about gaming-optimized kernels, which I did not realize was a thing.
I play competitive fighting games so input latency is a huge concern. Would love to hear from anyone else who’s been down this path.
With the exception of XWayland, all of the tests had input latencies within a very small range. No human could tell them apart by those latency differences alone. I would be amazed if someone could notice the 3ms difference jump to XWayland.
The difference could be much larger on a slower monitor. However the differences between Wayland and X11 as protocols is negligible. XWayland as an implementation looks to have a limitation.
The context here is the article's author is a twitchy FPS player, those extra 3ms are something that community agonizes over. I appreciated how much effort was put into controlling the variables in this test. There's a whole lot of witchcraft associated with these kinds of efforts and he sat down and did the measurements and got real numbers. My hat is off to OP.
As far as I'm aware, you get marginal FPS gains switching kernels. It helps some. It's just a matter of whether the effort is worth it to you. The main change is the scheduler: rather than trying to evenly distribute CPU time to processes, it will prioritize bursty processes.
I've been a fan of Hyprland for gaming so far. Much more configurable for things like VRR/tearing and other precise tweaks via Gamescope than when I was on AwesomeWM with X11. Been especially nice having Lua for configuration, which finally feels very familiar with my AwesomeWM roots.
I moved to OpenSuse and have the same experience/opinion. The only caveat I had in Wayland is Game Streaming. Sunshine/Moonlight work but the input lag is noticeable and there are artifacts in the game. I go back and forth between X11 and Wayland just because X11 better for game streaming but in time I'm sure I'll go full-time into Wayland.
He seems confused at the end why people think wayland is so slow, but don't you think it's because of his xwayland result? People were probably running x11 games on wayland and noticed that significant lag. Just a wild guess. Very nice article, wish people did actual measurements like this more often, of all sorts of things.
I've been using Linux since the mid 1990's. I'm no newbie to any of this. I literally can't tell the different between X11 and Wayland when using either of them and I don't care about all the arguing. This is just Vim vs Emacs and Gnome vs KDE all over again. At this point when I see people complaining about it I just click off the page. It's all stupid and pointless.
My biggest problem with wayland was how it was basically forced on the community. It broke innumerable things for years, put all the responsibility for implementing things down on the DEs and WMs themselves.
All of this hassle, forcing so much more work on DE/WM devs, for the sake of 'better security' in scenarios that don't really apply to 99% of linux users, with the promise of 'better latency' which this very article proves is false.
I tried to be an early adopter of wayland ~ 5 years ago. Found all sorts of things broken, and I'm now using linux mint xfce edition, as hopefully by the time xfce drags itself to wayland, all the bugs and tooling will be a solved problem.
And the attitude of just refusing to make things work because haha fuck you that's why. I think it was Kicad or Gimp or Blender where you can drag windows onto other windows to merge them and they had to add a warning saying this will never work on Wayland because Wayland doesn't want it to work.
This is the complete opposite of those discussions. It's taking a specific quantifiable thing and measuring it, with enough information for anyone to try and reproduce the results.
It's the epitome of science, comparing it to a generic vim vs emacs flamewar which is pure subjective opinion is pretty baseless.
I mean normally this type of discussion is silly, but in playing competitive shooters latency does make a huge difference, and it shows that XWayland is adding ~4ms of latency.
There is a native Wayland driver for Wine/Proton but it's enabled through an environment variable, not by default. This will probably be default in Wine 12/Proton 12 because Valve wants to squeeze as much performance out of SteamOS as possible. The gaming mode UI runs under Valve's own Wayland compositor (gamescope) already, but games are currently in nested XWayland windows.
It's really not like Emacs/vim. I use X because it works on my setup and always has, whereas Wayland has not, despite Wayland advocates claiming it's ready and X is deprecated for 10+ years.
Only xwayland showed that result. The difference was only a couple milliseconds. That’s in the range where I start to doubt that people are feeling the latency difference. If it was 10-20ms I could believe it, but not when it’s a couple milliseconds.
The author of this post did a good job of getting all of the other confounding settings out of the way. It’s possible that the people complaining that Wayland was slow were starting from an unoptimized situation and as part of switching to some low latency variant they set all the correct settings.
You might be right, 8 ms of total end to end latency is about 1 frame at 120 hz or half a frame of 60 hz, someone would need to be quite competitive to notice that. And the baseline was 4 ms, so going from half a frame of total e2e latency at 120 hz to 1 frame, not much of a difference. Also in 2026 I'm realizing it might be doubtful that many games would still be only x11, so I'm not sure how common it would be to encounter xwayland in a game today realistically.
Isn't Wayland always one frame delayed compared to Xorg to avoid tearing or has that been changed? If so, his very high refresh rate would minimize that effect
I don't think there was ever a design to be one frame behind.
Compositing requires the GPU to do some extra work to draw the frame to be presented. This typically takes very little time (much less than a full frame period). Additionally, most wayland compositors will bypass that extra step if an application is full screen (wlroots calls it "direct scanout").
Also some wayland compositors keep track of timing and delay the final composition until right before it is time to present the frame in order to reduce latency.
You can't just test one wayland compositor and talk about the performance of all wayland compositors. They're vastly different, especially when it comes to the extensions to wayland needed to handle input devices (ref: https://wayland.app/protocols/). It's not like how xorg is the standard strong reference implementation for X11 everywhere that works the same everywhere.
What's probably happening is that other wayland compositors are slower than KDE Plasma wayland which he tested. And people report that experience. Some other wayland compositors might even be faster than plasma. But what is for sure is that every wayland is very different from every other wayland.
You will also get different results by gpu, compiler, kernel, architecture, and then of course compositor. Even a slightly different version of some lib might throw off the results.
In any case the methodology in the post is sound and should be used for benchmarking in the future.
And FWIW, KDE probably make the most effort with their compositor. They have historically been well ahead of the curve for things that might affect this (e.g. VRR).
There's no such thing as "Wayland input latency". It's just a word salad, akin to "HTTP animation smoothness". The post is measuring Xorg vs. KWin (and also XWayland), other implementations of either X11 or Wayland will have different characteristics.
I wonder where the XWayland's added latency comes from though, it seems suspiciously high to just be easily hand-waved as overhead.
Yep, OP should certainly retest under gnome. Im not in the weeds myself but I get the vibe that gnome's wayland compositor is somewhat faster and more carefully designed than kwin (at least, Emacs in pgtk mode does substantially better under gnome, whereas on Kwin it uses substantial cpu when scrolling and even lags a bit at high resolutions)
This is an annoying style of argument from Wayland fans.
Wayland fan: You need to switch to Wayland. X is deprecated and has been for years! Wayland is the future.
User: Okay, I tried, and it's broken/worse.
Wayland fan: No, you don't understand, Wayland is just a protocol. It's your implementation of Wayland that is at fault, not Wayland itself! Wayland is still great!
I appreciate the article, it's cool to see such small differences across all those settings! It says they learned to solder, if you see solder joins that look like those in the first picture, keep learning! Those are some dry joins, which can easily lead to failure, or intermittant signal loss. Soldering can be a touchy sport!
This used a 500Hz display which hides a lot of the problems that would show up on slower displays.
The XWayland result is 3ms slower, which at refresh rates this high makes me wonder if it was one frame behind.
Running the tests at 120Hz or even 60Hz might be more interesting because we could start to separate out very small differences in timing from the much larger effects of being a full frame behind.
I am not super familiar with Wayland, but basically how composited rendering under X11 works, is if you have a fullscreen window, you can give an 'unredirect' hint to the compositor, in which case when nothing else is drawing your screen, it will stop compositing, and pass your app's swapchain directly to the screen.
This is pretty much optimal, and you can't really do much better than this.
Once a stray window appears on top, or something makes the compositor think it can't do this, it'll do the intermediate step of compositing your app window with others into a temp buffer, and render that.
Sometimes the unredirect breaks for some reason (I remember a case where for some inexplicable reason my app kept creating a window 1px smaller than the screen height), or you use XWayland, you get bad latency.
Since this is a fundamental constraint, other compositors on different OSes must work like this, and you can run into issues like this as well.
Another thing - Wayland afaik started exporing 'display planes' - which are a HW feature of GPUs, that allow it to composite multiple layers together - which means the game can render at full FPS and all the windows on top will be drawn into a different plane and get composited with no ill effects - not sure if this is actually used in production yet.
It looks like consoles and PCs have settled on somewhat different gaming configurations. Consoles usually try to target a fixed output frame rate, while the resolution is often dynamic. On PCs, by contrast, the resolution stays static, while the frame rate and frame-time pacing are dynamic. How does this fit into the latency discussion?
Especially in competitive gaming, I often see people targeting frame rates way beyond their display’s refresh rate. I’m not sure whether this actually provides a real benefit or whether they’re chasing a placebo effect.
Am I out of touch, or is it the children with colored LEDs on their DRAM sticks who are wrong?
As someone who is in the rendering space for work. Having a higher framerate does help, but in a weird way. Basically the start of the frame rendering is what mostly dictates where objects are rendered. By getting a higher framerate the position of objects that you see in game are much closer to their "real" position. So it's less about seeing more frames at that point and more about seeing the most up to date information possible. Technically it could be possible to render the frame in sync with the framerate and just offset the rendering so it finishes right before it's pushed to the screen, but if you're slightly wrong you'll get really bad stuttering and the execution time of gpus and the cpu submitting the work isn't really deterministic.
As the frame rate goes up the lag between the game state and display state goes down. It doesn't matter that the "extra" frames aren't displayed, the frame that is being displayed needs to be as up to date as possible.
Consoles are more and more supporting VRR, as are televisions.
But yes, given the limitations of the hardware, they often offer two modes - a high framerate but lower quality mode and a high quality but lower framerate mode.
>I often see people targeting frame rates way beyond their display’s refresh rate. I’m not sure whether this actually provides a real benefit or whether they’re chasing a placebo effect.
Pixel refresh is only one part of latency. A higher framerate will lower several other parts of the overall latency. Monitors Unboxed has charts that visualize the amount of latency for each step.
> I often see people targeting frame rates way beyond their display’s refresh rate. I’m not sure whether this actually provides a real benefit or whether they’re chasing a placebo effect.
If they are chasing a placebo effect, it's a really powerful one, since all the actual competitive people are often willing to sacrifice all detail and quite a lot of resolution to get those stupid high frame rates.
I can see the difference too, but the diminishing returns usually make it not worth it, since I prefer the eye candy better details and higher resolutions give me.
Also, some games can adjust the resolutions on the fly to keep a consistent frame rate. It's only become a feature on modern games, but I believe that's mostly a historical accident. PC games could often run on much worse hardware than they were actually designed for (with minimum requirements often being absolute minimums, and not 'this is what we developed for'), so people played them on low frame rates, so that kind of jank was often more culturally accepted on PC, and if you didn't want that experience, you could always upgrade. While on console, there was no upgrade path, and games were optimised for that one config, and thus never allowed to drop too far into the red (and dropping resolution is often a better option in those cases).
There’s an argument to be made that dropping all unimportant detail could make professional players react faster even if it doesn’t improve end-to-end latency in the PC.
This is something that could be tested experimentally, but isn’t, because the subjects we would need to test this on are all sponsored by hardware vendors.
The games I have in mind though still have those details present on lower settings. Instead they just look like shit rather than disappear. To be fair though, that just might make those details have higher contrast and not fade into the background as much.
Framerates beyond your display's refresh rate are not completely pointless, though a bit wasteful: they do mean that each frame as it is displayed shows a more up-to-date representation of the game state than if your framerate is matched to your refresh rate. In principle you don't need to render the excess frames: ideally your frame time is predictable enough you can kick off the render just before the display refresh, but the penalty is that if you miss the deadline you get some pretty jarring jankiness.
Many PC games have dynamic resolution, too. The reason consoles target 60fps is because that’s the frame rate you get with most TVs and everyone playing the game has the same hardware (or couple variants).
> Especially in competitive gaming, I often see people targeting frame rates way beyond their display’s refresh rate. I’m not sure whether this actually provides a real benefit or whether they’re chasing a placebo effect.
A newly rendered frame can cut-in during scan out. This shows up as tearing artifacts where the frame is changed while being sent to the display, but it allows fresher pixels to hit the screen below that tearing line. So each frame on the monitor can be a mix of multiple rendered frames.
It’s not as good as having variable refresh rate display with high refresh rate, but it does reduce latency.
For less action based games it’s common to turn vsync on and pace the frames to the refresh rate to eliminate this tearing.
Depends on the game. Many shooters have always targeted 60 fps (COD / BF iirc). A single player game may well target 4k30hz unless the user selects 'performance mode'.
In the PC version Resident Evil 5, hit reg with a particular boss is tied to fps, so I had to lock it to 30. Going from 165hz to 30hz was noticeable, everything above 60 just felt a bit smoother to me but 30-60 was night and day. I rarely notice it when playing on console.
Ironically the only game where I've ever felt I had to enable performance mode was Life is Strange. Not the sort of game you'd think would suffer from 30hz!
> Especially in competitive gaming, I often see people targeting frame rates way beyond their display’s refresh rate. I’m not sure whether this actually provides a real benefit or whether they’re chasing a placebo effect.
In video games you essentially have one giant loop that runs every frame (today it's more than that, but at its core it's still that). Producing frames faster than the display’s refresh rate can still reduce input latency because the next display refresh is more likely to use a recently generated frame. It does not necessarily mean the game receives more input events, but it can process and reflect those inputs sooner.
Not placebo, but diminishing returns become significant, and the benefit depends on frame queues, VSync, VRR, whether the game is CPU- or GPU-bound, and how its input and simulation loops are designed.
Running above the display refresh rate is only decent when you are wayyyy above it, at least double. If you have a 120hz display and run something variable around 150fps, its much worse. But 500fps does help, you're basically seeing more recent frames as it scans the screen.
"Especially in competitive gaming, I often see people targeting frame rates way beyond their display’s refresh rate. I’m not sure whether this actually provides a real benefit or whether they’re chasing a placebo effect."
Depends on the engine. Anyone remember Quake 3's multiple of 3 frame rate speed hack?
Been dealing with this a bit at Breaka Club. We teach kids to code with a modded version of Overcooked 2!.
We stream OC2[1] with our mod preinstalled over WebRTC. This ensures that kids/schools don't have to try and install the mod. This is particularly important since we support running on school provided hardware. Installing a game without a mod would be hard enough. Added advantage though is kids play with a virtual (on screen) gamepad on iPads in Mobile Safari.
Game instances run in Docker containers in Kubernetes/k3s atop very outdated nVidia hardware. Given we're already going across the Internet into school networks, we've tried very hard to optimize latency across the board. Using NVidia NVEnc with DMABuf (zero copy) etc. We're unfortunately using XWayland at present so experience the documented input overhead. Although our inputs are virtual devices at this point, so the overhead may be a bit different. Trying to optimize this whole thing end to end has been a challenge. I would say that performance is currently "acceptable".
The input latency that has been driving me crazy lately is the screen unblank latency to allow input to a program.
E.g. I have an old laptop running a browser playing some internet radio stream. Eventually the screen blanker (without locking) activates.
Some real life event makes me want to hit the space bar to pause music. But the modern screen blank has decided that it should eat/ignore key presses while blank. So hitting the key doesn't pause music. I have to wait for the screen to light up before it will be possible to trigger the pause, and this delay feels interminable!
I seem to recall that in the old days the input remained active to the focused window even if the screen was in a power saving state. This power saving was not conflated with screen-lock security etc. I much prefer that. I think this was because DPMS power saving was an underlying X server behavior, not delegated to a screensaver/lock application?
I'd also be partially satisfied with the async behavior of old terminal programs. My inputs should be buffered and processed even if the effects haven't returned to the screen yet. Then I could at least hit keys twice and be trained to know that one would unblank, the other would pause, and all would be well (eventually).
The current behavior is like having a temporarily numb hand, and being frustrated waiting for sensation to return before I can operate anything!
Question, does your monitor have a KVM? Cause I definitely notice this behavior, even on windows and os x with my monitor. I believe it was the same way on linux as well but it's been a minute since I used linux on my desktop
The Rock Band guitars have a photoresistor for precisely this purpose: the screen flashes and the guitar responds when the light hits it. It helps make the otherwise very painful calibration process transparent.
It would be so cool to get that to work in Linux. I know the instrument code is in hid-sony. Here are some open tabs I've got in case anyone's curious:
Latency numbers are written with three significant digits (4.21 ms). I'm curious about the accuracy of the measurement device. If it can measure tens of microseconds, I'm impressed. If it can't, the conclusions in this article should be taken more coarsely.
Great article! Thank you. Also in case others walked away with the same question I had, I'll save you the googling: use the utility vrrtest to help validate if VRR is properly configured on your machine.
Using Wayland (hyprland) for daily driver and it's the happiest I've been with an OS in a long time on feel. Feels crisp in that spartan way that windows & macos just don't - no animations, taskbar, popups or god knows what else.
gamescope is a custom wayland compositor that Valve built for gaming...
In steam big picture mode there is no xwayland or anything, just gamescope.
What you are reading from the readme notes that it calls into xwayland only when gamescope (wayland compositor) is nested within another compositor (say kwin or mutter).
gamescope itself is wayland only, and when run on SteamOS is has no xwayland latency...
I have a vague memory of (X?)Wayland being much worse than X11 before, and some patch or protocol making it out to all the relevant implementations, but I might have imagined that, since these result show virtually no difference, and only XWayland shows a marginal difference.
Or maybe it just came out of nowhere and was never true.
I wonder though, updated pixels might not have the same latency on the whole screen, and it might even be affected by some updates mechanisms, like panel replay. I.e. it would have been interesting to also measure the screen position as a dimension.
Also, both the input latency (usb controller, and its driver), and screen latency (input latency + processing + update delay) are supposedly also affecting all measurements, but hopefully somewhat consistent or at least filtered out.
If I understand the semantics of it all correctly, in that sense, there is no such thing as "Wayland". It is all a little bit confusing compared to the X11 ecosystem.
You don't run GNOME on Wayland. You run GNOME's Wayland compositor, which is an entirely different implementation than Plasma's Wayland compositor.
I've not used gnome for years, but I have a vague memory of gnome/mutter running on a single main thread which used to lock up quite a lot (javascript etc). And because in X it was X that used to manage things like rendering the mouse pointer every frame, whereas in Wayland it flipped to mutter having to do it directly, the stalls were way more obvious in wayland than X, which is where I think a lot of this perception came from.
Again, not sure how much of this is accurate, but that's the point I was trying to make.
Clicks and mouse movement do have different latency, but it doesn't matter for this test. The click is the constant part to show relative positioning of X11/wayland/low latency/vrr/etc. The relative performance should be the same ranking for mouse movement, just with different exact values.
This is awesome. I would like to see tests like this done at 60 Hz as well, and also with non-3D apps. I suspect the results might look different in those conditions. A 500 Hz monitor is not the common case. 2ms is a whole frame!
Linux is underrated and is only getting better. I am building a linux cloud platform so we can build unity and Godot games with mcps right in your browser. Not only is Linux the only option but I have realized it is the best option for AI
I wonder what is considered "unnecessary programs" by the author. Is "apparmor" or sandboxing considered in this? Or just user space applications (browser, discord, …).
I wonder if input latency would be improved if you ran setup as `root`. I wouldn’t do it for security sake, but just curious
Yeah, this particular experiment doesn't reveal much because the Nvidia driver is a black box. We know in the past it used its "composition pipeline" in Wayland situations, and that was a particularly noticeable increase in latency. Plus, for a while the Nvidia driver also was stuck on software cursors. And even on the DRM path the big two compositors only updated input once every frame, which was the reason for it "to have much worse input lag."
edit: I should also point out the mouse acceleration curve, which if you don't fix it is different between X11 and Wayland compositors. That really messes up the "feel" of things.
AMD's Mesa drivers are better, but if you already have an Nvidia card then you can still use it just fine with Wayland.
The biggest hit is Vulkan performance (~20% less than Windows iirc) but for desktop and casual gaming use, Nvidia's proprietary drivers are perfectly fine.
Seconding this. I'm not happy with the fact that the drivers are proprietary, I really prefer FOSS. But, I am tired of having to deal with FUD around performance and issues with NVIDIA devices which simply don't exist at scale.
I have friends who are stuck on Windows not because they play games with Windows-only anticheat, but because theyve been told by GNU heads that NVIDIA drivers simply don't perform acceptably on Linux.
I'd say it's deserved, not FUD. Nvidia did improve in some ways, but it's still way too heavy into blob even in case you plan to use it in the future with nvk. Nouveau / nova developers explained that Nvidia pushed a lot more into their firmware than AMD, turning the kernel driver into more of a shim than an actual driver. That's already a big downside.
From the "Similar Efforts" section toward the bottom:
---
David Ramiro built his m2p-latency and compared X11 vs Wayland in his article Building an Input Latency Meter (Because ‘Wayland Feels Off’ Isn’t a Metric) as well, coming to similar conclusions:
Native Wayland is on par with native X11 (all tied at ~7 ms), while XWayland roughly doubled the latency in his tests.
farnoy did extensive testing with the Open-Source-LDAT in his post Linux latency measurements and compositor tuning, also concluding that XWayland should be avoided.
One thing that's lovely about Linux is this kind of analysis is not only possible, but meaningful. These results will get reported back to the graphics software authors and the distribution packagers and the ecosystem will improve. There's no sense with Microsoft that kind of improvement is possible.
I recently switched to Linux after years on Windows desktop, mostly because the KDE Plasma desktop feels snappier than Windows 11. Also the feeling that if something isn't working right I can probably tinker and improve it. It's been really nice. If you haven't tried Linux desktops in awhile give Bazzite a whirl: it's a Fedora customized for gaming. Even if you don't game it's an easy way to get a very functional Linux desktop in no time at all.
> results will get reported back
Even better, most of the tech stack is open source and contributions are welcome!
>give Bazzite a whirl
I never really understood Bazzite's immutable fs thing. Can one install standard dev stuff (i.e. compilers, ides, etc) easily under bazzite?
This use case is the main reason why I lean towards maybe using cachyos
Yes of course, but you need to do it via the package manager. You can't "sudo make install". If you build your own software and want to install it system wide, you need to flatpack/brew it or run it in a docker container.
(obviously you can modify the filesystem if you really really really want to).
Bazzite is gaming oriented version of Fedora CoreOs. There are many different versions. I am running bluefin.
I've tried a lot of desktop linux distros, and to be honest, immutable linux feels like the future. Anything you do can simply be rolled back. Break something? Just roll it back.
And if you run something like Bazzite, but want to try out Bluefin-Dx which is developer oriented, then you can rebase your existing installation. If you don't like it, just revert back to Bazzite with a single command.
However, it's desktop oriented. Don't run CoreOs on a server.
The immutable linux is pretty great if you find one that fits you type of dev work. The prebuilt images are powered by a config (so you could build your own) but it's great that you can just follow someones good "config". Switching/updates are suddenly easy.
For example i just need docker for webdev and there is bazzite-dx basically bazzite with docker and few things added. Works pretty great, sometimes when something goes bad i rollback the image and wait for future version.
I use Bazzite for development bc it works like a Mac with sane defaults and a just works approach:
- IDEs are no problem. Editors will "just work" for anything you type into the app store - Bazzite handles the special cases for you and installs them through brew taps or Flatpaks.
- For development it's basically just like a Mac where you also can't install system-level packages: Node, Python etc work through brew / nvm / uv same as on Mac. Development that involves containers will be unchanged from a Mac. For compilers specifically, same as on Mac: Install it through brew, or if you need a Debian or Fedora base you do `distrobox create` and you can apt-install in a transparent podman container.
Bazzite mostly suggests installing stuff via Flatpak or Homebrew. (No, really, brew on Linux lol). You can also layer in things with rpm-ostree but it's clunky. I think the immutable OS makes sense for a consumer but not for a developer.
So I'm using Nobara instead. It's a different Fedora-for-gaming but has most of the same improvements. It is a traditional system, not immutable. CachyOS is also very popular and that gets you an Arch-for-gaming. Just yesterday I learned of PikaOS, a Debian-for-gaming.
The main thing all these gaming-customized systems are doing is getting graphics drivers and proprietary codecs installed for you easily.
Why not for developers? You can install tools from flatpak or in home directory. My workflow includes toolbox containers. Distrobox is also good option.
Many flatpaks aren't actually maintained by the actual developer nor the normal way the package is normally used. You may have bugs that aren't present in the package that the dev isn't aware of or interested in fixing especially if they support a different channel and the bug relates to sandboxing.
There is also a risk that the person may be malicious from the start, sell out, or simply get malware. Given the nature of the ecosystem a malicious release to a previously safe package could propagate incredibly quickly.
Where there are multiple steps for a package to get from developers machine to yours and each is slow enough for malicious behavior to be noticed each step adds friction and decreases the chance of ultimate success. Where all steps are nearly simultaneous your risk multiplies with each step in which a different person has their hands in it and if any of them are malicious or compromised you are screwed.
mostly i'm trying to get out of the terminal and most IDEs don't understand devbox/distrobox well without lots of finagling.
In addition to the other comment mentioning rpm-ostree, brew is installed on bazzite by default and works about as well as it does on macOS.
You can also use fedora toolboxes to create containers mounted on your home folder, though it is clunky.
You can modify the fs using rpm-ostree. It will overlay your changes on top of the immutable image. The recommended way is to create toolbox/distrobox which uses containers to isolate stuff.
Yes, rpm-ostree solves that
https://docs.bazzite.gg/Installing_and_Managing_Software/rpm...
https://coreos.github.io/rpm-ostree/
It competes with steamos. There are other gaming distros that work like normal distros.
Some things do indeed improve, but some other things require a massive effort that no hobbyist or small company can make happen.
This is most obvious in places where a lot of coordination is required, for example in supporting proper color correction throughout all applications, or decent support for advanced printer functions.
There are many incremental changes, but we often get stuck in local minima for years.
Still, I personally like that one can (relatively) easily watch what happens under the hood. It's not entirely clear to me why Windows and MacOS must remain closed source.
Because if it's open source you can't make money from it or have multiple tiers.
This kind of thing is certainly possible under Windows - you can basically patch any kernel API call, replace any COM object instantiation, install filter drivers that intercept any request to and from a device, replace userland DLLs with your own.
It's really scary what you can do, to the point that I often asked myself 'why allow this?' - seeing as hits on certain APIs took me to blackhat forums and articles about writing exploits.
Oh sure, but if I just need to troubleshoot why Minecraft-launcher (a first party app) won't launch ... it didn't give any output; it didn't even exist apparently, but only because MS were hiding it, had to crack out Process Explorer just to get something to troubleshoot on ... then it turns out the "turnkey" app from their first-party app store, loading the first-party app, on the same company's OS just failed with no indications to the user at all, not even a "this app crashed". Solution was to cut out as much of MS as possible ... it's just infuriating when it doesn't work, which seems to be all-the-time.
More power to Bazzite and Valve, the sooner games app run in other OS the better.
The developers of Process Explorer have a lot more to do with the OS than the makers of Minecraft, even if the company name is the same on all three
I think it depends who it is making noise. There are some famous Oculus-era stories about John Carmack tracing display latency issues and then writing lengthy screeds to Samsung engineers to get them to give him lower level access.
I bet if someone like him made enough noise, people at MS would pay attention.
Not Microsoft but Battlenonsense investigations on latency did get Nvidia to create reflex.
Afaik reflex is a bit different - it times the input to frame latency in your game, and tries to start the gameloop as late as possible in the frame so that your input gets sampled at the latest possible time and the frame still makes it to the vsync period, this isn't really a compositor tech.
Why would it need to measure input to frame latency? Surely it just needs to measure how long it takes the game to go from vsync period to frame ready and delay when the game thinks vsync happened to force it to poll later in the cycle?
Awesome article.
I switched my daily driver / gaming rig to Fedora a few months back.
Everything seems snappier compared to Windows, but not sure if it’s in my head, and I’ve been very curious about gaming input latency. This helps answer some questions.
I recently switched to hyprland and I’m very interested how that fits in these results. hyprland uses Wayland so I hope the author might revisit now that hyprland is gaining in popularity.
I’ve considered using gamescope to hopefully get in front of some of these concerns, but I’m on nvidia and there is some discussion about it not working well there.
Now the author's got me thinking about gaming-optimized kernels, which I did not realize was a thing.
I play competitive fighting games so input latency is a huge concern. Would love to hear from anyone else who’s been down this path.
With the exception of XWayland, all of the tests had input latencies within a very small range. No human could tell them apart by those latency differences alone. I would be amazed if someone could notice the 3ms difference jump to XWayland.
The difference could be much larger on a slower monitor. However the differences between Wayland and X11 as protocols is negligible. XWayland as an implementation looks to have a limitation.
The context here is the article's author is a twitchy FPS player, those extra 3ms are something that community agonizes over. I appreciated how much effort was put into controlling the variables in this test. There's a whole lot of witchcraft associated with these kinds of efforts and he sat down and did the measurements and got real numbers. My hat is off to OP.
As far as I'm aware, you get marginal FPS gains switching kernels. It helps some. It's just a matter of whether the effort is worth it to you. The main change is the scheduler: rather than trying to evenly distribute CPU time to processes, it will prioritize bursty processes.
I've been a fan of Hyprland for gaming so far. Much more configurable for things like VRR/tearing and other precise tweaks via Gamescope than when I was on AwesomeWM with X11. Been especially nice having Lua for configuration, which finally feels very familiar with my AwesomeWM roots.
I moved to OpenSuse and have the same experience/opinion. The only caveat I had in Wayland is Game Streaming. Sunshine/Moonlight work but the input lag is noticeable and there are artifacts in the game. I go back and forth between X11 and Wayland just because X11 better for game streaming but in time I'm sure I'll go full-time into Wayland.
When you say game streaming, is this the steam streaming stack?
He seems confused at the end why people think wayland is so slow, but don't you think it's because of his xwayland result? People were probably running x11 games on wayland and noticed that significant lag. Just a wild guess. Very nice article, wish people did actual measurements like this more often, of all sorts of things.
I've been using Linux since the mid 1990's. I'm no newbie to any of this. I literally can't tell the different between X11 and Wayland when using either of them and I don't care about all the arguing. This is just Vim vs Emacs and Gnome vs KDE all over again. At this point when I see people complaining about it I just click off the page. It's all stupid and pointless.
My biggest problem with wayland was how it was basically forced on the community. It broke innumerable things for years, put all the responsibility for implementing things down on the DEs and WMs themselves.
All of this hassle, forcing so much more work on DE/WM devs, for the sake of 'better security' in scenarios that don't really apply to 99% of linux users, with the promise of 'better latency' which this very article proves is false.
I tried to be an early adopter of wayland ~ 5 years ago. Found all sorts of things broken, and I'm now using linux mint xfce edition, as hopefully by the time xfce drags itself to wayland, all the bugs and tooling will be a solved problem.
And the attitude of just refusing to make things work because haha fuck you that's why. I think it was Kicad or Gimp or Blender where you can drag windows onto other windows to merge them and they had to add a warning saying this will never work on Wayland because Wayland doesn't want it to work.
By the time wayland becomes the default in xfce, maybe the wayland team will have added window positioning to the protocol...
This is the complete opposite of those discussions. It's taking a specific quantifiable thing and measuring it, with enough information for anyone to try and reproduce the results.
It's the epitome of science, comparing it to a generic vim vs emacs flamewar which is pure subjective opinion is pretty baseless.
You can measure a 3.3ms difference, but whether someone will notice and care is a different thing.
I mean normally this type of discussion is silly, but in playing competitive shooters latency does make a huge difference, and it shows that XWayland is adding ~4ms of latency.
There is a native Wayland driver for Wine/Proton but it's enabled through an environment variable, not by default. This will probably be default in Wine 12/Proton 12 because Valve wants to squeeze as much performance out of SteamOS as possible. The gaming mode UI runs under Valve's own Wayland compositor (gamescope) already, but games are currently in nested XWayland windows.
It's really not like Emacs/vim. I use X because it works on my setup and always has, whereas Wayland has not, despite Wayland advocates claiming it's ready and X is deprecated for 10+ years.
> and noticed that significant lag
Only xwayland showed that result. The difference was only a couple milliseconds. That’s in the range where I start to doubt that people are feeling the latency difference. If it was 10-20ms I could believe it, but not when it’s a couple milliseconds.
The author of this post did a good job of getting all of the other confounding settings out of the way. It’s possible that the people complaining that Wayland was slow were starting from an unoptimized situation and as part of switching to some low latency variant they set all the correct settings.
You might be right, 8 ms of total end to end latency is about 1 frame at 120 hz or half a frame of 60 hz, someone would need to be quite competitive to notice that. And the baseline was 4 ms, so going from half a frame of total e2e latency at 120 hz to 1 frame, not much of a difference. Also in 2026 I'm realizing it might be doubtful that many games would still be only x11, so I'm not sure how common it would be to encounter xwayland in a game today realistically.
Yeah at 120hz each frame is 8ms. So you're only missing a single frame around 30% of the time.
I certainly want my latency as low as I can get it. But I'm pretty skeptical that anyone is truly feeling the difference of a couple ms.
Try musicians that are used to playing extremely high speed semihemidemisemiquavers.
We notice latency. Neil Peart could almost get sample-precise timing, he was so godlike.
Isn't Wayland always one frame delayed compared to Xorg to avoid tearing or has that been changed? If so, his very high refresh rate would minimize that effect
I don't think there was ever a design to be one frame behind.
Compositing requires the GPU to do some extra work to draw the frame to be presented. This typically takes very little time (much less than a full frame period). Additionally, most wayland compositors will bypass that extra step if an application is full screen (wlroots calls it "direct scanout").
Also some wayland compositors keep track of timing and delay the final composition until right before it is time to present the frame in order to reduce latency.
You can't just test one wayland compositor and talk about the performance of all wayland compositors. They're vastly different, especially when it comes to the extensions to wayland needed to handle input devices (ref: https://wayland.app/protocols/). It's not like how xorg is the standard strong reference implementation for X11 everywhere that works the same everywhere.
What's probably happening is that other wayland compositors are slower than KDE Plasma wayland which he tested. And people report that experience. Some other wayland compositors might even be faster than plasma. But what is for sure is that every wayland is very different from every other wayland.
You will also get different results by gpu, compiler, kernel, architecture, and then of course compositor. Even a slightly different version of some lib might throw off the results.
In any case the methodology in the post is sound and should be used for benchmarking in the future.
And FWIW, KDE probably make the most effort with their compositor. They have historically been well ahead of the curve for things that might affect this (e.g. VRR).
There's no such thing as "Wayland input latency". It's just a word salad, akin to "HTTP animation smoothness". The post is measuring Xorg vs. KWin (and also XWayland), other implementations of either X11 or Wayland will have different characteristics.
I wonder where the XWayland's added latency comes from though, it seems suspiciously high to just be easily hand-waved as overhead.
Yep, OP should certainly retest under gnome. Im not in the weeds myself but I get the vibe that gnome's wayland compositor is somewhat faster and more carefully designed than kwin (at least, Emacs in pgtk mode does substantially better under gnome, whereas on Kwin it uses substantial cpu when scrolling and even lags a bit at high resolutions)
This is an annoying style of argument from Wayland fans.
Wayland fan: You need to switch to Wayland. X is deprecated and has been for years! Wayland is the future.
User: Okay, I tried, and it's broken/worse.
Wayland fan: No, you don't understand, Wayland is just a protocol. It's your implementation of Wayland that is at fault, not Wayland itself! Wayland is still great!
User: But X was working fine...
I appreciate the article, it's cool to see such small differences across all those settings! It says they learned to solder, if you see solder joins that look like those in the first picture, keep learning! Those are some dry joins, which can easily lead to failure, or intermittant signal loss. Soldering can be a touchy sport!
This used a 500Hz display which hides a lot of the problems that would show up on slower displays.
The XWayland result is 3ms slower, which at refresh rates this high makes me wonder if it was one frame behind.
Running the tests at 120Hz or even 60Hz might be more interesting because we could start to separate out very small differences in timing from the much larger effects of being a full frame behind.
I am not super familiar with Wayland, but basically how composited rendering under X11 works, is if you have a fullscreen window, you can give an 'unredirect' hint to the compositor, in which case when nothing else is drawing your screen, it will stop compositing, and pass your app's swapchain directly to the screen.
This is pretty much optimal, and you can't really do much better than this.
Once a stray window appears on top, or something makes the compositor think it can't do this, it'll do the intermediate step of compositing your app window with others into a temp buffer, and render that.
Sometimes the unredirect breaks for some reason (I remember a case where for some inexplicable reason my app kept creating a window 1px smaller than the screen height), or you use XWayland, you get bad latency.
Since this is a fundamental constraint, other compositors on different OSes must work like this, and you can run into issues like this as well.
Another thing - Wayland afaik started exporing 'display planes' - which are a HW feature of GPUs, that allow it to composite multiple layers together - which means the game can render at full FPS and all the windows on top will be drawn into a different plane and get composited with no ill effects - not sure if this is actually used in production yet.
It looks like consoles and PCs have settled on somewhat different gaming configurations. Consoles usually try to target a fixed output frame rate, while the resolution is often dynamic. On PCs, by contrast, the resolution stays static, while the frame rate and frame-time pacing are dynamic. How does this fit into the latency discussion?
Especially in competitive gaming, I often see people targeting frame rates way beyond their display’s refresh rate. I’m not sure whether this actually provides a real benefit or whether they’re chasing a placebo effect.
Am I out of touch, or is it the children with colored LEDs on their DRAM sticks who are wrong?
As someone who is in the rendering space for work. Having a higher framerate does help, but in a weird way. Basically the start of the frame rendering is what mostly dictates where objects are rendered. By getting a higher framerate the position of objects that you see in game are much closer to their "real" position. So it's less about seeing more frames at that point and more about seeing the most up to date information possible. Technically it could be possible to render the frame in sync with the framerate and just offset the rendering so it finishes right before it's pushed to the screen, but if you're slightly wrong you'll get really bad stuttering and the execution time of gpus and the cpu submitting the work isn't really deterministic.
As the frame rate goes up the lag between the game state and display state goes down. It doesn't matter that the "extra" frames aren't displayed, the frame that is being displayed needs to be as up to date as possible.
Yes, thanks for saying it more concisely/clearly than I did. Just cause I understand something doesn't mean I'm good at explaining it.
Consoles are more and more supporting VRR, as are televisions.
But yes, given the limitations of the hardware, they often offer two modes - a high framerate but lower quality mode and a high quality but lower framerate mode.
>I often see people targeting frame rates way beyond their display’s refresh rate. I’m not sure whether this actually provides a real benefit or whether they’re chasing a placebo effect.
Pixel refresh is only one part of latency. A higher framerate will lower several other parts of the overall latency. Monitors Unboxed has charts that visualize the amount of latency for each step.
> I often see people targeting frame rates way beyond their display’s refresh rate. I’m not sure whether this actually provides a real benefit or whether they’re chasing a placebo effect.
If they are chasing a placebo effect, it's a really powerful one, since all the actual competitive people are often willing to sacrifice all detail and quite a lot of resolution to get those stupid high frame rates.
I can see the difference too, but the diminishing returns usually make it not worth it, since I prefer the eye candy better details and higher resolutions give me.
Also, some games can adjust the resolutions on the fly to keep a consistent frame rate. It's only become a feature on modern games, but I believe that's mostly a historical accident. PC games could often run on much worse hardware than they were actually designed for (with minimum requirements often being absolute minimums, and not 'this is what we developed for'), so people played them on low frame rates, so that kind of jank was often more culturally accepted on PC, and if you didn't want that experience, you could always upgrade. While on console, there was no upgrade path, and games were optimised for that one config, and thus never allowed to drop too far into the red (and dropping resolution is often a better option in those cases).
There’s an argument to be made that dropping all unimportant detail could make professional players react faster even if it doesn’t improve end-to-end latency in the PC.
This is something that could be tested experimentally, but isn’t, because the subjects we would need to test this on are all sponsored by hardware vendors.
A fair argument at least for some games.
The games I have in mind though still have those details present on lower settings. Instead they just look like shit rather than disappear. To be fair though, that just might make those details have higher contrast and not fade into the background as much.
Framerates beyond your display's refresh rate are not completely pointless, though a bit wasteful: they do mean that each frame as it is displayed shows a more up-to-date representation of the game state than if your framerate is matched to your refresh rate. In principle you don't need to render the excess frames: ideally your frame time is predictable enough you can kick off the render just before the display refresh, but the penalty is that if you miss the deadline you get some pretty jarring jankiness.
Many PC games have dynamic resolution, too. The reason consoles target 60fps is because that’s the frame rate you get with most TVs and everyone playing the game has the same hardware (or couple variants).
> Especially in competitive gaming, I often see people targeting frame rates way beyond their display’s refresh rate. I’m not sure whether this actually provides a real benefit or whether they’re chasing a placebo effect.
A newly rendered frame can cut-in during scan out. This shows up as tearing artifacts where the frame is changed while being sent to the display, but it allows fresher pixels to hit the screen below that tearing line. So each frame on the monitor can be a mix of multiple rendered frames.
It’s not as good as having variable refresh rate display with high refresh rate, but it does reduce latency.
For less action based games it’s common to turn vsync on and pace the frames to the refresh rate to eliminate this tearing.
Consoles target 30fps though. Very rarely they target 60fps and even less rarely they can reach that.
That's outdated information. Most PS5 and the equivalent Xbox games target 60 fps nowadays. It's usually huge news when a game is only 30 fps.
Well, new GTA is 30 fps only. Some (most?) games have performance vs quality toggle. Quality mode being 30 fps, and performance mode looks too ugly.
Sure "only 30 fps" is big news, but pretty sure "quality mode target 30 fps" is still norm.
In Xbox, many games launched at 30 fps only, then gained 60 fps mode.
Until I see majority target at least 60 fps as minimal mode, my point IMO stands.
Depends on the game. Many shooters have always targeted 60 fps (COD / BF iirc). A single player game may well target 4k30hz unless the user selects 'performance mode'.
In the PC version Resident Evil 5, hit reg with a particular boss is tied to fps, so I had to lock it to 30. Going from 165hz to 30hz was noticeable, everything above 60 just felt a bit smoother to me but 30-60 was night and day. I rarely notice it when playing on console.
Ironically the only game where I've ever felt I had to enable performance mode was Life is Strange. Not the sort of game you'd think would suffer from 30hz!
> Especially in competitive gaming, I often see people targeting frame rates way beyond their display’s refresh rate. I’m not sure whether this actually provides a real benefit or whether they’re chasing a placebo effect.
In video games you essentially have one giant loop that runs every frame (today it's more than that, but at its core it's still that). Producing frames faster than the display’s refresh rate can still reduce input latency because the next display refresh is more likely to use a recently generated frame. It does not necessarily mean the game receives more input events, but it can process and reflect those inputs sooner.
Not placebo, but diminishing returns become significant, and the benefit depends on frame queues, VSync, VRR, whether the game is CPU- or GPU-bound, and how its input and simulation loops are designed.
Running above the display refresh rate is only decent when you are wayyyy above it, at least double. If you have a 120hz display and run something variable around 150fps, its much worse. But 500fps does help, you're basically seeing more recent frames as it scans the screen.
> I often see people targeting frame rates way beyond their display’s refresh rate
Could this be to reduce input lag?
"Especially in competitive gaming, I often see people targeting frame rates way beyond their display’s refresh rate. I’m not sure whether this actually provides a real benefit or whether they’re chasing a placebo effect."
Depends on the engine. Anyone remember Quake 3's multiple of 3 frame rate speed hack?
Been dealing with this a bit at Breaka Club. We teach kids to code with a modded version of Overcooked 2!.
We stream OC2[1] with our mod preinstalled over WebRTC. This ensures that kids/schools don't have to try and install the mod. This is particularly important since we support running on school provided hardware. Installing a game without a mod would be hard enough. Added advantage though is kids play with a virtual (on screen) gamepad on iPads in Mobile Safari.
Game instances run in Docker containers in Kubernetes/k3s atop very outdated nVidia hardware. Given we're already going across the Internet into school networks, we've tried very hard to optimize latency across the board. Using NVidia NVEnc with DMABuf (zero copy) etc. We're unfortunately using XWayland at present so experience the documented input overhead. Although our inputs are virtual devices at this point, so the overhead may be a bit different. Trying to optimize this whole thing end to end has been a challenge. I would say that performance is currently "acceptable".
OC2 coding: https://www.youtube.com/watch?v=ITWSL5lTLig (not streamed in this case)
[1] We've bought a limited number of copies of OC2 and pods claim a license on startup. If we're at capacity, kids play something else.
The input latency that has been driving me crazy lately is the screen unblank latency to allow input to a program.
E.g. I have an old laptop running a browser playing some internet radio stream. Eventually the screen blanker (without locking) activates.
Some real life event makes me want to hit the space bar to pause music. But the modern screen blank has decided that it should eat/ignore key presses while blank. So hitting the key doesn't pause music. I have to wait for the screen to light up before it will be possible to trigger the pause, and this delay feels interminable!
I seem to recall that in the old days the input remained active to the focused window even if the screen was in a power saving state. This power saving was not conflated with screen-lock security etc. I much prefer that. I think this was because DPMS power saving was an underlying X server behavior, not delegated to a screensaver/lock application?
I'd also be partially satisfied with the async behavior of old terminal programs. My inputs should be buffered and processed even if the effects haven't returned to the screen yet. Then I could at least hit keys twice and be trained to know that one would unblank, the other would pause, and all would be well (eventually).
The current behavior is like having a temporarily numb hand, and being frustrated waiting for sensation to return before I can operate anything!
Question, does your monitor have a KVM? Cause I definitely notice this behavior, even on windows and os x with my monitor. I believe it was the same way on linux as well but it's been a minute since I used linux on my desktop
This is actually a laptop with its builtin display.
The Rock Band guitars have a photoresistor for precisely this purpose: the screen flashes and the guitar responds when the light hits it. It helps make the otherwise very painful calibration process transparent.
It would be so cool to get that to work in Linux. I know the instrument code is in hid-sony. Here are some open tabs I've got in case anyone's curious:
- https://pascal.giard.info/techreports/nguyen-daniel-autocali...
- https://www.niangames.com/articles/reverse-engineering-rockb...
- https://github.com/torvalds/linux/blob/master/drivers/hid/hi...
These results are way better than what Dan Luu measured almost a decade ago:
https://danluu.com/input-lag/
Very interesting!
Latency numbers are written with three significant digits (4.21 ms). I'm curious about the accuracy of the measurement device. If it can measure tens of microseconds, I'm impressed. If it can't, the conclusions in this article should be taken more coarsely.
This website uses IP address blocking to censor itself, and deserves to be publicly shamed for it. Here's an archive link: https://archive.is/hrYZ6
Really hope to see a Windows result. What if Windows has a latency of 10 ms+. xD
Great article! Thank you. Also in case others walked away with the same question I had, I'll save you the googling: use the utility vrrtest to help validate if VRR is properly configured on your machine.
Using Wayland (hyprland) for daily driver and it's the happiest I've been with an OS in a long time on feel. Feels crisp in that spartan way that windows & macos just don't - no animations, taskbar, popups or god knows what else.
I guess you'd feel the same in i3 or Xmonad as well.
That’s interesting. Hyprland, spartan? It’s the fancy everything-and-the-kitchen-sink tiling WM, isn’t it?
SteamOS and Bazzite both use Xwayland through gamescope which is the worst option as it seems https://github.com/ValveSoftware/gamescope
>Avoid XWayland. It added 3.13 ms of latency, more than all other effects combined.
gamescope is a custom wayland compositor that Valve built for gaming... In steam big picture mode there is no xwayland or anything, just gamescope.
What you are reading from the readme notes that it calls into xwayland only when gamescope (wayland compositor) is nested within another compositor (say kwin or mutter).
gamescope itself is wayland only, and when run on SteamOS is has no xwayland latency...
I'm not sure I understand. If X11 software requires a X11 server and you have a Wayland compositor, how do you "avoid" XWayland?
Yes, we know wayland is not only slower but also with much less features.
I have a vague memory of (X?)Wayland being much worse than X11 before, and some patch or protocol making it out to all the relevant implementations, but I might have imagined that, since these result show virtually no difference, and only XWayland shows a marginal difference.
Or maybe it just came out of nowhere and was never true.
I wonder though, updated pixels might not have the same latency on the whole screen, and it might even be affected by some updates mechanisms, like panel replay. I.e. it would have been interesting to also measure the screen position as a dimension.
Also, both the input latency (usb controller, and its driver), and screen latency (input latency + processing + update delay) are supposedly also affecting all measurements, but hopefully somewhat consistent or at least filtered out.
A lot of people conflate Wayland being worse than X11 with Gnome on Wayland being worse than Gnome on X11.
Wayland has been great for me for a few years now. I don't use Gnome or nvidia though.
If I understand the semantics of it all correctly, in that sense, there is no such thing as "Wayland". It is all a little bit confusing compared to the X11 ecosystem.
You don't run GNOME on Wayland. You run GNOME's Wayland compositor, which is an entirely different implementation than Plasma's Wayland compositor.
That's what I mean though.
I've not used gnome for years, but I have a vague memory of gnome/mutter running on a single main thread which used to lock up quite a lot (javascript etc). And because in X it was X that used to manage things like rendering the mouse pointer every frame, whereas in Wayland it flipped to mutter having to do it directly, the stalls were way more obvious in wayland than X, which is where I think a lot of this perception came from.
Again, not sure how much of this is accurate, but that's the point I was trying to make.
The main issue is with mouse movement rather than clicks tho...
Clicks and mouse movement do have different latency, but it doesn't matter for this test. The click is the constant part to show relative positioning of X11/wayland/low latency/vrr/etc. The relative performance should be the same ranking for mouse movement, just with different exact values.
This is awesome. I would like to see tests like this done at 60 Hz as well, and also with non-3D apps. I suspect the results might look different in those conditions. A 500 Hz monitor is not the common case. 2ms is a whole frame!
I wonder how FreeBSD X11 compares to Linux X11
I saw a very similar post a month or two ago, is this the same author?
edit: no, this is the one I was remembering: https://farnoy.dev/posts/linux-latency
Linux is underrated and is only getting better. I am building a linux cloud platform so we can build unity and Godot games with mcps right in your browser. Not only is Linux the only option but I have realized it is the best option for AI
Wow, love this!! This is what makes HN great!
How does Windows compare to this?
Why isn't Wayland better than X11?
It's defective by design, in a DRM kind of way. It's designed to lack features and make them difficult to implement.
Amazing work. Thank you for putting this together.
This is why I read Hacker News. Thank you.
Very interesting analysis and setup.
I wonder what is considered "unnecessary programs" by the author. Is "apparmor" or sandboxing considered in this? Or just user space applications (browser, discord, …).
I wonder if input latency would be improved if you ran setup as `root`. I wouldn’t do it for security sake, but just curious
> A lot of people still use X11 over Wayland because Wayland is said to have much worse input lag
Wayland is fine. People should use AMD and KDE Plasma.
I'd avoid Nvidia to begin with.
Yeah, this particular experiment doesn't reveal much because the Nvidia driver is a black box. We know in the past it used its "composition pipeline" in Wayland situations, and that was a particularly noticeable increase in latency. Plus, for a while the Nvidia driver also was stuck on software cursors. And even on the DRM path the big two compositors only updated input once every frame, which was the reason for it "to have much worse input lag."
edit: I should also point out the mouse acceleration curve, which if you don't fix it is different between X11 and Wayland compositors. That really messes up the "feel" of things.
AMD's Mesa drivers are better, but if you already have an Nvidia card then you can still use it just fine with Wayland.
The biggest hit is Vulkan performance (~20% less than Windows iirc) but for desktop and casual gaming use, Nvidia's proprietary drivers are perfectly fine.
Seconding this. I'm not happy with the fact that the drivers are proprietary, I really prefer FOSS. But, I am tired of having to deal with FUD around performance and issues with NVIDIA devices which simply don't exist at scale.
I have friends who are stuck on Windows not because they play games with Windows-only anticheat, but because theyve been told by GNU heads that NVIDIA drivers simply don't perform acceptably on Linux.
Firefox is software rendered on linux-nvidia which leads to a bizarre situation of simple web animations lagging on decent computers.
I'd say it's deserved, not FUD. Nvidia did improve in some ways, but it's still way too heavy into blob even in case you plan to use it in the future with nvk. Nouveau / nova developers explained that Nvidia pushed a lot more into their firmware than AMD, turning the kernel driver into more of a shim than an actual driver. That's already a big downside.
From the "Similar Efforts" section toward the bottom:
---
David Ramiro built his m2p-latency and compared X11 vs Wayland in his article Building an Input Latency Meter (Because ‘Wayland Feels Off’ Isn’t a Metric) as well, coming to similar conclusions:
Native Wayland is on par with native X11 (all tied at ~7 ms), while XWayland roughly doubled the latency in his tests.
farnoy did extensive testing with the Open-Source-LDAT in his post Linux latency measurements and compositor tuning, also concluding that XWayland should be avoided.
X11 is a protocol. Xorg is an end of life'd project run by the Wayland team.
Xlibre is an actively developed and maintained X11 protocol display server.
Xfree86 is dead, long live Xorg. Xorg is dead, long live Xlibre!