Yeah, I was thinking nix will be probably one of the first things that can easily adapt Fil-C as it already packages in a way that allows different packages to be completely independent of each other, thus Fil-C's ABI compatibility does not matter. I assume other targets will be mostly enterprise distros where the perf hit and source compatibility issues are less of a concern, and memory safety is absolutely critical.
Fil-C compiled flatpaks might be a interesting target as well for normal desktop users. (e.g. running a browser)
I wonder if GPU graphics are possible in Fil-C land? Perhaps only if whole mesa stack is compiled using Fil-C as well, limiting GPU use to open drivers?
That is super cool, and I will probably start running it on at least a test box shortly.
How does python work? Of course I can just add filc.python to my system, but if I `python3 -m pip install whatever` will it just rebuild any C modules with the fil-c compiler?
Right now Python simply does not work at all. I was trying last night just to get the compiler to build, but ran into some Fil-C panics at the last moment when it tries to "freeze" some modules? But it should work.
My idea is to move towards defining Fil-C as a "platform", meaning it would have its own ABI value, so it would look like a target called "x86_64-unknown-linux-filc", and then the magic of Nixpkgs is that it can instantiate a full realm called "pkgsCross.filc" which automatically has every package, whose packages are built on your ordinary non-FilC platform as a cross compilation.
When that works—I hope it works, I probably need help to get it working—you should be able to use all the Nixpkgs packaging helpers like
This is already how Nixpkgs can have a pkgsCross.musl that builds everything with Musl as a cross compilation. So it seems reasonable and possible. It should be of wide interest for Nix users, since it would suddenly allow memory safety for whole swaths of the C/C++ packages, it would let NixOS use memory safe builds for core services like OpenSSHd, and so on...
I should probably try to gather some interest among people who are truly familiar with Nixpkgs cross compilation...
Yes filc as abi makes most sense. Do note that while fil-c seems to have pretty good source compatibility some stuff still require patches (like python!)
I don't know how it's going on the nixpkgs side, but my impression from Alpine is that it's pretty normal to carry some patches for any ABI/platform that isn't GNU.
Either Fil-C or a different implementation of the same idea seems essential to me. A great deal of software has been written in C, and without some way of running it, we lose access to that intellectual heritage. But pervasive security vulnerabilities mean that the traditional "YOLO" approach to C compilation is a bad idea for software that has to handle untrusted input, such as Web browsing or email.
Pizlo seems to have found an astonishingly cheap way to do the necessary pointer checking, which hopefully I will be able to understand after more study. (The part I'm still confused about is how InvisiCaps work with memcpy.)
tialaramex points out that we shouldn't expect C programmers to be excited about Fil-C. The point tialaramex mentions is "DWIM", like, accessing random memory and executing in constant time, but I think most C programmers won't be willing to take a 4× performance hit. After all, if they wanted to be using a slow language, they wouldn't be writing their code in C. But I think that's the wrong place to look for interest: Fil-C's target audience is users of C programs, not authors of C programs. We want the benefits of security and continued usage of existing working codebases, without having to pay the cost to rewrite everything in Rust or TypeScript or whatever. And for many of us, much of the time, the performance hit may be acceptable.
It's poor form to put the word "secure" (or any derivative of it) into a product that aims to provide more security.
Reason: there's no solution to the security problem. Fil-C doesn't solve security. Does it make things more secure? Yes! But there will always be more security issues. So imagine if I had called it "Secure C" (and then had a sexy compiler), and 10 years from now someone finds a comprehensive solution to the string injection problem in Secure C. What do they call their thing? Securer C? Secure C Pro? Secure Secure C?
I agree. I'd add that security is always relative to not just an existing level of security but to a threat model. There are threat models relative to which Fil-C doesn't make things more secure. (I can't think of one under which Fil-C makes things less secure, though.)
A similar criticism applies to "new". Newcastle is named after a castle built 945 years ago. Neuchâtel is named after a castle built 1014 years ago. Xavier (from Basque "etxeberri", "new house") is named after a castle built in the 10th century. Windows NT "new technology", etc.
Forgive me for not putting in the effort this deserves, but the description of pointers in this thread reads a lot like the current description of Fil-C pointers from the main site.
What is the 64-bit-presenting representation of pointers in Fil-C?
That is, what does %p return and how does that work as a pointer?
Re-reading tfm, it definitely had an answer to my dumb question(basically the same as any regular C pointer), but I still get the feeling that the explanations make jumps that may be obvious to you but absolutely not to me.
I sort of get a sense of what the flight pointers do after reading the linked page with the assembly listings( which on mobile looks like you're discussing hex dumps until one tries side scrolling, btw ) but my brain would expect something along the lines of: "this is the flight pointer and this is how it is managed for local variables, global variables, etc. This part of it is what your C program sees" at the beginning of the invisicaps article rather than "pointers are 64 bit like in regular C proceeds to talk about a tuple containing two pointers"(at least that's how it reads to me).
Yeah, fat pointers are definitely a viable approach, but a lot of the existing C code that is the main argument for Fil-C assumes it can put a pointer in a long. (Most of the C code that assumed you could put it in an int has been flushed out by now, but that was a big problem when the Alpha came out.) I'm guessing that the amount of existing C code in Apple's bootloader is minimal, maybe 1000 lines, not the billions of lines you can compile with Fil-C.
You’re off by a few orders of magnitude. I’ll grant you, what is the bootloader becomes a very complex question. Even if you scope it to just “what is the code physically etched into the chip as the mask ROM” (secureROM) you’re talking hundreds of thousands. If you’re talking about all the code that runs before the kernel starts executing you’re talking hundreds of millions.
No, I was only talking about the pre-existing C code that wasn't written for the bootloader, which therefore might have incompatibilities with fat pointers you had to hunt down and fix.
Also I'm really skeptical about your "hundreds of millions" number, even if we're talking about all the code that runs before the kernel starts. How do you figure? The entire Linux kernel doesn't contain a hundred millions of lines of code, and that includes all the drivers for network cards, SCSI controllers, and multiport serial boards that nobody's made in 30 years, plus ports to Alpha, HP PA-RISC, Loongson, Motorola 68000, and another half-dozen architectures. All of that contains maybe 30 million lines. glibc is half a million. Firefox 140.4.0esr is 33 million. You're saying that the bootloader is six times the size of Firefox?
Are you really suggesting that tens of gigabytes of source code are compiled into the bootloader? That would make the bootloader at least a few hundred megabytes of executable code, probably gigabytes, wouldn't it?
Maybe so; I haven't tried. Probably a lot less code depends on unsigned long wrapping at 2⁶⁴ than used to depend on unsigned int wrapping at 2¹⁶, and we got past that. But stability standards were lower then. Any code that runs on both 32-bit and 64-bit LP64 systems can't be too dependent on the exact sizeof long, and sizeof long already isn't sizeof int the way it was on 32-bit platforms.
For all the wrong code that assumes long can store a pointer, there's likely a lot more wrong code that assumes long can fit in 64 bits, especially when serializing it for I/O and other-language interop.
Also, 128-bit integers can't fit in standard registers on most architectures, and don't have the full suite of ALU operations either. So you're looking at some serious code bloat and slowdowns for code using longs.
You've also now got no "standard" C type (char, short, int, long, long long) which is the "native" size of the architecture. You could widen int too, but a lot of code also assumes an int can fit in 32 bits.
Ok, I think I follow. You'd widen the type under the hood but not expose this fact to user code.
However, most longs are just numbers that have no metadata. I guess you'd set the metadata portion to all zeroes in that case. This feels like a reified version of Rust's pointer provenance, and I think you would have to expose some metadata-aware operations to the user. In which case, you're inviting some code rewrites anyway.
While not as bad as the register/ALU ops issue, you're still making all code pay a storage size penalty, and still adding some overhead to handle the metadata propagating through arithmetic operations, just to accommodate bad code, plus it complicates alignment and FFI.
It would still be exposed to user code that checks its size with sizeof, but yeah the long would only have numerical values between 2^-63 and 2^63-1.
And yes, there would still be some overhead for storing and propagating the metadata, and struct alignment would change and FFI wouldn't work with longs.
There's a lot of code that makes assumptions about the number of bytes in a long rather than diligently using sizeof ... remember, the whole point here is low quality code.
You don't even need to reverse it. It's in the public clang, and I'm working on helping my team adopt it in some test cases.
And it's not just the bounds-checking that's great -- it makes a bunch of C anti-patterns much harder, and it makes you think a lot harder about pointer ownership and usage. Really a great addition to the language, and it's source-compatible with empty macro-definitions (with two exceptions).
It or something similar has apparently been upstreamed to clang as -fbounds-safety. I don't know if they're the same, but the RFC on -fbounds-safety does give some credit to the author of Fil-C, who also took credit for firebloom on this thread as well.
Also this is _de facto_ limited to userspace application for the mainstream OSes if my understanding is correct.
Reading Fil-C website's "InvisiCaps by example" page, I see that "Laundering Integers As Pointers" is disallowed. This essentially disqualifies Fil-C for low-level work, which makes for a substantial part of C programs.
(int2ptr for MMIO/pre-allocated memory is in theory UB, in practice just fine as long as you don't otherwise break aliasing rules (and lifetime rules in C++) - as the compiler will fail to track provenance at least once).
But that isn't really what Fil-C is aimed at - the value is, as you implied, in hardening userspace applications.
Fil-C already allows memory mapped I/O in the form of mmap.
The only thing missing that is needed for kernel level MMIO is a way to forge a capability. I don’t allow that right now, but that’s mostly a policy decision. It also falls out from the fact that InvisiCaps optimize the lower by having it double as a pointer to the top of the capability. That’s also not fundamental; it’s an implementation choice.
It’s true that InvisiCaps will always disallow int to ptr casts, in the sense that you get a pointer with no capability. You’d want MMIO code to have some intrinsic like `zunsafe_forge_ptr` that clearly calls out what’s happening and then you’d use that wherever you define your memory mapped registers.
Can you "launder" pointers through integers just to do things like drop `const`? It's a very common pattern to have to drop attributes like `const` due to crappy APIs: `const foo a = ...; foo b = (foo *)(uintptr_t)a;`
Hopefully Pizlo will correct me if I get this wrong, but I don't think Fil-C's pointer tagging enforces constness, which isn't needed for C in any case. This C code compiles with no warnings and outputs "Howlong\n" with GCC 12.2.0-14 -ansi -pedantic -Wall -Wextra:
Somewhat to my surprise, it still compiles successfully with no warnings as C++ (renaming to deconst.cc and compiling with g++). I don't know C++ that well, since I've only been using it for 35 years, which isn't nearly long enough to learn the whole language unless you write a compiler for it.
Same results with Debian clang (and clang++) version 14.0.6 with the same options.
Of course, if you change c[] to *c, it will segfault. But it still compiles successfully without warnings.
Laundering your pointer through an integer is evidently not necessary.
> > I don't know C++ that well, since I've only been using it for 35 years, which isn't nearly long enough to learn the whole language unless you write a compiler for it.
No one person could write a compiler for it, and even if they could they would forget as much in doing so as they could learn.
As kragen already posted, you can cast from const-pointer to non-const directly.
Not allowing a cast from integer to pointer is the point of having pointers as capabilities in the first place.
Central in that idea of capabilities is that you can only narrow privileges, never widen them.
An intptr_t would in-effect be a capability narrowed to be used only for hashing and comparison, with the right for reading and writing through it stripped away.
BTW, if you would store the uintptr_t then it would lose its notion of being a pointer, and Fil-C's garbage collector would not be able to trace it.
The C standard allows casts both ways, but the [u]intptr_t types are optional. However, C on hardware capability architectures' (CHERI, Elbrus, I dunno about AS/400) tend to make the type available anyway because the one-way cast is so common in real-world code.
If the laundering through integers is syntactically obvious - obvious that the cast back from int used a int that obviously can from a pointer - then I allow it.
I'll preface this by saying my experience is with embedded, not kernel, but I can't imagine MMIO is significantly different.
There would still be ways to make it work with a more restricted intrinsic, if you didn't want to open up the ability for full pointer forging. At a high level, you're basically just saying "This struct exists at this constant physical address, and doesn't need initialisation". I could imagine a "#define UART zmmio_ptr(UART_Type, 0x1234)" - which perhaps requires a compile time constant address. Alternatively, it's not uncommon for embedded compilers to have a way to force a variable to a physical address, maybe you'd write something like "UART_Type UART __at(0x1234);". I believe this is technically already possible using sections, it's just a massive pain creating one section per struct for dozens and dozens.
Unfortunately the way existing code does it is pretty much always "#define UART ((UART_Type*)0x1234)". I feel like trying to identify this pattern is probably too risky a heuristic, so source code edits seem required to me.
I'm curious, what's your strategy for integrating the GC with low-level code? I've been thinking about trying to use it for Arduino development. Mostly as a thought experiment for now (as I'm playing with Rust on RP2040).
If I wanted to go to kernel, I'd probably get rid of the GC. I've tweeted about what Fil-C would look like without GC. Short version: use-after-free would not trap anymore, but you wouldn't be able to use it to break out of the capability system. Similar to CHERI without its capability GC.
Arduino is kinda both. You have full control over the execution flow, but then you have to actually exercise the full control over the execution flow. The only major wrinkle are the hardware interrupts.
One interesting feature is that there might be some synergy there. The GC safepoints can be used to implement cooperative multitasking, with capabilities making it safe.
This is still within the userspace application realm but it's good to know that Fil-C does have explicit capability-preserving operations (`zxorptr`, `zretagptr`, etc) to do e.g. pointer tagging, and special support for mapping pointers to integer table indices and back (`zptrtable`, etc).
Yes, I think that's reasonable. I imagine you wouldn't have to extend Fil-C very much to sneak some memory-mapped I/O addresses into your program, but maybe having the garbage collector pause the program in the middle of an interrupt handler would have other bad effects. Like, if you were generating a video signal, you'd surely get display glitches.
> but I think most C programmers won't be willing to take a 4× performance hit.
At a 4x performance hit, you might as well use C# or Go.
> Fil-C's target audience is users of C programs, not authors of C programs.
Sure, but then they don't get it for free. There is a perf penalty from GC. Plus you need all the original sources, right?
> we lose access to that intellectual heritage.
Declining usage of C is going to make you lose intellectual heritage[1]. A language no one can read or write is a dead language, regardless if you can translate it to English or not.
[1] And that is outside Rust's or Zig's influence. It's an old language from when people thought you can trust the programmer. Which may well be the case if only people using it were Bell Labs engineers. It's got UB up the wazoo, no safety, and no sane package manager.
UB was added to C when it was 15 years old, and the current understanding of UB is from about 10 years after that. It isn't essential to C; an implementation that defines all the undefined behavior would still be compliant and capable of running nearly all existing C.
> There is a perf penalty from GC.
Not really, no. There's a perf penalty from bounds checking and runtime type checking. GC takes a little time but saves you time on free(), although it only becomes a real performance win when you remove the other, less efficient ways of tracking lifetimes, such as reference counting.
> Plus you need all the original sources, right?
Yes, but from my point of view, loss of sources is not a significant problem. I know it happened historically, especially last millennium, but really only for proprietary software running on a single platform such as MS-DOS. Unix software, free software, and software using source control systems have suffered almost no source code loss, except for particular old versions.
> Declining usage of C is going to make you lose intellectual heritage
I think there are more C programmers today than there have ever been, and I doubt that that number will ever fall to zero while there are still humans.
Tracing GC involves either a big performance penalty or a big memory footprint penalty compared to the ownership-based memory management in a language like Rust.
The pervasive use of reference counting that you find in languages like Swift is worse on throughput than typical GC, but can often avoid the memory overhead of GC due to deterministic destruction and typically gives you better worst-case latency, so there isn't a single winner between ARC and GC.
Its being not essential doesn't matter. If you have a Fil-C code that terminates on UB and a C code that doesn't, you have two provably semantically (subtly) different programs.
Proof: You have a program that halts on UB and one that continues running on UB.
> I think there are more C programmers today than there have ever been, and I doubt that that number will ever fall to zero while there are still humans.
Think that depends on more things than just there being humans. Where my BASIC programmers at?
You seem to be suggesting that code that crashes in Fil-C will often run successfully in YOLO-C, but that doesn't seem to be the case from Pizlo's LFS exercise. He's had to modify a few things, but not much. Generally, UB in C code is a bug, and usually a portability problem, so C code that runs on a VAX, a 68000, a SPARC, a 386, an Alpha, and AMD64 will also probably run on Fil-C.
Old BASIC programmers are mostly working on Pick and other "business BASIC" systems, and I still run into them from time to time. But most of that code is only useful within a single business, so I expect it to eventually die out. (Meanwhile, new BASIC programmers are proliferating in the retrocomputing hobby.)
By contrast, on my system here I have over a thousand libraries written in C or C++. A random sampling reveals libraries for: LevelDB; various JS interpreters; file format handlers for zipfiles, OpenEXR, DjVu, and JPEG; gamepad interaction; a sparse matrix solver (used in Octave); the RIST protocol (used by OBS Studio); simulation with finite element models, which uses a different sparse matrix solver (used by FreeCAD); inspecting and manipulating configuration of PCI devices; the MTP protocol; the Icecast protocol; the protocol FTDI devices speak over USB; and so on.
Nearly all software written today is either written in C, written in C++, or interpreted or compiled by an interpreter or compiler written in C or C++.
You absolutely can just transpile any language into any language. You just need to be willing to give up on performance, and fil-c is more or less doing just that.
Fair point, though it does seem to be more common in the Rust community to the point that it’s become a bit of a meme.
Related funny anecdote, I recently saw a Show HN post title that made sure to mention the thing was written in Rust, but forgot to mention what it actually did. Priorities lol.
I’m sure it sounds like I’m a Rust hater or something, but I’m really not. I like a lot of the new Rust tools being created and use a few myself (ripgrep, fd, and bat immediately come to mind but I’m sure there are more I’m using). I just find the almost religious fervor of the Rust community amusing.
Please don't assume the entire Rust community, or most of it, is intent on rewriting all legacy software. Frankly, I think the more familiar one is with Rust, the less one is inclined to blindly advocate for rewriting things in Rust, and the Rust language developers are among those most familiar.
Up to my eyeballs in trouble. Working two jobs, too, where I can't practice tech skills. Fortunately, I have the peace of Jesus Christ in hopeless situations. He might change them at any time, too.
I still try to squirrel away a little time to learn from and contribute to the tech community. Mostly here and on AI subs like r/mlscaling. The best stuff that I can't work on or even curate right now I'm saving in case time or funding open up in the future. Worst case, I can pass it onto bright minds with the skills to build it. Like before in security.
You were doing neat things with satellites and bootstrapping research last I checked. What interesting jobs or tech are you digging into now?
Mostly I've just been struggling to keep my head above water, but I've been experimenting with submilliwatt autonomous personal computing, LuaJIT, and radically simplified windowing systems. Planning to visit the US next year for the first time in almost 20 years.
Extraordinary project. I had several questions which I believe I have answered for myself (pizlonator please correct if wrong):
1. How do we prevent loading a bogus lower through misaligned store or load?
Answer: Misaligned pointer load/stores are trapped; this is simply not allowed.
2. How are pointer stores through a pointer implemented (e.g. `*(char **)p = s`) - does the runtime have to check if *p is "flight" or "heap" to know where to store the lower?
Answer: no. Flight (i.e. local) pointers whose address is taken are not literally implemented as two adjacent words; rather the call frame is allocated with the same object layout as a heap object. The flight pointer is its "intval" and its paired "lower" is at the same offset in the "aux" allocation (presumably also allocated as part of the frame?).
3. How are use-after-return errors prevented? Say I store a local pointer in a global variable and then return. Later, I call a new function which overwrites the original frame - can't I get a bogus `lower` this way?
Answer: no. Call frames are allocated by the GC, not the usual C stack. The global reference will keep the call frame alive.
That leads to the following program, which definitely should not work, and yet does. ~Amazing~ Unbelievable:
#include <stdio.h>
char *bottles[100];
__attribute__((noinline))
void beer(int count) {
char buf[64];
sprintf(buf, "%d bottles of beer on the wall", count);
bottles[count] = buf;
}
int main(void) {
for (int i=0; i < 100; i++) beer(i);
for (int i=99; i >= 0; i--) puts(bottles[i]);
}
Hmmm ... there's a danger here that people will test their programs compiled with Fil-C and think that they are safe to compile with a "normal" compiler. I would hope for an option to flag any undefined behavior.
I worked with the author of Fil-C at Apple while on the Safari team, and he's easily one of the brightest folks I've had the pleasure of knowing. Fil-C looks extremely cool.
There’s nothing about how Fil-C is designed that constrains it to x86_64. It doesn’t strongly rely on x86’s memory model. It doesn’t strongly rely on 64-bit.
I’m focusing on one OS and arch until I have more contributors and so more bandwidth to track bugs across a wider set of platforms.
C is immensely powerful, portable and probably as fast as you can go without hand-coding in the architecture-specific assembly. Most of the world's information systems (our cyberstructure) rely directly or indirectly on C. And don't get me wrong, I'm a great enthusiast of the idea of sticking to memory-safe languages like Rust from now on.
The hard truth is will live with legacy C code, period. Pizlo's heroic effort bridges the gap so to speak, it kind of sandboxes userspace C in a way that inherently adds memory safety to legacy code. There are only a few corner cases now that can't be bothered by any slow-down vis-a-vis unsafe C, and the great majority of code across every industry would benefit much more from the reduced surface of exposure.
This is already quite impressive. Many automatic memory managed languages have more than 4x worst-case slowdown. E.g. Google-backed Golang is ~1.5× to ~4× slower than optimized C++. I suppose there are many ways to further speed-up Fil-C if more resources were given to the project.
This seems likely. I would assume that the vast majority of memory access can be statically proven safe if you throw enough money at the problem, meaning the bookkeeping can be eliminated.
> I would assume that the vast majority of memory access can be statically proven safe if you throw enough money at the problem, meaning the bookkeeping can be eliminated.
You can't really do this without adding the same sorts of annotations as Rust or Safe C++. Especially if you care about keeping modular/separate compilation and code reuse.
If it's not guaranteed to succeed it's not really memory safe. The subset that can be fully optimized without any annotations is really limited, more like writing old-style FORTRAN or COBOL code where you don't dynamically allocate at all to begin with. It may be "safe" but it's also not really helpful.
Yes, think of something like Inferno, but Limbo now is AOT compiled instead of JITed.
However there are also kernel like commercial projects in Go, and apparently the related TamaGo fork might eventually get upstreamed into the reference implementation.
Great effort, but I find the whole idea somewhat flawed. If one needs speed, he can't use this C implementation, because it's several times slower. If speed isn't important, why not just using a memory safe language? And if both are important, why not using Rust?
Recompiling existing software written in C using Fil-C isn't also a great idea, since some modifications are likely needed, at least for fixing bugs found with usage of Fil-C. And after these bugs are fixed, why continue using Fil-C?
Most software works flawlessly without modifications on Fil-C, the performance isn't *that* bad and there are applications where security is more important than performance (for example military applications)
You could use Fil-C in combination with a memory-safe language like Rust to allow for both "safe" leaf references (potentially using ownership and reference counting to represent more complex allocated objects, but would not themselves "own" pointers to Fil-C-managed data, thus avoiding the need to trace inside these objects and auto-free them) and general Fil-C managed pointers with possible cycles (perhaps restricted to some arena/custom address space, to make tracing and collecting more efficient) to be contained in a singke origram. Due to memory safety, the use of "leaf" references could be ensured not to alter the invariants that Fil-C relies on; but managed pointers would nonetheless be available whenever tracing GC could not be avoided.
This would ultimately save much of the overhead associated with tracing GC itself.
Safety isn’t really optional. Up until now, we’ve had no choice but to run C and accept the bugs; now we’re allowed to also run it safely.
Yes, this means that C is one of the slower languages around. That’s fine; computers are fast. If you want to write high-performance code, there are plenty of faster languages to do it with.
There were several options, but the industry decided to bet on UNIX and C as main inspiration going forwards from the 1980's.
"A consequence of this principle is that every occurrence of every subscript of every subscripted variable was on every occasion checked at run time against both the upper and the lower declared bounds of the array. Many years later we asked our customers whether they wished us to provide an option to switch off these checks in the interests of efficiency on production runs. Unanimously, they urged us not to--they already knew how frequently subscript errors occur on production runs where failure to detect them could be disastrous. I note with fear and horror that even in 1980 language designers and users have not learned this lesson. In any respectable branch of engineering, failure to observe such elementary precautions would have long been against the law."
-- C.A.R Hoare's "The 1980 ACM Turing Award Lecture"
Guess what he means by "1980 language designers and users have not learned this lesson".
I'm working on packaging Fil-C for Nix, as well as integrating Fil-C as a toolchain in Nix so you can build any Nix package with Fil-C.
https://github.com/mbrock/filnix
It's working. It builds tmux, nethack, coreutils, Perl, Tcl, Lua, SQLite, and a bunch of other stuff.
Binary cache on https://filc.cachix.org so you don't have to wait 40 minutes for the Clang fork to build.
If you have Nix with flakes on a 64-bit Linux computer, you can run
right now!Yeah, I was thinking nix will be probably one of the first things that can easily adapt Fil-C as it already packages in a way that allows different packages to be completely independent of each other, thus Fil-C's ABI compatibility does not matter. I assume other targets will be mostly enterprise distros where the perf hit and source compatibility issues are less of a concern, and memory safety is absolutely critical.
Fil-C compiled flatpaks might be a interesting target as well for normal desktop users. (e.g. running a browser)
I wonder if GPU graphics are possible in Fil-C land? Perhaps only if whole mesa stack is compiled using Fil-C as well, limiting GPU use to open drivers?
That is super cool, and I will probably start running it on at least a test box shortly.
How does python work? Of course I can just add filc.python to my system, but if I `python3 -m pip install whatever` will it just rebuild any C modules with the fil-c compiler?
Right now Python simply does not work at all. I was trying last night just to get the compiler to build, but ran into some Fil-C panics at the last moment when it tries to "freeze" some modules? But it should work.
My idea is to move towards defining Fil-C as a "platform", meaning it would have its own ABI value, so it would look like a target called "x86_64-unknown-linux-filc", and then the magic of Nixpkgs is that it can instantiate a full realm called "pkgsCross.filc" which automatically has every package, whose packages are built on your ordinary non-FilC platform as a cross compilation.
When that works—I hope it works, I probably need help to get it working—you should be able to use all the Nixpkgs packaging helpers like
This is already how Nixpkgs can have a pkgsCross.musl that builds everything with Musl as a cross compilation. So it seems reasonable and possible. It should be of wide interest for Nix users, since it would suddenly allow memory safety for whole swaths of the C/C++ packages, it would let NixOS use memory safe builds for core services like OpenSSHd, and so on...I should probably try to gather some interest among people who are truly familiar with Nixpkgs cross compilation...
Yes filc as abi makes most sense. Do note that while fil-c seems to have pretty good source compatibility some stuff still require patches (like python!)
I don't know how it's going on the nixpkgs side, but my impression from Alpine is that it's pretty normal to carry some patches for any ABI/platform that isn't GNU.
Yep, you can look around in Nixpkgs and see a bunch of conditionally applied patches based on target platform.
My Fil-C flake already includes like 50 different patches extracted from the upstream port catalogue!
That's very exciting! Thank you!
Either Fil-C or a different implementation of the same idea seems essential to me. A great deal of software has been written in C, and without some way of running it, we lose access to that intellectual heritage. But pervasive security vulnerabilities mean that the traditional "YOLO" approach to C compilation is a bad idea for software that has to handle untrusted input, such as Web browsing or email.
Pizlo seems to have found an astonishingly cheap way to do the necessary pointer checking, which hopefully I will be able to understand after more study. (The part I'm still confused about is how InvisiCaps work with memcpy.)
tialaramex points out that we shouldn't expect C programmers to be excited about Fil-C. The point tialaramex mentions is "DWIM", like, accessing random memory and executing in constant time, but I think most C programmers won't be willing to take a 4× performance hit. After all, if they wanted to be using a slow language, they wouldn't be writing their code in C. But I think that's the wrong place to look for interest: Fil-C's target audience is users of C programs, not authors of C programs. We want the benefits of security and continued usage of existing working codebases, without having to pay the cost to rewrite everything in Rust or TypeScript or whatever. And for many of us, much of the time, the performance hit may be acceptable.
I like to share this every time there's a post about memory safe C:
Apple has a memory-safer C compiler/variant they use to compile their boot loaders:
https://saaramar.github.io/iBoot_firebloom/
That was my idea and I wrote a good chunk of the compiler and runtime.
(For those who didn't make the connection, pizlonator also wrote Fil-C.)
Why isn't the compiler frontend called fucc?
I was so tempted to call it that
Why not call it secc ("sexy") ?
It's poor form to put the word "secure" (or any derivative of it) into a product that aims to provide more security.
Reason: there's no solution to the security problem. Fil-C doesn't solve security. Does it make things more secure? Yes! But there will always be more security issues. So imagine if I had called it "Secure C" (and then had a sexy compiler), and 10 years from now someone finds a comprehensive solution to the string injection problem in Secure C. What do they call their thing? Securer C? Secure C Pro? Secure Secure C?
I agree. I'd add that security is always relative to not just an existing level of security but to a threat model. There are threat models relative to which Fil-C doesn't make things more secure. (I can't think of one under which Fil-C makes things less secure, though.)
A similar criticism applies to "new". Newcastle is named after a castle built 945 years ago. Neuchâtel is named after a castle built 1014 years ago. Xavier (from Basque "etxeberri", "new house") is named after a castle built in the 10th century. Windows NT "new technology", etc.
These are good points but no name is ever perfect. Just felt that secc was more memorable and would offer more levarage to this great project :-)
"Filthy" is already extremely memorable.
It isn't too late!
Forgive me for not putting in the effort this deserves, but the description of pointers in this thread reads a lot like the current description of Fil-C pointers from the main site.
What is the 64-bit-presenting representation of pointers in Fil-C?
That is, what does %p return and how does that work as a pointer?
https://fil-c.org/invisicaps
https://fil-c.org/invisicaps_by_example
Re-reading tfm, it definitely had an answer to my dumb question(basically the same as any regular C pointer), but I still get the feeling that the explanations make jumps that may be obvious to you but absolutely not to me.
I sort of get a sense of what the flight pointers do after reading the linked page with the assembly listings( which on mobile looks like you're discussing hex dumps until one tries side scrolling, btw ) but my brain would expect something along the lines of: "this is the flight pointer and this is how it is managed for local variables, global variables, etc. This part of it is what your C program sees" at the beginning of the invisicaps article rather than "pointers are 64 bit like in regular C proceeds to talk about a tuple containing two pointers"(at least that's how it reads to me).
Probably just laziness on my part.
That’s really good feedback about how to better talk about this stuff. Thank you!
InvisiCaps are counterintuitive. Took me a while to come up with them. I still haven’t found a really clean way to explain them to folks
Yeah, fat pointers are definitely a viable approach, but a lot of the existing C code that is the main argument for Fil-C assumes it can put a pointer in a long. (Most of the C code that assumed you could put it in an int has been flushed out by now, but that was a big problem when the Alpha came out.) I'm guessing that the amount of existing C code in Apple's bootloader is minimal, maybe 1000 lines, not the billions of lines you can compile with Fil-C.
You’re off by a few orders of magnitude. I’ll grant you, what is the bootloader becomes a very complex question. Even if you scope it to just “what is the code physically etched into the chip as the mask ROM” (secureROM) you’re talking hundreds of thousands. If you’re talking about all the code that runs before the kernel starts executing you’re talking hundreds of millions.
No, I was only talking about the pre-existing C code that wasn't written for the bootloader, which therefore might have incompatibilities with fat pointers you had to hunt down and fix.
Also I'm really skeptical about your "hundreds of millions" number, even if we're talking about all the code that runs before the kernel starts. How do you figure? The entire Linux kernel doesn't contain a hundred millions of lines of code, and that includes all the drivers for network cards, SCSI controllers, and multiport serial boards that nobody's made in 30 years, plus ports to Alpha, HP PA-RISC, Loongson, Motorola 68000, and another half-dozen architectures. All of that contains maybe 30 million lines. glibc is half a million. Firefox 140.4.0esr is 33 million. You're saying that the bootloader is six times the size of Firefox?
Are you really suggesting that tens of gigabytes of source code are compiled into the bootloader? That would make the bootloader at least a few hundred megabytes of executable code, probably gigabytes, wouldn't it?
Couldn’t one just make long bigger then to make it match?
Maybe so; I haven't tried. Probably a lot less code depends on unsigned long wrapping at 2⁶⁴ than used to depend on unsigned int wrapping at 2¹⁶, and we got past that. But stability standards were lower then. Any code that runs on both 32-bit and 64-bit LP64 systems can't be too dependent on the exact sizeof long, and sizeof long already isn't sizeof int the way it was on 32-bit platforms.
I'd actually keep it still wrapping at 2^64, with the extra metadata not participating in arithmetic operations...
That seems worse.
For all the wrong code that assumes long can store a pointer, there's likely a lot more wrong code that assumes long can fit in 64 bits, especially when serializing it for I/O and other-language interop.
Also, 128-bit integers can't fit in standard registers on most architectures, and don't have the full suite of ALU operations either. So you're looking at some serious code bloat and slowdowns for code using longs.
You've also now got no "standard" C type (char, short, int, long, long long) which is the "native" size of the architecture. You could widen int too, but a lot of code also assumes an int can fit in 32 bits.
No, it should only do arithmetic on the first 64 (or 32) bits. The extra metadata should be copied unchanged.
Ok, I think I follow. You'd widen the type under the hood but not expose this fact to user code.
However, most longs are just numbers that have no metadata. I guess you'd set the metadata portion to all zeroes in that case. This feels like a reified version of Rust's pointer provenance, and I think you would have to expose some metadata-aware operations to the user. In which case, you're inviting some code rewrites anyway.
While not as bad as the register/ALU ops issue, you're still making all code pay a storage size penalty, and still adding some overhead to handle the metadata propagating through arithmetic operations, just to accommodate bad code, plus it complicates alignment and FFI.
It would still be exposed to user code that checks its size with sizeof, but yeah the long would only have numerical values between 2^-63 and 2^63-1.
And yes, there would still be some overhead for storing and propagating the metadata, and struct alignment would change and FFI wouldn't work with longs.
There's a lot of code that makes assumptions about the number of bytes in a long rather than diligently using sizeof ... remember, the whole point here is low quality code.
It's going to break stuff one way or another.
You don't even need to reverse it. It's in the public clang, and I'm working on helping my team adopt it in some test cases.
And it's not just the bounds-checking that's great -- it makes a bunch of C anti-patterns much harder, and it makes you think a lot harder about pointer ownership and usage. Really a great addition to the language, and it's source-compatible with empty macro-definitions (with two exceptions).
> It's in the public clang
I think you’re thinking of something else
It or something similar has apparently been upstreamed to clang as -fbounds-safety. I don't know if they're the same, but the RFC on -fbounds-safety does give some credit to the author of Fil-C, who also took credit for firebloom on this thread as well.
Not the same
Thanks for clarifying, I had found a past HN comment stating they are the same.
Interesting! How do you get started?
A descendent of this is in clang as -fbounds-safety.
That is basically clang getting something like MSVC SAL, the more the merrier.
Not for much longer, replacing it with Swift is one of the reasons why Swift Embedded subset came to be.
"Why Embedded Swift"
https://youtu.be/LqxbsADqDI4?t=144
and the author of Fil-C worked on that!
Oh, somehow I missed that connection!
Also this is _de facto_ limited to userspace application for the mainstream OSes if my understanding is correct.
Reading Fil-C website's "InvisiCaps by example" page, I see that "Laundering Integers As Pointers" is disallowed. This essentially disqualifies Fil-C for low-level work, which makes for a substantial part of C programs.
(int2ptr for MMIO/pre-allocated memory is in theory UB, in practice just fine as long as you don't otherwise break aliasing rules (and lifetime rules in C++) - as the compiler will fail to track provenance at least once).
But that isn't really what Fil-C is aimed at - the value is, as you implied, in hardening userspace applications.
It’s not so fundamental of a limitation.
Fil-C already allows memory mapped I/O in the form of mmap.
The only thing missing that is needed for kernel level MMIO is a way to forge a capability. I don’t allow that right now, but that’s mostly a policy decision. It also falls out from the fact that InvisiCaps optimize the lower by having it double as a pointer to the top of the capability. That’s also not fundamental; it’s an implementation choice.
It’s true that InvisiCaps will always disallow int to ptr casts, in the sense that you get a pointer with no capability. You’d want MMIO code to have some intrinsic like `zunsafe_forge_ptr` that clearly calls out what’s happening and then you’d use that wherever you define your memory mapped registers.
Can you "launder" pointers through integers just to do things like drop `const`? It's a very common pattern to have to drop attributes like `const` due to crappy APIs: `const foo a = ...; foo b = (foo *)(uintptr_t)a;`
Hopefully Pizlo will correct me if I get this wrong, but I don't think Fil-C's pointer tagging enforces constness, which isn't needed for C in any case. This C code compiles with no warnings and outputs "Howlong\n" with GCC 12.2.0-14 -ansi -pedantic -Wall -Wextra:
Somewhat to my surprise, it still compiles successfully with no warnings as C++ (renaming to deconst.cc and compiling with g++). I don't know C++ that well, since I've only been using it for 35 years, which isn't nearly long enough to learn the whole language unless you write a compiler for it.Same results with Debian clang (and clang++) version 14.0.6 with the same options.
Of course, if you change c[] to *c, it will segfault. But it still compiles successfully without warnings.
Laundering your pointer through an integer is evidently not necessary.
Fil-C capabilities have a read only bit of this purpose.
Interesting, so you really can enforce constness on references? Does that mean the above code will crash on Fil-C?
I tried it and it doesn't crash when compiled with Fil-C, FYI. But I guess it wouldn't be too hard to implement.
Just using const in C isn’t enough, since C’a const is unsound, so I ignore it.
But if you try to write to a readonly global constant then you’ll panic. And there are a handful of ways to allocate readonly data via Fil-C’s APIs.
> I don't know C++ that well, since I've only been using it for 35 years
Ok that got a chuckle out of me haha
> > I don't know C++ that well, since I've only been using it for 35 years, which isn't nearly long enough to learn the whole language unless you write a compiler for it.
No one person could write a compiler for it, and even if they could they would forget as much in doing so as they could learn.
Walter Bright did, although maybe C++ has gotten enough bigger since then that your statement has become true.
Did you compile with -Wcast-qual?
No, that does give a warning.
As kragen already posted, you can cast from const-pointer to non-const directly.
Not allowing a cast from integer to pointer is the point of having pointers as capabilities in the first place.
Central in that idea of capabilities is that you can only narrow privileges, never widen them. An intptr_t would in-effect be a capability narrowed to be used only for hashing and comparison, with the right for reading and writing through it stripped away.
BTW, if you would store the uintptr_t then it would lose its notion of being a pointer, and Fil-C's garbage collector would not be able to trace it.
The C standard allows casts both ways, but the [u]intptr_t types are optional. However, C on hardware capability architectures' (CHERI, Elbrus, I dunno about AS/400) tend to make the type available anyway because the one-way cast is so common in real-world code.
Yes. That works.
If the laundering through integers is syntactically obvious - obvious that the cast back from int used a int that obviously can from a pointer - then I allow it.
I'll preface this by saying my experience is with embedded, not kernel, but I can't imagine MMIO is significantly different.
There would still be ways to make it work with a more restricted intrinsic, if you didn't want to open up the ability for full pointer forging. At a high level, you're basically just saying "This struct exists at this constant physical address, and doesn't need initialisation". I could imagine a "#define UART zmmio_ptr(UART_Type, 0x1234)" - which perhaps requires a compile time constant address. Alternatively, it's not uncommon for embedded compilers to have a way to force a variable to a physical address, maybe you'd write something like "UART_Type UART __at(0x1234);". I believe this is technically already possible using sections, it's just a massive pain creating one section per struct for dozens and dozens.
Unfortunately the way existing code does it is pretty much always "#define UART ((UART_Type*)0x1234)". I feel like trying to identify this pattern is probably too risky a heuristic, so source code edits seem required to me.
I'm curious, what's your strategy for integrating the GC with low-level code? I've been thinking about trying to use it for Arduino development. Mostly as a thought experiment for now (as I'm playing with Rust on RP2040).
If it's userlevel code, then it just works.
It's a concurrent GC.
If I wanted to go to kernel, I'd probably get rid of the GC. I've tweeted about what Fil-C would look like without GC. Short version: use-after-free would not trap anymore, but you wouldn't be able to use it to break out of the capability system. Similar to CHERI without its capability GC.
Arduino is kinda both. You have full control over the execution flow, but then you have to actually exercise the full control over the execution flow. The only major wrinkle are the hardware interrupts.
One interesting feature is that there might be some synergy there. The GC safepoints can be used to implement cooperative multitasking, with capabilities making it safe.
Check out this document to see how the Fil-C ports of Python and Perl and so on work:
https://github.com/mbrock/filnix/blob/main/ports/analysis.md
This is still within the userspace application realm but it's good to know that Fil-C does have explicit capability-preserving operations (`zxorptr`, `zretagptr`, etc) to do e.g. pointer tagging, and special support for mapping pointers to integer table indices and back (`zptrtable`, etc).
Yes, I think that's reasonable. I imagine you wouldn't have to extend Fil-C very much to sneak some memory-mapped I/O addresses into your program, but maybe having the garbage collector pause the program in the middle of an interrupt handler would have other bad effects. Like, if you were generating a video signal, you'd surely get display glitches.
> but I think most C programmers won't be willing to take a 4× performance hit.
At a 4x performance hit, you might as well use C# or Go.
> Fil-C's target audience is users of C programs, not authors of C programs.
Sure, but then they don't get it for free. There is a perf penalty from GC. Plus you need all the original sources, right?
> we lose access to that intellectual heritage.
Declining usage of C is going to make you lose intellectual heritage[1]. A language no one can read or write is a dead language, regardless if you can translate it to English or not.
[1] And that is outside Rust's or Zig's influence. It's an old language from when people thought you can trust the programmer. Which may well be the case if only people using it were Bell Labs engineers. It's got UB up the wazoo, no safety, and no sane package manager.
UB was added to C when it was 15 years old, and the current understanding of UB is from about 10 years after that. It isn't essential to C; an implementation that defines all the undefined behavior would still be compliant and capable of running nearly all existing C.
> There is a perf penalty from GC.
Not really, no. There's a perf penalty from bounds checking and runtime type checking. GC takes a little time but saves you time on free(), although it only becomes a real performance win when you remove the other, less efficient ways of tracking lifetimes, such as reference counting.
> Plus you need all the original sources, right?
Yes, but from my point of view, loss of sources is not a significant problem. I know it happened historically, especially last millennium, but really only for proprietary software running on a single platform such as MS-DOS. Unix software, free software, and software using source control systems have suffered almost no source code loss, except for particular old versions.
> Declining usage of C is going to make you lose intellectual heritage
I think there are more C programmers today than there have ever been, and I doubt that that number will ever fall to zero while there are still humans.
Tracing GC involves either a big performance penalty or a big memory footprint penalty compared to the ownership-based memory management in a language like Rust.
The pervasive use of reference counting that you find in languages like Swift is worse on throughput than typical GC, but can often avoid the memory overhead of GC due to deterministic destruction and typically gives you better worst-case latency, so there isn't a single winner between ARC and GC.
> It isn't essential to C;
Its being not essential doesn't matter. If you have a Fil-C code that terminates on UB and a C code that doesn't, you have two provably semantically (subtly) different programs.
Proof: You have a program that halts on UB and one that continues running on UB.
> I think there are more C programmers today than there have ever been, and I doubt that that number will ever fall to zero while there are still humans.
Think that depends on more things than just there being humans. Where my BASIC programmers at?
You seem to be suggesting that code that crashes in Fil-C will often run successfully in YOLO-C, but that doesn't seem to be the case from Pizlo's LFS exercise. He's had to modify a few things, but not much. Generally, UB in C code is a bug, and usually a portability problem, so C code that runs on a VAX, a 68000, a SPARC, a 386, an Alpha, and AMD64 will also probably run on Fil-C.
Old BASIC programmers are mostly working on Pick and other "business BASIC" systems, and I still run into them from time to time. But most of that code is only useful within a single business, so I expect it to eventually die out. (Meanwhile, new BASIC programmers are proliferating in the retrocomputing hobby.)
By contrast, on my system here I have over a thousand libraries written in C or C++. A random sampling reveals libraries for: LevelDB; various JS interpreters; file format handlers for zipfiles, OpenEXR, DjVu, and JPEG; gamepad interaction; a sparse matrix solver (used in Octave); the RIST protocol (used by OBS Studio); simulation with finite element models, which uses a different sparse matrix solver (used by FreeCAD); inspecting and manipulating configuration of PCI devices; the MTP protocol; the Icecast protocol; the protocol FTDI devices speak over USB; and so on.
Nearly all software written today is either written in C, written in C++, or interpreted or compiled by an interpreter or compiler written in C or C++.
> you might as well use C# or Go
Except, uh, you can't use C# or Go to run a program written in C/C++.
Oh, you mean we can solve all our problems by simply rewriting all legacy software? Right, I forgot!
You absolutely can just transpile any language into any language. You just need to be willing to give up on performance, and fil-c is more or less doing just that.
Let me know when the full "C++ to Go" toolchain is up and running, I'd love to try it and compare!
> Oh, you mean we can solve all our problems by simply rewriting all legacy software?
Isn’t that Rust’s raison d'être?
(Just kidding…mostly)
Pretty sure that's a programmer's raison d'être. Not invented here (NIH) syndrom predates Rust.
Fair point, though it does seem to be more common in the Rust community to the point that it’s become a bit of a meme.
Related funny anecdote, I recently saw a Show HN post title that made sure to mention the thing was written in Rust, but forgot to mention what it actually did. Priorities lol.
I’m sure it sounds like I’m a Rust hater or something, but I’m really not. I like a lot of the new Rust tools being created and use a few myself (ripgrep, fd, and bat immediately come to mind but I’m sure there are more I’m using). I just find the almost religious fervor of the Rust community amusing.
Please don't assume the entire Rust community, or most of it, is intent on rewriting all legacy software. Frankly, I think the more familiar one is with Rust, the less one is inclined to blindly advocate for rewriting things in Rust, and the Rust language developers are among those most familiar.
If you can convert Cobol to Java, you can convert C++ to a GC language as well. If you are so inclined.
Softbound + CETS was one of the old attempts I liked:
https://people.cs.rutgers.edu/~santosh.nagarakatte/softbound...
CCured was another:
https://people.eecs.berkeley.edu/~necula/Papers/ccured_popl0...
Good to see you back! I hope you're doing okay; I know you ran into some real trouble a couple of years ago.
Up to my eyeballs in trouble. Working two jobs, too, where I can't practice tech skills. Fortunately, I have the peace of Jesus Christ in hopeless situations. He might change them at any time, too.
I still try to squirrel away a little time to learn from and contribute to the tech community. Mostly here and on AI subs like r/mlscaling. The best stuff that I can't work on or even curate right now I'm saving in case time or funding open up in the future. Worst case, I can pass it onto bright minds with the skills to build it. Like before in security.
You were doing neat things with satellites and bootstrapping research last I checked. What interesting jobs or tech are you digging into now?
Mostly I've just been struggling to keep my head above water, but I've been experimenting with submilliwatt autonomous personal computing, LuaJIT, and radically simplified windowing systems. Planning to visit the US next year for the first time in almost 20 years.
Extraordinary project. I had several questions which I believe I have answered for myself (pizlonator please correct if wrong):
1. How do we prevent loading a bogus lower through misaligned store or load?
Answer: Misaligned pointer load/stores are trapped; this is simply not allowed.
2. How are pointer stores through a pointer implemented (e.g. `*(char **)p = s`) - does the runtime have to check if *p is "flight" or "heap" to know where to store the lower?
Answer: no. Flight (i.e. local) pointers whose address is taken are not literally implemented as two adjacent words; rather the call frame is allocated with the same object layout as a heap object. The flight pointer is its "intval" and its paired "lower" is at the same offset in the "aux" allocation (presumably also allocated as part of the frame?).
3. How are use-after-return errors prevented? Say I store a local pointer in a global variable and then return. Later, I call a new function which overwrites the original frame - can't I get a bogus `lower` this way?
Answer: no. Call frames are allocated by the GC, not the usual C stack. The global reference will keep the call frame alive.
That leads to the following program, which definitely should not work, and yet does. ~Amazing~ Unbelievable:
Hmmm ... there's a danger here that people will test their programs compiled with Fil-C and think that they are safe to compile with a "normal" compiler. I would hope for an option to flag any undefined behavior.
No discussion, but just on the front page last week (31 points) https://news.ycombinator.com/item?id=45655519
Previous discussion:
2025 Safepoints and Fil-C (87 points, 1 month ago, 44 comments) https://news.ycombinator.com/item?id=45258029
2025 Fil's Unbelievable Garbage Collector (603 points, 2 months ago, 281 comments) https://news.ycombinator.com/item?id=45133938
2024 The Fil-C Manifesto: Garbage In, Memory Safety Out (13 points, 17 comments) https://news.ycombinator.com/item?id=39449500
I worked with the author of Fil-C at Apple while on the Safari team, and he's easily one of the brightest folks I've had the pleasure of knowing. Fil-C looks extremely cool.
Somewhat related, safe C++ proposal is not being continued
https://news.ycombinator.com/item?id=45234460
posted multiple times, x86 only last time I checked
Yeah because I’m limiting my test matrix.
There’s nothing about how Fil-C is designed that constrains it to x86_64. It doesn’t strongly rely on x86’s memory model. It doesn’t strongly rely on 64-bit.
I’m focusing on one OS and arch until I have more contributors and so more bandwidth to track bugs across a wider set of platforms.
Hey Filip – while we're talking about memory architecture, have you started looking at ARM's EMTE extension (e.g https://security.apple.com/blog/memory-integrity-enforcement...)? Could it eventually replace invisicaps?
No. MTE is probabilistic. Fil-C is deterministic.
If you're taking votes, I would vote for AArch64 next. :D
noted :-)
All the more reason to make it portable. I wonder if this can be implemented via LLVM?
It is implemented via LLVM.
TLDR: 4x slowdown in the normal case
the performance overhead of this approach for most programs makes them run about four times more slowly
> TLDR: 4x slowdown in the normal case
4x slower isn't the normal case. 4x is at the upper end of the overheads you'll see.
That's good to know!
C is immensely powerful, portable and probably as fast as you can go without hand-coding in the architecture-specific assembly. Most of the world's information systems (our cyberstructure) rely directly or indirectly on C. And don't get me wrong, I'm a great enthusiast of the idea of sticking to memory-safe languages like Rust from now on.
The hard truth is will live with legacy C code, period. Pizlo's heroic effort bridges the gap so to speak, it kind of sandboxes userspace C in a way that inherently adds memory safety to legacy code. There are only a few corner cases now that can't be bothered by any slow-down vis-a-vis unsafe C, and the great majority of code across every industry would benefit much more from the reduced surface of exposure.
> 4x is at the upper end of the overheads
This is already quite impressive. Many automatic memory managed languages have more than 4x worst-case slowdown. E.g. Google-backed Golang is ~1.5× to ~4× slower than optimized C++. I suppose there are many ways to further speed-up Fil-C if more resources were given to the project.
This seems likely. I would assume that the vast majority of memory access can be statically proven safe if you throw enough money at the problem, meaning the bookkeeping can be eliminated.
> I would assume that the vast majority of memory access can be statically proven safe if you throw enough money at the problem, meaning the bookkeeping can be eliminated.
You can't really do this without adding the same sorts of annotations as Rust or Safe C++. Especially if you care about keeping modular/separate compilation and code reuse.
You can do it without annotations if you only do it as an optimization that's not guaranteed to succeed.
If it's not guaranteed to succeed it's not really memory safe. The subset that can be fully optimized without any annotations is really limited, more like writing old-style FORTRAN or COBOL code where you don't dynamically allocate at all to begin with. It may be "safe" but it's also not really helpful.
Every time I read C and memory safety, I just think Golang. Especially for user space
Go programs are not fully memory-safe if they use multiple treads, due to possible data races with fat pointers: https://news.ycombinator.com/item?id=44672003
Still much better than C will ever be, even if I disagree with some of its design decisions.
Yes, think of something like Inferno, but Limbo now is AOT compiled instead of JITed.
However there are also kernel like commercial projects in Go, and apparently the related TamaGo fork might eventually get upstreamed into the reference implementation.
https://www.withsecure.com/en/solutions/innovative-security-...
Great effort, but I find the whole idea somewhat flawed. If one needs speed, he can't use this C implementation, because it's several times slower. If speed isn't important, why not just using a memory safe language? And if both are important, why not using Rust?
Recompiling existing software written in C using Fil-C isn't also a great idea, since some modifications are likely needed, at least for fixing bugs found with usage of Fil-C. And after these bugs are fixed, why continue using Fil-C?
Most software works flawlessly without modifications on Fil-C, the performance isn't *that* bad and there are applications where security is more important than performance (for example military applications)
You could use Fil-C in combination with a memory-safe language like Rust to allow for both "safe" leaf references (potentially using ownership and reference counting to represent more complex allocated objects, but would not themselves "own" pointers to Fil-C-managed data, thus avoiding the need to trace inside these objects and auto-free them) and general Fil-C managed pointers with possible cycles (perhaps restricted to some arena/custom address space, to make tracing and collecting more efficient) to be contained in a singke origram. Due to memory safety, the use of "leaf" references could be ensured not to alter the invariants that Fil-C relies on; but managed pointers would nonetheless be available whenever tracing GC could not be avoided.
This would ultimately save much of the overhead associated with tracing GC itself.
And after these bugs are fixed, why continue using Fil-C?
Because Fil-C might be a good way to debug future code?
Your question reads like, "Why use a debugger?"
Safety isn’t really optional. Up until now, we’ve had no choice but to run C and accept the bugs; now we’re allowed to also run it safely.
Yes, this means that C is one of the slower languages around. That’s fine; computers are fast. If you want to write high-performance code, there are plenty of faster languages to do it with.
There were several options, but the industry decided to bet on UNIX and C as main inspiration going forwards from the 1980's.
"A consequence of this principle is that every occurrence of every subscript of every subscripted variable was on every occasion checked at run time against both the upper and the lower declared bounds of the array. Many years later we asked our customers whether they wished us to provide an option to switch off these checks in the interests of efficiency on production runs. Unanimously, they urged us not to--they already knew how frequently subscript errors occur on production runs where failure to detect them could be disastrous. I note with fear and horror that even in 1980 language designers and users have not learned this lesson. In any respectable branch of engineering, failure to observe such elementary precautions would have long been against the law."
-- C.A.R Hoare's "The 1980 ACM Turing Award Lecture"
Guess what he means by "1980 language designers and users have not learned this lesson".
Security issues happen because of the bugs you won’t find under any normal use or with any tests you have
That’s why even and especially if a C program runs in Fil-C with zero changes, you should use the Fil-C version in any context where security matters
You're glossing over the "just rewrite everything in Rust" part.