It's unfortunate that change IDs aren't considered. There was a discussion [1]
in 2025, and it has resurfaced a couple of times since.
Basically, the idea is to attribute a new kind of ID to an initial 'change'. During review, or whenever a commit is rebased, the change ID is kept, whereas the commit of course changes. This allows tooling to identify all previous versions of a change, and is what enables "per-commit" code review à la Gerrit [2] (which IMO is a much better experience than the branch-review-squash model that GitHub normalized). It's also used in jj, although I'm not familiar with that.
As of today, any tool that wants a change ID needs to somehow encode it in commit message bodies. The proposed discussion was about making a change ID a standard header field that git would natively keep across rebases.
JJ and GitButler already create and inject this into the commit headers (using the same interoperable reverse-hex format), which is recognized by Gerrit and some forges like Tangled for incremental commit based review.
I doubt that core Git will adopt it anytime soon as it was not discussed at this years contributor summit (last week) and doesn't seem to be a hot topic on the ML.
What I would like to see is support for `git rebase` not dropping it, which is the current main issue. The `git replay` command, as well as commands based on the same sequencing code (`git history` for example) do not drop custom headers like this, so there is partial non-breakage, but several of the other history editing commands do drop custom headers.
Having switched to jj I don't really agree. Everything I do is basically the same as with branches, just that I get randomly generated tip names rather than naming them myself, which is both slightly convenient and slightly annoying
Yeah, "standardizing" change IDs would make it much easier to develop further tooling around it. In particular decentralized review is something that I'd be interested in.
For example, if GitHub is down, that would not be a blocker to access review comments or to do reviews. And maybe you could push your reviews to a GitLab mirror if you want a UI.
They aren't really. It's basically like getting an auto generated ref/branch name for each new commit, with convenience rewriting every ref when you rebase.
It's a bit more convenient if you prefer referring to a non-leaf commit directly rather than relative to the leaf branch a la master~2
change ids are more analogous to commit messages than branches, eg suppose that in a feature branch you have a "Delete deprecated classes" commit; in git there is a clear idea of "cloning" this commit (eg rebase, cherrypicks, maybe reverts) and the common sense that the new commit inherits the same commit message. Change ids are the same thing but in hex id form that can be created for every new commit/stash/index.
They allow for example to identify all the clones of a commit and they allow to give stable identities across rebases eg suppose you rebase a typo at the beginning of a feature branch without change ids a reviewer sees n new unrelated commits while with change ids it is possible to clearly identify which commits where changed/added/removed since the previous review iteration.
> suppose you rebase a typo at the beginning of a feature branch without change ids a reviewer sees n new unrelated commits while with change ids it is possible to clearly identify which commits where changed/added/removed since the previous review iteration.
A rebase can introduce change to a commit in cases such as handling conflicts or squashing.
Also, a commit already retains it's commit message after rebasing.
> A rebase can introduce change to a commit in cases such as handling conflicts or squashing.
and with change ids you can quickly separate commits that changed from commit that did not.
> Also, a commit already retains it's commit message after rebasing.
but commit message are not ids, there is no command for checking out a commit by its message, nor any sense that commit with the same message are somehow functionally related
Another thing that becomes easier with change IDs is reviewing multiple related commits together, essentially "stacked pull requests".
If you treat a branch as your unit of review, then it becomes super difficult for someone to submit a chain of related changes. You'll be constantly rebasing your pull requests onto each other as you get feedback from dependent branches.
I heard that the github CLI recently introduced support for this, but since in git there's no concept of dependent branches (a branch isn't even an object in git, just a reference to a commit), I think this approach will always be clunkier than reviewing commits related by a change ID.
> If you treat a branch as your unit of review, then it becomes super difficult for someone to submit a chain of related changes. You'll be constantly rebasing your pull requests onto each other as you get feedback from dependent branches.
Does it mean that when switching trop sha1 to sha256 you need to forcepush and rewrite all history? Wouldn’t that be a massive source of potential vulnerabilities?
Yes, you do need to do that. However, there is also much more work after that.
Git will not intermingle SHA-256 and SHA-1 enabled repositories, even in things like submodules, so anything used in that manner will need to keep both versions into the indefinite future. If you rely on a submodule that has not yet converted, you will have to convert it yourself and try to keep it up to date, or the forge will have to automatically keep a bidirectional mirror (if you have submodules in various forges, you'll have to wait for all of them to do it), etc.
This means that every SHA referenced anywhere on the internet, in commit messages, in issues, in code comments is now invalid and needs a mapping to find the rewritten one for forever.
It also means that every commit signature ever made is now invalid and will probably have to be stripped from the rewritten new 256 history because it's impossible to resign everything.
Companies like Google and GitHub are working on keeping two versions of each repository so that there can be long stages of ecosystem migrations, but no matter what, it's going to be a huge pain for millions of developers for years to come.
i guess that for now only the default will change for new repositories. support for sha1 is not going to be dropped, so most existing repositories won't switch any time soon. if you want to switch then yes, it sounds like a force push might be needed, although it could also be that simply switching is not possible, but that instead you have to create a new repo and import the history from the old repo, forcing everyone to clone the new repo intentionally.
It is a giant format change, but in the current documentation [0] sounds more like a repack than a force-push. git keeps a lookup table of the SHA1 object ids similar to an index file and some interop is allowed between SHA1 repositories and SHA256. (Primarily if you still needed to use GitHub as an SHA1 server because of some support hiccup, but needed your local repo to be SHA256 for security or other reasons, that's partially/mostly supposted.) Objects need to be resigned with their SHA256 id, but for different reasons than rebase/force-push and with a subtly different developer experience. In theory using that compatibility index of SHA1 hashes a good UI could show both signatures.
Couldn't you write something that checks every commit's content and message is byte equal to the old tree? One scan through the history to verify it should be relatively simple if not cheap. Should be built into git.
The contents of the files don’t change, only the Merkle tree. You can verify that the content blobs all have the same sha1 by literally rehashing. Then you can verify that the contents of the clone are the same. That doesn’t stop history corruption, but it does prevent malicious injection into the current state of the tree before the migration.
> The Git 2.45 release added reftables as a more efficient way of storing refs. It is a binary file optimized for both space efficiency and quick access. Since then, it has been possible to create a repository that uses a reftable rather than the old file-based mechanism, but that has never been the default. Switching to reftable should have no visible consequences (other than better performance) for users of Git itself, but it can be a problem for users of other software packages that access Git repositories. In his email, carlson mentioned libgit2 as a potential concern.
Looking forward to losing all references at once vs just the current one...
I've noticed persistent Git/fs interaction where on crash the current ref can just disappear...
I'm looking forward to reftable to become the default. It solves many problems with branches, like branches with weird characters created with non-standard clients making a fetch impossible; case insensitive branches with "same" names doing the same; or impossible to create branch FOO because FOO/Something exists.
All those problems just go away when branches are no longer files on disk.
I enabled it in setup script of one large repo I maintain; the main issue is the incompatibility with some people's personal tooling based on libgit2 (some git status tooling in oh-my-zsh), but people do find workarounds.
So Git, in version 3.0, is moving to SHA-256 by default for SHA-1 ain't considered that strong anymore but...
What about future attacks by quantum computers? Is Git safe from quantum computers for it's all hashes only? Or shall there be issues with quantum attacks?
I'm asking for there are several projects that are already moving to quantum-resistant schemes (like OpenSSH who uses an hybrid scheme [1]).
At this point, it does not appear that (reasonably strong and modern) symmetric cryptography (ciphers, hash functions, etc.) is realistically threatened by quantum computing: https://words.filippo.io/128-bits/
Thanks for the great link! This has been vexing me, as intuitively it seems like if quantum computers can test all factors they should be able to test all keys.
But the article helps. Basically Grover’s is not as potent as Shorr’s. And it seems like everyone is convinced there is no dramatically better quantum algorithm than Grover’s?
No. Not quite.
In fact, that blog post ignores something important from the very papers it cites.
Grover's assumes the function is a black box that you cannot look inside and that your only way of finding a certain result is through repeated invocation.
Under this assumption, Grover's is optimal in the number of invocations of the function required to find the result.
However, this assumption may be quite wrong for AES and friends. It may be the structure allows for non brute force attacks that are totally impractical classically but not subject to Grover's optimality limitation quantumly.
The only thing you are guaranteed here is that if you cannot take advantage of structure at all then Grover's is the best you can do.
Given that we have pretty much always found a way to take some advantage of structure, I would bet we will do so here.
That may or may not make it viable to break at all, I just wouldn't bet that it must be treated like a black box forever.
Yes. Structure comes in many forms.
In this case, Shor is turning the function into a strongly periodic algebraic structure.
I'll explain it without going too far into why any of this is true, which is much more complicated to prove. This will let me use relatively simple math.
Let's say you want to factor N. Pick some number that is coprime to N, which we'll call a, and consider f(x) = a^x (mod N).
Since it's a modular function, it repeats at some point. Shor calculates the period of this function (r), rather than seeing which of the 2^n numbers is "the answer".
Once you know the period of this function, there is a high chance that the factors fall out of gcd(a^(r/2) - 1, N) and gcd(a^(r/2)+1, N).
The point here is not to explain Shor's as much as to point out it is finding a strong amount of structure to take advantage of, quantumly.
This is actually the same way the oracle separation of BQP and the entire polynomial hiearchy works[1] - It depends on forrelation, which is a problem where quantum computers can extract a global property of the function without needing to learn all the individual values, by taking advantage of structure.
Which is why i go to "The idea that there is literally no structure that can be taken advantage of in AES strikes me as a bad bet".
There are already reduced round quantum attacks on AES as well. Again, more to the point, the idea that symmetric key ciphers and cryptographic hashes in general are safe because grover's is slower than shor's is not a thing i would bet on at all. Even if AES ends up relatively safe, that tells you basically nothing about the other practically-used ciphers and functions since there are a lot of different construction mechanisms being used.
[1] People still seem to believe there are no functions which quantum computing models have been been proven to be faster at than classical computing models. This is false. Forrelation is the canonical example - and shows that BQP can perform things exponentially faster than you can classically even given access to an infinite polynomial hierarchy.
It is the current physical actualization of these computing models that have the "is it really faster than classical computers" issue, not the theory ;)
(IE it is a perfect example of "in theory there is no difference between theory and practice, and in practice, there is")
Thanks so much for the patient and accessible explanation. So much more useful and grounded than the naive “quantum is faster because it can consider all possible solutions at once” canard.
Ah it's interesting, AIUI cryptographic hashes are safe from quantum attacks (for there's an infinity of secrets that, once hashed, give a specific hash and hence somehow it's not possible to use a quantum computer to forge what you'd want).
And from the other comment, symmetric cryptography is safe too from QC attacks.
So it's apparently as you wrote: it's really only asymmetric crypto that is at risk.
> AIUI cryptographic hashes are safe from quantum attacks (for there's an infinity of secrets that, once hashed, give a specific hash and hence somehow it's not possible to use a quantum computer to forge what you'd want).
Quantum algorithms require some sort of quantum 'trick' to actually have any speedup over classical computers. The most general quantum trick is Grover's algorithm, which lets you find f⁻¹(x) (given f and x) in sqrt(N) queries rather than N queries, where N is the size of the set from which x is drawn. This cuts the bit security of every algorithm in half, although for things like cryptographic hashes, it really means that a second preimage is now only as 'easy' as finding a collision (due to the birthday attack).
The other really well-known quantum trick is QFT, which allows you to find the period of an unknown periodic function really quickly. This is what allows quantum computers to break asymmetric algorithms based on integer factoring or elliptic curves, since they can both be expressed in terms of the QFT.
Why am I not surprised that GitHub is dragging its heels on sha256? I assume they just aren't able to change fundamental parts of their system now. So no sha256, no IPv6 etc. They can only sprinkle bits around the edges.
Length extension attacks are not an issue for git, because every object has two fields in its header, which is prepended to the object before hashing: the object type and the length in bytes.
Interesting. Anyway, since they're going through the pain of changing the hash function, why not using the latest standard? SHA3 has been standardized for some time now, and using SHA256 isn't any easier than using SHA3-256.
> they're going through the pain of changing the hash function
They've already gone through the pain, deciding on it on 2018[0] (and functional & non-experimental 3 years ago per TFA). What's left is just changing the default (and some stragglers to complete support). Changing the function now would push back changing the default by a couple additional years until the new git version gets widespread deployment (incl. on LTS distros and whatnot).
Create a WIP branch, and keep working in there committing as often as you want, then squash the history if you don't want it all when you are ready to put the changes in the “real” branch that you are working on?
What I've done since before git was a thing is a variant of my backup process: my main work areas are synced using rsync⁰ to a copy¹ that is the head of a series of snapshots. If this ends up containing any newly created/modified files²³ a new snapshot is created using `cp -al`. This way I don't have to remember to commit regularly, and I have an automatic trace of everything I've done to a certain granularity⁴. The snapshots are given a name based on the contents of a text file, if present, so I can label points in time (otherwise the snapshot names are just timestamps). Tidying up is easy, just delete old snapshots with `rm -rf`, you could automate this if you like⁵ but I've never felt the need to. The not having to remember to do anything is key for me - over the years it has saved me⁶ from harmful edits not noticed for some time that might otherwise have been more of a pain to recover from. You could do similar per repo with the WIP-branch-in-git option: have script that scans for projects in that named branch, for any found check `git status`, if there are any changes commit with the timestamp as the commit message.
--------
[0] set to ignore a few things like .git directories and some artefacts that I would list in .gitignore
[1] off on a server, that isn't key but it does give me protection against the work machine going boom as well as from accidents off my own doing
[2] detected by looking for files with only one link to them, this can be an expensive check over huge numbers of files but not for what I'm using it on
[3] the sync deletes files too, though I don't use such changes on their own as a reason to create a new snapshot
[4] much higher than the 24-hour granularity that my normal backups have, about 1440 times smaller in fact
[5] keeping them for a maximum amount of time, perhaps, and/or more complex heuristics like not keeping too many copies that are only a few minutes or less apart
[6] only a few times, but more than enough to make me glad I implemented it!
Not sure what you are looking for. What's wrong with stash? What ceremony are you referring to? `git stash` - `git stash pop` is as simple as it gets.
Then you can also do `git diff > changes.diff`. Or simply `rsync -avPh repo/ repo.snap/`, if your repo isn't huge. Or consider putting your repo in a filesystem that can do CoW snapshots.
I used to do this purely so that I could attach a name, then I found out that you can add a message when stashing: `git stash push -m "trying a thing"`
Stash is just a stack of commits. If you use stash this way too much probably you will mess something, since it needs to keep that stack data structure.
The solution: create a branch or tag with the things that you're trying. If you want to apply it, use `git merge --squash`. This way your unfinished work lives outside the stash stack!
git stash is given the UX of a stack, but it's more a list or a set of commits. The commits in your git stash don't need to be related (their commit parent pointers can point at very different branches). At some point if you are heavily using lots of stashes you tend to switch to referring to them by commit message and/or stash number instead of thinking it a stack you build bottom to top and always and only pop from the top.
There is a post[1] on the gitbutler blog where they collect a subset of defaults that allegedly git core developers use. It's where I got most of my config from.
receive.denyCurrentBranch should be updateInstead by default (or at the very least mentioned in the help message, rather than it recommending ignore or warn or refuse, none of which do what is wanted)
I wouldn't agree with all of those reasons, but it's very definitely not "just for the sake of it." One of the better reasons so many people look to writing some things in Rust is that we now have pretty ample evidence than trying to write a binary file format parser in C is a cornucopia of CVEs that are just simply absent in Rust, and the excuse of "well, but a sufficiently smart programmer doesn't write bugs in C" doesn't cut it anymore.
Somehow we have binary file format parsers written in C everywhere, so the real world shows it is possible and we do have programmers capable of doing it.
Somehow we also have memory safety bugs everywhere, too. So real world shows bugs in C code are possible. What even is your argument? Real men write asm?
Sure, we can write a binary file format parser in C. We just can't figure out how to write one that isn't buggy and lets someone infect your computer if you give it sufficiently inventive garbage.
The issue isn't whether it's possible to have parsers, but whether it's possible to have them be secure, and periodic CVEs "everywhere" suggest we don't
Aside from memory safety, which is solved by using a compiler that just doesn't allow unsafe memory operations (so not GCC or Clang upstream), which CVEs specifically would have been ameliorated by a parser written in Rust instead of C?
> Aside from memory safety, which is solved by using a compiler that just doesn't allow unsafe memory operations
I don't see how that's possible without turning the language into something that isn't C, either by adding significant new functionality (e.g. fat pointers) or subtracting enough functionality that it's a much less capable language (e.g. disallowing dynamic memory allocation).
fwiw, the use of C is infinitely more "coerced" than the use of Rust.
on my Linux system, C takes ownership of a 'top-level' /usr/include directory, all the kernel APIs have their canonical definitions in C headers, a lot of system features like nsswitch require dynamically linked C libraries etc. etc.
Rust is just something that programs can choose to be written in and that doesn't inconvenience me in any way.
Of the codebases i know that have adopted Rust, it has always been because some of their maintainers wanted to do so.
Maybe git's case is different though. Do you have more info about it? Are you a git maintainer who was coerced to use Rust, or do you know of such cases?
If that's the case, they should stop using git and Linux right now, because it's everything written in C. Having 0.1% of the code in a safe language will not change anything, it's only a bad security blanket.
> Having 0.1% of the code in a safe language will not change anything, it's only a bad security blanket.
Just because something does provide an immediate perfect solution does not mean it isn't not worth investigating and/or pursuing.
Also consider that bugs tend to be more prevalent in new code (e.g., [0]) as a result, you are likely to see more of a benefit from writing new code in a memory-safe language than raw line count proportions would indicate.
You don't believe in slowly and iteratively improving a codebase over time? Should git stick with its weird mishmash of C and perl and shell scripts forever, for tradition's sake, performance and maintainability be damned?
I don't understand your reasoning. Why should they quit git and Linux (and presumably all applications written in C) if they believe Rust is more secure than C?
It's the classic "Yet you participate in society. Curious!" response. You don't get dislike the current state of the world if you exists in it, apparently.
Rewriting it all in rust with bug for bug compatibility and byte identical outputs won’t cost more than $100k in tokens, but I don’t think this is an answer you’re looking for
> It is a binary file optimized for both space efficiency and quick access. Since then, it has been possible to create a repository that uses a reftable rather than the old file-based mechanism,
Good, are there (m)any other plans to ditch the slow files and use proper database? Or is it only reserved for various post-git competitors?
> Good, are there (m)any other plans to ditch the slow files and use proper database?
The filesystem is a proper database, just not a relational one.
Linus focused heavily on performance when he wrote git; he used the filesystem because, as the main Linux kernel maintainer, he knew that the Linux VFS and filesystems were fast enough for these use cases.
(It's the use cases that have changed; it was not expected back then to have more than a few hundred refs in a single repository.)
Ah, yeah, "you're holding it wrong", though use cases haven't changed, it's closer to the expected common case of expectations turning out wildy wrong (Why would you ever expect people to stop NAMING things at scale???)
But also the core property of the filesystem database has always been low performance for a bunch of tiny things
Not so much a "you're holding it wrong" as much more directly "we didn't expect it to be used that way". The Linux Kernel team was using it in a DVCS way with a mailing list as the primary "remote work in progress ref storage" and local refs mostly just local personal branches and tags. The "Hub" model of everyone on a project having access to nearly any and all refs in the project is different from the model of the original git developers. Neither model is "wrong" just one is more unexpected when working on the other.
(As a Windows user, I certainly can't argue that sometimes the filesystem as database has been a performance hit when using git. Though Windows filesystem performance isn't always slow, just performs differently, especially with corporate anti-virus tools involved.)
Yes, Patrick Steinhart (GitLab) has been working not only on reftables and pluggable backends for the references data, but also pluggable backends for object storage, so that you can use any database backend format (sqlite, s3, special large file storage options, etc) to store objects if you want (in addition to loose objects and packfiles).
This is work that Patrick and GitLab have been doing for years now and it's very impressive and nearly complete.
It's unfortunate that change IDs aren't considered. There was a discussion [1] in 2025, and it has resurfaced a couple of times since.
Basically, the idea is to attribute a new kind of ID to an initial 'change'. During review, or whenever a commit is rebased, the change ID is kept, whereas the commit of course changes. This allows tooling to identify all previous versions of a change, and is what enables "per-commit" code review à la Gerrit [2] (which IMO is a much better experience than the branch-review-squash model that GitHub normalized). It's also used in jj, although I'm not familiar with that.
As of today, any tool that wants a change ID needs to somehow encode it in commit message bodies. The proposed discussion was about making a change ID a standard header field that git would natively keep across rebases.
[1] https://lore.kernel.org/git/Z_OGMb-1oV0Ex05e@pks.im/T/#mf941...
[2] https://gerrit-review.googlesource.com/Documentation/user-ch...
JJ and GitButler already create and inject this into the commit headers (using the same interoperable reverse-hex format), which is recognized by Gerrit and some forges like Tangled for incremental commit based review.
I doubt that core Git will adopt it anytime soon as it was not discussed at this years contributor summit (last week) and doesn't seem to be a hot topic on the ML.
What I would like to see is support for `git rebase` not dropping it, which is the current main issue. The `git replay` command, as well as commands based on the same sequencing code (`git history` for example) do not drop custom headers like this, so there is partial non-breakage, but several of the other history editing commands do drop custom headers.
Having switched to jj I totally agree. Change IDs are a huge UX win.
Having switched to jj I don't really agree. Everything I do is basically the same as with branches, just that I get randomly generated tip names rather than naming them myself, which is both slightly convenient and slightly annoying
Yeah, "standardizing" change IDs would make it much easier to develop further tooling around it. In particular decentralized review is something that I'd be interested in.
For example, if GitHub is down, that would not be a blocker to access review comments or to do reviews. And maybe you could push your reviews to a GitLab mirror if you want a UI.
> if GitHub is down
Surely you mean when GitHub is down.
As an aside, I thought it a bit worrisome that the move to Sha256 is apparently delayed due to GitHub dragging their feet on this.
How is this different from branches?
They aren't really. It's basically like getting an auto generated ref/branch name for each new commit, with convenience rewriting every ref when you rebase.
It's a bit more convenient if you prefer referring to a non-leaf commit directly rather than relative to the leaf branch a la master~2
change ids are more analogous to commit messages than branches, eg suppose that in a feature branch you have a "Delete deprecated classes" commit; in git there is a clear idea of "cloning" this commit (eg rebase, cherrypicks, maybe reverts) and the common sense that the new commit inherits the same commit message. Change ids are the same thing but in hex id form that can be created for every new commit/stash/index.
They allow for example to identify all the clones of a commit and they allow to give stable identities across rebases eg suppose you rebase a typo at the beginning of a feature branch without change ids a reviewer sees n new unrelated commits while with change ids it is possible to clearly identify which commits where changed/added/removed since the previous review iteration.
> suppose you rebase a typo at the beginning of a feature branch without change ids a reviewer sees n new unrelated commits while with change ids it is possible to clearly identify which commits where changed/added/removed since the previous review iteration.
A rebase can introduce change to a commit in cases such as handling conflicts or squashing.
Also, a commit already retains it's commit message after rebasing.
> A rebase can introduce change to a commit in cases such as handling conflicts or squashing.
and with change ids you can quickly separate commits that changed from commit that did not.
> Also, a commit already retains it's commit message after rebasing.
but commit message are not ids, there is no command for checking out a commit by its message, nor any sense that commit with the same message are somehow functionally related
You amend or rebase a commit and can still reference it by the same name.
A branch is mutable and holds only one version of history at a time.
If you never rewrite history, you could achieve something similar, but it precludes you from having a “tidy” branch.
Whether or not you’re into rewriting history is a different discussion that has been hashed out over and over again.
Another thing that becomes easier with change IDs is reviewing multiple related commits together, essentially "stacked pull requests".
If you treat a branch as your unit of review, then it becomes super difficult for someone to submit a chain of related changes. You'll be constantly rebasing your pull requests onto each other as you get feedback from dependent branches.
I heard that the github CLI recently introduced support for this, but since in git there's no concept of dependent branches (a branch isn't even an object in git, just a reference to a commit), I think this approach will always be clunkier than reviewing commits related by a change ID.
> Another thing that becomes easier with change IDs is reviewing multiple related commits together, essentially "stacked pull requests".
What's wrong with branches?
> If you treat a branch as your unit of review, then it becomes super difficult for someone to submit a chain of related changes. You'll be constantly rebasing your pull requests onto each other as you get feedback from dependent branches.
Does it mean that when switching trop sha1 to sha256 you need to forcepush and rewrite all history? Wouldn’t that be a massive source of potential vulnerabilities?
It's much, much worse than that.
Yes, you do need to do that. However, there is also much more work after that.
Git will not intermingle SHA-256 and SHA-1 enabled repositories, even in things like submodules, so anything used in that manner will need to keep both versions into the indefinite future. If you rely on a submodule that has not yet converted, you will have to convert it yourself and try to keep it up to date, or the forge will have to automatically keep a bidirectional mirror (if you have submodules in various forges, you'll have to wait for all of them to do it), etc.
This means that every SHA referenced anywhere on the internet, in commit messages, in issues, in code comments is now invalid and needs a mapping to find the rewritten one for forever.
It also means that every commit signature ever made is now invalid and will probably have to be stripped from the rewritten new 256 history because it's impossible to resign everything.
Companies like Google and GitHub are working on keeping two versions of each repository so that there can be long stages of ecosystem migrations, but no matter what, it's going to be a huge pain for millions of developers for years to come.
It's not quite that bad. Just freeze/archive the old, rewrite everything to the new, then dump a lookup table somewhere to remap SHA1s.
Tools and scripts will have to be migrated of course, but I'm the brave new world of agentic coding it shouldn't be too hard.
The main pain is providing user support to developers who are curiously not very tech/OS savvy (has anyone else noticed this phenomenon?)
That's odd. Why not compute both sha1 and sha256 for all git objects for the foreseeable future?
Failing that, have a kind of git object that wraps another and says hey this is in sha1 don't mess with it
i guess that for now only the default will change for new repositories. support for sha1 is not going to be dropped, so most existing repositories won't switch any time soon. if you want to switch then yes, it sounds like a force push might be needed, although it could also be that simply switching is not possible, but that instead you have to create a new repo and import the history from the old repo, forcing everyone to clone the new repo intentionally.
It is a giant format change, but in the current documentation [0] sounds more like a repack than a force-push. git keeps a lookup table of the SHA1 object ids similar to an index file and some interop is allowed between SHA1 repositories and SHA256. (Primarily if you still needed to use GitHub as an SHA1 server because of some support hiccup, but needed your local repo to be SHA256 for security or other reasons, that's partially/mostly supposted.) Objects need to be resigned with their SHA256 id, but for different reasons than rebase/force-push and with a subtly different developer experience. In theory using that compatibility index of SHA1 hashes a good UI could show both signatures.
[0] Migration document: https://git-scm.com/docs/hash-function-transition
Couldn't you write something that checks every commit's content and message is byte equal to the old tree? One scan through the history to verify it should be relatively simple if not cheap. Should be built into git.
I don't know much about this. How does that enable vulnerabilities exactly?
Trusting a forced push w/o any other verification means nefarious history changes can be slipped in.
Semi-relevant-ish: https://blog.citp.princeton.edu/2013/10/09/the-linux-backdoo...
You can still verify the contents - the content blobs don’t change after the migration. Not sure if there’s a practical attack one could do but maybe
Um. how do you verify the contents? The history is for the contents you now have, not what might have been
The contents of the files don’t change, only the Merkle tree. You can verify that the content blobs all have the same sha1 by literally rehashing. Then you can verify that the contents of the clone are the same. That doesn’t stop history corruption, but it does prevent malicious injection into the current state of the tree before the migration.
Shallow clones and such would break but you could rehash the local history manually and compare the SHA256 hashes commit by commit, no?
Issue is it would be pretty slow so you'd want it to be a one time thing.
> Try LWN for free for 0 month: no payment or credit card required.
Quite a generous offer!
</aside>
`git add --resolved` is a wonderful idea, and definitely something I would start using.
It seemed obvious to me that the next version number after 2.56 would be 5.12
> The Git 2.45 release added reftables as a more efficient way of storing refs. It is a binary file optimized for both space efficiency and quick access. Since then, it has been possible to create a repository that uses a reftable rather than the old file-based mechanism, but that has never been the default. Switching to reftable should have no visible consequences (other than better performance) for users of Git itself, but it can be a problem for users of other software packages that access Git repositories. In his email, carlson mentioned libgit2 as a potential concern.
Looking forward to losing all references at once vs just the current one...
I've noticed persistent Git/fs interaction where on crash the current ref can just disappear...
It appears that BitBucket also does not currently support SHA256 hashes.
https://jira.atlassian.com/browse/BCLOUD-23729
I'm looking forward to reftable to become the default. It solves many problems with branches, like branches with weird characters created with non-standard clients making a fetch impossible; case insensitive branches with "same" names doing the same; or impossible to create branch FOO because FOO/Something exists.
All those problems just go away when branches are no longer files on disk.
I enabled it in setup script of one large repo I maintain; the main issue is the incompatibility with some people's personal tooling based on libgit2 (some git status tooling in oh-my-zsh), but people do find workarounds.
So Git, in version 3.0, is moving to SHA-256 by default for SHA-1 ain't considered that strong anymore but...
What about future attacks by quantum computers? Is Git safe from quantum computers for it's all hashes only? Or shall there be issues with quantum attacks?
I'm asking for there are several projects that are already moving to quantum-resistant schemes (like OpenSSH who uses an hybrid scheme [1]).
[1] https://www.openssh.org/pq.html
At this point, it does not appear that (reasonably strong and modern) symmetric cryptography (ciphers, hash functions, etc.) is realistically threatened by quantum computing: https://words.filippo.io/128-bits/
Thanks for the great link! This has been vexing me, as intuitively it seems like if quantum computers can test all factors they should be able to test all keys.
But the article helps. Basically Grover’s is not as potent as Shorr’s. And it seems like everyone is convinced there is no dramatically better quantum algorithm than Grover’s?
No. Not quite. In fact, that blog post ignores something important from the very papers it cites.
Grover's assumes the function is a black box that you cannot look inside and that your only way of finding a certain result is through repeated invocation.
Under this assumption, Grover's is optimal in the number of invocations of the function required to find the result.
However, this assumption may be quite wrong for AES and friends. It may be the structure allows for non brute force attacks that are totally impractical classically but not subject to Grover's optimality limitation quantumly.
The only thing you are guaranteed here is that if you cannot take advantage of structure at all then Grover's is the best you can do.
Given that we have pretty much always found a way to take some advantage of structure, I would bet we will do so here.
That may or may not make it viable to break at all, I just wouldn't bet that it must be treated like a black box forever.
To me that would be a very bad bet.
Thank you again! That’s exactly what my intuition was reaching for by my expertise was too short to support.
And, if I’m following you, that’s the key difference in Grover’s and Shorr’s: Shorr’s takes advantage of structure?
Yes. Structure comes in many forms. In this case, Shor is turning the function into a strongly periodic algebraic structure.
I'll explain it without going too far into why any of this is true, which is much more complicated to prove. This will let me use relatively simple math.
Let's say you want to factor N. Pick some number that is coprime to N, which we'll call a, and consider f(x) = a^x (mod N).
Since it's a modular function, it repeats at some point. Shor calculates the period of this function (r), rather than seeing which of the 2^n numbers is "the answer".
Once you know the period of this function, there is a high chance that the factors fall out of gcd(a^(r/2) - 1, N) and gcd(a^(r/2)+1, N).
The point here is not to explain Shor's as much as to point out it is finding a strong amount of structure to take advantage of, quantumly.
This is actually the same way the oracle separation of BQP and the entire polynomial hiearchy works[1] - It depends on forrelation, which is a problem where quantum computers can extract a global property of the function without needing to learn all the individual values, by taking advantage of structure.
Which is why i go to "The idea that there is literally no structure that can be taken advantage of in AES strikes me as a bad bet".
In part because it's already false if you go literature searching. For example, https://www.sciencedirect.com/science/article/abs/pii/S00200...
There are already reduced round quantum attacks on AES as well. Again, more to the point, the idea that symmetric key ciphers and cryptographic hashes in general are safe because grover's is slower than shor's is not a thing i would bet on at all. Even if AES ends up relatively safe, that tells you basically nothing about the other practically-used ciphers and functions since there are a lot of different construction mechanisms being used.
[1] People still seem to believe there are no functions which quantum computing models have been been proven to be faster at than classical computing models. This is false. Forrelation is the canonical example - and shows that BQP can perform things exponentially faster than you can classically even given access to an infinite polynomial hierarchy.
It is the current physical actualization of these computing models that have the "is it really faster than classical computers" issue, not the theory ;)
(IE it is a perfect example of "in theory there is no difference between theory and practice, and in practice, there is")
Thanks so much for the patient and accessible explanation. So much more useful and grounded than the naive “quantum is faster because it can consider all possible solutions at once” canard.
Good point here I had never thought about, but it's still good to upgrade to 256 bits when possible for peace of mind.
This has nothing to do with SHAttered
But their post-quantum-algorithm also uses sha256. Afaik only asymetric crypto is in danger from quantum computers.
Ah it's interesting, AIUI cryptographic hashes are safe from quantum attacks (for there's an infinity of secrets that, once hashed, give a specific hash and hence somehow it's not possible to use a quantum computer to forge what you'd want).
And from the other comment, symmetric cryptography is safe too from QC attacks.
So it's apparently as you wrote: it's really only asymmetric crypto that is at risk.
> AIUI cryptographic hashes are safe from quantum attacks (for there's an infinity of secrets that, once hashed, give a specific hash and hence somehow it's not possible to use a quantum computer to forge what you'd want).
Quantum algorithms require some sort of quantum 'trick' to actually have any speedup over classical computers. The most general quantum trick is Grover's algorithm, which lets you find f⁻¹(x) (given f and x) in sqrt(N) queries rather than N queries, where N is the size of the set from which x is drawn. This cuts the bit security of every algorithm in half, although for things like cryptographic hashes, it really means that a second preimage is now only as 'easy' as finding a collision (due to the birthday attack).
The other really well-known quantum trick is QFT, which allows you to find the period of an unknown periodic function really quickly. This is what allows quantum computers to break asymmetric algorithms based on integer factoring or elliptic curves, since they can both be expressed in terms of the QFT.
As far as I understand, quantum computing can cut it in half, to 128 effective bits. Still way too large to brute force from there.
Why am I not surprised that GitHub is dragging its heels on sha256? I assume they just aren't able to change fundamental parts of their system now. So no sha256, no IPv6 etc. They can only sprinkle bits around the edges.
TFA literally notes that one if the key sha256 devs is a github employee and in favor of the transition.
And sha256 is in private preview at GitHub: https://github.com/bk2204/talk-rust-in-git/blob/dev/presenta...
They don't seem to have any problems cramming in ever more AI garbage down our throats.
Priorities!
It is on the horizon: https://github.com/bk2204/talk-rust-in-git uses SHA256
Why not transitioning to SHA3 directly? IIRC lenght extension attacks are not currently feasible on SHA2, but still theoretically possible.
Length extension attacks are not an issue for git, because every object has two fields in its header, which is prepended to the object before hashing: the object type and the length in bytes.
Interesting. Anyway, since they're going through the pain of changing the hash function, why not using the latest standard? SHA3 has been standardized for some time now, and using SHA256 isn't any easier than using SHA3-256.
> they're going through the pain of changing the hash function
They've already gone through the pain, deciding on it on 2018[0] (and functional & non-experimental 3 years ago per TFA). What's left is just changing the default (and some stragglers to complete support). Changing the function now would push back changing the default by a couple additional years until the new git version gets widespread deployment (incl. on LTS distros and whatnot).
[0]: https://github.com/git/git/commit/0ed8d8da374f648764758f1303...
SHA3 does not supersede SHA2. They are different types of hashes useful for different purposes.
I'm still looking for a simple way to "save" a snapshot of my work in git, without all the ceremony of stashing etc
What could be simpler than "$ git stash" and "$ git stash pop" ?
Create a WIP branch, and keep working in there committing as often as you want, then squash the history if you don't want it all when you are ready to put the changes in the “real” branch that you are working on?
What I've done since before git was a thing is a variant of my backup process: my main work areas are synced using rsync⁰ to a copy¹ that is the head of a series of snapshots. If this ends up containing any newly created/modified files²³ a new snapshot is created using `cp -al`. This way I don't have to remember to commit regularly, and I have an automatic trace of everything I've done to a certain granularity⁴. The snapshots are given a name based on the contents of a text file, if present, so I can label points in time (otherwise the snapshot names are just timestamps). Tidying up is easy, just delete old snapshots with `rm -rf`, you could automate this if you like⁵ but I've never felt the need to. The not having to remember to do anything is key for me - over the years it has saved me⁶ from harmful edits not noticed for some time that might otherwise have been more of a pain to recover from. You could do similar per repo with the WIP-branch-in-git option: have script that scans for projects in that named branch, for any found check `git status`, if there are any changes commit with the timestamp as the commit message.
--------
[0] set to ignore a few things like .git directories and some artefacts that I would list in .gitignore
[1] off on a server, that isn't key but it does give me protection against the work machine going boom as well as from accidents off my own doing
[2] detected by looking for files with only one link to them, this can be an expensive check over huge numbers of files but not for what I'm using it on
[3] the sync deletes files too, though I don't use such changes on their own as a reason to create a new snapshot
[4] much higher than the 24-hour granularity that my normal backups have, about 1440 times smaller in fact
[5] keeping them for a maximum amount of time, perhaps, and/or more complex heuristics like not keeping too many copies that are only a few minutes or less apart
[6] only a few times, but more than enough to make me glad I implemented it!
Not sure what you are looking for. What's wrong with stash? What ceremony are you referring to? `git stash` - `git stash pop` is as simple as it gets.
Then you can also do `git diff > changes.diff`. Or simply `rsync -avPh repo/ repo.snap/`, if your repo isn't huge. Or consider putting your repo in a filesystem that can do CoW snapshots.
I always just create a branch
I used to do this purely so that I could attach a name, then I found out that you can add a message when stashing: `git stash push -m "trying a thing"`
Stash is just a stack of commits. If you use stash this way too much probably you will mess something, since it needs to keep that stack data structure.
The solution: create a branch or tag with the things that you're trying. If you want to apply it, use `git merge --squash`. This way your unfinished work lives outside the stash stack!
git stash is given the UX of a stack, but it's more a list or a set of commits. The commits in your git stash don't need to be related (their commit parent pointers can point at very different branches). At some point if you are heavily using lots of stashes you tend to switch to referring to them by commit message and/or stash number instead of thinking it a stack you build bottom to top and always and only pop from the top.
Even more lightweight is a tag -- 'git-log --all' will list these for you.
Jujutsu
Will we ever get integer commit references like we have in mercurial ?
I will always mourn hg losing to git in the DVCS wars, however it was generally considered bad practice to rely on the integer commit references.
Why, because it shifts?
That is not a problem for local use + constant repo state.
so it is a problem for majority of use
Are they going to fix all the bad defaults in Git 3.0?
For reference, what would you say those bad defaults are? (I would like to know if I should consider changing my configuration)
There is a post[1] on the gitbutler blog where they collect a subset of defaults that allegedly git core developers use. It's where I got most of my config from.
[1] https://blog.gitbutler.com/how-git-core-devs-configure-git
Here are a few:
1. Git push should default to --force-with-lease --force-if-includes.
2. push.autoSetupRemote should be enabled by default.
3. The default conflict style should be zdiff3.
4. diff.submodule should be 'log' by default (gives much nicer submodule diffs).
5. Submodule updates / clones should be recursive by default. (There is a setting for this but I can't remember it.)
Also would be nice:
receive.denyCurrentBranch should be updateInstead by default (or at the very least mentioned in the help message, rather than it recommending ignore or warn or refuse, none of which do what is wanted)
> 3. The default conflict style should be zdiff3.
This one always baffled me. The default conflictstyle is so hard to read it's almost useless. Using diff3 is mandatory.
I hadn't heard of zdiff3, I'll give it a shot.
rerere should be on
[flagged]
It is regrettable that they're trying to coerce the use of Rust everywhere just for the sake of it. It's a nonsense that is now forced on everyone.
The comments gives a link to a recent talk about the motivation for using Rust in Git: https://github.com/bk2204/talk-rust-in-git/blob/dev/presenta...
I wouldn't agree with all of those reasons, but it's very definitely not "just for the sake of it." One of the better reasons so many people look to writing some things in Rust is that we now have pretty ample evidence than trying to write a binary file format parser in C is a cornucopia of CVEs that are just simply absent in Rust, and the excuse of "well, but a sufficiently smart programmer doesn't write bugs in C" doesn't cut it anymore.
Somehow we have binary file format parsers written in C everywhere, so the real world shows it is possible and we do have programmers capable of doing it.
Somehow we also have memory safety bugs everywhere, too. So real world shows bugs in C code are possible. What even is your argument? Real men write asm?
Sure, we can write a binary file format parser in C. We just can't figure out how to write one that isn't buggy and lets someone infect your computer if you give it sufficiently inventive garbage.
The issue isn't whether it's possible to have parsers, but whether it's possible to have them be secure, and periodic CVEs "everywhere" suggest we don't
Aside from memory safety, which is solved by using a compiler that just doesn't allow unsafe memory operations (so not GCC or Clang upstream), which CVEs specifically would have been ameliorated by a parser written in Rust instead of C?
Aside from the fact that it's not solved by using an alternative compiler, why would you put the core advantage aside?
What?
> Aside from memory safety, which is solved by using a compiler that just doesn't allow unsafe memory operations
I don't see how that's possible without turning the language into something that isn't C, either by adding significant new functionality (e.g. fat pointers) or subtracting enough functionality that it's a much less capable language (e.g. disallowing dynamic memory allocation).
Behold: https://fil-c.org/
An important improvement over rust is that "Fil-C has no unsafe statement."
As everything, there are compromises and prices to pay.
In case of fil-c, it is about 1.5-4x slower performance, and a memory overhead.
So, let's not present it as a panacea to all problems: there could good reasons to use it, but it isn't a magic trick.
Rust is slower too, and git is IO bound anyway, and routinely calls bash.
Rust is not 1.5-4x slower at all. Git is not IO bound at all, it is not saturating your IO device, it just performs IO a lot.
fwiw, the use of C is infinitely more "coerced" than the use of Rust.
on my Linux system, C takes ownership of a 'top-level' /usr/include directory, all the kernel APIs have their canonical definitions in C headers, a lot of system features like nsswitch require dynamically linked C libraries etc. etc.
Rust is just something that programs can choose to be written in and that doesn't inconvenience me in any way.
It doesn't need to be that way: https://gobolinux.org/at_a_glance.html
Of the codebases i know that have adopted Rust, it has always been because some of their maintainers wanted to do so.
Maybe git's case is different though. Do you have more info about it? Are you a git maintainer who was coerced to use Rust, or do you know of such cases?
I don't think it's "just for the sake of it". I think they believe that the Rust code will be safer.
If that's the case, they should stop using git and Linux right now, because it's everything written in C. Having 0.1% of the code in a safe language will not change anything, it's only a bad security blanket.
> Having 0.1% of the code in a safe language will not change anything, it's only a bad security blanket.
Just because something does provide an immediate perfect solution does not mean it isn't not worth investigating and/or pursuing.
Also consider that bugs tend to be more prevalent in new code (e.g., [0]) as a result, you are likely to see more of a benefit from writing new code in a memory-safe language than raw line count proportions would indicate.
[0]: https://security.googleblog.com/2024/09/eliminating-memory-s...
You don't believe in slowly and iteratively improving a codebase over time? Should git stick with its weird mishmash of C and perl and shell scripts forever, for tradition's sake, performance and maintainability be damned?
I don't understand your reasoning. Why should they quit git and Linux (and presumably all applications written in C) if they believe Rust is more secure than C?
It's the classic "Yet you participate in society. Curious!" response. You don't get dislike the current state of the world if you exists in it, apparently.
I hope you don't use seatbelts in your car, as they won't help you against a fire.
Rewriting it all in rust with bug for bug compatibility and byte identical outputs won’t cost more than $100k in tokens, but I don’t think this is an answer you’re looking for
Is all use of Rust "coerced" and "forced on everyone", or is there a way to write things in it that makes sense?
Waiting for 3.0 to land and emancipate us from our 'master's. Bringing about true social change, one SHA at a time.
> It is a binary file optimized for both space efficiency and quick access. Since then, it has been possible to create a repository that uses a reftable rather than the old file-based mechanism,
Good, are there (m)any other plans to ditch the slow files and use proper database? Or is it only reserved for various post-git competitors?
> Good, are there (m)any other plans to ditch the slow files and use proper database?
The filesystem is a proper database, just not a relational one.
Linus focused heavily on performance when he wrote git; he used the filesystem because, as the main Linux kernel maintainer, he knew that the Linux VFS and filesystems were fast enough for these use cases.
(It's the use cases that have changed; it was not expected back then to have more than a few hundred refs in a single repository.)
There are lots of places it'd be useful to use Git that don't have filesystems.
> more than a few hundred refs
Ah, yeah, "you're holding it wrong", though use cases haven't changed, it's closer to the expected common case of expectations turning out wildy wrong (Why would you ever expect people to stop NAMING things at scale???)
But also the core property of the filesystem database has always been low performance for a bunch of tiny things
Not so much a "you're holding it wrong" as much more directly "we didn't expect it to be used that way". The Linux Kernel team was using it in a DVCS way with a mailing list as the primary "remote work in progress ref storage" and local refs mostly just local personal branches and tags. The "Hub" model of everyone on a project having access to nearly any and all refs in the project is different from the model of the original git developers. Neither model is "wrong" just one is more unexpected when working on the other.
(As a Windows user, I certainly can't argue that sometimes the filesystem as database has been a performance hit when using git. Though Windows filesystem performance isn't always slow, just performs differently, especially with corporate anti-virus tools involved.)
Yes, Patrick Steinhart (GitLab) has been working not only on reftables and pluggable backends for the references data, but also pluggable backends for object storage, so that you can use any database backend format (sqlite, s3, special large file storage options, etc) to store objects if you want (in addition to loose objects and packfiles).
This is work that Patrick and GitLab have been doing for years now and it's very impressive and nearly complete.
Not sure if this post is an exhaustive list. Is that stuff making it to 3.0 or further out?
By "proper" I assume you mean relational? Or ACID? Or you mean using existing database software? What is so improper about the way git stores data?
It's not just slowness, but what about case insensitive filesystems?