Tangential, but I almost never read assembly [1], but I do read Java bytecode pretty frequently, primarily because doing that can sometimes be a good substitute for benchmarking [2], which I do not enjoy.
The thing that never seems to stop tripping me up is the different “dup” codes that compile. At some point I really need to properly learn the difference between dup_x2 and dup2_x1 and dup2_x2.
[1] not out of like an ethical objection, just my career has involved almost no reverse engineering and it’s also never been a path I have been super interested in to pursue on my own.
[2] e.g. if two competing chunks of code emit the same bytecode, you don’t need to pull out JMH. My go to example for this is using if statements vs switches, which will usually emit the same code so performance arguments are moot.
Thus finally the 0F FF believers were rewarded by being give the honor of op code UD0 making it the one and true original invalid opcode permanently disgracing the 0F B9 adherents with the shame of UD1.
We also got UDB (D6), the one-byte variant that arrived with x86-64 for 64-bit mode.
And we have always had UDW (FF FF), aka group #5 (1st FF) with a modrm byte of mod=11b r/m=111b (/7) reg=111b (2nd FF) -- that one matters for memory with all bits set to 1, or for buses terminated to all 1 when no device claims an access.
I'm not much of an x86 person but on other architectures you can raise software interrupts/exceptions. Does x86 not have this or did those facilities not cover enough use cases?
Maybe because if the code wants to call the invalid opcode interrupt handler (INT6), it needs extra code to populate the flags and registers expected by that handler, whereas actually triggering an invalid opcode exception will get all those parameters populated automatically.
Already since Intel 8086, x86 has the instruction "INT vector_number", whose purpose is to allow software to invoke directly any of the many kinds of exception handlers or hardware interrupt handlers that are specified by the ISA or implemented by the hardware designer, which are normally invoked when various conditions arise, as determined by software execution or by I/O events.
So you can invoke the handler of the invalid instruction exception with the INT instruction, but as another poster mentioned, the INT instruction alone is not enough for this, but you need to setup the stack in such a way so that it will contain the information expected by the exception handler, which requires multiple instructions.
This kind of invocation may be acceptable when you write a test program for the invalid instruction exception handler, but it is not acceptable when you want to initialize some guard memory with values that will trigger the exception, to signal that your program has attempted to execute instructions from an area that should not be executable. Setting a memory area as non-executable through the access rights has only page granularity, so it is not useful when a page must contain both some executable code and some non-executable data.
If Intel had not defined an official opcode that is guaranteed to remain unused forever, to be able to reliably trigger the invalid instruction exception, the workaround would have been for the user to reserve one of the 256 interrupt vectors for the invocation through software of the invalid instruction exception. For that vector, a simple handler could have been used, which would have setup the stack in the right way, before jumping to the invalid instruction handler.
But this workaround would have had the disadvantage that any chosen interrupt vector could have conflicted with some choice made by the hardware designers of some computers, so it would have been required for it to be a configurable parameter of the operating system kernel, and also of the user applications that need it, like compilers, unless it would have been standardized by some organization.
Just reserving an opcode at Intel and AMD was simpler, with no other requirements for standardization or changes in the existing software.
#UD has the same stack frame as a software interrupt, there's no error code pushed. But most likely, executing INT 06 from ring 3 will generate a protection fault instead, since the gate descriptor would be set up to not be reachable from that privilege level.
(exceptions that do push an error code couldn't be emulated at all using INT, since the error code is the last thing pushed by the CPU, after flags and return address)
> the INT instruction alone is not enough for this, but you need to setup the stack in such a way so that it will contain the information expected by the exception handler,
Wait, what? The x86 CPUs construct the stack frame themselves before jumping to the interrupt handler, otherwise e.g. INT3 wouldn't work.
It's basically a convention. The alternative is to raise interrupts of course, but that might be application specific, or use other invalid instructions than the designated one, but they might work differently on other processor types.
There's a bit of convention and practicality The only thing you really need is that your "fatal error" instruction and "syscall" instruction can be reasonably discriminated without needing to set registers at the call site. Needing to set register to identify a fatal error is not great for code size, especially in languages that generate a lot of them (memory safe languages, mostly).
Though, yes, convention does play a role. On ARMv8 you get both SVC <imm> and BRK <imm>. SVC and BRK raise different exception codes (which satisfies the "easy to distinguish requirement) but in principle you could just use BRK with a well-known immediate and eliminate the need for SVC since BRK's immediate is reported in the exception status register. And, anyways, if you have an SVC instruction and a BRK instruction, you may as well use the SVC instruction for syscalls since it's right there.
ARMv8 also gives you a a UDF imm, for a guaranteed undefined insn with an immediate payload.
The reason to want a true UDF imm with an immediate comes down to it being pretty solidly guaranteed that it's going to turn into your language/OS equivalent of a SIGILL insn. In theory an OS could by convention allocate some subset of BRK space for arbitrary userspace purposes, but in practice none did, so trying to use BRK gets you dumped into a debugger, or doesn't have consistent behaviour. It's nice for userspace to have something that doesn't need active OS support.
It's common to use int3 for some of the scenarios mentioned in the article. (Like non-reachable code) This instruction is often used to trigger a break in the debugger.
I expect that UD2 stops instruction fetching (beyond the current block) and conversion to µops. A software interrupt or supervisor call should probably do neither because most of the time, these instructions eventually return and continue executing the next instruction.
An interrupt or SYSCALL instruction could do anything, which includes remapping or overwriting the memory location it returns to. So no, these instructions can't be prefetched in any case.
exception handling requires that some unrelated region of memory is initialized and intact and ready to do the right thing, whatever that is, and that region is outside the scope of your control, it belongs to the operating system or the the embedded ROM, and it may not have been laid out to take care of your case.
assembly/machine code is operating at a lower layer: "I don't know what larger thing I'm a part of, but I know I need to stop."
What the instruction does is well documented, and the history of other invalid instructions being used for the same purpose in the past I'm guessing there is also well known, though a quick search doesn't turn up any official Intel documentation on the matter.
Given who this is and the overall quality of his output over the years, I'm willing to trust it isn't pure guesswork - and anyway, I'd trust his guesswork over many other people's absolute facts.
A: and B: were not specific to the type of drive. It was common to have two drives before fixed storage became common, often you would have the application disk in one and your data disk in the other though there were other common use patterns for two drives also (with the OS, or at least the core of it, resident in memory you can copy and otherwise manage data over two data disks, and so on).
The first hard-drive in a system was made C: to reserve A: and B: in part because there was software out there that assumed A: and B: were floppy drives and could cause problems if something else was allocated to those signifiers. There are many things that are due to long forgotten compatibility issues like this (try naming file LPT1 under Windows to see another). Another reason is that the BIOS on many PCs was just hard-wired to assume two floppy drives so DOS would see that even if there were no drives really there.
From what I recall from the time I still used MS-DOS, even if you only have one floppy disk drive, it's accessible as both A: and B: and it prompts you to insert the other floppy disk when you change the drive letter. That is, it's "virtualizing" two floppy disk drives using a single physical one. That explains why the first hard disk drive was always C: even when you only had one floppy disk drive (and of course you had at least one floppy disk drive, how could you use a computer without one?)
> and of course you had at least one floppy disk drive, how could you use a computer without one?
They were relatively rare, but there absolutely were DOS-era machines without a floppy drive – found in schools, corporations, government agencies.
The most common implementation used Novell NetWare's RPL protocol to boot using an Ethernet card with an installed RPL BOOT ROM. The machine didn't need any disk drives (hard disk or floppy) at all, it loaded MS-DOS off a NetWare server, which was also used for all files or data.
You could also install a hard disk for local storage, without installing a floppy drive, but the more common configuration was no disks at all.
Also, the original IBM PC supported external floppy drives; as the 80s went on, external floppy connectors became increasingly rare, but if you specifically wanted an ISA FDC card with an external connector, they were still making new ones into the 1990s. (These didn't use anything like USB; they took the signals of the internal floppy ribbon cable, and surfaced them on a port on the rear of the card.) So, you could have a machine with a hard disk but without a floppy most of the time, and you'd plug in an external drive when you needed it, and you could keep it locked in a cupboard out of the hands of the plebian users the rest of the time. I doubt many people did this – diskless workstations were a lot less work – but probably somebody did at some point.
Which is very convenient (and necessary for backwards compatibility) because programs could be (and were) written to assume they're in B: (and the DOS disk is in A:), or that they're in A: and the user's data is in B:, or a two-disk program is in both. All of that just worked (tm) even with a single drive, as long as the program didn't try to interleave accesses to both drives (I think it would still work because DOS was single-process, single-thread, it would just annoy the user!)
Both of these things are true: commands like diskcopy have special handling to allow the same drive to be used as both source and destination, and the DOS I/O has special handling to request disk changes from A: to B: on single-drive systems for programs that lack specific support.
Even further back into time, before 3.5" disks, both A: and B: were 5.25" disks. And, although my memory is hazy, 3.5's were commonly slotted into B: at first, because no one had 3.5" boot disks until the drives became somewhat common.
You are forgetting 8” floppies. IBM used only the soft sector ones (one hole punched close to the spindle to mark sector 0) but there were also hard sector ones (a ring of 32 holes close to the spindle hole).
There was a lot of experimentation back in those days. The Wikipedia’s page on floppy disks is surprisingly long!
8" floppy drives were not compatible mechanically with IBM PC cases, but there were 8" floppy drives with their own enclosures and power supplies, which could be put on a desktop along the PC case.
IBM PCs have never supported 8" floppy drives, but it was easy to make an adapter between IBM PC floppy cables and 8" floppy drives, so there have existed IBM PC clones from countries where 5¼" floppies were scarce (e.g. Eastern Europe), which supported the attachment of 8" floppy drives.
The original 5¼" floppies had only the size advantage, but they had both a lower capacity and a lower speed than 8" floppies, so attaching 8" floppy drives would have been a higher performance option in the beginning.
Only after the IBM PC/AT introduced the high-density 1.2 Mbyte floppy disks, the 5¼" format matched (actually very slightly exceeded) the performance of the old 8" floppies.
But in the context of drive lettering, although CP/M supported 8" floppies, MS-DOS never did. MS-DOS adopted the naming scheme, but not the boot style.
Floppies, superfloppies, removable hard disks (platter-only, as opposed to modern external hard drives that incorporate the entire drive mechanism), magneto-optical disks, rewritable CD/DVD, even today magnetic tape is still used as removable storage in applications where large capacity is required and shelf life is more important than fast random access.
The floppies got A and B because hard drives were expensive and for a time a lot of people got by without them, though everybody had at least one floppy.
Even if you only had one floppy, it was both A and B, so you could say "copy a:DOC.TXT b:" and it would read the doc and then ask you to insert the floppy you're considering B (and depending on file size and available memory, sometimes switch back and forth a few more times)
I have vague memories of something like this from my childhood, where my family's IBM 486SX had the 3.5" drive on A and the 5.25" drive on B but my grandparents' Epson 286 had the opposite. Also learning about disk densities when none of the disks from the 486 worked on the 286.
> It’s called ud2 because the 0F FF variant was retroactively named ud0, and the 0F B9 variant was retroactively named ud1, leaving ud2 as the recommended undefined opcode.
It was a surprise for me as a reader. When I came to this sentence I assumed that 0f ff would become #1 and 0fb9 -- #2. But no, Intel counts from zero, so there is a third ud.
> Better to stick with ud2. Its behavior is consistent and architecturally guaranteed.
Ah, finally an undefined instruction whose behaviour is consistent and architecturally guaranteed!
Schrodinger's instruction: simultaneously defined and undefined
Tangential, but I almost never read assembly [1], but I do read Java bytecode pretty frequently, primarily because doing that can sometimes be a good substitute for benchmarking [2], which I do not enjoy.
The thing that never seems to stop tripping me up is the different “dup” codes that compile. At some point I really need to properly learn the difference between dup_x2 and dup2_x1 and dup2_x2.
[1] not out of like an ethical objection, just my career has involved almost no reverse engineering and it’s also never been a path I have been super interested in to pursue on my own.
[2] e.g. if two competing chunks of code emit the same bytecode, you don’t need to pull out JMH. My go to example for this is using if statements vs switches, which will usually emit the same code so performance arguments are moot.
Thus finally the 0F FF believers were rewarded by being give the honor of op code UD0 making it the one and true original invalid opcode permanently disgracing the 0F B9 adherents with the shame of UD1.
0 undoubtedly comes first, but we all know 1 == true...
That will show'em.
Hyrum’s Law reaching all the way down to the instruction decoder. And for those who care the ud2 opcode is 0F 0B
Nowadays UD0 UD1 UD2 are in the SDM and APM.
We also got UDB (D6), the one-byte variant that arrived with x86-64 for 64-bit mode.
And we have always had UDW (FF FF), aka group #5 (1st FF) with a modrm byte of mod=11b r/m=111b (/7) reg=111b (2nd FF) -- that one matters for memory with all bits set to 1, or for buses terminated to all 1 when no device claims an access.
I'm not much of an x86 person but on other architectures you can raise software interrupts/exceptions. Does x86 not have this or did those facilities not cover enough use cases?
Maybe because if the code wants to call the invalid opcode interrupt handler (INT6), it needs extra code to populate the flags and registers expected by that handler, whereas actually triggering an invalid opcode exception will get all those parameters populated automatically.
And this is code that will (hopefully) almost never run, so you don't want it to take up much space in you your program, and especially cache lines.
Already since Intel 8086, x86 has the instruction "INT vector_number", whose purpose is to allow software to invoke directly any of the many kinds of exception handlers or hardware interrupt handlers that are specified by the ISA or implemented by the hardware designer, which are normally invoked when various conditions arise, as determined by software execution or by I/O events.
So you can invoke the handler of the invalid instruction exception with the INT instruction, but as another poster mentioned, the INT instruction alone is not enough for this, but you need to setup the stack in such a way so that it will contain the information expected by the exception handler, which requires multiple instructions.
This kind of invocation may be acceptable when you write a test program for the invalid instruction exception handler, but it is not acceptable when you want to initialize some guard memory with values that will trigger the exception, to signal that your program has attempted to execute instructions from an area that should not be executable. Setting a memory area as non-executable through the access rights has only page granularity, so it is not useful when a page must contain both some executable code and some non-executable data.
If Intel had not defined an official opcode that is guaranteed to remain unused forever, to be able to reliably trigger the invalid instruction exception, the workaround would have been for the user to reserve one of the 256 interrupt vectors for the invocation through software of the invalid instruction exception. For that vector, a simple handler could have been used, which would have setup the stack in the right way, before jumping to the invalid instruction handler.
But this workaround would have had the disadvantage that any chosen interrupt vector could have conflicted with some choice made by the hardware designers of some computers, so it would have been required for it to be a configurable parameter of the operating system kernel, and also of the user applications that need it, like compilers, unless it would have been standardized by some organization.
Just reserving an opcode at Intel and AMD was simpler, with no other requirements for standardization or changes in the existing software.
#UD has the same stack frame as a software interrupt, there's no error code pushed. But most likely, executing INT 06 from ring 3 will generate a protection fault instead, since the gate descriptor would be set up to not be reachable from that privilege level.
(exceptions that do push an error code couldn't be emulated at all using INT, since the error code is the last thing pushed by the CPU, after flags and return address)
> the INT instruction alone is not enough for this, but you need to setup the stack in such a way so that it will contain the information expected by the exception handler,
Wait, what? The x86 CPUs construct the stack frame themselves before jumping to the interrupt handler, otherwise e.g. INT3 wouldn't work.
It's basically a convention. The alternative is to raise interrupts of course, but that might be application specific, or use other invalid instructions than the designated one, but they might work differently on other processor types.
There's a bit of convention and practicality The only thing you really need is that your "fatal error" instruction and "syscall" instruction can be reasonably discriminated without needing to set registers at the call site. Needing to set register to identify a fatal error is not great for code size, especially in languages that generate a lot of them (memory safe languages, mostly).
Though, yes, convention does play a role. On ARMv8 you get both SVC <imm> and BRK <imm>. SVC and BRK raise different exception codes (which satisfies the "easy to distinguish requirement) but in principle you could just use BRK with a well-known immediate and eliminate the need for SVC since BRK's immediate is reported in the exception status register. And, anyways, if you have an SVC instruction and a BRK instruction, you may as well use the SVC instruction for syscalls since it's right there.
ARMv8 also gives you a a UDF imm, for a guaranteed undefined insn with an immediate payload.
The reason to want a true UDF imm with an immediate comes down to it being pretty solidly guaranteed that it's going to turn into your language/OS equivalent of a SIGILL insn. In theory an OS could by convention allocate some subset of BRK space for arbitrary userspace purposes, but in practice none did, so trying to use BRK gets you dumped into a debugger, or doesn't have consistent behaviour. It's nice for userspace to have something that doesn't need active OS support.
It's common to use int3 for some of the scenarios mentioned in the article. (Like non-reachable code) This instruction is often used to trigger a break in the debugger.
I expect that UD2 stops instruction fetching (beyond the current block) and conversion to µops. A software interrupt or supervisor call should probably do neither because most of the time, these instructions eventually return and continue executing the next instruction.
An interrupt or SYSCALL instruction could do anything, which includes remapping or overwriting the memory location it returns to. So no, these instructions can't be prefetched in any case.
>you can raise software interrupts/exceptions
exception handling requires that some unrelated region of memory is initialized and intact and ready to do the right thing, whatever that is, and that region is outside the scope of your control, it belongs to the operating system or the the embedded ROM, and it may not have been laid out to take care of your case.
assembly/machine code is operating at a lower layer: "I don't know what larger thing I'm a part of, but I know I need to stop."
x86 has...
INT Ib INT1 INT3 INTO BOUND
I recently had to debug builds that failed randomly and a ud2 from V8 was there waiting for me.
Is this just his speculation? Or is there evidence for it?
What the instruction does is well documented, and the history of other invalid instructions being used for the same purpose in the past I'm guessing there is also well known, though a quick search doesn't turn up any official Intel documentation on the matter.
Given who this is and the overall quality of his output over the years, I'm willing to trust it isn't pure guesswork - and anyway, I'd trust his guesswork over many other people's absolute facts.
This is a first hand account of somebody very knowledgeable and respected in the industry at the time of the events. I would say he is the evidence.
It's like why the first (hard) drive letter is C.
A: drive is 3.5; B: drive is 5.25; C: drive is hard disk
A: and B: were not specific to the type of drive. It was common to have two drives before fixed storage became common, often you would have the application disk in one and your data disk in the other though there were other common use patterns for two drives also (with the OS, or at least the core of it, resident in memory you can copy and otherwise manage data over two data disks, and so on).
The first hard-drive in a system was made C: to reserve A: and B: in part because there was software out there that assumed A: and B: were floppy drives and could cause problems if something else was allocated to those signifiers. There are many things that are due to long forgotten compatibility issues like this (try naming file LPT1 under Windows to see another). Another reason is that the BIOS on many PCs was just hard-wired to assume two floppy drives so DOS would see that even if there were no drives really there.
From what I recall from the time I still used MS-DOS, even if you only have one floppy disk drive, it's accessible as both A: and B: and it prompts you to insert the other floppy disk when you change the drive letter. That is, it's "virtualizing" two floppy disk drives using a single physical one. That explains why the first hard disk drive was always C: even when you only had one floppy disk drive (and of course you had at least one floppy disk drive, how could you use a computer without one?)
> and of course you had at least one floppy disk drive, how could you use a computer without one?
They were relatively rare, but there absolutely were DOS-era machines without a floppy drive – found in schools, corporations, government agencies.
The most common implementation used Novell NetWare's RPL protocol to boot using an Ethernet card with an installed RPL BOOT ROM. The machine didn't need any disk drives (hard disk or floppy) at all, it loaded MS-DOS off a NetWare server, which was also used for all files or data.
You could also install a hard disk for local storage, without installing a floppy drive, but the more common configuration was no disks at all.
Also, the original IBM PC supported external floppy drives; as the 80s went on, external floppy connectors became increasingly rare, but if you specifically wanted an ISA FDC card with an external connector, they were still making new ones into the 1990s. (These didn't use anything like USB; they took the signals of the internal floppy ribbon cable, and surfaced them on a port on the rear of the card.) So, you could have a machine with a hard disk but without a floppy most of the time, and you'd plug in an external drive when you needed it, and you could keep it locked in a cupboard out of the hands of the plebian users the rest of the time. I doubt many people did this – diskless workstations were a lot less work – but probably somebody did at some point.
Which is very convenient (and necessary for backwards compatibility) because programs could be (and were) written to assume they're in B: (and the DOS disk is in A:), or that they're in A: and the user's data is in B:, or a two-disk program is in both. All of that just worked (tm) even with a single drive, as long as the program didn't try to interleave accesses to both drives (I think it would still work because DOS was single-process, single-thread, it would just annoy the user!)
I thought MS-DOS had special handling for A: and B: since it allowed you to copy from A: to B: even with just one floppy drive.
It does.
https://github.com/microsoft/MS-DOS/blob/2d04cacc5322951f187...
Yes, it did, or something did (maybe the individual commands rather than it being through the OS). I'd completely forgotten about that…
Both of these things are true: commands like diskcopy have special handling to allow the same drive to be used as both source and destination, and the DOS I/O has special handling to request disk changes from A: to B: on single-drive systems for programs that lack specific support.
Even further back into time, before 3.5" disks, both A: and B: were 5.25" disks. And, although my memory is hazy, 3.5's were commonly slotted into B: at first, because no one had 3.5" boot disks until the drives became somewhat common.
You are forgetting 8” floppies. IBM used only the soft sector ones (one hole punched close to the spindle to mark sector 0) but there were also hard sector ones (a ring of 32 holes close to the spindle hole).
There was a lot of experimentation back in those days. The Wikipedia’s page on floppy disks is surprisingly long!
But those weren't on "IBM PC"-compatibles, just older micros? The 5150 had 5¼" drives. That is: (Q|MS|PC)-DOS never supported 8" floppies, right?
8" floppy drives were not compatible mechanically with IBM PC cases, but there were 8" floppy drives with their own enclosures and power supplies, which could be put on a desktop along the PC case.
IBM PCs have never supported 8" floppy drives, but it was easy to make an adapter between IBM PC floppy cables and 8" floppy drives, so there have existed IBM PC clones from countries where 5¼" floppies were scarce (e.g. Eastern Europe), which supported the attachment of 8" floppy drives.
The original 5¼" floppies had only the size advantage, but they had both a lower capacity and a lower speed than 8" floppies, so attaching 8" floppy drives would have been a higher performance option in the beginning.
Only after the IBM PC/AT introduced the high-density 1.2 Mbyte floppy disks, the 5¼" format matched (actually very slightly exceeded) the performance of the old 8" floppies.
MS-DOS 1.25 supported 8" disks. I think 2.0 still did.
Trust me, I will never forget 8" floppies.
But in the context of drive lettering, although CP/M supported 8" floppies, MS-DOS never did. MS-DOS adopted the naming scheme, but not the boot style.
MS-DOS 1.25 supported 8" disks. I think 2.0 still did.
Floppies, superfloppies, removable hard disks (platter-only, as opposed to modern external hard drives that incorporate the entire drive mechanism), magneto-optical disks, rewritable CD/DVD, even today magnetic tape is still used as removable storage in applications where large capacity is required and shelf life is more important than fast random access.
The floppies got A and B because hard drives were expensive and for a time a lot of people got by without them, though everybody had at least one floppy.
Even if you only had one floppy, it was both A and B, so you could say "copy a:DOC.TXT b:" and it would read the doc and then ask you to insert the floppy you're considering B (and depending on file size and available memory, sometimes switch back and forth a few more times)
I have vague memories of something like this from my childhood, where my family's IBM 486SX had the 3.5" drive on A and the 5.25" drive on B but my grandparents' Epson 286 had the opposite. Also learning about disk densities when none of the disks from the 486 worked on the 286.
Yeah it's a relic from when computers booted off floppy and hard disks were rare and expensive
MS-DOS didn't have built in support for 3.5" drives until version 3.2.
> It’s called ud2 because the 0F FF variant was retroactively named ud0, and the 0F B9 variant was retroactively named ud1, leaving ud2 as the recommended undefined opcode.
It was a surprise for me as a reader. When I came to this sentence I assumed that 0f ff would become #1 and 0fb9 -- #2. But no, Intel counts from zero, so there is a third ud.