Read on Twitter
OK SO with computers we talk about their bitness a lot.
Like the C64 is an 8bit computer, the SNES is 16bit, the 386 PCs were 32bit, and modern PCs and phones are 64bit, right?
Like the C64 is an 8bit computer, the SNES is 16bit, the 386 PCs were 32bit, and modern PCs and phones are 64bit, right?
and what that means is... complicated. Because often it& #39;s called "bus width" which is rarely accurate.
The idea of it being bus width means the CPU talks to the memory in chunks of that size. 8-bit CPUs talk to ram in 8-bit chunks, right?
The idea of it being bus width means the CPU talks to the memory in chunks of that size. 8-bit CPUs talk to ram in 8-bit chunks, right?
the other meaning is that it& #39;s the "word size" of the processor.
Like, the registers and math done on an 8-bit CPU are 8-bit, and 32bit on a 32-bit CPU.
MAKES SENSE?
Like, the registers and math done on an 8-bit CPU are 8-bit, and 32bit on a 32-bit CPU.
MAKES SENSE?
and you can see how that works on a processor like the 6502.
It& #39;s an 8-bit processor, it talks to memory in 8-bit chunks, and it has three registers: A, X, & Y which are all 8-bit
It& #39;s an 8-bit processor, it talks to memory in 8-bit chunks, and it has three registers: A, X, & Y which are all 8-bit
it& #39;s also got S, the stack pointer, which is 8-bit, and PC, the program counter, which is 16bit
WAIT
WAIT
why& #39;s the 8-bit CPU got a 16-bit register in it?
and the answer is basically "uhh, it& #39;s not that simple".
Even 8-bit CPUs need to do 16-bit math sometimes and address things with 16-bit address, so the program counter is 16-bit
Even 8-bit CPUs need to do 16-bit math sometimes and address things with 16-bit address, so the program counter is 16-bit
but maybe this is a rare case of a weird design from the dawn of Computer Time and it all makes simple sense from then on.
I doubt it, but maybe.
I doubt it, but maybe.
So, Intel 386, launched in 1985. It& #39;s a 32bit CPU. Main registers? 32bit. Index registers? 32bit. Program counter? 32bit.
Definitely 100% a 32bit processor, that.
Definitely 100% a 32bit processor, that.
OK not exactly. For historical reasons it boots up in a 16-bit mode because it wants to be compatible with software and operating systems written for the earlier 8086/8088 CPUs. But ignoring the fact it can pretend to be a 16bit CPU and does so by default, it& #39;s 32bit, right?
well mostly.
Remember about the "bus width" thing? 8-bit CPUs have 8-bit buses, and 16-bit CPUs have 16-bit buses, and 32bit CPUs... usually have 32bit buses.
Remember about the "bus width" thing? 8-bit CPUs have 8-bit buses, and 16-bit CPUs have 16-bit buses, and 32bit CPUs... usually have 32bit buses.
See Intel figured that a lot of computers weren& #39;t going to need all the high-end POWER of the 386, so they made a cut-down version which interfaces with the outside world like it& #39;s a 16-bit CPU.
which greatly simplifies the motherboard& #39;s design, since it means half as many data lines going around and simpler memory.
BUT OKAY.
that& #39;s a weird cut-rate 386 Intel made back in 1988, surely since then it& #39;s gotten much simpler, CPUs are just consistently the width you expect them to be?
that& #39;s a weird cut-rate 386 Intel made back in 1988, surely since then it& #39;s gotten much simpler, CPUs are just consistently the width you expect them to be?
NOPE FUCK YOU!
OK so to explain why the 486, the next processor after the 386, isn& #39;t a 32bit processor like it says on the box, we have to go back to the original IBM PC.
And farther, in fact.
And farther, in fact.
the idea is that this isn& #39;t a full CPU on its own. It just Does Math for an attached 8086/8088 CPU.
It specifically does floating point math, which has to be handled differently from integer math that standard CPUs are so good at.
It specifically does floating point math, which has to be handled differently from integer math that standard CPUs are so good at.
So it& #39;s not really a CPU.
But if it was a CPU, how many bits would we say it is?
Keep in mind it& #39;s from 1980, this thing is older than the c64 or NES, older than the IBM PC, it came out the same year as Pac-man.
But if it was a CPU, how many bits would we say it is?
Keep in mind it& #39;s from 1980, this thing is older than the c64 or NES, older than the IBM PC, it came out the same year as Pac-man.
And why& #39;s that matter for the 486?
Because in the Intel 80486DX variant, Intel brought the Math Coprocessor on-chip for the first time.
Because in the Intel 80486DX variant, Intel brought the Math Coprocessor on-chip for the first time.
so the Intel 80486 is a "32bit processor" which has a 32bit bus, does most math in 32bit, talks to memory in 32-bit chunks, and has nine 32 bit registers...
and then EIGHT 80-bit registers.
and then EIGHT 80-bit registers.
but okay, that& #39;s the end of it. sure 32bit processors have these weird 80-bit secondary registers but none of that changed until we moved to 64bit, right?
RIGHT?
RIGHT?
OK so the next chip after the 486 was the Pentium.
And the original pentium was 32bit and did 32bit in all the same ways, but of course had those 80bit registers in it.
And the original pentium was 32bit and did 32bit in all the same ways, but of course had those 80bit registers in it.
but in 1997 Intel released the first Pentium with MMX CPUs.
These are 32bit CPUs which have an added instruction set called MMX, which can do special fast math on... 64bit registers!
These are 32bit CPUs which have an added instruction set called MMX, which can do special fast math on... 64bit registers!
so it& #39;s a fully 32bit CPU which accesses memory in 32bit chunks and does most math in 32bit EXCEPT it can also just do 64bit math if it wants do. as a treat.
fun fact: those 64bit registers?
They& #39;re not actually new.
It& #39;s actually just using part of the 80-bit floating point registers.
They& #39;re not actually new.
It& #39;s actually just using part of the 80-bit floating point registers.
Anyway a few years later, AMD develops x86-64, Intel adopts it as well, and now our processors are 64-bit! Great.
They& #39;re fully 64-bit (well except for those 80-bit registers which are STILL AROUND). They can talk to the memory in 64bit, do math in 64bit, they& #39;re 64bit!
They& #39;re fully 64-bit (well except for those 80-bit registers which are STILL AROUND). They can talk to the memory in 64bit, do math in 64bit, they& #39;re 64bit!
and they still are.
Your desktop? 64bit.
there definitely hasn& #39;t been any expansions to the CPU between 2000 and today.
it& #39;s definitely still 64bit. Check your windows settings or your /proc/cpuinfo, it& #39;ll say "64bit".
Your desktop? 64bit.
there definitely hasn& #39;t been any expansions to the CPU between 2000 and today.
it& #39;s definitely still 64bit. Check your windows settings or your /proc/cpuinfo, it& #39;ll say "64bit".
OK SO in 1999 Intel releases the Pentium III.
This adds a replacement for MMX called SSE, which tries to solve the major weakness of MMX, which is that it reuses those 80-bit floating point registers as MMX registers.
This adds a replacement for MMX called SSE, which tries to solve the major weakness of MMX, which is that it reuses those 80-bit floating point registers as MMX registers.
How many bits is the Pentium 3?
Well, it& #39;s... 32bit. It& #39;s definitely 32bit. It came out in 1999, AMD didn& #39;t publish x86-64 until 2000, Intel couldn& #39;t have launched a 64-bit processor in & #39;99.
Well, it& #39;s... 32bit. It& #39;s definitely 32bit. It came out in 1999, AMD didn& #39;t publish x86-64 until 2000, Intel couldn& #39;t have launched a 64-bit processor in & #39;99.
But it& #39;s 32bit, but it has those 80bit floating point registers, and now it& #39;s got the XMM registers to use with SSE.
Which are 128bits.
Which are 128bits.
SO NOW our "32bit" processor has 128bit registers in it. GREAT! it can do 128-bit math. THAT& #39;S FUN.
ok so by 2008, we& #39;re on 64bit processors. and there& #39;s still these 128-bit registers in there for SSE, but at least we& #39;re getting close. We& #39;re only off by a factor of two.
And that& #39;s why and when Intel announces Advanced Vector Extensions.
And that& #39;s why and when Intel announces Advanced Vector Extensions.
AVX adds some more registers and some more instructions to work on them and oh by the way, the registers are now 256-bits wide.
so now your "64bit" CPU has 256-bit registers in it. OKAY WHATEVER.
So, in 2013, Intel announces AVX-512.
Guess what: new registers now, and they& #39;re 512-bits wide.
Guess what: new registers now, and they& #39;re 512-bits wide.
So the 64bit processor has 512-bit registers. OK.
But they& #39;re still talking to memory in 64bit chunks, right?
ehh....
But they& #39;re still talking to memory in 64bit chunks, right?
ehh....
So the original DDR SDRAM was 64-bits wide.
Fun fact: This came in 1998, when the PCs you were going to be using this on were 32-bits. Don& #39;t think about that too hard.
Fun fact: This came in 1998, when the PCs you were going to be using this on were 32-bits. Don& #39;t think about that too hard.
and it turns out the rest of DDR versions have remained at 64-bit. they transfer in chunks of 64 bits at a time.
And there& #39;s a variant of DDR SDRAM that& #39;s used in graphics cards: Graphics DDR SDRAM.
GPUs use memory differently and need it quickly and are very parallel and more predictable, so it makes sense they have different RAM
GPUs use memory differently and need it quickly and are very parallel and more predictable, so it makes sense they have different RAM
there& #39;s GDDR1 (actually DDR SGRAM) through GDDR6X but they retain the 64bit nature of DDR SDRAM.
But that& #39;s not the only way to get to memory on a graphics card, OH NO
But that& #39;s not the only way to get to memory on a graphics card, OH NO
There& #39;s also High Bandwidth Memory.
HBM is an interface which uses 3D-stacked SDRAM, with side bit widths and multiple channels added together.
HBM is an interface which uses 3D-stacked SDRAM, with side bit widths and multiple channels added together.
so HBM natively uses 2-channel 128-bit dies, but it stacks them, that& #39;s the whole point.
So you can have an HBM stack of four dies has 8 channels in total, or 8*128: 1024 bits wide.
So you can have an HBM stack of four dies has 8 channels in total, or 8*128: 1024 bits wide.
BUT WHY STOP THERE
So the Nvidia Quadro GV100, launched in March 2018, has a Volta-based CPU which uses 4 HBM2 memory stacks. Each stack has 4 dies, and there& #39;s 2 channels per die, and 128bits per channel.
That& #39;s 4096 bits. that& #39;s 4 KILOBITS.
That& #39;s 4096 bits. that& #39;s 4 KILOBITS.
if that& #39;s not enough for you, they& #39;ve announced (and might even be selling) the NVidia A100, an Ampere-based GPU Accelerator, which has 40 gigabytes of HBM2 memory onboard, accessed with a 5120-bit bus.
it also costs 25,000$, which... ok fair
which makes this diagram in their PDF amusing
"hey, look, if you spend A HUNDRED THOUSAND DOLLARS on four A100s, you need to connect them properly"
"hey, look, if you spend A HUNDRED THOUSAND DOLLARS on four A100s, you need to connect them properly"
if I& #39;m spending that much on some graphics cards I expect Nvidia is gonna send a tech over to install them for me
anyway the whole point of this thread was that I looked up the info on the Nvidia Volta architectures (used on the Titan V and Quadro GV100 cards) and saw wikipedia just casually drop "bus width: 4096 bit" and had to take a moment to rant my way up to how mind boggling that is
