Thread by @Foone, So when you format a floppy disk you can pick between full [...]

So when you format a floppy disk you can pick between full and quick format, and I think I only really understood the difference and what "full format" does relatively recently.
SO A THREAD

At a high level:
Quick format doesn't format the disk, it just erases the file metadata by setting up a new FAT12 filesystem.
Full format actually formats the disk and erases all the content, and also sets up a FAT12 filesystem.

And the confusion comes down to how "formatting" really means three things:
Low Level Formatting
High Level Formatting
Creating the Filesystem

Basically, for quick formatting, you only do #3.
But what are the other two? And confusingly, "formatting" is different for hard drives, and different for both kinds of hard drives.

So let's start by explaining how a memory card stores data.
An old memory card, one that isn't Smart... even if it's called a "Smart Media Card".

This uses flash memory, which uses a bunch of little transistors that can store a tiny charge. They're grouped together into "pages" which you read/write at once, and those are grouped into sectors, which is how the PC reads and writes them.

But these sectors are physical things: it's a grouping of transistors.
It starts at sector 0 and goes to how ever many sectors the device has. In the case of this 32mb one and standard 512-byte sectors, that's 65536 sectors.

So to format it, you could go through and write the same value to all those sectors, making it blank.
Additionally, you set up the filesystem, where there's FAT12 headers at the start, telling you where files are/will be.

And you're done. That's formatting.

Part of the problem with understanding this is that basically all data storage devices pretend they work like this in the modern day, in order to make things simpler.
Sectors 0 to MAX, you can write them, read them, and that's about it. They're there before you format it.

So modern flash memory (SD cards, USB sticks, and SSDs) make this more complicated, while still pretending it works this way.
With flash memory, each cell has a limited lifespan measured in a number of writes, so to avoid wearing a cell out, they do wear leveling.

Basically there's a tiny computer inside the memory chip that translates between the sector number you request and the internal sector number, and external-sector-to-internal-sectors mapping can change, to evenly use the entirety of the memory card.

the idea is that this is hidden from you and the memory card acts just like it doesn't do this, except the memory lasts a lot longer.

And so onto hard drives:
Spinning-rust hard drives don't work like this at all, but since Logical Block Addressing they pretend to.
They have some number of sectors, and your PC sees 0-MAX sectors, with no info about where they are.

But inside, your hard drive is a stacked pile of 1 or more circles of magnetic material, with an arm that can move the drive heads in and out, and it spins.

And the way this works is explicitly named in the old CHS scheme: "Cylinder/Head/Sector"
Cylinder selects which circular track to use (further in or farther out), head selects which read head to use (top/bottom of each disc), and sector (which little segment of that track)

But because of LBA, the PC just sees this as a series of individual sectors, and the hard drive controller can convert those into an actual physical location on disk... OR CAN IT?

So look at this hard drive platter. Do you see sectors? or tracks? NO! they are invisible. And how did they get there, anyway? are they physical structures, like the grove on a record? are they magnetic? or are they entirely logical?

So it turns out the answer is that they're magnetic! and they're on the disc from the factor, but not when the disc platter is created. The platter is the same all across, but there are servo markings on them

Servo markings are put down on the disk at the factory, using a super-accurate system. They mark out where the tracks are and where the sectors are, and the head can use them to navigate around the disk: it finds the right track, then waits for the right sector to rotate around

And importantly, modern hard drives cannot put these servo tracks on the platters themselves.
You used to be able to do it (back when hard drives were in the 40mb range!) but now it's designed so that it's not needed

and making it not be needed saves on cost for the drive, as it only has to be accurate enough to find a sector, not accurate enough to put it down in the right place to begin with.

And so, back to formatting:
The putting down of these servo tracks is called low-level-formatting, so when you "format" a modern hard drive (meaning anyone since like 1991) you only do the high-level-formatting and the making of the filesystem. Low level has already been done

BTW, this also means that if your hard drive gets erased by a magnet, you can't fix it. Those servo tracks can't be rewritten, and they're gone permanently.
(Fortunately that doesn't really happen: the hard drive wouldn't really be affected by most magnets)

OK SO that's how it works on flash and hard drives, let's go back to floppies.
The floppies used with PCs (both 5.25" and 3.5") are what's called soft-sectored. There are no servo tracks.

So the servo tracks in a hard drive are needed for two things:
1. to find the track (moving the head in and out)
2. to identify the sector (as it rotates around)

Hard drives need them for #1 because they use a voice coil. It's positioned based on the current going through it, and it's analog. So different amounts of electricity positions it at different positions, and it uses the servo tracks to figure out where it is and adjust.

floppy drives, on the other hand, use stepper motors.
Stepper motors have a system where you can adjust how the motor turns, in 90-degree steps. So by using it with a worm drive, it can position the head in discrete steps.

So to figure out how far to move to a track, you just need to know where you currently are, and then count how many steps to make and in which direction.

So if you know you're on track 5, track 7 is just 2 steps up.

But you have to figure out where are to start with. This is done in two simple ways, depending on the drive:
1. Go down one track until you trigger a track-zero sensor
2. Go down more tracks than the disk has.

3.5" PC floppy drives have a track zero sensor which detects when the track hits the minimum position. So when it starts up, it just goes down until that sensor activates, then it knows it's on track 0, then it remembers which track it's at from then on.

Apple II disk drives, on the other hand, save money by not having a sensor.
They just know that Apple II disks have 35 tracks, so when they power on they go down 35 tracks, so no matter what track they started on, they're on zero now.

You know that THUDTHUDTHUDTHUDTHUD noise when an Apple II boots up? That's the drive head hitting the zero-position and failing to got back any farther.

OK SO: Floppy drives don't need servo tracks to know where tracks are.
How do they know where the sectors are?
If you just start reading at any point, you'll find some data, but is it the sector you want? the second half of another sector? HOW DO YOU START?

and the answer is, as you might have guessed:
low level formatting!

When you format a floppy, you're not just doing the "write each sector with a blank sector" part, like you'd do with a hard drive or flash storage device, you're also creating the sectors.

so the actual magnetic format on disk is going to be something like:
[sector header for sector 1][sector data][sector footer]
[inter-sector gap]
[sector header for sector 2][sector data][sector footer]
[inter-sector gap]
...

each sector header has info like what track it's supposed to be on, which sector number it is, and the checksum of the sector (which may be in the footer, depending on the format)

this checksum, btw, is how you can tell the difference between a floppy disk that has been magnetically erased and formatted.
A formatted disk will have blank sectors with accurate checksums, an erased disk will have no sectors or corrupted sectors.

But as the floppy is spinning around, this gets written out like it's a linear tape, until it completes a full rotation. It just writes the sectors with metadata and the gaps between the sectors, going all the way around.

BTW, the metadata here isn't things like "what file is stored here?" it's just to let the drive locate the sector itself.

Things like filesystem info is stored as data inside sectors, not inside their metadata.

and the gaps are very interesting: You'd think you wouldn't want gaps, because that's parts of the magnetic surface that you're not using. it's wasted storage space!

But the problem is that disks are read-write devices, and they're not perfect.

imagine you write out a disk with three sectors, with no gaps, back to back, like:
[sector 1][sector 2][sector 3]
Now you need to rewrite sector 2. Uh oh. You have to time this PERFECTLY or you'll overwrite part of sector 1 or sector 3 because of the positioning!

And it's a timing thing because, to paraphrase something 4am said, the disk keeps spinning no matter what the CPU is doing.
If you take too long or not long enough, your head is in the wrong place.

So inter-sector padding gives you some slop so you can safely rewrite a sector without risking stepping on the sectors around it.

You could, in theory, read the entire track into RAM, replace the sector in RAM, then write the whole thing back out... but that's like 16-32 kilobytes of data (for a 1.44mb disk). That's a lot of RAM!
It's also slow. You'd have to rewrite the track every time you write 1 sector

so in practice the only things that ever did track-at-once writing were things like disk duplicators, which were specialized.

SO, to summarize how this works: When you format a floppy disk, you do three things:
1. Decide where the sectors will be (on each track) by writing their headers/footers
2. Write blank sectors, effectively clearing the disk.
3. Create a filesystem inside those sectors

But 1 and 2 are discrete steps, even if they will (in practice) be merged when doing this. Your disk controller doesn't actually write a sector and then re-write it to be blank, it just writes it blank in the first place.

And it's important to realize they're discrete steps because they're not done in all situations, even when we're talking about "formatting".
Format a hard drive, and there's no low-level formatting being done.

Interestingly this is also true of some types of floppies!
Zip Disks, for example, use servo tracks. This is one of the ways they store so much more data than a 1.44mb disk

And it's also true in some optical discs. Optical discs are usually read-only and just have one long spiral track (like a vinyl record) but they use something similar to let the head determine where it is on the disc. But in writable discs, it's different.

Your CD-R stores data using a pregroove (created when the disc is created), which the head of the drive follows as it burns or doesn't burn the dye in small segments along that groove, affecting the reflectivity and allowing it to be read later.

The simplest way to do this is "disc at once" mode: It starts at the beginning and follows the whole groove until the end.
But there's also "track at once", which only does a part of the disk, allowing more data to be added later.

importantly, "tracks" in this case are fully virtual things. The table of contents of the disc says where they start and end, based on which sectors belong to which tracks. Like:
Track 1: sectors 1-2000
Track 2: sectors 2001-6000
Track 3: sectors 6001-9999

But they're not the only optical writeable discs:
there's also Magneto-Optical disks, and they're hard sectored. Instead of needing to low-level format them, it's encoded into the disc itself.

and the cool part? Those sectors are VISIBLE! If you open a magneto-optical disk you can see exactly where each sector starts and ends.

but yeah. This whole thread was basically just to explain the background to show how I went through multiple levels of understanding about what "formatting" a floppy meant over 25 years of dealing with this shit

1. what's formatting?
2. formatting is erasing all the files
3. formatting is writing blank bytes/sectors to the disk
4. formatting is creating the sectors, so the drive can find/read/write them again later

BTW, the fact that formatting a floppy drive is #4 and not #3 actually indicates something about what you should do if you get a bad sector on a disk.

if you think of formatting as just writing zeros(/blank sectors), if you get an error then that must mean the disk itself is bad, physically.
throw it out!

but when you understand that it's creating the sectors, you can see that you might get an error writing a sector because THAT SECTOR ISN'T THERE ANYMORE (thanks, magnets) and the disk itself will be fine, once you reformat it again.

admittedly, NOT the most useful thing to know in 2019, even for me.
But hey, it's good to know.

Anyway, if you liked this Old Tech Rant, feel free to give me a dollar or two on ko-fi: https://ko-fi.com/fooneturing

Buy Foone Turing a Coffee. ko-fi.com/fooneturing

Buy a Coffee for Foone Turing with Ko-fi.com

https://ko-fi.com/fooneturing

or set up a monthly donation on patreon: https://www.patreon.com/foone

Foone Turing is creating Tweets, Weird Hardware Things, and Videos | Patreon

Become a patron of Foone Turing today: Read posts by Foone Turing and get access to exclusive content and experiences on the world’s largest membership platform for artists and creators.

https://www.patreon.com/foone

or get one of my weird art PCBs on etsy: https://www.etsy.com/listing/736823648/copyprocontrol-pcb-v12

CopyProControl PCB v1.2

This is the CopyProControl 1.2! Its a functional PCB I designed to control a CopyPro 2000 floppy disk duplicator, but when I went slightly mad with power and covered it in fun references and such,...

https://www.etsy.com/listing/736823648/copyprocontrol-pcb-v12

and you might like to read my older thread on how CHS turned into LBA in PC hard drives and the weird hack we did to make that transition smother: https://twitter.com/Foone/status/1165455650823950337

https://twitter.com/Foone/status/1165455650823950337

BTW, I was taking special care to say "blank" sector instead of "zeroed" for most of the thread, because the DOS format command doesn't erase it to zeros: it uses 0xF6.
This has no real practical impact but is definitely a thing, so it's the perfect piece of trivia.

Latest Threads Unrolled: