[BillPg] has been designing a fantasy 1980s-era home computer. As part of the exercise, he’s reevaluating all the assumptions that have grown organically over time in the small computer landscape. Hindsight is, so they say, 20/20, but sometimes hindsight can also be colored by modern thinking. Sometimes an idea that seems stupid today made sense in the context of its time. In particular, [Bill] has thoughts on the much-maligned 8086 memory segments.
If you haven’t run into it before, the 8086/8088 had a problem. It wanted to be more or less conceptually software compatible with the 8080 and Z80 computers, which had 16-bit addresses, leading to a limit of 64K of memory. When Intel was designing the next generation of chips, it knew that 64K had to go, but telling developers that code would require huge reengineering was a non-starter. So the idea was to provide multiple 64K spaces broken up into segments.
As with most things, there is theory, and there is practice. In theory, a 16-bit segment provided four extra address bits to add to the existing 16-bit address, producing a 32-bit address, even though the CPU only had 20 bits of address bus. Code that fit in 64K could pretend like that was the whole world, and a tricked-out system could have 16 worlds. Future systems could, in theory, have had more.
In practice, Intel made the segment the top 16 bits of a 32-bit address and then added it to the ordinary 16-bit address. So address 0000:0010 (segment=0, address=10 hex) is the same memory location as 0001:0000. Address 0010:0010 is the same as address 0000:0110 and 0001:0100. This wasn’t really the intent, just a byproduct of how the chip worked.
Eventually, the segments would become indices into a table (like the title graphic), but by then, bad practices wiped out a good idea. It is doubtful that the original designers thought anyone would take advantage of the overlapping address, but, of course, they did.
By the time the 80286 and beyond produced segments that were really keys which defined a block of memory, everyone was already in the mode of using the segment and offset as a large pointer. C compilers even had “modes” that let you treat the segment as just more address bits. Because of that, even on newer processors, people had a tendency to build a “flat” segment and use it. That is, make a segment that starts at 0, ends at the end of memory, and then forget about segments.
In fact, many people independently discovered that you could define a flat segment in protected mode, return to real mode, and then enjoy a flat address space. This was later christened unreal mode, and a topic we’ve covered a few times before.
We agree with [Bill]. Segments were a good idea at the time and might have been more important if people had used them the “right” way. Of course, there would have been ups and downs. Proper segments might have allowed for easy virtual memory, for example. But at the price of possibly swapping in and out huge segments instead of relatively small pages. Today, most of what segments were supposed to do is part of the memory management unit and is mostly hidden from the application developer. Still, interesting to reflect on why Intel made that choice and how we got to where we are today.
The Intel segmented memory architecture was an abomination. The only good thing that came out of it was the understanding that it was so bad and the ‘386 needed to include a proper memory management system rather than the kludge that was the ‘286.
I think that people’s memories have been softened with the passage of time.
This wired architecture belongs in the same bin as non-binary word sizes and octal.
I agree…
Said in hindsight, but I always liked the flat memory model of the Motorola CPUs more (6×00 series, 68k series). Just think how diffirent compute could have been if we never had the Intel segmentation model (which was pointless, as the 8080 died almost immediately after the 8086 was released, and Z80 code never made it to the IBM PC).
I do like backward compatibility, but again in hindsight…that early compatibility should have been dropped, as it was never used.
CP/M and the CALL 5 interface had some significance during the development of early DOS software, though.
It allowed porting of CP/M-80 software without much changes.
Probably even automatically through conversion tools.
Of course, the 68k (and ARM!) chips had their own pointer related problems, as when RAM sizes could still comfortably fit in a 24-bit address space some OSes used the higher bits of pointers for housekeeping purposes. Which made a bunch of stuff simply not work once those bits were needed for actually addressing RAM.
Of course, while inconvenient this was still a lot less hassle to deal with than segments…
It wasn’t really, unless you wanted to use contiguous memory > 64KiB, which wasn’t easily possible. Remember the 8086 was a 16-bit CPU, so it didn’t have registers that could contain an address > 64KiB. So adding a segment prefix made all the sense – it was like a built-in mapper the 8-bitters of the time used. When the IBM PC first came out, it only had 16-64KiB of memory anyway, no-one expected memory would become so cheap in the years to come.
+1
Also, the IBM PC model 5150 was half home computer, still.
The BASIC in ROM and the datasette port are an remembrance of that.
At the time of release, the IBM PC was an electronic typewriter released in a market of home computers such as VIC-20 or ZX81.
Its biggest rival was the Apple II, maybe, whose edge connectors slots were an inspiration to the PC bus slots.
At that time, 64 or 128 KB of RAM was considered huge, still.
And the 8086/8088 perhaps seemed future-proof enough thanks to its 1024 KB address bus range.
I mean, during design phase, it wasn’t even clear if the IBM PC would be a success.
The older PC 5100, 5110 and 5120 weren’t much of an success, at least.
For comparison, in 1981, the modern Commodore SuperPET had 96 KB of RAM.
https://www.computinghistory.org.uk/det/54596/Commodore-SuperPet-(SP9000)/
“The later E-stepping level of the 80286 was a very clean CPU,
free of the several significant errata that caused problems for programmers
and operating system writers in the earlier B-step and C-step CPUs (common in the AT and AT clones)”
https://www.computinghistory.org.uk/userdata/products/6187.htm
“In theory, a 16-bit segment provided four extra address bits”
huh???
Segment registers were 16 bits. The physical address was calculated as follows: shift the segment register to left by 4, and add the 16 bit offset. This will give a 20 bit physical address, thus “four extra address bits”. This was real pain when comparing C pointers, for example.
You know, I wrote 20 bit and then second guessed myself to 32 thinking of 80386 PM so you can see I bounced back and forth a little ;-)
The 80286 still had 16-bit address size limits on the segments, so it couldn’t make a flat address space larger than 64KB.
It wasn’t until the 80386 that it was possible to go to a flat 32-bit address space. The 80386 could also switch back to real mode without a hardware reset.
The CMOS memory was required to handle the kludge where the 80286 needed a hardware reset to switch from protected mode back to real mode. It would save the registers and a magic number in the CMOS memory, then hardware-reset the CPU, and the BIOS would restore the registers. Ugly, but a functional workaround.
The 80286 was half-baked at best, the I/O virtualization was not very good at all.
The 80386 fixed all of these problems, and only the silly and counter-intuitive little-endian addressing remained (and still remains, to this day). Little-endian addressing means that an 8-bit, 16-bit and 32-bit value can be accessed from the same address, because the bytes are stored backwards. So if you write a 32-bit value of 0x12345678 into memory, and read back the bytes on an 80386 or higher CPU, they are stored as: 0x78, 0x56, 0x34, 0x12.
The 80286 supported memory-protection based segmentation.
Segments could be defined as active code, data etc.
Thanks to “flat-mode”, “modern” 32-Bit software got code overruns and other nice things.
It took ages until x86 got extensions such as PAE+NX-Bit (no execute) and DEP (on Windows) to feature something equivalent what the 80286 Protected-Mode just had offered naturally. ;)
Btw, the 80286 had LOADALL instruction, which later versions of Himem.sys used on 80286 PCs.
It allowed accessing all memory without switching between real mode and protected mode.
So there was no need to reset the 80286, it could remain in real mode on plain DOS.
The 80386 had some sort of LOADALL, too, though it was incompatible to the real 80286 version.
For compatibility, later 386 PC/AT BIOSes then did emulate the 80286 LOADALL through an exception handler or something along these lines.
That being said, the 80386 was a fine piece of silicon, too.
The 80286 and 80386 are siblings, after all. :)
On Windows/386, the bigber 80386 provided multiple V86 VMs and its MMU emulated LIM 4 EMS.
That way, EMS-ware Windows 2.x applications could overcome memory limitations.
(Windows 2.x itself was limited to a single DOS VM with conventional memory and EMS).
On Windows 3.0, it was then possible for sophisticated Windows applications such as databases to take advantage of 32-Bit and linear memory of the 80386, if needed.
A third-party software, the WIN386 extender by Watcom was most notable here, but there were others, too.
Windows 3.1 then first added the obscure WinMem32 API and then had the well known Win32s extension provided, later on.
The previous 386 Windows extenders could still be used, but remained at Win 3.0 API level, maybe.
In retrospect, all this trickery made it possible to have Windows running on different CPU generations.
I think that’s quiet interesting. Other platforms such as Macintosh used hybrid binaries, rather.
Yes, physical address was 16*segment register+(pointer/constant/some other addressing mode). As segment was multiplied by 16 and not 2¹⁶, it only have 4 extra bits addressable memory.
An obvious typo or editing error. It adds 16 bits, hence it producing a 32-bit address if just added to it. But in the end it only added 4 bits, because the address bus was only 20 bits wide.
Sorry, what is so hard to understand with real mode addressing. This is no 32-bit addess and never was. It is a physical 20 bit address redundantly encoded in 2x 16 bits. Maybe because it is so brainless and silly, younger generations get confused. Why add 16 bits for of segment registers when all you get out of it are 4 more bits for your physical address. Well, ask Intel (or Steve Morse)!
My guess: this is what happens, when you just need a “quick fix” to counter your advanced competition (Motorola 68000, Zilog Z8000; NS320xx, etc) while waiting for the “big new thing” (iAPX432).
I don’t think you really got the point of 8086-style segmented addressing. x86-16 code used absolute addresses. That isn’t a problem as long as you want to have only one program in memory at a time.
Now comes 8086, you have more memory, bigger address space… room for multiple programs/pieces of code at once in memory like TSRs. Maybe you want to split your software into modules that invoke each other through software interrupts (like network drivers do) etc.
If you’d divided 20bits into 16 segments with 64k each you run into a problem. For each piece of code that runs individually you’d have to waste 64k of memory, even if the TSR only takes up 100 bytes.
The segmentation scheme intel chose allows you/DOS to provide as many segments with a usable start address of 0 as you like, wasting no more than 16bytes of memory for each.
To give an example: If you wanted to handle an interrupt for a sound card or your kezboard, you’d have to install an Interrupt Service Routine by putting the appropriate segment:offset address into the Interrupt Vector Table.
You could happily ask DOS for a few hundred bytes anywhere in memory, put your code with 16byte alignment into it, and compute the segment:offset address so your code runs with offset=0 when invoked as interrupt.
Yup. It wasn’t arranged to let you run a single big program. It was set up to let you run lots of small 64kB+64kB programs independently. This was back in the day when that was quite a lot of space, remember. The 8086 and 80286 segment system is best thought of as a very simple virtualisation system. It did it pretty well: there were Unixes which would run happily on such a system (including C compilers). Minix 2 was such a one, and provided a very comfortable system which could recompile itself on an 8086 with minimal memory.
Later on there were a few attempts to use 286 segmentation as core functionality. I once saw a Smalltalk implementation where every object had its own segment, using the segment limits to give real memory protection to object accesses. I do wonder how effective it was given that the segment tables only supported a few thousand segments (8192, I think?).
Yes it doesn’t scale well. I found it very crude when, back in the times, I first read that you have to put absolute offsets for LDT/GDT entries into the segment registers rather than some sort of index. Why would they? And what happens if you put something odd in there that still makes up a valid entry? :D
If one would have to make the same design choices Intel had to make back then it becomes quite obvious why they’ve made it that way:
If you want to keep old 64k code running you have to keep most of the registers as they are.
Now if you want several programs in memory you would introduce some sort of “offset register” that offsets all the absolute addresses. All registers are 16bit, so you make it 16bit as well.
Finally you want to address 20bits. The offset register obviously doesn’t need to be byte-based, so you shift it left by 4.
Voila you got 8086 segment registers. Now you just have to provide several of them for code/data/stack/etc.
No. You put what is called a selector, wich is the index plus some entra control bits (such as privilege level, enabling using call/jmp to switch protection level) and if its a gdt or an ldt descriptor. Even the descriptor offsets are “linear offsets” to be resolved by the paging mechanism. Thats how commonly different applications had the same loading offset.
I had to look it up. You can either load the offset +/OR some bits for the access level OR the index << 3 +/OR some bits for the access level. It’s the same value.
Probably some literature back then wrote “offset plus …” instead of “index << 3” so that’s where the confusion comes from.
Of course the latter explanation makes much more sense.
IIRC x86-64 still allows the hardware to distinguish between objects and integers based on the LSB, it’s controlled by a bit in one of the CRs. I’ve always wondered who Intel added that for… I’d suspect somebody using big servers with lots of VM for a legacy OS.
The 80286 had introduced a variable segment size.
Range was from 1 Byte to 64 Kilobyte, I think.
However, the default size of 64 KB was rarely changed.
16-Bit Windows kept it in 16-Bit Protected-Mode,
probably for binary compatibility with Windows applications that also run on Windows in Real Mode on 808x PCs.
Merely 386 Enhanced Mode of Windows 3.x operated on 4 KB segments, if I remember correctly.
This was transparent to 16-Bit Windows applications, I think.
haha i read the replies here because i challenged hackaday readers to tell me something good about the awful design and you did not disappoint! thanks!
it really is slick how a .COM is just loaded into memory and then branch to offset 100h. By far the simplest dynamic loader i’ve ever used. IIRC the “TSR” call you would just pass how much of that 64kB you’re still using, and it would save that much for you.
There was a special DOS function call (int 21h, ah=31h, dx=number of 16-byte paragraphs to keep resident) for TSRs. So one would put the actual stay-resident part at the beginning of the .com or .sys and the loader at the end, and then the final thing the loader would do is jettison the loader part
One thing I do sort of miss is direct hardware access via memory mapped I/O it made interfacing with small robotics projects, readying the system clock or just beeping the intermal speaker pretty easy.
Which has exactly nothing to do with the segmentation. Memory-mapped I/O wasn’t really a thing on the 8086 since it had a seperate I/O space accessed via IN and OUT instructions, inherited from the 8080. There’s not that much difference between memory-mapped and register-mapped I/O for a programmer. (And note that the video memory was memory-mapped, as it lived on an add-on card, not on the mother board.)
Everyone else was using memory-mapped I/O instead and so it also appeared on the 8086. The whole address space beyond 640k (0xA0000 to 0xFFFFF) was reserved for it, all text/graphic adapters used it and lots of NICs too.
Looking back I/O ports were some sort of legacy low performance 8bit I/O system, like SMBUS is today.
You do still have memory mapped I/O. In fact, that’s typically all you have, instead of the disaster that was the separate I/O space.
Expansion busses are dynamically mapped now, so you need to look up the address, but “sleazy solutions” still just open /dev/mem and seek to an address, because safety is for suckers. Obviously USB stuff is different, but that’s because those are just complicated serial ports with their own behaviors.
It’s actually reasonably safe with IOMMUs and VFIO now, so with that, user space PCI (and its inherited families) is pretty simple.
But the number of times I see some custom hardware thingy just do mmap in user space and act like it’s 1980 is fairly common.
It’s the OS stopping you doing that.
It was still a thing on Windows 95 and Pentium CPUs.
Same CPU and Windows NT, sorry, no hardware access from unprivileged code.
i think if you are using mmio directly from ‘userspace’ then that is the sine qua non of ’embedded programming’ in my mind. so like today i’m lumping the real-mode DOS experience in with stm32 / esp32 / rp2040 / pic / atmel sort of environments :)
I don’t think it was a matter of porting code more easily. It was just to allow 16 bit registers to address more than 64K. Intel wasn’t ready to make the jump to 32 bit registers like the 68000 did.
Indeed, I don’t think there was much too it besides that. It was like an 8-bit CPU with built-in mapper.
Intel sure sold it as about making porting code easier.
And it was in fact a really bad idea, then and now.
There’s nothing inherently wrong with segmentation, particularly if segment size limits are removed as they were on some later x86 CPUs.
The real problem is that on the ‘286 and its successors the LDT and GDT were still limited to 64K bytes each, hence there was an absolute limit of 16K descriptors per process. If you used the descriptors in the way that they were initially documented by Intel which implied- for example- that every code segment needed two descriptors so that it could both be executed and written to during debugging, or if you have every IP message a descriptor and used that to move it around between rings, then 16K segments was woefully inadequate.
Once Protected Mode was available in 386, the segmentation was left unused as all major OSes relied on linear addressing. Also, linear addresses are used in other architectures (m68k, MIPS, SPARC, PowerPC, ARM) and OS developers decided to have single unified code that relied on this feature.
If someone remembers an x86 protected-mode OS that actually used segmented addresses for something, please chime in.
16-Bit OS/2 made good use of 80286 features.
Especially OS/2 v1.3 was quite mature at its time of release was and used in networking environments/software development.
It used rings 0, 2 and 3 of x86 ring scheme, I think. And virtual memory (1GB address range, but split into 2x 512 MB).
The “Family API” supported by OS/2 made it possible to compile binaries that work on both 8088 based IBM PCs running DOS 3.x and 80286/80386-based ATs running OS/2.
When run on DOS, a minimal OS/2 runtime would run the application code within the limits of an Real-Mode environment.
When run on OS/2, the native OS/2 application code could make use of preemptive multitasking and virtual memory was available, too.
And unlike a DOS application, a Family API appliction didn’t have to run in the restricted DOS box.
The Family API was still used in the 90s by “DOS” compilers such as MS C 6 or PDS 7.x.
OSes such as OS/2 or NT would run them natively instead via an virtual DOS machine.
More information:
http://www.malsmith.net/blog/visual-c-os2-family/
http://www.os2voice.org/vnl/past_issues/vnl0801h/vnewsf4.htm
The memory segmentation isn’t the best technical solution, but we need to be aware that our analysis is retrospective, e.g. 50 years after the event having all the experience afterwards. Maybe back in time this design choice seemed justifiable to Intel for the moment and later they did a ton of horrible things to the x86 ISA based on the “compatibility” excuse. I wont even mention the ISA wars between Intel and 3rd-party manufacturers.
Btw, if I remember correctly Patterson & Hennessy wrote in their book (“A Quantitative Approach to Computer Architectures”) that sooner or later due to technology advancement a computer architecture will reach a point where the (originally designed) address space will turn out to be insufficient. ISA designers will try to fix it with weird extensions (segmented memory, PAE, you-name-it), but this only delays the inevitable death.
Hehe, does anyone still remember F-CPU by Yann Guidon & friends?
i remember clearly the moment in 1995 that i put this all behind me. My DOS programming skills were growing, and i had just finished my first program that used EMS paging. And at the same time i had had linux on a different machine long enough that i was starting to get used to it. And after all that i went through just to access a couple MB, it suddenly occurred to me that on the Linux machine i could just malloc(210241024) and there would be no far pointer, no explicit paging, just memory. And i just swore to myself on that day, i would never write another DOS program again. Good riddance!
The crazy thing is both professionally and personally i keep running into software that really bends over backwards to fit into a 16-bit system, often with a bunch of “#ifdef LOW_MEM” surrounding egregious hacks and compromises. I am still struggling to fully implement that vow, 30 years later.
haha formatting. malloc(2 times 1024 times 1024)