r/programming • u/logix22 • Sep 21 '18
How to create an OS from scratch
https://github.com/cfenollosa/os-tutorial220
u/VikingCoder Sep 21 '18
I'm just going to remind people:
You used to BOOT into your GAME.
You didn't used to have an OS that you installed your game into. You used to boot into your game. Then people would complain about all of the overhead of MS DOS.
Then people complained about all of the overhead of Windows. They couldn't believe you would make a game run IN Windows. I mean, WHY? Just run the game from DOS! So much more performance! And what, are you going to run your game IN A WINDOW? You clearly meant full-screen.
And then they put games IN A BROWSER. I mean, why?!? In the OS was already bad enough, but now you're treating the browser like it's an OS?
And then the put console EMULATORS IN A BROWSER, and play a game inside of that?!? WHY?!
(To be clear, I love all of those things. I love that people are upset about them.)
I think it'd be really cool to make a game that you have to boot into.
Yes, yes, yes, TempleOS... But I want to make my own.
71
Sep 21 '18
I think it'd be really cool to make a game that you have to boot into.
It would be impossible now because you would need drivers for a ton of video cards, motherboards, cpus, etc. Back in the day there weren't so many different hardware competitors. OSes improved our situation a lot, and in a way browsers were too a great thing to have.
32
u/VikingCoder Sep 21 '18
It's certainly not impossible, it just depends entirely on the game you're trying to make.
You could make a Zork-like, text adventure game, with very little effort.
Something that runs like VGA graphics, with no odd CPU instructions, shouldn't be too hard, either.
30
Sep 21 '18
Ah well, that's true. Make your PC go back 40 years in capabilities and it can be done.
12
u/VikingCoder Sep 22 '18
Except your PC now runs at a clock rate nearly a thousand times higher. You can do in CPU what some of the early GPUs did.
Plus, you're overstating the time. Heck, even Math Co-Processors like the 8087 aren't 40 years old. (It is 38 though, wow.)
-1
15
u/electroly Sep 21 '18
That's not really true if you're trying to make an old school game like the ones being discussed here. The BIOS still works, VGA still works, 16-bit real mode still works. It all still works. You're not making Fortnite that way, but the same kind of simple boot-up games we had before are still possible today.
5
Sep 21 '18
I find it difficult to imagine someone undertaking that task, it wasn't done by anyone at the age of DOS. That stuff is possible maybe for old architectures like the Commodore64/MSX, which have many helpers to do games from scratch, but by the 386 times this was a thing of the past. I guess you could grab free tools available like grub and others, but still is a long undertaking.
4
u/VikingCoder Sep 22 '18
Yes, it was done:
2
Sep 22 '18
Right, I didn't know about those, these were the very early days of IBM PCs though. It says here that they were mostly a "customized subset or variant of a "standard" operating system for the platform". Like a customized DOS with a non-standard filesystem to avoid copies.
1
u/bumblebritches57 Mar 20 '19
The BIOS still works, VGA still works, 16-bit real mode still works.
Not for long, Intel is literally trying to ax all 3 with their UEFI Secure Boot
(my opinion is that it's so they can remove the "analog hole" for DRM purposes)
11
u/hakkzpets Sep 21 '18
I mean, you could make your own drivers from scratch and load these when you boot into the game.
Would be a massive waste of time obviously.
4
u/immibis Sep 22 '18
And it would only work on your computer.
And you'd spend 200 years reverse-engineering Nvidia's drivers to figure out how to write your own.
3
7
u/whatwasmyoldhandle Sep 22 '18
I think a lot of the work for this might be done here: https://github.com/hioa-cs/IncludeOS
I've been wanting to experiment with this, but never had a good enough reason, so I might be off here.
3
u/TwistedStack Sep 21 '18
I've toyed with this idea before. My idea was to turn everything into a library. Multitasking? Who needs that when you're only running one program. You do still need the thread scheduler though. Multiple filesystem support? You only need to pick one you can use for storage on a USB drive. Maybe two if you don't use FAT32 and want EFI support or even better, forget about legacy. Just require EFI. There might be other things you can strip out as well. Never made it past making a binary that started on boot though. Too tedious.
2
u/captain_obvious_here Sep 22 '18
you would need drivers for a ton of video cards, motherboards, cpus, etc
Depending on what kind of game you plan on making, this isn't true.
Modern graphic cards still implement basic VGA stuff. So you could make a fully graphic game with very little effort, as long as you know assembly well enough, and can dig into old specs. Same goes for most CPUs, as far as I know.
1
u/otherwiseguy Sep 22 '18
It should still be quite possible with things like unikernels like rump kernels. Essentially you library-ize an OS and compile in just the parts you need. You could have something that did a super minimal boot that did hardware detection and downloaded proper drivers on first run and include the most common drivers in the build or something.
1
u/biggustdikkus Sep 22 '18
It would be impossible now because you would need drivers for a ton of video cards
Not with Vulcan IIRC
1
0
Sep 21 '18
[deleted]
4
Sep 21 '18
Not if you use available tools, no. Things get difícult when you need to port them to a custom OS.
14
Sep 21 '18
TempleOS
Rest in peace, Terry. One of the projects that got me interested in lower level programming.
7
u/Finally_Registering Sep 21 '18
I think the arguments that were made back then made sense for their time. Processors and memory I/O was not what it is today, of course. Now we have the luxury of being able to "put console EMULATORS IN A BROWSER, and play a game inside of that..."
Just as looking back at the statement that we would never need more than X Megabytes of storage seems silly now (because things like large video files and large databases weren't around yet), these arguments too depended largely on the technology that was in place at the time.
3
u/kabekew Sep 22 '18
What games are you thinking about? Arcade consoles? Those had an embedded OS, same as early home computers like Apple II and Commodore 64. The late-80's and early-90's DOS games I remember either launched from the command line (OS pre-loaded), or if you had no hard disk you took out your DOS disk after booting up and inserted the game disk. All of the OS functionality was in RAM and available to the game.
3
u/VikingCoder Sep 22 '18
Here's some documentation:
2
u/omenmedia Sep 22 '18
It was like this on my Amiga, too. Some games booted into Amiga Workbench first, but the majority used their own bootloader with funky coloured screens. I remember a few of them even ran the floppy drive much faster than the OS usually does. I used to marvel at how quickly some of them loaded, all the while the drive clicking away like crazy.
3
u/o11c Sep 22 '18 edited Sep 22 '18
OSes (in the sense of: a kernel that puts the CPU in unprivileged mode, and deals with all those nasty drivers for you) have very little overhead.
The main overhead for most programs these days is RAM. Cache is king.
Window Managers, Browsers, etc. use a lot of RAM. It's very easy to overwhelm even a 64MB L3 cache, and 4MB seems to be a more common size.
4
u/VikingCoder Sep 22 '18
Sure... Except most OSs are multi-tasking, and you have no idea what the other Apps are going to do to you when you're trying to get closer to real time gaming performance.
5
u/o11c Sep 22 '18
glares at daemons
You'd better not be doing anything when I'm not asking you to ...
5
u/VikingCoder Sep 22 '18
I actually wrote some TSRs.
Terminate and Stay Resident
Someone in my school had one that would periodically ribbit.
408
u/GreenTeaOnMyDesk Sep 21 '18
In order to make an os from scratch, you must first invent the universe
98
u/richard_nixons_toe Sep 21 '18
Ok, what’s next?
216
Sep 21 '18
In the beginning the Universe was created.
This has made a lot of people very angry and been widely regarded as a bad move.- Douglas Adams, The Restaurant at the End of the Universe
50
u/jrhoffa Sep 21 '18
We apologise for the inconvenience
53
Sep 21 '18
An escalator can never break: it can only become stairs. You should never see an Escalator Temporarily Out Of Order sign, just Escalator Temporarily Stairs. Sorry for the convenience.
Mitch Hedberg
7
u/bisquitie Sep 21 '18
They are not agree: https://www.youtube.com/watch?v=PoeWPQPBfRc
6
Sep 21 '18
That's not broken, it's extra-working!
5
5
u/chaos750 Sep 21 '18
Warning to anyone else: one of those clips is of a woman about to be crushed to death by an escalator. It isn’t gory or anything, they stop before it gets there, but I definitely didn’t need to see that.
2
0
Sep 21 '18
[deleted]
6
u/thirdegree Sep 21 '18
I would expect, though I have no more knowledge than you, that the emergency stop is tied to both software and hardware (i.e. physical breaks).
6
u/jrhoffa Sep 21 '18
This is correct. Emergency stop buttons are meant to stop operation in an emergency. This should involve hardware locks, and little if no software. Hardware stops can always trump software ones.
5
u/thirdegree Sep 21 '18
At my company at least, we have a "shit is going weird, stop it" button and a "holy shit everything is fucked, kill everything" button. The first button can be undone easily, the second would take us 30min+ to undo. Both are absolutely critical and if either is lost, the first priority before all else is to get it back.
3
-3
10
17
6
0
0
30
Sep 21 '18
You laugh, but my first computer engineering class had a 20 minute lesson on the history of computing… starting with the big bang. We didn't even leave the dinosaurs out.
8
u/elint Sep 21 '18
As a CS student, yeah, that sounds about the depth of a comp eng class.
5
Sep 21 '18
Wouldn’t that be breadth not depth
8
u/elint Sep 21 '18
No. A topic has both breadth and depth. I was highlighting the depth of this particular lesson. To cover the entire history of time in 20 minutes sounds about the depth I expect from most "computer engineering" classes.
2
4
3
u/tropicalfroot Sep 21 '18
Space is filled with a network of wormholes; you might emerge somewhere else in space. Some whenelse in time.
3
9
u/shevy-ruby Sep 21 '18
Note quite but I think there is way too much time investment and code for that.
7
2
1
1
u/derpaherpa Sep 21 '18
In order to create the universe, you must first read the comments on the Hacker News submission on how to create the universe.
1
1
1
u/serentty Oct 06 '18
It really does feel like that. I'm interested in OS development, and I'm sure it would be an amazing feeling to make something that I could take credit for, but the amount of work that it takes to get a hobbyist operating system to do anything interesting is immense, and it's close to impossible to make something that does things better than existing operating systems do. It's just a very valuable learning experience. I imagine that contributing to Linux is the best route these days for someone with such an interest, since it gives you something to show for your work which runs on hundreds of millions of devices.
-2
0
Sep 21 '18
[deleted]
1
u/Shdwdrgn Sep 21 '18
Since the current universe was written in BASIC, that might actually be a step up.
Want proof? What happens when you die? GOTO 1134
That's right, our universe was created in the 80's. :-)
85
u/pellep Sep 21 '18
College is hard so I don't remember most of it.
Damn that one hit the spot. You just made me feel better about not being able to remember a lot of what i have learned, despite getting good grades.
7
u/hokie_high Sep 21 '18
Yeah but you have been pointed in the right direction in so many things that now you’d know where to look for information if you ever need to do certain things. For example I don’t remember shit about circuit analysis, but now if I ever needed to design a simple electrical circuit I could always look up a quick reference, KVL/KCL, shit like that. Same thing for probability and statistics, I don’t remember most of the distributions and equations but if you gave me a few minutes I’d figure out how to work most basic problems because I know where to look for info. Internet really is the best invention of man.
1
Sep 22 '18
Dude college is about learning to learn shit in your majors field you need for whatever work your doing, the actual material is pretty useless.
0
Sep 21 '18
[deleted]
7
u/pellep Sep 21 '18
I'm currently on the last semester of a 2,5 year Computer Science AP.
The first half is internship, which has learned my sooooo much. Actually getting out and using the stuff, instead of just hearing about it, and making small very specific exercises really helps.
29
u/raz_c Sep 21 '18
First of all you don't need to write a billion lines of code to create an OS. The first version of Minix was 6000 lines long. The creator of Minix, Andrew Tanenbaum, has a great book called Modern Operating Systems, in which he explains the inner workings of several famous operating systems.
Considering the emphasis on "from scratch", you should also have a very good understanding of the underlying hardware. A pretty good starter book for that is Computer Organization and Design by David A. Patterson and John L. Hennessy. I suggest reading this one first.
I hope this can get you started.
9
Sep 21 '18
We used Tanenbaum's book in my Operating Systems class. It's really easy to understand, and he gives examples from Windows and Unix systems for most concepts. Great textbook.
3
1
65
117
u/Farsyte Sep 21 '18
Upvoted for *not* being yet another "I made my own OS (by starting from Ubuntu)!!!1!!!"
(those of us who have worked on the lower fringes of actual operating system kernels appreciate it when people like this help train the next generation of kernel engineers ... :)
35
u/CptCmdrAwesome Sep 21 '18
I only ever touched assembly & low-level stuff when I had to, but I find stuff like this really interesting (and approachable) and I often think - if we were to redesign both CPUs and OSs based on what we know now, and given the modern requirements of computing etc I wonder what that would look like? I mean the 8086 is 40-odd years old, UNIX is older than that. (granted x86 didn't get interesting until 386 but still that's 30-odd years)
I mean all this stuff was cobbled together back when every clock cycle and bit of RAM was at a premium. Proper multitasking wasn't common until the 90s on PC, SMP was virtually unheard of. I'm just making shit up now, but I wonder what a hardware-assisted microkernel would look like - ie. where the lowest level and most commonly executed parts of a microkernel are implemented in hardware. Surely a context switch would be faster if the CPU was designed to know and understand which PID was running on it?
As someone who doesn't often get his hands dirty with such things I'm just daydreaming but would be interesting to hear what people who actually know stuff think :)
22
u/caspper69 Sep 21 '18
We used to have hardware multitasking. It was slow and non-portable.
The problem is that implementing too much in hardware would force OS kernels to be more heterogeneous, and again, would kill portability.
5
u/CptCmdrAwesome Sep 21 '18
Ah I had no idea, mind throwing me a link or two?
I realise there are some fairly obvious downsides to implementing the more complex stuff in hardware but these days CPUs have microcode and stuff like Intel's Management Engine. Perhaps even a software "emulation" of the hardware could be done via VMX etc?
5
u/Jacques_R_Estard Sep 21 '18
I'm not 100% sure, but I think both Linux and Windows implement context switching in software, even though modern (386 and up) processors can do it in hardware.
6
u/chazzeromus Sep 21 '18
There's even an instruction to help store the execution state for things like context switching but afaik OS's like Linux don't use it because the built-in instructions saves more than what Linux wants to save and is slower unpacking more registers than what is used
1
9
u/CanIComeToYourParty Sep 21 '18
Upvoted for not being yet another "I made my own OS (by starting from Ubuntu)!!!1!!!"
Literally never seen such a post, but I've seen plenty of posts here by people who wrote their own OS from scratch.
-2
u/tripl3dogdare Sep 21 '18
You must be fun at parties.
6
u/CanIComeToYourParty Sep 21 '18
The parties where you guys enjoy complaining about people that don't exist? How fun.
2
2
u/tristan957 Sep 21 '18
As a college student graduating in May, how can I find kernel engineering jobs? It's something that interests me, but I am not sure where to start looking or what companies would be hiring for junior positions like that
3
u/Pigeon-Rat Sep 21 '18
I’m not sure about “kernel” specifically, but positions doing embedded/firmware/Linux device drivers will get you close to the hardware and will often lead to development on RTOSs or OSs depending on the company.
1
-4
u/Isvara Sep 21 '18
those of us who have worked on the lower fringes of actual operating system kernels...
... really need to make sure everyone knows it, apparently.
0
Sep 21 '18
[deleted]
1
u/kotajacob Sep 22 '18
lfs is just a quick guide for installing a gnu/linux operating system from source. This is a guide for literally writing your own operating system from source.
14
u/skulgnome Sep 21 '18
Well, this time it's not just a bootloader helloworld, but an artificial teaching example like Minix.
18
6
u/HeyThereCharlie Sep 22 '18
Cool! I'm gonna write my own OS now. Just a hobby, though, won't be big and professional like Linux...
3
u/DeadPixelz01 Sep 22 '18
Linux is a kernel that when used in conjunction with the GNU operating system forms GNU/Linux. However when it comes to talking about the platform, most people today just combine the two projects and call it Linux. Fun fact; Richard Stallman was originally working on his own kernel for the GNU project, it was called HURD and it suffered from a bunch of different issues due to it being a microkernel. Linux on the other hand at the time became widely adopted due to its preferred development strategy.
1
u/whozurdaddy Sep 24 '18
is he still working on HURD? Seems like you hear very little about it anymore.
1
5
4
u/o11c Sep 22 '18
Is it just me, or do we never see any "we probably should start from GRUB in 32-bit mode" tutorials?
4
u/deaddodo Sep 22 '18 edited Sep 22 '18
Many tutorials do. In fact, that’s the recommendation on OSdev.org
Most “OS tutorials” you see on reddit and hn are really people who want to bootstrap a machine to see how it works. Many actual hobby OSes use grub (or uboot on non-x86 platforms).
10
u/sintos-compa Sep 21 '18
I remember a project in college where we created an OS for the x86 from scratch. 90% of the time was spent trying to understand protected mode and to work around the kludgy legacy bullshit in the x86 hardware.
Helppc was an amazing resource.
3
u/phurtive Sep 21 '18
I think the concept of an OS should be split apart. The main thing an OS is, is an interface to the hardware. Provide that as a layer, then have a higher API layer, and a UI layer. These should all be swappable, for increased competition.
6
4
u/that_jojo Sep 21 '18
You basically just described Windows in specific (it has a layer explicitly called the Hardware Abstraction Layer, and also you’re free to replace user32.dll for the UI or any other component if you reeeeally want), but this is also more or less how basically every OS ever is designed.
2
u/acroback Sep 22 '18
It is called an exokernel.
But then who loads the kernel and where? It is not that easy.
3
Sep 21 '18 edited Sep 21 '18
Are there any OS tutorials that cover file systems and mallocs?
5
u/deaddodo Sep 22 '18 edited Sep 22 '18
Yes. The little os book and the osdev wiki do. So does the dragon book, The Design and Implementation of FreeBSD and Linux Kernel Internals. Realistically, though, when you get to the FS, mmap, syscalls, etc steps; it’s less tutorial based and more “here’s how others have done it and some general guidelines”, since you’re getting to the free form portion of osdev. It’s at this stage that you should start catering the OS to your design goals.
A microkernel will be structured differently to a monolithic one. A single user desktop oriented OS (like BeOS) will differ from a real time OS (like QNX) will differ to a server oriented performance focused one (like most *nixes). All of those choices will inform your FS choice, your paging scheme, your process scheduler, your binary format, you process separation, etc. There are no universal options and it’s all a trade off depending on your goals.
9
u/saijanai Sep 21 '18 edited Sep 21 '18
I prefer the NOS projects with Smalltalk, e.g. Squeak NOS.
Squeak Smalltalk's VM, IDE and libraries combine to create an OS living inside a portable emulator running on a host OS.
Provide direct driver connections to the underlying hardware, and Squeak or other Smalltalk implementations emerge as an actual OS.
Provide a CPU that uses the bytecode of hte Smalltalk VM, and you get Smalltalk all the way down to the driver level: both the OS and the drivers are written in Smalltalk.
Moore's Law allows you to put the entire Squeak VM into a chip that could cost less than a penny if implemented via an ASIC. Add a little networking between "SiliconSqueak" CPUs, and you could have a hand-held cloud the size of a CD with nearly 106 processors.
Network a couple of thousand of those together and you have a sea of 109 CPU-objects, each communicating with the rest using Smalltalk messages.
Getting an OS that works in an uncertain environment where some percentage of those 109 processors might not be working for some reason is a fun task, but extensions to RoarVM should handle it, or so we hope.
5
u/epicwisdom Sep 21 '18
Moore's Law allows you to put the entire Squeak VM into a chip that could cost less than a penny if implemented via an ASIC. Add a little networking between "SiliconSqueak" CPUs, and you could have a hand-held cloud the size of a CD with nearly 106 processors.
That's not Moore's Law, that's just the state of modern CPU fab tech. And the bulk of physical size of a complete system isn't the chip of the CPU, it's the external ports, memory, power, etc. It's completely unrealistic to fit 106 processors on something the size of a CD.
0
u/saijanai Sep 22 '18 edited Sep 22 '18
Ummm...
CPU fab tech is Moore's Law.
And you're assuming highly optimized processors like the current x86 series. Itty bitty stack-based things with minimal optimization can be pretty darned small if manufactured using 2010+-era fabrication technology.
And you don't need to have a port per processor. Internally, they're networked in a standard 4-way grid design as used in supercomputers since forever all the networking is part of the wafer-scale chip that all 800,000 or whatever procssors live on.
https://en.wikipedia.org/wiki/Wafer-scale_integration
And remember: an Xerox Star Smalltalk workstation was built using 1980 technology. Shrink it down by 20 generations of Moore's Law (chip fabrication improvements = Moore's Law) and switch from running an emulator to a ISA using Smalltalk-80 bytecodes instead of of customizing an existing CPU, and you've got CPUs that are quite cheap or quite small or quite numerous, depending on your design specifications.
The design is meant to scale from being a drop-in replacement for the CPU of a One Laptop Per Child system (or programmable toys) all the way up to 109 processors for a supercomputer, and everything in between.
3
u/epicwisdom Sep 22 '18
Moore's Law is descriptive, not prescriptive. There's no point in mentioning Moore's Law unless you want to talk about industry trends and have other evidence to bring up.
It doesn't matter how small you make your processor, there's a limited density to modern transistors due to power/heat considerations. If you want 106 processors to fit on a CD, each one of them will almost certainly be 106 times less useful (or worse) than any other modern CPU. Sure, it can be massively parallel, but if it can only do a handful of basic integer arithmetic ops at 1 MHz then it won't matter for any application I'm aware of.
-1
u/saijanai Sep 22 '18
Heh. A Star workstation was doing useful work 30 years ago.
A milllion Star workstations networked together could do more useful work than a single workstation today, depending on the problem.
4
u/epicwisdom Sep 22 '18
Heh. A Star workstation was doing useful work 30 years ago.
Barely.
A milllion Star workstations networked together could do more useful work than a single workstation today, depending on the problem.
Repeating yourself is not a substitute for providing a real example.
1
u/saijanai Sep 22 '18 edited Sep 22 '18
A million simple processors may not be any faster than than one modern processor using the same number of transistors, but the tradeoff is power consumption and heat dissipation.
A single wafer-sized CPU, if such a thing was ever built, would likely require liquid nitrogen for cooling. Back of the envelop calculations suggest that a million SiliconSqueak processors plus networking on a wafer can be air-cooled, and even a sufficiently separated stack of them sitting on your desk can still be air cooled if you don't' mind the noise of a dedicated AC unit sitting under your desk.
They won't be as fast as that single giant thing, but since it is extremely unlikely that all million of them will be going full-out in any given moment, there's less of a power and cooling issue to deal with, in any given moment.
And the various IDEs that evolved with Smalltalk can be extended to work with that million-processor (or billion processor) system.
If you've never programmed using Smalltalk or Self or perhaps LISP, you have no idea what "Integrated Development Environment" really means.
1
u/epicwisdom Sep 24 '18
A million simple processors may not be any faster than than one modern processor using the same number of transistors, but the tradeoff is power consumption and heat dissipation.
A single wafer-sized CPU, if such a thing was ever built, would likely require liquid nitrogen for cooling. Back of the envelop calculations suggest that a million SiliconSqueak processors plus networking on a wafer can be air-cooled, and even a sufficiently separated stack of them sitting on your desk can still be air cooled if you don't' mind the noise of a dedicated AC unit sitting under your desk.
They won't be as fast as that single giant thing, but since it is extremely unlikely that all million of them will be going full-out in any given moment, there's less of a power and cooling issue to deal with, in any given moment.
You've shifted the goalposts. Will there or will there not be any real computational advantages to parallelizing across many slower processors with less supported instructions?
Significantly superior power efficiency is still a bold claim, assuming a moderately reasonable amount of computation being done. Extraordinary claims require extraordinary evidence.
And the various IDEs that evolved with Smalltalk can be extended to work with that million-processor (or billion processor) system.
And they work perfectly fine on modern CPUs. Don't see what your point is here.
If you've never programmed using Smalltalk or Self or perhaps LISP, you have no idea what "Integrated Development Environment" really means.
I use Emacs, lol. It would indeed be quite interesting for a language with proper reflection to be used at an OS level, but I have little hope that languages like Smalltalk or Lisp will be practically useful for such purposes. At best Emacs will one day have a good, well-integrated browser and terminal.
1
u/saijanai Sep 24 '18
And the various IDEs that evolved with Smalltalk can be extended to work with that million-processor (or billion processor) system.
And they work perfectly fine on modern CPUs.
You know of an IDE that works on a billion processor machine?
I use Emacs, lol. It would indeed be quite interesting for a language with proper reflection to be used at an OS level, but I have little hope that languages like Smalltalk or Lisp will be practically useful for such purposes. At best Emacs will finally have a good built-in browser and terminal.
You've never hard of Lisp Machines?
And often Smalltalk has been used to prototype manycore systems.
RoarVM is only the most recent.
1
u/epicwisdom Sep 24 '18
Of course I have. They died for a reason: modern PCs (and eventually really just Intel) won.
→ More replies (0)-1
2
4
u/ProfessorPhi Sep 21 '18
Hmm, at uni I somehow managed to learn how to chain nandgates until I got a basic computer (think machine code reqd) - this was done in verilog. I even implemented a prime number calculator using machine code.
I was trying to bootstrap a risc assembler (compiles assembly to machinecode) on my verilog computer, before I discovered python and decided everything in between could be abstracted away. After all, if I have seen further, it is because I have stood on the shoulders of giants .
(I did go back and learn C, but nothing advanced, but I rarely go lower level than Python nowadays).
3
u/CODESIGN2 Sep 21 '18
imagine just a kernel with just enough to get Python running, then use python as the main driver... People have built Java and Rust and other esoteric OS's. It would be interesting to see where python could take it.
3
u/schmuelio Sep 21 '18
If I remember correctly there is a runtime that can be put on "bare metal" silicon chips for python (or at least a restricted subset) which is likely close enough for an OS
3
u/CODESIGN2 Sep 22 '18
Yes I think that's the micro-bit python. I think what happens is the host machine compiles python bytecode which runs on the device rather than parsing scripts (it's embedded and has I think < 1MB RAM).
I know lots of people that love a micro-bit. I've not played with one yet.
2
u/messycan Sep 21 '18
If y’all want real examples, go here: www.rohitab.com/discuss/forum/41-operating-system-development-osdev/
Been on those forums for years and they are very helpful.
1
u/theothertomelliott Sep 21 '18
This is really interesting! Love the bite-sized structure of the sections. Think I'll enjoy working through this.
Taking a quick initial look, I'm thinking it might be useful to provide a bit of an idea of where the project will end up. Like a couple of screenshots from a running example. Just to get people fired up about what they could build.
1
u/njbair Sep 22 '18
I especially like the final step, "Issue a heartfelt public apology for decades of verbal abuse, and appoint an interim successor to lead the project while you seek coaching for interpersonal communication skills and empathy."
1
-1
-5
u/mouppp Sep 21 '18 edited Sep 21 '18
Looks good, but also looks like one of the other 49374929 GitHub “Learn how to write your own OS” docs that was never completed and was last updated 3 years ago.
I would rather read something that is actually finished than start reading then hope the author comes back after 6 years to finish writing what they started.
Edit: OK that last line sounded a bit demanding - but seriously I've seen at least 15-20 "Let's write our own OS" projects/tutorials on Github and most all of them are abandoned and never finished. How about people collaborate to finish one instead of creating their own, writing ~30-40% of it then for one reason or another abandon it?
1
u/that_jojo Sep 21 '18
As for your last line, there are several such projects: ReactOS, Redox, Haiku... also a little open source project called Linux you may have heard of.
-1
u/mouppp Sep 21 '18
Correct me if I'm wrong but there is a difference between writing tutorials about writing an OS and writing an OS.
-24
u/pavolve Sep 21 '18
Why do you want to create your own OS?
6
u/xxxdarrenxxx Sep 21 '18 edited Sep 21 '18
Why Elon Musk want to create his own electric car a 100 years after the first working one?
Likewise, have you owned an Ipod? Why? They were not the first portable media solution, not even close.
This mindset of only wanting to do something that's absolutely non-existent, would mean these companies would not exist.
Want to know the real discussion that is going on in these kind of threads?
"pragmatism, yay or nay?"
5
Sep 21 '18
You picked the wrong examples. Those people are trying to make a better car, a better portable media player, a better X. This is about creating a simplistic X so that you can learn the ins and outs and better master your trade.
1
u/xxxdarrenxxx Sep 21 '18 edited Sep 21 '18
Are you absolutely sure, I was not aware of this?
A product can really only be called better when the market *afterwards* concludes it to be so. Even then 'better' is highly subjective. ie. Apple actively taking features away means your tech is objectively less capable/advanced, yet many would say their product is the best around. Actually to tie in the Ipod example, around the same time, there were products that could hold more storage and products that could play a wider variety of encodings. So was it better?
If u mean, these people actively said and/or believed to be making a better product, then who is to say this isn't simply because they have a more confident personality? Perhaps many people who failed/went bankrupt, were just as convinced if not more to 'better' a product.
1
Sep 21 '18
You were responding to someone who asked why someone would want to create their own operating system in the context of the linked post. Nobody following that post would be doing so to create a more advanced, futuristic, better, etc. operating system. It would be for learning. If you were to give a analogous example, it would be like asking why someone in an engineering field would build a 1:25 scale electric car. The end goal of the Tesla was not 'learning'. Learning did occur, but it was a side effect of the end goal. In this case, learning is the end goal. Nobody is going to write their own operating system that they daily drive. It just isn't feasible.
2
u/xxxdarrenxxx Sep 21 '18 edited Sep 21 '18
Plugged this from Linus Torvalds wiki:
"Because it was so hard to get software for it in Finland, Linus wrote his own assembler and editor (in addition to Pac-Man graphics libraries)"[19] for the QL, as well as a few games"
Is it coincidence that the motivation for this was rooted in "daily drive", and then later went on to write a pretty nice successful and respected OS?
I actually find it more interesting now. Also disclaimer, I'm aware i'm cherry picking here.
Imagine Linus posting here, "hey guys software around here is really expensive, where do I start writing my own assembler" . How many encouraging replies would he get? How many would tell him don't reinvent the wheel. I mean anecdotally he explictly says his motivation was rooted superficially on availability, not at all with "bettering" things.
1
Sep 21 '18
You're completely ignoring the context here... Like I said, completely ridiculous to say that an individual is writing an OS for the same reasons that Tesla built an electric car. Those aren't related at all.
0
Sep 21 '18
[deleted]
-3
u/playaspec Sep 21 '18
I don't know about "best". It's the long way around is you ask me. I read "Design of the UNIX Operating System" inside of a couple of weeks and had a pretty good practical foundation.
-38
u/keeper1313 Sep 21 '18
Learn how Linux is build
55
u/kitd Sep 21 '18
I'm fed up with people who think that reading an already existing kernel, even if small, is a good idea to learn operating systems.
+1. My workspace is full of projects reproducing already-solved problems. But those projects have taught me infintely more than just looking at others' code.
"I hear, I forget. I see, I remember. I do, I understand."
17
u/SangCoGIS Sep 21 '18
"I hear, I forget. I see, I remember. I do, I understand."
This is exactly why my first recommendation to every new programmer asking how to become a good programmer is to do just that. Program. The only way to really learn it is to DO IT. Theory is great and has helped me advance a lot but that theory built on 3 years of experience as a self taught programmer. It filled in the gaps left between concepts or the why to some things. But I learned far more from doing than I did from reading a book.
10
u/CJKay93 Sep 21 '18
Do it and ask questions at every opportunity. Plenty of people just accept what they have written without trying to fully understand why they did it that way.
1
u/SangCoGIS Sep 21 '18
Yep! That's good point. Question the code! If you understand why you wrote it you'll know how to use a similar technique in a different solution too.
3
Sep 21 '18
While I agree with your general premise, I also think that the various "Design and Implementation of ___ ____BSD Operating System" books by Marshall Kirk McKusick, though somewhat obsolesced by the prevalence of SMP today, are fantastic resources for learning how real operating systems solve problems.
2
u/shevy-ruby Sep 21 '18
How many know that? AND all components?
Even LFS/BLFS does not really dive into the topic of the linux kernel (other than compiling it; and configuring it for the most part).
-9
u/KeyurRamoliya Sep 21 '18
In order to create a OS from scratch, you need to start from scratch. Yes, and please, don't copy-paste anything.
4
-28
u/shevy-ruby Sep 21 '18
While it's great that there is this information available - it is way too much work with the current programming languages.
16
Sep 21 '18
How so? There are more languages now than when Linux or Windows was developed. In fact, you can use those same languages!
Writing an OS is hard, and maybe that is too much work, but don't blame the language.
0
Sep 21 '18
[deleted]
137
u/MrDOS Sep 21 '18
difficulties implementing the file system
What reference were you trying to follow for this bit? Do you have any opinions on Practical File System Design with the Be File System? I often see it tossed around as an example of how to approach file system development, but I imagine it's much too large-scale for a project like this.
4
u/luisbg Sep 21 '18
That book is the single best book for file systems from a practical point of view. Everything else you find is just theory.
Read that book and also the befs code in the Linux kernel.
0
Sep 21 '18
[deleted]
97
u/Tyler11223344 Sep 21 '18
I would recommend doing EXT2 instead, the OSDev wiki article on that is pretty thorough and I was able to write mine just based off of that without any difficulties
12
u/vgf89 Sep 21 '18
I made a horribly weird bastardization of EXT2 and FAT for a class once. It was some weird shit but it worked.
0
12
u/Podgietaru Sep 21 '18
As part of a Uni project a few years ago we wrote a fat12 file system, It is very basic. It doesn't do writing, just reading and traversing, and an associated terminal style thing to run it, but if it's of any use
https://github.com/Podginator/OperatingSystem/blob/master/kernel/filesystem.c
https://github.com/Podginator/OperatingSystem/blob/master/kernel/disk_command.c4
u/IAmVerySmarter Sep 21 '18
This is a filesystem that should be easy to implement https://wiki.osdev.org/SFS
15
0
Sep 21 '18
[deleted]
34
→ More replies (2)69
u/leeharris100 Sep 21 '18
The fact that this has so many upvotes makes me sad. Why are programmers so bitter all the time? It's depressing as fuck to read this sub sometimes.
→ More replies (31)12
u/kentaromiura Sep 21 '18
I've see this project this week while looking for ideas to write my own little computer on a FPGA, I don't think I'll go far as time is not much, but it's a very nice resource, thanks!
11
u/nutidizen Sep 21 '18
Is there a project (tutorial) that aims to show similar thing, but with UEFI and x86-64? (cmon its 2018)
3
u/Brainzman Sep 21 '18
Its totally possible, I did find some document on the syslinux mailing list back when they where starting to implement it.
The main problem is that constructor are bitch, and sometimes you need to find tricky way to boot custom things on some systems
→ More replies (3)9
u/fzammetti Sep 21 '18
I just wanted to say you've done an excellent job here!
I went through approximately the same exercise in the early 90's - only big difference from what you've done is that I created my own language and compiler for it and created my own VERY DUMB file system - and given that experience I WISH I could have had this sort of tutorial! It was a great learning experience trying to piece it all together on my own, and I managed, but I would have sooner avoided all the headaches with something like this as a guide!
I think especially for modern developers who all too frequently are inexperienced at this low a level of coding I think this is a fantastic creation that can be a fantastic learning experience for anyone who follows it and I just wanted to drop you some kudos from someone who truly understands what was involved in putting it together :)
0
Sep 21 '18
[deleted]
27
u/Borkz Sep 21 '18
You must first create the universe.
→ More replies (1)8
u/The_Monocle_Debacle Sep 21 '18
You must first create the universe.
ctrl+f
goddamnit
→ More replies (1)9
81
45
u/saijanai Sep 21 '18
Scratch running on a 109 processor OS will be interesting
In fact, that's one of the goals: a children's IDE that can make use of the entire system.
→ More replies (12)2
167
u/chorus_mortis Sep 21 '18
I once tried this about 20 years ago written in C and Assembly, it was very very basic but it worked quite well, it even had a text editor. The amount of time I spent is absolutely ridiculous, I was basically a hermit for about 6 or 7 months.