--------------------------------------------------------------------------------
__ _____ __ _
/ /___ _ / __(_)___ ____ ____/ /___ _____ (_)
/ / __ `/ / /_/ / __ \/_ / / __ / __ `/ __ \/ /
/ / /_/ / / __/ / / / / / /_/ /_/ / /_/ / / / / /
/_/\__,_/ /_/ /_/_/ /_/ /___/\__,_/\__,_/_/ /_/_/
-------------------------------------------------------------------------------- | blog | blog_archive | about | contact | tutoring | --------------------------------------------------------------------------------
$ cat blog/* wayless_presentation.md: 2019-11-10 [permalink] [raw] --------------------------------------------------------------------------------
I had the opportunity to talk about Wayless, an operating system kernel I've been working on, at the Portland Mozilla offices last Wednesday. The slides from the talk are up at https://finzdani.net/rust_osdev.pdf
Thanks everyone who made it out! I'm excited I got the chance to speak about what I've been working on. Here's the links from the last slide of my presentation:
Phil Opperman's BlogOS: https://os.phil-opp.com
OSDev Wiki: https://wiki.osdev.org
AMD's x86_64 Programmer's Manual, Volume 2: https://www.amd.com/system/files/TechDocs/24593.pdf
Muchless: https://gitlab.com/waylon531/muchless
Wayless: https://gitlab.com/wayless/wayless
-------------------------------------------------------------------------------- -------------------------------------------------------------------------------- threads.md: 2019-11-10 [permalink] [raw] --------------------------------------------------------------------------------
Here's a short tutorial on what threads are and why they're important. This is inspired by a question I got on Discord, I figured that I might as well write up a long-form response as it might help other people. Hopefully there'll be more blog posts incoming in the near future :)
Early personal computers were only able to run a single program at a time. This affected early operating systems like MS-DOS--whenever you run a program in MS-DOS it gets sole control of your CPU. This works perfectly fine when the only way to use the computer is through a command-line interface--you'll type a program's name, it'll run for a bit, and eventually it'll exit and once again MS-DOS will be in control of your CPU.
However, this is kind of limiting. You can't edit a text file and listen to your collection of MIDI music files at the same time. You can't run multiple different command-line sessions. And worst of all, if you have a graphical interface to your computer, you can't have multiple windows open at the same time.
Single-task operating systems had some severe limitations. Pretty much none of these operating systems are around today: most modern systems are multitasking.
Multitasking operating systems have the ability to run multiple programs at the same time. Not only does this now let you run programs in the background of your computer, it now means that you can have your browser open at the same time as your IDE. It's probably hard to imagine a computer that doesn't let you run multiple programs.
To accomplish this your operating system has to do a bit of work, your processor can still only run a single program at one time. To give the illusion of multiple programs running simultaneously the operating system has to rapidly switch between all currently running programs, called threads, letting each run for a fraction of a second at a time. This process is called context switching, specifically thread context switching. However, the operating system has to figure out exactly when to switch threads. There's two major ways to do this, through cooperative multitasking and preemptive multitasking.
Cooperative multitasking is named as such because threads are expected to cooperate with each other to make thread switches happen. Each thread can yield control to the operating system, letting the operating system know that it's ready for a thread switch to happen. Yielding typically happens through a system call provided by the operating system. Each thread is expected to yield withing a reasonable time, and this method will get great performance. However, as soon as you get a malicious process that refuses to yield your computer is frozen, running that malicious process until the end of time (or a hard reset).
Using preemptive multitasking threads will automatically be switched after a specified period of time, typically somewhere between 1ms and 100ms. The operating system will use a timer to generate periodic interrupts, stopping the current process and letting the operating system handle a switch into the next process. Typically the operating system will also let threads yield in addition to automatically preempting them. Preemptive multitasking fixes the issue of malicious program locking up the computer: now if a program tries to run forever without yielding it'll just get preempted by the operating system, letting other programs run like normal.
A thread of execution represents part of a running program. Some programs may run as a single thread, some may run using multiple threads. Many operating systems also make a distinction between threads and processes, but I won't here. To run, each thread needs its own set of registers, as well as its own virtual memory space. Each thread keeps track of its own instruction pointer, a pointer to the next assembly instruction to execute. Depending on the operating system some extra information might need to be stored with each thread (Wayless keeps track of a capability space for each thread). To switch between threads the operating system saves the values of the instruction pointer and every register. The operating system calls its scheduler, which figures out what thread to run next. Once the operating system knows what thread to run it loads the values of all the registers from the new thread, and switches from the old address space to the new space. Then the operating system sets the instruction pointer to that of the new thread. Now the computer is all set up to start executing the new thread, right where the thread left off.
The scheduler keeps track of all of the threads and decides what order to run them in. There's a lot of different scheduling algorithms, so I'll just cover a few here. One of the simplest ways to schedule threads is using a round robin scheduler. This scheduler uses a queue of threads, which can be implemented using a linked list. When the operating system switches threads it saves the current thread to the end of the queue. Then, the next thread to run is dequeued from the front of the queue. This ensures that every thread gets ran, and that threads aren't able to starve the system of CPU time. It also allows for a simple implementation. However, it doesn't allow for priority levels and potentially limits the responsiveness of high-priority threads.
A priority-based round robin scheduler adds priority levels. A different round-robin queue exists for each priority level. Processes from higher priority queues get ran first--each thread from the highest priority level is run, then each thread from the next-highest priority level, and so on. This scheduling algorithm extends the round robin scheduler, adding slightly to its complexity. Now programs that need to be highly responsive, like graphical programs, are guaranteed to run before lower priority programs.
Some computers have multiple cores, or multiple physical CPUs. A thread can run on every single core, potentially providing a huge increase in speed. As each program runs using at least one thread, this automatically provides a benefit when running many different programs simultaneously. However, breaking your program up into multiple threads usually takes a bit of work (modulo programming language). If you can break your program up into multiple threads then each of those threads can run in parallel, given enough cores, giving a massive boost to performance. If your program is only a single thread then it won't be able to take advantage of the multiple cores.
https://wiki.osdev.org/Context_Switching
https://wiki.osdev.org/Multitasking_Systems
-------------------------------------------------------------------------------- -------------------------------------------------------------------------------- gdb.md: 2019-11-21 [permalink] [raw] --------------------------------------------------------------------------------
If you've never run GDB before, you can start it up with the following command:gdb <program>
If you've been compiling to a.out then you'd start GDB up withgdb ./a.out
Make sure to add the -g flag when compiling your program! This would look something likeg++ -g my_program.cpp
And here's some conventions I'm using:
[arg] -- This is an optional argument<arg> -- This is a required argument<foo|bar> -- This is a required argument that can be either foo or barThere's a few tricks I've learned that make GDB easier to use. First of all, you don't actually need to type out the full command name to run a command. You can type the first few characters of a command, or potentially even a single character. This means instead of typing layout you can type lay, and instead of typing break you can type b.
Sometimes you want to run the same command a bunch of times in a row. Typing next a bunch of times is a pain. However, if you enter an empty command and just hit Enter then GDB will repeat the last command. Instead of typing next a bunch of times, you can type it a single time and then repeatedly hit the Enter key.
run [args] -- This command will start your program. You can optionally specify arguments, these will get passed as command-line arguments to your program.
layout src -- This command displays the source code of your program above the main GDB command window.
backtrace -- This command will show all function calls that were made to reach the current location. This is particularly useful if your code breaks inside of a library--you can see which of your functions called the library's function.
frame <frame number> -- Each function that backtrace shows you has its own stack frame, with a frame number shown in the leftmost column. You can enter one of these frames with the frame command, bringing GDB back to that function. This shows you the relevant code and lets you look at variables inside of that function.
print <variable> -- This command prints out the contents of a variable. Any valid C/C++ will work here, you can access arrays (a[10]), access fields of structs or classes (person.name), and even dereference pointers (*ptr).
display <variable> -- This command works similarly to print. However, display will print out your variable whenever you move to a new line of code.
break <function|function:label|file:line|line> -- This command sets a breakpoint at the desired location. GDB will stop execution once the breakpoint is hit, letting you inspect problematic areas of code. The breakpoint location can be given in a couple of forms. You can use function names as breakpoint locations, or you can give a filename and line number in the form file.cpp:42. If you find yourself using file:line on the same location repeatedly, consider adding a label there, and using function:label to give it a stable location and convenient name.
continue -- This will continue the execution of your program after a breakpoint is hit.
del [breakpoint number] -- This will either delete all breakpoints if given no arguments, or delete a single breakpoint specified by the breakpoint number. When creating a new breakpoint GDB will show you its number.
step [count] -- This will make GDB move to the next line of code, moving into any functions encountered. A count can be specified to step multiple times
next [count] -- This command works almost the same as step. However, next will step over functions. This can be useful when you want to avoid stepping through library functions.
help [command] -- If no command is given this will show you information about all GDB commands. Otherwise, if a command is given, help will show more information about that specific command.
layout <reg|asm> -- There are layouts designed specifically for debugging assembly code and for reverse engineering programs. Similar to src, asm will show the relevant assembly code in a pane above the main GDB window. reg shows both the assembly code and the values of all registers, each in their own pane.
x[/FMT] <memory location|variable|$register> -- x is used to examine the value of a variable or memory location. x, like print, is used to print out information about your program. However, x will work with arbitrary memory locations and allows you to examine the contents of registers. The FMT string lets you change how the value is displayed, you can display values as hex, decimal, and in many different sizes. help x has more information on FMT.
info registers -- This command prints out the values of all registers.
This work is licensed under the Creative Commons Attribution-ShareAlike 4.0 International License. If you're interested in modifying this document you can obtain the original markdown at https://finzdani.net/gdb.md
-------------------------------------------------------------------------------- -------------------------------------------------------------------------------- forward.txt: 2019-01-23 [permalink] [raw] --------------------------------------------------------------------------------
Local mail is a pretty cool concept that has all but died out. Because it is centered around one system mail gets delivered directly to each user, letting users customize how they recieve mail. One of the ways they can do this is though the .forward file. ---------------------------------------------------------------------------- The .forward file: sendmail will look for a file called .forward in your home directory. This file contains a list of entries to forward messages to. For example, if my .forward file consisted of: .forward: ``` steven paul uelen ``` then all messages sent to me (uelen) would get sent to steven, paul, and then a copy would finally get sent to me. Interestingly, the .forward file supports commands in addition to just usernames. You can stick a pipe in your .forward file, followed by a command, to pipe all of your email messages to a specified program. My .forward file looks like this: ~/.forward: ``` |/usr/bin/procmail ``` I pipe all of my messages to procmail, which takes them out of the mail spool and sends them to my Maildir. mutt, my preffered email client, can then interact directly with this mail. I can also get rid of the "You have new mail" message without actually deleting my mail :P I'll cover procmail in a future textfile
-------------------------------------------------------------------------------- --------------------------------------------------------------------------------