failing like never before


My Time with Arch

I've been a proud and content Arch Linux user for a little over one year and nine months now, far longer then I've spent with any other Linux distribution. Arch put a happy end to my constant distro-hopping lifestyle, and I've been so pleased with its simplicity and performance that I've spared nary a glance at any other distribution over these past 21 months. But in more recent times, I've been having some disagreements with my Arch system and so I've slowly started reverted to frequenting the old haunts of my distro-hopping days (i.e., and  other such distro news sites).

Stability has always been a much touted feature of Linux in general, but some distributions lay a greater claim to that attribute then others. Arch in particular has tended to be slightly more bleeding-edge then other distros, sacrificing stability for the newest features; packages are updated in the repository as soon as new versions are released and with  a relatively minimal amount of time (extremely minimal compared to other Linux distros like Debian) spent in testing in an Arch environment, just enough to ensure that the packages don't completely break the system. While this strategy has its benefits, namely that it allows users to get the latest and greatest software right when it comes out, it comes at the cost of stability (and security to some degree). And the more packages that I've added to my system, the more I've started to notice just how unstable Arch can be.

I generally run "pacman -Syu" to do a full system update at least every week, and I try not to let my system stay without an update for three weeks at the longest, so in general I'll stay pretty well up-to-date. But it has not been uncommon, that after performing a full update, that my system completely locks up or goes completely nuts. Take for example, just a few weeks ago, when a full system update made my Arch Linux partition completely unbootable and required that I boot a live CD and futz around in the configuration files. When my laptop was finally usable again, I had to mess around some more with my wireless drivers to get them working again. And lately, after my most recent system update, I've been having some problems where my laptop will occasionally freeze up and become totally unresponsive to everything except a hard reboot, and system logs show no behavior to be out of the ordinary. Of course, not all of the breaks in Arch have been this bad. About four months ago, a system update made it so that I could no longer hibernate, a problem which was easily remedied by a quick visit to the Arch wiki and a few short commands. Sadly, the list of weird errors goes on (although its not that long).

Two years ago, when I was moving off of Debian testing, Arch's bleeding edge packages were quite welcoming, but not that I've matured a little bit, I don't care as much about the latest features (Lets face the facts, the programs I've been using the most these past few weeks, have been vim, GCC, SVN, and Zoom). My first priority these days, is getting shit done. And if my laptop decides to go bat-shit-crazy now and then, it seriously hampers my ability to work properly. I don't mind a few bugs now and then, and I could probably even live with a rare kernel panic, but sometimes I get the feeling that Arch is maybe just a little too bleeding edge for me.

I mentioned earlier that another one of the costs of having the latest and greatest software, is security. A lot of the newest software releases tend to be not as well hammered out and therefore are slightly more prone to have security holes. I'm only a slightly paranoid Linux user, so while the lack of security is a little worrying to me, its not a huge deal breaker. Arch's lack of solid support for more powerful RBAC security modules like SELinux or AppArmor has also been a little worrying to me. I would love to be able to slap on some powerful RBAC policies on my laptop to give me greater piece of mind, but Arch's normally awesome wiki is a little lacking in help (although it seems that reccently, the SELinux page has gotten a little more meat to it).

This next point is a rather silly and illogical thing to hold against a distribution, but I feel that it needs to be said because its entered my thoughts a few times in the past year. Whenever I go in for an interview, I generally try to play up my Linux expereince (which is not incredible, but still fairly impressive enough). The logical question for an interviewer to ask of course, is "what distribution(s) do you use?" As soon as the words "Arch Linux" comes out of my mouth, I can see the interviewers knocking some points off of my interview. People always assume that Arch is just another one of those random "edge" distros that is basically just an Ubuntu/Fedora knockoff with some sparkles thrown in, and no maybe how much I explain it to them, I know that they don't respect an Archer as much as they respect a Slacker. So yeah, I'm a little shallow, but I do care about what people think about me, especially in interviews. A part of always wishes that Arch was just a little more mainstream and a little more well known.

So I've taken some pretty mean shots at Arch, but my comments shouldn't be miscontrued to indicate that hate I Arch. Quite the contrary in fact, I've loved using Arch. My old Arch review enumerates out more clearly the points of Arch that I really like, but I'll list them out here quickly.

  • Fast! - Compiled for i686 and lightweight with no extra cruft thrown in
  • Clean - there is nothing on my Arch system that I didn't put there
  • Simple - things tend to be very straightforward and elegently simple
  • Awesome documentation and user community - Arch's comprehensive wiki is in my opinion, one of its strongest selling points, also the forums are quite helpful.
  • Rolling updates - its nice not to have do some big update every six months...

All the reasons that I first came to love using Arch still hold true, its simply that as time has worn on, I've changed a bit: I don't care as much for bleeding edge features, stability and security have become bigger issues, and I've started caring about what other people think about me. So I've been asking myself, "is Arch still for me?" And I think the answer might be no. I purchased a used IBM Thinkpad reccently, and I don't think Arch is going to be my first choice for it.

It seems like its time for Arch and I to "take a break" in our long relationship. But don't worry Arch, its not you, its me.

Tagged as: , , 1 Comment

Smashing the Stack for Extra Credit

(So this one is a little old... I have a habit of writing up drafts, stashing them away to be uploaded later, and then completely forgetting about them.)

A quick intro to buffer overflow attacks for the unlearned (feel free to skip this bit).

I would highly recommend reading AlephOne's Smashing the Stack for Fun and Profit if you really want to learn about buffer-overflow attacks, but you can read my bit instead if you just want a quick idea of whats its all about.

Programs written in non-type safe languages like C/C++, do not perform bounds checking on memory when doing reads and writes, and are therefore often to susceptible to what is known as a buffer-overflow attack. Basically, when a program allocates some array on the program stack, the possibility exists that if the programmer is not careful, her program may accidentally overwrite the bounds of the array. One could imagine a situation where a program allocates N amounts of bytes on the stack, reads in input from stdin using scanf (which terminates reading input when it hits a newline) and stores the read bytes in the allocated array. However, the user sitting at the terminal might decide to enter in more then N bytes of data, causing the program to overwrite the bounds of its array. Other values on the stack could then be unintentionally altered which could cause the program to execute erratically and even crash.

But a would-be attacker can do more then just crash a program with a buffer-overflow attack, they could potentially gain control of the program and even cause it to execute arbitrary code. (By "executing arbitrary code" I mean that the attacker could make the program do anything.) An attacker can take control of a program by writing down the program stack past the array's bounds and changing the return address stored on the stack, so that when the currently executing function returns control to the calling function, it actually ends up executing some completely different segment of code. At first glance, this seems rather useless. But an attacker can set the instruction pointer to anything, she could even make it point back to the start of the array that was just overwritten. A carefully crafted attack message, could cause the array to be filled with some bits of arbitrary assembly code (perhaps to fork a shell and then connect that shell to a remote machine) and then overwrite the return address to point to the top of the overwritten array.

A problem with the generic buffer overflow attack is that the starting location of the stack is determined at runtime and therefore can change slightly. This makes it difficult for an attacker to know exactly where the top of the array is every single time. A solution to this, is to use a "NOP slide," where the attack message doesn't immediately begin with the assembly code but rather begins with a stream of NOPs. A NOP is an assembly language instruction that basically does nothing (I believe that it was originally included in the x86 ISA to deal with hazards) so as long as the instruction pointer is reset to point into somewhere the NOP slide, the computer will "slide" (weeeeee!!!!) into the rest of the injected assembly code.

Sounds simple so far? Just you wait....

The trials and tribulations of my feeble attempt to "smash the stack."

The professor for my security class threw in a nice little extra credit problem on a homework assignment last quarter. One of the problems in the homework asked us to crash a flawed web-server using a buffer overflow attack, but we could get extra credit if we managed to root the server with the buffer overflow. Buffer overflow attacks on real software are nontrivial, something my professor made sure to emphasize when he told us that in his whole experience of teaching the class, only one student had ever successfully rooted the server. Now Scott Adams, author of Dilbert, mentioned in one of his books that the best way to motivate an engineer, is to tell them that a task is nearly impossible and that if they're not up for the challenge, then its no big deal because so-and-so could probably do it better. This must be true, because after discussion section, I went straight to a computer and started reading Smashing the Stack for Fun and Profit, intent on being the second person to "smash the stack" in my school.

I was able to crash the server software within less then one minute of swapping the machine's disk image in, as it was a ridiculously simple task to guess where an unbounded buffer was being used in the server code and force a segmentation fault. It took me a few more minutes to trace down the exact location of where the overflow was occurring (there was a location in the code where the program would use strcat() to copy a message), but as soon as I did, I booted up GDB and started gearing up for some hard work.