Thursday, October 23, 2014

The Best UNIX Computing Environments for Scientific and Technical Computing

So, I promised to write on a wide variety of topics.  This entry is about Unix and Unix-like environments.  If you have absolutely no clue what UNIX is, you can probably skip this post (unless you are just here to learn something new, in which case, kudos for you). 

Let me paint the picture of the kind of person this post is intended for.  You spent most of your life using computers, fine with Windows (or for some of you, OS-X) without a care in the world.  Then you start studying computer science, physics, engineering, or some related discipline and realize that there are a bunch of UNIX programs out there used in your field which may not have easily installed Windows versions and you do not want to go trudging to the lab every time you need a Unix environment.  Besides, it would be nice to have a UNIX environment available for you at home to learn on, would it not?

The intent of this post is not really to show you how to set up a specific Unix-like environment.  There are plenty of other sites that do that.  The intent is to run-down the options that you have.

What Is Unix?

If you are reading this, you probably have some vague idea about what Unix is and there are plenty of other websites which will give you a detailed history or technical definition.  I am just here to inform you what the various parts of Unix are that you may need in order to set up your own environment.

  • The Kernel handles operating system tasks at the lowest levels.  Everyone needs a Unix kernel or kernel emulator, which is going to be part of any environment we discuss here.
  • Developer Tools are what you need if you intend to do any programming yourself or build open-source tools from source.
  • Utilities are the low-level tools that you need to interact with the operating system, for instance, copying files or editing text.  You probably want a system that supports the widest variety of standard utilities as possible out of the box.
  • Package Managers allow you to install new open-source programs, either by compiling them from source or by downloading them, and help keep them updated. 
  • X11 is the standard way to display Unix graphics.  This is really, the first optional component.  If you want to do everything through a text interface, you do not need X11.  If you want to see graphics, you do.
  • Windows Managers allow you to manage multiple X11 programs in a sensible way.  If you need X11, you need some kind of Windows manager whether it be a simple interface to a terminal (like Putty running with Xming) or a full blown desktop environment, like KDE or GNOME.

Unix Environments

 

Macintosh OS-X

If you have a Macintosh then you have genuine POSIX-certified UNIX.  Congratulations!  You technically are done.  The actual Macintosh operating system is called Darwin and is a port of an old UNIX-like operating system called Free BSD, which Apple adopted as its core operating system kernel for the same reason Microsoft adopted Windows NT.  Unlike the old Macintosh operating systems, BSD was very stable at its core.  The interface that most Apple users think of as the operating system is actually a desktop environment called Aqua.  Buried somewhere in there is a program called terminal that gives you access to Apple’s take on F-BSD. 

In order to install common scientific a developer tools, you need to install X11, which is relatively easy.  Actually getting the tools is a bit more difficult.  In fact, in my opinion, in many ways getting Unix tools to run properly on OS-X is at least as difficult as getting Unix tools to run on Windows, despite the fact that OS-X is fully Unix.  But if you already own an Apple product and you are willing to put in the time, there are projects such as MacPorts, Fink, and other package managers.  

Boot Linux

By far, the most popular Unix-like environment is Linux.  There are thousands of flavors of Linux out there, the most popular being Ubuntu, which is linked in the above subtitle.  I am going to talk about Ubuntu, since it is the most popular desktop distribution of Linux out there, but feel free to look for others better-suited to your needs.  The advantage of booting Linux is that it installs full speed onto your hardware along-side or in place of your existing operating system (probably OS-X or Windows).  The disadvantage of booting Linux is that you cannot usually access your primary operating system while Linux is running (though you can access your files) and Linux driver support varies wildly.  In some cases, it works perfectly.  In most cases, there are one or more minor issues with your hardware.  In some cases, there are critical problems, like complete failure to see any video. 

By the way, there are some  alternatives to Linux, the most common being variants of BSD and Solaris.  Everything I am writing about Ubuntu is also true of other flavors of Linux and is probably true of BSD and Solaris if you decide that is your cup of tea.  However, I strongly urge you to choose Ubuntu or another major flavor of Linux due to greatly increased support and user friendliness.

Virtualize Linux

Virtualization is a great way to run Linux.  It allows your operating system to simulate a complete computer, so you can, in effect, run two different operating systems at the same time (or three, four, five. . . ).  The downside is that operating systems are going to be competing for the same resources, storage, CPU cycles, and most importantly, RAM.  If you have less than 4 GB of RAM, this is probably not the best option and if you want a complete Linux desktop environment, I suggest 8 GB of RAM. 

There are many different virtualization options.  The one I suggest is VMware.  We will get to why in the next section.  A link to the 14.04 VMware image of Ubuntu is here.  The VMware player (needed to run the virtual machine) is free for non-commercial use here.

The gist of this, and why it is one of my preferred solutions for higher-end computers (quad core processors and 6GB of RAM or more) is that you get a no-compromised, complete Linux environment that you can call upon at any time.  Also, Virtualization software companies typically supply all the drivers you need to get the operating system working correctly, whereas proper drivers for your real hardware might not exist for Linux.  Also, you can usually move the virtual machine from computer to computer without running into missing hardware drivers.

 

Virtualize Individual Linux Windows

VMware has a neat feature called Unity.  It allows you to run just the graphical windows of the operating system you are virtualizing, so they appear largely as if they are being run by the host operating system, other than the fact that you are using the clients windowing system.  If you do this, I suggest you install a second desktop manager like XFCE.  You can then log into this light-weight desktop environment when you want to use unity mode (don’t confuse VMware’s Unity option with Ubuntu’s Unity Desktop Environment) and back into your heavy-weight manager (Unity, GNOME, KDE or whatnot) when you want to run Linux in full-screen mode. 

unity

As you can see from the above photo, Kubuntu is being run with the XFCE windows manager in Unity mode under Windows.  This enables use of a native Linux environment and relatively good 3D acceleration using VMware’s open-GL drivers.  It might not work for playing games, but seems to do fine rotating 3D graphics with Mathematica or Celestia. 

Connect to a Linux Server via SSH

If you have a Linux server available or you want to create one, this is a great option.  While theoretically, you can forward the entire desktop, in the ideal situation you just want to have a terminal with an X-server.  If you are running OS-X, you already have SSH and just need to install X11.  If you have Windows, there are a lot of different configuration options, but I have found MobaXTerm to be the best.  It comes in a free version with an X11 server built-in. 

The upside to this is that you can also install a Linux server on your network and render the output on your desktop.  I might post about how to do this some other time, but if you have an extra computer lying around somewhere, you can install something like Ubuntu server on it and then run Linux programs from inside Windows via SSH. 

In my opinion, when it is available it is the best solution because:

  • Other than rendering X11 graphics, your computer doesn’t waste resources running Linux/Unix.
  • On a high speed network, X11 forwarding usually has no significant delay except in the most graphically complicated programs.
  • If you are connecting to school or work, someone else is does the dirty business of administrating the Linux environment.

However, it has some downsides such as:

  • If you are connecting to a remote computer (such as one at work or school) graphical applications using X11 might become extremely slow.
  • A few programs do not operate well over SSH, such as one of my favorite text-editors KATE. 

Cygwin

There are a number of different Linux emulation environments for windows.  Microsoft’s own SUA is deprecated and scheduled for removal after Windows 8.  Cygwin is the most robust and supported, is relatively fast, and it comes with X11 support built-in.  It basically emulates a complete Unix environment under Windows.  Unfortunately, the package manager leaves much to be desired, especially in terms of automatically updating software, but at least it is relatively straightforward to setup compared to various Macintosh options.  Unfortunately, it is also probably the least-supported version of Unix I have talked about and for many open-source science programs, you may have to manually build from source.  However, it has the advantage of giving you a complete, Windows-native Unix environment without any significant overhead like you have with Linux virtualization.  Follow the link in the subtitle for more information.

Conclusion

This blog entry is supposed to serve as a starting point for the different options you have available, focusing on the ones I have explored and found preferable.  Feel free to leave your opinions in the comments section. 

No comments:

Post a Comment