Installing scientific Python with NumPy, SciPy, matplotlib, PyMC, and Basemap/geos on Mac OS X 10.9 Mavericks, 10.10 Yosemite or 10.11 El Capitan

I love OS X for day-to-day work, especially compared to its main alternative, Windows. But Apple neglects data monkeys who install tools under the hood. For my research, I use Python, NumPy, SciPy, matplotlib, PyMC, and Basemap/geos, among other modules. These tools used to break with each new version of OS X, although as OS X has matured, the disruptions with each new release have become less.

I’ve tested these instruction on OS X 10.8 Mountain Lion, 10.9 Mavericks, 10.10 Yosemite, and 10.11 El Capitan when they were reasonably current (I usually stay about one OS X release behind). Over the years these instructions have evolved, and Homebrew formulae have been added and improved. I haven’t been going back and re-testing these instructions on older OS X releases, so your mileage might vary if you do this.

Consider starting with a clean install of OS X.

The dirty upgrade works great if you’re an everyday user who stays on the UI level, but in the distant past it made a terrific mess of my tools installed under the hood. I’ve ended up with multiple MySQL servers and multiple apache servers. But, as OS X has become more mature, I’m noticing less of a need to do this.

If you decide to take my advice and do a clean installation, first make sure that you have a backup. Time Machine works spectacularly well for this purpose, or you can install onto a new hard drive or carbon copy your old drive. Erase your working hard drive, install OS X, then have Migration Assistant import user accounts and applications from the backup (but leave the Other Files & Folders box unchecked). Migration Assistant takes care of restoring ordinary documents, which Apple does well, yet it provides a relatively blank slate under the hood to freshly install tools.

If you decide that you don’t want to start with a clean install of OS X, then go through these instructions anyway. For each package, use brew to uninstall, and then reinstall each package in sequence. Use my directions to keep you on track as you update each package individually. Don’t forget to also test each package after each reinstallation, as I have demonstrated in my instructions, so that if there are problems, you know exactly what the problem is.

Download Xcode from the App store and install the command line tools.

Xcode is a dependency for Homebrew, which uses Xcode’s gcc compiler to compile everything from source. Once you have Xcode installed, you need to install the command line tools for the gcc compiler to work. The quickest way to do this is from Terminal:

$ xcode-select --install

Check the systemwide PATH variable.

Open a Terminal window, and type:

$ cat /etc/paths

Make sure that /usr/local/bin occurs before /usr/bin. If they don’t, then you need to change this order. Edit /etc/paths using vi or your favorite text editor. I love and use TextWrangler. Close your Terminal window and open a new Terminal window for this change to take effect.

Obtain Homebrew. 

Homebrew is a great package manager for OS X that installs everything in /usr/local/bin and does not require sudo.It then creates symlinks to the expected locations so that the various tools can find one another. Because the packages are centralized in the Cellar, they are easily updated and/or removed.

$ ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

Fix any problems that Homebrew detects.

$ brew update
$ brew doctor

Follow brew doctor‘s instructions. brew doctor usually complains about Xcode. If I’m guessing the error correctly, here is the solution that brew doctor will suggest (assuming OS X 10.10 – note the version since this affects what you will type into Terminal):

$ cd /Applications/Xcode.app/Contents/Developer/Toolchains/
$ sudo ln -s XcodeDefault.xctoolchain OSX10.10.xctoolchain 
# note the version (10.10) and modify accordingly.

Run brew doctor again. Continue to follow brew doctor‘s instructions until it tells you that “Your system is ready to brew.”

Install Homebrew’s gcc.

This step, overlooked in the other blogs I’ve seen, has been required in the past for successfully installing SciPy and PyMC (newer Homebrew formulae may have fixed this problem). Without this, installation has failed with a vague gfortran error.

$ brew install gcc

Install Homebrew’s Python.

$ brew install python

Update OS X’s Python symlink to point to Homebrew’s Python.

This symlink controls the version of Python that runs when a user types python in Terminal. These directions should have appeared at the end of your brew installation, but in case you missed them, here they are (These instructions assume Python 2.7.6 – note the version since this affects what you will type into Terminal):

$ cd /System/Library/Frameworks/Python.framework/Versions
$ sudo rm Current
$ ln -s /usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/Current
# note the version (2.7.6) and modify accordingly.

Now, we will verify homebrew’s Python installation. If there’s a problem, we need to fix it now before proceeding with installations.

$ which python
# should see /usr/local/bin/python

Don’t touch or relink OS X’s system Python. Change your hashbang habits.

This is another place where this blog differs from other blogs.

OS X exasperatingly contains yet another copy of Python, called system Python, located at /usr/bin/python. The standard hashbangs !#/usr/bin/python or !#/usr/bin/env python will call Homebrew’s Python when the script is run by the local user, but will call system Python if the script is called by a launchctl daemon/agent. Other blogs have recommended deleting system Python and symlinking the deleted file to homebrew’s Python installation, but this solution creates serious problems with OS X Server, which depends on the original system Python installation being left intact.

Therefore, just change your habits to always use this hashbang: #!/usr/local/bin/python and all of these problems go away, no matter where or who your script is executed by. Easy-peasy.

Install NumPy, SciPy, and matplotlib.

First try each installation using homebrew. In the past, Homebrew didn’t yet have formulae for some of these, so I used pip whenever a formula was not available.

$ brew tap homebrew/python
$ brew install numpy
$ brew install scipy # this module used to require homebrew's gcc to succeed.
$ brew install matplotlib
# let's test our installations
$ python
>>> import numpy
>>> import scipy
>>> import matplotlib
>>> exit()

Install Basemap.

Basemap depends on geos, which needs to be installed first.

$ brew install geos

Download the latest non-Windows Basemap tarball. Unpackage it by double clicking on the compressed file on your desktop (which is easier) or use tar -xzf. Then, cd into the directory.

$ python setup.py install
$ cd examples
$ python simpletest.py 
# this tests your installation

Install PyMC.

This module is for Markov chain Monte Carlo (MCMC) simulations. If you don’t use MCMC then you should skip this. PyMC won’t install without a successful SciPy installation, which, in turn, has depended upon Homebrew’s gcc compiler.

$ git clone https://github.com/pymc-devs/pymc.git
$ cd pymc
$ python setup.py build
$ python setup.py install
$ python # let's test our installation
>>> import pymc
# if you get a warning about statsmodels and/or patsy you can probably ignore it. 
# I did, and my simulations have worked fine.
# (I hope!)
>>> exit()
$ rm -R pymc

You’re done! Happy programming!

I didn’t come up with all of this on my own. I acknowledge:

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s