Statistically Significant

This week was a literary one. At work, I spent quite a bit of time reading statistical genetics journal articles. I’m in the process of writing up some results to be submitted for publication, and I’m in “literature review” mode. Unfortunately for me, my relative lack of knowledge about the field of statistical genetics (without any formal coursework/degree and one year of doing something closer to statistical programming than statistical genetics) means that everything is new and shiny and requires at least a few of readings to really become internalized. The rate of progress is frustrating, but I do enjoy learning and it’s giving me a lot to think about.

And in my spare time, I’ve been reading a few good books. Still in the realm of statistics are Bayesian Data Analysis (BDA) and Statistics in a Nutshell (SiaN). Though I never took a class on Bayesian statistics, I purchased BDA for a PhD-level biostatistics class that I took during my senior year. I didn’t end up using it much, but I’ve always been intrigued by the Bayesian framework (and I enjoy reading Professor Gelman’s blog), so I’m spending some time every day outlining it and doing the examples/problems. It seems like Bayesian methods can be extremely powerful alternatives to their frequentist complements and are definitely useful to have in your statistical bag of tricks. The latter is just a review book that I picked up in an attempt to get my brain thinking about the big topics before I head out to Berkeley. The Lady Tasting Tea, which I was a couple of months ago, was an amazing overview of the titans and history of the field of statistics, and I’m hoping that SiaN will provide a slightly more technical overview of things soon to come.

And unrelated to statistics, though not unrelated to my life ambitions, is The Fountainhead. That’s right: I’ve decided to become an architect and dye my hair orange. No, not quite. But after being introduced to Howard Roark and company a couple of years ago, I try to reread this book every 6-12 months. I’m not sure that I agree with all of Ayn Rand’s philosophies, but whenever I put the book down, I feel empowered to be the best I can possibly be, to throw myself into my work, and to push forward while trying to please nobody but myself. There’s something to be said for that. Love it or hate it, it’s definitely worth a read.

This past week, I learned of a cool little resource for people learning a new programming language: Rosetta Code. It’s a wiki-style site that contains user-submitted solutions to many common programming tasks in a variety of languages. R is definitely my language of choice these days, and while many R solutions had already been provided, I filled in a few blanks this week: Amb, Luhn, pangram, binomial, permutation, Hailstone, reversalGame. (Yes, I did beat the reversalGame in 4 moves. No, I can’t do that consistently.)

A few thoughts I’ve been having about programming lately:

- Writing code, especially solving little problems like these, is a lot of fun!

- I once watched an MIT Open Courseware data structures class introductory lecture, and one of the first things that the professor was, “To get good at programming, you can either do it for 10 years or you can take a data structure class and then do it for 1 year.” For about January-March, I was working through an introductory Python book, which exposed me to some new data structures and programming ideas. Applying some of those ideas to my R programming made me a much better programmer almost overnight. It was great.

- J appears to be an insanely powerful language with an equally insane learning curve. Consider, for example, the Hailstone Sequence task on Rosetta Code. My solution in R is pretty straightforward… even if you don’t know R, it’s pretty easy to just read along and assume that the syntax is correct. Now consider the J function that does the same thing:

hailseq=: -:`(1 3&p.)@.(2&|) ^:(1 ~: ]) ^:a:"0

Crazy. I saw this when I was doing Project Euler problems, too. Note to self: learn J. But only after improving your Python skills, learning C, and beefing up your scripting skills.

- Most of what I’ve been doing for this past year has been programming, not statistics. But programming and statistics are different activities. Therefore, I expect a pretty serious transitionary period away from writing code and towards doing theoretical statistics problem sets. (I don’t expect to write very much code at all next year, although my R skills will definitely come in handy when I need to.)

On a related note, I subscribe to Scott Adams’ (author of Dilbert) blog, and he made an interesting observation that really rang true with me in a recent post:

During one period of my life I wrote a number of computer programs that involved intense manipulation of objects in my mind, for hours each day. I discovered that it was difficult to be social at night when my mind had been manipulating object during the day. It felt as if I were deep inside a cave and yelling to the people who stood at the cave opening. It seemed as if the practice of programming interfered with, or exhausted, the part of my brain that handles social skills.

That metaphor, of standing deep inside of a cave and yelling to the people standing at the entrance, so accurately captures the way that I feel sometimes after a long day of being totally in the programming zone. Any other programmers experience a similar phenomenon?