Tuesday, June 25, 2013

On the Automation of Data Analysis

I spent the day analyzing data today. I rigged up some Excel spreadsheets to do what I wanted them to do, but it wasn't pretty. I swear by Darwin's ample beard that once I am done with the paper I am trying desperately to get out quickly, and will thus have some time, I will write some Python code to automate the whole process I just did and spit me out my results.

Darwin's Ample Beard. I wonder what he would've though about a wonderful math-doing machine that can keep track of and analyze all the morphometric data about finches you can throw at it.

I know enough about Python (thanks to the Software Carpentry course I took) that I can envision exactly what I want my code to do. (If a cell in this column is blank, fill it in with the contents of the cell above it; Add a new column with labels depending on the value of another column; Delete all rows with a number less than this value in this column; etc.) It is just a matter of sitting down and learning the syntax and functions, which is an investment that will definitely pay off over time, but requires a lump cost up front. (Relevent XKCD) This particular data flow is something I can see the lab I am in spending an hour on every month for at least the next 5 years, which according to the XKCD chart means I can spend up to two days writing code to automate it before it becomes no longer worth it. Challenge accepted! As soon as I finish this paper, I promise...

1 comment:

Please keep comments respectful-- I do not currently moderate comments.