NOTICED on The Carpentries

Software Carpentry on TalkPython

Thu, 05 Jan 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

On January 3rd, 2017 TalkPython published a podcast interview with our Executive Director, Jonah Duckles, that was recorded on December 6th, 2016. If you have friends who are interested to know about Software Carpentry and are fan of listen podcast (or not) the record is a great way for them to discovery what we are and how we operate.

In the past, our staffs were interviewed in other podcasts by Open Science Radio and Podcast.init that podcast fans should check out.

A Reproducibility Reading List

Tue, 01 Nov 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Prof. Lorena Barba has just posted a reading list for reproducible research that includes ten key papers:

Schwab, M., Karrenbach, N., Claerbout, J. (2000) Making scientific computations reproducible, Comp. Sci. Eng. 2(6):61–67, doi: 10.1109/5992.881708
Donoho, D. et al. (2009), Reproducible research in computational harmonic analysis, Comp. Sci. Eng. 11(1):8–18, doi: 10.1109/MCSE.2009.15
Reproducible Research, by the Yale Law School Roundtable on Data and Code Sharing, Comp. Sci. Eng. 12(5): 8–13 (Sept.-Oct. 2010), doi:10.1109/mcse.2010.113
Peng, R. D. (2011), Reproducible research in computational science, Science 334(6060): 1226–1227, doi: 10.1126/science.1213847
Diethelm, Kai (2012) The limits of reproducibility in numerical simulation, Comp. Sci. Eng. 14(1): 64–72, doi: 10.1109/MCSE.2011.21
Setting the default to reproducible (2013), ICERM report of the Workshop on Reproducibility in Computational and Experimental Mathematics (Providence, Dec. 10–14, 2012), Stodden et al. (eds.), https://icerm.brown.edu/tw12-5-rcem/ // report PDF
Sandve, G. K. et al. (2013), Ten simple rules for reproducible computational research, PLOS Comp. Bio. (editorial), Vol. 9(10):1–4, doi: 10.1371/journal.pcbi.1003285
Leek, J. and Peng, R (2015), Opinion: Reproducible research can still be wrong: Adopting a prevention approach, PNAS 112(6):1645–1646, doi: 10.1073/pnas.1421412111
M. Liberman, “Replicability vs. reproducibility — or is it the other way around?,” Oct. 2015, http://languagelog.ldc.upenn.edu/nll/?p=21956
Goodman, S. N., Fanelli, D., & Ioannidis, J. P. (2016). What does research reproducibility mean? Science Translational Medicine 8(341), 341ps12–341ps12, doi: 10.1126/scitranslmed.aaf5027

The papers themselves are great, but what really adds value is the way they're ordered, analyzed, and connected. If you're trying to make sense of all this, or trying to help others do so, it's a great place to start.

Tracy Teal on Research in Action

Tue, 01 Nov 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The latest podcast from Research in Action features Dr. Tracy Teal, Executive Director of Data Carpentry. In 33 minutes (plus a couple of bonus clips), Tracy talks about the mission of Data Carpentry, how it came to be, and how people can get involved as learners, instructors, and lesson developers. It’s a great introduction for newcomers, and has a lot of tidbits that long-time participants will find fun and interesting as well.

Seymour Papert 1928-2016

Tue, 02 Aug 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Seymour Papert passed away on Sunday at the age of 88. I never had the privilege of meeting him, but Software Carpentry would probably never have existed if I hadn’t stumbled across his inspirational book Mindstorms. You can read more about his life and work here; when you’re done, please go and help someone learn something—I think he’d have liked that.

Announcing the Open Science Grid User School 2016

Mon, 28 Mar 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

If you could access hundreds or thousands of computers for your scholarly work, what could you do?

How could it transform your work?

What discoveries might you make?

We are seeking applicants for the 2016 Open Science Grid (OSG) User School, which takes place 25–29 July at the beautiful University of Wisconsin in Madison. Participants will learn to use high throughput computing (HTC) to harness vast amounts of computing power for research, applicable to nearly any field of study (e.g., physics, chemistry, engineering, life sciences, earth sciences, agricultural and animal sciences, economics, social sciences, medicine, and more).

Building Software, Building Community: Lessons from the rOpenSci Project

Wed, 17 Feb 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

[Carl Boettiger]({{site.baseurl}}/team/#boettiger_carl), [Scott Chamberlain]({{site.baseurl}}/team/#chamberlain_s), [Ted Hart]({{site.baseurl}}/team/#hart_t), and [Karthik Ram]({{site.baseurl}}/team/#ram_k) have just published a paper titled "[Building Software, Building Community: Lessons from the rOpenSci Project](http://openresearchsoftware.metajnl.com/articles/10.5334/jors.bu/)", in which they describe what they've learned by growing the [rOpenSci](http://ropensci.org/) project There are a lot of great ideas here that other groups could borrow, and of course you can keep up with their news by [following their blog](http://ropensci.org/blog/).

NGS Summer 2016: Analyzing Next-Generation Sequencing Data

Sun, 14 Feb 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A two-week residential course on next-generation sequencing is being offered at the Kellogg Biological Station at Michigan State University on August 8-19, 2016. The course's directors are Prof. Matt MacManes (U. New Hampshire) and Prof. Meg Staton (U. Tennessee, Knoxville), and instructors will include Prof. Ian Dworkin (McMaster U.), Prof. Torsten Seemann (U. Melbourne), Shaun Jackman (PhD candidate, UBC) and others. More information, or to register, please see [http://bioinformatics.msu.edu/ngs-summer-course-2016](http://bioinformatics.msu.edu/ngs-summer-course-2016).

Note: if you are running a course that might be of interest to our community, please let us know.

Open Science Radio Podcast

Tue, 09 Feb 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Matthias Fromm and Konrad Förstner recently interviewed Greg Wilson for their Open Science Radio podcast. You can listen to it [here](http://www.openscienceradio.de/2016/02/09/osr038-software-carpentry-with-greg-wilson-en/), or explore their other recordings.

Two Pages of Evidence

Tue, 02 Feb 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Andreas Stefik and his colleagues have written [a two-page summary](http://quorumlanguage.com/evidence/evidence.pdf) of what we actually know about usability and programming language design. There aren't nearly as many results yet as we want or need, but what the people working in the field have shown is that we actually can answer these questions scientifically, and that answering them correctly actually does have an impact.

Our Introduction to Git Has Been Published

Thu, 21 Jan 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Our short paper "A Quick Introduction to Version Control with Git and GitHub" has been [published in PLOS Computational Biology](http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1004668). It is freely reusable under a Creative Commons Attribution license, and we hope you and your colleagues find it useful.

A Data Programming CS1 Course at the University of Washington

Sun, 03 Jan 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

People who are interested in integrating Data Carpentry and Software Carpentry ideas into the undergraduate curriculum may enjoy reading Anderson et al's paper "A Data Programming CS1 Course" (*SIGCSE'15*, http://dx.doi.org/10.1145/2676723.2677309). From the abstract:

This paper reports on our experience teaching introductory programming by means of real-world data analysis. We have found that students can be motivated to learn programming and computer science concepts in order to analyze DNA, predict the outcome of elections, detect fraudulent data, suggest friends in a social network, determine the authorship of documents, and more. The approach is more than just a collection of “nifty assignments”; rather, it affects the choice of topics and pedagogy.

Lessons as Lab Protocols

Sun, 03 Jan 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A roundabout chain of references led me to Abbott et al’s “Programs for People: What We Can Learn from Lab Protocols” (presented at VL/HCC 2015) which looks at how lab protocols are similar to and different from programs. On the one hand, a lab protocol describes the steps to be followed to (for example) prepare a particular kind of sample for analysis. On the other hand, “human plans are more like descriptions or predictions than prescriptions of what actions a person will take given the most likely sequence of future actions; they are resources for anticipating likely future events. In programming language terms, perhaps, they are more declarative than imperative in that people draw on knowledge of the whole plan rather than blindly following each step.”

Introducing the Research Bazaar

Wed, 09 Dec 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

One of the lessons Software Carpentry has learned over the years is that there's no substitute for a face-to-face learning experience. The unscripted interactions that occur between instructors, helpers and participants at our workshops are as important, if not more important, than the formal teaching syllabus itself. Of course, we aren't the only educator in the digital research space to which this lesson applies. The social aspect of in-person learning experiences is equally important to the various organisations, eResearch departments and libraries around the world tasked with "up-skilling" the next generation of digital researchers.

The Morea Framework

Fri, 20 Nov 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I first met Philip Johnson, a professor of Computer Science at the University of Hawaii, through shared interests in empirical software engineering research and Google Summer of Code. He has recently been developed the Morea Framework for creating structured course websites using GitHub and Jekyll. "Morea" stands for "Modules, Outcomes, Readings, Experiences, and Assessments", which are the five main elements the framework supports. As you can see from the project gallery, it's much more structured than our lessons. It also requires more tooling—Morea Framework sites are built using custom Jekyll plugins, and the source relies much more heavily on include files than our template—and it's geared very strongly toward traditional semester-long courses.

Test-Driven Data Analysis

Thu, 19 Nov 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

My former colleague Nick Radcliffe has started posting a series of article on test-driven data analysis, in which he explores systematic ways of checking whether one-off data analyses are correct. It'll be really interesting to see how the series unfolds, and comments on the posts would be very welcome.

Ten Simple Rules for Digital Data Storage

Fri, 23 Oct 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Edmund Hart, Pauline Barmby, David LeBauer, François Michonneau, Sarah Mount, Timothée Poisot, Kara Woo, Naupaka Zimmerman, and Jeff Hollister have just posted a pre-print on PeerJ titled Ten Simple Rules for Digital Data Storage. The paper is a distributed collaborative effort spawned from a thread on the Software Carpentry instructors mailing list and further carried out on GitHub. There are a lot of good ideas in it, many of which we should fold back into our lessons, and we hope it will spark more collaborations in our community.

Software Development Practices in Academia

Thu, 18 Jun 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Derek Groen, Xiaohu Guo, James Grogan, Ulf Schiller, and James Osborne have just submitted a paper to arXiv.org titled "Software development practices in academia: a case study comparison". From the abstract:

Academic software development practices often differ from those of commercial development settings, yet only limited research has been conducted on assessing software development practises in academia. Here we present a case study of software development practices in four open-source scientific codes over a period of nine years, characterizing the evolution of their respective development teams, their scientific productivity, and the adoption (or discontinuation) of specific software engineering practises as the team size changes. We show that the transient nature of the development team results in the adoption of different development strategies. We relate measures of publication output to accumulated numbers of developers and find that for the projects considered the time-scale for returns on expended development effort is approximately three years. We discuss the implications of our findings for evaluating the performance of research software development, and in general any computationally oriented scientific project.

A Lesson on Reproducible Computational Analysis

Mon, 15 Jun 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Titus Brown has recorded a two-hour lesson on reproducible computational analysis and posted it to YouTube. Many thanks to the folks at ICER for making it available.

Routinely Unique

Sun, 14 Jun 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Back in April, Jeffrey Chang wrote wrote an article for Nature in which he pleaded for those who analyze bioinformatics data to be recognized as creative collaborators in need of career paths. In it, he observed:

To give greater support to researchers, our centre set out to develop a series of standardized services. We documented the projects that we took on over 18 months. Forty-six of them required 151 data-analysis tasks. No project was identical, and we were surprised at how common one-off requests were. There were a few routine procedures that many people wanted, such as finding genes expressed in a disease. But 79% of techniques applied to fewer than 20% of the projects. In other words, most researchers came to the bioinformatics core seeking customized analysis, not a standardized package.

Coding for Librarians

Fri, 22 May 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Andromeda Yelton (who has featured in this blog before) has written a set of six articles for Library Technology Reports titled Coding for Librarians: Learning by Example. From the introduction:

[This] draws from more than fifty interviews with librarians who have written code in the course of their work. Its goal is to help novice and intermediate programmers understand how programs work, how they can be useful in libraries, and how to learn more.

Three chapters discuss use cases for code in libraries. These include data import, export, and cleanup; expanded reporting capability; and patron-facing services such as improvements to catalog and LibGuide usability. Most of the programs discussed are short&mash;under a hundred lines—so that implementing or modifying them is within the reach of relatively novice programmers. Where possible, links to the code itself are provided. Several scripts are explained in depth.

Additional chapters focus on nontechnical aspects of library code. One chapter outlines political situations that have been faced by librarians who code and the solutions they have employed. Another chapter shares interviewees' advice on specific resources and strategies for learning to code.

Experiences with Geoscientists

Thu, 21 May 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Christian Jacobs, Gerard Gorman, and Lorraine Craig have written a paper titled "Experiences with efficient methodologies for teaching computer programming to geoscientists" that describes how their intro to computing course has changed over the last few years. It includes discussion of ideas they've borrowed from Software Carpentry, and some data on hwo students have responded. It's a good read, and we'd welcome more experience reports of this kind.

Achintya Rao's PhD Starter Kit

Sat, 02 May 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Achintya Rao started a PhD last January, and in response to a request for advice from a friend, wrote a PhD Starter Kit that lists useful tools and practices. Most of them involve software of one kind or another, and it's interesting to compare the list to what we teach: if nothing else, it tells me that we really do need to figure out what to teach people about publishing science in the 21st Century.

Research in the Cloud in London

Sat, 02 May 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Mark Stillwell and others are running a Research in the Cloud workshop in Feltham (near London) on July 15-17. Along with the standard Software Carpentry curriculum, they'll teach modules on cloud computing, including deployment, configuration, and management of virtual machines. Please see his blog post for details, or the workshop web site to register.

Van Lindberg's Keynote: Say Thanks

Sat, 25 Apr 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Van Lindberg, the chair of the Python Software Foundation, gave a really insightful keynote at PyCon 2015 last week. In a nutshell:

The PSF's greatest challenge is that it's short of time: it has one full-time and three part-time employees.
What can you do to help?
- Say thanks to people for what they're doing.
- Raise up and mentor others.
- Persevere when trying something new. (Bonus points if you help someone out who doesn't look like you.)
How can a new organization grow up to be the PSF?
- Don't rush it. (In particular, don't put too much process in place too soon.)
- Default to openness.
- Build a culture of service.

There's lots more, and you should watch the whole thing, but I think these are good guidelines for Software Carpentry (and most other things, too).

Invitation to Millions of Compute Hours: Announcing the Open Science Grid User School

Fri, 17 Apr 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

If you could access thousands or even millions of hours of computing, how would it transform your research? What discoveries might you make?

Each year the NSF-funded Open Science Grid (OSG) selects a group of 25-30 students to attend the OSG User School, a week-long dive into high-throughput computing approaches, technologies, and skills, within a larger context of computational research design that students can take into their future careers as researchers. Students across the country from nearly any research discipline are invited to apply, and selected applicants will obtain direct access to the OSG beyond the duration of the school.

The Future Then and Now

Mon, 13 Apr 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Jon Udell's Internet Groupware for Scientific Collaboration taught me how to think about the web. He started work on an update a couple of months ago, and it has now been published by PLOS. What strikes me upon re-reading the first is how far we've come; upon reading the second, is how far we still have to go to make all of this normal. As William Gibson said, "The future is already here—it's just not very evenly distributed."

The Most Viewed PLOS Biology Paper of 2014

Tue, 03 Mar 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We were very pleased to learn that the most viewed article in PLOS Biology in 2014 was "Best Practices for Scientific Computing", written by Dhavide Aruliah, C. Titus Brown, Neil Chue Hong, Matt Davis, Tommy Guy, Steven Haddock, Katy Huff, Ian Mitchell, Mark Plumbley, Ben Waugh, Ethan White, Paul Wilson, and yours truly. According to Stavroula Kousta, the article has been viewed more than 66,000 times.

Scientific Coding vs. Software Engineering

Sun, 08 Feb 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Daisie Huang recently wrote a great article for the Software Sustainability Institute's blog titled "Scientific coding and software engineering: what's the difference?". As a professional programmer who has become a scientist, rather than a scientist who's learned how to program, she has a fresh take on the differences between what those two groups do. Comments on her post would be very welcome.

Software Sustainability Institute's Collaborations Workshop 2015

Tue, 03 Feb 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Software Sustainability Institute's Collaborations Workshop 2015 (CW15), which will be held on March 25-27, 2015 in Oxford, UK, focusses on software, best practice and the social side of working past the boundaries of traditional disciplines and roles to accelerate research outcomes—or put differently, interdisciplinarity done right!

What will you learn by attending:

The social and technical sides of interdisciplinary working.
Examples of best practice of when it's been effective - and horror stories of when it has not.
Examples of techniques and technologies that could be applied from one discipline to another - discipline hopping.

Practical Computing for Biologists (and Other Scientists)

Wed, 14 Jan 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are big fans of Steve Haddock and Casey Dunn's book Practical Computing for Biologists, which covers everything we do and more (and is suitable for all kinds of scientists, not just biologists). We were therefore very pleased to learn that they are running a course at Friday Harbor Laboratories this summer. The outline is below, and applications are due on February 1st.

Results of Software Sustainability Institute Survey

Sat, 13 Dec 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Software Sustainability Institute's recent survey of researchers at top UK universities is out. Headlines figure are:

92% of academics use research software
69% say that their research would not be practical without it
56% develop their own software (worryingly, 21% have no training in software development)
70% of male researchers develop their own software, and only 30% of female researchers do

For the full story, see this post on their blog.

International Workshop on Software Engineering for High Performance Computing in Science

Thu, 04 Dec 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The 2015 International Workshop on Software Engineering for High Performance Computing in Science is being held in conjunction with the International Conference on Software Engineering in Florence, Italy in May 2015. Topics to be covered include:

You Should Read Juha Sorva's Thesis

Thu, 06 Nov 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

If you really want to dig deeper into educational research and how it applies to teaching programming, you should grab a copy of Juha Sorva's PhD thesis. The UUhistle system he built is interesting, and the research he did with it is thought-provoking, but what's really great is the summary of educational research in the first third of the thesis. I've extracted that part with his permission, and I can't recommend it highly enough.

A Reproducible Science Hackathon

Sat, 04 Oct 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

NESCent is organizing a "Reproducible Science Hackathon" where participants will develop material and tools for teaching and facilitating a broad adoption of reproducible science. The deadline for application is next Friday (Oct 10th); see the full announcement for details.

UCOSP as a Model

Sun, 28 Sep 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Software Carpentry's two-day workshops are just one of many ways to teach people practical skills. Term-long group projects are another model that I'm very fond of, and earlier this year, the four people who've been running the UCOSP program in Canada wrote a paper about what they've learned. Some of the lessons rhyme with what we've learned from Software Carpentry, but other insights are new. If you know of papers describing lessons learned from other innovative teaching projects, pointers in the comments section would be very welcome.

Nature Interview with Kaitlin Thaney

Wed, 03 Sep 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Nature has just published a short interview about Software Carpentry with Kaitlin Thaney, the director of the Mozilla Science Lab, and two bootcamp attendees (Harvard's Rebecca Perry and USNIDA's Brian Sadacca). As always, version control seems to have been one of their big take-aways, but as Kaitlin says, two days isn't enough for people to become proficient: our bootcamps really are just the first step.

Open Source Comes to Campus

Wed, 03 Sep 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Open Source Comes to Campus helps the next generation of open source contributors get started by teaching one-day workshops aimed at undergraduate and graduate students from all disciplines. This workshop introduces students to the tools and culture of open source development. We familiarize students with tools such as IRC, issue trackers and version control; talk plainly about aspects of open source that might be confusing or intimidating; introduce them to open source professionals who can talk about career and educational opportunities; and help them make their first contributions to open source projects.

OSCTC will be hosting events this fall at over a dozen schools, including the City College of San Francisco, Bucknell University, the University of Victoria, Hartnell College, the University of Washington, and the University of Connecticut. We're always looking for volunteers to mentor at our events, and if you want to bring an event to your school, please get in touch.

The Fifth ANGUS Course

Thu, 21 Aug 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Titus Brown recently blogged a summary of the fifth run of the Analyzing Next Generation Sequencing (ANGUS) course at Michigan State. It includes some interesting observations on what's working and what needs to be improved, and some thoughts on assessment—he'd welcome feedback.

Collected Links

Thu, 05 Jun 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A lot of people have been writing about Software Carpentry on other blogs of late. Here, in no particular order, are a few of their posts:

Damien Irving: A vision for data analysis in the weather and climate sciences
Deb Paul: Tales from A Data Carpentry Workshop: In Demand
Tracy Teal: Inaugural Data Carpentry Workshop
Oxana Sachenkova: Software Carpentry at SciLifeLab, Sweden
Rob Davey: From the workshop to the workstation: Software Carpentry training from a bioinformatics perspective
Stephen Turner: Collaborative lesson development with GitHub
Karin Lagesen: The one where I went to Sweden
Damien Irving (again): Software Carpentry for Biomedical Imaging

Workshops at SESYNC

Sat, 19 Apr 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Applications are now open for the 2014 Computational Summer Institute at the National Socio-Environment Synthesis Center (SESYNC). Small teams of researchers are invited to apply for spaces at the one-week institute which will be held July 7-11 in Annapolis, Maryland. The workshop will offer participants hands-on training in managing the lifecycle of their data and code with a focus on using open source tools, including R. Topics will include

Data Science Workshops in Seattle

Tue, 18 Mar 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Via Sumana Harihareswara: the Community Data Science Workshops are a series of project-based workshops being held at the University of Washington for anyone interested in learning how to use programming and data science tools to ask and answer questions about online communities like Wikipedia, Twitter, free and open source software, and civic media.

The workshops are for people with no previous programming experience. The goal is to bring together both researchers and academics as well as participants and leaders in online communities. The workshops will all be free of charge. Participants from outside UW are encouraged to apply.

There will be three workshops held from 9am-4pm on three Saturdays in April and May. Each session will involve a period for lecture and technical demonstrations in the morning. This will be followed by a lunch graciously provided by the eSciences Institute at UW. The rest of the day will be followed by group work on programming and data science projects supported by more experienced mentors.

For more information, see the full announcement.

A Letter from John von Neumann

Fri, 14 Mar 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Plans to standardize and publish codes of various groups have been made in the past, and they have not been very successful so far. The difficulty is that most people who have been active in this field seem to believe that it is easier to write a new code than to understand an old one. This is probably exaggerated, but it is certainly true that the process of understanding a code practically involves redoing it de novo. The situation is not very unlike the one which existed in formal logics over a long initial period, where every new author invented a new symbolism. It took several decades until a few of these found wider acceptance, at least within limited groups. In the case of computing machine codes, the situation is even more difficult, since all logics refer, at least ideally, to the same substratum, whereas the machine codes frequently refer to physically different machines.

I think, nevertheless, that if a competent mathematician...is interested in working on this problem, he ought to be encouraged. The task may turn out to be a thankless one, but it is always possible that a competent and energetic man will in the end come up with something useful. My personal doubts are limited to the near future, and even if it will take a non-trivial number of years to produce something, there is no harm in starting early.

— John von Neumann, in a letter to Marston Morse, April 23, 1952

Everything Old is New Again

Fri, 14 Mar 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Yesterday, the New York Times R&D Lab announced streamtools, a web-based graphical tool for working with streams of data. It lets users create dataflow systems to re-mix and process the streams they believe most data will consist of in the near future.

Streamtools looks nice, but I do have some questions:

Andromeda's Advice

Thu, 19 Dec 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We're very pleased to announce that Andromeda Yelton will be coming to Toronto in mid-January to help teach a bootcamp for librarians. Her advice on how to do this is online, along with her reflections on what she's learned herself. There's lots of good stuff in both, and we're looking forward to lots of new ideas.

There Ought to Be a Badge

Tue, 10 Dec 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I feel like there should be a Software Carpentry badge for drawing one of these for someone for the first time.
— Amy Brown on Twitter

Women in Tech Workshop at PyData NYC

Wed, 13 Nov 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Software Carpentry has always focused on scientists at the graduate level and above, but getting people into the pipeline is even more important as getting them through it. Last week, thanks to backing from NumFOCUS and others, a group of volunteers helped do just that by running Python workshops for high school girls in conjunction with PyData NYC. Judging from Julia Evans' writeup, it was a great success. We look forward to seeing some of their alumni in our classes some day...

Python for Biologists

Mon, 14 Oct 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Programmers, at least as found in scientific research institutions, generally fall into two groups. There are people with a computer science background, for whom programming is a way of Thinking About The World, and people with a scientific background, for whom programming is a way of Getting Things Done.

The Future: Today

Fri, 27 Sep 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Two papers appeared on my radar this week that give a taste of the kind of science we want everyone to be able to do in five years' time. The first, by Aide et al, is "Real-time bioacoustics monitoring and automated species identification"; the second, by Omberg et al, is "Enabling transparent and collaborative computational analysis of 12 tumor types within The Cancer Genome Atlas". They're both excellent pieces of 21st Century science, and they caught my eye because:

They're open access: I was able to read them on the off chance that they'd be interesting.
They show how a few scientists with serious software skills can accelerate the research of hundreds of others, provided those others have a few skills themselves.

Code and Data for the Social Sciences

Thu, 26 Sep 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Matthew Gentzkow and Jesse Shapiro have written an excellent guide "Code and Data in the Social Sciences". It's short (only 38 pages), very readable, and full of practical advice for scientists of all stripes:

Jonathan Dursi Joins Compute Canada

Fri, 30 Aug 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Jonathan Dursi, a long-time supporter of Software Carpentry and one of our first instructors, has been named Interim Chief Technology Officer for Compute Canada. Congratulations!

Video Interviews from SESYNC Workshop

Thu, 22 Aug 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The folks at SESYNC interviewed Titus Brown, Kaitlin Thaney, and Greg Wilson while they were in town for a workshop a few weeks ago, and the video is now up on their web site. It's a nice, short summary of the problem we're trying to solve and the approaches we're taking.

Miscellaneous Videos

Sat, 20 Jul 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

People have been talking about Software Carpentry in a bunch of different venues recently. Katy Huff and Matt Davis did a tutorial at SciPy'13 that was captured in these three videos; Damien Irving gave a talk at PyCon Australia; Luis Figueira talked about learning how to learn; and Ariel Rokem and Shreyas Cholia talked about their experiences teaching in New Zealand in a recent Mozilla community call. (You need to enter a name—any will do—on the launch page for the recording, and then scroll ahead manually to 00:04:54 to hear them because the Flash player won't let me hyperlink directly to a particular time mark. The irony of both hasn't escaped us...)

The Fourteenth Anniversary

Fri, 19 Jul 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Fourteen years ago this October, I had the good fortune to attend a one-day workshop titled Open Source/Open Science at Brookhaven National Laboratory. A lot of the tools we work with have changed since then, but the central message remains the same: when it comes to science, the opposite of "open" isn't "closed" — it's "broken".

Workshop for e-Infrastructure Trainers

Sat, 06 Jul 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Simon Hettrick, of the Software Sustainability Institute, recently posted this:

If you have an interest in training researchers how to use computing, software or data, you should attend the workshop for e-Infrastructure trainers on 14 August 2013 at the newly opened Hartree Centre.

For more information, please see the full announcement.

Announcing Hack4ac

Thu, 16 May 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Hack4ac is a one-day hackathon in London, England, on July 6. Its goals are:

Demonstrate the value of the CC-BY licence within academia. We are interested in supporting innovations around and on top of the literature.
Reach out to academics who are keen to learn or improve their programming skills to better their research. We're especially interested in academics who have never coded before.

It looks like fun—if you're in the area and interested in getting involved, they'd be happy to have you there.

A Mention in Science Careers

Tue, 14 May 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Vijee Venkatraman has written a good article for Science Careers titled "When All Science Becomes Data Science", which mentions Software Carpentry.

Make It Easier to (Re)use Your Data

Fri, 03 May 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Software Carpentry has focused on computing for most of its 14 years (primarily because that's what I'm most familiar with) but it's increasingly clear that we need to tackle other parts of the research cycle. One is the new ideas clustered around publication, discovery and metrics, which I'll discuss in a future post. The other is data management; we only touch on the topic right now, but it's as important to most scientists as crunching numbers, and how best to do it is changing rapidly. Luckily, a few of our friends have written a guide for the perplexed:

A Short Report from Utah State

Sat, 02 Feb 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

(The first in an occasional series about other ways people are teaching computing to scientists.)

My "Advanced Programming and Database Management for Biologists" course at Utah State is based almost entirely on Software Carpentry (though the full curriculum is obviously not just what we teach in two days). It is a blended classroom: for about 2/3 of the semester the students view SWC material before class, at the beginning of class I provide 5-10 minutes of additional introduction to the material or answer questions the students had after viewing the videos, and then they spend the rest of class working on problems related to the material. They also do an independent project related to their research and the last 1/3 of the semester involves primarily me helping individual students work on their research projects (plus I present a set of more advanced or discipline relevant topics that they choose by voting on, e.g., the GPU peta cloud). It's been fairly popular for a rather nerdy graduate class and scores off the charts.

Next-Generation Sequencing Course 2013

Fri, 01 Feb 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Analyzing Next-Generation Sequencing Data

http://bioinformatics.msu.edu/ngs-summer-course-2013

June 10th–20th, 2013
Kellogg Biological Station, MSU
Course sponsor: NIH.

Instructors: Dr. C. Titus Brown, Dr. Ian Dworkin, and Dr. Istvan Albert.

Course Description

This intensive two week summer course will introduce attendees with a strong biology background to the practice of analyzing short-read sequencing data from Illumina and other next-gen platforms. The first week will introduce students to computational thinking and large-scale data analysis on UNIX platforms. The second week will focus on mapping, assembly, and analysis of short-read data for resequencing, ChIP-seq, and RNAseq.

PLoS Ad for Software Carpentry

Thu, 10 Jan 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The folks at PLoS Computational Biology kindly placed an ad for Software Carpentry on their home page today—thanks!

Why We Teach

Fri, 04 Jan 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Data Sharing and Management Snafu in 3 Short Acts: you have to laugh, because otherwise you'd cry...

Sample Data Management Plans

Fri, 21 Dec 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Neil Chue Hong recently pointed us at DataONE's Data Management Planning page, which has two really useful things:

a link to DMPTool, which will help you build a data management plan for your project; and
five sample data management plans, all written for the NSF, which people can use as reference models.

Data management is something we deliberately left out of "Best Practices" paper—it would have increased the paper's length by half again—but what we should put together is some sample code management plans. Any volunteers?

Lorena Barba's Reproducibility PI Manifesto

Sat, 15 Dec 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Prof. Lorena Barba, of Boston University, presented an excellent manifesto at the ICERM meeting on reproducible research:

I will teach my graduate students about reproducibility.
All our research code (and writing) is under version control.
We will always carry out verification and validation.
For main results in a paper, we will share data, plotting script & figure under CC-BY.
We will upload the preprint to arXiv at the time of submission of a paper.
We will release code at the time of submission of a paper.
We will add a "Reproducibility" declaration at the end of each paper.
I will keep an up-to-date web presence.

So say we all.

A Mostly Successful Decade

Wed, 14 Nov 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A video of Fernando Perez's keynote at PyCon Canada, titled "Science And Python: retrospective of a (mostly) successful decade", has been posted. If you'd like to see what the IPython Notebook can do, and how it fits into a broader picture, it's well worth watching.

{% assign video_title = "Science And Python: retrospective of a (mostly) successful decade" %} {% assign video_slug = "F4rFuIb1Ie4" %} {% assign video_time = "1:00:58" %} {% include youtube %}

Web 4 Science

Tue, 13 Nov 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Titus Brown (a long-time supporter of, and contributor to, this project) has a series of posts on his blog about using the web for science. As he's well ahead of the curve in doing this himself, I thought readers of this blog would be interested:

There's lots here to chew on—I'm sure he'd welcome comments from our readers.

Winter School on Reproducible Research

Sun, 04 Nov 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

It is a pleasure to announce that the 13th Geilo Winter School will take place January 20th-25th, 2013 at Dr. Holms Hotel, Geilo, Norway. As you can see from the attached flyer, the Winter School will give an introduction to reproducible science and modern techniques for scientific software development, a topic we hope many of you find interesting. The aim of the winter school is that participants will be able to apply the learned techniques to make their own research reproducible, and topics that will be covered include reproducible research, verification and validation, software testing, and continuous integration. More information is available on the winter school webpages, and participants are also encouraged to present their own research in a poster.

A List of Bioinformatics Courses

Tue, 30 Oct 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Titus Brown is compiling a list of bioinformatics courses. If you know one that should be added, he'd enjoy hearing from you.

Excel Isn't Intrinsically Evil

Mon, 22 Oct 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Excel and other spreadsheets aren't intrinsically evil, but like any power tool, they can easily take off fingers when used carelessly. From Neil Saunders' blog:

...a text download of 450K methylation from the Cancer Genome Atlas project reveals that Excel has had its evil way with the data at some point. Gene names such as MAR1, DEC1, OCT4 and SEPT9 are now reformatted as dates.

As he goes on to say, "Despair at the quality of public data, fears about reproducibility in science. Must be Monday."

Why We Teach Version Control

Fri, 12 Oct 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Convergent Evolution

Fri, 05 Oct 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Earlier this week, a Google search turned up a Software Carpentry workshop at the ICHEC in Dublin—which was surprising, because we didn't know they were running one. It turns out they'd reinvented the term independently, though their "carpentry" was a lot more advanced than ours—according to ICHEC's Simon Wong:

Half of the workshop was on software development concept and techniques but specifically aimed at deployment on the cloud, e.g. test-driven development, code management, SQL vs NoSQL databases. The rest of the workshop was more getting the attendees familiarised with various Amazon Web Services (e.g. Elastic MapReduce)

The Software Is Open (even if the interviews aren't)

Wed, 12 Sep 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

FLOSS for Science, a site devoted to free/libre/open source software for scientific computing, has just published two volumes of interviews with people in the community. The content doesn't appear to be open, but it's only $9.99 for the pair.

An Interview with Titus Brown

Mon, 27 Aug 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The folks at Simply Statistics interviewed Prof. Titus Brown earlier this month; there's lots of good stuff in the post for people interested in doing computational science (or any kind of science) better.

Two Posts on Scientific Workflows

Tue, 26 Jun 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Carly Strasser recently posted two articles on informal and formal scientific workflows, both of which are likely to be interesting to readers of this blog.

Pretty Well Sums It Up

Fri, 15 Jun 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Speaking at a conference in the UK today, Andrew Eland (of Google) said:

Think of the things scientists won't build because they're not exposed to computing in school.

+1 to that.

All Entries for the Executable Paper Grand Challenge

Thu, 14 Jun 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I don't know how I missed this, but at last year's International Conference on Computational Science, two dozen different groups presented papers in response to Elsevier's Executable Paper challenge. PDFs of their work are all online, and links are included below. As I've said before, one of our goals is to give researchers the skills they need to do this kind of thing; looking through these papers, we've still got a ways to go.

First Workshop on Maintainable Software Practices in e-Science

Sun, 10 Jun 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

First Workshop on Maintainable Software Practices in e-Science

9 October 2012
Co-located with 8th IEEE International Conference on eScience, Chicago

This workshop will focus on the issues relating to the development and maintenance of software that can endure past the limited periods of defined project durations and project funding, and go beyond software engineering best practice to address aspects of cultural, organisational and policy change. By bringing together all those with an interest in ensuring the longer term development and use of software for research, including researchers, developers, research computing specialists, software engineers, infrastructure providers, facilitators, and funders, the goal of this workshop is to understand what software practices can be successfully applied and which lead to long-term improvements in the development of software for e-Science.

Ten Simple Rules

Thu, 07 Jun 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The "Ten Simple Rules" series being run in PLoS Computational Biology has a lot of useful gems. Written by editor-in-chief Philip Bourne and others, the entire collection to date is available as a single PDF, but for those who prefer bite-sized reading, here are ten simple rules for:

...Starting a Company (PDF)
...Getting Involved in Your Scientific Community (PDF)
...Teaching Bioinformatics at the High School Level (PDF)
...Developing a Short Bioinformatics Training Course (PDF)
...Getting Help from Online Scientific Communities (PDF)
...Building and Maintaining a Scientific Reputation (PDF)
...Providing a Scientific Web Resource (PDF)
...Getting Ahead as a Computational Biologist in Academia (PDF)
...Editing Wikipedia (PDF)
...Organizing a Virtual Conference—Anywhere (PDF)
...Chairing a Scientific Session (PDF)
...Choosing between Industry and Academia (PDF)
...Combining Teaching and Research (PDF)
...Organizing a Scientific Meeting (PDF)
...Aspiring Scientists in a Low-Income Country (PDF)
...Graduate Students (PDF)
...Doing Your Best Research, According to Hamming (PDF)
...a Good Poster Presentation (PDF)
...Making Good Oral Presentations (PDF)
...a Successful Collaboration (PDF)
...Selecting a Postdoctoral Position (PDF)
...Reviewers (PDF)
...Getting Grants (PDF)
...Getting Published (PDF)

So, what would your ten simple rules for good computational science be?

Software Carpentry: The E-Book Version?

Mon, 04 Jun 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

If I could send email two years into the past, I'd tell myself to spend a lot less time making videos for Software Carpentry, and a lot more time exploring interactive formats. One that I've been following recently is the IPython Notebook, a "live" lab notebook that allows scientists to mix, share, and explore text, code, graphs, and the like. Another, which I've just come across through this post on Mark Guzdial's ever-informative blog, is an interactive version of "How to Think Like a Computer Scientist". As Brad Miller says in his description:

Spot the Workshops

Thu, 24 May 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Responsible Conduct

Wed, 23 May 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Titus Brown, Ethan White, and I have been talking about what Responsible Conduct of Research (RCR) standards would look like in computational science. Titus has posted a four-point summary; we would welcome your input over on his blog.

The Most Important Scientific Result Published in the Last Year

Fri, 18 May 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

J.M. Wicherts, M. Bakker, and D. Molenaar: "Willingness to Share Research Data Is Related to the Strength of the Evidence and the Quality of Reporting of Statistical Results". PLoS ONE, 6(11): e26828, 2011, doi:10.1371/journal.pone.0026828.

Background

The widespread reluctance to share published research data is often hypothesized to be due to the authors' fear that reanalysis may expose errors in their work or may produce conclusions that contradict their own. However, these hypotheses have not previously been studied systematically.

Fooling the Internet

Tue, 15 May 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A recent article in The Atlantic titled, "How the Professor Who Fooled Wikipedia Got Caught By Reddit" describes how GMU's Prof. T. Mills Kelly has had students fake history online, and how their most recent effort unraveled. There's lots to think about here regarding what scientists should know about using the web, trusting it, and making it their own...

Run My Code

Fri, 11 May 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

RunMyCode is a web site and service intended to support reproducible research (initially in computational economics). Authors create companion web sites for papers that include the software they used; other people can then re-run their models, and (crucially) play with parameters, using cloud-based instances of those environments. They only support MATLAB, R, and SAS right now, but are hoping to add more tools soon. It's a cool idea, and we'd welcome your impressions.

Fish and Bugs

Thu, 10 May 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The May/June 2012 issue of Washington Monthly has an article by Alison Fairbrother titled "A Fish Story". Near the top, it says, "In 2009, a routine methodological upgrade at NOAA—and the subsequent discovery of a few lines of faulty computer code—forced the start of a profound shift in the ASMFC's estimates of menhaden stocks." A few pages later, we get more details:

The Architecture of Open Source Applications: Volume 2

Tue, 08 May 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are very pleased to announce that The Architecture of Open Source Applications: Volume 2 is now available from Lulu. A PDF version will go on sale in the next few days, and e-book will become available as soon as we can produce it. Many thanks to everyone who contributed, and to the indefatigable Amy Brown for pulling it all together. As always, all royalties will go directly to Amnesty International, so if you buy a copy, you'll be helping to make the world a better place.

A Future Student

Mon, 09 Apr 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I don't know what we'll be teaching in 20 years, but I hope Tristan Davis will be there to learn.

Titus Brown Finds a Theme

Fri, 06 Apr 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Titus Brown just posted "Big Data Biology — why efficiency matters", in which he explains the academic, practical, and algorithmic reasons why efficient computation is good for science. His opinions are all grounded in his extensive personal experience working in the overlap between biology and computing, and, combined with a couple of things Michelle Levesque showed off at our Oakland workshop last week, have me thinking that we ought to include half an hour on performance analysis and tuning in the Software Carpentry core. Damn you, Titus—I thought I had this curriculum figured out :-)

Upcoming Events for Webmaking Instructors

Tue, 03 Apr 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Software Carpentry has had several homes over the years. Right now, it's part of a larger effort by the Mozilla Foundation to teach people—all kinds of people—how to make the web their own. And as part of that, Mozilla is running events over the next couple of months in Boston, San Francisco, Toronto, and London to bring together people who are trying to teach this stuff. The details are on Michelle Levesque's blog; we hope you'll be able to join us.

Object-Oriented Programming in Fortran 2003

Fri, 23 Mar 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Damian Rouson is teaching a class in Berkeley March 26-28, 2012, and again April 10-12, on object-oriented programming in Fortran 2003. The March class is full, but there are still seats available for April. Fortran is still the workhorse of scientific computing, and recent dialects like 2003 have a lot of powerful features—it's well worth checking out.

And While We're Stuck Here With 21 Seconds Worth of Music to Fill...

Sun, 18 Mar 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Scholarship in the Age of the Internetatron:

{% assign video_title="Scholarship in the Age of the Internetatron" %} {% assign video_slug="8omwsw8ibzw" %} {% assign video_time="00:02:42" %} {% include youtube %}

The IPython Notebook

Mon, 12 Mar 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Titus Brown has created a short video showing what the IPython Notebook can do. The short answer is, a lot: it's an interactive notes-plus-graphics-plus-live-coding tool that runs in the browser. (The video doesn't really get going 'til the 2:50 mark, so you may want to skip ahead to then.) The question isn't whether we want to using this in Software Carpentry, but when. Installing it is (in my experience) still painful, but it holds tremendous potential. If we can get to the point where 90% of our intended audience can "just run it", and the other 10% need only a couple of minutes' help, we'll begin the switch.

I Resemble That Remark

Wed, 07 Mar 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Titus Brown's recent post "Top 12 Reasons You Know You are a Big Data Biologist" may resonate with some readers of this blog. I particularly like the idea (in the comments) of using links in email to drive garbage collection of data sets.

Open Education Week

Mon, 05 Mar 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Today marks the start of Open Education Week. To celebrate, please post a link here to your favorite online educational resource.

Inscight from Trieste

Thu, 01 Mar 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

In the latest Inscight podcast, the inimitable Katy Huff talks to several of the participants in the bootcamp she and Tommy Guy are leading at the International Centre for Theoretical Physics. It's great to hear so many people from all over the world learning so much.

ULP (or, This is tricky and perhaps profound)

Thu, 01 Mar 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Bruce Dawson's "Comparing Floating Point Numbers (2012 edition)" is, as he says of its subject, tricky, and perhaps profound—worthwhile reading for anyone pushing numbers around.

Worth Reading, Worth Watching

Wed, 29 Feb 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The PLoS Computational Biology article "Ten Simple Rules for Getting Help from Online Scientific Communities" has a lot of good advice.
Mark Guzdial's talk "Helping Everyone Create with Computing" is full of good ideas—see in particular the discussion of computational scientists starting around 06:15.

Reproducibility Redux

Tue, 28 Feb 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A recent editorial in Nature, and a longer article by Darrell Ince, Les Hatton, and John Graham-Cumming titled "The case for open computer programs", are just two signs of the growing pressure to raise standards around computational work in science. We can debate the exact meaning and value of reproducibility, but there's no arguing with the fact that if scientists want to do better work, they'll need better skills. That's the real long-term goal of Software Carpentry.

Granules of Research

Wed, 22 Feb 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Cameron Neylon recently posted an article titled "Github for science? Shouldn't we perhaps build TCP/IP first?" His argument is that the web's a good way to move text around, because it was built by programmers, and programmers work with text. It's not (yet) well suited to moving science around, because we don't (yet) have something as granular and portable as text for scientific ideas. Yes, any particular piece of research can be represented as text, but so can any image, or any audio stream, or anything else—it's the structure that adds meaning, and we haven't (yet) agreed on structures. Circling back to today's first post, part of what we're trying to do is give scientists the background to understand and take part in conversations like these...

How They See Us, Part N

Thu, 16 Feb 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This week's Ed-Tech Podcast from Steve Hargadon and Audrey Watters discusses Software Carpentry a bit around the 23:00 mark [1]. In answer to Hargadon's point about home schooling, and whether the way people learning programming even fits the notion of class, we have a couple of answers. First, most of the people we're trying to help don't know enough (yet) to know what to type into Google, how to recognize when they've stumbled upon an answer to their problem, or what to tag a question with on Stack Overflow. Some can climb that hill themselves; a handful can't, but most won't (see below), so one of our goals is to help them get from A to B so that they can get from B to Z.

Analyzing Next-Generation Sequencing Data

Wed, 15 Feb 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Analyzing Next-Generation Sequencing Data

http://bioinformatics.msu.edu/ngs-summer-course-2012

June 4th — June 15th, 2012
Kellogg Biological Station, MSU
Course sponsor: NIH.

Instructors: Dr. C. Titus Brown, Dr. Ian Dworkin, and Dr. Istvan Albert.

Board of advisors: Dr. Kevin White; Dr. Paul Sternberg; Dr. Rich Lenski; Dr. Robin Buell; Dr. Jim Tiedje; Dr. Lincoln Stein

Applications are being accepted through March 1st (midnight Pacific)!

And Speaking of New...

Wed, 15 Feb 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

...check out Bret Victor's talk at CUSEC 2012—jump in around the 7:00 mark and watch for a couple of minutes. You'll want to go back and watch the whole thing...

Advanced Scientific Programming in Python

Fri, 10 Feb 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Advanced Scientific Programming in Python

A Summer School by the G-Node and the Institute of Experimental and Applied Physics, Christian-Albrechts-Universität zu Kiel

Scientists spend more and more time writing, maintaining, and debugging software. While techniques for doing this efficiently have evolved, only few scientists actually use them. As a result, instead of doing their research, they spend far too much time writing deficient code and reinventing the wheel. In this course we will present a selection of advanced programming techniques, incorporating theoretical lectures and practical exercises tailored to the needs of a programming scientist. New skills will be tested in a real programming project: we will team up to develop an entertaining scientific computer game.

Our Long Tail

Thu, 26 Jan 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Frequency a page is viewed (as a percentage of total views) vs. pages (ordered by frequency); data taken over the last 90 days. I guess there's something to this "long tail" stuff after all.

Test-Driven Public Speaking

Tue, 24 Jan 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Once again, Cameron Neylon explains things much better than I ever could:

"The impact factor of a journal is a better predictor of the chances of a paper being retracted than...of the number of citations."

New Features in Excel for Scientists

Tue, 13 Dec 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Carly Strasser, of the Digital Curation for Excel Project, has posted some ideas about new features that could be added to Excel to make it more useful for scientists. In brief, the list is:

Generate metadata.
Generate a data citation for the data file.
Check the spreadsheet for export compatibility.
Link to archive services.

Feedback and suggestions would be welcome...

Surviving the Tsunami

Mon, 14 Nov 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The October 2011 issue of ACM Queue features an article by Bruce Berriman and Steven Groom titled "How Will Astronomy Archives Survive the Data Tsunami?" The figures are scary: astronomers already have a petabyte of publicy-available data, and are adding half a petabyte per year, a rate which will increase dramatically as new instruments come online. The only way to avoid this all becoming write-only is to bet on emerging technologies, from general-purpose GPUs to cloud computing. The problem, of course, is that "emerging" usually means "flaky", both because the tools haven't had time to mature, and because we, their users, don't have the years of experience needed to know how best to use them. (As far as I'm concerned, we're still trying to figure out how best to use object-oriented programming in science, and we've been at it for thirty years...)

The Best vs. the Good

Tue, 08 Nov 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Cameron Neylon recently posted another thought-provoking piece, this one titled, "Building the perfect data repository...or the one that might get used". In it, he talks about why big institutional efforts to create scientific data repositories have mostly failed to take off, and points at simpler grassroots efforts that scientists might actually adopt because they immediately and obviously solve problems that scientists actually realize they have. One that he points to is DataStage, "a secure personalized 'local' file management environment for use at the research group level, appearing as a mapped drive on the user's PC." Another is If This Then That, which is the simplest "dataflow" tool you could imagine, and which I've been using regularly since it burst on the scene a couple of months ago.

Slides from Hans-Martin

Fri, 21 Oct 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Hans-Martin von Gaudecker has posted the slides for his course "Effective programming practices for economists". They're a nice complement to the ones on this site—thanks, HM.

American Scientist Article on Empirical Studies of Software Engineering

Wed, 19 Oct 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

An article that Jorge Aranda and I wrote for American Scientist about empirical studies of software engineering is now up on the web. We hope it's a good introduction to the area, and we look forward to your feedback. If you'd like to know more about the area, please check out our joint "neat results" blog at http://www.neverworkintheory.org/.

The Science Code Manifesto's Five C's

Fri, 14 Oct 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Science Code Manifesto comprises five core principles:

Code: All source code written specifically to process data for a published paper must be available to the reviewers and readers of the paper.
Copyright: The copyright ownership and license of any released source code must be clearly stated.
Citation: Researchers who use or adapt science source code in their research must credit the code's creators in resulting publications.
Credit: Software contributions must be included in systems of scientific assessment, credit, and recognition.
Curation: Source code must remain available, linked to related materials, for the useful lifetime of the publication.

In a way, Software Carpentry's goal is to give people the skills they need to do these things right: to create code that other people can read, to share and evolve that code in maintainable ways, and to create code that's useful in the first place. If you agree with their principles, please take a moment to endorse them, and please help make them a reality.

The Simplest Web That Could Possibly Work

Sat, 17 Sep 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A new web tool called If This Then That generated a flurry of interest this week (see for example Scott Hanselman's blog post). Simply put, IFTTT lets you connect web services together using if-then rules. It's as simple as:

Click on "this" to start.
Select "Twitter", and tick off boxes to create a rule (such as "when Alan Turing tweets").
Click on "that".
Select what you want to happen (e.g., echo the tweet to your own Facebook page).
There is no step 5.

That's the beauty of it: there is no step 5. It knows how to work with all sorts of services, from Instapaper and Pinboard to weather reports (yes, weather reports), and there is no step 5. Just as Twitter stripped blogs down to their bare essentials, IFTTT takes graphical workflow tools like Yahoo! Pipes and says, "What's the simplest version of this idea that could possibly be useful?"

Renting Cycles Has Never Been Easier (For Some Definition of 'Easier')

Thu, 01 Sep 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This video, titled "Building an AWS Cluster in 10 Minutes", is both inspiring and depressing: inspiring, because you really can set up a cheap supercomputer for almost no money in just a few minutes, and depressing, because it's effectively out of reach for most scientists, who won't understand the terms being thrown around (certainly not well enough to get themselves out of trouble if anything goes wrong). The good news is, Titus Brown's course has shown that scientists can get from zero to cluster in just two (full-time) weeks.

Software Carpentry in HPCWire

Fri, 22 Jul 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

HPCWire has run an interview with me about Software Carpentry (a follow-up to one they did several years ago). Regular readers will have seen the main points before, but I hope it's a good summary of the current state of play, and of what's wrong with equating "scientific computing" and "high performance computing".

And Speaking of Titus Brown...

Wed, 20 Jul 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Michigan State's Titus Brown recently ran his course on analyzing next-generation sequencing data for the second time. Judging from his report, it was just as successful as last year's. Congrats!

How Much Do You Need?

Wed, 20 Jul 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Michigan State's Titus Brown has posted a good discussion of what kind of computing hardware you need to do bioinformatics, and why. Long story short, it's about $100K/year (US).

Material from Newcastle Workshop Now Available

Wed, 20 Jul 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Back in June, we mentioned a workshop on computing skills at the University of Newcastle. It reportedly went very well, and presentations and other materials are now available. Many thanks to Elizabeth Petrie, the other organizers, and the presenters.

The Case of Abinit

Wed, 20 Jul 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The latest issue of Computing in Science & Engineering has a good article by Yann Pouillon, Jean-Michel Beuken, Thierry Deutsch, Marc Torrent, and Xavier Gonze titled "Organizing Software Growth and Distributed Development: the Case of Abinit" (unfortunately behind a pay wall). It describes the infrastructure and practices used to manage Abinit, a half-million line open-source program. It's a lot more engineering than most teams need, but it provides a lot of insight into what small projects might grow into. If anyone would like to do a similar paper about what a smaller-scale team uses, it would be a welcome counterpoint.

Reproducible Computational Geophysics

Wed, 06 Jul 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A summary of a recent workshop on Open Software Tools for Reproducible Computational Geophysics is now online. Lots of interesting stuff, and once again, the things we teach in this course are prerequisites for doing/using/understanding much of it.

Mentioned in Nature Methods

Fri, 01 Jul 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A recent article in Nature Methods by Jeffrey Perkel titled "Coding your way out of a problem" makes mention of Software Carpentry. Elsewhere, Mike Croucher has written a nice article called "In defense of inefficient scientific code", which makes many of the same points we've been making here.

It Will Never Work in Theory

Wed, 29 Jun 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Inspired in part by Lambda the Ultimate, which reports on what's new in programming language research, Jorge Aranda and I have started a new blog called "It Will Never Work in Theory" to bring you the latest results in empirical studies of software engineering. The first posts discuss:

Rahman and Devanbu's "Ownership, Experience, and Defects: A Fine-Grained Study of Authorship", which found that code worked on by one developer (rather than many) is more often implicated in defects, but that a developer's experience with a particular file (rather than the project in general) reduces defect rates.
Stolee and Elbaum's "Refactoring Pipe-like Mashups for End-User Programmers", which applies the "code smells" meme to Yahoo! Pipes (and by implication shows that refactoring ideas can be applied to other end-user programming systems).
Mockus's "Organizational Volatility and its Effects on Software", which found that an influx of newcomers into a project doesn't increase fault rates (since they're usually given simple tasks to start with), but that organizational change can still account for about 20% of faults.

Our aim in starting this blog is to continue the work begun in Making Software: to let practitioners know what researchers have discovered, and what kinds of questions they can answer, and to give researchers feedback on what's useful, what isn't, and what they ought to look at next. We look forward to your feedback.

Michael Nielsen Talks About Open Science in San Francisco on June 29

Wed, 22 Jun 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As per his blog post, the inimitable [1] Michael Nielsen will be talking about "Why the net doesn't work for science—and how to fix it" next Wednesday in San Francisco. It's sure to be both informative and enjoyable—hope you can make it.

[1] Well, you try to imitate an Australian quantum physicist turned open science advocate...

Health Informatics Resources

Sat, 18 Jun 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Via William Hopper, a list of online healthcare informatics resources that might be of interest to some readers. If you have others, I'm sure he'd enjoy hearing from you.

Practical Computing for Everyone (not just biologists)

Tue, 07 Jun 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Steven Haddock and Casey Dunn:
Practical Computing for Biologists.
Sinauer Associates, 2010, 0878933913.

My copy of Practical Computing for Biologists arrived last week, and I've been very impressed. It is a well-written, well-paced guide to basic computing skills for scientists and engineers of all stripes (not just biologists). Many of the topics will be familiar:

editing text files (including how to use regular expressions in an editor)
the Unix shell
basic Python programming (including debugging strategies)
relational databases
SSH
installing and configuring software

There are also a few that we don't cover, such as interacting with hardware, and some that are covered in more depth than we give them, like image manipulation. The pace is gentler than Software Carpentry, but the last couple of years have convinced me that's a good thing: I think Haddock & Dunn have it right for this target audience. And it's beautifully produced: full-color printing and great graphical design make this book a joy to read. If I ever do turn Software Carpentry into a book, I might skip the topics PCB covers and just tell people to go and buy it.

Programming for Scientists at Newcastle University: June 20, 2011

Sat, 04 Jun 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

From the announcement:

Programming is becoming an increasingly important part of scientific research, yet many scientists are self-taught programmers with little formal training. This means that we are often unfamiliar with simple tools that can make programming and dealing with data faster, more reliable and more reproducible. This event is a day-long workshop to develop awareness of the skills and tools that help make computing more efficient and provide results that are less prone to error. If you've ever thought "Surely there must be a better way to do this", then this is the event for you!

'The Architecture of Open Source Applications' is Now Available

Mon, 23 May 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

It has been slightly over a year in the making, but it's finally here: The Architecture of Open Source Applications has been published. You can buy the book directly from Lulu.com at http://www.lulu.com/product/paperback/the-architecture-of-open-source-applications/15819207, or view the contents online at http://aosabook.org.

My thanks to all the people who contributed to the book, and especially to Amy Brown, my tireless and diligent co-editor. We hope you enjoy it.

More Interested in the Asides

Sat, 14 May 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

So how did the term go, you ask? Here's what traffic on this site looked like:

The three spikes in March where we had up to ten times our usual 250-a-day visitors were the articles on tuple spaces, literate programming, and graph layout—in other words, the articles that weren't about teaching scientists and engineers basic computational skills. As the man said, "So it goes."

The Architecture of Open Source Applications

Fri, 06 May 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

My apologies for how quiet this site has been recently—finding a new job takes time [1], and we had to make one final push on the other project I've been working on. But I'm pleased to announce that The Architecture of Open Source Applications is now at the printer's—we're going to give the first copy a final going-over, then make it available for purchase (hopefully within two weeks). I'm very pleased with how it looks, and very grateful to my co-editor Amy Brown, the contributors, and the reviewers for all their hard work.

In Praise of Street Fighting

Fri, 22 Apr 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Sanjoy Mahajan's Street-Fighting Mathematics is subtitled "The Art of Educated Guessing and Opportunistic Problem Solving". As the author says in the introduction, "Too much mathematical rigor teaches rigor mortis: the fear of making an unjustified leap even when it lands on a correct result." It's only 134 pages long (including the index), but it's packed full of practical ideas for tackling mathematical problems; in a sense, it aims to train people to do the kind of rough-and-ready calculations that David MacKay deploys so effectively in Sustainable Energy—Without the Hot Air.

Holding Up a Mirror

Mon, 18 Apr 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Cameron Neylon always has interesting things to say. In a recent talk, he commented on my tendency to idealize laboratory practice when contrasting it with the sorry state of computational work. Thought-provoking...

And I'm on a Horse

Sat, 26 Mar 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Patrick Mackenzie (whom I've never met) gave a good lightning talk at the Business of Software that sums up a lot of what we haven't done for Software Carpentry. Any reworking of the material really (really) has to be built around what you (the researchers) need, rather than what we (the programmers) know.

A Better Way to Teach Programming to Scientists

Thu, 24 Mar 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This year's SIGCSE conference on computer science education featured a very cool paper by Robbins, Senseman, and Pate, of the University of Texas at San Antonio, called "Teaching Biologists to Compute using Data Visualization". In it, they describe CS 1173: Data Analysis and Visualization using MATLAB, which introduces students in the life sciences to programming using a problem-first approach. Like the media-first approach that Mark Guzdial and his colleagues introduced at Georgia Tech, this rewards students right away—they don't have to wade through a "CS first" morass of data types and arcane rules about Boolean expressions for three or four weeks in order to get to the useful bits. Given another year, I'd have rewritten our intro to Python this way...

I'd Settle for 0.1%

Tue, 22 Mar 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

In a recent article about computational thinking, Carnegie-Mellon's Jeannette Wing says:

...every scientific directorate and office at the National Science Foundation participates in the Cyber-enabled Discovery and Innovation, or CDI, program, an initiative started four years ago with a fiscal year 2011 budget request of $100 million. CDI is in a nutshell "computational thinking for science and engineering."

0.1% of that would keep Software Carpentry going for another year... Or if I'm allowed to be a curmudgeon for a moment, hands up those people who believe that $100 million is going to do 1000 times more for science and engineering than another year of work on these materials? *sigh*

Next-Generation Sequencing Course at MSU

Wed, 16 Mar 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Analyzing Next-Generation Sequencing Data

June 6th — June 17th, 2011

Kellogg Biological Station, Michigan State University

Instructors: Dr. C. Titus Brown, Dr. Ian Dworkin, and Dr. Istvan Albert.

Applications must be received by March 25th for full consideration. See the full announcement at http://bioinformatics.msu.edu/ngs-summer-course-2011 for more information.

Course Description:

This intensive two week summer course will introduce students with a strong biology background to the practice of analyzing short-read sequencing data from Roche 454, Illumina GA2, ABI SOLiD, Pacific Biosciences, and other next-gen platforms. The first week will introduce students to computational thinking and large-scale data analysis on UNIX platforms. The second week will focus on mapping, assembly, and analysis of short-read data for resequencing, ChIP-seq, and RNAseq.

What To Demand

Sat, 12 Mar 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Peter Norvig (formerly of NASA, now at Google) recently gave a talk titled "What to Demand From a Scientific Computing Language". It's a good talk (and not just because he explains why he's a fan of Python). I was a bit disappointed, though, by this list:

Shouldn't there be a version control system on this list? And some sort of provenance tool? OK, that's a trick question: there aren't any provenance tools in widespread use, but what about testing tools? There are lots of those, and there's even a name ("xUnit") for the whole JUnit-style family of tools across different languages. Saying "these aren't really core to scientific computing" is sort of like saying "disinfection isn't really core to surgery".

Science Illustrated

Fri, 11 Mar 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Videos of lectures from a two-day symposium on scientific visualization called Science Illustrated are now online. Many thanks to Mubdi Rahman for organizing it.

Advanced Scientific Programming in Python

Wed, 09 Mar 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Advanced Scientific Programming in Python

A Summer School by the G-Node and the School of Psychology, University of St Andrews

High Tech That Looks Low Tech

Wed, 09 Mar 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

These videos about climate change are great: they look low tech, with hand-drawn diagrams and low-fi narration, but I imagine a lot more work went into their animation than into our slideshows.

We Got a Mention in Comm. ACM

Fri, 25 Feb 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The March 2010 issue of Communications of the ACM has an article on grid computing that mentions Software Carpentry and Titus Brown's course at Michigan State—yay! Unfortunately, Software Carpentry didn't get a URL, and the bit.ly URL for Titus's course currently redirects to the wrong place.

Later: the bit.ly URL is apparently working again.

Scientific Computing Podcast

Fri, 18 Feb 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

inSCIght is a podcast that focuses on scientific computing in all of its various forms. Every week we have four panelists engage head-to-head on poignant and interesting topics. The panelists are drawn from all across the scientific computing community. From embedded systems experts to very high level language gurus, from biologists to nuclear engineers, the hosts of inSCIght use computers to solve science and engineering problems everyday. This podcast throws people, ideas, and opinions into an audio-blender hoping to educate and entice each other and the world.

Reddit on Scientific Programming

Thu, 17 Feb 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Nature's article on scientific computing has attracted 75 comments on Reddit. Some interesting ideas being kicked around; please go ahead and add your own.

I Want Their Software

Wed, 16 Feb 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This blog post about the Khan Academy might gush a bit, but the embedded video is a must-watch for anyone who cares about education. I really want their software...

Thinking Like the Web

Wed, 26 Jan 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Jon Udell's 1999 book Practical Internet Groupware was a revelation for me: it was the first coherent explanation I'd ever read of how the disparate collection of technologies and social conventions that we call "the web" fit together, and what the deeper patterns and concepts beneath them are. After a lot of further work and thought, Jon has condensed those ideas into seven principles—or as he puts it, "Seven Ways to Think Like the Web". These concepts are the most meaningful definition yet of what the phrase "computational thinking" actually means, and of what people who aren't programmers need to know in order to use the web effectively. As I said in the post on Tom Limoncelli's plea to software vendors, we'll know we're teaching the right things, the right way, when people who have done this course understand these principles and how to apply them.

Making System Administrators' Lives Easier

Wed, 19 Jan 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Google's Thomas Limoncelli (author of Time Management for System Administrators) has an article in ACM Queue titled "A Plea to Software Vendors from Sysadmins—10 Do's and Don'ts". His list is:

Do have a "silent install" option.
Don't make the administrative interface a GUI.
Do create an API so that the system can be remotely administered.
Do have a configuration file that is an ASCII file, not a binary blob.
Do include a clearly defined method to restore all user data, a single user's data, and individual items.
Do instrument the system so that we can monitor more than just, "Is it up or down?"
Do tell us about security issues.
Do use the built-in system logging mechanism.
Don't scribble all over the disk.
Do publish documentation electronically on your Web site.

Every one of these applies equally to scientific software, big and small. We'll know we're teaching the right things, the right way, when people who have done this course know how to write software that obeys these rules and (more importantly) understands why they should.

Open Research Computation

Mon, 27 Dec 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

By now, many of you have (hopefully) seen the announcement of Open Research Computation, a new journal devoted to "...peer reviewed articles that describe the development, capacities, and uses of software designed for use by researchers in any field." The editorial board includes several friends of this course; as one of them, Titus Brown, observed in his blog:

...the problem with the online world for scientists [is] there's no real systematized incentive to any of this online stuff. And that makes it really tough. I'm going through Reappointment right now... Nowhere on there is there a place for "influential blog posts" — how would you measure that, anyway? Same with software — I listed my various software releases on the "scientific products" page of the form, and have since been asked to describe and discuss the impact of my software. Since I don't track downloads, and half or more of the software hasn't been published yet and can't easily be cited, and people don't seem to reliably cite open source software anyway, I'm not sure how to document the impact.

Executable Papers

Mon, 20 Dec 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Elsevier is sponsoring an "Executable Paper Grand Challenge". If you have more than just ideas about the future of scholarly publication in computational science, it may be a good way to get them some press.

Programmer Competency Matrix

Thu, 02 Dec 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

There's no scientific research behind this tabulation of what programmers ought to know, and some of the categorizations are unactionably vague, but it's still a useful guide. Most of Software Carpentry is Level 1, with some Level 0 and Level 2 stuff blended in.

Red-R

Thu, 02 Dec 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

From a reader, a link to Red-R, a visual programming environment for R. The six-minute screencast on the documentation page shows what it's capable of. I don't know of any similar projects for Python—I'd appreciate pointers.

Making Software Screencast

Wed, 17 Nov 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A short screencast about Making Software is now up on Amazon. In each of the book's 30 chapters, leading researchers in software engineering explain what we actually know about code review, statistical prediction of software faults, how programmers communicate, whether using design patterns makes code better, the relative quality of open and closed source software, and a host of other topics. We hope you enjoy it.

Nature Article on Scientific Programming

Thu, 14 Oct 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Nature has just published an article by Zeeya Merali titled "Computational science: ...Error" that looks at what's wrong with scientific computing today. Software Carpentry gets a mention, along with some of the scientists who are trying to raise awareness and standards. And in a companion article, Nick Barnes makes the case for scientists publishing their research software. Here's hoping both will be widely read.

43% Independent

Mon, 16 Aug 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Cecilia d'Oliveira and colleagues recently wrote an essay in Science about MIT's OpenCourseWare initiative, ten years after its inception. Among the stats:

OCW currently receives upwards of 1.5 million visits each month from 900,000 unique individuals. Students have grown to 42% of the audience, and educators and independent learners now constitute 9% and 43% of visitors, respectively. Twelve percent of educators responding to a March 2010 visitor survey indicated that they do incorporate OCW materials into their own content as anticipated, but educators more frequently use OCW for personal learning (37%), to adopt new teaching methods (18%), and as a reference for their students (16%). Students were largely expected to use the site as a supplement to materials they received in their own classes, a use identified by 40% of students. Just over 43%, however, indicated that they also use OCW for personal learning beyond the scope of their formal studies, and a further 12% use it as an aid in planning their course of study. Independent learners use OCW in a variety of personal (41%) and professional (50%) contexts, including home-schooling children and keeping up on developments in their professional field. 66% of visitors indicate they are mostly or completely successful at meeting their educational goals for visiting the site.

Mark Guzdial on Software Carpentry

Wed, 28 Jul 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Mark Guzdial, a leading researcher in computing education, blogged a few days ago about the Texas Advanced Computing Center's training program for computational scientists, and asked, "Given the importance of computational science, what do all scientists and engineers need to know about high-performance computing?" As you might expect, I replied to say that the question was almost always premature: we should first ask what scientists and engineers need to know about computing in general before tackling HPC.

A Gentle Introduction

Sun, 18 Jul 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Via several routes, I've been pointed at MIT Open CourseWare's "A Gentle Introduction to Programming Using Python", as taught by Sarina Canelake in January 2010. An hour of lecture a day for 10 days (two weeks), plus two hours of lab per day, it is... "a gentle introduction to programming using Python for highly motivated students with little or no prior experience in programming computers". The pace is brisk:

HPC and Programmability

Sat, 10 Jul 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Via Andrew Lumsdaine, a pointer to an interesting article in Communications of the ACM by Eugene Loh titled "The Ideal HPC Programming Language". A few key quotes:

These programmability studies began with a focus on programming languages, but the focus quickly shifted to other topics. Existing languages—notably Fortran...—proved remarkably adequate. Programming challenges stem mostly from other factors.

The DARPA HPCS program...sponsored the development of new programming languages: Chapel from Cray, Fortress from Sun, and X10 from IBM. Proponents of those languages would show early on how rewriting familiar HPC benchmarks in the new languages could reduce source-code volume substantially—tenfold reductions were not surprising—but rewriting these benchmarks even in Fortran achieved similar source-code reductions and corresponding improvements in expressivity.

Eric Lander on Genomics

Thu, 24 Jun 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This lecture by MIT's Eric Lander is a great overview of where genomics is, how we got here, and where we might be going. "Biology as an information science" is proving to be a very rich idea...

SIAM News Article About Software Carpentry

Thu, 24 Jun 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The June 2010 issue of SIAM News (published by the Society for Industrial and Applied Mathematics) has an article I wrote about Software Carpentry called "Get More Done with Less Pain". Here's hoping the course can live up to that billing :-)

For World Cup Fans (and Everyone Else)

Fri, 18 Jun 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Mike Knell has posted a funny piece titled "If Sports Got Reported Like Science" [accessed via Internet Archive]. Perhaps I should have put quotes around "funny"...

A Voice from the Back of the Room

Wed, 16 Jun 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Mark Guzdial recently posted another thought-provoking piece, this one about how teachers are biased toward assessing a class's progress by their interactions with its brightest students, and how important it is to assess understanding systematically to find out what's actually going on. We're still wrestling with this in Software Carpentry: what can we put on the web to help students determine whether they have understood a particular topic and are ready to move on?

Who Reports On The Other 97 Per Cent?

Tue, 01 Jun 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The BBC has posted a nice graphic of the latest Top 500 list of supercomputers around the world. It's pretty impressive, particularly if you're a Linux fan (check out the display by operating system). However, nobody would think that a list of the world's 500 richest people told them much about the state of the economy; I'd be much more interested in statistics on how much computing power is employed by the 97% or so of scientists who don't use a supercomputer.

Jim Graham on Reproducibility

Sat, 29 May 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

In response to Titus Brown's not-really-joking spoof of how most scientists manage their data, Scimatic's Jim Graham has asked, "What is reproducibility, anyway?" His main point is that if I re-run your code on your data, and get the same result that you got, I haven't actually added to the sum total of human knowledge. If I get an equivalent answer using different code, on the other hand, then our confidence in the answer is usefully increased. What do you think?

Archiving Experiments to Raise Scientific Standards

Tue, 25 May 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

An NSF Workshop on Archiving Experiments to Raise Scientific Standards begins today (May 25, 2010) at the University of Utah — the schedule has links to some of the presentations, and there's a live video feed as well. Topic areas range from networking and compilers to geophysics and the life sciences — it'll be interesting to see what comes out, and how it interacts with the Open Provenance work.

Evaluating Methods and Protocols

Wed, 19 May 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

From their home page:

ScienceCheck.org is the only site dedicated specifically to sharing objective evaluations of published experimental methods/protocols, from researchers with real-world experience.

Sadly, the "computer science" category (filed under "Other") is currently empty, and there isn't a "computational science" category as far as I can see. Still, it's a beginning...

File Sharing for Scientists

Fri, 16 Apr 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A scientist I recently met in Toronto had a problem: how to share large files with colleagues. Each file is a couple of hundred megabytes; dozens are produced each week, but each is only interesting for a couple of months; and there are confidentiality issues, so some kind of password protection is needed. Conventional file-sharing services like Dropbox aren't designed for data that size, so in the end she bought a domain and set up secure FTP.

Teaching Open Source

Mon, 12 Apr 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Over at opensource.com, Red Hat's Greg DeKoenigsberg has a post about a new collaboratively-authored textbook on open source software aimed squarely at undergrad courses. As Máirín Duffy points out in the first comment, it's very code-centric, but in my experience, that's the right approach: students won't be ready for discussion of design until they're proficient in coding [1]. I'm looking forward to borrowing lots from the book for Software Carpentry...

Models To Imitate

Thu, 01 Apr 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

My father once told me that a week of hard work can sometimes save you an hour of thought. In that spirit, I've been looking for asynchronous online courses to imitate. I previously mentioned MIT's Open Courseware, CMU's Open Learning Initiative, and (closer to my scale) Saleem Khan's Khan Academy. Google Code University's lessons on programming languages are also on my radar—I'll blog more about them once I finish the Python material—but another model that I'm looking at closely is Teaching Open Source, a collaborative effort to get more open source into college and university courses. I first encountered them through POSSE (Professors' Open Source Summer Experience), which they describe as:

Periodic Table of Science Bloggers

Wed, 31 Mar 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

David Bradley has created a periodic table of science bloggers that regular readers might enjoy:

Summer Course: Analyzing Next-Generation Sequencing Data

Thu, 25 Mar 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Analyzing Next-Generation Sequencing Data

May 31 — June 11th, 2010
Kellogg Biological Station, Michigan State University
CSE 891 s431 / MMG 890 s433, 2 cr

http://bioinformatics.msu.edu/ngs-summer-course-2010

Applications are due by midnight EST, April 9th, 2010.

Course sponsor: Gene Expression in Disease and Development Focus Group at Michigan State University.

Instructors: Dr. C. Titus Brown and Dr. Gregory V. Wilson

This intensive two week summer course will introduce students with a strong biology background to the practice of analyzing short-read sequencing data from the Illumina GA2 and other next-gen platforms. The first week will introduce students to computational thinking and large-scale data analysis on UNIX platforms. The second week will focus on mapping, assembly, and analysis of short-read data for resequencing, ChIP-seq, and RNAseq.

Panton Principles

Sun, 28 Feb 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Via Cameron Neylon: the Panton Principles are guidelines for open data in science. In full:

Science is based on building on, reusing and openly criticising the published body of scientific knowledge.

For science to effectively function, and for society to reap the full benefits from scientific endeavours, it is crucial that science data be made open.

By open data in science we mean that it is freely available on the public internet permitting any user to download, copy, analyse, re-process, pass them to software or use them for any other purpose without financial, legal, or technical barriers other than those inseparable from gaining access to the internet itself. To this end data related to published science should be explicitly placed in the public domain.

BEACON Funded!

Mon, 22 Feb 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Congratulations to Titus Brown and others on the NSF's announcement that it will fund the BEACON (Bio/computational Evolution in Action Consortium) Science and Technology Center. BEACON "...BEACON is focused on studying the evolution of organization across multiple scales—from genomic and cellular, to multicellular, to inter-multicellular (a.k.a. social)—using techniques from experimental evolution, modeling, and digital life systems." Long story short, this means that Michigan State University and its partner institutions "...has money explicitly for supporting students doing really sexy interdisciplinary work combining computation and biology."

Two Views

Fri, 12 Feb 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Darrell Ince (who, as far as I know, hasn't worked directly with climate scientists or their software) wrote about ClimateGate for The Guardian. In response, Steve Easterbrook (who has) wrote a detailed rebuttal. Here's hoping the latter is as widely read as the former.

It Seems That Everyone Cares

Sun, 24 Jan 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Ars Technica isn't primarily a science site, but even they are now worried about reproducibility in computational science. I think it no longer matters how important this "crisis" actually is—sooner or later, major funding agencies are going to mandate adoption of something like the Open Provenance Model. Problem is, given the current skill set of the average scientist, that will almost certainly translate into burden without benefit.

Big Science == Big Skills Gap

Wed, 20 Jan 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Over on Nature News, Eric Hand's article "'Big science' spurs collaborative trend" is subtitled, "Complicated projects mean that science is becoming more globalized." It talks about the benefits of international collaboration, but what it doesn't say is that sharing ideas, results, procedures, and software requires skills that aren't part of the standard curriculum. One of the main goals of the rewrite of Software Carpentry is to teach scientists some of what they need to know in order to do what Hand describes without heroic effort. I'd be grateful for suggestions about topics and tools that ought to be on the list, but aren't.

Whatcha Gonna Do When They Come For You?

Wed, 13 Jan 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

First it was pharma companies withholding "unhelpful" data, then it was ClimateGate, and now there's this:

One of the founders of the controversial 'Baby Einstein' range of products is taking the University of Washington to court in an attempt to force the institution's scientists to release their raw data to him...William Clark...wants records relating to two studies published in 2004 and 2007. The latter found an "association between early viewing of baby DVDs/videos and poor language development" while the former suggested "efforts to limit television viewing in early childhood may be warranted".

Dudley and Butte on Software Skills

Sun, 27 Dec 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Via Titus Brown, a new PLoS paper titled "A Quick Guide for Developing Effective Bioinformatics Programming Skills" by Joel Dudley and Atul Butte. Their recommendations are:

Programming languages
Embracing open source
Unix command-line skills
Keeping projects documented and manageable
Preserving source code with version control
Embracing parallel computing paradigms
Structuring data for speed and scalability
Understanding the capabilities of hardware
Embracing standards and interoperability
Put a high value on your time

I think all these things matter, but:

Why Opening Up (Probably) Wouldn't Help

Fri, 11 Dec 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Good post from Steve Easterbrook on why open-sourcing climate models probably wouldn't make a difference.

Caesar's Wife

Thu, 26 Nov 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Improving the way scientists use computers isn't just about making them more productive: it's also essential to defending the integrity of their work. Stories like this one about a researcher struggling in vain for three years to replicate someone else's results can only undermine public confidence at a time when we need to make a lot of hard decisions in a hurry. Sadly, we have no one to blame but ourselves...

Serendipitous and Unexpected

Sun, 22 Nov 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Via Ryan Lilian:

Most research effort does not produce what is thought of as a traditionally publishable result. That doesn't mean, however, that nothing was gained by conducting the research. These results, whether they are failures or merely perplexing, can provide valuable insights into open problems and prevent other researchers from duplicating work. We started a journal that focuses on serendipitous (I have no idea why this worked) and unexpected (it seems like this technique should work on this problem but it doesn't) results. The goal of the journal is to provide a venue where ideas can flow and be debated.

Special Issue of Computing in Science and Engineering

Wed, 18 Nov 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A special issue of Computing in Science & Engineering that Andy Lumsdaine and I edited, devoted to software engineering in computational science, is now available. We'd like to thank everyone who contributed:

Report on the Second International Workshop on Software Engineering for CSE, by Jeffrey Carver (University of Alabama)
Managing Chaos: Lessons Learned Developing Software in the Life Sciences, by Sarah Killcoyne and John Boyle (Institute for Systems Biology)
Scientific Computing's Productivity Gridlock: How Software Engineering Can Help, by Stuart Faulk (University of Oregon), Eugene Loh and Michael L. Van De Vanter (Sun Microsystems), Susan Squires (Tactics), and Lawrence G. Votta, (Brincos)
Mutation Sensitivity Testing, by Daniel Hook (Engineering Seismology Group Solutions) and Diane Kelly (Royal Military College of Canada)
Automated Software Testing for MATLAB, by Steve Eddins (The MathWorks)
The libflame Library for Dense Matrix Computations, by Field G. Van Zee, Ernie Chan, and Robert A. van de Geijn (University of Texas at Austin), and Enrique S. Quintana-Ortí and Gregorio Quintana-Ortí (Universidad Jaime I de Castellón)
Engineering the Software for Understanding Climate Change, by Steve Easterbrook (University of Toronto) and Timothy Johns (Hadley Centre for Climate Prediction and Research)

Cloud Computing for Beginners

Sun, 15 Nov 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Ana Nelson has posted step-by-step instructions showing how to use Amazon's EC2 cloud computing platform to run simulations. There are still a lot of fiddly details, but the barriers to entry are getting lower all the time...

Python in Science

Fri, 06 Nov 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Guido van Rossum just posted a description of a variety of scientific Python projects. The diversity is pretty impressive...

Creating New Niches

Wed, 21 Oct 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

"Publish or perish" is the central credo of academic life: despite all the hoopla about the blogosphere and online what-not, the reality for most of us is that if our work doesn't get into a respected journal or conference, it doesn't count.

But what do you do if there isn't a home for your kind of work? People working in scientific computing have been struggling with this for at least a quarter century: while there are many places to submit the results of programs, there are very few places where you can publish a description of the program itself, even if building it took years and required one intellectual breakthrough after another. In contrast, if you design a new telescope, there are at least half a dozen places you could turn.

Another Reason to Care About Provenance

Mon, 21 Sep 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A vice president at ETH is resigning today because of accusations of fraud. A key lab notebook is missing, retractions have been retracted, and according to ETH President Ralph Eichler, "there is now no legal way of finding out for sure who was responsible for the falsifications."

Question: if someone accused you of falsifying results, how well and how easily could you defend yourself? How long would it take you to pull together all the notes, data, and programs you used three years ago to produce the paper being challenged? And would you career survive? Even if you "won", you would probably lose weeks or months of research time.

Two Links

Fri, 11 Sep 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Mendeley has been getting some good press—anyone have first-hand experiences to share?
A Nature special on data sharing. Bryn Nelson's piece on empty archives is particularly worth reading.

Is The Future Waving At You?

Sun, 30 Aug 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Cameron Neylon has been playing with Google Wave, and he likes it. His presentation at Science Online in London in August explains why (you can also watch video, though sadly there's no soundtrack). He's even writing robots to automate some scientifically interesting tasks. Nature News liked Wave too, which (a) reminds me yet again of how prescient Jon Udell's "Internet Groupware for Scientific Collaboration" was ten years ago, and (b) makes me wonder (also again) how much this course should be re-thought.

Bad News and Good News

Mon, 24 Aug 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The bad news is, retractions of scientific papers have risen tenfold since 1990. The good news is, the rate has gone from 0.0007% to 0.007%. Going back to bad news, though, some estimates of how many papers ought to be retracted are around 1%, so we still have a long way to go.

Playing Safe

Mon, 24 Aug 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This thoughtful article from the New York Times asks whether the current grant system for funding research discourages researchers from taking risks. My personal experience undoubtedly biases me, but I tend to agree—the problem is coming up with something better.

Science and JoVE

Mon, 24 Aug 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Science and JoVE, the Journal of Visualized Experiments, have partnered to produce and publish scientific videos online. The aim is to enhance scientific articles published in Science through video demonstrations of experimental techniques. See the announcement for a link to the first joint work.

Who Owns Your Data?

Mon, 24 Aug 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Nature's "Great Beyond" blog reports another attempt to force climate scientists to release their data. I'm of at least two minds on this: I believe openness is absolutely crucial to science (and society as a whole), but I understand scientists' concern about being scooped, and equally their concern about having their work misrepresented or quoted out of context. Mostly, I come down on the side of openness—how about you?

It's Like Not Wearing Your Cleats in the House

Sat, 15 Aug 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Carl Zimmer, one of my favorite science writers, recently posted about three new books aimed at scientists: Unscientific America, Am I Making Myself Clear?, and Don't Be Such a Scientist. All three are aimed squarely at the biggest problem modern science faces—the inability of most scientists to explain themselves to non-specialists—and all three are now on my read-soon list. I don't think communication skills will ever be part of this course, but given the problems our planet faces, they damn well need to be part of every scientist's education.

You Can Do a Lot Without Programming

Sat, 15 Aug 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

From Cameron Neylon, a short video showing how to embed molecules from the ChemSpider service into a wiki page. I was surprised and impressed to discover during his visit to Toronto just how little programming Cameron does: mostly, he leverages his understanding of how information moves around the Internet to plumb existing tools and services together. This is part of (or dependent on) what Jeannette Wing calls "computational thinking", and one of the goals for the next revision of this course is to focus more on those kinds of skills.

What Is Open Science?

Mon, 03 Aug 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Over at the OpenScience Project, Dan Gezelter is trying to define what "open science" means. He thinks there are four key points:

Transparency in experimental methodology, observation, and collection of data.
Public availability and reusability of scientific data.
Public accessibility and transparency of scientific communication.
Using web-based tools to facilitate scientific collaboration.

What changes need to be made to this course to prepare people to do these things effectively?

A Good Afternoon

Fri, 31 Jul 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Yesterday afternoon, the students and ninety other guests were treated to six engaging talks about Science 2.0 from Titus Brown, Cameron Neylon, Victoria Stodden, David Rich, Michael Nielsen, and Jon Udell. We'll post slides and video here as soon as we get them; until then, you can catch up on what happened in the FriendFeed room or by reading Steve Easterbrook's real-time blog of the event.

Our thanks once again to everyone who made the day possible:

Martin Fenner on SciBarCamp

Sun, 26 Jul 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Nature's Martin Fenner has blogged a summary of what he heard and saw at SciBarCamp'09 in Palo Alto a couple of weeks ago. Cameron Neylon was there too—in fact, I'd be willing to bet that a healthy number of attendees posted their experiences in some form or another. Maybe the next challenge for scientific publishing is something that will aggregate and summarize disparate reports of such events?

Elsevier's Future, Version 0.1

Tue, 21 Jul 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Elsevier (a big scientific publisher) have unveiled their guess of what the future of scientific publishing looks like. Tabs? Got 'em. Audio? Yup. RSS feeds for updates, corrections, and comments? The raw data, complete with provenance information so other researchers can try to reproduce the results? Still waiting...

Quantum to Cosmos: October 15-25 in Waterloo

Sun, 19 Jul 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Perimeter Institute is organizing a ten-day science festival in Waterloo this October called "Quantum to Cosmos". It promises to take you from the strange subatomic world of the quantum to the outer reaches of the cosmic frontier. And for those who can't make it, all events will be streamed online live and on demand 24 hours a day. Hope to see you there!

Quality Control and Traceability

Mon, 29 Jun 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Nature News recently reported work by Gurt Vriend at other to clean up records in the widely-used Protein Data Bank (PDB). This is great news, but what's missing is a way to track forward from entries to see what already-published papers have relied on data that's now known to be incorrect. One of the many goals of this course is to give scientists the understanding they need to tackle this problem.

The Environmental e-Science Revolution

Mon, 29 Jun 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Steve Easterbrook has summarized some papers from a recent Royal Society workshop on how the web is changing the way environmental science is done. We'll be asking students in the Software Carpentry course how much of this they're doing, and how much they would like to.

Neylon's Head in the Clouds

Mon, 15 Jun 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Cameron Neylon (a guest speaker at this summer's offering of the course) is putting together a paper for the new BMC journal Automated Experimentation titled "Head in the Clouds: Re-imagining the experimental laboratory record for the web-based networked world". It's an excellent description of what a web-native lab notebook could/should look like, and much more besides.

Big Code vs. Science 2.0

Mon, 01 Jun 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The original goal of this course was to give scientists and engineers the skills they needed to build large pieces of software without heroic effort. It's increasingly clear, though, that another goal is equally important: to help them take part in what's sometimes called "Science 2.0": the sharing of information through the Internet. Cameron Neylon's slides on capturing process are, like Jon Udell's on computational thinking, a good summary of the skills required—skills which overlap with, but are distinct from, those needed by software developers.

SECSE Workshop

Mon, 01 Jun 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The latest in a series of workshops on "Software Engineering for Computational Science and Engineering" was held in Vancouver on May 23, just after ICSE'09. Steve Easterbrook has written a good summary of what was discussed, and Jeffrey Carver's longer summary will appear in a future issue of Computing in Science and Engineering.

What If Scientists Didn't Compete?

Fri, 01 May 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Interesting summary in the New York Times of some collaborative science done by Dr. Sean Cutler. One of the goals of this course is to give scientists the skills they need to do things like this routinely. This raises a few questions:

Should one of the "graduation exercises" for the course be to write some kind of search tool (or a wrapper around several search tools)? Finding potential collaborators is sometimes the biggest challenge.
Should more emphasis be placed on sharing and merging data sets? Or on looking for inconsistencies in them?
What else do you need to be able to do in order to collaborate more effectively with your colleagues?

Madagascar Course in Delft June 12-13

Mon, 27 Apr 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Via Victoria Stodden, a link to a course on reproducible research with Madagascar being run in Delft on June 12-13. Focus will be seismic research, but the ideas are more generally applicable.

Cameron Neylon on the Three Opens

Fri, 03 Apr 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Cameron Neylon has another good post up, this one on open data, open source, and open process. Like many advocates of open science, he feels he has to choose between using open source software on one hand, and getting more science done on the other. I sympathize, especially since my colleagues and I have to choose what to use and not use in the July 2009 run of the Software Carpentry course.

Inference for R

Wed, 25 Mar 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Inference for R lets users nest the R statistical language in Word and Excel. It's a neat idea, and another example of the kind of bottom-up innovation that I predict will eventually lead to fully-fledged extensible programming systems. (If Bespin [merged into Ace, as of Jan 2011] made it easier to do things like this, I might understand all the excitement...)

Open Science Panel at Columbia

Wed, 18 Feb 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Via Jon Pipitone: there's a panel discussion tomorrow at Columbia titled "Open Science: Good For Research, Good For Researchers?" Jean-Claude Bradley, Barry Canton, and Bora Zivkovic are all going to be there, and yes, video will be distributed. I'm looking forward to it—it'll be a lot of thinking on computer supported collaborative science in one place.

Enough Players to Hand Out Medals

Mon, 16 Feb 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I blogged last August about the first and second Provenance Challenge, in which the creators of systems for tracking scientific data and workflows were given sample problems, then asked to have their tools answer a variety of questions. (Results from the first were reported in Concurrency and Computation, but ironically, those articles are not openly available; the third challenge will kick off soon.) Chasing down one of those references again, I came across the Open Notebook Science Challenge, which "...calls upon people with access to materials and equipment to measure the solubility of compounds (aldehydes, amines and carboxylic acids are a priority) in organic solvents and report their findings using Open Notebook Science". This isn't quite the same thing as automatically tracking data provenance; instead, it is "...the practice of making the entire primary record of a research project publicly available online as it is recorded". There are lots of interesting research questions for computer scientists here, ranging from privacy and security issues to notification, peripheral awareness, ontological engineering, and more — for example, see Cameron Neylon's latest post synthesizing discussion about using OpenID to identify scientific researchers and their contributions.

Carl Zimmer's Readers' Reading List

Wed, 11 Feb 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Carl Zimmer (prolific and talented writer on biology and evolution) has posted a crowd-sourced reading list of great science writing. Lots of good stuff...

MTEST

Wed, 11 Feb 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Steve Eddins has posted an xUnit-style testing harness for MATLAB called MTEST on the MATLAB Central File Exchange. It's a nice piece of work, and I hope numerical programmers will make heavy use of it.

Python Textbooks for Biotech

Wed, 11 Feb 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Jan Erik Moström recently posted a request for textbook recommendations for teaching programming (using Python) to biotechnology students. He has now posted the responses [no longer online] he received — might be interesting to some readers of this blog.

Sharing Data Isn't That Easy

Fri, 06 Feb 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Interesting post from Systeme D called "ShareAlike considered harmful" (for geo data, anyway). I should give this to my students and ask them to think it through...

Cameron Neylon Says Interesting Things

Wed, 04 Feb 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This time, he has blogged about best practices for making scientific data available. I think this kind of thing will have a much bigger impact on scientists' productivity than any amount of parallel supercomputing, and that computer scientists could have a lot of impact by helping "real" scientists figure out how to do it better.

Communicate First, Standardize Second

Fri, 30 Jan 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

That quote from Jean-Claude Bradley is on slide 34 of Cameron Neylon's presentation "Open Access, Open Data. Open Research?" Very worthwhile...

Web Native Lab Notebooks

Tue, 27 Jan 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Good post from Cameron Neylon about the lab notebooks of the future: "The traditional paper notebook is to the fully integrated web based lab record as a card index is to Google."

A New Kind of Big Science

Fri, 23 Jan 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Via Michael Nielsen, a guest column by Aaron Hirsh at the NY Times on "A New Kind of Big Science":

There is another way to extend our scientific reach, and I believe it can also restore some of what is lost in the process of centralization. It has been called Citizen Science, and it involves the enlistment of large numbers of relatively untrained individuals in the collection of scientific data. To return to our architectural metaphor, if Big Science builds the high-rise yet higher, Citizen Science extends outward the community of villages.

I Want To Be A Number

Sat, 10 Jan 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Via Science in the Open's summary of the workshop on open science at PSB'09, a link to a paper explaining why scientists will want to identify themselves with unique serial numbers.

A Healthy Dose of Scepticism

Wed, 24 Dec 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Titus Brown's latest post (which opens with, "The latest hot idea for making a protein-protein interaction database leaves me lukewarm") should be read by every computer scientist who's "just trying to help":

...while tools can be helpeful, the fundamental problem is much more, well, fundamental: science is hard. Connecting the dots is hard. Thinking clearly about the problem and separating the wheat from the chaff, so to speak, is hard. I worry that for the majority of biologists, new tools are going to be more distracting than helpful. We need to build simpler, easier-to-use tools, not more complicated tools; we need to keep our focus on the goal (solving biological problems) and not just on intermediate stages like improving databases and building better prediction tools.

Google Pulls the Plug on Scientific Data Sharing Project

Fri, 19 Dec 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Google has decided not to launch its scientific data sharing service — another victim of the recession, I suppose. Bummer :-(

The National Academy Would Like to Hear From You

Fri, 19 Dec 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Via Carl Zimmer: the US National Academy of Sciences would like you to fill in a two-minute survey about what science topics you care about most. Current results are:

Three Reasons to Distrust Microarray Results

Wed, 10 Dec 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Interesting post:

...the paper actually demonstrated that is it possible to distinguish microarray experiments conducted on one day from experiments conducted another day. That is, batch effects from the lab were much larger than differences between patients who did and did not respond to therapy... As is so often the case, data were mislabeled. In fact, 3/4 of the samples were mislabeled.

Getting the Science Right-Or At Least, Less Wrong

Thu, 20 Nov 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Via The Great Beyond:

The US National Academy of Sciences has created an initiative that will link TV and movie directors with scientists and engineers to incorporate more accurate science content into entertainment.

Press release here, web site here. That would be a cool job...

Science Lessons for MPs

Mon, 17 Nov 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Via Nature: politicians from the UK Conservative Party will be required to take science lessons. On the one hand, kind of sad that they didn't learn the basics in grade school. On the other hand, yay!, and when will Canadian parties require the same?

Finding and Re-using Open Scientific Resources

Mon, 27 Oct 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Via Cameron Neylon, a workshop in London in November on finding and re-using open scientific resources. Wish I could go...

Science 2.0: the Future of Online Tools for Scientists

Thu, 04 Sep 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A pub night and panel with Timo Hannay, Cameron Neylon, and Michael Nielsen, hosted by Nature Network Toronto

What does the future hold for the way we do science? Are online repositories such as GenBank and the physics preprint ArXiv, or social tools such as Nature Network, about to change science profoundly? To find out, join Nature Network Toronto for an interactive panel discussion over drinks at the pub.

Bil Lewis Works With Biologists...

Fri, 22 Aug 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

...and occasionally finds it frustrating.

Data Provenance Challenge

Wed, 13 Aug 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

John's summary of our discussion about what to teach scientists about reproducible research if they already believe it's a good thing, and want to start doing it reminded me that I never posted about the Provenance Challenge. It has been run twice so far; each time, authors of tools to track the provenance (or lineage) of scientific data have to implemented some workflows, then answers questions about where data came from, what was done to it, and so on. The results of the first challenge are described system-by-system in these papers (sorry, but it's behind a wall — if you google for combinations of the authors' names, you can find PDF preprints). This is a very cool research area, and I hope one of my incoming grad students will want to do something with it.

SciFoo, eGY, and Splitting

Mon, 11 Aug 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

OpenWetware has posted notes from SciFoo. I'm sorry I missed it; looking forward to "Science in the 21st Century" even more than before.
Those notes pointed me at the Electronic Geophysical Year declaration. I agree in principle, but think that something like the Open Source Initiative's "certified open" badge would be more useful than UN-ish statements like, "Effort should be made to identify and rescue critical Earth system data and ensure persistent access to them."
We're very close to re-launching the Software Carpentry site as a wiki, and I'm wondering if I should move the science-and-computing thread out of this blog into a separate one at that site. Thoughts?

Badge of Reproducibility

Wed, 23 Jul 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Coming back to the badge meme from earlier this week, John Cook's new Reproducible Research blog pointed me at this page on the EPFL site advertising a paper called "What, Why and How of Reproducible Research in Signal Processing". Notice the "Reproducible Research" badge? The "add your evaluation" link takes you to a formlet that lets you choose between:

I have tested this code and it works
I have tested this code and it does not work (on my computer)
I have tested this code and was able to reproduce the results from the paper
I have tested this code and was unable to reproduce the results from the paper

It's a good start...

Kevin's Been Busy

Tue, 01 Jul 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Kevin Brown has been busy — he's been coordinating, installing, maintaining, fixing, and figuring out how to use a new $20 million supercomputer for cancer research. No word on how much money will be spent training people how to use it effectively, but hey, I'm easy to reach... :-)

Faking Results

Fri, 06 Jun 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Via BoingBoing, a story about scientists Photoshopping experimental results. Sometimes it's outright fakery; sometimes they're just "cleaning up" or "correcting". Either way, it raises an interesting question: how often are people doing this with computational results? Without scientists' code, or any other way to reproduce their work, we'll probably never know.

The Retractions Just Keep Coming In

Wed, 02 Apr 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Via Titus Brown, yet another published result retracted because of a bug in code.

Scientific Groupware Revisited

Tue, 26 Feb 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Nature would like "to hear from physicists what kind of tools would help in managing the ever-growing tide of information from, and the exciting possibilities of, the internet." (I presume scientists in lesser disciplines will be asked to chip in at a later date ;-). It will be interesting to see how much (if anything) has changed since Jon Udell wrote "Internet Groupware for Scientific Collaboration" eight years ago.

O'Reilly Creating a Web Version of Mathematica

Wed, 20 Feb 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

According to this press release, O'Reilly is partnering with Wolfram Research to create a Web 2.0 version of Mathematica that will "be browser accessible and, utilizing AJAX technologies, will emulate the desktop version of the software with remarkable fidelity". I'll be very interested to see how they handle equations (MathML isn't well supported) and grahpics (IE7 doesn't do SVG, so what, Flash? Silverlight?).

Grumpy Minds Think Alike

Thu, 14 Feb 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Ned Gulley pointed me at this talk by fellow-Mathworkser Steve Eddins that hits a lot of the same core ideas as Software Carpentry. Time to convert SC to a wiki, I think, and start pushing it again...

SciBarCamp in Toronto March 15-16

Sat, 02 Feb 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

SciBarCamp is weekend unconference for scientists, artists, and technologists. The goal is to create connections between science, entrepreneurs and local businesses, and arts and culture. I think this is a great idea, and wish I could attend (I already have a plane ticket for SIGCSE in Portland); it's free, but there is only space for around 100 people, so please register by sending an email to Jen Dodd.

The Burning Man of HPC

Fri, 26 Oct 2007 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

It's a small world: Andy Oram, my co-editor on Beautiful Code, just interviewed Brent Gorda, co-author of the first version of the Software Carpentry course, about the cluster challenge that's running at Supercomputing'07. Don't be fooled by the purple-on-black titles: this is a very cool idea. Students teams get fixed time (and fixed electrical power) to put together a cluster and run some benchmarks. Doing this earned you a PhD twenty years ago; ten years ago, it would have been a line item in a department's budget. It's now a contest for students, and I look forward to finding a cluster in the bottom of a cereal box before I die... ;-)

Openness and (the promise of) XML

Wed, 05 Sep 2007 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Over at O'Reilly, people are gloating about ISO's decision not to ratify Microsoft's OOXML document "standard". At the same time, Jon Udell is saying much more interesting things about what smart document formats could do for the sciences. To quote him quoting Clifford Lynch, "Scientific literature that is computed upon, not merely read by humans."

Random Survey about HPC

Fri, 31 Aug 2007 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This was just forwarded by a friend:

The High Performance Computing (HPC) community is initiating a study to develop education and training pathways in order to enable scientists and engineers to advance the pace of discovery by taking advantage of high performance computing (HPC) and grids.

In order to inform our study, we invite you to complete a survey about your experiences. We would like you to evaluate a list of categories of expertise according to how relevant they are for your work. We also would like to collect additional information about your training experiences.

How Not to Collaborate

Tue, 31 Jul 2007 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I posted a note a while back about an upcoming workshop at Microsoft Research on computational education for scientists. If you read the call for papers, you'll discover that there aren't any instructions on how to submit material; nor is there any contact information, other than the generic "contact us" link the bottom corner that gives you nothing more than Microsoft's generic 1-800 number ("press 1 for Windows Vista support..."). After bouncing around for five minutes, I got a human being who told me that she couldn't give out any phone numbers, but I was welcome to fax my question to them...

Computational Education for Scientists

Wed, 18 Jul 2007 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Via Zachary Dodds: Microsoft is hosting a workshop on computational education for scientists in September. I'm going to try to be there—anyone else?

Win a Trip to Reno!

Thu, 05 Jul 2007 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Francis Sullivan is running a contest: describe what you'd do if someone gave you a desktop petaflop machine, and you could ~~win a trip to Supercomputing '07 in Reno~~. 1000 words, due by August 10.

Inspirational Videos

Wed, 20 Jun 2007 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This, on scientific computing and visualization, is pretty fricken cool.

Nature Precedings

Mon, 18 Jun 2007 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Tim O'Reilly posted a note from Timo Hannay, of Nature magazine, about their new "Precedings", which:

giv[es] researchers a place to post documents such as preprints and presentations in a way that makes them globally visible and citable. Submissions are filtered by a team of curators to weed out obviously inappropriate material, but there's no peer-review so accepted contributions appear online very quickly — usually within a couple of hours.