Software Carpentry on The Carpentries

Software Carpentry Governance Committee - Goals for 2026

Mon, 15 Jun 2026 00:00:00 +0000

Looking for New Committee Members!

The Software Carpentry Governance Committee is excited to share our plans for this year below.

We are looking for a couple of members to join the team, as several of this year’s committee members had to step away due to other commitments. If you are excited about helping to shape the future of the Software Carpentry lesson program and would like to participate, please fill in the application form, or send an email to Toby Hodges to indicate your interest or ask questions.

Calling for new Lesson Program Governors

Tue, 29 Jul 2025 00:00:00 +0000

The Lesson Program Governance Committees (LPGCs) of Library Carpentry and Software Carpentry are looking for new members. This is an exciting time for both projects as they consider expanding their range of workshops. Joining the community governance body is a great way to guide the development and support the progress of a lesson program.

What is a Lesson Program Governor?

LPGCs oversee the strategy of the lesson program, advocate for the project in their networks, and act as a point of contact for the project e.g., liaising with the Board of Directors, the Core Team, and the wider community. Learn more about the powers and responsibilities of the committees in a previous blog post and check out the Lesson Program Governors Handbook for other resources that support this community role.

The Carpentries’ Websites Relaunched!

Mon, 18 Nov 2024 00:00:00 +0000

We are pleased to announce that today, 18 November, we have relaunched the main Carpentries website and the three lesson program websites! All four websites have been redesigned to allow for integrated navigation between them, allowing the viewer to navigate to any other one of the websites from another. The URLs for the main Carpentries website, as well as the Software Carpentry, Data Carpentry, and Library Carpentry websites all remain the same.

Recently, we outlined how the websites were redesigned to provide an overview of The Carpentries, primarily for the general public, Carpentries member organisations, funders, and sponsors, as well as new members to the community or members who would like a refresher on The Carpentries’ core operations and community structure.

Almost Ready: The Carpentries’ new websites and Handbooks

Wed, 30 Oct 2024 00:00:00 +0000

Introduction

We shared that The Carpentries is on a journey towards consolidating its resources and lesson programs - Data Carpentry, Software Carpentry, and Library Carpentry - under one unified identity. To achieve this goal, we migrated the blog posts from the three lesson program websites to The Carpentries main blog. Shortly after that, we announced that the content on these lesson programs’ websites was pared down to only display each lesson program’s lessons, past and upcoming workshops, and its governance structure, with all other content incorporated into the main website of The Carpentries.

Lesson programs' websites content pared down

Tue, 30 Jul 2024 00:00:00 +0000

The content on the three Carpentries lesson programs websites - Data Carpentry, Software Carpentry, and Library Carpentry - has been pared down to only display each lesson program’s lessons, past and upcoming workshops, and its governance structure. All other content pertaining to each lesson program has been incorporated into the main website of The Carpentries. Users of the satellite websites can be redirected from the lesson program websites to The Carpentries website. This change was effected on 29 July 2024.

Translating Software Carpentry lessons into Ukrainian language: a status update

Fri, 17 May 2024 00:00:00 +0000

This blog post was orginally published on Dr. Olexandr Konovalov’s website as “Software Carpentry for Ukraine: status update”. Minor modifications in style have been made to the original so that this version is in keeping with The Carpentries blog post conventions.

This is an overview of the current state of the project of translating Software Carpentry lessons into Ukrainian language. It accompanies the slide of my lightning talk at the Collaborations Workshop 2024.

Lesson programs' blog posts migration

Wed, 20 Mar 2024 00:00:00 +0000

As of today, blog posts will no longer appear on the three lesson program websites. All blog posts that were on the Software Carpentry, Data Carpentry, and Library Carpentry websites are now merged with the blog posts at The Carpentries blog.

In addition to this change, you will also see a pagination feature at the bottom of the list of blog posts to cycle through the list of posts. Also, you will see a new sidebar that lets you view the entire blog archive or browse posts by author or browse posts by tag.

Changes to Software Carpentry's Lesson Program Governance Committee and their work

Tue, 19 Mar 2024 00:00:00 +0000

Hi all!

Like you all, we were surprised, upset, and a little taken aback by the loss of several members of the Carpentries Core Team. Our thoughts and best wishes are with all those who have been affected.

What does a reduction in Core Team support available mean for Software Carpentry Governance and the maintenance and development of lesson materials? These are the most difficult conversations we have had in the past year as the Lesson Program Governance Committee (LPGC).

Welcoming the New Lesson Program Governance Committees

Mon, 08 May 2023 00:00:00 +0000

I am delighted to announce the new Lesson Program Governance Committees for Data Carpentry, Library Carpentry, and Software Carpentry. These committees will provide community leadership for each lesson program, with responsibility to oversee project strategy, advocate for the lesson program and its community, and to communicate with the Executive Council, the Core Team, and the Carpentries community as a whole.

These committees are essential to the long-term health of The Carpentries as a community-led organisation, and I am deeply grateful to the following community members for stepping into the role.

Calling for Volunteers to join a Lesson Program Governance Committee

Tue, 24 Jan 2023 00:00:00 +0000

This post was updated on 3rd February 2023, extending the deadline for applications to the end of Wednesday 15th February 2023 Anywhere on Earth.

The Carpentries has three lesson programs, Software Carpentry, Library Carpentry, and Data Carpentry. In November, I published a post describing the governance structure of each lesson program, and outlining a plan to introduce more community governance in the form of Lesson Program Governance Committees (LPGCs). Today, 24 January we open the call for volunteers to form these new committees. The call for volunteers will remain open until the end of 5th February, and I encourage anyone interested in community leadership and guiding the development of one of the lesson programs to apply.

Planning for Lesson Program Governance Committees

Mon, 21 Nov 2022 00:00:00 +0000

In January 2023, The Carpentries will call for community leaders to volunteer to join Lesson Program Governance Committees for Software Carpentry, Library Carpentry, and Data Carpentry.

Where We Have Been

The Carpentries was formed in January 2018, after Software Carpentry and Data Carpentry merged to form a single organisation. Our third lesson program, Library Carpentry, joined in November that year, and the structure of the organisation has remained relatively stable ever since. Before the merger, Software Carpentry & Data Carpentry had Steering Committees, which combined into a single entity when the two organisations agreed to merge, and became the first Executive Council of The Carpentries, following an election of new members in late 2017.

Running Newcastle University's First Online Software Carpentries Workshop

Thu, 24 Sep 2020 00:00:00 +0000

How it all started

On April 1, 2020, Newcastle University’s Research Software Engineering (RSE) team became one year old. We are a very new team, but have great plans for the future. One is to badge as many as possible of our team as Carpentries Instructors so that we can run Carpentries workshops at the university for research staff and students.

I managed to attend an online instructor training workshop towards the end of March 2020 just as lockdown started. Inspired and enthused at how well the instructors’ workshop worked online I quickly completed the checkout process and then started exploring opportunities for an excuse to run a workshop. Being the first to complete the instructor training, hopefully, a few more of my colleagues will follow in September 2020.

Teaching Software Carpentry at the University of Botswana

Thu, 10 Oct 2019 00:00:00 +0000

On 16-19 September, I taught a Software Carpentry workshop at the University of Botwsana.

The workshop was hosted by the Southern Africa Innovation Support (SAIS) Universities-Industry-Government (UIG) Co-creation Platform and had more than 30 learners.

Because Professor Tshiamo Motshegwa was travelling during the workshop, Badisa Mosesane was in charge of talk about The Carpentries and introduce me.

Badisa Mosesane explaining about The Carpentries.

During the workshop, I taught navigating files and directories with Bash, working With files and directories with Bash, for-loop with Bash, Git, GitHub, for-loop with Python, if-clause with Python, Python functions, Python scripts, debugging Python code, and Make.

Carpentry on a Chromebook

Fri, 15 Mar 2019 00:00:00 +0000

Since grad school, I’ve appreciated the ability to jump between the command line terminal, desktop apps, and the browser. I recently needed a new laptop. I have long been primarily a Mac user, and I looked at new and refurbished Mac, linux, windows, and ultimately Chromebook laptops.

The price of a Chromebook made it an extremely attractive choice, but could it serve as my main laptop? There are many blog posts and reddit subthreads describing hacks for dual boot or VM terminals in unstable developer mode. However, I discovered Google recently released a Linux (beta) app within the stable ChromeOS channel, called “Crostini”. This allows users to simply opt-in to Linux on the settings page; no flags, no developer mode, no complicated hoops. The system requirements for Crostini mean that only a few Chromebooks are currently supported; I went with the Acer Chromebook 11 CB311-8H.

Want to Organise a Workshop on Image Processing?

Mon, 17 Sep 2018 00:00:00 +0000

I wanted to organise a workshop on image processing for PhDs in the medical domain. I usually use Software Carpentry lessons in my workshops, but there are no lessons on image processing. Before starting to re-invent the wheel, I thought I should ask the Software Carpentry community if anybody had given a workshop like that recently.

Hey guys, image processing lesson anyone?

I was hit by a wave of suggestions and existing materials — and although not everything fitted in the scope of my workshop, it sure made my life a lot easier! Here is a summary of some of the useful links which were shared.

Running workshops on limited budgets

Fri, 13 Apr 2018 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Anelda van der Walt and co-host Toby Hodges ran the first themed discussion session in March to brainstorm ideas about running workshops with little to no funding. Attendees from Australia, the US, the UK, and Canada joined to hear and contribute ideas.

Existing Funding Models

The Brisbane model is to use Eventbrite to charge a low fee (around AUD $60) for attendance at workshops. This has a few benefits - waitlists can be managed, the booking widget appears in the workshop website, fees can be easily refunded up to a week before the workshop, and attendance is generally good because people have paid out of their own pockets or have been sponsored by their school to come. The fee covers the costs of room hire and the fairly lavish morning and afternoon teas that keep people from wandering off in search of sustenance during breaks. The Brisbane instructor pool is fairly large so there is no need to fly people in, which means we don’t have to factor in those costs. One drawback is having to pay the catering and room hire costs up-front by personal credit card as the Eventbrite payout comes only after the workshop. This is a break-even charge - we could charge more and return some money to The Carpentries, but our main interest is in keeping costs low.

Software Carpentry: Considering the Future

Fri, 23 Mar 2018 00:00:00 +0000

When Software Carpentry became part of the merged Carpentries this January, outgoing Software Carpentry steering committee members Rayna Harris and Christina Koch were tasked with continuing the work of the steering committee. For the past several weeks, we’ve been working together with a small group to determine what Software Carpentry needs as a sub-community to grow and thrive in the future, and we have identified two primary goals.

One first goal is a restatement of Software Carpentry’s mission. The Carpentries are growing by leaps and bounds, so this seems like a timely moment to stop and think about what Software Carpentry is doing as a branch of The Carpentries and why. This exploration of mission and identity isn’t just so we can have a nice tagline, but to provide direction and a reference point for future decisions that impact the community.

Webinar with Rochelle Tractenberg: Debrief

Tue, 20 Mar 2018 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

On February 2, the Assessment Network held a well-attended webinar with Rochelle Tractenberg. Dr. Tractenberg holds two PhDs, one in Cognitive Sciences and the other in Statistics, and directs the Collaborative for Research on Outcomes and Metrics at Georgetown University, where she is a tenured professor in the Department of Neurology. It was a great privilege for our community to be able to engage in a conversation with her and to learn from her expertise.

Welcome to New Trainers

Wed, 14 Mar 2018 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are excited to welcome seventeen newly badged Trainers to our community. The group recently finished their program with Karen Word and are now certified to teach new Carpentry Instructors.

Join us in welcoming Tania Allard, Anne C. Axel, Mik Black, Murray Cadzow, Mesfin Diro, Caroline Fadeke Ajilogba, Anne Fouilloux, Alejandra Gonzalez-Beltran, Claire Hardgrove, Toby Hodges, SherAaron Hurt, Senzo Mpungose, David Perez-Suarez, Juliane Schneider, Nicholas Tierney, Jessica Upani, Elizabeth Williams.

Revival of the African Task Force

Tue, 13 Mar 2018 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Over the past five years, the Carpentries have gained considerable traction in Africa. Since the first online instructor training in 2015, more than 100 African-based instructors have been trained of which more than 40 had qualified. An estimated 50 workshops have taken place on the continent in countries including South Africa, Namibia, Botswana, Mauritius, Kenya, Ghana, Gabon, and Ethiopia. More than 15 instructors from other continents have visited the continent to join local instructors in building community and teaching workshops. These include instructors from the UK, several states in the USA, Canada, the Netherlands, and New Zealand.

Carpentries for Latin America

Mon, 12 Mar 2018 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The idea of translating Carpentries’ lessons into Spanish and other languages is not new [1, 2, 3]. At the end of 2017, a group of volunteers (see Initial Group members below) embarked on the goal of making Spanish translations a reality. We are very happy that many more volunteers joined us in this effort. We did it, and today we can make the fruit of these months of work known to the entire community!

Carpentries para Latinoamerica

Mon, 12 Mar 2018 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

La idea de traducir las lecciones de Carpentry al español y otros lenguajes no es nueva [1, 2, 3]. A finales del 2017, un grupo de voluntarios (ver Grupo inicial) nos embarcamos en la meta de hacer las traducciones al español una realidad. Estamos muy felices de que muchas más voluntarias y voluntarios se nos han unido en este esfuerzo y hoy podemos dar a conocer el fruto de estos meses de trabajo a toda la comunidad. ¡Lo logramos!

Mentoring Groups are Back!

Mon, 12 Mar 2018 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

In February, we held our mentoring groups virtual showcase, and community members engaged with mentors and mentees virtually to hear about the accomplishments they made over the course of their mentoring experience. Not only were mentees able to finish their instructor checkout tasks and contribute to the CarpentryCon taskforce, mentors started local communities and reconnected with community members globally.

You’ve expressed to us and each other the benefits these mentoring groups have had on your growth in and outside of the Carpentries. You are building powerful, and meaningful connections by developing peer communities. Because of that, we want to continue to support these groups, and give more community members the opportunity to join, either as a mentor, or a mentee.

Announcing the first joint Carpentries Bug BBQ

Thu, 08 Mar 2018 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Bug BBQ is a Carpentry-wide event to improve all our lessons – both existing Software and Data Carpentry lessons, and new releases. We welcome contributions from the Community on all of our lessons. If you want to contribute, but you are not sure where to start, head over to the Bug BBQ website where we will highlight how to get involved.

The Carpentries are preparing to publish the Social Sciences and the Geospatial Data Carpentry curricula on Zenodo. The Social Science Lessons will be published in April, and the Geospatial Lessons in June. This will be the first publication for these lessons. We regularly publish our lessons (SWC, DC) to provide stable identifiers for polished versions of the lessons. This enables referenced discussions of the lesson materials and gives contributors a verifiable product to cite on their CVs or resumes.

Call for Code of Conduct Committee Volunteers

Thu, 08 Mar 2018 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Carpentries are dedicated to developing and empowering a diverse community of enthusiasts around computational methods for research and data science. Whether you are an Instructor, learner, Maintainer, Mentor, Trainer, Executive Council member, Champion, or member of our Staff, you belong to this community. We are committed to creating avenues for you to contribute that are welcoming and inclusive, whether in-person or online.

Our Code of Conduct (CoC) serves a vital role in this commitment, as it outlines our commitment to provide a welcoming and supportive environment to all people regardless of who you are, or where you come from. Our CoC outlines very detailed reporting guidelines and an enforcement policy, so that, should violations occur, members of our community can rest assured that their concerns are being handled appropriately and in a timely manner.

Call for Contributions: Data Carpentry Ecology and Software Carpentry Curriculum Advisory Committees

Tue, 06 Mar 2018 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

In mid-2017, Data Carpentry piloted a Curriculum Advisory Committee (CAC) for the Genomics curriculum. The goals for this committee were to provide general oversight, vision, and leadership for the full Genomics lesson stack, to ensure that the lessons stay up-to-date with existing best practices in the field, and to continue to serve the needs of genomics practitioners attending our workshops. Genomics Curriculum Advisors met after the initial Genomics lesson publication in November to discuss proposed structural and major topical changes to the lessons and will be helping the Genomics Maintainer team to make decisions about these changes as we prepare for a second release this year.

First African Carpentries Instructor Training of 2018.

Tue, 06 Mar 2018 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

On 21-23 February 2018, the fourth South African Carpentries instructor training for African based Instructors took place in Kleinmond, Western Cape. The workshop was funded through the Rural Campuses Connection Project Phase II (RCCPII) Tertiary. The lead Trainer, Erin Becker, is the Associate Director of The Carpentries and currently based near San Francisco, California. Other Trainers included Senzo Mpungose, Scientific Data Center and Infrastructure Manager for Mathematical Sciences at WITS, Caroline Ajilogba, a Postdoctoral Researcher from the Microbial Biotechnology group, NWU Mafikeng Campus, and Martin Dreyer, an eResearch support consultant at NWU IT.

CarpentryCon - Hotel Accommodation Options

Wed, 28 Feb 2018 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Hotel Accommodation for CarpentryCon 2018

We have two hotel options for you for CarpentryCon 2018. Both hotels are offering us a special CarpentryCon rate.

The special rates will be valid for the period of the event and available until the cut-off date; anyone looking to book after that date will be offered the best available rate at the time.

So please book before 18 April if you want to lock in the cheaper rate.

Lesson Infrastructure Subcommittee 2018 February meeting

Wed, 28 Feb 2018 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

On 14 February 2018 at 18:00 UTC+0, the Lesson Infrastructure Subcommittee had their 2018 February meeting. This post will cover the topics discussed and their resolutions.

Software Carpentry and Data Carpentry merge

Software Carpentry and Data Carpentry have merged. Their Executive Council was elected, they have a new fiscal sponsor and a new logo. Also, they have hired Dr. François Michonneau to lead curriculum development efforts.

State of the State: Instructor Checkout

Tue, 27 Feb 2018 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This blog post is the second in a series examining the roles and contributions of the different parts of the Carpentry community. In case you missed it - read the first post in this series, about Maintainers.

Carpentry Instructors are the core of our community. Without Instructors, there would be no workshops. Because of the vital role that Instructors play in advancing the Carpentry mission, we as a community take preparing Instructors very seriously. Before becoming certified Instructors, trainees must show familiarity with our curriculum, demonstrate their teaching skills (with a focus on the Carpentry pedagogical model), and interact with the broader Carpentry community. Software Carpentry Instructors also need to demonstrate familiarity with Git and GitHub.

Library Carpentry Governance - An Update

Tue, 20 Feb 2018 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Eight of us from Australia, the UK, the US, Canada, and South Africa met last week via zoom to discuss taking the Library Carpentry project forward.

We discussed the creation of a new website to replace this one, talked about what constitutes a Library Carpentry workshop, discussed who can teach the material, and and started thinking about what kind of governance structure we need to lead the work from now on. There was an update on the status of the IMLS-funded Library Carpentry Coordinator role to be based at CDL.

Mentoring Groups Showcase their Accomplishments

Mon, 19 Feb 2018 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We just finished our second round of mentoring groups and had an amazing showcase of their work and ideas. In this round, we were more specific and focused on multiple topic areas. There were groups on community building, lesson maintenance, and preparing for instructor checkout.

The feedback and outcomes were great! Participants were able to focus on specific goals, including teaching their first workshop and developing new lesson contribution material. Being part of a group that addressed something important to them, (e.g. developing new communities in Japan and Singapore), made the mentoring groups powerful and enjoyable.

My Favorite Tool - Docker

Thu, 15 Feb 2018 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

What kind of tool is it?

Docker is a virtualization tool.

Why I like it

I use Docker every day - it’s the one piece of software that has changed the way I work in recent years. There are many other approaches to virtualisation but its versatility and ubiquity is compelling. I use it for many purposes - for rapidly obtaining and executing other open source software, for providing a predictable environment in which to test code that I’ve written, and for building and deploying my own and third-party applications. We use it in the RSE team at Imperial College to ensure that others can rapidly install and evaluate our work.

Valerie Aurora to Keynote at CarpentryCon 2018

Wed, 14 Feb 2018 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Carpentries are excited to announce that Valerie Aurora will be one of four keynote speakers at this May’s CarpentryCon in Dublin.

Valerie is a software engineer turned diversity and inclusion consultant.

We want CarpentryCon 2018 to be a truly global, diverse and inclusive event, which is why we are so happy that Valerie has accepted our offer to speak there.

Making the Case for CarpentryCon

Thu, 08 Feb 2018 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Getting a PhD is, was or will be a major career milestone for many in our community.

Yet getting a PhD, momentous as that achievement is, is really just the first step on the career ladder. Employers increasingly want other skills as well.

In all kinds of surveys, the same key skills keep popping up:

Communication skills. Teamwork. Problem solving. Ethical and intercultural understanding. Leadership. Critical thinking.

Carpentry Champions Call

Wed, 07 Feb 2018 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

At the 14 February Carpentry Champions call we’ll be talking about the development of a new resource called the Carpentries Cookbook.

This is an open contribution, community-developed document to help members and supporters share strategies to build local Carpentries communities. We’ve begun this work here.

The Champions call is scheduled for 8pm UTC, 14 February, 2018. The meeting agenda, connection details (via Zoom) and sign up are on the etherpad, and you can check the local time and date for you.

Unveiling Our New Logo

Wed, 07 Feb 2018 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

It must be the worst-kept secret in cyberspace - that the Carpentries have a new logo.

Now that Software and Data Carpentry have merged, we wanted a new logo to celebrate our coming together as the Carpentries, and to give that project its own distinct identity.

The new logo retains a ‘Carpentry’ feel - at the basic level, it represents a wrench around a hexagonal bolt. Yet it also conveys a sense of exhilaration and celebration - that magic moment when you ‘get’ something and your arms shoot up in celebration. There are many such ‘aha!’ moments in Carpentries’ workshops, so it is fitting that our logo represent not just the hard work of learning (the wrench) but the satisfaction of achievement and mastery that we gain (the ‘Yay!’).

Carpentries Transition From Fiscally Sponsored Project to NumFOCUS Community Alliance Member

Tue, 30 Jan 2018 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Software Carpentry and Data Carpentry have combined their separate projects into a new project, now known as The Carpentries.

As part of this transition, Software Carpentry and Data Carpentry are moving from Fiscally Sponsored Projects with NumFOCUS to The Carpentries with Community Initiatives, whose fiscal sponsorship administration services are better aligned with our emerging needs. The Carpentries looks forward to new opportunities with NumFOCUS and will continue to participate in the NumFOCUS Community as a new member of the NumFOCUS Community Alliance.

Be our Advocate

Wed, 24 Jan 2018 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Dear members of our community, we need your help.

Most of you probably know that our inaugural CarpentryCon will take place from 30 May - 1 June, 2018 at University College Dublin.

We would now like to ask for your help in helping us find sponsors to support this event.

We have written blog posts that lay out the case for support from commercial enterprises as well as from support from foundations and other grant-making bodies.

Foundations and Funders: Why Should You Sponsor CarpentryCon 2018?

Tue, 23 Jan 2018 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

CarpentryCon 2018 planning is now well underway. We have chosen a venue and a date, identified keynote speakers, and roughed out a program, but we are still missing one vital piece:

SPONSORS!

CarpentryCon 2018 will teach the practical skills people need to lead 21st century research within academia and industry. By supporting us, you will help develop that next generation of research leaders, people who will attend CarpentryCon to develop the skills needed to take research forward.

Why Should You Sponsor CarpentryCon 2018?

Thu, 18 Jan 2018 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The inaugural CarpentryCon 2018 will take place on 30 May-1 June at University College Dublin.

Planning is now well underway. We have chosen a venue and a date, identified keynote speakers, and roughed out a program, but we are still missing one vital piece:

SPONSORS!

CarpentryCon 2018 will teach the practical skills people need to lead 21st century research within academia and industry.

Workshop Template Enhancement Proposal

Tue, 16 Jan 2018 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Over the last couple of years, many emails and issues on GitHub requested moving the setup instructions in the Workshop Template to a separate page for valid reasons. To provide some reference, see swcarpentry/DEPRECATED-bc#415, swcarpentry/DEPRECATED-bc#729, swcarpentry/workshop-template#194 and swcarpentry/workshop-template#408.

What did we discover during those years and trials?

Finding the balance to accommodate long time and advanced instructors, novice or intermediate instructors and learners is as hard as carrying a watermelon with a tea spoon. So why are we having this discussion again?

How do instructors get placed at workshops?

Sun, 14 Jan 2018 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

If you are a badged Carpentries Instructor, you have most likely received emails about upcoming workshops that need Instructors. After the emails go out, we sometimes get questions about how Instructors are actually chosen and placed, so we want to share our methods with you.

First, it is not first come, first served.

That would not be fair, as that just shows who was lucky enough to be online and able to check their calendars at the moment my email went out. Rather, we give it a few days, or sometimes even a couple of weeks, depending on how quickly the workshop may be coming up.

My Favorite Tool - Midnight Commander

Sun, 14 Jan 2018 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

What kind of tool is it?

A text editor and file manager.

Why I like it

My favourite tool is Midnight Commander and its text editor (which also works as a standalone tool) Mcedit. These two utilities are powerful yet easy to use. They both run on the command line in Unix operating systems, but instead of typing commands, they work mostly by using menus and selecting things with arrow keys or a mouse. This makes them a kind of halfway house between a traditional command line and a GUI utility and in my opinion they bring together the best of both worlds.

The Centrality of the Code of Conduct

Wed, 10 Jan 2018 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This is the first in a series of posts about Carpentries’ teaching practices. Subsequent posts will cover the other practices - live coding, sticky notes, helpers, challenges, etherpads - that make Carpentries’ workshops the success that they are.

I gave a talk recently for the Australian National Data Service on ’teaching the Carpentries way’. Originally I planned to cover six reasons why our workshops are effective, but ended up covering thirteen, with the thirteenth being the Carpentries’ Code of Conduct.

Teaching Statistics in the 21st Century

Tue, 09 Jan 2018 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The late 1980s saw a wave of new undergraduate programs launched in computational physics, as the advent of affordable workstations and PCs made the power to compute and simulate more accessible. A decade later, though, many of those programs had drastically scaled back their ambitions or quietly wound down. The problem wasn’t the programming: the problem was that whenever a curriculum is designed as “X plus some Y”, it’s the Y that gets cut when time runs short, budgets are squeezed, or tough hiring decisions need to be made. “Computational physics” became “the physics we’ve always taught, but with assignments on computers” and then just “the physics we’ve always taught”.

Fourteen and Counting - People's Favorite Tools

Mon, 08 Jan 2018 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Thank you to all the people who have sent in short posts about their favourite tools. We are up to fourteen now, and the variety of tools is great to see. Finding out what other people use - and possibly more importantly - why they use it, and for what, is a great shortcut for researchers. There are a bewildering number of research tools around. Getting a tip from someone in your discipline, or from someone who is doing a similar task to you, can really help.

My Favorite Tool - Twitter

Mon, 08 Jan 2018 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Why do I like Twitter?

At the risk of sounding like a shallow person with a short attention span, Twitter really is a favourite of mine. I tweet as cloudaus.

It’s not really Twitter’s fault that people use it for shameless self-promotion, for marketing, or for fighting culture wars. That side of it has probably turned many people off, which is a shame because Twitter has so many practical uses.

A Successful 2nd RSE Conference

Wed, 03 Jan 2018 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The second RSE conference took place on the 7th and 8th of September 2017 at the Museum of Science and Industry (MOSI). There were over 200 attendees, 40 talks, 15 workshops, 3 keynote talks, one of which was given by our very own head honcho Mike Croucher (slides here), and geeky chats galore.

My Favorite Tool - QGIS

Tue, 02 Jan 2018 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

QGIS (formerly Quantum GIS) is an open source Geographic Information System.

QGIS is capable of advanced analysis and cartography, but I don’t use it for that.

In my research in hydrodynamic modelling, I deal with a lot of spatial data - coastlines, bathymetry, and the like - and this will eventually be processed and plotted using R, MATLAB or Python.

But if I’ve received a file and simply want to take a quick look at it, or if I want to quickly compare two files that use different coordinate systems and see if things line up, most of the time I can throw the file at QGIS and it will show it to me with a few clicks. This approach lacks the reproducibility of a coded solution, but it’s an awful lot quicker for a throwaway visualisation.

My Favorite tool - A Coding Sandbox

Thu, 21 Dec 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

My favorite tool is repl.it, a ‘sandbox’ tool to help me test and refine Python code.

Why do I like it?

It is an online tool for testing/running code. It allows you to write and run code in a number of different languages, and see the results. No R support I’m afraid, but it does have Python (2 and 3), Ruby, Haskell, Go, JavaScript and many more.

Green Stickies for 2017: Report from the Happy Holidays Green Sticky Party Calls

Thu, 14 Dec 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

When Carpentries’ staff score a win, we call it a #greensticky. So for our final community calls for 2017, we hosted two Happy Holidays Green Sticky Parties to celebrate wins we have had this year. The calls highlighted successes, both personal ones, and wins for the wider community.

Among the pluses reported were:

The upcoming Data and Software Carpentry merger
Plans to bring Library Carpentry into the fold
Planning for our 2018 CarpentryCon event
Starting bi-lingual (English/Spanish) teaching demos
Getting ideas for starting a community from the Community Champions calls
Teaching a Software Carpentry workshop for the first time
Successfully using Carpentry-style teaching in an undergraduate programming course
Getting to meet community members in person

Tracy Teal said that calls like these are the #greensticky, and no-one disagreed.

Our 2017 Community Service Award winner: Anelda van der Walt

Wed, 13 Dec 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Carpentries are happy to honor Anelda van der Walt as our 2017 Community Service Award winner. We received seven independent nominations for Anelda this year, which is a testament to her commitment to both individual people and the broader community.

Starting from scratch, Anelda planted the tiny seed that has now become the phenomenal growth of Software and Data Carpentry in South Africa, not to mention its spread to an ever-growing list of other African countries, such as Namibia, Botswana, Ghana, Gabon, Mauritius, and Ethiopia.

Announcing the 2018 Executive Council for the Carpentries

Tue, 12 Dec 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Voting in the election for community governance of the Carpentries (Executive Council, formerly named Steering Committee or Board of Directors) closed last week. Out of the 501 members eligible for voting, 147 ballots were cast (29% turnout).

We are pleased to announce the four newly elected members of the Executive Council:

Raniere and Lex received the highest number of votes and will serve two year terms; Amy and Elizabeth will serve one year terms.

Challenges Assessing Data Science

Mon, 11 Dec 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Assessment Network was established as a space for those working on assessment within the open source/research computing community to collaborate and share resources. During our quarterly meeting in November, we engaged one another in a conversation revolving around data science education. This meeting was organized and hosted online by Kari Jordan, and six community members attended.

First, we discussed the definitions of data scientist, data analyst, and data engineer; second, we worked in pairs on a set of questions about assessing data science education.

My Favorite Tool - Todoist

Mon, 11 Dec 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

My favorite tool is Todoist, a task manager.

Why do I like it?

Todoist is a cross-platform ’to-do’ tool. I went through many different ones before settling on Todoist a few years ago and I now use it to organize everything from my household chores, grocery shopping and gift ideas for family members to my many work projects and non-work projects (things I do that are work-related but not apart of my actual job).

Poster Competition Now Open

Mon, 11 Dec 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Design a Promotional Poster for CarpentryCon 2018

The Carpentries are excited to announce our poster design competition where some lucky member of our community will win a free registration for CarpentryCon 2018.

Are there any artists out there who could design a fantastic poster? If so, we want to see your design!

The winning designer will take home some fantastic Carpentries swag and gain free registration to CarpentryCon 2018. This includes registration for the conference only. It does not include travel to and from Dublin.

When Do Workshops Work? A Response to the 'Null Effects' paper from Feldon et al.

Mon, 11 Dec 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Author: Karen R. Word

Contributors: Kari Jordan, Erin Becker, Jason Williams, Pamela Reynolds, Amy Hodge, Maxim Belkin, Ben Marwick, and Tracy Teal.

“Null effects of boot camps and short-format training for PhD students in life sciences” is the provocative title of a recent article in the Proceedings of the National Academy of Sciences. Those of us who enthusiastically design and deliver short-format training promptly took note, then scratched our heads a bit. We waited a little for a response, wondering if one or more of the programs that participated in the study might step up to their own defense. Nothing happened. We thought about letting it go - we’ve got our own programs, with distinct goals, and our own assessment data, so maybe this broad-brush study isn’t so important. But … it keeps being raised. Someone will bring it up here and there, asking what we think about it. Whenever this paper comes up in conversation, its title certainly throws some weight around.

CarpentryCon 2018 - Website is Live!

Thu, 07 Dec 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are excited to announce that our website for CarpentryCon 2018 is live.

CarpentryCon 2018 will take place at University College Dublin from 30 May - 1 June, 2018, with the overarching theme of “Building Globally, Connecting Locally”.

The three-day event will focus on professional development, community building and networking, as we aim to help skill up the next generation of research leaders through a combination of talks, workshops and breakout sessions.

Celebrate the Wins of 2017: Join the year's final community call

Tue, 05 Dec 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

When Carpentries staff get a win, we call it a #greensticky. These are things to celebrate with a resounding ‘Yay!’

So we decided to turn our regular December community call (our last for the year) into a Happy Holidays GreenSticky Party.

2017 has been a BIG year for our community. We negotiated the Carpentries merger, got planning underway for 2018’s CarpentryCon, trained a whole bunch of new instructors and trainers, and restarted our mentoring groups - and of course a ton of workshops were taught all round the globe. As we wind down for the holidays, it is worth taking time out to think over the great moments of the year.

Lesson Infrastructure Subcommittee 2017 November meeting

Mon, 04 Dec 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

On 16 November 2017 at 15:00 UTC+0, the Lesson Infrastructure Subcommittee had their 2017 November meeting. This post will cover the topics discussed and their resolutions.

Software Carpentry and Data Carpentry merger

With the merger in 2018, some Git repositories will be owned by a new GitHub organization. The Instructor Training course material has already been moved, you can now find it at http://carpentries.github.io/instructor-training/. Date for the migration will be announced in 2018. Instructions for migrating the repository can be find here.

Upcoming Membership Webinar

Mon, 04 Dec 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Organizational memberships are a great way to build your local Carpentries’ instructor community.

Members organizations can run Software and Data Carpentry workshops more often, and can organize workshops for specific communities within their institution. As part of our membership services, we provide members with instructor training and mentorship, and give you easy access to pre- and post-workshop assessment surveys. As your organization leverages their membership to grow your pool of experienced instructors, we seek to find opportunities to connect common needs across our membership.

satRday Cape Town 2018

Sun, 03 Dec 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

satRdays are community-led, regional conferences to support collaboration, networking and innovation within the R community.

satRdays are the brainchild of Stephanie Locke, and were inspired to a large extent by the success of SQLSaturdays. SQLSaturdays focus on developing technical knowledge around SQL Server and related technologies. The first SQLSaturday was held in 2007, and there are now well over 100 events across the globe.

Jessica Upani: Nomination for 2018 Steering Committee

Fri, 01 Dec 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

2018 Election : Jessica Upani

Who am I?

My name is Jessica Upani. I am Namibian and I am a high school teacher. I am also a 4th year Mathematics and Computer Science undergraduate student at the University of Namibia. I am an executive of the Python Namibia society. We run Python workshops, meetups and Conferences all over the country.

Previous involvement.

My involvement with the carpentries started in April 2016 when I was one the instructors that were trained by Aleksandra Pawlik in Potchefstroom, South Africa. I then continued with my training and became a certified Software Carpentry instructor on 7th July 2016. My final checkout was done by Greg Wilson. He gave me my qualification. I did not do this alone. I was mentored by David Perez-Suarez. Since then, I hosted and taught two workshops locally. We paired our first workshop with Bertie Seyffert from South Africa. I also had a chance to meet the new instructors that were trained in Cape Town, May 2017.

Community Building Catchup

Thu, 30 Nov 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The last Carpentries’ Champions call led by Jonah Duckles was well attended, with representatives from Australia, South Africa, Ethiopia, the US, the UK, New Zealand and Norway.

After the introductions and a review of the last call, attendees were divided up into four breakout rooms to discuss (while keeping the following three questions in mind) an event that promoted a sense of excitement about new research tools.

OpenCon in Berlin - Impressions

Thu, 30 Nov 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Along with fellow Software Carpenters Rayna Harris and Paula Martinez, I attended OpenCon 2017 held over the weekend of 11-13 November, 2017 in Berlin. The conference was held in the Harnack Haus in Dahlem, the home of the Max Planck Society, where the friendly ghosts of Einstein, Heisenberg and other stellar scientists smiled on our endeavours to promote open access, open education and open data.

People's Favorite Tools

Wed, 29 Nov 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A big thank you to everyone who has responded so far to the call for short posts about their favourite tools.

So far we have had Paula Martinez on R, with Bianca Peterson enthusiastically seconding, Jeff Oliver sharing his love of Git and GitHub, Kellie Ottoboni talking up IPython, and Thomas Arildsen on how the Jupyter Notebook facilitates his teaching. Juliane Schneider weighed in on the wonders of OpenRefine. Clifton Franklund likes RStudio, while Francesco Montanari is a fan of emacs.

My Favorite Tool - Asking for Help

Tue, 28 Nov 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

My favorite tool is asking for help. You may not think of it as a tool, but it’s something I use frequently to solve a wide range of problems, so I think it qualifies. Whether it’s saying, “I don’t know what to do next – does anyone have any ideas?” when teaching, or cold-calling people to see if they’ll host a workshop, asking for help has probably done more for me than Emacs and version control combined.

Our Steering Committee Candidates

Sun, 26 Nov 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Eight people have nominated to serve on the 2018 Steering Committee of the new, merged Carpentries.

The nominees so far are:

Samantha Ahern (UK)
Martin Callaghan (UK)
Auriel Fournier (US)
Amy Hodge (US)
Lex Nederbragt (Norway)
Raniere Silva (Brazil, but currently in the UK)
Juan Steyn (South Africa)
Jessica Upani (Namibia)
Elizabeth Wickes (US)

There is still time to put your name forward. Nominations will close on 1 December.

Call for Applicants: Mentoring Subcommittee Co-Chair

Tue, 21 Nov 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Dear Carpentries Community,

How would you like to work first-hand on developing our ever growing instructor pool? A Mentoring Subcommittee Co-Chair position has just opened!

The leader serving in this year-long position (December 2017- December 2018) will share duties with the co-chair of the mentoring subcommittee, with a workload averaging 1.5 hours/week.

The expectations of the mentoring co-chairs are to host the monthly mentoring subcommittee meetings, facilitate opportunities for building connections across our community to better serve our instructors, and to manage the weekly instructor discussion sessions hosted on this etherpad.

Amy Hodge: Nomination for 2018 Steering Committee

Mon, 20 Nov 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

2018 Election : Amy Hodge

A Little About Me

Hi all! My name is Amy Hodge, and I work for Stanford University Libraries. I have a PhD from Yale in Molecular Biophysics and Biochemistry, and spent about 10 years working at science database companies, where I discovered the elegance of SQL and the perfect role for me in enabling people to do better science. Since my undergraduate days I’ve been involved in teaching, mentoring, and software training, and my involvement with the Carpentries is a natural extension of these activities and my desire to enable people to do better science.

Samantha Ahern: Steering Committee Nomination

Mon, 20 Nov 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Samantha Ahern: Steering Committee Nomination

How am I involved with the Carpentries?

I have been involved with Software Carpentry for four years. I participated in a Bootcamp back in 2013 and since then have been a helper for a number of Bootcamps and have taught in three in total - two in the last year.

I am a very newly qualified Instructor, literally completing the sign-off tasks in the last few weeks, but have been involved in Education and this type of learning in particular for a long time.

Work first-hand on developing our ever growing instructor community

Mon, 20 Nov 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

How would you like to work first-hand on developing our ever growing instructor community? A Mentoring Subcommittee Co-Chair position has just opened!

The leader serving in this year-long position (December 2017- December 2018) will share duties with the co-chair of the mentoring subcommittee, with a commitment averaging 1.5 hours/week.

The expectations of the mentoring subcommittee co-chairs are to host the monthly mentoring subcommittee meetings, facilitate opportunities for building connections across our community to better serve our instructors, and to manage the weekly instructor discussion sessions hosted on this etherpad.

Martin Callaghan: Nomination for 2018 Steering Committee

Thu, 16 Nov 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

2018 Election: Martin Callaghan

So, who are you?

I’m Martin Callaghan and I’m a Research Computing Consultant (part research software engineer, part trainer, part consultant and part outreach) at the University of Leeds in the UK, where I provide programming and software development consultancy across a diverse research community, including the Arts and Social Sciences, for Cloud and High Performance Computing.

It’s an exciting job where I get to work with researchers to understand their research questions and support them in developing and selecting computational platforms, tools and applications to help them answer those questions.

Raniere Silva: Nomination for 2018 Steering Committee

Thu, 16 Nov 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Who am I?

My name is Raniere Silva. I’m a Brazilian who currently lives in Manchester, UK, where I work for the Software Sustainability Institute as a Community Officer and help run the Institute’s Fellowship Programme. The programme supports, among other activities, Carpentry workshops and this incredible community in the UK and beyond.

Previous Involvement

I discovered Software Carpentry in 2013 after reading this blog post, being redirected to it by this announcement. Months later, I joined the 6th round of instructor training along side Leszek Tarkowski, Michael Crusoe, Christina Koch, Mark Laufersweiler, Jonah Duckles and many other fabulous instructors that I would only meet in person years later.

Lex Nederbragt: Nomination for 2018 Steering Committee

Tue, 14 Nov 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

2018 Election: Nomination by Lex Nederbragt

Who am I?

My name is Lex Nederbragt. I am a bioinformatician (senior engineer) at the Institute of Biosciences, University of Oslo, Norway. I also have a 20% associate professor position at the Institute of Informatics at the University of Oslo. My research and teaching involve genomics, bioinformatics and programming for biologists.

Previous involvement with The Carpentries

I hosted the first Software Carpentry workshop at our university in 2012 (we were very fortunate to have Greg Wilson, one of the founders of Software Carpentry, teach the workshop himself!). I became a Software Carpentry instructor in 2013 and a Data Carpentry instructor in 2016. I have taught at numerous Software and Data Carpentry workshops, both in Oslo, in Norway and abroad. In 2016, I also became an Instructor Trainer and I have co-taught a couple of Instructor Training workshops since then. Together with Karen Lagesen, I started a ‘Carpentry’ initiative at the University of Oslo which now has grown to a dozen local instructors, and another dozen helpers, and many workshops, some full two-day, standard Software or Data Carpentry workshops, but increasingly half to one-day workshops teaching just one lesson of the Carpentries material.
I have contributed to the unix, git, python and make lesson, and this year, I became one of the maintainers of the Data Carpentry ‘Wrangling Genomics Data’ lesson.
Finally, I am very proud of having been one of the authors of the ‘Good Enough Practices in Scientific Computing’ paper that was published earlier this year.

My Favourite Tool - Videoconferencing

Mon, 13 Nov 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

My favorite tool: Videoconferencing

I know most of the blog posts so far in this series have been about the tools people use to enable them to conduct research. However, I feel quite strongly that the video conference systems that allow me to speak to other people have had the most profound impact on my development as a researcher and educator over the past three years.

Nominating for 2018 Steering Committee

Mon, 13 Nov 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Three nominations have come in so far for the new 2018 Steering Committee of the merged Carpentries.

The nominations received are from Juan Steyn (South Africa), Auriel Fournier (US) and Elizabeth Wickes (US).

There is still time to put yours in - nominations close on 1 December. Here is all the information you need to nominate.

There will be a Meet the Candidates opportunity on our upcoming community calls on 16/17 November (date and time will vary, depending on time zones).

Elizabeth Wickes: Nomination for 2018 Steering Committee

Sun, 12 Nov 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

2018 Election: Nomination by Elizabeth Wickes

Who am I?

I’m Elizabeth Wickes and I work as a Lecturer with the School of Information Sciences at the University of Illinois at Urbana-Champaign. I began this position in mid-2017, and previously worked as a Data Curation Specialist for the University Library at UIUC for 2 years. I have my MSLIS, and worked for 5 years at Wolfram|Alpha before beginning my master’s program.

Auriel Fournier: Nomination for 2018 Steering Committee

Thu, 09 Nov 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

2018 Election: Auriel Fournier

I am excited for the opportunity to stand for election for the 2018 Steering Committee of The Carpentries.

About Me

I am currently a postdoctoral research associate at Mississippi State University, based out of the Coastal Research and Extension Office in Biloxi, MS on the Gulf Coast. I am part of the leadership of a large (>45 partner, 200 person) cooperative network which is using structured decision making to inform decision making about bird conservation in response to the Deepwater Horizon Oil Spill. I received my PhD in Biology from the University of Arkansas in May 2017 and before that received a B.S. in Wildlife Ecology and Management from Michigan Technological University. I’m an avid gardener, crochet/knitter and lover of dogs.

CAB-Alliance Bioinformatics Workshop in Franceville, Gabon

Thu, 09 Nov 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Over the past five years, researchers from the University of New Orleans (UNO) in the USA and the Université des Sciences et Techniques de Masuku (USTM), Franceville, Gabon and University of Buea (UB) - have been working closely together as part of a larger collaborative initiative to map patterns of genomic and phenotypic variation in rainforest species across central Africa. This international partnership known as the Central African Biodiversity (CAB) Alliance is made up of a number of institutions in the US, Africa and Europe and is primarily funded through the National Science Foundation’s Partnership in International Research and Education program.

Juan Steyn: Nomination for 2018 Steering Committee

Thu, 09 Nov 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

2018 Election: Juan Steyn

Hi! I’m Juan Steyn, currently employed at the South African Centre for Digital Language Resources as project manager. I’m an enabler at heart and I want to contribute and give back to the African and international Carpentry community. I especially want to help ensure that the core values of the Carpentries get transferred and integrated into how our workshops are conducted within new loci. Africa has A LOT of potential and I would like to play my part to enable and catalyze this potential into action.

Running effective online meetings with Zoom (or Google Hangouts, or ...)

Wed, 08 Nov 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Online meetings are a fact of life for most of us in the Carpentries and other distributed projects. So how do we make them as effective as we can?

This post came about after the Twitter discussion generated by this tweet from Titus Brown.

Here are some points I have thought of to make things work more smoothly. Feel free to comment.

Before the meeting:

Familiarize yourself with the agenda for the meeting. Are there things you need to prepare or figure out beforehand? Are there things you are wondering about? If there is no agenda, ask the chair to send one out.

Apply to Become a Carpentry Maintainer

Tue, 07 Nov 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Software and Data Carpentry are currently accepting applications to join the lesson Maintainer team. Carpentry Maintainers work with the community to make sure that lessons stay up-to-date, accurate, functional and cohesive. Maintainers monitor their lesson repository, make sure that PRs and Issues are addressed in a timely manner, and participate in the lesson development cycle including lesson releases. They endeavor to be welcoming and supportive of contributions from all members of the community.

My Favourite Tool - RStudio

Tue, 07 Nov 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

My favorite tool is RStudio. This integrated development environment (IDE) is really a force multiplier for data analysis and reporting.

RStudio provides an excellent interface to the R programming language (that much should be obvious). However, it is much more than that. This application affords me easy workflows and shortcuts to interact with most of the other tools that I regularly use. These include:

My Favourite Tool - R

Mon, 06 Nov 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

R can do anything - from making presentations, analyzing and plotting data to version control (Git).

I use it for absolutely everything! It’s not difficult to use and it’s free!

– Bianca Peterson, Temporary lecturer (Microbiology) & Project Coordinator (IT), South Africa

Have you got a favourite tool you would like to tell us about? Please use this form to add a bit of detail and we will do the rest. You can read the background to these posts here, or see what other tools people have written about.

Skills Training for Librarians: Expanding Library Carpentry

Mon, 06 Nov 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Post from the University of California Curation Center of the California Digital Library

In today’s data-driven, online and highly interconnected world, librarians are key to supporting diverse information needs and leading best practices to work with and manage data. For librarians to be effective in a rapidly evolving information landscape, training and professional development opportunities in both computational and data skills must be available and accessible.

My Favorite Tool - Emacs

Fri, 03 Nov 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

My favorite tool is Emacs.

Emacs provides an extensible and unified framework to access nice interfaces to several other tools. For instance, it allows you to:

Keep notes, maintain TODO lists, plan projects, edit and automatically export documents to many formats (Org-mode).
Conveniently edit Python source code and, at the same time, send code regions to a Python shell, permitting a piece-by-piece interactive programming (Python mode).

Pack Your Bags for Dublin!

Thu, 02 Nov 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Carpentries are excited to announce that the 2018 CarpentryCon will take place from 30 May - 1 June, 2018 at University College Dublin (UCD).

Yes, we are going to Ireland for the inaugural CarpentryCon!

A huge thank you to UCD for their bid - we are confident that this will be a fantastic venue.

We are also grateful to all the other community members who proposed bids to host CarpentryCon. All were compelling, and it was very hard to select one from so many fantastic options.

My Favourite Tool: OpenRefine

Mon, 30 Oct 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

My favorite tool is OpenRefine.

OpenRefine all the way, baby! I’m the curator of eagle-i, an RDF open access repository of stem cells, viruses, mice, core facilities, lab equipment and people. I use OpenRefine to facet by URI, by date, by label, to clean the data up, search for anomalies and export in csv, Excel or whatever format is needed for my purpose.

I use OpenRefine’s General Refine Expression Language functionality (GREL) to perform complex search and replace, to transform the data, to break up and combine strings.

My Favourite Tool: Jupyter Notebook

Sat, 28 Oct 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

My favorite tool is … the Jupyter Notebook.

One of my favourite tools is the Jupyter notebook. I use it for teaching my students scientific computing with Python.

Why I like it:

Using Jupyter with the plugin RISE, I can create presentations including code cells that I can edit and execute live during the presentation to demonstrate the various aspects of Python that I introduce. It is a very good compromise between running scripts and typing in the command prompt on the projector screen vs. just showing static slides with code.

My Favourite Tool: IPython

Fri, 27 Oct 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

My favorite tool is … IPython.

IPython is a Python interpreter with added features that make it an invaluable tool for interactive coding and data exploration. IPython is most commonly taught via the Jupyter notebook, an interactive web-based tool for evaluating code, but IPython can be used on its own directly in the terminal as a replacement for the standard Python interpreter.

Why I like it

You can run Unix commands directly in IPython. For instance, if you want to load a file from another directory, it is convenient to cd into the directory from within the IPython window.

My Favourite Tool: Git/GitHub

Wed, 25 Oct 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

My favorite tool is …

I love Git and GitHub.

I only use them for work I care about. Examples include lesson development for R workshops, my recent performance review packet, and collaborative projects on species distribution modeling. Oh, and Software Carpentry workshop websites (obviously).

Why I like Git:

The version control system and the companion remote host system (GitHub or Bitbucket or CloudForge, etc.) provide a great versioning and collaboration platform, the highlights of which have been enumerated many times over and in such depth that I won’t talk about them here.

Call for Nominations to Joint Board

Wed, 18 Oct 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

EDITED 2017-11-07: Board of Directors is a legal term and can’t be used for a sponsored project. Changed to “Steering Committee”

Call for Joint Carpentries Leadership: Stand for election to the joint Steering Committee of the merged Carpentries organization

As most of you know, Software Carpentry and Data Carpentry are merging into a new organization, provisionally called “The Carpentries”. This new organization will officially begin on January 1, 2018, with a hugely talented staff and dedicated community already in place.

Our long-term assessment results are in!

Wed, 18 Oct 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A discussion of learner outcomes more than six months after attending a Carpentries workshop

What concrete changes are people implementing in their computational research practices as a result of completing a Carpentries workshop?

Our long term survey report shows that two-day Software or Data Carpentry workshops are effective for increasing skills and confidence, and the adoption of reproducible research perspectives. We see gains in our survey measures for learners’ motivation to continue their learning, change in reproducible research behavior, and frequency of use of computational skills and tools. We find this very exciting, especially since a recent general survey of bootcamps and short-format trainings reports no measurable impact on skill development or research productivity.

RFCs and lessons learned

Tue, 17 Oct 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Thanks to everyone for sharing questions and comments on our recent Request for Comments regarding the upcoming merger of Software Carpentry and Data Carpentry. Now that the official response period for the RFCs has ended, the GitHub issues specific to the RFCs will be closed. We received comments from individuals across our community representing multiple stakeholders, including the Software Carpentry Advisory Council (representing member organizations) and the community at large (through the GitHub repository, private comments via the Google Form, and personal communication). Those of us working on the merger have been actively considering and discussing this valuable feedback and have been identifying areas requiring additional attention and clarification. We will be publishing blog posts over the next few weeks summarizing our thoughts and identifying future actions for each of the RFCs. You are welcome to continue communicating your questions or concerns to us through new issues in the GitHub repo. Below we describe the context behind a few themes and issues emerging from the RFC discussions which may not be immediately apparent but have factored heavily into our approach.

Blogging for the Carpentries - We Want to Hear From You

Mon, 16 Oct 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This is an open invitation to our community members to share their knowledge.

We have a global community working in many different disciplines using a vast range of tools. On our blog, we would like to tap into that community experience and then share that hard-won knowledge.

This kind of information could help people new to a discipline, or might inspire others to try out a tool they have never used before.

My Favourite Tool: R

Mon, 16 Oct 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

My favorite tool for data analysis is R.

I have used it for a few years now, I feel at ease when I need to work with tabular data. Using R makes my work enjoyable. I can clean my data in one place - RStudio - and then work with it creating new graphics. I also enjoy learning new things in R. Its constant development gives you the chance to learn a bit more every day. This is thanks to a huge collaborative community that supports you, with quick answers, with examples, and with new packages.

Carpentries Mentorship Program - 2.0

Thu, 12 Oct 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We’re starting a new round of mentoring groups, centered on specific lessons

Mentorship is an important part of the Carpentry experience. As Instructors, we both teach and mentor our Learners. We also mentor each other as Instructors, learning something new from each other every time we teach and interact with one another. The Mentoring Subcommittee offers guidance to new and continuing Instructors through weekly discussion sessions, where Instructors from the global Carpentry community gather to share their experiences and learn from each other. This is a fantastic opportunity to interact with other Carpentry Instructors from around the world.

All about Membership

Sun, 08 Oct 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are fortunate in the Carpentries to have many member organizations who support our work. However, if we are to continue reaching out to new disciplines and to build communities in under-served countries, we need a broader and more diverse membership base.

What are the benefits of membership?

Members receive priority access to instructor training and guidance about capacity building at their organization. Once institutions have a pool of local instructors, they can readily run low-cost local workshops that teach foundational computational and data skills to their staff and students. Memberships give Software and Data Carpentry revenue to ensure the ongoing development and maintenance of the lessons demanded by research communities. We work to give your local instructors support, ongoing mentorship and a forum for community lesson development. In addition, we have just launched round two of a targeted mentorship program to help new instructors develop their skills.

Mentoring is Back! Round Two of the Carpentries Mentoring Program begins October 25th

Thu, 05 Oct 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Mentoring groups provide experienced instructors with the chance to help small groups develop confidence in teaching, lesson maintenance and community building

The inaugural Carpentries mentoring program was a great success, and we have used the feedback we received from both mentors and mentees to craft a new and improved mentoring experience in round two. The next round will run October 25th - January 10th.

Maintaining Lessons - Community Perspectives

Tue, 03 Oct 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Our September community call on lesson maintenance brought up many good ideas around the lesson maintenance process for Software and Data Carpentry lessons. If you weren’t able to make the call, below is a summary of our discussion and potential avenues for growth.

Our discussion focused around two major questions: what do the lesson maintainers do, and what are some of the reasons to be a lesson maintainer?

Trainer Training Announcement

Tue, 03 Oct 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As the Carpentry community continues to grow, our instructor training is increasingly in demand! In September, we welcomed 13 new Instructor Trainers who will help us to meet that need. We are now accepting applications for the next group of new Trainers. In this round, we welcome all applicants, but are particularly keen to recruit trainers who can work in Latin America, Africa, Australia and New Zealand. We would also like to recruit new Trainers who are fluent in Spanish.

1 - 30 September, 2017: Future of the Carpentries, New Staff Members, Community Service Awards, CarpentryCon

Mon, 02 Oct 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Highlights

All the details on the joint future of the Carpentries explained. Please feel free to ask any questions that you need answered.
The very first ever Data Carpentry in Ethiopia was held and it was an extraordinary experience.
The Lesson Infrastructure Subcommittee had their 2017 September meeting and have come up with some resolutions.
We are pleased to anounce we have three new staff members who will be working for the Carpentries part time: SherAaron Hurt is the new Workshop Adminstrator for the Carpentries, Elizabeth Williams has joined as a part-time Business Administrator, and Karen Word has joined the Carpentries and will be the Deputy Director of Instructor training.
If you feel there is a community member who is working extra hard to help our organization, please consider nominating them for a Community Service Award.

Tweets

Why is Python growing so quickly?
Carpentry instructors - some great advice here on making workshops better for people with Dyslexia.
Want to submit a Carpentries blog post but nervous about GitHub? You can use a form.
Know an unsung hero of the Carpentries? Nominate them for a Community Service Award.
Want to report on a workshop? Write a Software Carpentry blog post.
Read our newsletter, Carpentry Clippings, to keep tabs on our community.
Want more Carpentries? Support our growth by becoming a member organization.
Report of recent in-person staff meeting for both Carpentries - what we discussed.
We now have a discussion group just for #assessment, led by @DrKariLJordan.
Our newsletter will appear on Tuesday. Not a subscriber yet? Sign up here.

General

Everything you need to know about CarpentryCon.

Toads in Vancouver: using Stencila to teach SQL and R at UBC

Fri, 29 Sep 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

One of Stencila’s goals is to create an easy way for people who don’t yet code to learn data science and statistics skills, and to feel comfortable trying out powerful scientific computing languages like R, Python and Julia. We’re doing that by providing interfaces that are similar to the word processors and spreadsheets that they already use. It’s a way for people to “dip their toe” into code - without having to dive into the daunting ocean of IDEs, text editors, packages, version control etc.

Invitación a Participar / Invitation to Participate

Mon, 25 Sep 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Invitación a participar

Las Carpentries, el Nodo Nacional de Bioinformática México (NNB), y la Sociedad Iberoamericana de Bioinformática (SoIBio) les invitan a participar en el proyecto Carpentry para América Latina.

Las Carpentries han generado material para enseñar a investigadores y estudiantes, las habilidades computacionales necesarias para realizar su trabajo de manera eficiente. Actualmente, las carpentries cuentan con más de una docena de lecciones creadas con técnicas de pedagogía actual. Estas lecciones se han promovido en talleres en más de 37 países. Carpentry para América Latina tiene la intención de promover este movimiento con la comunidad hispana.

Introducing Elizabeth Williams and Karen Word

Tue, 19 Sep 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

At our recent in-person staff meeting in Davis, California, we introduced three new members to the team, SherAaron Hurt, Elizabeth Williams and Karen Word. All will be working with the Carpentries part-time.

Elizabeth has joined Software and Data Carpentry in a part time role as Business Administrator to assist with onboarding and supporting Member organizations and general business and financial operations.

Here’s what Elizabeth has to say about herself:

Request for Comment: Share your thoughts on the future of The Carpentries

Tue, 19 Sep 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are now requesting comments on plans related to The Carpentries!

A blog post last week provided history and some context behind the planning still in progress for the eventual merger of Data Carpentry and Software Carpentry into a unified organization, tentatively called The Carpentries. An outline of the planned structure, roles, and responsibilities of The Carpentries is now available, and we request your feedback through a series of Requests for Comment and related GitHub issues by October 6, 2017.

Software Carpentry Lesson Maintenance: Be Part of the Conversation!

Tue, 19 Sep 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Come share your experience and ideas about maintaining the Software Carpentry lessons

We invite all members of the Software Carpentry community to participate in two upcoming events centered around maintaining our lessons:

Community Call this week on Thursday, September 21
Task Force for the future of the Software Carpentry lesson organization

Community Call

We will be discussing lesson maintenance on this week’s community call, happening on Thursday.
On the call, we’ll gather community aspirations and concerns about the current lessons and their maintenance. We also will ask for everyone’s input on how we can best support and recognize the valuable work of our maintainers and contributors. See the etherpad for times, agenda and to sign up. Anyone from the community is welcome to join.

Community Service Awards - 2017 Edition

Mon, 18 Sep 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Is there a Software Carpentry community member you’ve noticed working extra hard to help our organization?

If so, consider nominating this person for a Community Service Award!

The Community Service Award was inaugurated last year to recognize the crucial role of volunteer contributions to the work of the Software Carpentry Foundation. This award acknowledges individuals whose work, in the Steering Committee’s opinion, significantly improves the Foundation’s fulfillment of its mission and benefits the broader community.

The Carpentries' In-Person Staff Meeting

Mon, 18 Sep 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Software and Data Carpentry staff teams met for a two-day in-person meeting on 12-13 September in Davis, California. This was a very productive meeting - more on this below - and it also provided some staff the opportunity to meet others face-to-face for the first time. The team is geographically scattered with staff in the US, Australia and New Zealand. At the meeting, we welcomed new Workshop Administrator SherAaron Hurt, Business Administrator Elizabeth Williams, and Deputy Director of Instructor Training Karen Word (all part-timers) to what will be a merged staff team from 2018, when the planned merger of the two Carpentries will be complete.

New Staff Member

Wed, 13 Sep 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are delighted to announce that SherAaron Hurt has accepted the job of Workshop Administrator with the Carpentries. SherAaron is joining our team of workshop coordinators who manage workshop logistics, communicate with hosts and instructors, and respond to general workshop inquiries.

SherAaron lives in Detroit, Michigan, USA. She has been very active in the National Society of Black Engineers, and has a strong background in logistics, marketing, and training and managing both staff and volunteers. Not only has she planned and run events both large and small, but she has a Masters in hospitality management to back up her experience. She is passionate about Software and Data Carpentry’s mission of teaching foundational computational and data skills to researchers, and is keen to help ‘lighten the load’ for instructors and workshop hosts.

Lesson Infrastructure Subcommittee 2017 September meeting

Tue, 12 Sep 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

On 5 September 2017 at 14:00UTC+0, the Lesson Infrastructure Subcommittee had their 2017 September meeting. This post will cover the topics discussed and their resolutions.

Software Carpentry and Data Carpentry merge

With the upcoming merge, this subcommittee needs to start thinking about streamlining the thinking process and disconnect from the organisations that maintain the lessons. The responsibilities for this subcommittee will stay unchanged:

Reporting on the Second Software Carpentry Workshop in Namibia

Sun, 10 Sep 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Background

After attending this year’s African Instructors Meetup in Cape Town South Africa, Jessica Upani and Gabriel Nhinda from Namibia started laying the groundwork for another Software Carpentry workshop for Namibia. The previous workshop took place about a year ago as part of a 12-month programme to build computing capacity in Africa. This year however, the workshop was initiated, organised and taught by exclusively local instructors and helpers, Gabriel and Jessica with assistance from Ruber.

Software Carpentry Introduces Mauritian HPC Users to Tools for Data Analysis

Sun, 10 Sep 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Square Kilometre Array (SKA) has been branded as one of the biggest scientific projects to date and spans not only country borders but also continents. Although the SKA’s primary aim is to address many of the questions around our universe, the spin-offs of this project will touch people from all research disciplines as well as communities around the SKA sites.

One of the spin-offs is expanded human and infrastructure capacity in terms of High Performance Computing (HPC) in other African countries. The South African Centre for High Performance Computing (CHPC) is involved in a programme named the HPC Ecosystems Project. The Ecosystems Project focuses on the distribution of decommissioned HPC equipment to be used as mid-tier systems at various sites across Africa. This is followed with training of the system administrators to run the equipment.

The First Ever Data Carpentry in Ethiopia

Sun, 10 Sep 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Ethiopian Education and Research Network (EthERNet) from the Ministry of Education in collaboration with the German International Cooperation (GIZ) Sustainable Training and Education Programme (STEP), the Education Strategy Center (ESC) and Talarify organised its first ever Data Carpentry workshop for young academics and researchers in Ethiopia. The workshop was conducted over two and a half days from 14-16 August 2017 at Addis Ababa Institute of Technology (AAiT). The main aim was to increase data literacy for researchers and establish a community of good research data practice in Ethiopia in order to increase the presence of Ethiopian researchers in the global research community. Note: UNESCO Statistics Institute reveals that in 2016 1.1% of the global research community are researchers coming from Sub-Saharan Africa. On average 30.4% from all Sub-Saharan researchers are females, whereas Ethiopia counts 13.3 % female researchers out of all Ethiopian researchers.

Joint future for Software Carpentry and Data Carpentry

Sat, 02 Sep 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

“If you want to go fast, go alone. If you want to go far, go together.”

Software Carpentry and Data Carpentry are sister organizations focused on teaching computational best practices to scientists. They are currently independent organizations with their own fiscal sponsorship, Steering Committees, governance model, and bank accounts. However, as is perhaps no surprise the organizations’ operations have evolved to share memberships, infrastructure for workshop coordination, an instructor training program, and even some staff members. This ‘separate but collaborative’ organizational structure has allowed us to build a shared community of instructors with more than 1000 certified instructors and 47 current Member Organizations around the world.

Waxing poetical in a Software Carpentry workshop

Thu, 31 Aug 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Hi everyone! I just finished running my first segment of a Software Carpentry workshop. I massively overprepared last evening and my notes for the first four lessons of unix shell are on google drive if anyone would care to reuse them.

In any case, one of my students (who had audited one of my other classes) made the following tweet: “Never heard anyone present fundamental computing concepts so poetically as @DenubisX @swcarpentry #MacquarieUni”

All About CarpentryCon

Wed, 30 Aug 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

What is CarpentryCon?

CarpentryCon aspires to become a major learning, skill-building and networking event for our global Carpentries community. We want to help Carpenters develop the skills they need to get the careers they want. We want to help them create supportive local communities based on our values of openness and sharing. We also want to give newcomers the chance to network with more experienced people in our community, so that a range of hard-won knowledge can be passed along to a new generation of leaders.

Feedback of Champions

Thu, 24 Aug 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Jonah Duckles and I hosted our first Community Champions call on 22 August (23 August for us southern hemisphereans). Twenty-five people signed up for the call. We had attendees from the US (several locations), the UK, Canada, the Netherlands, plus me in Australia and Jonah in New Zealand. We also had a range of expertise - some old hands, and some keen to kickstart a brand new community.

Publishing our lessons, Version 2017.08

Thu, 24 Aug 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are pleased to announce the latest publication of Software Carpentry lesson materials, from release Version 2017.08. Although most of our lessons are fairly mature, we had
almost 100 new contributors. We have used this opportunity to improve the process through which lessons are released, some parts of which were previewed in this blog post. This release includes linked ORCID identifiers (visible on the Zenodo page next to the author’s name) and has allowed for individuals to opt-out as contributors. You can learn about the release process on GitHub, and can view this and previous archived versions on the release page.

15 July -15 August, 2017: Writing a blog post, Instructor Training Curriculum, Merger, League of Champions.

Fri, 18 Aug 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Highlights

It should not be a painful experience to write a blog post, so please reach out to us!
Have a look at the approved motions for the Carpentries merger.
We are pleased to announce the Instructor Training Curriculum has been updated and will be released end of August.

Tweets

Our Carpentry Clippings newsletter is out. Read it here if you missed it.
Contributed to a lesson? Let us know so you get credit.
Donate to Software Carpentry to support workshops in new places.
Want to get a Software Carpentry workshop at your institution? Here’s how.
Show your support for opensource datascience — become a NumFOCUS member.
Did you know we have a mailing list to discuss our R lesson + other R-related issues? Post questions there.
Building a community: 3 months of Library Carpentry.

General

Please feel free to provide suggestions to our Community Lead via the Google form.

The Totally Bearable Lightness of Being an Instructor ... So Finish that Training!

Thu, 17 Aug 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Also, there’s this thing at UCLA going on September 7-8 …

Greetings, my Library Carpentry community peeps! Hope your summer is going as well as mine, which means about as fine as the coffee in Twin Peaks, or a Fellini film - take your pick.

Several months ago, I wrote a post about the amazing time I had in Portland at csv,conf and at the Library Carpentry Instructor Training that Belinda Weaver and Tim Dennis taught, ably assisted by myself, John Chodacki and a spatula. We had a wonderful cohort of instructors-in-training, all of whom I know would rock a room with their Bash and OpenRefine instructional skills.

Instructor Curriculum Updated

Mon, 14 Aug 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Sixty-seven pull requests were successfully merged during the recent update of the Software and Data Carpentry instructor training curriculum. Most of these PRs were merged during the recent 24-hour Bug BBQ that ran over 3-4 August. More than 20 instructors and trainers were involved with the update. A few pull requests remain to be merged before the material is ready to be re-released. This should happen by the end of August.

Motions approved for Data Carpentry & Software Carpentry Merger

Mon, 07 Aug 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I am happy to announce that the Steering Committees of both Software Carpentry and Data Carpentry have approved 4 motions regarding the structure and leadership of the mergered Carpentries organization. The approved motions are:

Motion 1

The Board of Directors for the combined organization will be composed of 9 members, each serving a two year term without limits on the number of terms. Five members will be appointed through a process of nomination to the board followed by voting by board members. The other four members will be elected by the membership of the organization.

The Champions League

Mon, 07 Aug 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

When people sign up for our newsletter, Carpentry Clippings, many mention community building as one of their key interests. We are lucky in the Carpentries to have so many people who want to build local groups. These people are our community’s champions. They are the hard workers who volunteer their time to organise and run local workshops, recruit new instructors and helpers, teach and maintain our lessons, serve on committees and task forces, and generally help further the Carpentries’ mission of skilling up researchers to do more efficient, reproducible science.

Keep Calm and Write a Blog Post

Tue, 25 Jul 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Writing a blog post should not be hard. A blank screen can seem very daunting, but if you populate it with a few questions, such as Who? What? Where? When? Why? and How?, then you have the genesis of a post.

Workshop report? Once you’ve answered the questions above, you’re pretty much done. Throw in something that went well or a funny story about something that didn’t, and that’s it.

Feedback on feedback

Thu, 20 Jul 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

When I started as Community Development Lead, I posted a Google form to ask people in our community for feedback on what they thought was the main issue I should address.

I got a range of responses, from concerns about lesson maintenance to a call for more briefings to help people stay connected.

Also raised were instructor retention - how do we prevent burnout so good people don’t leave? - and instructor involvement - how do we increase the numbers of instructors reading our blog posts, signing up for the newsletter, getting involved in community calls and committees like mentoring?

1 - 15 July, 2017: Learner Impact, Instructor Training, Community Building, Author Information.

Mon, 17 Jul 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Highlights

Our pre- and post-workshop surveys show that the Carpentries have a significant impact on the learners.
We are working to better the instructor training material to ensure we have a good up to date curriculum.
You do not have to be an expert to be a good instructor.

Tweets

Are you on our mailing list yet? Sign up here to get our newsletter.
The teams at @swcarpentry and @datacarpentry keep fighting the good fight! Good enough practices in scientific computing.
Help bring workshops to new countries + communities in 2017 - make a donation to Software Carpentry.
Read our experiences in ELIXIR with @swcarpentry and @datacarpentry.
Research libraries and tools curation in digitalhumanities: new report.
Wonder what’s going on @swcarpentry? You can see our community calendar here - it will correct for your time zone.

General

You do not need to be an expert to be a workshop helper.

Credit for lesson contributors

Sun, 16 Jul 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

In 2016, Software Carpentry instituted a twice-yearly release schedule for our lessons (see here for previous releases). Lesson releases, representing a snapshot in time of the material’s development, are published through Zenodo and given a DOI for citation purposes. Releases serve a few purposes, primarily to provide a trackable record of the development of our materials and to allow attribution to community members who contribute to lessons. The lesson maintainers are included as editors, while all other contributors who have made changes (as tracked through GitHub) are represented as authors.

First time for everything

Tue, 11 Jul 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This spring I found myself back in a new situation, an anxious instructor in front of a room full of anxious students. My last teaching was many years back, as a teaching assistant in graduate school, and then a few lectures as part of an intermediate-level course for graduate students. But those were rather theoretical, blackboard affairs. This was my first attempt at teaching software, and with live coding demos. Didn’t Steve Jobs himself say “never give a software demo”? (Okay, maybe not.) In the end though, I was happily surprised by the experience.

Help Update the Instructor Training Materials

Tue, 11 Jul 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Carpentry Instructor Training curriculum helps prepare new instructors to teach Carpentry workshops. It also impacts instructors’ teaching practices when they teach in other contexts, helping to spread the Carpentry pedagogical model and evidence-based teaching practices around the world!

We last published this curriculum in February. Since then, we’ve taught over 150 new instructors at a dozen training events. We’ve also welcomed ten new Instructor Trainers to our community, with fifteen more to join in September. We’ve learned a lot over the past six months and want to incorporate what we’ve learned before our next publication (scheduled for August 10th). Please join the Trainer community in updating these lessons!

Two Workshops at NASA DEVELOP (or, Python De-Fanged)

Tue, 11 Jul 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

On 8-9 June, 2017, Katie Moore, Deputy Data Management Team Lead for the NASA CERES Science Team, and Ryan Avery, Geoinformatics Fellow with the NASA DEVELOP National Program, ran a Self-Organised workshop in Hampton, VA.

On 12-13 June, 2017, Kunal Marwaha, a software engineer with Palantir, Kelly Meehan, Geoinformatics Fellow with NASA DEVELOP, and Ryan ran another Self-Organised workshop in Norton, VA.

Both workshops focused on building skills in NASA DEVELOP participants, who work on 10-week feasibility projects that demonstrate how to apply NASA Earth observations to environmental concerns to enhance project partner decision-making. In the process, both partners and participants gain a better understanding of NASA’s Earth-observing (EO) capabilities and improve their professional and technical capacity to use EO data. For Katie, Kelly, and I, these were the first workshops in which we taught entire lessons and we found it to be an extremely rewarding experience.

Analysis of Software Carpentry Workshop Impact

Mon, 10 Jul 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We’ve begun to look at our pre- and post- workshop surveys and are sharing the draft reports to stimulate conversation about our workshops and their impacts on learners. For this first cut, we looked at our post-workshop survey data.

From this survey data, we can see that learning to program can be quite intimidating for many learners. About 44.5% of Software Carpentry learners who responded to the post-workshop survey feel that at least one of the tools covered in the workshop they attended was either slightly or very intimidating to them before attending a workshop.

Open Channels

Thu, 06 Jul 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I was writing a talk about community building in the Carpentries for the ANDS Tech Talk series, and found myself surprised at how many things we do.

Obviously, workshops are our key community-building activity. They help us reach new learners, give instructors the chance to practise teaching, and draw in new helpers. Many learners and helpers go on to become instructors themselves, which builds further momentum.

Why be a helper at Software Carpentry workshops?

Sat, 01 Jul 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

“I’m not an expert on R”, “I don’t know any Python”, “I’ve never used Git” - these may be true statements but they should never stop you from helping out at a Software Carpentry workshop.

Even the workshop instructors themselves may not be “experts” - and all the better if they are not!

Experts don’t necessarily make the best teachers. Many have lost sight of - or, worse, patience with - the beginner mindset.

15 - 30 June, 2017: Reorganisational Timeline, HPC-in-a-day, Good Enough Practices for Scientific Computing, Opensource Survey.

Fri, 30 Jun 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Highlights

The reorganization timeline has been drafted and awaiting approval from the respective steering committees.
Learn about the HPC-in-a-day course that brings the fun back to HPC.
we are pleased to anounce the publication of Good Enough Practices for Scientific Computing.
We are very excited for Maneesha Sane that is moving from Program Coordinator to Program Manager of the Carpentries.

Jobs

Software and Data Carpentry are looking to hire a part-time Workshop Administrator to help set up workshops and ensure that they run smoothly.

Tweets

Top 10 tips for image acquisition if you are going to do image analysis. Spread the word and happy quantifying!
Just out, “Good Enough Practices in Scientific Computing”
Ten simple rules for collaborative lesson development.
Good essay on value of empathy in work. Empathy of @swcarpentry instructors humanizes technical skills for learners.
How open is your opensource? Check out the Open Source Survey.

General

Our secretary has commited to write a monthly blog post to ensure better transparency between the Steering Committee and the community.
Macquire University held their first ever instructor training and it went very well.
17 workshops were run over the past 30 days. For more information about past workshops, please visit our website.
Upcoming Workhshops:

Job Posting: Workshop Administrator

Mon, 26 Jun 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

With the growth of Carpentry workshops all over the world, we are excited that Maneesha Sane is moving from Program Coordinator to Program Manager of Software and Data Carpentry. As Program Manager, she will continue to be involved in workshop coordination and instructor training and will oversee and ensure the quality and consistency of program operations. She will also work to develop processes, infrastructure and communications to consistently improve the workshop experience for instructors, hosts and learners.

Steering Committee composition and operations

Sun, 25 Jun 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

You've hopefully seen [Belinda Weaver's post](https://software-carpentry.org/blog/2017/06/weaver_cdl.html) announcing her hire as Community Development Lead for Software and Data Carpentry, which also mentioned her stepping down from the Steering Committee. This means we (the Steering Committee) will continue our term of service with only six members. We are confident in our ability to continue our operations as normal despite this change. Thanks to Belinda for her service on the committee over the last year and a half, and we're looking forward to interacting with her in this new capacity.

Good Enough Practices in Scientific Computing

Thu, 22 Jun 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are pleased to announce the publication of “Good Enough Practices in Scientific Computing” by Greg Wilson, Jennifer Bryan, Karen Cranston, Justin Kitzes, Lex Nederbragt, and Tracy K. Teal, which is intended as a complement to 2014’s “Best Practices for Scientific Computing”. As the summary says:

Computers are now essential in all branches of science, but most researchers are never taught the equivalent of basic lab skills for research computing. As a result, data can get lost, analyses can take much longer than necessary, and researchers are limited in how effectively they can work with software and data. Computing workflows need to follow the same practices as lab projects and notebooks, with organized data, documented steps, and the project structured for reproducibility, but researchers new to computing often don’t know where to start. This paper presents a set of good computing practices that every researcher can adopt, regardless of their current level of computational skill. These practices, which encompass data management, programming, collaborating with colleagues, organizing projects, tracking work, and writing manuscripts, are drawn from a wide variety of published sources from our daily lives and from our work with volunteer organizations that have delivered workshops to over 11,000 people since 2010.

HPC in a day?

Tue, 20 Jun 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Preface

In today’s scientific landscape, computational methods or efficient use thereof can be at the heart of the race for new insights, if not at the heart of the race with the academic competition. Learning how to automate tasks from data analysis to data preprocessing as taught by the carpentries provides the technical concepts to enter this race with an advantage.

If you just graduated a software/data carpentry boot camp and want to go beyond your laptop’s capabilities, the next step in academia is typically to approached the data center of your university or alike. There, a user account application has to be filed for the High Performance Computing (HPC) facilities. After some more formalities, storage and computing time is awarded and you can successfully log into the cluster. And then? Then either our carpenter is given a link to the wiki of the local cluster and how to use it. Sometimes there can be a short course on the mechanics of the HPC cluster and how to use tools that are installed on the cluster. That’s it and good luck.

Instructor Training at Macquarie University

Tue, 20 Jun 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Software Carpentry Executive Director Jonah Duckles and I ran instructor training for new member institution Macquarie University in Sydney on 19-20 June.

There were a few firsts at this workshop - Jonah’s first time as an instructor trainer, my first day in the job as the new Community Development Lead for Software and Data Carpentry, and Macquarie’s first-ever instructor training event.

We may also have had the biggest-ever workshop etherpad, coming in at 1400+ lines, but that would be a tough claim to substantiate. Certainly the eventual size of the etherpad mirrored the deep engagement of attendees, who all worked very hard.

New monthly updates from the secretary

Tue, 20 Jun 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

After 6 months, I feel that I’m settling into my role as Software Carpentry Steering Committee Secretary. In an effort to improve communication and transparency between the Steering Committee and everyone else, I am commiting to writing or co-writing a montly blog post to keep you abreast on news from each meetings.

Check out these new blog post summaries of the in-person Software Carpentry Steering Committee Meeting and of the in-person SWC/DC merger group meeting.

1 - 15 June, 2017: Steering Committee retreat, Community Development Lead, Library Carpentry Instructors,CarpentryCon.

Mon, 19 Jun 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Highlights

The Carpentries might look a bit different in future.
The Software Carpentry Steering Committee retreat went really well and they have provided five key areas in which the committee wil provide oversight.
We are pleased to welcome Belinda Weaver as our new Community Development Lead we wish her all the best!

Tweets

For detailed review computation and data: Research Software Engineers report.
A good read. A Comprehensive Survey on Open Source.
Awesome, Tireless, Excitable Library Carpentry Champion Belinda Weaver is the new Software Carpentry Community Development Lead!
If you care about research data practices & have #genomics #bioinformatics experience we’ve got a great opportunity.
Hear about how Software Carpentry built community on campus through University of Oklahoma Libraries from Carl Grant and Sarah Clayton.
Sheffield University is now a Software Carpentry Parnter!

General

Library Carpentry will have some new instructors soon and we cannot wait to welcome them to the instructor community.

Timeline for the Data Carpentry & Software Carpentry Reorganization

Sat, 17 Jun 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

On Jun 8 & 9, we met in-person to continue discussing how to proceed with the merger of the Software Carpentry and Data Carpentry organizations to best achieve our strategic goals and serve our communities. We are happy to report that is was a very productive meeting.

We spent most of the meeting discussing the structure of the future organizational leadership. We have drafted a number of motions that we will present to our respective steering committees for approval at our next meetings. Among the motions, we will propose that the combined Carpentries governing structure be made up of both appointed and elected members. Additionally, we propose that it is in the best interest of the organization to seed the future steering committee with bylaws, which will be enumerated, written, and approved in the coming months.

The road to CarpentryCon 2018

Sat, 10 Jun 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Imagine a global community event where Carpentries members and aficionados come together for a short period to exchange experiences, discuss lessons and debate teaching practices. Visualize a space where keynote speeches are closely complemented by community-driven lightning talks, and round-table panels are supported with ad-hoc informal meetings to share stories about challenges and successes. Finally, to complete the picture, consider a number of satellite events, such as social gatherings, carpentry workshops and new lesson developments. Well, this is our vision for CarpentryCon 2018!

Announcing Belinda Weaver as our Community Development Lead

Wed, 07 Jun 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We’re excited to announce Belinda Weaver has accepted our offer to become the Community Development Lead. Belinda will join the Carpentries staff later this month, please give her a warm and enthusiastic community welcome!

Many of you may already know of Belinda from her work all over the community as a Software Carpentry Steering Committee Member, the Mentorship Subcommittee, the Trainers Subcommittee and a champion and leader of Library Carpentry. As part of our bylaws Belinda will be stepping down from the Software Carpentry Steering Committee on Friday June 16th and the Committee will finish the 2017 term with a membership of 6. For those of you who don’t know her yet, look for her introductory blog post later today. In short, Belinda is a very active community member and a delight to work with, we’re incredibly excited to have her contributing to our community full-time very soon!

The Endless Sprint

Wed, 07 Jun 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The recent Library Carpentry sprint exceeded my expectations, and then some!

Around 107 people signed up on our organising etherpad, either to work remotely or to contribute at one of the 13 sites in seven countries around the globe. That’s a big step up from the 2016 numbers of six sites and around twenty people working on the material.

The sprint was organised as part of the 2017 Mozilla Global Sprint. Library Carpenters comprised around one-sixth of that sprint workforce, and our more than 850 GitHub events - pull requests, forks, issues raised, commits and merges - outpaced the rest of the field by miles.

Instructor Access to Workshops

Tue, 06 Jun 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Software and Data Carpentry share a mission to spread computational and data literacy to the broader research community at a wide range of institutional types across the world. We also share core community values of openness and inclusiveness.

Recently, we learned that some of our host sites have selective criteria that limit physical access to their sites on the basis of nationality. These policies have the effect of excluding some Carpentry instructors from teaching at these sites. We recognize that often these criteria are imposed by the broader security and bureaucratic apparatus at the organization that the host operates within and are beyond the control of our contacts at those sites. Nevertheless, these policies can cause conflict with our mission and core community values.

New Community Development Lead

Tue, 06 Jun 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I am very pleased to be starting as the new Community Development Lead for Software and Data Carpentry. Building communities, helping people connect, fostering skills and learning, brokering solutions - these are the things that drive me.

Jobs I have had include librarian, repository manager, newspaper columnist, Internet trainer and IT project manager. I ran the library system for the City of London libraries in the 1980s, and have just finished working for a non-profit organisation making eResearch infrastructure available to university researchers. In my spare time, I like to read (a LOT), bake bread (and cakes), see as many films as I can, especially foreign/art house ones, and grow herbs and vegetables in the garden.

Apply to Become a Carpentry Instructor Trainer!

Wed, 31 May 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Carpentry community is growing! This month we welcomed ten new Instructor Trainers to our community. Now we are looking for the next group of new Trainers.

Carpentry Instructor Trainers run instructor training workshops, lead online teaching demonstrations, and engage with the Trainer community about how best to train new instructors. Trainers are also actively involved in developing and maintaining the instructor training curriculum. We meet regularly to discuss our teaching experiences and stay up to date on policies, procedures, and changes to our curriculum.

Summary of the 2017 Software Carpentry Steering Committee Retreat

Wed, 31 May 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Software Carpentry Steering Committee met in person on June 7-8 in Davis, California. The committee members have unique journeys that brought them through the community to having a seat on the steering committee. Importantly, we have a unified vision for the future of the Software Carpentry Foundation and the roles of the steering committee. Below is a brief summary of the motions passes and detailed description of the five key areas over which the Steering Committee will continue to provide oversight.

Summary of May Community Call: Restructuring the Carpentries

Fri, 19 May 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Thanks to those of you who attended the May Community Calls, the topic of which was restructuring of Software Carpentry (SWC) and Data Carpentry (DC) into a single unified (umbrella) organization, with SWC and DC continuing as lesson organizations within the umbrella.

As the Chair of the SWC Steering Committee, I am one of the representatives from each organization tasked with developing a plan of action for how the restructuring will proceed over the next ca. six months. I am gratified by how much our community cares about its future, and what we (as community) believe is important to target as we proceed in planning. The following items highlight major discussion points synthesized from both community calls, and a complete set of notes is available on the etherpad.

Three Instructors, Two Coasts and One Spatula

Wed, 17 May 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

In the space of a year, interest and participation in the Library Carpentry community has exploded like an amoeba who over-ate at an algae banquet and attempted one too many pseudopods.

For Library Carpentry, though, this is a good thing; the pseudopods are propelling us forward across institutions, disciplines, and continents. The community, grounded in collaborative tools like Github and Gitter (I always want to type Glitter) is coalescing around lesson development and holding new workshops. Why is the buzz so strong? I think it’s a combination of relentless energy from people like Belinda Weaver and Tim Dennis (to name just a few), the acceptance and active encouragement of new people who want to contribute in some way, and the mutual recognition by all of us that in any one thing, we are all absolute beginners, and we all give each other permission to be terrible until we aren’t.

1 - 15 May, 2017: Lesson Printing, Instructor training, Successes of the Carpentries.

Mon, 15 May 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

##Highlights

Printing our lessons has never been easier!
Sign up for the community call about the restructuring of the carpentries.

##Tweets

Heartworming to hear about the success of Software and Datacarpentry from Dr Karli Jordan.
Share your fun coding applications to inspire learners.

##General

The second in-person Instructor training for South Africa took place and was very successful.
What are the reasons why we volunteer our time so enthusiastically toward the Carpentries.

May Community Call: Restructuring the Carpentries

Sun, 14 May 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Software Carpentry Steering Committee (SWC SC) announced back in February that we were initiating discussion with Data Carpentry about potentially merging into a single organization. Members from the SWC SC and the Data Carpentry Board have been meeting regularly since then, identifying the advantages and logistical challenges associated with a restructured organization. More recently, we passed a motion approving the following structure: “A Carpentry umbrella organization and a set of Lesson Carpentries. The umbrella organization is primarily responsible for finances, administration, and facilitating policy. Each Lesson Carpentry is primarily responsible for lesson development and maintenance.”

What is the reward for empowering others?

Sun, 14 May 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Following a few round-table discussions with Software Carpentry instructors, I gained some insight into what drives so many of us to volunteer our time so enthusiastically. I came to the conclusion that many of us share a sense of accomplishment when we empower others through mentoring and education. Software Carpentry gives us a fountain-full of opportunities to contibute through community developed lessons, community supported workshops, and world-wide community conversations. We start out volunteering a little when there is time, but we soon carve out more and more space in our schedule for Software Carpentry events and activities that provide the opportunity to empower others.

Instructor Training in South Africa 2.0

Tue, 09 May 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

One of the main goals of our community is to make Software and Data Carpentry workshops accessible for all researchers from all disciplines and all regions of the world. In the past three years workshops in South Africa and other African countries have started gradually picking up and I think we may soon see exponential growth!

This has been made possible thanks to a lot of effort and hard work from a number of amazing people, including Anelda van der Walt. In 2014 we realised that the biggest obstacle to running workshops in Africa, was the lack of qualified trainers on the continent. Since then we’ve put significant focus on building a local instructor pool to enable research organisations in the region to run Software and Data Carpentry training. In 2015 several instructors were trained online. Last year North West University at Potchefstroom hosted the first in-person Instructor Training with attendees from South Africa and other African countries. Just last week, between 30th April and 2nd May, we (Anelda, Kari Jordan and Aleksandra Pawlik) ran another Instructor Training in Cape Town with 28 participants.

How to print our lessons?

Mon, 08 May 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

From the time that I joined Software Carpentry, I remember two feature requests that often showed up on our list of issues. The first one was links to help navigate the lessons and the second one was a way to print the lessons. Adding these two features was challenging because the limited information that is available as metadata by Jekyll (or Pandoc, which we used for a while). In addition, we want to have a “real” solution, instead of a workaround that is going to stop work in a few months. Although as software developers we knew that at every release of Jekyll, or any other library, that we are using exist a chance that our pipeline will break.

Plans for Windows Installer

Thu, 04 May 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

In March, we had two email threads on our discuss mailing list related to nano on Windows machines The first thread was about “nano not found”, a bug on our installer that we never managed to trace. The second thread was about some “misbehaviour” of nano on Windows, and the suggestion to use Atom as the default text editor. The suggestion to use Atom started a long discussion where instructors described their reasons to use nano and why Atom is inadequate which lead us to start investigating ways to install nano properly on Windows. On May 3rd, Kate Hertweck, Maneesha Sane, Naupaka Zimmerman, Rémi Emonet and I met online to draft a plan—taking into consideration all of the feedback that was provided in the past month—to solve the problem with nano on Windows.

1 - 30 April, 2017: Library Carpentry, Instructor training, relevant to Learning.

Sun, 30 Apr 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

##Highlights

Great news for Library Carpentry, we won the British Library Labs Awards 2016.
University of Puerto Rico taught an Instructor Training Workshop and it was special in many ways.
Libarary Carpentry along with Mozilla science lab will have a lesson sprint in June 2017.

##Tweets

Have you considered making a donation to Software Carpentry?
Author Carpentry a research training initiative complementary to Software and DataCarpentry.
A great atricle for a Software Carpentry Workshop.
To folks in Software Carpentry: please take the survey in this blog post and sign up for the community calls!

##General

Library Carpentry sprint in June

Wed, 26 Apr 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Lesson maintenance is necessarily an ongoing task, especially with fledgling lessons where workshops taught reveal gaps or issues with existing material. Development of new lessons may also spring from the same source. Who hasn’t thought, after teaching: “Wouldn’t it be great if we could add x?”

But lesson maintenance and new development are probably best done as a shared activity, which is why the Library Carpentry community has signed up to work on lessons intensively during the 2017 Mozilla Science Lab global sprint on 1-2 June. Check out our sign up etherpad for information about our plans.

Instructor Training in Puerto Rico

Mon, 24 Apr 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

On March 24-25, Rayna Harris, Sue McClatchy, and Tracy Teal co-taught an instructor training workshop at the University of Puerto Rico (UPR). This was a very special workshop in many ways, and we are excited to share some of the highlights with you.

Also, be sure to check out an accompanying blog post by Humberto Ortiz Zuazaga about the combined Replicathon and Instructor Training events.

Software tools for unix shell: Survey and April community call

Fri, 14 Apr 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Software Carpentry Code of Conduct includes a somewhat tongue-in-cheek reference to nondiscrimination based on “choice of text editor.” While a choice of software may seem trivial, this particular discussion has led to more than one heated exchange among programmers. In fact, the Discuss list in March was very active concerning issues with the Windows installer, which provides access to the nano text editor through Git for Windows (used in both the Software Carpentry Shell and Git lessons). That discussion eventually transitioned to consideration of alternative software tools we might consider using and moved over to GitHub, where you can read more about the opinions and options voiced by participants.

Optimised for Learning

Fri, 07 Apr 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

In 2014 I fell in love with Software Carpentry. I wasn’t quite sure what it was that was *that* appealing about the workshops, but I knew we had to run more of these in South Africa.

I had been following the activities and blogs of Software Carpentry since 2012 when I started working at a next generation sequencing (NGS) facility as lead (only) bioinformatician. Working with our clients I found that most of them had no idea how to deal with NGS data and very often didn’t know how/where to access computing infrastructure that would allow them to analyse their data.

What's new in Library Carpentry

Thu, 06 Apr 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

There is lots of current Library Carpentry activity. Our big news is Library Carpentry won the British Library Labs award in 2016 for teaching and learning. The prize money has since been used to fund Library Carpentry workshops in the UK, with one in Sheffield just completed, and another upcoming in Taunton in June.

Tim Dennis (UCSD) and I are running Software Carpentry instructor training for 30 librarians in Portland, Oregon, on 4-5 May, after csv,conf. I am then teaching Library Carpentry at MIT with Juliane Schneider (ex-UCSD, now working for Harvard) on 15 May.

Our first work cycle - Prometheus

Sun, 26 Mar 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We’re wrapping up our first work cycle! Here’s what we accomplished over the past six weeks and what we’re still working on. To help with any of these projects, please get in touch!

Planning for Data Carpentry Ecology Lessons Release

What did we do? This cycle we started the process for Data Carpentry’s first lesson release - for our Ecology lessons. During the Issue Bonanza 3/16-3/17, fifteen energetic members of our community and staff submitted a total of 134 issues forcus on items related to our lesson release checklist. Data Carpentry staff and maintainers will be going through these issues in preparation for our Bug BBQ (4/6-4/7). Stay posted for announcements about the Bug BBQ!

Get Involved With Mentoring

Wed, 08 Mar 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Are you a new Software or Data Carpentry instructor? Do you remember what it was like to be a new instructor? Are you interested in improving your own teaching skills? Do you want to connect with other instructors to share teaching ideas and experience?

The mentoring committee is a group of Software and Data Carpentry community members who organize initiatives to support instructors and we’re looking for new members.

1 - 28 February, 2017: Inclusivity, Career Pathway Panel, Community Development Lead, BaseCamp.

Tue, 28 Feb 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

##Highlights

The Carpentries stand for inclusivity and freedom of movement for each and every volunteer, researcher, or professional in industry. If you have been affected by any action or policy that is preventing you from partaking in the carpentries in any way, please contact us.
The second Career Pathway Panel will be held on 1 March 2017, please register.

##Tweets

Have you considered making a donation to Software Carpentry?
ResBaz 2017 was a big success! Thank you to all that participated.
Need a helping hand to get started with coding? Have a look at our website.
Have you subscribed to our monthly newsletter?

##Jobs

A Year to Build a Software and Data Carpentry Community at the University of Florida - The Impact of a Local Instructor Training Workshop on Building Computing Capacity

Wed, 22 Feb 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This January was the one year anniversary of our effort to bring regular Software Carpentry and Data Carpentry workshops to the University of Florida. These workshops are aimed at helping students, staff, and faculty gain the computing skills they need to be successful in our data-driven world. The Carpentries are international organizations that provide materials, instructor certification, and organization of multi-day workshops on basic software development and data analysis tools. In January 2016 a Software Carpentry instructor training workshop held at the University of Florida Informatics Institute provided the start of our efforts. Since then, instructors trained here as well as experienced instructors already in the UF community have held four workshops, reaching 98 participants, including 70 students, 14 staff and 11 faculty. The participants received training in programming languages like R and Python, version control with Git and GitHub, SQL database queries, OpenRefine, and Excel spreadsheets.

Beginning the conversation: Potential merger with Data Carpentry

Wed, 22 Feb 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Newcomers to our community frequently request clarification regarding the distinction between Software Carpentry (SWC) and Data Carpentry (DC). SWC and DC are two independently established and operated organizations that share a common goal of promoting education in reproducible science skills, both in data literacy (DC) and software development (SWC).

Despite our separate organizational structures, SWC and DC maintain close ties, and have begun moving over the last year or so towards increased connectivity. Staff interact with leadership and members of both communities, and are sometimes shared hires between both groups. We offer joint institutional memberships. We’ve worked together to implement shared policies, and have released statements which reflect our commitment to shared values.

Carpentries Career Pathways Panel - Marianne Corvellec, Bernhard Konrad, Aleksandra Pawlik

Wed, 22 Feb 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Wednesday, Mar 1, 3pm PST / 6pm EST / 11pm UTC / 9am AEST (next day)

On Wednesday, March 1, the Carpentries will host the second of three Career Pathway Panels, where members of the Carpentry communities can hear from three individuals in careers that leverage teaching experience and Carpentry skills. (Note: The date of this second panel was shifted from the originally-proposed date of Feb 22 due to scheduling considerations.)

Job Opportunity: Community Development Lead

Tue, 21 Feb 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Software Carpentry and Data Carpentry are hiring a Community Development Lead!

We are excited to announce a position as a full-time staff member to lead community development activities! Software and Data Carpentry have an active global community of researchers, volunteers, learners and member organizations. This person will cultivate and grow this community, developing communication strategies and opportunities for the community to connect with and support each other. You will become an active member of our team of staff and will work with people around the world to advance our mission of building a community teaching digital research skills to make research more productive and reproducible.

How we're getting things done

Thu, 16 Feb 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Adopting work cycles

The Data and Software Carpentry staff have been working together to make progress on projects that are important for our community. To help us do this, we’re trying out a new work process based on BaseCamp’s six week work cycle. You can read their blog post if you’re interested in the details of how structuring a work cycle works. We’re picking a small handful of projects to focus on for each six week cycle, with each staff member working on one or two projects. For each project, we’re setting realistic goals we know we can accomplish before the end of the cycle and holding ourselves accountable to meeting those goals. We’re spending the first two weeks of the cycle planning those goals, dividing up the work into teams, and setting timelines to make sure we stay on track. We envision this workflow having some specific advantages, including:

Standing for Inclusivity

Thu, 02 Feb 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Our goal as Software and Data Carpentry is to build a community teaching digital research skills. We teach essential computing skills to thousands of researchers in academia and industry each year. As an international organization we rely on the volunteer efforts of hundreds of researchers and professionals from around the world. Our volunteers come from diverse backgrounds, countries of origin, and beliefs. These individuals generously donate their time with the goal of helping to speed the discovery of new knowledge and the creation of new technology.

Moving Forward

Tue, 31 Jan 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As of January 30th, Greg Wilson has stepped down from his role as Director of Instructor Training to start a new position as Shopify’s Computer Sciences Education Lead.

Instructor training will continue under the guidance of Erin Becker, Data Carpentry’s Associate Director and Maneesha Sane, Data and Software Carpentry’s Program Coordinator. Erin has a strong background for this role from her postdoc at University of California, Davis studying the effectiveness of training methods for transforming instructional practices. She has been involved with the Carpentry community as an instructor trainer, a member of the Mentorship Subcommittee, and leader of the effort to form an instructor mentoring program.

Announcing the 2017 Steering Committee

Mon, 30 Jan 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I’m pleased to announce our new steering committee for 2017.

The new steering committee is comprised of

![]({{ site.url }}/files/people/hertweck_kate.jpg) Kate Hertweck
![]({{ site.url }}/files/people/harris_rayna.jpg) Rayna Harris
![]({{ site.url }}/files/people/koch_christina.jpg) Christina Koch
![]({{ site.url }}/files/people/kuzak_mateusz.jpg) Mateusz Kuzak
![]({{ site.url }}/files/people/lagesen_k.jpg) Karin Lagesen
![]({{ site.url }}/files/people/mcclatchy_sue.jpg) Sue McClatchy
![]({{ site.url }}/files/people/weaver_belinda.jpg) Belinda Weaver

Turnout and Voting Tallies

We had 31.6% turnout representing 176 of the 557 eligible voters casting a ballot in this election. Official election results are available from electionbuddy.

Carpentries Career Pathways Panel: Raniere Silva, Geneviève Smith, Tiffany Timbers

Fri, 20 Jan 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Tuesday, January 24, 7am PST / 10am EST / 3pm UTC / 2am AEST (next day)

On Tuesday, January 24, the Carpentries will host the first of three Career Pathway Panels, where members of the Carpentry communities can hear from three individuals in careers that leverage teaching experience and Carpentry skills.

Anyone who has taught at a Carpentry workshop in the last three months is invited to join, and should register ahead of time in order to be invited to the call. Registration is limited to 20 people per session, so please only commit if you are sure you will attend. Attendees can register for any number of these sessions. Future, monthly, panel sessions will occur on different days and at different times. Each session will last one hour and will feature a different set of panelists.

South Africa's North-West University Becomes Software and Data Carpentry’s first African Partner

Fri, 20 Jan 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

In November 2014 the first large-scale Software Carpentry event was run in South Africa as part of the eResearch Africa conference in Cape Town. Since then 15 more Software, Data, and/or Library Carpentry events were run by the Southern African community across many disciplines and several institutions.

![Feedback from the NWU Genomics Data Carpentry Workshop in September 2016]({{ site.filesurl }}/2017/01/Feedback-SA-DC-Genomics.png)

The North-West University has been heavily involved in further developing the Southern African Carpentry community. In 2015 NWU led the development of a 12-month proposal that kicked off in April 2016 with the first South African in-person instructor training event. Since 2015 NWU has been involved in four internal Software and Data Carpentry events as well as four events run at other Southern African institutions. The university currently has five qualified instructors as well as two preparing for check-out. Instructors hail from diverse disciplines such as genomics, digital humanities, chemistry, and IT.

1 - 15 January, 2017: CarpentryCon, Steering Committee Elections, Rubic for online Instructor training, TalkPython.

Sun, 15 Jan 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

##Highlights

If you are interested in attending or helping with the planning of CarpentryCon, please sign up!
Steering Committee elections will take place on January 23-27, and all members will be receive a ballot via email. Please look at the membership details to ensure you get the email.

##Tweets

Have you considered making a donation to Software Carpentry?
Talkpython episode #93 Spreading Python through the sciences with Software Carpentry.
Huge thank you to the instructors who joined the new mentoring program!
8 best practices to improve your scientific software.
Sign up for the Collaborations Workshop 2017.
First DTL programmers meeting for 2017 scheduled for January 20 in the Netherlands.

##General

Software Carpentry Steering Committee Candidates 2017

Sun, 15 Jan 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

2017 Steering Committee elections and January Community Call

We are pleased the announce the 2017 Steering Committee elections, which will occur from January 23-27. All members will receive a ballot via email to cast their vote via electionbuddy. Be sure to look at the membership list and let us know if you feel you’ve been left off by mistake; membership details are here.

Community Call

As a preface to the election, we are using our January Community Call to allow the candidates time to introduce themselves. We have seven outstanding candidates to fill the seven Steering Committee seats. Our governance rules require us to hold the election, albeit uncontested, but we also feel this is a vital way for the community to exhibit commitment to the organizational leadership. Please sign up to attend one of two meetings on January 20 to hear about our candidates’ plans for 2017. What would you like to see the Steering Committee accomplish? What are you excited for Software Carpentry to tackle this year?

Announcing for Lesson Infrastructure Subcommittee

Fri, 13 Jan 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

One of the issues that we have with the styles, lesson-example, and workshop-template repositories is that some issues or pull request just sit around for a long time because of the lack of ownership of those repositories. Going on with the Proposal for Lesson Infrastructure Subcommittee, that will try to solve that issue, we would like to announce that Subcommittee calendar for 2017.

Software Carpentry on TalkPython

Thu, 05 Jan 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

On January 3rd, 2017 TalkPython published a podcast interview with our Executive Director, Jonah Duckles, that was recorded on December 6th, 2016. If you have friends who are interested to know about Software Carpentry and are fan of listen podcast (or not) the record is a great way for them to discovery what we are and how we operate.

In the past, our staffs were interviewed in other podcasts by Open Science Radio and Podcast.init that podcast fans should check out.

Announcing the CarpentryCon Proposal

Wed, 04 Jan 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

To date, we have not yet had an opportunity to bring together all the members of our communities. Enter: CarpentryCon! We are proposing a two-and-a-half day conference, tentatively held in May 2017 at the University of Michigan.

We envision an event that brings together members of the Carpentry community, including instructors, partners, advocates, and staff, together with people sharing similar interests from around the globe. We will have a “come and learn” format that is different from most conferences with session on topics such as teaching methods, curriculum development, community organization, and leadership skills. Opportunities will be provided for participants to come together informally to share stories about challenges and successes. There will be at least one session where attendees can share how they have incorporated Carpentry techniques into their own research and teaching, and/or how they have grown their local Carpentry community. The final list of sessions will be determined by the program committee in consultation with the community, balancing “who wants to learn what?” with “who’s willing to teach what?”.

Rubric for Open Instructor Training

Tue, 03 Jan 2017 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Software Carpentry Foundation’s Steering Committee recently resolved to run four open online instructor training sessions per year in order to help support people whom we otherwise might not be able to reach. Since there are likely to be many more applications than spaces, we have developed a rubric for ranking requests for training. This applies only to people who are applying for spots in open training sessions: people who are receiving training through institutional partnership agreements will continue to be nominated by their institution as before.

1 - 31 December, 2016: Instructor Training, Community Service Awards, Career paths, Steering Committee Elections.

Sat, 31 Dec 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

##Highlights

We are pleased to announce the first ever Library Carpentry instructor training in 2017.
The very first Software Carpentry community service awards have been awarded.
Greg Wilson will be taking up a position as Computer Science Educator Lead at Shopify, and will continue to work as a volunteer with the Carpentries.
We are exited about the upcoming series of panel discussions on the variety of career paths available for our community members.

##Jobs

Career Pathways Panel Discussions

Wed, 28 Dec 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Carpentries are excited to announce an upcoming series of panel discussions designed to help our community members become informed about the variety of career paths available to computationally literate members of their fields. Panel discussions will be held virtually in the months of January, February and March (tentative dates, below) with each session featuring 3-4 senior community members in Carpentry-related professions, including: tenured faculty, communicators/consultants, research software engineers, industry scientists, etc.

2017 Election: Belinda Weaver

Fri, 23 Dec 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Hello everyone. I am standing again as a candidate for the Steering Committee, having served on it this year for the first time.

About me

I have been involved with Software Carpentry for about three years. I organised the first workshop in Brisbane in 2014. I certified as an instructor myself in 2015 and taught at two workshops that year. In 2016, I taught at eight workshops, all of which I either organised or helped to organise. During 2016, I and other Queensland instructors have taken Software Carpentry to five cities in Queensland - Brisbane, Townsville, Toowoomba, Gold Coast and Rockhampton - a huge improvement on the number of workshops run in 2015 (three).

2017 Election: Karin Lagesen

Fri, 23 Dec 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

2017 Election: Karin Lagesen

I have been a member of the Software Carpentry Steering Committee for two years, first as secretary and then as vice chair. My involvement with the SCF started in 2012, when I attended a workshop in Oslo, Norway. I signed on as an instructor in 2013 and became an instructor trainer this year. I have around 10 workshops, including instructor training, under my belt. As a member of the SC, I have mainly focused on Software Carpentry operations and on the development of instructor training and instructor trainers. In addition to serving on the Steering Committee, I am also a member of the mentoring committee. Although I have not been able to be as active in the mentoring committee this year, I have previously worked on developing programs focusing on integrating new instructors and on helping new instructors teach their first workshop.

2017 Election: Mateusz Kuzak

Fri, 23 Dec 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Hi, I’m standing up for election to the Software Carpentry Steering Committee 2017.

Me & Carpentries

My background is life science. I currently work at the Netherlands eScience Center in Amsterdam. Apart from being a developer on various research software projects, I’m involved in Center’s training activities and software development best practices.

I got sucked into in Software Carpentry world through Elixir Data Carpentry Pilot organised by Aleksandra Pawlik. During hackathon in Helsinki, together with Francois Michonneau and few other people we kickstarted ggplot2 part in R-ecology-lesson. From day one I learned a lot about the mechanics of SC, lesson development and logistics of the workshops.

2017 Election: Christina Koch

Thu, 22 Dec 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

To begin, I present:

Notable events in Christina’s Software Carpentry career:

2013-April: Graduate from grad school (Masters degree in mathematics).

2013-May: Attend Software Carpentry workshop (and see git and Python for the first time).

2014-January: Teach first Software Carpentry workshop.

2014-March: First time travelling abroad for Software Carpentry.

2014-May: Hear about a job through the Software Carpentry blog.

2014-October: Get the job (which is still my job).

2014-November: Become a lesson maintainer.

2015-December: First time leading an instructor training.

2016-January: Switch roles, from lesson maintainer to mentoring committee chair.

2016-June: Teach first Data Carpentry workshop.

To this list, I would like to add:

Christmas Instructor Discussion

Thu, 22 Dec 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

On December 20th we hosted our two last Instructor Discussion Session from 2016. We started 2016 with Post-workshop Instructor Debriefing Sessions that run twice (in different times) every two weeks and during the year we change its name to reflect the modifications made to better accomodate their inclusion on the Instructor Training pipeline as well as the increase of offers (twice every week). The Steering Committee is very grateful for all the work from the Mentoring Subcommittee as well all Instructor Discussion Session hosts without help would be impossible to provide all the hours of sharing experience among members of our community.

Instructor Traininig Intercontinental

Thu, 22 Dec 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The end of the calendar year is a usual opportunity to make look back at the past 365 days. The summary of mine would be “Instructor Training Intercontinental”. I have always considered myself to be incredibly lucky working with Software and Data Carpentry and certainly in 2016 I hit another jackpot. I run 7 Instructor Training workshops, in 6 countries on 3 continents. But the highlight was, as it has always been with SWC and DC, the people.

2017 Election: Kate Hertweck

Mon, 19 Dec 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I am excited to stand for election for the 2017 Software Carpentry Steering Committee (SWC SC). I hope you’ll consider supporting me in this year’s election so I may continue to serve our community.

Previous experience with Software Carpentry:

Some folks might consider it insanity for a tenure-track assistant professor to volunteer so much energy to the Carpentries, but in reality, the time I’ve spent on the SWC SC has been essential to my development of an effective model of leadership. The following points highlight my contributions to our community:

2017 Election: Rayna Harris

Mon, 19 Dec 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I am really excited to stand in election for the 2017 Software Carpentry (SWC) Steering Committee so that I can continue to serve this amazing community.

##Software Carpentry Involvement I was first exposed to Software Carpentry by April Wright, who suggested that I attend the Instructor Training Workshop at UC Davis in January 2015. In 2015 I co-taught workshops at UT Arlington and New Mexico State University, I co-organized the Austin-based Instructor/Helper Retreat, served on the Mentoring and Assessment Subcommittee.

Rayna Harris's Year in Summary 2016

Mon, 19 Dec 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A year in review

I want to thank you for giving me the prilege to serve on the Software Carpentry Steering Committee. Here is a very brief recap of the year. Click this link to read my 2017 election post.

A year ago, I said I would focus my efforts on:

integrating data from the mentoring and debriefing sessions with the assessment surveys to understand the degree of workshop effectiveness

Software Carpentry workshop in severe conditions

Mon, 19 Dec 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Usually the main struggles preceding Software and Data Carpentry workshops involve the laptop setup, signposting the room and making sure the washrooms are unlocked. But the recent workshop in New Zealand, held at National Institute of Water and Atmospheric Research (NIWA) was facing much more extreme conditions.

The day before the workshop New Zealand was hit by what is now known Kaikoura earthquake. Wellington, where the NIWA offices are, was also affected. Fortunately, NIWA buildings remained safe and the hosts, Fabrice Cantos and Wolfgang Hayek, decided to go forward with the workshop.

Teaching Support IT Job at UCL Physics and Astronomy

Thu, 15 Dec 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We’re looking for someone with IT skills, and interests in teaching and physics, to work with us at the Department of Physics and Astronomy at UCL (University College London).

This is a system manager job with an emphasis on supporting our teaching, and will involve a wide range of responsibilities, including managing a Linux cluster and interfacing PCs to lab equipment as well as providing first-line support for the university Windows environment.

Next Steps

Wed, 14 Dec 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Software Carpentry has accomplished an amazing amount over the past six and a half years, but a new opportunity has come up for me here in Toronto, and after a great deal of thought, I’ve decided to pursue it. At the end of January 2017, I will be taking a position as a Computer Science Education Lead at Shopify, where I will help with their CS education partnerships. I’m excited to have a chance to work for change locally, but also look forward to continuing to be involved in the Carpentries as a volunteer, and to many more discussions of teaching, open science, and how awful Git truly is.

Community Service Awards 2016

Tue, 13 Dec 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are very pleased to announce the recipients of the Software Carpentry Foundation's first community service awards, which are given annually to recognize work that improves the Foundation's fulfillment of its mission and benefits the broader community:

After becoming an instructor, Christina Koch immediately started taking on extra responsibilities. She has taught countless workshops, been a lesson maintainer, played a major role in the mentoring committee, and become an instructor trainer.

Don't forget to submit your post to stand for the 2017 Steering Committee

Mon, 12 Dec 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

If you’ve considered running for the 2017 Software Carpentry Steering Committee, please submit your blog post announcing your intentions by December 23. More information about what to include in your post can be found in the original announcement.

Feedback on Communications

Mon, 12 Dec 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Software and Data Carpentry have at their core a collaboration-driven ethos, and communication is key to that collaboration. We’re reaffirming our commitment to open and transparent communication, because we know we can do better! We want to give community members opportunities to talk to each other, staff and Steering Committees, to get updates on efforts and activities and to generate ideas and participate in discussions.

Instructor Training for Library Carpentry

Thu, 08 Dec 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are pleased to announce that we are partnering with csv,conf (a community conference for data makers everywhere) to run an instructor training class specifically geared for people interested in Library Carpentry. The class will take place in Portland, Oregon, on May 4-5, 2017; for details, please see the full announcement.

Making art with Python: Projects after Software Carpentry

Thu, 08 Dec 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This March I signed up for a Software Carpentry class and learned Python for the first time. I had a great time at the workshop, and I wanted to share one of the first Python projects I completed thanks to Software Carpentry.

I originally signed up for the workshop for my PhD research in mosquito behavior. I needed to automate some video analysis tasks, and several friends recommended learning Python. I ended up making the video analysis work (thanks Software Carpentry!), but this blog is actually about a Python art project that I worked on right after finishing the class.

2017 Election: Sue McClatchy

Mon, 05 Dec 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Hi,

I’m a bioinformatician at a research lab in rural Maine, U.S.A. My path here has been winding, varied, and fraught with good luck. I had the good fortune to find Software Carpentry some years ago, and in my travels have never found anything quite like it. I’m honored to be part of this community and now want to give back by contributing my experience and expertise.

15 - 30 November, 2016: Instructor Training, UCSF Library, Code of Conduct, Announcement List, Steering Committee minutes.

Wed, 30 Nov 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

##Highlights

We are gratefull that Open Instructor training is going well with participants from all over the world.

##Jobs

The UCSF Library’s Data Science Initiative is looking for a biomedical researcher, please visit the UCSF recruit site.

##Tweets

Remember to abide by the Code of Conduct.
Incase you missed it, the low volume announcements list is up, please sign up.
Think of replacing awk with bioawk, you can contibute to the development.

##General

UCSF is Hiring

Wed, 30 Nov 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The UCSF Library’s Data Science Initiative is hiring! We are looking for a biomedical researcher with an entrepreneurial spirit and a passion for programming in R/Python, bioinformatics, data curation, statistics, data visualization (or all of the above) to serve as the Scientific Lead for our Data Science Initiative. We are taking a broad approach to data science, and are looking for someone who will work to identify the data science needs of the UCSF research community, help build a Library-based hub for data science activities, develop programs and events, and teach workshops and classes. To find out more about this position please visit https://aprecruit.ucsf.edu/apply/JPF01144.

Minutes of Steering Committee Meeting

Mon, 21 Nov 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Steering Committee informs that the minutes of their third quarter are now linked on [the Software Carpentry Foundation page]({{site.url}}/scf/) as well on the [README of the board GitHub repository]({{site.github_url}}/board). If you have critics and suggestions related to the minutes please send those by email to our Secretary, Raniere Silva, at [raniere@rgaics.com](mailto:raniere@rgaiacs.com).

In the forth and last quarter, the Steering Committee will focus to wrap and documenting some procedures to make the transition to the next Steering Committee smoother than this year, something that started with the Amending Steering Committee Election Procedures that you voted last October.

Open Instructor Training

Sat, 19 Nov 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

(Originally posted on the Data Carpentry blog.)

After workshops and conferences, we frequently get questions from people who are interested in teaching with the Carpentries. We’re overjoyed by this interest and excited to bring more committed and enthusiastic instructors into our community. Unfortunately, until recently, we haven’t had the resources to open up our instructor training program, and have been holding training events primarily with Partnering institutions.

1-15 November, 2016: Research in Action, The Jackson Laboratory, Reproducability of Research, Announcement List, RStudio.

Fri, 18 Nov 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

##Highlights

Listen to the postcast about Research in Action by Dr. Tracy Teal, Executive Director of Data Carpentry.

##Jobs

The Carter Lab at The Jackson Laboratory is seeking a Postdocoral fellow in Computational Genetics.
The Carter Lab at The Jackson Laboratory is seeking an Associate Research Scientist in the genetics of Alzheimer’s disease.
The Churchill Lab at The Jackson Laboratory is seeking a Postdortoral fellow in systems biology.
The Jackson Laboraty has a scientific sofgtware engineer position avalable.

##Tweets

Programming with GAP

Fri, 18 Nov 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Links to the GAP lesson in this blog post have been updated to reflect the lesson moving to The Carpentries Incubator.

Software Carpentry is more than just a set of workshops and lessons. It is also a way to develop lessons, one that we have used successfully to create a lesson on Programming with GAP.

GAP is an open source system for discrete computational algebra. It provides a programming language with the same name; thousands of functions implementing various algebraic algorithms; and data libraries containing extensive collections of algebraic objects. GAP distribution includes its detailed documentation; even more materials on learning GAP and on using it in teaching a variety of courses are available on GAP homepage here.

Software engineer position at The Jackson Laboratory

Tue, 15 Nov 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A scientific software engineer position is available immediately at The Jackson Laboratory in Bar Harbor, ME. This position reports to the Associate Director of Jackson’s Computational Sciences Scientific Computing (CSSC) team that is primarily responsible for developing software applications for scientific research programs. The individual in this position is responsible for developing software applications and systems to support genetics and genomics research including but not limited to web-based technologies and systems. A senior level candidate will lead development of complex projects, from high-level requirements, involving teams that may include other software developers, bioinformatics analysts, statisticians and scientists.

Computational Genetics Postdoc Position with The Jackson Laboratory

Fri, 04 Nov 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The [Carter Lab] (https://www.jax.org/research-and-faculty/research-labs/the-carter-lab) at The Jackson Laboratory is seeking a Postdoctoral Fellow in computational genetics and systems biology. Our group is developing novel computational methods to derive biological models from large-scale genomic data. The strategies we pursue involve combining statistical genetics concepts such as epistasis and pleiotropy to understand how many genetic and environmental factors combine to control disease-related processes in animal models and human studies. We are especially interested in dissecting the genetic complexity of autoimmune disease, neurodegeneration, and cancer.

Research Scientist Position at The Jackson Laboratory

Fri, 04 Nov 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Carter Lab at The Jackson Laboratory is seeking an Associate Research Scientist in the genetics of Alzheimer’s disease. [Our group] (https://www.jax.org/research-and-faculty/research-labs/the-carter-lab) is developing novel computational methods to derive biological models from large-scale genomic data. The strategies we pursue involve combining statistical genetics concepts such as epistasis and pleiotropy to understand how many genetic factors combine to control disease-related processes in neurodegeneration. We are therefore seeking an individual with expertise in epistasis analysis as it pertains to studies of Alzheimer’s genetics in humans.

Systems Biology Postdoc Position with The Jackson Laboratory

Fri, 04 Nov 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The [Churchill Lab] (http://churchill-lab.jax.org/website/) at The Jackson Laboratory is seeking a Postdoctoral Fellow in systems biology. Our group applies a systems approach to study the genetics of health and disease. We develop new methods and software to improve the power of quantitative trait locus mapping and high throughput sequence analysis. We are especially interested in the genetics of aging and metabolic disorders.

RStudio Training and Consulting Directory

Wed, 02 Nov 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

RStudio maintains a directory of people who provide training and consulting for R using their flagship product. If you have taught R for Data Carpentry or Software Carpentry, have an established training and/or consulting practice, can provide some positive references from previous clients, please contact Garrett Grolemund about adding your profile.

A Reproducibility Reading List

Tue, 01 Nov 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Prof. Lorena Barba has just posted a reading list for reproducible research that includes ten key papers:

Schwab, M., Karrenbach, N., Claerbout, J. (2000) Making scientific computations reproducible, Comp. Sci. Eng. 2(6):61–67, doi: 10.1109/5992.881708
Donoho, D. et al. (2009), Reproducible research in computational harmonic analysis, Comp. Sci. Eng. 11(1):8–18, doi: 10.1109/MCSE.2009.15
Reproducible Research, by the Yale Law School Roundtable on Data and Code Sharing, Comp. Sci. Eng. 12(5): 8–13 (Sept.-Oct. 2010), doi:10.1109/mcse.2010.113
Peng, R. D. (2011), Reproducible research in computational science, Science 334(6060): 1226–1227, doi: 10.1126/science.1213847
Diethelm, Kai (2012) The limits of reproducibility in numerical simulation, Comp. Sci. Eng. 14(1): 64–72, doi: 10.1109/MCSE.2011.21
Setting the default to reproducible (2013), ICERM report of the Workshop on Reproducibility in Computational and Experimental Mathematics (Providence, Dec. 10–14, 2012), Stodden et al. (eds.), https://icerm.brown.edu/tw12-5-rcem/ // report PDF
Sandve, G. K. et al. (2013), Ten simple rules for reproducible computational research, PLOS Comp. Bio. (editorial), Vol. 9(10):1–4, doi: 10.1371/journal.pcbi.1003285
Leek, J. and Peng, R (2015), Opinion: Reproducible research can still be wrong: Adopting a prevention approach, PNAS 112(6):1645–1646, doi: 10.1073/pnas.1421412111
M. Liberman, “Replicability vs. reproducibility — or is it the other way around?,” Oct. 2015, http://languagelog.ldc.upenn.edu/nll/?p=21956
Goodman, S. N., Fanelli, D., & Ioannidis, J. P. (2016). What does research reproducibility mean? Science Translational Medicine 8(341), 341ps12–341ps12, doi: 10.1126/scitranslmed.aaf5027

The papers themselves are great, but what really adds value is the way they're ordered, analyzed, and connected. If you're trying to make sense of all this, or trying to help others do so, it's a great place to start.

Tracy Teal on Research in Action

Tue, 01 Nov 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The latest podcast from Research in Action features Dr. Tracy Teal, Executive Director of Data Carpentry. In 33 minutes (plus a couple of bonus clips), Tracy talks about the mission of Data Carpentry, how it came to be, and how people can get involved as learners, instructors, and lesson developers. It’s a great introduction for newcomers, and has a lot of tidbits that long-time participants will find fun and interesting as well.

18 - 31 October, 2016: Community Call on Assessment, Steering Committee, Library Carpentry, Machine Learning.

Mon, 31 Oct 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

##Highlights

Several members of our community have published a new paper on Ten Simple Rules for Digital Storage.
Library Carpentry is one year old and still gaining momentum.
The Steering Committee election is taking place in January 2017, please consider standing.

##Tweets

@UniOslo is an affiliate of the @Swcarpentry foundation
The latest comnbined capentries newsletter is now online.

##General

A new book on machine learning with python has been released. You can find it at O’Reilly Media

Close Cousins

Sun, 30 Oct 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Our process for developing and maintaining lessons has grown and changed over time. Simultaneously but separately, an organization called the Programming Historian has crafted a diverse set of open, reusable lessons on computing skills for people working in the digital humanities (DH), and their process is different from ours in some interesting ways.

The main elements of our approach are:

A first version is created by:
- someone writing something on their own (or translating something they’ve written before),
- a group of people getting together at a hackathon to create a roadmap, or
- someone driving an open design process of the kind used for our new Python lesson or Zack Brym’s new visualization lesson.
The lesson is put in a GitHub repository. Everyone is invited to submit enhancements and changes by filing issues and/or submitting pull requests, and to comment on other people’s submissions. This is a doubly-open process: both the submissions and the reviews are tied to the GitHub usernames of their creators, which in turn are usually tied to their actual identities.
One or two people act as the lesson’s maintainers (a term we borrowed from open source software projects). Their role is primarily editorial: they either review PRs themselves or make sure that other people review them, and have final say over whether changes are merged or not.
We publish our lessons twice a year by tidying them up and then archiving them at Zenodo, which gives each of them a DOI. Everyone whose work has been merged into the lesson is listed as a contributor, and the maintainers are listed as editors (because that’s a role everyone in academia understands).

The strengths of this approach are that the community maintains the lessons (we’ve had about 400 distinct contributors in the past three years), while the editor-vs-contributor distinction allows us to recognize people who are doing extra work. Its weaknesses are that big changes are more difficult to make than they would be if there was a single author, and there’s no incentive for people to do reviews: someone’s name doesn’t show up in the bibliographic record for a lesson if “all” they did was craft hundreds of lines of thoughtful feedback.

New Book: Tidy Text Mining with R

Sat, 29 Oct 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A new online book has recently been published that may be of interest to our community:

Tidy Text Mining with R

This book provides resources and examples for people who want to use tidy tools from the R ecosystem to approach natural language processing tasks. The intended audience for this book includes people who don’t have extensive backgrounds in computational linguistics but who need or want to analyze unstructured, text-heavy data. Using tidy data principles can make text mining easier, more effective, and consistent with tools already in wide use like dplyr, broom, and ggplot2. Topics covered in the book include how to manipulate, summarize, and visualize the characteristics of text, sentiment analysis, tf-idf, and topic modeling.

The Rest Is Yet To Come

Sat, 29 Oct 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I co-taught an instructor training workshop earlier this week, then taught a second one on my own a couple of days later. I made some pretty big mistakes in both: I kept interrupting my co-instructor in the first, while in the second, I told too too many stories, made jokes about hipsters and Javascript programmers when I’d told participants not to belittle people in class, and shut down discussion a couple of times when I had no authority to do so.

What the Carpentries Mean To Me

Wed, 26 Oct 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

October 26 marks my 3rd Github cakeday. It also marks my 3 year anniversary since my first Software Carpentry workshop as a learner. The icing on the cake (haha?) is that it’s also Open Access Week.

My first computer science course was in high school. I got though the class with a healthy amount of struggling, but I never thought I’d make it in computer science because some of my fellow classmates got though the class so effortlessly. My rationale at the time was: if this is what it takes to be good at computer science, I’d never make it. I graduated with a BA in psychology/behavioral neuroscience, and minors in biology and computer science.

Call for Candidates for the 2017 Steering Committee

Mon, 24 Oct 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Software Carpentry will hold its annual election for the Steering Committee of The Software Carpentry Foundation on January 23-27, 2017. Please consider standing for this election to help shape the future direction for our commmunity. The roles and responsibilities of members of the Steering Committee are available here.

If you are a qualified instructor who has taught at least twice in the past two years, or have done non-teaching work for the community, you can both stand for election and vote. Please visit the list of current members to see who is eligible to stand and vote in our election. If you believe you qualify as a member but are not listed there, please contact us as soon as possible.

Programming as Theory Building

Sun, 23 Oct 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I was recently reminded of a thought-provoking but often-overlooked essay by Peter Naur from 1985 called “Programming as Theory Building” (scan here, plain text here). He suggests that, “…programming properly should be regarded as an activity by which the programmers form or achieve a certain kind of insight, a theory, of the matters at hand. This suggestion is in contrast to what appears to be a more common notion, that programming should be regarded as a production of a program and certain other texts.” His thoughts on what programmers actually do, especially when modifying programs, seem directly relevant to most research software development. In particular, when the Jupyter Notebook and R Markdown are discussed as ways to make research more reproducible, I wonder if part of that is to encourage the programmer to make her theory of what she’s doing explicit.

A Comparison of Online and In-person Instructor Training Workshops

Sat, 22 Oct 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I have co-taught three instructor training workshops this year (one online and one in-person with Christina Koch and one online with Greg Wilson.

Overview

Too long to read? Here’s an overview in tabular form.

Feature	In-person	Online
networking	excellent	poor to very good
Etherpad use	poor	excellent
displayed on screen	the web? slides?	the instructor webcast view of other learners
time commitment	prep + class time + travel + paperwork	prep + class time
private communication	in-person	Slack & email
technical difficulties	medium	high

Networking

In-person: I think the most amazing feature of an in-person workshop is the networking. There is something about face to face conversation during activities and over coffee, beer, and dinner that really solidifies personal relations. I became a certified instructor in January 2015 at UC Davis instructor training workshop taught by Greg Wilson, Tracy Teal, Bill Mills, and Aleksandra Pawlik; I consider all four of these people to be close colleagues now, and I’m going to be visiting a handful of other workshop participants in November when I visit California. Also, I regularly talk to SWC/DC instructors on-line, but there is something about meeting them in-person that feels like its the first time we’ve met face-to-face.

Library Carpentry is One Year Old

Sat, 22 Oct 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The indefatigable James Baker recently wrote a blog post summarizing what’s happened with Library Carpentry in the past year. It summarizes their lessons, their workshops, how Library Carpentry is managed, and much more. Announcements and initial discussion take place on Gitter, and new members are welcome to join–please check them out.

Cambridge Instructor Training 19-20 September 2016

Thu, 20 Oct 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Last month, Steve and Karin taught an instructor training workshop at the University of Cambridge, sponsored by the R Consortium. The event was organized by Laurent Gatto, a Software Sustainability Institute Fellow, with help from Paul Judge and Gabriella Rustici from the University of Cambridge Bioinformatics Training facility. 25 trainees from a diverse set of backgrounds spent two days getting to know each other and learning how to teach, and several have already completed the checkout process. You can read a full write-up on the SSI website, and we hope to be able to organize a repeat in the new year.

Community Call on Assessment

Thu, 20 Oct 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Dr. Kari L. Jordan, Data Carpentry’s Deputy Director of Assessment, will be speaking on the Community Call on the impact our workshops have on our learners. Kari will cover the results of our post-workshop surveys including learner attitudes, learner data management and analysis skills prior to and following our workshops, and learner attitudes toward Data Carpentry instructors. Come and hear about our successes and share your thoughts. It’s going to be an informative community call.

Ten Simple Rules for Digital Data Storage

Thu, 20 Oct 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are pleased to announce the publication of a new paper whose author list includes several members of our community:

Edmund M. Hart, Pauline Barmby, David LeBauer, François Michonneau, Sarah Mount, Patrick Mulrooney, Timothée Poisot, Kara H. Woo, Naupaka B. Zimmerman, and Jeffrey W. Hollister: “Ten Simple Rules for Digital Data Storage”. PLOS Computational Biology, Oct 20, 2016, http://dx.doi.org/10.1371/journal.pcbi.1005097.

Their ten rules are:

Software Carpentry at Oklahoma State

Wed, 19 Oct 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I recently instructed Git for the third time at a Self-Organised workshop on the Oklahoma State University main campus. I enjoy instructing and helping with the Software Carpentry workshops (and hopefully will get to do a Data Carpentry soon) and each workshop is always different from the last, so I was excited to participate again.

Because of a scheduling conflict, we only had the lab for five hours on Day One. In five hours (with two 15-minute breaks), we went through the entire bash shell lesson without many hiccups. One thing we did create for the bash shell lessons was a series of three-question quizzes we launched live on Socrative before each break. The learners had good feedback about this and felt it was a good way to break up the lessons and provided a good review of concepts they had just learned.

12 September - 17 October, 2016: Community Service Awards, Steering Comminty Elections, Lesson Maintainers Meeting, Newsletter, Carpentries Community Call.

Mon, 17 Oct 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

##Highlights

The very first Community Service Awards will be awarded before the end of the year.
We are pleased to welcome two new Intsructor Trainers to the Carpentries.
Please vote on the special election to amend the procedure of the Steering Committee elections.
Some highlights on the progress made by our lesson maintainers on streamlining our decision-making process.
The first joint Carpentries newsletter is out, please sign up.

##Jobs

Machine Learning with Python

Mon, 17 Oct 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A new book has recently been published that may be of interest to our community:

Introduction to Machine Learning with Python

Data driven approaches have taken over many empirical sciences and many business application. Machine learning algorithms are one of the most important tools for extracting knowledge and making decision based on complex datasets. This book takes a practical approach to machine learning, using Python and the scikit-learn library. Starting from the basics, it explains how and when to use machine learning, discusses common methods and points out pitfalls for beginners. Every method and example comes with code in the form of Jupyter notebooks. The book requires a basic understanding of the Python programming language and some familiarity with NumPy. Experience with matplotlib is helpful to gain a better understanding of the visualizations.

October 2016 Maintainers' Meeting

Thu, 13 Oct 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This week’s meeting of our lesson maintainers made some great progress in streamlining our decision-making process, and we hope to begin implementing some of the changes discussed in the next few weeks. The major highlights are:

Creation of a developers’ subcommittee: We’re creating a new subcommittee that will be in charge of decision-making for template and style changes common to all lessons. Each maintainer will still be responsible for PRs/issues specific to their lesson. I will contact folks who indicated interest in participating, and am pleased we have folks from both SWC and DC; if anyone not currently acting as a maintainer would like to take part, please contact Kate Hertweck.

In Memoriam: Hans Petter Langtangen

Tue, 11 Oct 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Hans Petter Langtangen’s books Python Scripting for Computational Science and A Primer on Scientific Programming with Python taught many of us how to do numerical computing with Python. He passed away yesterday after a long struggle with cancer; while I only had the privilege of meeting him in person twice, we corresponded frequently during Software Carpentry’s early years, and he was always helpful, insightful, and enthusiastic. He will be missed.

Beth Duckles on the Practice of Measuring

Mon, 10 Oct 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Dr. Beth Duckles, who did a valuable study of our instructor community earlier this year, gave a talk at the recent Measuring the Impact of Workshops workshop titled “The Practice of Measuring”. It’s a very useful 50 minutes, particularly for those of us who have backgrounds in the physical rather than social sciences.

Vote Next Week to Amend Steering Committee Election Procedures

Mon, 10 Oct 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Last month, the Steering Committee announced a special election to amend the procedure for Steering Committee elections. The election itself will take place next week (Oct 17-21, 2016). All members will receive a ballot via email to cast their vote regarding this revision via electionbuddy. Please take a moment to check the current membership list and contact us by email to let us know of any omissions. As per our bylaws, our membership includes any certified instructor who has taught at least twice in the past two years (i.e., since October 2014) or has made other significant contributions to Software Carpentry in the opinion of the Steering Committee.

Request for Review: ESIP's Software Guidelines

Wed, 05 Oct 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

ESIP (the Federation of Earth Science Information Partners) has been developing research code/software guidelines for the earth observation and geosciences communities, and would appreciate feedback on the current draft before the end of October. If you have suggestions or feedback for:

Interoperability
Jupyter notebooks as code or documentation
The proposed progression model
Sustainability or adoption/reuse

please chime in. (Look in the right margin of the browser page for the hypothes.is controls that will let you add and view comments.) As a taste of what they’re doing, here’s a table of their stakeholders’ user cases and desired outcomes:

And Now There Are Three

Tue, 04 Oct 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A new book has just been published that covers much of the same material as Software Carpentry, and a great deal more: Paarsch and Golyaev’s A Gentle Introduction to Effective Computing in Quantitative Research: What Every Research Assistant Should Know. It covers almost everything I would want to see in a one-semester course for new research students: the Unix shell, data organization, the basics of Python, data analysis, “geek stuff” (including hardware and algorithm analysis), numerical analysis, some worked examples, Python extensions, and preparing manuscripts with LaTeX.

Congratulations to Our New Instructor Trainers

Tue, 04 Oct 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are very pleased to welcome Karin Lagesen and Anelda van der Walt to our team of instructor trainers. Karin and Anelda have been very active in all of our activities–Karin is now in her second year on the Steering Committee, and Anelda has been the driving force behind our growth in South Africa–and we are very grateful that they are willing to give even more to our community.

Python as a Second Language

Tue, 04 Oct 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Donny Winston, Joey Montoya, and I taught a one-day class for Lawrence Berkeley National Laboratory on Python as a Second Language last week. As its introductory blurb says, “This lesson is an introduction to programming in Python for people who are already comfortable in some other language such as Perl or MATLAB.” The notes are still very much under development, but having delivered it twice, we’re pretty confident that it can actually be delivered in one day. We would be very grateful for feedback: please file issues in the GitHub repository to let us know what you think, to add more exercises and bullet points, or anything else.

Perth Software Carpentry - A Tale of Three Trainers

Fri, 30 Sep 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Andrew

Perth’s journey into Software Carpentry began when Andrew Rohl attended eResearchNZ in 2014, for which he has to thank Nick Jones of NeSI for financial support. There he met the director of the Mozilla Science Lab, Kaitlin Thaney, and learned about the Software Carpentry movement. Fast forward to the end of the year and David Flanders was arranging a “Train the Trainer” Software Carpentry course in Melbourne and he even had funds to cover travel costs for those selected to attend! Andrew convinced Raffaella Demichelis, a fellow computational chemist, to also apply and they were both fortunate enough to be chosen. They enjoyed the training in February 2015, followed by the first Research Bazaar conference and returned to Perth enthused.

Two Studies of Online Communities

Fri, 30 Sep 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Two recent papers may be of interest to this community. The first is from Adam Crymble at The Programming Historian a distributed group of digital humanities scholars that has built some excellent tutorials on software tools. Its title is Identifying and Removing Gender Barriers in Open Learning Communities, and its abstract reads:

Open online learning communities are susceptible to gender barriers if not carefully constructed. Gender barriers were identified in The Programming Historian, through an open online discussion, which informed an anonymous user survey. The initial discussion pointed towards two barriers in particular: a technically challenging submission system and open peer review, as factors that needed consideration. Findings are put in context of the literature on gender and online communication, abuse, and online learning communities. The evidence suggests that open online learning communities such as The Programming Historian should work actively to promote a civil environment, and should listen to their communities about technical and social barriers to participation. Whenever possible, barriers should be removed entirely, but when that is not feasible due to financial or technical constraints, alternatives should be offered.

Software Carpentry Workshop Attendance: a New Zealand Perspective

Wed, 28 Sep 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Having taught and helped at a series of workshop over the past few months Tom Kelly, PhD Candidate in Genetics at the University of Otago, wrote up some of his reflections on the issues related to workshop attendance. This spurred further discussion via email among the New Zealand instructors. We decided to put these thoughts together in hope that this could help other sites struggling with the attendance problems.

SWC: First Impressions

Wed, 28 Sep 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This post is a simple telling of the beginning of my experience with SWC and hopefully first of many encounters with SWC as a community as well as a learning experience. I attended a session a couple of months previously and was very impressed by my experience. I had a chat to the other attendees and found that their experience as complete beginners was as positive as someone who had some experience in python, git and shell as I did. I have always held a firm belief that digital literacy in research and education is both vital and sadly lacking, my own experience is entirely self taught throughout a university education, so it is no surprise that myself and my fellow attendees were drawn to the SWC workshop to fill that gap. SWC addressed the digital gap in research, and had me so hooked that I just had to stay involved. Over the next couple of months I looked for any opportunity; I incorporated just the python novice material into an independent workshop that I ran on Python for data pipelines, and now I have just instructed my first SWC session. The more involved I get the more I feel like SWC is something that I want to be involved in, because it addresses an important need in research, and because it does so in a way that is accessible and tailored by its community. I think my experience with SWC is not unique, and that hundreds of people around the globe have been in the same boat as I was. In fact I am sure it is not unique, as every contact I have seems to be with people as interested as I am.

Teaching Programming to the Blind

Fri, 23 Sep 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Andreas Stefik (who discusses what we know about the usability of programming languages in this entertaining podcast) has worked extensively on computing education and programming tools for the visually impaired. When asked earlier this week how to teach programming to the blind, he sent the response below. We’re grateful for his comments, and for Evan Williamson’s recent pull request to improve the accessibility of our lessons.

Teaching at the Board

Tue, 20 Sep 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This post originally appeared on Chris Hamm’s personal blog.

Software Carpentry is a non-profit organization that teaches basic computer skills. The lessons for these courses assume no prior knowledge among the learners. I am a certified instructor for Software Carpentry and its sister organization, Data Carpentry.

I have taught two Software Carpentry workshops at the Federal Reserve Board in Washington, DC and they were very different from one another. Both workshops were successful but for very different reasons. This post explores the reasons why (I think) they were so different.

Systems Biology Postdoc Position with The Jackson Laboratory

Mon, 19 Sep 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Carter Lab at The Jackson Laboratory is seeking a Postdoctoral Fellow in computational genetics and systems biology. Our group is developing novel computational methods to derive biological models from large-scale genomic data. The strategies we pursue involve combining statistical genetics concepts such as epistasis and pleiotropy to understand how many genetic and environmental factors combine to control disease-related processes in animal models and human studies. We are especially interested in dissecting the genetic complexity of autoimmune disease, neurodegeneration, and cancer.

Show Me Your Model

Sun, 18 Sep 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As far as I can tell, there are no published studies showing that version control is better than mailing files around or sticking them in shared drives. I believe it is–I wouldn’t work on a project that didn’t use version control–but nobody’s ever gathered data, compared it to a model, and published the result.

One reason, I think, is that we don’t know how to measure the productivity of programmers. “Lines of code per hour” clearly isn’t right: good programmers often write less code, or spend their time on the parts of problems that have the highest thinking-to-coding ratio. Without some operationalization of “better” and “worse”, it’s hard to rank or compare alternatives.

Community Service Awards

Sat, 10 Sep 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Software Carpentry Foundation relies on volunteer efforts to achieve many of its goals. It is now inaugurating a Community Service Award as a way for its Steering Committee to recognize work which, in its opinion, significantly improves the Foundation’s fulfillment of its mission and benefits the broader community. Details are available on this page; nominations are welcome at any time, and we will make the first awards before the end of this year.

The Discussion Book

Sat, 10 Sep 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Hot on the heels of Small Teaching (which we reviewed last week) comes Brookfield and Preskill’s The Discussion Book. Its subtitle is “50 great ways to get people talking”, and that’s exactly what it delivers: one succinct description after another of techniques you can use in classes or meetings to get everyone talking productively. Each one is covered in three or four pages with the headings “Purposes”, “How It Works”, “Where and When It Works Well”, “What Users Appreciate”, “What to Watch Out For”, and “Questions Suited to This Technique”.

17 August-12 September, 2016: Steering Committee, Google Summer of Code, rOpenSci, Small Teaching, Ten Simple rules.

Fri, 09 Sep 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

##Highlights

We are amending the steering committee election procedures.
Some members of our community co-authored research papers, and would appreciate some feedback.
A very succesful Google Summer of Code 2016 has come to an end.
Openness can still lead to a frustrating situation.
You can now log in to the etherpad to let us know if you will be attending the monthly Carpentries Community Call.

##Tweets

Excel might be to blame for your conversion mistakes.
Software Sustainability Institute fellowship programme 2017 applications now open.
Learn how to make your research analysis better and reproducible at PyConZA 2016.
Read more about Academic archetypes.
What would you like to have at the Brisbane Research Bazaar (ResBaz)?
A good read on how to be a Modern Scientist.
If you have innovative ideas for open science the arnold foundation may be the place for you.

##Vacancies

September Carpentries Community Call

Fri, 09 Sep 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Our next Carpentries Community Call (formerly called Lab Meeting or Town Hall meeting) will be Thursday, September 15 (September 16 Aus/NZ). These meetings will now be monthly on the third Thursday of every month. It would be great to see instructors there! These calls are a great chance to connect with other Carpentry instructors and get updates and information on important and interesting topics for the community.

Post-doc Position with rOpenSci

Wed, 07 Sep 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The rOpenSci project based at the University of California, Berkeley seeks to hire a postdoctoral scholar to work on the research activities funded by the grant titled “Fostering the next generation of sustainable software and reproducible research practices in the scientific community”. The project develops open source software to promote reproducible research practices in the scientific community. The postdoctoral scholar will focus on a research topic aligned with their own interests in order to better understand and improve scientific software practices. Possible topics include but are not limited to:

Google Summer of Code 2016 ended

Mon, 05 Sep 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As announced in April, we had some Google Summer of Code Students working with us this year.

Manage workflow for Software Carpentry and Data Carpentry instructor training

Chris Medrela, under the mentoring of Greg Wilson and Piotr Banaszkiewicz, worked on AMY implementing instructor training workflow that is already in use as we reopened instructor trining.

Result-aggregation server for the installation-test scripts for Software Carpentry and Data Carpentry

Prerit Garg, under the mentoring of Piotr Banaszkiewicz and Raniere Silva, worked on a web server that receive and store information provided by the installation script.

Small Teaching

Mon, 05 Sep 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Elizabeth Green’s Building a Better Teacher changed how I think about teaching, and sparked some good discussion in our community. Therese Huston’s Teaching What You Don’t Know had a similar impact a few years earlier, and now there is James Lang’s Small Teaching.

As its title suggests, Lang’s book focuses on little things that teachers can do right now to improve their teaching, rather than on big, systemic changes that might have larger impact, but which require larger effort (and probably buy-in from other people). To be included, a practice had to:

We Still Can't Have Nice Things Together

Mon, 05 Sep 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Last year I used YAML and Norway to explain why why we can’t have nice things. We’ve just stumbled over a problem that has forced us to re-do some of the work we did to publish our lessons a couple of months ago, and which illustrates how openness can still be frustrating to actually do. Are you sitting comfortably? Then let’s begin.

GitHub can publish repositories as website.

If the user’s ID is gloom, and the project’s name is despair, then the GitHub repository’s URL is http://github.com/gloom/despair. If that repository has a branch called gh-pages, GitHub automatically creates a website at http://gloom.github.io/despair.

Feedback Sought on Two Papers

Fri, 02 Sep 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We would be very grateful for feedback on two papers co-authored by members of our community:

Taschuk & Wilson: “Ten Simple Rules for Making Research Software More Robust”.
Wilson, Bryan, Cranston, Kitzes, Nederbragt, & Teal: “Good Enough Practices for Scientific Computing”.

Each paper has a link at the top to send us email; we look forward to hearing from you.

Election Announcement: Amending Steering Committee election procedures

Thu, 01 Sep 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Steering Committee will be holding a special election on October 10-14 regarding the following amendment to Steering Committee elections:

“Following an election, the new Steering Committee will meet jointly with the previous Steering Committee for no less than 60 days. For meetings during the first 30 first days, the new SC will not have voting privileges. After 30 days, voting privileges are transferred to the new SC.”

Ten Ways to Turn Off Learners

Fri, 19 Aug 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

PLOS has published a very useful set of articles called Ten Simple Rules that covers everything from effective statistical practice to winning a Nobel Prize. I’m just as interested in what not to do and what mistakes to avoid, so as part of our instructor training course, I’d like to put together a list of ten simple ways you can turn off your learners. My first five are listed below; if you’d like to add your own, comments would be very welcome.

Teaching Library Carpentry to Librarians at UCSD

Wed, 17 Aug 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I sort of knew what I was getting into. I’d done the excellent instructor training in February at UC Davis, which is a good thing, because I didn’t know the first thing about instruction. I didn’t know the first thing about organizing workshops, either, but I figured what the hell; my colleague and partner-in-crime Tim Dennis had reserved the big conference room in our library, which is really hard to get. If you’re in academia you know the first rule, which is never waste a reserved prime conference room. With lots of prompts and help from my colleague Tim Dennis, we put together a Library Carpentry workshop at UC San Diego.

1 -16 August, 2016: Assessment Deputy Director, Policy Subcommittee, Code of Conduct, Workshop Resources, Bug BBQ, and Vacancies.

Mon, 15 Aug 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

##Highlights

Dr. Kari L. Jordan has been appointed as Data Carpentry’s new Deputy Director of Assessment.
Join the Policy Subcommittee and/or provide feedback on the Reporting Guide for handling potential Code of Conduct Violations.
Data Carpentry has put together a terrific resource for workshop organisation.
Read about our recent Bug BBQ that was held in the lead up to publishing our lessons.

##Vacancies

NCSA at the University of Illinois are looking for applicants for the position of Training Coordinator.
NumFOCUS is looking for a full-time Projects Director.

##Tweets

2016 Bug BBQ Summary

Mon, 15 Aug 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

At the beginning of the summer, the Software Carpentry community joined forces hold their first ever Bug BBQ. The goal of this event was to squash as many bugs in our core lessons as possible before we published and shipped the new version (2016.06) of the lessons. In addition to the goal of getting a large amount of work done as quickly as possible, we also aimed to use this event to engage and connect with our world-wide community.

Training Coordinator Position at NCSA

Wed, 10 Aug 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Computational Science and Engineering program and the National Center for Supercomputing Applications at the University of Illinois at Urbana-Champaign seek applicants for the position of Training Coordinator. This position will report jointly to the Director, CSE and the Assistant Director for Scientific Software and Applications at NCSA. The Training Coordinator will enable cutting-edge research involving the use and/or development of advanced digital software and hardware across all disciplines by delivering, coordinating, and administrating training programs for students and researchers at the University of Illinois and area institutions, including industry. The individual in this position will play a key role in the research life of the campus, identifying important computational and data science skills and technologies and demonstrating how they can be used to solve problems in computational science and engineering and other disciplines.

Code of Conduct and Call for Volunteers for Policy Subcommittee

Mon, 08 Aug 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Carpentries are proud to share a common Code of Conduct (CoC), which outlines acceptable standards of behavior for our community members and those interacting with the Carpentries at in-person events and online spaces. Historically, however, we have not had an official process for reporting potential Code of Conduct violations or for adjudication and resolution of reported incidents. Thanks to input from our community, we recognize that defining these procedures is an important step in ensuring that any such issues are dealt with transparently in order to keep our community welcoming and safe for all.

Resources for Running Workshops

Mon, 08 Aug 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A successful workshop is the result of coordinated effort among many different types of participants, including instructors, helpers, hosts, learners and staff. Software and Data Carpentry offer two types of workshops: Self-Organised and Centrally-Organised. These workshop types differ in terms of instructor training requirements, fee structures, and participant responsibilities, with local hosts and instructors at Self-Organised workshops taking on administrative responsibilities normally handled by Carpentry staff.

Data Carpentry's New Deputy Director of Assessment

Tue, 02 Aug 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Data Carpentry has just announced that Dr. Kari L. Jordan will be joining them as the Deputy Director of Assessment. Kari holds a PhD in STEM Education from Ohio State University, and has worked most recently at Embry-Riddle Aeronautical University in Florida, where her postdoctoral research focused on understanding factors that influence faculty adoption of evidence-based instructional practices. To learn more, please see the post on the Data Carpentry website or follow Kari as @drkariljordan on Twitter.

Heuristic Evaluation for Novice Programming Systems

Tue, 02 Aug 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I have recently been reading and enjoying a new paper by Michael Kölling and Fraser McKay titled “Heuristic Evaluation for Novice Programming Systems”. In it, the authors say:

With the proliferation of competing systems [for novices], the problem [of evaluation ] has become more complicated. Not only should we ask the question whether such kinds of tools are helpful at all (which many instructors strongly believe them to be, even in the absence of hard evidence), but we need to decide which of a significant number of competing systems is “better” for a given task in a given context. Educators have to make choices, not only between using an educational IDE or not, but between a number of direct competitors.

How Well Do Developers Understand Open Source Licenses?

Tue, 02 Aug 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

You are invited to participate in a survey on software licensing designed to investigate how well software developers understand common open source software licenses. We are looking for software developers that have built or are currently building on open source software in their projects (and I am personally interested in hearing from people building open source software for research). The study is being conducted by Prof. Gail Murphy (murphy@cs.ubc.ca) and graduate student Daniel Almeida (daa@cs.ubc.ca); participating in the anonymous online survey will take approximately 30 minutes. If you are interested in participating, please go to:

NumFOCUS Project Director

Tue, 02 Aug 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

NumFOCUS (the organization which shelters Software Carpentry, Data Carpentry, and several other open science projects) is seeking to hire a full-time Projects Director to develop and run a sustainability incubator program for NumFOCUS fiscally sponsored open source projects. This is the first program of its kind, with opportunity for enormous impact on the open source ecosystem. The learnings from this program will be public, meaning it has the potential to change how all open source projects are managed. For more information, please see the job posting.

Seymour Papert 1928-2016

Tue, 02 Aug 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Seymour Papert passed away on Sunday at the age of 88. I never had the privilege of meeting him, but Software Carpentry would probably never have existed if I hadn’t stumbled across his inspirational book Mindstorms. You can read more about his life and work here; when you’re done, please go and help someone learn something—I think he’d have liked that.

Library Carpentry in Toronto

Sat, 30 Jul 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

On July 28-29, a group of volunteers from the University of Toronto’s libraries ran a two-day workshop for thirty-five of their fellow librarians. People came from as far away as Sudbury, Ottawa, New York City, and even Oregon to spend two days learning about:

regular expressions,
XPath and XQuery,
OpenRefine,
programming in Python, and
scraping data off the web.

While there were inevitably some hiccups getting software installed on learners’ machines, everything ran pretty much on schedule, and the instructors got through most of the material they had planned to cover. What was particularly nice was the way the modules fit together: the Python lesson closed by showing people how to write programs using regular expressions, while the scraping lesson referred back to the XPath material.

Library Carpentry workshop at James Cook University, Townsville

Fri, 29 Jul 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We held a two-day Library Carpentry workshop at James Cook University, Townsville on 14–15 July, 2016. The workshop was a first on several fronts - the first course run by Software Carpentry trainers Jay van Schyndel and Daniel Baird from the JCU eResearch Centre and the first Library Carpentry course run at JCU. Collin Storlie, the local QCIF eResearch Analyst, agreed to help out too.

How was the workshop proposed?

1 - 28 July, 2016: Lesson Publication, Instructor Training Open, Creating New Material, Revamped Lesson Template, and Instructor Testimonial.

Thu, 28 Jul 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

##Highlights

Congradulations and thanks to everyone who contributed to Version 2016.06 of the Software Carpentry lessons which have just been published.
You can now partner with Software and/or Data Carpentry to develop new lessons. Get in touch to discuss your ideas.
Please take note of our new lesson template.

##Intstructor Training

Apply now for online instructor training.
Some frequently asked questions about the reopening of the Instructor training have been addressed.
Why attend a Software & Data Carpentry instructor training workshop?
About ten hours is all you will need to qualify as an instructor after the initial training event.

##General

How Software Carpentry can help you switch careers

Wed, 27 Jul 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

You’ve heard how Software Carpentry is important not only to get your research done but also to make it better and thus more likely to be published.

It might even get you a job in your favourite field of research.

But, for those of us who do not like to do the same thing forever, Software Carpentry skills can help you switch to another career.

More on Instructor Training

Tue, 26 Jul 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Since we announced yesterday that we are re-opening applications for instructor training, several questions have come in by email and on Twitter. We’ve answered the general ones below, and will update this post as others arrive.

Why are you running open-enrollment classes?: Our primary goal is to increase our reach: many geographic regions and research disciplines are sparsely represented in our instructor pool, or not represented at all, and we are always conscious of the need to maintain and improve diversity.
How does this relate to instructor training for partner organizations?: Partner organizations are guaranteed a certain number of instructor training slots as part of their agreement with us, and those trainings are given priority over open-enrollment offerings.
How many classes are being offered, and how many spaces are available?: We are currently planning to run two classes, each for about 30 participants.
Where will the training take place?: These two classes will take place online, and will run over two full days. We will choose dates and times to accommodate as many people as possible.
How are participants going to be selected?: We will select participants based on previous involvement with Software and Data Carpentry, location, research discipline, previous teaching experience, ability to commit time to teaching and mentorship, and all of the other factors in the application form.
When will selections be made?: We will start notifying people in a few weeks (i.e., the second half of August).
In what time zones will classes take place?: We haven’t decided yet, but they will be aimed at different timezones so that we can accommodate the widest possible range of participants.
Will you offer more open-enrollment classes in future?: Yes.
Wil people have to re-apply to take part?: No: we will keep all applications, and contact people as spots become available to see if they’re still interested.

Software Carpentry at Curtin

Tue, 26 Jul 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The first Software Carpentry workshop to be held at Curtin University (and the third so far in WA) started on Monday 18 July. We decided early to experiment with the timetable and spread the course over four half days. Monday was for the [Unix Shell](https://twitter.com/CurtinIC/status/754850455797190657), Tuesday and Wednesday for [Programming with Python](https://twitter.com/CurtinIC/status/755613286192721920) and Thursday for [Version Control with Git](https://twitter.com/CurtinIC/status/755959937574907904).

We offered 40 places, and it was already fully booked with a couple of weeks to go. Most participants were from Curtin University, but there were also some attendees from CSIRO and other places.

Genomics R Software Carpentry workshop at the University of Auckland, New Zealand

Mon, 25 Jul 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A two day Software Carpentry workshop with R was held at the University of Auckland Winter Bootcamp on 11-12 July. After a brief battle with the projector in the room, Day 1 consisted of an eventful morning session on Unix Shell, the spontaneous explosion of a glass door, followed by an introduction to programing with R.

Reopening Instructor Training

Mon, 25 Jul 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

For the last ten months, the Software Carpentry Foundation has worked toward three goals for its instructor training program:

Make the content more relevant.
Increase the number of people able to deliver instructor training.
Find a format that meets everyone’s needs in a sustainable way.

We have made a lot of progress on all three, and are therefore now able to offer instructor training once again to people who aren’t affiliated with our partner organizations, but would like to teach Software Carpentry, Data Carpentry, or both (as the course is shared by both organizations). If you wish to apply to take part in one of the two open-enrollment classes we will offer this fall, please fill in the form at:

Showing Changes When Teaching

Mon, 25 Jul 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A key - but challenging - part of learning to program is moving from writing technically-correct code “that works” to writing high-quality code that is sensibly decomposed into functions, generically-applicable and generally “good”. Indeed, you could say that this is exactly what Software Carpentry is about - taking you from someone bodging together a few bits of wood in the shed, to a skilled carpenter. As well as being challenging to learn, this is also challenging to teach: how should you show the progression from “working” to “good” code in a teaching context?

A Tale of Two Workshops

Fri, 22 Jul 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Brisbane and Toowoomba are 125 km apart in Queensland. Software Carpentry workshops were held in both cities a week apart (11-12 July and 18-19 July). I taught at both.

The Brisbane R workshop was held at The University of Queensland. This was a tie-in workshop for the annual UQ Winter School in Mathematical and Computational Biology. Attendees are generally very keen to learn as they want to emulate the amazing computational work they have seen demonstrated during the week by the stellar speaker line-up at the Winter School. We had no trouble filling a workshop with 40 places, and no-one left the workshop. It was one of the best workshops I have organised - the buzz in the room was palpable and the feedback was overwhemingly positive.

Software Carpentry workshop at the University of Auckland - Winter bootcamp, New Zealand

Wed, 20 Jul 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A two day Software Carpentry Workshop wiht Python was held at the University of Auckland 11-12th July as part of Winter Bootcamp. For myself,it was the first time helping with Software Carpentry training. It was a great experience to assist as a collaborator, helping others resolving problems with software installation and hands-on exercises, but also learning from all the unexpected situations that could happen with different people around the room, with different laptops, configurations, and different sets of skills.

Lesson Incubation

Tue, 19 Jul 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Data Carpentry and Software Carpentry Steering Committees recently approved [a process for supporting the incubation of new lessons. The goal is to provide a clear path for creating new material and getting it into the hands of people who can teach it while taking into account the resources we have available and the need to maintain lessons as well as create them. If you would like to build something substantial for either of the Carpentries, we’d enjoy hearing from you. (Please also see the announcement on the Data Carpentry site.)

Publishing Our Lessons, Version 2016.06

Tue, 19 Jul 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are very pleased to announce the publication of Version 2016.06 of the Software Carpentry lessons. Thanks to a lot of hard work by their maintainers and Rémi Emonet (who has acted as release manager), we have:

removed the need to build and commit HTML (everything except our R lessons is now pure Markdown);
updated the appearance of the templates (including a top menu, a nicer color scheme, and previous/next links); and
merged over 3100 pull requests from over 200 people.

Our materials are far from perfect, but we're very proud of what our community has built. Please see the releases page for links to the archived release, and the main lessons page for links to the updated lessons.

Using RMarkdown with the new lesson template

Fri, 08 Jul 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Our lesson template is getting a face-lift. Actually, it is a lot more than that: all the internal mechanics are also affected.

What’s new with the template?

The lesson maintainers have developed a new template that comes with features that have been repeatedly requested such as an easy way to navigate among episodes within a lesson: there are now “previous” and “next” arrows in each episode.

Why attend a Software & Data Carpentry instructor training workshop?

Wed, 06 Jul 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

*Note:* the following post was written by Carrie Andrew, University of Oslo, after a request to write a testimonial of her participation in a Software/Data Carpentry instructor training workshop.

The SWC/DC initiative is a cutting-edge program that promotes computer and data skills to those who need the greatest help, but are often the most put-out to learn: beginners. It is taught by an ever-increasing, diverse assemblage of people across private and public, academic and non-academic, research institutions. The common denominator is that all have an interest and motivation to promote computer and data skills within STEM organizations. Even more interesting: The program is designed to be self-destructing. Once educational and research organizations are globally saturated with keen, well-trained individuals who can stand-in as the SWC/DC person(s) for their institutions, the initiative promises to end. When the need is met, it self-destructs, thus promoting change from the bottom-up, a grass-roots initiative for our futures!

Instructor Training Completion Times

Tue, 05 Jul 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

How long does it take to complete instructor training once the class itself is done? Two dozen people who recently qualified told us this:

	Reading Lesson(s)	Writing Exercise	Discussion Session	Lesson Demo	Total
	1.0	0.5	2.0	0.5	4.0
	5.0	1.0	2.0	1.0	9.0
	5.0	3.0	2.0	3.5	13.5
	2.0	0.5	1.0	1.0	4.5
	10.0	3.0	1.0	5.0	19.0
	5.0	2.0	1.0	3.0	11.0
	1.0	2.5	1.0	10.0	14.5
	16.0	4.0	4.0	8.0	32.0
	1.0	1.0	1.0	1.5	4.5
	2.0	1.0	2.0	2.0	7.0
	1.0	0.5	1.5	0.5	3.5
	5.0	2.0	2.0	1.0	10.0
	7.0	2.0	2.0	1.0	12.0
	3.0	0.5	2.0	1.5	7.0
	3.5	1.0	1.0	2.5	8.0
	1.0	1.0	2.0	3.5	7.5
	6.0	1.0	2.0	3.0	12.0
	2.5	1.0	1.0	2.0	6.5
	5.5	1.0	2.5	3.0	12.0
	2.0	2.0	1.5	2.0	7.5
	10.0	2.5	3.0	1.5	17.0
	5.0	1.0	4.0	1.5	11.5
	5.0	3.0	2.0	0.5	10.5
	6.0	2.0	1.5	1.0	10.5
	2.5	3.5	2.0	5.0	13.0
	1.5	0.5	0.8	2.5	5.3
Min	1.0	0.5	0.8	0.5	3.5
Ave	4.4	1.7	1.8	2.6	10.5
Median	4.3	1.0	2.0	2.0	10.3
Max	16.0	4.0	4.0	10.0	32.0

There are a few outliers (which may be due to different interpretations of what time should be assigned where), but overall these numbers are pretty consistent, and will be shared with future training course participants.

Software Carpentry workshop at the University of Otago, New Zealand

Mon, 04 Jul 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A three day Software Carpentry workshop was held at the University of Otago from June 29 - July 1. The instructors for the course were Mik Black, Nick Burns, Murray Cadzow, David Eyers, Tanya Flynn and Tom Kelly, with Anthony Shaw and Riku Takei providing additional assistance as helpers. This was Tanya’s and Nick’s first time teaching at a SWC workshop, so it was great to add them to our Otago carpentry team.

1 - 30 June, 2016: Efficacy and Usefulness, Minutes, Discussions, Onboarding Documents, Teaching Undergraduates, and Library Carpentry Material

Thu, 30 Jun 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

##Highlights

Have you previously attended a Software Carpentry workshop? Get involved with the follow-up study on “The Efficacy and Usefulness of Software Carpentry Training” by completing a survey. Feedback on your experience will be greatly appreciated!
Minutes of the Steering Committee Meeting is available - suggestions to modify the workflow of the minutes are welcome!

##Discussions

Please add your comments to the topics in ongoing discussions.
There is a discussion on the mailing list about “onboarding” documents for laboratories or small research groups. Also see Lab Carpentry.
Mexican/Spanish speaking instructors are needed for a Software Carpentry workshop later in the year. Please let us know if you’re interested to teach.
Read about approaches and experiences on teaching python to undergraduate geoscientists.

##Other

Software Carpentry workshop at the University of Canterbury, New Zealand

Thu, 30 Jun 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A two day Software Carpentry course was hosted at the University of Canterbury (UC) the 22nd and 23rd of June. The instructors, all of whom are from UC and are certified Software Carpentry instructors, were Paul Gardner (School of Biological Sciences), Richard Clare (QuakeCoRE), and Constantine Zakkaroff (Accounting & Information Systems). They were aided by three helpers: Christopher Thomson, Viktor Polak and Michael Gauland. Organising the event was greatly assisted by Sung Bae of the New Zealand eScience Institute (NeSI). NeSI also sponsored the refreshments for the coffee breaks during the workshop.

Three workshops in Brazil

Thu, 30 Jun 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

In Brazil, like in many other countries that we are starting to run workshops, there are many open questions that we need to answer such as

how do we advertize our workshops?
do we charge a registration fee? If yes, how much do we charge?
how do we create a local community after the workshop?
how do we get new partners after create a local community?

Thanks to the work of many instructors like Anelda van der Walt, Belinda Weaver, Bill Mills, Lex Nederbragt, Selene L. Fernandez-Valverde, and Tiffany Timbers we are getting some hypothesis to the questions above across the world. In terms of Brazil, the three workshops that we ran in May added more data to help answering the questions.

Minutes of Steering Committee Meeting

Thu, 23 Jun 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Steering Committee informs that the minutes of their last meeting held last week are now available on [GitHub](https://github.com/swcarpentry/board/blob/master/minutes/minutes-2016-06-16.md).

As the Secretary, I’m sorry for the delay to release the minutes of the meetings due some experiments on the workflow to write the minutes. Now I’m happy with the workflow that we have in place that will assign one ID for all our motions making easy for us to refer them. I still want to make a few improvements on the workflow to make the minutes more machine redable.

Teaching Library Carpentry

Wed, 22 Jun 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

## Teaching Library Carpentry

I ran a two-day ‘Library Carpentry’ workshop for librarians at The University of Queensland on 13-14 June. The class covered

jargon-busting and data structures
writing and using regular expressions
command line tools to find data in files
version control
OpenRefine for data cleanup

Eighteen people from six organisations attended, out of a pool of 26 who intially expressed interest. (In the small lab I was using, I only had room for 18.)

Workshop Satisfaction Survey

Wed, 22 Jun 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

## Calling all Software Carpentry workshop attendees!

I’d like 10 minutes of your time to complete a survey about the Software Carpentry workshop you attended.

Why should you bother?

The data will be useful for Software Carpentry in the future planning of workshops.

While individual Software Carpentry workshops are assessed at the time of delivery, no long term follow-up study has been done on the efficacy of the training delivered, nor of any impact the training might have had on attendees’ work practices, further skills acquisition, or subsequent career paths.

Ongoing Discussions

Mon, 20 Jun 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We've been using GitHub issues to manage discussions for the last eighteen months or so. It's working pretty well: when something on the main mailing list looks like it's going to run for a while, we push the conversation over to GitHub so that people who want to go into more detail can do so without flooding other people's mailboxes.

I've being going through those discussion issues while tidying up our lessons for their next release, and found a few that are still relevant, but which never reached a resolution. If you have thoughts on any of these, please go ahead add your comments—if your ideas don't get folded into the lessons this time, they will next.

Teaching Python to undergraduate geoscientists: A summary of our approaches and experiences over the years

Sat, 18 Jun 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

In 2010, a new course commenced at Imperial College London that aimed to teach undergraduate geoscience students how to program in Python. Over the course of five years (2010 to 2014 inclusive) our teaching methodology evolved significantly in response to student feedback and examination performance. The course lecturers (Gerard Gorman and myself, Christian Jacobs) along with several of our Teaching Assistants also undertook Software Carpentry instructor training which offered a fantastic pedagogical insight into how to effectively teach novices how to program. As a result, many of our teaching techniques were influenced by this which helped to greatly improve our course. In this blog post we summarise what we changed throughout the five years and why, in the hope that this will benefit other instructors currently offering, or planning to offer, an undergraduate computer programming course.

Software Carpentry and HPC class hits Palmerston North (New Zealand)

Thu, 16 Jun 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

On 7th and 8th June 2016 we gave a Software Carpentry training consisting of four blocks. We started with bash (presented by Markus, Landcare Research Hamilton), Python (Ben, Landcare Research Palmerston North), went on to git (Hannes, Massey University) and finally had a session around using High Performance Computing (HPC) as it is offered by the NeSI, the New Zealand eScience Infrastructure (Wolfgang, NIWA Wellington and NeSI). This last session is not one of the official SWC modules, but was taylor-made and rounded up the training very well, as it used the skills acquired during the preceding three sessions. It included hands-on training, working remotely on the NeSI Pan cluster in Auckland; all attendees managed to log on and submit jobs to the SLURM scheduler successfully.

This was a very successful workshop with almost 40 attendants, organised and presented by a mixed team of new and more experienced instructors. The participants were very responsive and there was a lot of learning happening for the attendees and for instructors. The feedback was mostly positive and the negative feedback was very constructive. An aspect mentioned by several participants was that the number of attendees was probably too large, possibly with respect to the venue size - something we have to consider in the future. The logistical support with refreshments, received from the Massey in the person of Colleen Blair, was behind the scenes, but timely and well prepared. So we had a hydrated group, and a lot of fun was had by all. The new friendships made will hopefully lead to more cooperation between colleagues. Hannes’ favourite critique was: “It is impossible to listen and type as well!” – Note to self :)

Workshop at Unicamp

Mon, 13 Jun 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A Software Carpentry workshop took place at Unicamp (State University of Campinas, Brazil) on May 23rd and 24th. Raniere Costa was the host and lead instructor, while Felipe Bocca, Monique Oliveira, and I completed the instructor's team.

Day 1: Although we had issues with the first booked classroom due to the student’s strike against budget cuts in public university education, Renato Santos gave the Unix shell lesson without problems. To solve the complication with facilities, we found an available computational lab to continue the workshop with the help of professor Doctor Francisco de Assis Magalhães Gomes Neto, IMECC’s director. After lunch, Felipe Bocca taught some R and then Raniere Costa taught RMarkdown.

Updating Library Carpentry

Mon, 06 Jun 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A global team worked to update the [Library Carpentry](https://github.com/LibraryCarpentry) curriculum and lesson material at this year’s [Mozilla Science Lab Global sprint](https://science.mozilla.org/programs/events/global-sprint-2016).

The work kicked off in Brisbane, where Clinton Roy, Natasha Simons and I worked with Carmi Cronje in Sydney to start the ball rolling. Matthias Liffers came online in Perth two hours later. At 5.30 pm our time, we handed over to teams in South Africa, the Netherlands, and the UK, led by Anelda van der Walt, Mateusz Kuzak, and James Baker respectively.

9 - 31 May, 2016: The First Bimonthly Report, Instructor Data Analysis, R Instructor Training, Measuring the Right Stuff, RSE Conference, and a Bug Barbeque

Tue, 31 May 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

##Highlights

The 2016 Steering Committee’s first bimonthly report provides an overview on happenings in the community in the past two months. The joint partnership agreement with Data Carpentry is now available, together with a written version of the instructor training model, a call for subcommittees and task forces, and an official space on Facebook. Make sure to read the Code of Conduct in support of our welcoming, friendly, and diverse community.

##Instructor training

Further Analysis of Instructor Training Data

Tue, 31 May 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Following [Erin Becker's analysis of our instructor training data]({{site.baseurl}}/blog/2016/05/first-analysis.html), Byron Smith has done [another analysis](http://blog.byronjsmith.com/instructor-survival.html) using survival statistics. The three key figures are:

What Digital Humanists Also Do

Mon, 30 May 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Following up on [an earlier post]({{site.baseurl}}/blog/2016/05/what-digital-humanists-do.html), here are two more user stories from digital humanists to help us figure out what they need and how we can help.

Gamma

Gamma is a senior scholar working on two projects. The first looks at communities of workers and architects in the Medieval and Renaissance eras, and relies on records of specific buildings, diaries of artisans, sketches, and building plans (some of which are very large: the width of a sidewalk, and the length of a city block). The end goal is to learn about the people who created the buildings, who they did the work with, the processes they used, what they were paid, and so on, and more broadly to understand the social network of people involved in creating the great buildings of the time.

What Digital Humanists Do

Wed, 25 May 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I have started gathering user stories from humanists to give us a better sense of what they do and how (or whether) our kind of skills training might help. Two of these are summarized below; there's a lot to think about, but the most important observation is that DH really is different in important ways from what Software and Data Carpentry have been doing so far.

Alpha

Alpha is studying the meeting of cultures in a text which mixes vernacular (Judaeo-Provençal) and Hebrew. The text’s author was a physician, and says in the introduction that the vernacular was for women and children, while the Hebrew was for men. The text exists in only one hand-written manuscript from 1401, which is based on the book of Esther used in the Purim holiday.

Darwin Data Carpentry at Charles Darwin University

Mon, 23 May 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

##First Data Carpentry workshop in Darwin Australia##

The COMBINE Data Carpentry R workshop at Charles Darwin University was filled with joy, enthusiastic attendees and extra degrees of warm.

This was a Self-Organised workshop led by COMBINE with help of local organisers mainly from the Menzies School of Health Research. After some emailing back and forth, three months before the workshop, we planned to have an R data carpentry workshop based on the answers of a custom-made survey (prepared by Steven). Then, two COMBINE instructors were called, so Westa and I volunteered for this workshop and started with preps. The workshop was capped for 30 people with a nominal fee to cover for catering ($50 AUD). People were well-disposed and a good sign was that 45 days before the workshop the registration reached its limits. Later on, there were 2 dropouts and we were able to add 2 people from the waiting list.

First Analysis of Instructor Training Data

Fri, 20 May 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Following up on [Wednesday's post about instructor training stats]({{site.baseurl}}/blog/2016/05/looking-for-a-model.html), Erin Becker (Data Carpentry's new Associate Director) has [posted an analysis](http://www.datacarpentry.org/blog/instructor-metrics/). I was very surprised to discover that less than 20% of people trained over a year ago haven't taught yet: I believed the number to be much higher. 51% of those trained in the last 12 months haven't taught yet, but that's less surprising, since in many cases there simply hasn't been time. Overall, we seem to be doing pretty well...

A Common Sense Review of a Software Carpentry Workshop

Wed, 18 May 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Re-posted with permission from the author's blog.

Rationale

This Fall, I am teaching graduate-level biostatistics. I have not had the good fortune of teaching many graduate-level offerings, and I am really excited to do so. A team of top-notch big data scientists are hosted at NCEAS. They have recently formed a really exciting collaborative-learning collective entitled ecodatascience. I was also aware of the mission of Software Carpentry but had not reviewed the materials. The ecodatascience collective recently hosted a carpentry workshop, and I attended. I am a parent and use common sense media as a tool to decide on appropriate content. As a tribute to that tool and the efforts of the ecodatascience instructors, here is a brief common sense review.

Looking for a Model

Wed, 18 May 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Updated: this CSV file has information on who taught when. The three columns are the person's unique identifier, the date on which they first qualified, and the dates on which they taught. (If someone has taught multiple times, there is one record for each teaching event.) People who haven't taught at all are at the bottom with empty values in the third column. Erin Becker's analysis of this data is posted on the Data Carpentry blog and discussed here.

R Instructor Training Applications Open

Mon, 16 May 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Thanks to generous sponsorship from the [R Consortium](https://www.r-consortium.org/), Software Carpentry is running a two-day R instructor training class in Cambridge, UK, on September 19-20, 2016. If you are active in the R and/or Software and Data Carpentry communities, and wish to take part in this training, please fill in [this application form](https://docs.google.com/forms/d/1py7-vm6ZGHKZD_lixAlaAeQnWk9llEnK5sOEydI2Ipc/viewform). We will select applicants, and notify everyone who applied, by June 30, 2016; those who are selected will be responsible for their own travel and accommodation. If you have any questions, please mail [training@software-carpentry.org](mailto:training@software-carpentry.org).

Please note that as a condition of taking this training:

Software Carpentry in Brisbane

Sat, 14 May 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Brisbane Software Carpentry sputtered into life some time in 2014 when scattered local supporters finally met and began to form a group. Having heard about the initiative via Twitter, I managed to contact Nathan Watson-Haigh who got me on to the Aus/NZ mailing list. Then I met Philipp Bayer (now in Perth) and we started planning our first workshop at The University of Queensland. PyCon Australia was coming up in Brisbane and we were able to get Damien Irving from Melbourne and Tim McNamara from New Zealand to teach for us as they were in town for that. So we ran our first ever Software Carpentry workshop with Python at UQ in July 2014.

First bimonthly report from 2016 Steering Committee

Wed, 11 May 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

On May 8th and 9th the Steering Committee had an in-person meeting at Cold Spring Harbour Laboratory to conclude the discussions during the first two months and make plans for the next ten months of activities. In the following weeks we will write more about the results of the in-person meeting on the blog but for now we want to give you an overview.

Partnerships

Our Executive Director is doing an amazing job increasing the number of Software Carpentry partners. Last year, we were asked to provide a join partnership agreement with Data Carpentry and after months of work this agreement is now available.

First Conference of Research Software Engineers: Call for Participation

Tue, 10 May 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The [RSE Conference](http://www.rse.ac.uk/conf2016) on September 15-16, 2016, in Manchester will be the first conference to focus exclusively on the issues that affect people who write and use software in research. We're looking for submissions that will investigate and communicate ideas and expertise from the RSE community.

This is not a standard academic conference! We welcome researchers, but we also want to hear from people who may not typically attend conferences.

24 April - 4 May, 2016: Subcommittees and Task Forces, Partnerships, Instructor Training, A Vacancy, Lab Meeting, Bug Barbeque, Discuss, and New Videos and a Book

Mon, 09 May 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Highlights

The 2016 Steering Committee invites you to help organize and develop resources and activities to support our community members by proposing new initiatives in the form of subcommittees and task forces. Full instructions on the proposal process can be found here. The Steering Committee will gladly help! Also check out the subcommittee listing for information on existing and past subcommittees.
Joint partnerships are now offered by Data Carpentry and Software Carpentry! Read the partnership information on how to become a partner and get in touch.

Vacancy

Data Carpentry is hiring a Deputy Director of Assessment. Visit the Data Carpentry jobs page for a full job description and application procedure.

Events

Tune in to the Software Carpentry Lab Meeting on May 10th to discuss what’s new and upcoming in the community. Check out the Lab Meeting Etherpad for the schedule and connection details.
Join the worldwide Software Carpentry Bug Barbeque on June 13 to help fix bugs in Version 5.4 of Software Carpentry lessons before publication. All contributors will receive credit for your hard work with a citable object.

Conversations

A recent conversation on the discuss mailing list prompted the creation of two new issues open for discussion in GitHub:
- How should we deal with high-volume discussions on the mailing list?
- Is the Code of Conduct saying what the community want/need it to say and how should it be enforced?

Videos and Books

Greg Wilson published a video of “things not to do while teaching”. The material is great to use during instructor training to demonstrate to instructor trainees what we’re trying to avoid when teaching and is based on suggestions made in this discussion on GitHub.
Support Data Carpentry by buying the book How to Be a Modern Scientist written by Jeff Leek. The book includes guides for reviewing papers, reading papers, career planning, and giving talks.
Iñigo Aldazabal Mensa (CSIC-UPV/EHU) provided a video of him explaining Software Carpentry in great detail at the 2016 HPC Knowledge Portal Conference - Software Carpentry: teaching computing skills to researchers.

Other

Did you recently participate in a Software Carpentry workshop and had questions that weren’t answered by the lessons? Do you teach workshops and hear questions from participants that can’t be addressed by the existing materials? Please add these questions to this post.
What have we learned from our lesson discussion sessions so far? We welcome all new instructors to provide feedback about their experience of the sessions by commenting on the post.
Aleksandra Pawlik recently blogged about her experience when she visited South Africa for the first face-to-face Software and Data Carpentry Instructor Training in Africa.
Planning on running a Software Carpentry workshop with R? Read how three sticky notes and chocolates helped participants in the Sotware Carpentry workshop at Griffith University.
Staff members from iDigBio gave their perspective on attending a Software Carpentry workshop. They encourage all researchers who’d like to work with the iDigBio datasets to particpipate in a workshop near them or request one at their own institution.
16 workshops were run over the past 16 days. For more information about past workshops, please visit our website.
Upcoming Workhshops

Our Code of Conduct

Mon, 09 May 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The amazing Software and Data Carpentry community of instructors and learners is the foundation of our organizations. We have more than 500 instructors from 30 countries and have had over 20,000 learners in our workshops.

Software and Data Carpentry are community driven organizations. We value the involvement of everyone in this community - learners, instructors, hosts, developers, steering committee members, and staff. We are committed to creating a friendly and respectful place for learning, teaching and contributing. All participants in Software and Data Carpentry events or communications are expected to show respect and courtesy to others.

Software Carpentry Bug BBQ

Wed, 04 May 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Software Carpentry is having a Bug BBQ on June 13th

Software Carpentry is aiming to ship a new version (5.4) of the Software Carpentry lessons by the end of June. To help get us over the finish line we are having a Bug BBQ on June 13th to squash as many bugs as we can before we publish the lessons. The June 13th Bug BBQ is also an opportunity for you to engage with our world-wide community. For more info about the event, read-on and visit our Bug BBQ website.

Software Carpentry with R at Griffith University

Wed, 04 May 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We had 27 people register to attend, with 22 attending the first day. We offered these workshops free of charge so we had been cautious about drop-out numbers.

I had 2 who had previously mentioned they were only coming to partial sessions and one dropped out the next day. We accepted everyone who registered and attempted the software install (our capacity was 30).

Save the Date: Software Carpentry Lab Meeting May 10

Tue, 03 May 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Software Carpentry’s next Lab Meeting calls will be at 14:00 UTC and 23:00 UTC, May 10th; all are welcome to join in to discuss what’s new and upcoming in the Software Carpentry community. Connection details are on the Lab Meeting Etherpad.

Our first feature for the month is the new Subcommittee and Task Force program, recently announced by the 2016 Steering Committee. Following on the success of our subcommittees in 2015, the Steering Committee has decided to open up the process of proposing and running projects and committees that advance Software Carpentry’s mission and enrich our community to all of our community members. More details on the proposal process can be found here; tune into the Lab Meeting to here more comments and ask questions from the program designers.

New Joint Partnerships with Data Carpentry

Mon, 02 May 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We’ve been hearing of the interest of organizations to build local capacity for training and to be able run both Data and Software Carpentry workshops. We are excited to announce that Data Carpentry and Software Carpentry are now offering joint partnerships! These partnerships will give member organizations the benefits of running workshops from either the Software Carpentry or Data Carpentry community. At the Silver and above tiers there will also be instructor training and capacity building services provided.

Buy This Book and Support Data Carpentry

Fri, 29 Apr 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Thanks to the efforts of Len Epp at Leanpub and Tracy Teal at Data Carpentry, 50% of the royalties from the book *[How to Be a Modern Scientist](https://leanpub.com/modernscientist/)* will be donated to Data Carpentry. The book is pay what you want with a suggested price of $10. I am very excited to help support the efforts of Data Carpentry since I believe data science education is a fundamental need in the modern scientific era.

About the book: The face of academia is changing. It is no longer sufficient to just publish or perish; we are now in an era where Twitter, Github, Figshare, and Alt Metrics are regular parts of the scientific workflow. Here I give high level advice about which tools to use, how to use them, and what to look out for. This book is appropriate for scientists at all levels who want to stay on top of the current technological developments affecting modern scientific careers. The book is based in part on the author’s popular guides including guides for

Summarizing Our Lesson Discussion Sessions

Fri, 29 Apr 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

For the first four months of this year, we ran hour-long lesson discussion sessions to give people going through instructor training a chance to ask questions of people who had taught our material several times. Trainees told us they were useful for getting information and as a way to meet more of the community. We have now decided to merge those sessions with our weekly workshop debriefing sessions in order to turn up the volume on both aspects, so this seems like a good time to summarize what we’ve learned so far ourselves.

Call for Software Carpentry Foundation Subcommittees and Task Forces

Sun, 24 Apr 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Purpose and Rationale

Software Carpentry is volunteer-powered organization that is rapidly growing. One of the greatest things about Software Carpentry is the huge diversity of people that have come together to collaborate, teach, and share ideas and resources. Your contributions to the community have made Software Carpentry a huge success!

To meet the needs of our growing community, the 2015 Steering Committee Launched the first Subcommittee program to allow groups of members to organize and develop resources and activities that support our members.

Data Carpentry is Hiring a Deputy Director of Assessment

Sun, 24 Apr 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

[Data Carpentry is hiring](http://www.datacarpentry.org/jobs/)!

Data Carpentry seeks to hire a full-time staff member to direct its assessment activities. This person will design, implement, monitor, analyse, and report on a comprehensive system of metrics to help the Data Carpentry project and its sibling organization, Software Carpentry, evaluate the impact and effectiveness of the training they offer, to both learners and instructors.

As the Deputy Director of Assessment, you will have primary responsibility for developing methods and standards for the evaluation of all aspects of Data Carpentry’s training including relevance of curriculum, learning experience, long-term adoption of tools and skills and impact on productivity and reproducibility. You will also work with the Software Carpentry Foundation to build evaluation of the instructor development program, including effectiveness of instructor training and mentorship and the longer term impacts of instructor training on career development for instructors. You will also have the opportunity to collaborate with the training coordinators from related organizations to coordinate strategies and initiatives.

Questions, Answers, and Lessons

Sun, 24 Apr 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

While working on an outline of a new lesson on Python, I began thinking about the overall coherence of what we teach. In particular, I started to worry that we might be teaching some things because we teach them, i.e., that the curriculum might lose its connection to researchers' actual issues.

One method for keeping things grounded in the other field I still occasionally work in (empirical software engineering) is called Goal, Question, Metric. As the name suggests, it defines three questions: what are you trying achieve, what questions do you need answered in order to achieve it, and what metrics will you accept as answers to those questions. An educational equivalent is Question, Answer, Lesson: what questions do novices have, what answers do competent practitioners give them, and what lessons are needed to teach those answers. (The "do novices have" modifier is crucial: in order for our workshops to be appealing, they must answer the questions that novices actually have, not the ones we wish they would ask.)

Welcome to Google Summer of Code Students

Sat, 23 Apr 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Thanks to a lot of hard work from some members of our community and members of others open source scientific projects we are happy to welcome the students and mentors that will participate on Google Summer of Code this year under NumFOCUS umbrela.

Student	Project	Mentors
Bhargav Srinivasa	Dynamic Topic Models for Gensim	Lev Konstantinovskiy, Radim Rehurek and Devasena Inupakutika
Chris Medrela	Manage workflow for Software Carpentry and Data Carpentry instructor training	Greg Wilson and Piotr Banaszkiewicz
Akash Goel	Upgrade to datapackage.json standard for EcoData Retriever	Henry Senyondo and Ethan White
Patrick Kofod Mogensen	Improving the state of Optim.jl for JuliaOpt	Miles Lubin
Prerit Garg	Result-aggregation server for the installation-test scripts for Software Carpentry and Data Carpentry	Piotr Banaszkiewicz and Raniere Silva
Ramchandran Muthukumar	Presolve Routines for LP and SDP within Convex.jl for JuliaOpt	Madeleine Udell
Hannah Aizenman	Categorical Axis for matplotlib	Michael Droettboom and Thomas Caswell

This is the second year that NumFOCUS participate on Google Summer of Code and they have more than the double of slots from last year. Selecting the students was very difficult due the high quality of the proposals that are archived on GitHub.

Instructor Training in South Africa

Thu, 21 Apr 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Thanks a lot of hard work from Anelda van der Walt and the support from the [North-West University](http://www.nwu.ac.za/), [University of Cape Town](http://www.uct.ac.za/) and [Talarify](http://www.talarify.co.za/) we were able to run the first face-to-face instructor training in South Africa. We trained 23 new instructors from South Africa, Namibia, Zimbabwe and Kenya. The event was a great success and we are looking forward to expansion on Software and Data Carpentry workshops South Africa and other African countries.

Anelda has been developing Software and Data Carpentry in South Africa for a while now and her efforts led to this first face-to-face Instructor Training. We started off on Sunday, 17th April . evening with a short ice-breaking session of lightning talks during which the participants talked about “The coolest thing about their job” for 2 minutes. Everyone had something interesting to say. What was even more impressive, nobody ran over time, which is not the usual case.

10 tips and tricks for instructing and teaching by means of live coding

Wed, 20 Apr 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

One of the key teaching practices used at Software and Data Carpentry workshops is ’live coding’: instructors don’t use slides, but work through the lesson material, typing in the code or instructions, with the workshop participants following along. Learning how to teach using live-coding is best done in practice, with feedback from peers (this is why it is included in instructor training). Nonetheless, this post lists ten tips and tricks to help you get started.

So You Want to Make a Screencast

Wed, 13 Apr 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I am the author of [Learning Cython](http://shop.oreilly.com/product/0636920046813.do), a new series of videos teaching [Cython](http://cython.org/). There are over 70 videos, making up around five and a half hours of content. If you're involved in education, you may have considered making a screencast like these and I'd like to give a short overview of what that entails.

Publisher

You may be happy to self-produce and self-host, and I would be too except that I was approached by a publisher and made an offer to produce the videos for distribution on their site. How this happened is interesting: in 2015, on a whim, I decided to submit a Cython talk proposal for PyCon AU. My talk got accepted, and a few weeks before the conference the publisher contacted me and made the offer.

Carpentry week 2016 at the University of Oslo

Tue, 12 Apr 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Cross-posted from the author’s blog.

In March 14-18 2016 we organised the first Carpentry week at the University of Oslo. After a mini-Seminar on Open Data Skills, there was a Software Carpentry workshop, two Data Carpentry workshops and a workshop on Reproducible Science as well as a ‘beta’ Library Carpentry workshop.

The Software and Data Carpentry effort at the University of Oslo, aka ‘Carpentry@UiO’, really started in 2012 when I invited Software Carpentry to give a workshop at the university. The then director, Greg Wilson, came himself and gave an inspirational workshop – recruiting Karin Lagesen and I to become workshop instructors in the process. Karin and I graduated from instructor training spring 2013 and have been giving a couple of workshops in Oslo and elsewhere.

Installation Video Tutorials

Tue, 12 Apr 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I recently created videos for installing the software needed for most Software Carpentry workshops. Links for these videos are now added to the lessons template for a workshop and listed below.

Windows:

Mac:

So far, we only have Windows and Mac videos. If anyone wants to make Linux versions, I’m happy to answer any questions. Or if you want to make the recordings and not the voiceover, I’m willing to narrate.

Changes on Mentoring Subcommittee

Mon, 11 Apr 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The first quarter of 2016 is almost over and so far the Mentoring Subcommittee have been very quiet although we have continued hosting the post workshop debriefing sessions and the pre-workshop helping session.

We have been quiet because we were busy with the Steering Committee election and reorganization of the volunteer hours from our members since six of them (Belinda Weaver, Bill Mills, Karin Lagesen, Kate Hertweck, Rayna M Harris and I) were elected as Steering Committee members. In addition to this some members had to temporarily reduce their volunteer hours for various reasons but are now back to help us.

Designing a New Novice Python Lesson

Sun, 10 Apr 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Last November, I volunteered to pull together a new full-day lesson on Python suitable for people with no previous programming experience. It has taken longer to come together than I expected, but that’s partly because I saw it as an opportunity to create a full-length example of the “backward” lesson design method we teach in the instructor training class.

What we have so far is in this GitHub repository, which is rendered at this site. We would be very grateful for input: if you think crucial topics have been left out, please say so, while if there’s stuff that could be removed or re-ordered, please say that as well.

Maintaining a Lesson

Mon, 04 Apr 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

After two years as a Software Carpentry lesson maintainer of r-novice-inflammation, I am stepping down so that I can spend more time on my research (this thesis appears unwilling to write itself) and with my family. This therefore seems like a good time to summarize what’s happened and what I’ve learned for the benefit of anyone planning to become a maintainer.

A Bit of Background

Software Carpentry really started growing the number of workshops and instructors in 2012. At the time, there were two main sets of lessons in place: the version 4 SWC lessons written by Greg Wilson and others in 2010-11, and a set developed by The Hacker Within, a student group at the University of Wisconsin - Madison. (These were referred to as the THW lessons, and can be found in the old boot-camps repository.)

AMY release v1.5.1

Sun, 03 Apr 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We’re very pleased to announce the newest (v1.5.1) release of AMY, our workshop management tool. This release is special because:

it contains more bugfixes than new features
we had a number of submissions from prospective Google Summer of Code 2016 (GSoC) students.

GSoC students that submitted fixes or new features to AMY:

Shubham Singh added “Notes” field to instructor profile update form
Nikhil Verma found and fixed “List duplicates” page error when no duplicates existed
Shubham Singh added new tag “hackaton”
Chris Medrela found and fixed 404 page for revisions that didn’t exist

Additionally Nikhil Verma is working on getting certificates to generate from within AMY, but this will probably land in v1.5.2.

rOpenSci is Looking for a Community Manager

Tue, 29 Mar 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Thanks to their recent funding, rOpenSci is now looking for a community manager. Their mission is to expand access to scientific data and promote a culture of reproducible research and sustainable research software; to aid that, the community manager’s job will be to broaden the understanding and reach of rOpenSci to the researcher community. For details, please see the full announcement.

An R-based Instructor Training Sponsored by the R Consortium

Mon, 28 Mar 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The R consortium Infrastructure Steering Committee awards have been officially announced and we are happy to confirm that our proposal to organise an R-based instructor training was successful.

We will organise an in-person instructor training that will focus on R. While the content of the training will not change as such, code-related exercises and examples that the participants will be working with will be focused on R, and will be made available to the R instructor community to help them improve their teaching. The full proposal is available here.

Announcing the Open Science Grid User School 2016

Mon, 28 Mar 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

If you could access hundreds or thousands of computers for your scholarly work, what could you do?

How could it transform your work?

What discoveries might you make?

We are seeking applicants for the 2016 Open Science Grid (OSG) User School, which takes place 25–29 July at the beautiful University of Wisconsin in Madison. Participants will learn to use high throughput computing (HTC) to harness vast amounts of computing power for research, applicable to nearly any field of study (e.g., physics, chemistry, engineering, life sciences, earth sciences, agricultural and animal sciences, economics, social sciences, medicine, and more).

Hello, Spatio-temporal Data Carpentry

Mon, 28 Mar 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Data Carpentry have just announced the availability of an introductory lesson on working with geospatial data developed in conjunction with NEON (the National Ecological Observatory Network). For details, please see the full announcement on Data Carpentry’s blog.

2015 Annual Report

Wed, 23 Mar 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We’re pleased to announce the publication of the first SCF annual report summarizing what we have done this year to to further our goals of promoting reproducibility and reliability in all branches of science, and helping researchers be more productive.

You can download the annual report here.

Python Education Summit at PyCon 2016

Wed, 23 Mar 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Python Education Summit is a gathering of educators from all venues (school, colleges, universities, community workshops, online courses, government) that share interest and passion in teaching Python. It is an interesting opportunity for Software Carpentry instructors to get inspiration and tips from educators in completely different contexts.

Schedule

The Summit will take place at the Oregon Convention Center, same site of PyCon, on Sunday, May 29th, the 2nd day of the tutorials and the day before the main conference starts. It will run from 9am to 4:30pm.

4 - 18 March, 2016: Instructor Trainee Mentoring, Debriefing vs Lesson Sessions, Version Control, Big Data in Biology Summer School, New Lessons

Fri, 18 Mar 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

##Instructor Training

How can we help instructor trainees complete their training? Belinda Weaver is working hard to support new instructors in Australia. Do you have any other ideas?
Several opportunities are now available for new and experienced instructors to join the conversations about workshops. Should we be combining various sessions to better integrate new and old instructors and give more options for participation?
You can also read about the abovementioned discussions in the latest summary of instructor debriefing rounds 4 and 5.

##Conversations

Software and Data Carpentry Instructor Training Comes to Africa

Fri, 18 Mar 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

North-West University eResearch, UCT eResearch, and Talarify are excited to announce that a Software & Data Carpentry Instructor Training event will take place in Potchefstroom, North-West Province, South Africa from 17 - 20 April 2016.

Our lead trainer will be Aleksandra Pawlik and several of the more experienced South African instructors will also join the workshop to work with the trainee instructors.

In line with a previous post from Belinda Weaver about helping new instructors through the pipeline, this workshop will form part of a larger 12-month programme to help new instructors truly integrate into the community. The programme, an initiative of the three hosting organisations and currently under development, will include supporting instructor trainees to:

New Maintainers

Thu, 17 Mar 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are pleased to announce that [Harriet Dashnow]({{site.baseurl}}/team/#dashnow_harriet) and [Daniel Chen]({{site.baseurl}}/team/#chen_daniel) have agreed to take over maintenance of [the R inflammation lesson]({{site.github_url}}/r-novice-inflammation). Our thanks to them for volunteering, and to [John Blischak]({{site.baseurl}}/team/#blischak_j) and [Denis Haine]({{site.baseurl}}/team/#haine_denis) for all their hard work over the past year and more.

We are also grateful to Erik Bray, who has volunteered to take over maintenance of our Windows installer. As we have grown, we have come to depend on our community to do more than just teach (and yes, I just said “just”). If you would like to get involved in maintaining lessons, mentoring new instructors, or other non-teaching tasks, please give us a shout.

2016 Post-Workshop Instructor Debriefings, Rounds 04 and 05

Thu, 10 Mar 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

On February 23, Rayna Harris, Sheldon McKay, and Tiffany Timbers ran the 4th round of post-workshop instructor debriefing. On March 8th, Rayna Harris, Raniere Silva, and Belinda Weaver ran the 5th round of post-workshop instructor debriefing.

New lesson materials

Data Carpentry Genomics Lesson. Greg Wilson recently sent out an email about updating the lesson maintainer roles for Software Carpentry to jump-start finalizing the Ecology and Genomic lessons. These aren’t final yet, but the Iowa State University instructors spent about considerable time putting together a genomic lesson. You can view the Git repository with their lessons here.

3rd Annual Big Data in Biology Summer School

Tue, 08 Mar 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Center for Computational Biology and Bioinformatics at The University of Texas at Austin is hosting the 3rd Annual Big Data in Biology Summer School May 23–26, 2016.

The 2016 Summer School offers eleven intensive courses that span general programming, high throughput DNA and RNA sequencing analysis, proteomics, and computational modeling. These courses provides a unique hands-on opportunity to acquire valuable skills directly from experts in the field. Each course will meet for three hours a day for four days (either in the morning or in the afternoon) for a total of twelve hours.

A Proposal for Helping Instructor Trainees Finish

Tue, 08 Mar 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Instructor training is one way we grow our community worldwide. Yet many people who go through instructor training never go on to teach at a Software Carpentry workshop. Can that be fixed? I suggest that shepherding people though the final stages might help with completion rates.

When the idea of running instructor training in Brisbane was originally mooted, I set up a survey, very early on, to record expressions of interest from people who might want to train. I tweeted that several times in the months leading up to the training. The survey captured name, email address, institution, discipline and how each applicant had heard about the survey. I wish now I had also asked ‘why do you want to train’ or ‘why do you deserve a place over someone else?’ as that would have helped me sift through the candidates.

Should We Combine Debriefing and Lesson Discussion?

Tue, 08 Mar 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Belinda Weaver recently posted [a proposal for helping instructor trainees finish]({{site.baseurl}}/blog/2016/03/proposal-instructor-trainees.html). While reading her post and writing out [the current instructor training workflow]({{site.github_url}}/board/pull/81) for the benefit of GSoC students, I started wondering if it might also make sense to combine the workshop debriefing sessions with the trainee mentoring sessions: a couple of regularly-scheduled slots each week (times chosen to accommodate all comers) would be easier to wrangle, and trainees would get to meet more existing instructors and hear about their in-class experiences. We'd still need some sort of sign-up mechanism to keep numbers manageable, and we'd need someone to report back on which trainees had asked well-informed questions, and it would mean less coupling between the discussion and the specific lesson(s) trainees were working on, but the first two aren't new work (we're doing them already) and the third might actually be a good thing, particularly if we follow up on another suggestion and have people attend a discussion *before* submitting their lesson change. Please add comments to this blog post to let us know what you think of the idea, and to [Belinda's]({{site.baseurl}}/blog/2016/03/proposal-instructor-trainees.html) to give her feedback as well.

Complexity vs. Subtlety

Sat, 05 Mar 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I gave a lightning talk on Software Carpentry for the [OICR](http://oicr.on.ca/) yesterday, and in discussion afterward, [Jonathan Dursi](http://www.dursi.ca/) made an observation that I've been thinking about since. He wondered whether the key difference between commercial software and scientific software is *complexity* versus *subtlety*. For example, the software that manages workplace insurance payouts for the province of Ontario is complex because it has to handle every regulatory change since the mid-1920s. None of the its rules and exceptions are intellectually taxing, but by the time you turn them into a service, provide a dozen different interfaces for different business roles, and make the whole thing fault tolerant, the software is incredibly tangled.

A lot of scientific software is relatively straightforward by comparison, so long all you look at is the control flow. It’s the specific calculations that are hard: what differencing scheme or statistical test to use, what convergence criteria or significance measure to apply, and so on. And yes, there are a lot of fiendishly tricky algorithms in science, but they’re often hidden in libraries built and maintained by specialists who work and think like software engineers.

16 February - 3 March, 2016: New Steering Committee, Software Carpentry Value Proposition, Webinar, Vacancies, Community Building, Instructor Training, and Modern Scientific Authoring

Thu, 03 Mar 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

##Highlights

Our new steering committee has been elected. Congratulations to Rayna Harris, Kate Hertweck, Karin Lagesen, Bill Mills, Raniere Silva, Belinda Weaver, and Jason Williams!
Looking for a good model to show potentially interested stakeholders what value Software Carpentry brings? Jonah Duckles, our executive director, shared his views. Let us know how we can improve on this?

##Webinars

DataONE is hosting Greg Wilson for a webinar titled Research Computing Skills for Scientists: Lessons, Challenges, and Opportunities from Software Carpentry on 8 March 2016 at 9am Pacific Time.

##New

Communities: The Foundation of Impactful Workshops

Tue, 01 Mar 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I spend a good deal of my time trying to communicate with member organizations about what it is that the Software Carpentry Foundation can do to help them meet their own goals. It is part of my job to showcase to them the return on investment that they’ll see in various areas. The three areas in which this is most apparent are impacts on skills transfer to learners who attend workshops, the lesson material that is publicly available and built by the community, and the capacity building that comes from mentoring instructors who are thinking about impactful instruction in short workshops.

Applications due March 1st: 2016 eScience Data Science for Social Good summer program

Fri, 26 Feb 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The University of Washington eScience Institute, in collaboration with Urban@UW and Microsoft, is excited to announce the 2016 Data Science for Social Good (DSSG) summer program. The program brings together data and domain scientists to work on focused, collaborative projects that are designed to impact public policy for social benefit.

Modeled after similar programs at the University of Chicago and Georgia Tech, with elements from our own Data Science Incubator, sixteen DSSG Student Fellows will be selected to work with academic researchers, data scientists, and public stakeholder groups on data-intensive research projects. Graduate students and advanced undergraduates are eligible for these paid positions.

Bioinformatics Training Impact Coordinator

Mon, 22 Feb 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Bioinformatics Training Facility of the School of Biological Sciences, University of Cambridge is looking for a Training Impact Co-ordinator. The post-holder will develop (building on existing work), implement, monitor, analyse and report on a comprehensive system of training metrics/key performance indicators across the portfolio of [ELIXIR](http://www.elixir-europe.org/) and [EXCELERATE](https://www.elixir-europe.org/news/elixir-accelerates-major-horizon-2020-funding) bioinformatics training activities. For more information, please see [the full ad](http://www.jobs.cam.ac.uk/job/9571/).

Welcome to the 2016 Steering Committee!

Sat, 20 Feb 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The results are in. Your 2016 steering committee, in alphabetical order by last name is:

Rayna Harris
Kate Hertweck
Karin Lagesen
Bill Mills
Raniere Silva
Belinda Weaver
Jason Williams

Thank you to all of the candidates for standing for election. We look forward to an exciting year of the new committee’s contribution and leadership.

For those interested, the raw results are available here.

More of a Difference Than You Realize

Fri, 19 Feb 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We received this after an online instructor training workshop earlier this week, which reminded me that small differences for some people can be large ones for others:

Thanks for a great workshop the last two days… I wanted to share a separate positive comment that I should have included on the Etherpad: I’m profoundly/severely hard-of-hearing in both ears, and depend quite a bit on lip-reading when listening to people. As such, I have great difficulty with online material if the audio is bad, the speaker is not well lit, or the speaker is simply not on video.

Building Software, Building Community: Lessons from the rOpenSci Project

Wed, 17 Feb 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

[Carl Boettiger]({{site.baseurl}}/team/#boettiger_carl), [Scott Chamberlain]({{site.baseurl}}/team/#chamberlain_s), [Ted Hart]({{site.baseurl}}/team/#hart_t), and [Karthik Ram]({{site.baseurl}}/team/#ram_k) have just published a paper titled "[Building Software, Building Community: Lessons from the rOpenSci Project](http://openresearchsoftware.metajnl.com/articles/10.5334/jors.bu/)", in which they describe what they've learned by growing the [rOpenSci](http://ropensci.org/) project There are a lot of great ideas here that other groups could borrow, and of course you can keep up with their news by [following their blog](http://ropensci.org/blog/).

A Counterpoint to Collaborative Lesson Design

Tue, 16 Feb 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Discussion of our proposed lesson on [on modern scientific authoring]({{site.github_url}}/modern-scientific-authoring) is an instructive counterpoint to our previous post extolling the virtues of [collaborative lesson development]({{site.baseurl}}/blog/2016/02/designing-lessons-collaboratively.html). The aim of the lesson is to show researchers how to write and publish in the early 21st Century---or more honestly, to persuade them to stop mailing each other copies of Microsoft Word files and start using something else. After a long-winded opening (which I'll cut substantially), the [current introduction]({{page.site}}/01-mess.html) summarizes the strengths and weaknesses of six options:

WYSIWYG on the desktop (Microsoft Word).
WYSIWYG on the web (Google Docs).
Desktop typesetting (LaTeX).
Web-based typesetting hybrids (Authorea, Overleaf).
HTML.
Markdown.

It concludes that for the foreseeable future, many researchers will continue to prefer WYSIWYG tools rather than typesetting tools requiring compilation (such as LaTeX and Markdown). However, since most researchers are already familiar with WYSIWYG tools, and since typesetting tools are easier to integrate into reproducible workflows, the lesson will focus on LaTeX for mansucripts and Markdown for the web.

Checking the Balance

Tue, 16 Feb 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Added 2016-02-22: this strong critique of the Terrell et al preprint mentioned in the opening paragraph of this post is worth a careful read.

It's been a depressing couple of weeks. On top of yet more reports of universities turning a blind eye to sexual harassment for years, a new paper of gender bias in open source shows that, "...women's contributions tend to be accepted more often than men's. However, when a woman's gender is identifiable, they are rejected more often." This comes on top of earlier studies (like this one) showing that women are substantially under-represented in online forums like Stack Overflow, even when compared to computing as a whole.

Our New Instructor Pipeline

Tue, 16 Feb 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Last fall, we decided to [reboot]({{site.baseurl}}/blog/2015/09/rebooting-instructor-training.html) our [instructor training course]({{site.github_io_url}}/instructor-training/). We've [tried a lot of things]({{site.baseurl}}/blog/2015/12/three-flavors-of-instructor-training.html) since then, and one of the biggest successes has been [our new checkout procedure]({{site.baseurl}}/blog/2015/12/instructor-training-checkout-procedure.html). In brief:

After completing the instructor training course, the trainee picks one of the core lessons from each of the Carpentries she wants to teach and submit a new exercise for that lesson.

Designing Lessons Collaboratively

Mon, 15 Feb 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A few days ago, I asked for feedback on a new Python lesson aimed at people who've never programmed before. The [outline]({{site.github_url}}/python-novice-gapminder/blob/gh-pages/index.md) had already received several rounds of feedback from a handful of people, but there were still lots of comments:

As always, the choice of tools attracted a lot of discussion. Jeremy Metz opened by saying, “I worry that the use of a more abstract environment like the Jupyter Notebook might confuse and add an additional barrier to people wanting to ‘really’ use Python.” I agree that the Notebook imposes an extra cognitive load, since it’s so different from anything else novices are likely to have seen, but it has become the tool of choice for many scientists for good reasons: it’s stable, cross-platform, encourages reproducible practices, and has a great, supportive community. We’re also starved for alternatives: the audience for this class isn’t required to have seen the shell, so running scripts from the command line is out, and all of the Python IDEs we’ve tried have significant shortcomings for our audience.

NGS Summer 2016: Analyzing Next-Generation Sequencing Data

Sun, 14 Feb 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A two-week residential course on next-generation sequencing is being offered at the Kellogg Biological Station at Michigan State University on August 8-19, 2016. The course's directors are Prof. Matt MacManes (U. New Hampshire) and Prof. Meg Staton (U. Tennessee, Knoxville), and instructors will include Prof. Ian Dworkin (McMaster U.), Prof. Torsten Seemann (U. Melbourne), Shaun Jackman (PhD candidate, UBC) and others. More information, or to register, please see [http://bioinformatics.msu.edu/ngs-summer-course-2016](http://bioinformatics.msu.edu/ngs-summer-course-2016).

Note: if you are running a course that might be of interest to our community, please let us know.

Announcing New Unix Shell Maintainer

Fri, 12 Feb 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

After an application and selection process, Ashwin Srinath has been selected as a new co-maintainer of the Unix Shell lesson, joining Gabriel Devenyi and replacing outgoing maintainer Christina Koch.

Introducing Ashwin

Ashwin is a Ph.D. candidate in Mechanical Engineering at Clemson University, and soon-to-be member of Clemson’s Cyberinfrastructure Technology Integration (CITI) research computing group. His research areas include computational fluid dynamics, and high-performance computing. He spends most days programming and making mistakes, and some days teaching others how not to make them. Ashwin has formerly served as co-maintainer for the Software Carpentry MATLAB materials.

2016 Post-Workshop Instructor Debriefing, Round 03

Thu, 11 Feb 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

On Tuesday, February 9th, we ran the 3rd round of Post-Workshop Instructor Debriefing Sessions. Rayna and Raniere hosted the morning debriefing session. The even session was cancelled due to low attendance.

New lesson materials

Blogging as a Collaborative Git Exercise. The SFU group created a new Git exercise in which pairs of learners wrote guest blog posts for the others site. Then learners experienced conflict resolution with a controlled lesson where paid edited the same line of the guest blog post. This was an engaging exercise that allowed learned to see successful collaboration and conflict resolution. Stay tuned for a dedicate blog post from Bruno Grande about the exercise.

What worked well

A 5-day Hybrid Data / Software Carpentry Workshop. The 2016-01-25-Utrecht group covered a lot of material in this 5 day workshop (Spredsheets, OpenRefine, Python 3, Unix, SQL, Git, Make, and HPC)! While most of the materials were used as is, updates to the Data Carpentry Python lesson have already been pushed and merged.

Context when teaching Numerical Methods

Thu, 11 Feb 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As a director of the [Centre for Doctoral Training in Next Generation Computational Modelling](http://ngcm.soton.ac.uk/) I spend plenty of time thinking about, working with, and particularly teaching Numerical Methods. The [general intro course](https://github.com/IanHawke/NumericalMethods) I give is Maths focussed, has around 100 participants, has about 50 contact hours over 12 weeks, and gets a range of interested students from Maths, Physics and Engineering.

The past two weeks I’ve been teaching Numerical Methods for the Centre for Doctoral Training in New and Sustainable Photovoltaics. With around a dozen participants, many of whom had minimal theory or coding background, and only 10 hours available, some changes in the teaching were required. Most of these followed Software Carpentry approaches: live coding, Jupyter notebooks, sticky notes, and a set of “authentic” examples to train underpinning principles.

A New Lesson on GAP

Tue, 09 Feb 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are pleased to announce that [Alex Konovalov](http://blogs.cs.st-andrews.ac.uk/alexk/) and his colleagues have created a Software Carpentry-style lesson on [the computational algebra system GAP](http://www.gap-system.org). The lesson is at [http://alex-konovalov.github.io/gap-lesson/](http://alex-konovalov.github.io/gap-lesson/), and the repository for it is at [https://github.com/alex-konovalov/gap-lesson](https://github.com/alex-konovalov/gap-lesson). It was taught at [a recent workshop](https://kkwakwa.github.io/2015-11-16-manchester-codima/), and feedback can be viewed [here](https://github.com/alex-konovalov/2015-11-16-manchester-codima-feedback/blob/master/raw-feedback.md).

We are now starting to develop a lesson on SageMath. We invite collaborators, please watch the repository if you’re interested in following along, and add a comment to this issue if you’re interested in contributing.

Open Science Radio Podcast

Tue, 09 Feb 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Matthias Fromm and Konrad Förstner recently interviewed Greg Wilson for their Open Science Radio podcast. You can listen to it [here](http://www.openscienceradio.de/2016/02/09/osr038-software-carpentry-with-greg-wilson-en/), or explore their other recordings.

Correlations

Mon, 08 Feb 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We've run instructor training both online and in person for several years, so it's time to look at how they compare. The raw data shows:

the event's start date
its unique identifier (which we call a "slug")
whether it was online
how many people took part
how many completed training after this course (both as an absolute number and as a percentage)
how many completed after taking a later course (in both forms)
how many have never completed
how many have taught at least once since taking the course

The plot below then shows completion rates and follow-through teaching rates (as percentages) versus cohort size, tagged by whether the training event was online or in person, for all events that took place at least one year ago. (I've used that cutoff to give participants a fair chance to teach after completing their training.) It also shows the absolute number of participants and the follow-through teaching percentage by date.

Come a Long Way, Got a Long Way to Go

Sun, 07 Feb 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

[Simon Oxenham recently reported](http://bigthink.com/neurobonkers/we-need-to-rewrite-the-textbook-on-how-to-teach) on [a new report](http://www.nctq.org/dmsStage/Learning_About_Learning_Report) from the National Council on Teacher Quality that examined how well teacher training courses and textbooks convey evidence-based teaching practices. The sad answer is, hardly at all:

The report finds that out of 48 texts used in teacher-training programs none accurately described fundamental evidence-based teaching strategies comprehensively. Only 15 percent had more than a single page devoted to evidence-based practices; the remainder contained either zero or only a few sentences on methods that have been backed up by the decades of scientific findings that exist in the field of educational psychology.

2016 Post-Workshop Instructor Debriefing, Round 02

Fri, 05 Feb 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

On January 26, we ran the 2nd round of Post-Workshop Instructor Debriefing Sessions. Rayna and Christina hosted the morning debriefings while Bill and Raniere hosted the evening debriefing. A topic of interest today was on installations. Read on for details.

Thoughts on the “check installation” scripts

There is in an ongoing discussion about whether or not the “check installation” scripts for workshop pages should be updated. So, we asked the attendees how often these were being using and how useful they were. Some comments we received were:

Software Carpentry as a University Course

Fri, 05 Feb 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The inaugural Software Carpentry and Data Carpentry Instructor and Helper Retreat is over! It was a long day, packed with tutorials, demos, and discussions. I (Daniel Chen) led a round table discussion with Tiffany Timbers and Jenny Bryan. You can watch the discussion and/or read the notes on etherpad.

Active Workshops

Tue, 02 Feb 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A few days ago, we were asked what was the greatest number of simultaneous workshops we'd ever run. I didn't know, but it was an easy enough question to answer from our records:

I’m sure better visualizations are possible (if you want to create one, the raw data is here), and we should find some way to show that the burst starting in April 2014 is actually a couple of workshops that were spread out over several weeks, but I think the trend line is pretty cool.

Two Pages of Evidence

Tue, 02 Feb 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Andreas Stefik and his colleagues have written [a two-page summary](http://quorumlanguage.com/evidence/evidence.pdf) of what we actually know about usability and programming language design. There aren't nearly as many results yet as we want or need, but what the people working in the field have shown is that we actually can answer these questions scientifically, and that answering them correctly actually does have an impact.

17 January - 1 February, 2016: SCF Election Candidates, Lessons Learned, Instructor Survey Results, and an Intermediate R Lesson

Mon, 01 Feb 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

##Highlights: Steering Committee Election

The 2016 Software Carpentry Foundation Steering Committee election dates are set for February 15-19. Remember to join our Community Lab Meeting on 9 February to meet the candidates.
Read more about the candidates for the 2016 elections by following the links below:

##Published

SSI is hiring a Communications Officer!

Mon, 01 Feb 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The [Software Sustainability Institute (SSI)](http://software.ac.uk/) is recruiting a [Communications Officer](http://software.ac.uk/news/2016-01-27-were-hiring-communications-officer-help-us-disseminate-software-best-practice) to communicate the activities of the Institute, and raise its profile in the research community and the general public. SSI has been Software Carpentry's long-standing partner and coordinates our workshops in the UK. They are looking for an enthusiastic individual with a track record in dissemination and outreach. Ideally, they'll have had experience of working with researchers and/or writing technical articles, as well as an understanding of what it takes to nurture a great social media presence and successfully promote events.

For more details and to apply, please see the advert on the University of Edinburgh website (search for vacancy reference ID 035330).

Elsewhere on the Web

Fri, 29 Jan 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Our instructors teach a lot more than just Software Carpentry. For example, [Christie Bahlai]({{site.baseurl}}/team/#bahlai_christie) has started teaching a course on "Open Science and Reproducible Research". You can [follow the course blog](https://osrrcourse.wordpress.com/) or check out the materials in [the course repository](https://github.com/cbahlai/OSRR_course)---contributions from the community are very welcome. The course already includes lots of interesting ideas, and there's more to come.

Elsewhere, Tiffany Timbers is organizing several half-day workshops at Simon Fraser University as part of Titus Brown’s Data Intensive Biology training program. These workshops will be hands on and supported by several TA’s in addition to the instructor. All are welcome to participate, regardless of discipline or training level—please feel free to distribute these events widely. And if you cannot attend locally because of geography, please note that several of them are being streamed on YouTube:

A New Version of 'Lessons Learned'

Thu, 28 Jan 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A new version of "Software Carpentry: Lessons Learned" is [now available on F1000](http://f1000research.com/articles/3-62/v2). We think it's an interesting complement to the [instructor survey]({{site.baseurl}}/blog/2016/01/instructor-survey.html), and we hope you enjoy them both.

Instructor Survey Report

Wed, 27 Jan 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Back in the fall we surveyed our then 450-strong instructor community for feedback on what being an instructor has done for them.

To recap, we asked the following questions:

How did you get interested in doing Software Carpentry/Data Carpentry?
How do you use the skills you teach in Software Carpentry/Data Carpentry in your daily work?
What is the benefit (personally and/or professionally) to you in being involved with the Software Carpentry/Data Carpentry Community?
What examples do you have of how the Software Carpentry/Data Carpentry Community have benefited others (students, labs, university groups)?
What suggestions do you have for improvement?

There is a ton of great information in this report that can be useful for building momentum for Software and Data Carpentry instructor training and workshops at your own institution. It is clear that we have a very active and driven pool of instructors. This is a large part of what made me so excited and passionate about the community when I first became an instructor. I hope that we can learn from each other and broaden the impacts that this community is having around the world!

Meet the 2016 Election Candidates

Thu, 21 Jan 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The candidates for the 2016 election are set. The election will take place during the week of February 15th-19th. Look for electionbuddy ballot information in your email if you’re a member.

Meet the candidates at the February 9th Community Lab Meeting Calls (14:00 & 23:00 UTC). Call coordinates will be posted to the etherpad when they’re finalized.

In alphabetical order by last name the candidates are:

Our Introduction to Git Has Been Published

Thu, 21 Jan 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Our short paper "A Quick Introduction to Version Control with Git and GitHub" has been [published in PLOS Computational Biology](http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1004668). It is freely reusable under a Creative Commons Attribution license, and we hope you and your colleagues find it useful.

Presenting Materials for Intermediate UseRs

Mon, 18 Jan 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We’ve been teaching R workshops for over a year now at the University of Melbourne, and one thing we’ve noticed is the disparity in skill level among workshop attendees. Researchers generally fall into one of two catergories:

Absolute novices: those who have heard of R, but have never touched a programming language before.
Regular users: those who are using R in their research, possibly on a regular basis. They can modify scripts, and have a general understanding of the language basics, but want to extend their knowledge.

What Are Your Favorite Cartoons?

Mon, 18 Jan 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As we're updating our lessons this year, I'd like to add a few more cartoons that relate directly to what we teach. Two of my favorites are [this description of how most scientists manage revisions](http://www.phdcomics.com/comics/archive.php?comicid=1531) and [this summary of how many people react to unwelcome evidence](http://www.felienne.com/archives/4404). What are yours, and what lessons would you add them to? Please add links in the comments on this page (and please make sure what you link to has enough information for us to contact artists and make sure we have permission to re-use their work).

6 January - 16 January, 2016: Election Candidates, SCF Strategic Plan, New Book, Pre-Workshop Help Sessions, AMY Version 1.3, Mistakes, and Recorded Lessons

Sat, 16 Jan 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

##Highlights: Steering Committee Election

The 2016 Software Carpentry Foundation Steering Committee election dates are set for February 15-19. Please make sure your name is on the members list to cast your vote.
Read more about the candidates for the 2016 elections by following the links below:
The Software Carpentry Foundation strategic plan as developed by the current steering committee is available.

##New

18 Months of Progress

Fri, 15 Jan 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

[My talk at SciPy](https://youtu.be/1e26rp6qPbA?t=1748) in 2014 quoted one of our instructors as saying:

The most frustrating things for me are

people are more willing to submit new material…that to improve existing material or review another person’s PR and 2) instructors teaching bootcamps using other lesson material. This experiment in collaborative lesson development is doomed to fail if

people don’t make iterative changes to lesson material and 2) the end product isn’t used in live bootcamps.

18 months later, I’m very pleased by how much better we’re doing. Almost all of our instructors are now using our lessons rather than legacy material of their own. just as importantly, dozens of people have submitted pull requests in the last couple of months alone. Some of those are exercises required to complete instructor training, but others range from minor bug fixes to major refactorings.

2016 Election: Anelda van der Walt

Fri, 15 Jan 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

## Who am I?

I have always been involved with, and am passionate about, multidisciplinary, multi-institutional projects. Over the last six years most of my work has focused on collaboratively providing support and training to researchers at various institutions across South Africa. Initially the support focussed specifically in areas of Next Generation Sequencing as I was employed by two national *omics platforms, but more recently I’ve participated in local eResearch initiatives. Through eResearch I’ve been fortunate to collaborate with researchers from a broad array of disciplines including life scientists, medical doctors, engineers, linguists, historians, and more. While I may not be the most computationally advanced person in this community, I have a knack for creating connections between those who need help and those who can help. I specifically love to work with people who are blissfully unaware of the power of technology, and upon exposure, blossoms into the greatest advocates and users with huge impact.

2016 Election: Giacomo Peru

Fri, 15 Jan 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

2016 Election: Giacomo Peru

Ciao a tutti. Many thanks for taking a minute or two to read this post about my candidature for our Software Carpentry Steering Committee. As you’ll see, it was a long way from programming that I was reared, yet this opportunity for closer involvement with our work and its future brings together so much of what I have been doing all these years. Here’s why…

2016 Election: Jonathan Guyer

Fri, 15 Jan 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

##Background

My training is in Materials Science and Engineering and I have done both computational and experimental research in fields like semiconductor crystal growth, electrochemical interfaces, and additive manufacturing. I co-authored the FiPy partial differential equation Python framework to support my simulations and to enable others to use these methods. I have spent my professional career at the US National Institute of Standards and Technology and been leader of the Mechanical Performance Group for the past two years.

2016 Election: Bill Mills

Thu, 14 Jan 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Meet Bill

I’m Bill Mills, a scientific software developer based in Vancouver, Canada, currently spearheading web development for the GRIFFIN collaboration at TRIUMF.

In two years with SWC, I’ve taught seven workshops in addition to five live Instructor Trainings, across five countries. Over the past year I have sat on SWC’s Mentorship Committee; my most visible work there was as co-organizer of our first Instructor and Helper Retreat.

2016 Election: Dhavide Aruliah

Thu, 14 Jan 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

My name is Dhavide Aruliah. I am standing for election to the Steering Committee of Software Carpentry for 2016.

Who am I?

By training, I am an applied mathematician and computer scientist. From 2004 to 2009, I was a tenure-track assistant professor at UOIT (the University of Ontario Institute of Technology in Oshawa, ON, Canada). From 2009 to 2015, I was an associate professor with tenure at UOIT. In mid-2015, I left my academic career to join Continuum Analytics.

2016 Election: Jason Williams

Thu, 14 Jan 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

#2016 Election: Jason Williams

With benthic sensuous pleasure I offer up myself as candidate for the 2016 Steering Committee.

There are those (mom, the U.S. Department of Justice) who have labeled my apoplectic fits a sign of ‘weakness’, but I have found them to be a source of extraordinary strength. With this strength I pledge to bring Software Carpentry to places no one imagined it could (or should) go. As other candidates hail HYDRA, I’m proud to be the first and only candidate to come out in favor of all the things you believe in, and strongly (though perhaps not uncategorically) opposed to the things that displease you.

2016 Election: Karin Lagesen

Thu, 14 Jan 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

For the past year, I have served as the Software Carpentry Steering Committee’s secretary. My involvement with the SCF started in 2012, when I attended a workshop in Oslo, Norway. I signed on as an instructor in 2013, and have taught 9 workshops since then. In addition to serving on the Steering Committee, I am also a member of the mentoring committee, where I have focused on ways of giving our instructors hands-on experience with the teaching material, with the goal of making it easier for new instructors to get started. I am also currently training to become an instructor trainer.

I have a PhD in bioinformatics from the University of Oslo, and am currently employed at the Norwegian Veterinary Institute and the University of Oslo. My background is in both computer science and molecular biology. Since I have formal training in both fields, I am frequently the one to translate the biological problem into a computational one. I have often been called upon to teach people with little to no training in computer science how to do their bioinformatics analyses. This means introducing them to Unix, to command-line work and to basic programming. Working in such multi-disciplinary situations has made me very aware of how hard it can be to move into a field far removed from your core area of expertise. This makes the values and skills that Software Carpentry teaches particularly important to me.

2016 Election: Lauren Michael

Thu, 14 Jan 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

## About Me

B.Sc. Biology, Chemistry - Truman State University
M.Sc. Biophysics - University of Wisconsin-Madison
3 years instructing with SWC; certified Fall 2014
9 SWC workshops organized/helped; 8 as an instructor
1 DC workshop organized/helped

Relevant Volunteer Work

Social Media Manager for Midwest Ultimate, 2013-2014
Science Editor and Writer for the Daily Cardinal, 2011-2012

Currently

Research Computing Facilitator at UW-Madison’s large-scale computing center, the Center for High Throughput Computing, where I lead user interaction efforts and spend much of my time working directly with researchers to:

2016 Election: Raniere Silva

Thu, 14 Jan 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

My name is Raniere Gaia Costa da Silva and I'm standing for re-election to Software Carpentry Steering Committee because working on the Steering Committee in 2016 was fun and a great way to meet some of the amazing members of Software Carpentry community.

Background

I have a B.Sc. in Apply Mathematics from the University of Campinas, Brazil, and in the last year I worked as a freelancer software developer using most of the time Python and Javascript. This year I will move to the UK for work on the University of Manchester and collaborate with The Software Sustainability Institute.

2016 Election: Rayna Harris

Thu, 14 Jan 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

##Background

I am pursuing a PhD in Cell and Molecular Biology at The University of Texas in Austin. My thesis research in Hans Hofmann’s lab focuses on understanding transcriptional responses to spatial learning with single neuron resolution.

Since 2012, I’ve been deeply involved with enhancing graduate student education through 1) the Neural Systems and Behavior (NS&B) course at the Marine Biological Laboratories and 2) the Center for Computational Biology and Bioinformatics (CCBB) at UT Austin. At NS&B, I teach molecular approaches to neuroscience and supervise student research projects. With the CCBB, I organize workshops and symposia to promote sharing of ideas and expertise across departmental boundaries. It’s been awesome to see theses, manuscripts, tweets, and blogs acknowledge these programs for advancing student’s professional development.

2016 Election: Kate Hertweck

Wed, 13 Jan 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Hello fellow educators and coding enthusiasts! I’m terribly excited to offer myself as a candidate for the Software Carpentry Steering Committee. See below for answers to a few questions you might have about whether I’d be a good fit.

Who are you?

I’m an assistant professor at the University of Texas at Tyler in small but diverse Biology Department. My position is officially described as bioinformaticist, but I specialize in comparative genomics and collaborate on many different types of biological data analysis. I teach graduate and undergraduate classes in bioinformatics, genomics, and plant taxonomy.

A New Book from Mark Guzdial

Wed, 13 Jan 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Regular readers will know that I am a huge fan of Mark Guzdial, a professor at Georgia Tech whose group does world-class work on computing education, and who [blogs about it](https://computinged.wordpress.com/) regularly and incisively. Mark has just [released a new book](https://computinged.wordpress.com/2015/12/23/book-released-learner-centered-design-of-computing-education-research-on-computing-for-everyone/); you can [preview the contents](https://dl.dropboxusercontent.com/u/2635522/LCD-CE-TOC-Preface-Dec2015.pdf) if you want, or just head straight to [Amazon](http://www.amazon.com/Learner-Centered-Design-Computing-Education-Human-Centered/dp/1627053514/) and order a copy.

In it, Mark asks what it means to talk about teaching everyone to program, and whether we should we have the same goals for a mass audience as we do for professional software developers. If not (and he makes the “not” case pretty convincingly), then how do we design computing education that works for graphic designers, high school teachers, and everyone else? His answers are based on both his own experience and his comprehensive knowledge of CS Ed research, and is a solid, readable, and purposeful introduction to the latter. It’s definitely going to be on the reading list for future instructor training classes…

Archiving Videos

Wed, 13 Jan 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

People have been asking for videos of our lessons more and more frequently over the past few months, so we have begun archiving links on [the lessons page]({{site.baseurl}}/lessons/#video). If you have more, please send them our way (either as a pull request against the lessons page in [our website's GitHub repo](https://github.com/swcarpentry/website/blob/gh-pages/pages/lessons.html) or [by email](mailto:{{site.contact}}).

Pre-workshop help sessions for 2016

Wed, 13 Jan 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Are you preparing to teach a workshop in a few months? Have you been trained as an instructor but are hesitant to sign up to teach? Has it been awhile since you taught and are interested in learning what’s new in a lesson? Are you thinking about teaching a new (or just new-to-you) lesson?

The mentoring subcommittee is pleased to announce the institution of regularly scheduled help sessions for folks who are interested in feedback on their workshop plans. The first hangouts of 2016 are planned for Wednesday, January 20 at 10:00 and 19:00 EST and are open to the entire community. You can view possible topics for discussion and sign up to attend on the etherpads for the morning and evening sessions. These discussions are planned for every two weeks and will be included on the community calendar.

2016 Election: Leanne Wake

Tue, 12 Jan 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

## Academic background

You can find me in the Department of Geography in the University of Northumbria at Newcastle, UK. In 2010 I obtained my PhD in Geophysics from Durham University, UK. I would describe myself, broadly, as an Earth system modeller. I try to understand the workings of a complex world through the 0s and 1s of code but I have been known get my hands dirty with fieldwork!

2016 Post-Workshop Instructor Debriefing, Round 01

Tue, 12 Jan 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

On January 11, 2016 we ran the 1st round of post-workshop instructor debriefing of the calendar year! The mentoring committee ran 22 debriefing session in 2015, and we are exciting about continuing to host and improve these sessions this year.

Interestingly, all instructors present participated in 3-day format Software Carpentry workshops. Read on to find out more about the changes made, what worked well, and what didn’t.

A Strategic Plan for the Software Carpentry Foundation

Tue, 12 Jan 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Software Carpentry Foundation Steering Committee held a strategic planning meeting in August. In that two-day, intensive meeting, we focused on developing a long term strategic plan for the Software Carpentry Foundation. The process emphasized identification of our stakeholders and mission. It also used the results of our community survey to identify our strengths and weaknesses as well as the opportunities and threats that face the organization. The resulting Strategic Plan was developed over the course of the in-person meeting and summarized into a document. With vital feedback from the Advisory Council, whose role is to offer advice and guidance to SCF on strategic matters, we arrived at the document that now resides here.

Online Workshops from UC Davis

Tue, 12 Jan 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We have been experimenting with half-day workshops on specialized topics that are broadcast over the web. They have been [going very well](http://ivory.idyll.org/blog/2015-d3.js-annoc.html), so we are planning to do more in the first half of 2016, with about half of them broadcast this way. The tentative schedule is below; all workshops will run 9:15am-12:15pm, Pacific Time. If you would like to present something yourself in one of the open slots, please [file an issue](https://github.com/dib-lab/dib-training/issues) or leave a comment on [this post](http://ivory.idyll.org/blog/2015-training-for-q12-2016.html) --- we think this could be a really good way to field-test new material that Data Carpentry and Software Carpentry instructors could use for more advanced audiences.

2016-01-20: Camille Scott, pydoit - local+remote
2016-01-27: Raniere Silva, advanced git - local+remote
2016-02-17: Tiffany Timbers, regular expressions and Python - local+remote
2016-02-19: Ariel Rokem, scipy.optimize - local+remote
2016-02-29: Adelaide Rhodes, sphinx + webhooks + bitbucket - local+remote
2016-03-07: Titus Brown, Amazon Web Services - local+remote
2016-03-28: Titus Brown, Short-read trimming and quality eval - local+remote
2016-04-06: Daniel Chen, intro git (SWC lesson on git) - local+remote
2016-04-08: Daniel Chen, advanced git (branching and merging, etc.) - local+remote
2016-04-11: open
2016-04-13: Titus Brown, TBD
2016-04-18: Titus Brown, TBD
2016-04-27: Meeta Mistry, TBD
2016-05-11: Marian Schmidt, Rmarkdown - local+remote
2016-05-13: Ted Hart, TBD - local+remote
2016-05-18: Heer group on VEGA - local+remote
2016-06-06: Titus Brown, TBD
2016-06-10: Titus Brown, TBD
2016-06-13: open
2016-06-15: open
2016-06-22: Titus Brown, TBD
2016-06-29: Titus Brown, TBD

Change Strategies in STEM Education

Sat, 09 Jan 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I recently had a chance to read:

Maura Borrego and Charles Henderson: "Increasing the Use of Evidence-Based Teaching in STEM Higher Education: A Comparison of Eight Change Strategies". *Journal of Engineering Education*, 103(2), DOI 10.1002/jee.20040.

The abstract says:

Background Prior efforts have built a knowledge base of effective undergraduate STEM pedagogies, yet rates of implementation remain low. Theories from higher education, management, communication, and other fields can inform change efforts but remain largely inaccessible to STEM education leaders, who are just beginning to view change as a scholarly endeavor informed by the research literature.

Good Communities (Kinds Of)

Sat, 09 Jan 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Back in October, [Sarah Sharp](http://sarah.thesharps.us/) posted a really useful article titled "[What makes a good community?](http://sarah.thesharps.us/2015/10/06/what-makes-a-good-community/)". In it, she divided online tech communities into six levels (numbered from zero, of course) according to how welcoming and supportive they are. I would put Software Carpentry and Data Carpentry in Level 1, and say that we're starting to do meaningful work toward Level 2. I would appreciate feedback of two kinds:

If you agree, what should we do in 2016 to get to Level 2?
If you don’t, where are we falling short of Level 1 (or Level 0)?

What the Data Says About Novice Programming Mistakes

Sat, 09 Jan 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I recently had a chance to catch up with this paper from 2014:

Neil C. C. Brown and Amjad Altadmri: “Investigating Novice Programming Mistakes: Educator Beliefs vs Student Data”. ICER'14, http://dx.doi.org/10.1145/2632320.2632343.

Its abstract says:

Educators often form opinions on which programming mistakes novices make most often – for example, in Java: “they always confuse equality with assignment”, or “they always call methods with the wrong types”. These opinions are generally based solely on personal experience. We report a study to determine if programming educators form a consensus about which Java programming mistakes are the most common. We used the Blackbox data set to check whether the educators’ opinions matched data from over 100,000 students – and checked whether this agreement was mediated by educators’ experience. We found that educators formed only a weak consensus about which mistakes are most frequent, that their rankings bore only a moderate correspondence to the students in the Blackbox data, and that educators’ experience had no effect on this level of agreement. These results raise questions about claims educators make regarding which errors students are most likely to commit.

2016 Election: Cam Macdonell

Fri, 08 Jan 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Background & SWC Involvement ----------

I am an Assistant Professor in the Department of Computer Science at MacEwan University in Edmonton, Alberta, Canada. My involvement with Software Carpentry began in 2009 as a helper. I helped at a handful of Edmonton bootcamps before completing instructor training in 2013. I have been an instructor for 9 bootcamps, 6 for Software Carpentry and 3 for Data Carpentry.

A Year Of Software Carpentry in South Africa

Wed, 06 Jan 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

After the [Software Carpentry workshop]({{site.baseurl}}/blog/2014/12/cape-town-swc.html) ran jointly with the [eResearch Africa conference](http://www.eresearch.ac.za/eresearch-africa-2014)in November 2014, we were looking for opportunities to run more workshops in South Africa in 2015. In May we ran our [first remotely instructed workshop]({{site.baseurl}}/blog/2015/06/remote-workshop-cape-town.html) (with thanks to [Laurent Gatto](https://twitter.com/lgatt0) and Software Carpentry for the remote option) focusing only on R to counter possible time loss due to technical glitches. Then it was onto [remote instructor training]({{site.baseurl}}/blog/2015/05/online-instructor-training.html) together with two groups on other continents (one in the USA and another in the UK) in June. Six new instructors were trained, four of them having gone on to teach one or more subsequent workshops. In July we joined forces with [UCT eResearch](http://www.eresearch.uct.ac.za/) to host a [third local workshop](http://www.eresearch.uct.ac.za/news/software-carpentry-powerful-initiative), joined by an experienced SWC instructor, [Matt Lammens](https://mlammens.github.io/) from the US, who happened to be visiting. Another workshop was offered in conjunction with the [SAEON Graduate Student Network Indibano](http://www.saeon.ac.za/enewsletter/archives/2015/october2015/doc11) in September. The next one was co-organised by [Adrianna Pińska]({{site.baseurl}}/blog/2015/10/jozihub-workshop.html) from the Cape Python User Group in October in Johannesburg. In November 2015 we ran the sixth Software Carpentry workshop in 12 months at [North-West University](http://www.nwu.ac.za/).

In all, six new instructors were trained, approximately 28 helpers recruited, and more than 200 learners from over 18 local organisations have participated in the 6 workshops between November 2014 and November 2015. There have been a number successes, but of course also some large, and small, bumps in the road.

Announcing the Data Science Journal

Wed, 06 Jan 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The [Data Science Journal](http://datascience.codata.org/) is, as its title suggests, a journal dedicated to the advancement of data science. The first thing that's good about it is that you won't get random emails about it with poor grammar and wild claims about its impact factor that begin with DEAR ESTEEMED RESEARCHER....

Even though it’s about data science, it’s not obsessed with building ever better recommender algorithms for Netflix or mining twitter feeds. Its focus is very much on its application in the policies, practices, and management of open data. It tries to take as wide a definition as possible when considering the subject. Data can be originally digital or converted from other sources, and it also considers every research discipline. The journal will have digital humanities papers rubbing shoulders with bioinformatics papers with social science papers.

AMY Version 1.3

Tue, 05 Jan 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

In the past month we've seen two releases of AMY: v1.2.1 and v1.3. This blog post (originally published on [my personal blog](http://piotr.banaszkiewicz.org/blog/2016/01/01/amy-release-13/)) contains a joined release notes for both of them.

Bug fixes

wrong URL used in event validation or import/update features is now indicated (and we won’t receive wrong notifications about it)
properly throw 404 on some pages (previously: 500)
spaces are striped from Person and ProfileUpdateRequest fields (names, emails)
disable location inputs on event details page if online country was preselected

New features

use custom-built jQuery-UI (so that we no longer have conflicts with Bootstrap’s tooltip module)
Greg updated the script used to send instructors “Hey, update your info” mails (it’s getting removed later on)
it’s possible to add memberships per host
new badge: DC instructor
new logic for dealing with two instructor badges
timeline of TO-DO items
basic models (e.g. lessons, tags, academic levels, etc.) are now manageable from Django’s admin interface
all persons view: add filtering by workshop type person taught at
remove blurred production database in favor of generated fake database
mailing script turned into better Django management command
bulk upload now shows generated username and suggested people with matching names
show preview of event on SWC website
API: filter events by tags

No longer with us

Greg removed some unused scripts (test-command-line-upload.sh) and commands (parse-instructor-info.py)
notifications about invalid HTTP header Host
other removed scripts and commands

January and February don’t seem busy for me, so I hope to have more done on AMY in the coming months. I also want to thank Prof. Ethan White for his support of my work through December, and for extending his support for the next two months.

A Data Programming CS1 Course at the University of Washington

Sun, 03 Jan 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

People who are interested in integrating Data Carpentry and Software Carpentry ideas into the undergraduate curriculum may enjoy reading Anderson et al's paper "A Data Programming CS1 Course" (*SIGCSE'15*, http://dx.doi.org/10.1145/2676723.2677309). From the abstract:

This paper reports on our experience teaching introductory programming by means of real-world data analysis. We have found that students can be motivated to learn programming and computer science concepts in order to analyze DNA, predict the outcome of elections, detect fraudulent data, suggest friends in a social network, determine the authorship of documents, and more. The approach is more than just a collection of “nifty assignments”; rather, it affects the choice of topics and pedagogy.

Lessons as Lab Protocols

Sun, 03 Jan 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A roundabout chain of references led me to Abbott et al’s “Programs for People: What We Can Learn from Lab Protocols” (presented at VL/HCC 2015) which looks at how lab protocols are similar to and different from programs. On the one hand, a lab protocol describes the steps to be followed to (for example) prepare a particular kind of sample for analysis. On the other hand, “human plans are more like descriptions or predictions than prescriptions of what actions a person will take given the most likely sequence of future actions; they are resources for anticipating likely future events. In programming language terms, perhaps, they are more declarative than imperative in that people draw on knowledge of the whole plan rather than blindly following each step.”

Discussion Sessions

Sat, 02 Jan 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

At the end of last year [we announced a new checkout procedure for instructor training]({{site.baseurl}}/blog/2015/12/instructor-training-checkout-procedure.html). This new procedure has "an hour-long group discussion led by an experienced instructor" and we are looking for instructors interested in lead the sessions.

Welcome to 2016

Fri, 01 Jan 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Mentoring Subcommittee hope that you and your family had a incredible Christmas and amazing New Year.

We are slowly backing from holidays but we already have a busy agenda for January so we give a heads up for a few important dates:

Mentoring Subcommittee Meeting on January 11th.

This will be our first meeting this year and we will have some discussions about our plans for 2016. If you want to join us you will find informations at http://pad.software-carpentry.org/scf-mentoring. You can use the etherpad for suggestions or send them by email to mentoring@lists.software-carpentry.org.

Plans for 2016

Mon, 28 Dec 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Twelve months is a long time. A year ago I was wrapping up my second month without an income and had serious doubts about whether Software Carpentry was going to be viable. Today, our new Executive Director and Program Coordinator are in place, our workshop administration tool is ticking over nicely, we're about to elect our second Steering Committee, we've published our lessons, we have a growing number of partners and affiliates, our reboot of instructor training has taught us a lot, and those are just the things I've bookmarked.

New Words Needed

Sat, 26 Dec 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Titus Brown recently asked (on behalf of a friend) what people think of the term and practice of "hardening" software, by which he meant making research software more robust, more easily usable, and possibly scalable. Several people responded that "hardening" is usually used to mean "make more secure", and that what Titus's correspondent was really asking about was something we don't have a simple word for: making software ready for production use. As Morgan Taschuk wrote, this involves eliminating hard-coded paths, adding useful error messages, incorporating command-line flags to turn features on and off, and many other things that aren't really necessary if the program is only ever used by its author on her own machine and data, but are essential to institutional scale-up, i.e., to making the software usable by other people, in other places, on other problems.

Assessment Update - 2015

Tue, 22 Dec 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

About a year ago, I hoped to help answer the question, "Is our students learning?" We are now a bit closer to an answer, and I wanted to update the community about what we have accomplished this year. First, I want to thank and acknowledge the assessment subcommittee and Blake Joyce, who helped to define tasks, possibilities, and provided significant feedback. Especially here I wanted to thank Daniel Chen, Jeramia Ory, and Rayna Harris who put in a lot of hours moving things along.

Pushing Ahead in Puget Sound

Sun, 20 Dec 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As regular readers will know, our instructor training classes are heavily over-subscribed: even after the classes we ran this fall, we still have over 400 people on our waiting list, some of whom have been there for months.

One of these is Billy Charlton, who works for the Puget Sound Regional Council. Keen to sharpen his team's technical skills, and unable to wait any longer for a training course, he taught himself as much as he could about what and how we teach, then went ahead and ran a workshop using our materials. His report on his experiences is inspiring, as is the feedback he received from participants, and a great way to end a busy year.

Instructor Training Checkout Procedure

Fri, 18 Dec 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

After a lot of discussion, we have come up with a new procedure for completing instructor training for Data Carpentry and Software Carpentry. Its goals are to ensure that people are familiar with the lesson material while introducing them to our community. The steps are:

Three Flavors of Instructor Training

Fri, 18 Dec 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

It's been a busy few months for instructor training: along with a two-day class at the University of Manchester, we also wrapped up a pilot of a new kind of multi-week course and a two-day version of the same material. We have also been training four new trainers, and have put the first complete draft of the training materials online. Here's some of what we think we've learned so far.

More on Educational Engineering

Wed, 16 Dec 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

In my experience, the term "educational engineers" doesn't seem to have caught on to describe any particular activity, but I can provide some examples of people bringing ideas from the research to inform teaching. There's a range depending on what counts as "building".

November 11 - December 16, 2015: 2016 Steering Committee Election, Projects, Instructor Training and More, Lessons, Workshop Feedback, and Data Carpentry is Hiring.

Wed, 16 Dec 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Highlights

The Software Carpentry Foundation will hold its annual Steering Committee election in February 2016. Active Software Carpentry community members are invited to both stand for and vote in the elections.
The first candidate for the Steering Committee elections is Belinda Weaver. Learn more about her history with Software Carpentry and see what she plans to do for the community in 2016.
Want to contribute to Software Carpentry, but don't know how? There's a fantastic array of projects including development of new lessons, fixing up old ones, and much more. Jump in or contact us if you're interested in participating.

Other News

The past month saw a huge amount of Software Carpentry activities including:
Instructor Training events and feedback
Many Software Carpentry workshops were run and several people provided great write-ups about these. Write-ups are also available for a number of other workshops from which the Software Carpentry community could learn a great deal.
A few exciting developments in Data Carpentry
New lessons were developed and ideas to improve existing lessons were discussed
Several other posts have been published by community members

2016 Election: Belinda Weaver

Tue, 15 Dec 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I co-organise, run and teach at Software Carpentry workshops in Queensland, Australia, and frequently tweet about them and other Software Carpentry activities as @cloudaus

I first heard of Software Carpentry on Twitter in 2013 and thought it was a great initiative. Through the A/NZ mailing list, I made contact with Australian Software Carpentry trainers, and was able to organise the inaugural Brisbane bootcamp in July 2014, flying in instructors from Melbourne and Auckland.

Educational Engineering

Tue, 15 Dec 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

One of the participants in this week's instructor training course mailed me to say, "[We] were discussing some of the ideas we were talking about in educational research and I feel like we are missing 'educational engineers'. Does this discipline exist? I feel like we don't really have the people to take the research that's been done and build something with it."

The answer is that we actually do have educational engineers: at the K-12 level, many of the people who write textbooks and other learning materials have lots of training in pedagogy, and the curriculum they create goes through the same sort of careful review that plans for a new dam go through. But that's usually not true in higher education: most of the people who write textbooks at the post-secondary level are domain experts with little or no training in pedagogy. And even in K-12, people with no background in education at all frequently overrule experts on both content and method.

Community Calendar Available

Thu, 10 Dec 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

During this year we host many events for the community including lab meetings, sub-committees and de/briefing sessions. Unfortunately we fail to let everyone know of the events and to try solve this problem next year we are launching a community calendar. If you want to add the community calendar into your agenda you should use https://calendar.google.com/calendar/ical/oseuuoht0tvjbokgg3noh8c47g%40group.calendar.google.com/public/basic.ics.

Teaching For Loops

Thu, 10 Dec 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A few days ago, Karin Lagesen asked people what metaphors they use when teaching for loops. It kicked off an entertaining thread. When we talk about pedagogical content knowledge (PCK) in instructor training, this is exactly the kind of thing we mean:

Feedback on Practicum Proposal

Wed, 09 Dec 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We have wanted for some time to give newly-trained instructors more experience with workshop materials and our teaching methods before sending them into the field. The mentoring committee and other community members have now created a proposal for doing this, and would like to test it out early in 2016 with participants in the two training session that ran this fall. We have therefore created a short questionnaire to find out whether our plan is feasible and takes us in the right direction.

Introducing the Research Bazaar

Wed, 09 Dec 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

One of the lessons Software Carpentry has learned over the years is that there's no substitute for a face-to-face learning experience. The unscripted interactions that occur between instructors, helpers and participants at our workshops are as important, if not more important, than the formal teaching syllabus itself. Of course, we aren't the only educator in the digital research space to which this lesson applies. The social aspect of in-person learning experiences is equally important to the various organisations, eResearch departments and libraries around the world tasked with "up-skilling" the next generation of digital researchers.

Software and Data Carpentry Workshop in Stockholm

Wed, 09 Dec 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

On April 14, three Software Carpentry Instructors from Stockholm, Radovan Bast, Olav Vahtras and myself got an email from Greg Wilson:

Subject: Any interest in putting together a workshop for Stockholm this summer?

Bank of America Merrill Lynch to sponsor the first Workshop for Women in Science and Engineering in the UK

Tue, 08 Dec 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are very happy to announce that the first UK Software Carpentry Workshop for Women in Science and Engineering received generous support from Bank of America Merrill Lynch. The event will take place on 14 and 15th December at the University of Manchester.

Call for Candidates for the 2016 Steering Committee

Tue, 08 Dec 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

From February 15-19, Software Carpentry will hold its annual election for the Steering Committee of The Software Carpentry Foundation. The inaugural Steering Committee has been an incredible team of dedicated community members working to position Software Carpentry so that it has a strong foundation and continued impact in scientific communities around the world. Please consider giving back to the community by standing for election!

Software Carpentry and Data Carpentry on Podcast.init

Tue, 08 Dec 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A couple of weeks ago I was interviewed by the Podcast.__init__ team to talk about Software Carpentry and Data Carpentry. The request for an interview actually came in a couple of months ago and I asked Greg Wilson about doing it, telling him I was a big fan of the podcast. He said he'd be happy to have me do it myself, giving me the chance to star in one of my favorite shows. I decided to do it, in pursuit of my goal of promoting Software Carpentry and Data Carpentry (and maybe more public exposure for myself).

Announcing Instructor Training Materials

Mon, 07 Dec 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are pleased to announce that the lessons for our instructor training course are now available online and in this GitHub repository. They have evolved a lot in the three and a half years since we started running this class, and more improvements would be welcome: please file issues or send pull requests to fix, extend, or otherwise improve the material.

Launching Pre Workshop Help Session

Fri, 04 Dec 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Organizing one workshop isn't easy and sometimes things out of your control go wrong in the last minute like the internet access on the room is very slow. After months of discussions of what could be done to help instructors and hosts when organizing their workshops and being prepared for the show time the Mentoring Subcommittee decided to test some helping sessions.

First Workshop in Venezuela

Thu, 03 Dec 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Last November Francisco Palm taught the first ever Software Carpentry workshop in Venezuela, the second country from Latin America to enter in our list of previous workshops. We are very excited with our expansion across Latin America and we hope to add more countries to that list next year.

Learners from the first workshop in Venezuela.

Francisco allowed us to share some of his words about the workshop:

Intel to sponsor the first Workshop for Women in Science and Engineering in the UK

Thu, 03 Dec 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are very happy to announce that the first UK Software Carpentry Workshop for Women in Science and Engineering received generous support from Intel. The event will take place on 14 and 15th December at the University of Manchester.

Software Carpentry workshop at EITN in Paris

Wed, 02 Dec 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Python workshop on November 19-20th at the EITN was the first course organised at European Insitute of Neuroscience in Paris. The course offered a practical introduction to Python programming and scientific software development for PhD students and post-docs. The programme followed closely the syllabus recommended by the Software Carpentry.

Data Science for Social Good: an Experiment in Data Science Training

Tue, 01 Dec 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Data science faces many challenges in the traditional academic setting. At the same time, many research fields are becoming increasingly dependent on data science tools and techniques. A key element in tackling these challenges is the education of a new generation of researchers that are fluent in both their research domain and in data science methodologies. In this post, we discuss an immersive approach to training in data science, the University of Washington eScience Institute's inaugural Data Science for Social Good (DSSG) program.

Software Sustainability Institute Funding

Tue, 01 Dec 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Software Sustainability Institute, a partner organization which is committed to cultivating world-class research through software, has received £3.5M in funding to continue its valuable support for the UK's research software community. Two new funders, the Biotechnology and Biological Sciences Research Council (BBSRC) and Economic and Social Research Council (ESRC), have joined forces with the Institute's original funder, the Engineering and Physical Sciences Research Council (EPSRC), to continue to invest in research that is underpinned by software until at least 2019. For the full announcement, please see this post on the SSI site.

Congratulations!

Assistant Director Position with Data Carpentry

Mon, 30 Nov 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Data Carpentry seeks a full-time Associate Director to lead the organization’s community engagement and education activities, to cultivate a healthy supportive community and provide mentorship and training to current and future instructors. The Associate Director is one of the two key roles providing leadership to Data Carpentry’s core efforts and is expected to shape the organization’s operational functioning, influence training, and contribute to strategic planning.

Data Carpentry is a not-for-profit organization developing and teaching workshops on fundamental data skills needed to conduct research. Its mission is to provide researchers high-quality, domain-specific training covering the full lifecycle of data-driven research. Data Carpentry lessons are intentionally domain specific, and span the life, physical, and social sciences. Data Carpentry workshops create an environment friendly to learners who have little to no prior computational experience, and are designed to empower researchers to apply the skills learned to data driven discovery in their own research.

The Associate Director position is initially funded for 2 years through a grant from the Gordon and Betty Moore Foundation. They will be hired and paid as a contractor of NumFOCUS, Data Carpentry’s fiscal sponsor. Review of applications will begin December 18th, 2015, and the position will remain open until filled.

December Instructor Training - Announcing Selected Groups

Mon, 30 Nov 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Thank you to all the teams that applied for instructor training that sent us applications. Selection of the teams was a very difficult tasks because

the applications were excellent,
we had a small amount of time to discuss all the applications and
we need to improve capacity to support the demand we see for instructor training.

The mentoring subcommittee has selected the teams that inform at their cover letters that booked a room for the instructor training at

December Instructor Training Selection Debrief

Wed, 25 Nov 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Many of you who applied for instructor training in December are undoubtedly receiving disappointing news. We had 250 applicants on 45 teams from a wide range of locations. I'd like to talk through a few things about this process and how it played out as I think it will help the community understand the situation we were in with instructor training demand and our ability to meet demand.

2015 Post-Workshop Instructor Debriefing, Round 22

Tue, 24 Nov 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

There were relatively few attendees at the 22nd round of debriefing for instructors on November 10, but we thought we'd include a few quick notes about two specific topics: using Software and Data Carpentry workshops to reach out to potential users of high-performance computing (HPC) resources, and exploring new lesson materials related to databases.

High Performance Computing

University of Manitoba hosted a Software Carpentry workshop at the end of October in conjunction with Compute Canada. In addition to the normal half-day lessons in Unix shell, Git, and intro to Python, this workshop featured an introduction to using WestGrid/Compute Canada resources on the afternoon of the second day. The instructor for this workshop, Hossein Pourreza, said that this last lesson module was used as a capstone for helping reinforce and integrate material from the previous lessons, and appeared well-received by the students. Given that there are few training opportunities for using the cluster, this appears to be a great way to introduce new potential users to the basic tools they'll need to get started with larger-scale analysis.

2015 Software + Data Carpentry Instructor and Helper Retreat

Tue, 24 Nov 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Software and Data Carpentry recently held their first ever Instructor + Helper Retreat. The aim of this event was to bring instructors and helpers together remotely and at local sites around the world for a day of sharing skills, trying out new lessons and ideas and discussing all things instructor and helper related. This Retreat attempted to meet these goals via 17 local meetups in North America, Europe and Australia, 14 sessions broadcast globally via Google Hangouts on Air, and many people participating remotely by watching and asking questions at the global broadcast sessions.

In addition to bringing the community together, the Retreat also has generated additional resources surrounding many topics of interest to Software and Data Carpentry instructors and helpers; the global broadcast sessions are archived on YouTube (pending session leaders leaving them up) and can be watched at any time. The links to these videos are archived on the Retreat website. Notable other moments from retreat as captured via twitter can be accessed here.

A New Lesson on Testing

Sun, 22 Nov 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Katy Huff has just posted a new lesson on testing and continuous integration with Python drawn in part from the book that she and Anthony Scopatz recently published. There's a lot of useful material in here, all of it tested in the classroom—please check it out.

Applications for December Instructor Training Are Now Closed

Fri, 20 Nov 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Applications to take part in December's two-day instructor training class are now closed. We received more than three dozen applications from four continents, including over 400 people, and will let people know early next week whether they have been selected. Groups that we can't include in this round will be given priority to take part in the new year.

The Morea Framework

Fri, 20 Nov 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I first met Philip Johnson, a professor of Computer Science at the University of Hawaii, through shared interests in empirical software engineering research and Google Summer of Code. He has recently been developed the Morea Framework for creating structured course websites using GitHub and Jekyll. "Morea" stands for "Modules, Outcomes, Readings, Experiences, and Assessments", which are the five main elements the framework supports. As you can see from the project gallery, it's much more structured than our lessons. It also requires more tooling—Morea Framework sites are built using custom Jekyll plugins, and the source relies much more heavily on include files than our template—and it's geared very strongly toward traditional semester-long courses.

rOpenSci Announces $2.9M Award from the Helmsley Charitable Trust

Thu, 19 Nov 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

rOpenSci, whose mission is to develop and maintain sustainable software tools that allow researchers to access, visualize, document, and publish open data on the Web, is pleased to announce that it has been awarded a grant of nearly $2.9 million over three years from The Leona M. and Harry B. Helmsley Charitable Trust. The grant, which was awarded through the Trust's Biomedical Research Infrastructure Program, will be used to expand rOpenSci's mission of developing tools and community around open data and reproducible research practices.

Test-Driven Data Analysis

Thu, 19 Nov 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

My former colleague Nick Radcliffe has started posting a series of article on test-driven data analysis, in which he explores systematic ways of checking whether one-off data analyses are correct. It'll be really interesting to see how the series unfolds, and comments on the posts would be very welcome.

A Practical Computing Course

Sun, 15 Nov 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

While it was not a SWC course, this summer Casey Dunn and I taught a 12-day Practical Computing summer class at Friday Harbor Labs. It was a great group of students—mostly somewhat beginner level—and we covered regular expressions, the shell, Python, R + ggplot, Git, graphics, and bit of electronics. The centerpiece (which the students really got into) was a personal project that was applicable to their own interests and which they solved with some combination of the tools we covered. Many of the students went from zero experience to having a script that actually gave them insight into the real-life research back home.

CourseSource: A(nother) New Hope

Sun, 15 Nov 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I came across CourseSource a few weeks ago, and I'm pretty excited:

CourseSource is an open-access journal of peer-reviewed teaching resources for undergraduate biological sciences. We publish articles that are organized around courses in biological disciplines and aligned with learning goals established by professional societies representing those disciplines.

Miscellaneous Projects

Sun, 15 Nov 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This post is a bit of a link fest, but after talking about how to contribute at yesterday's instructor retreat, I thought it might be useful to post a few additions to our projects page.

Python Lesson Rewrite

Sun, 15 Nov 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Recent analysis and introspection regarding Software Carpentry's Python lesson has led us to conclude that the lesson would benefit from a complete rewrite. The subcommittee on lesson development met twice to plan much about the new lessons (yes, that's plural). This post will describe our plans for the new lessons.

We're going to write two new Python lessons, one intended for students who are completely new to programming (the "novice" lesson), and another for students with some programming experience in any language (the "intermediate" lesson). Data Carpentry also targets new programmers so we'll be developing the new novice lesson in collaboration with DC, and both DC and SWC will use the lesson. We decided to use Gapminder data for both the novice and intermediate lessons in order to have a consistent experience and approachable, meaningful data (and because some lessons are already using it).

2015 Post-Workshop Instructor Debriefing, Round 21

Fri, 13 Nov 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The mentoring subcommittee held the 21st round of debriefing for instructors on October 27. We had some great discussions with a few instructors about recent workshops and their experiences with implementing a module on imposter syndrome and methods for integrating lessons throughout the workshop.

Data Carpentry Instructor Certification for Software Carpentry Instructors

Thu, 12 Nov 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Re-posted from the Data Carpentry blog.

Data Carpentry has been growing over the last year, and now with the addition of Maneesha Sane as Program Coordinator, we have the chance to run even more workshops in more domains. To date, we have used some great instructors from the Software Carpentry community as teachers; what we would like to do now is put that on a more regular footing.

We want to update the instructor records to see who’s interested and qualified to teach Data Carpentry, so we’ve created a Data Carpentry Instructor FastTrack certification. If you have already taught a Data Carpentry workshop, you’re registered as a Data Carpentry instructor and don’t need to go through the FastTrack certification process. If you are currently a Software Carpentry instructor and would like to teach core data wrangling skills to people who are new to computing, this program is for you!

October 27 - November 10, 2015: Maneesha Sane, Retreat Activities, Instructor Training, Code Review Revisited, and a WiSE Workshop.

Tue, 10 Nov 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Highlights

Meet Maneesha Sane, the Software Carpentry (and now Data Carpentry) course coordinator.
Our instructor/helper/community retreat is planned for 14 November with participating sites around the world. Line-ups include roundtable discussions, open house sessions, Worldwide Library Hour, and more. Join us virtually or live for the whole day or only the sessions of interest.

Conversations

Do you have any experience with code review? David Pèrez-Suárez described his experience over ten years and asks some interesting questions.

R Foundation Announces Code of Conduct Policy

Sun, 08 Nov 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The R Foundation recently announced that all conferences it supports must have a code of conduct. They encourage other R meetings to adopt codes of conduct as well, stating that:

A code of conduct serves two important purposes. Firstly, it sends a clear message to those outside the community that an R conference is a professional and comfortable working environment for all participants. Secondly, it provides a mechanism for reporting and monitoring any incidents of harassment that may occur.

Clarification about December Instructor Training

Fri, 06 Nov 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Since we announced that the applications for December Instructor Training we have received many questions. We are now amending the announcement to clarify some things that have come up.

Teaching Bimodal Workshops with a Large Range

Thu, 05 Nov 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Teaching a technical class always has its challenges. Teaching a technical workshop covering 3 or more topics over the span of 2 days is what makes us Software Carpentry instructors.

A common observation instructors have is the class demographics are bimodal/multimodal. There are students who attend the workshop with little to no experience, some have been using the tools we teach but want a more formal course, and finally, a student may know some of the tools, but use the workshop to learn Git or Python, for example. How do we as instructors deal with workshops where the range of student skills is extremely high?

This post is a case study of my most recent workshop and a call for help and discussion about we should do to make our workshops enjoyable for everyone. In many ways it is a complement to Peter and Cam's post Pulling In Those Left Behind.

Introducing Maneesha

Mon, 02 Nov 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This is a long overdue post introducing myself to all of you. I've met many of you (at least virtually!) over the past few months but I wanted to formally introduce myself to the Software Carpentry and Data Carpentry community.

2015 Post-Workshop Instructor Debriefing, Round 20

Sun, 01 Nov 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

On October 13th, the mentoring subcommittee held the 20th round of debriefing for instructors. At this round, we have a great discussion text editors, workshop organization and challenges for learners.

Adrianna Pińska wrote a great post about the workshop in Johannesburg that you should check out.

Pulling In Those Left Behind

Thu, 29 Oct 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A common challenge that arises before or during a workshop is that participants' prior expertise in programming or broadly speaking their abilities of using computers for science is distributed randomly. At best, this distribution peaks at the expectations of the instructor. Usually, this distribution is quite wide and thus a considerable portion of the participants do lack the necessary predispositions for the workshop level, or have a slower learning rate or simply are too shy to ask questions. I recently taught a follow-up workshop to the Software Carpentry (SWC) Novice material and struggled to keep the pace of teaching at a level so all learners would come along. Given the feedback on the SWC mailing list (see the original post), this problem occurs quite often. Thus, this blog post is a summary of the discussion initiated among fellow SWC instructors on how to pull in those learners again that fall behind or how to pace/design a course so that a minimal portion of learners fall behind.

2015 Post-Workshop Instructor Debriefing, Round 19

Wed, 28 Oct 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

On September 29th the mentoring subcommittee held the 19th round of Instructor Debriefing and had some interesting discussions about our lessons and opportunity for learners after the workshop.

Code Review - a Needed Habit in Science

Wed, 28 Oct 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

More than a year ago, Marian Petre and Greg Wilson wrote an article about reviewing code, but I was not aware of it until last week when Greg talked about their findings in a workshop in London (though it had been mentioned here before). I have been reviewing code for years, however I have not realised that I was doing so till GitHub made it easy. Reading code from others feels the same as proofreading a Spanish text (or any other text), where you have to pay attention to the orthography, the grammar and whether it says what it's meant to say (or it does what it's meant to do).

October 19 - 26, 2015: 500 Workshops and 16,000 Participants, Retreat, Debriefing Sessions, Digital Data Storage, and A Science Competition.

Wed, 28 Oct 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Highlights

The Software Carpentry team has now run more than 500 workshops for 16,000 participants!
Remember to join our global instructor/helper retreat on 14 November. If you care about training, join a site near you or dial in via Google Hangouts.

Dates

Please take note of the planned dates for upcoming debriefing sessions for instructors.

Publications

Ten Simple Rules for Digital Data Storage: A new article on the importance of considering data storage and metadata tagging that started as a discussion on the Software Carpentry mailing list.

Opportunities

Are you building products or developing services to advance Open Science? Read more about the global science competition launched by the Wellcome Trust and NIH.

Site Planning for the Instructor and Helper Retreat

Wed, 28 Oct 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

With less than three weeks to go before Software Carpentry and Data Carpentry's first ever Instructor and Helper Retreat coming up on November 14, many of the lead organizers for sites around the world got together this week to discuss plans and ideas for the event. Our goal for the retreat is to create opportunities for instructors, helpers, and the community to get together to practice teaching, break ground on new lessons, and start discussions. Here are just a few of the ideas coming up that you can get involved with.

Software Carpentry for Women in Science and Engineering UK

Wed, 28 Oct 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Software Sustainability Institute in collaboration with ARCHER and Women in HPC is organising the first Software Carpentry workshop for Women in Science and Engineering (WiSE) in the UK. The event will take place at the University of Manchester on 14-15 December 2015.

Visualizing Repository Activity

Tue, 27 Oct 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I am updating the lessons learned paper, and would like to include histograms showing how many people have contributed how often to our lessons. More specifically, I have 9 data sets (one for each lesson), ranging in size from 5 to 16 records, in which each record shows a number of commits and how many people have committed that often. For example, the data for our SQL lesson is:

Call for Applications to December Instructor Training

Mon, 26 Oct 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This blog post was amended with the information from this other post.

This blog post was updated with the answers that we got so far.

As we announced previously, we will run another round of the free/open Instructor Training in December. People who are interested in participating in this round can now apply to take part.

Recent Statistics

Sun, 25 Oct 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

At some point in the last few weeks, we ran our 500th workshop for our 16,000th learner:

Debriefing Sessions and Winter Recess

Sat, 24 Oct 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are getting close to the begin of 2016 and although there will be some workshops between Christmas and New Year the Mentoring Subcommittee will give a winter recess for the debriefing session. But there will be some debriefing sessions before then:

Ten Simple Rules for Digital Data Storage

Fri, 23 Oct 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Edmund Hart, Pauline Barmby, David LeBauer, François Michonneau, Sarah Mount, Timothée Poisot, Kara Woo, Naupaka Zimmerman, and Jeff Hollister have just posted a pre-print on PeerJ titled Ten Simple Rules for Digital Data Storage. The paper is a distributed collaborative effort spawned from a thread on the Software Carpentry instructors mailing list and further carried out on GitHub. There are a lot of good ideas in it, many of which we should fold back into our lessons, and we hope it will spark more collaborations in our community.

Open Science Prize

Wed, 21 Oct 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Wellcome Trust and National Institutes of Health have launched a global science competition for new products or services to advance open science. Up to six teams stand to win US$80,000 each to develop their ideas into a prototype or to advance an existing early stage prototype. The prototype judged to have the greatest potential to further open science will receive $230,000. For more information, see the announcements on the Wellcome site and from the NIH, or visit the prize website.

Programming Historian Live

Wed, 21 Oct 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Originally posted at http://cradledincaricature.com/2015/10/21/programming-historian-live/.

On 19 October curious historians descended on the British Library for Programming Historian Live. The Programming Historian is a suite of open access, peer reviewed lessons that provide practical instruction to historians thinking about using data, code, and software in their research. It is co-edited by 2013 Software Sustainability Institute Fellow Adam Crymble and it does an amazing job at bringing the methods and motivations of the (small but growing) Digital History community to the wider historical profession. This "Live" spin-off, funded by my 2015 Software Sustainability Institute Fellowship, was designed to take into account the fact that whilst some of us learn just fine through self-directed tutorials, others need the mental space, in person support, and peer pressure of seminar style learning.

Inserting Software Carpentry Graduates into Coding Communities

Mon, 19 Oct 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

One of the issues I'd like to see Software Carpentry tackle is what happens to learners after they've attended a workshop.

October 5 - 18, 2015: Jonah Duckles, Instructor/Helper Retreat Still Growing, Peer Reviewed Lessons, Data Management, and AMY 1.0 Released.

Sun, 18 Oct 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Highlights

Meet our new executive director, Jonah Duckles.
14 sites worldwide have registered for the first ever Software Carpentry and Data Carpentry instructor and helper retreat on 14 November. If you care about training, join a site near you or dial in via Google Hangouts.

Lessons

The obvious next step for the Software Carpentry lessons: Peer review

Contribute

Rayna Harris suggested we look at how Software Carpentry and Data Carpentry can play a role in teaching or disseminating information about data management plans. Please let us know if you are aware of good resources or would like to discuss how we can contribute as community?
AMY 1.0 was released. Django web programmers are welcome to join the development team.

Journals as Repositories

Sat, 17 Oct 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I had a really good conversation yesterday with Cath Brooksbank and Sarah Morgan, who do training at EMBL-EBI in Hinxton. During the conversation we touched on CourseSource, a peer-reviewed journal in which people can publish undergraduate biology lessons—not studies of the lesson's effectiveness, but the lessons themselves.

This is a brilliant idea, and thinking about it has made me realize why I've never been excited about online lesson repositories. We already have repositories for the things academics do: they're called journals. And we have portals (or aggregators, or whatever you want to call them): they are things like PubMed. What we don't have is people putting things into the system in the first place. Growing a separate parallel system to do those things for lessons hasn't worked: as far as I can tell, most of what's uploaded to lesson repositories just sits there.

2015 Post-Workshop Instructor Debriefing, Round 18

Thu, 15 Oct 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

On September 15th, the mentoring subcommittee held the 18th round of debriefing for instructors. At this round, we have a great discussion about teaching (or not) branches during the Git lesson and about Python packages used in examples.

Feedback from a Software Carpentry workshop at PyConZA 2015 in Johannesburg

Wed, 14 Oct 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

On the 3rd and 4th of October we ran a Software Carpentry workshop associated with PyConZA 2015, a conference for Python developers in South Africa. The workshop ran concurrently with the post-conference sprints, in a neighbouring room in the same venue.

Because we expected little overlap between the attendees of the conference (who are mostly existing Python programmers) and the target audience of the workshop, we organized the workshop as a separate event with separate tickets.

AMY 1.0 Released

Sat, 10 Oct 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are very pleased to announce the release of Version 1.0 of AMY, our online workshop administration tool. A complete list of features is included in this blog post, and this milestone describes what we're planning to do for Version 1.1. If you enjoy web programming in Django, you'd be welcome to join in.

Assessing Assessment

Sat, 10 Oct 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Jason Williams and I met with two consultants at the University of Texas at the end of September to get feedback on Software Carpentry's post-workshop survey. They gave us detailed suggestions for improving six of the questions, and felt the rest were OK as they are. The feedback is given below; even without the whole questionnaire (which we will post shortly), we hope it's helpful.

A Summary of Debriefing Feedback on Our Python Lesson

Thu, 08 Oct 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Last month, we discussed results from a survey of how our instructors are teaching Python. We now have a summary of the feedback we've received in our bi-weekly debriefing meetings. The recurring themes are:

A greater choice of exercises and multiple choice questions would allow instructors to select domain specific examples and cater to varying levels of learner.
Some instructors added an explanation of the Jupyter notebook or Spyder IDE environments.
Some instructors added an explanation of basic Python datatypes before presenting the lessons.
Comments that there is too much material to fit into a workshop and that some sections seem rushed.
Request for a better explanation of the advantages of the Anaconda Distribution at the start of workshops and resources for post workshop learning.
Comments on presenters style were positive for funny and entertaining examples and negative for highly mathematical examples.

A Workshop in Brisbane

Wed, 07 Oct 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A sea of green stickies: that really sums up the 28-29 September Brisbane Software Carpentry bootcamp—it ran extremely smoothly. We had 40 people signed up (with a waitlist of six) but lost one at the last minute to acute appendicitis—ouch—and another to project deadlines. The remaining 38 comprised post docs (7), Master's candidates (2), PhD candidates (19), research technical support (4), and four people from industry (one from an NGO and three from a virtual lab). We also had one undergrad (a first for us) and one very bright high school student.

Data Management Plans: A Role for Software and Data Carpentry

Wed, 07 Oct 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I spent the better part of the last three weeks working on an NSF-IOS Doctoral Dissertation Improvement Grant (DDIG) proposal. Pretty much daily, I consulted this list of publically available grant proposals in the biological sciences to look at other people's proposals. It's an awesome resource if you want to see how people write their project description, but there are no links to example data management plans, facilities, summaries, etc. Where does one go for examples of or advice on these supplementary documents?

At least part of the answer is "here". The last page of NSF's information about Data Management Plan requirements, updated on October 1, urges readers to check out Data Carpentry and Software Carpentry for resources and training. This is a huge shout-out (see these tweets), so how can SWC and Data Carpentry do more?

More About Jonah Duckles

Tue, 06 Oct 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I'm thrilled to be stepping into the role of Executive Director of the Software Carpentry Foundation. I come to you as an experienced instructional community member excited to share my administrative, grant writing and professional background to help the Software Carpentry Foundation become sustainable and reach exciting new goals.

September 20 - October 4, 2015: A New Executive Director, Instructor and Helper Retreat, Data Visualisation Lesson, Teaching, and Lesson Citations

Sun, 04 Oct 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Highlights

Please welcome our new executive director, Jonah Duckles!
Please join us on 14 November for the first international multi-site instructor and helper retreat. The blog post explains how you can get involved or host a site.

Lessons

Read about the new lesson on Data Visualisation with D3 and the pros of interactive data visualisation in Isabell Kiral-Kornek and Robert Kerr's post.

Contribute

We're actively revising our teaching practice and lesson design. The latest feedback from people who've taught the Python lessons is summarised and there's another opportunity for you to add your thoughts via a short survey or longer discussion. We'd love to hear from you!
We're still talking about the citation format for our lessons. You can follow the latest discussion and contribute your suggestions at the blog post.

A Case for Online Data Visualization

Fri, 02 Oct 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This article originally appeared in The Research Bazaar.

Thorough data analysis is only one part of good research. Equally important is communicating the outcome well and accessibly. And visible research is accessible research.

Our main motivations for publishing our research results are:

making them openly accessible to the public,
informing fellow researchers about new outcomes that will help them in their research, and
strengthening our professional profiles.

Please Welcome Our New Executive Director

Thu, 01 Oct 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are very pleased to announce that Jonah Duckles has accepted the position of Executive Director of the Software Carpentry Foundation, and will start on Monday, October 5, 2015. Jonah was most recently the Director of Informatics and Innovation at the University of Oklahoma where he partnered with researchers to improve their computational workflows while developing maker spaces for the campus. He holds a BS in Physics and an MS in Forestry and Natural Resources, both from Purdue, and has been a very active contributor to Software Carpentry for several years.

Citation Format

Wed, 30 Sep 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Earlier this month, we published our lessons by giving them DOIs through Zenodo. As we said in an earlier post, though, we've been struggling to figure out (a) how to cite them in text and (b) how to express their metadata in standard bibliographic formats to produce those human-readable citations.

Thinking About Teaching

Mon, 28 Sep 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A little over a year ago, we blogged about jugyokenkyu, or "lesson study", a bucket of practices that Japanese teachers use to hone their craft, from observing each other at work to discussing the lesson afterward to studying curriculum materials with colleagues. Getting the Software Carpentry Foundation up and running almost immediately pushed that aside, but now that the SCF is up and running, it's time to return to the subject. Discussion of how teaching practices are transferred is part of that; so are two other developments this week.

Announcing the 2015 Instructor and Helper Retreat

Tue, 22 Sep 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Update: to stay current, please visit the Etherpad at {{page.etherpad}}.

The Mentorship Committee is very excited to announce the first ever Software & Data Carpentry Instructors & Helpers Retreat, happening worldwide on November 14. We're inviting all Software and Data Carpentry instructors and helpers to get together at sites around the world for a day of sharing skills, trying out new lessons and ideas and discussing all things instructor and helper related.

September 6 - 19, 2015: A Retreat, New Mentors, Instructor Training Update, Preparing Researchers, Interactive Excercises, and a Student's Experience.

Sat, 19 Sep 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Highlights

The first Software Carpentry and Data Carpentry instructor and helper retreat will be held on 14 November 2015! Please get involved by hosting a local session or attending one. Remote participation will also be possible.
We have seven new members on the Mentoring Subcommittee. Thanks to Rayna Harris, Christina Koch, Sue McClatchy, Mariela Perignon, Phil Rosenfield, Michael Sarahan, and Belinda Weaver for joining us in this important task.
Instructor training has been revised. Read the blog to learn about the new plans for 2015 and beyond.
Jazib Askari, a remarkable 6th form Student at Altrincham Grammar School for Girls wrote about her Software Carpentry work experience.

Recommended

Christina Koch mentioned some fun interactive excercises for explaining concepts related to automation and version control in her latest blog.

Contribute

"What would you teach to prepare researchers better to do good science given the changing technological landscape?" An interesting post by Naupaka Zimmerman about being wrong, provenance, and structured data. What are your ideas?

Events

Join the Software Credit Workshop on 19 October 2015 in London to participate in the conversation about career advancement for research software developers.

Software Credit Workshop in London, 19 October 2015

Fri, 18 Sep 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Securing credit for research software is the subject of the Software Credit Workshop. This is taking place at the Natural History Museum, London, UK on the 19th October 2015.

Explore what contributions software can and should make for career advancement. Discuss ways in which you see software tools and applications supporting the current needs of researchers and software developers seeking credit for their involvement in research software. Identify and propose ideas for improving the way software’s contribution to better research are connected and how this should support appropriate reputational credit for research software enablers.

Join other funders, publishers, software developers, researchers, leaders, citation experts and altmetrics visionaries to make your thoughts heard and shape the conversation.

Find out more and register at www.software.ac.uk/software-credit.

Software Engineering Practices in Science

Wed, 16 Sep 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Dustin Heaton and Jeffrey Carver have just published a paper titled Claims About the Use of Software Engineering Practices in Science: A Systematic Literature Review:

Teaching to the Workflow

Tue, 15 Sep 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

"Teaching to the test" has a deservedly bad reputation, but what about "teaching to the workflow"? A group of us came together at the NCEAS Open Science Codefest last year and put together a paper on open science in ecology. In it, we sketch three examples of possible open science workflows (Figure 2 in the paper). In response, I was asked what Software Carpentry should teach to prepare people for working in those ways. My top three things are (in order):

Not Quite Lesson Material

Mon, 14 Sep 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Originally posted on the author's blog.

Lesson plans are a funny thing. The Software Carpentry lessons are an odd balance of a lesson to be read (like a textbook) and a lesson plan for instructors, where they bring the text to life via speaking, live coding, and doing exercises.

This text format doesn't lend itself well to a few activities I've used in Software Carpentry workshops. So I'm going to throw them up here for other instructors to look at - feel free to add your own activities in the comments, or submit another blog post!

Rebooting Instructor Training

Mon, 14 Sep 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Instructor training has been key to Software Carpentry's growth, but it was clear by the time our last online class finished in April that we couldn't and shouldn't keep doing it the way we had been: couldn't because of the time it ate up, and shouldn't because it wasn't doing enough to prepare people for actually teaching.

After five months of thinking, talking, and revising, we finally have a plan for rebooting the course. We are going to run it twice between now and the end of the year: once in the usual multi-week format, and once more compressed into two full days. We are also going to introduce some preparatory requirements, mentoring for new instructors, and mini-lessons to help ease people into organizing and running workshops. Finally, we will start training a few more people to run instructor training so that we can afford to give trainees more personal attention.

The full proposal is included below, and we will be contacting people who have applied for instructor training over the next few days to invite them to take part. I apologize right now for not being able to offer everyone a place right away, but if this reboot goes well—and I think it will—we should be able to start clearing the rest of our backlog soon.

How Teaching Knowledge Is Transferred

Sun, 13 Sep 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I hesitate to say so, but I believe that we pedagogues tend to exaggerate greatly the amount of change in educational practice that results from reading what other people say should be done...

— Stephen Corey, 1951

Being back in Edinburgh thirty years on has occasioned much reflection upon lessons learned. This has occasioned a re-reading of some papers by Prof. Sally Fincher, whose research group at the University of Kent studies the teaching and learning of computer science. In particular, I have been looking at what they have discovered about how educators share teaching practices. I hope these excerpts and reflections are of interest. (Note: section titles link to papers.)

Workshop at the University of Arizona

Sun, 13 Sep 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Software Carpentry Workshop

Who: Practicing and aspiring research scientists

When: October 3 and 4, 2015 (9am - 5 pm on both days)

Where: UA Integrated Learning Center

How: Register at http://iplantsc.eventbrite.com

2015 Post-Workshop Instructor Debriefing, Round 17

Wed, 09 Sep 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The mentorship committee held their latest round of debriefing for instructors of recent workshops on Tuesday, September 1. Then, on Friday September 4th mentorship committee met for a regular bi-weekly meeting. Some comments from the committee meeting have made their way into this debriefing post.

Reporting on a Commercial Workshop

Wed, 09 Sep 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

On August 22, Marty McGuire (@schmarty) and I (@joshua_r_smith) taught a for-profit workshop based on Software Carpentry materials, particularly the novice Python lesson. This post is a debrief of that workshop.

August 27 - September 5, 2015: Lessons Published, ReScience, SSI Fellowships Open, Interview, and Mailman Threads as GitHub Comments.

Sat, 05 Sep 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Highlights

Our lessons are now published! Please find important citation information in the blog post.

New

You can now publish your computational studies that replicates previous findings in ReScience.
The Art of Data Science by Roger D. Peng and Elizabeth Matsui has been published.
Greg Wilson's recent CS Education Zoo interview about Software Carpentry is available via YouTube.

Contribute

Do you know of a tool that can convert Mailman mailing list threads into comments on a GitHub issue?

Opportunities

Applications are open for the Software Sustainability Institute's Fellowship programme. Apply before 1 October 2015.

Our Lessons Have Now Been Published

Sat, 05 Sep 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

It's been a long time coming, but we have finally published Version 5.3 of our core lessons. Please cite them as:

Announcing ReScience

Thu, 03 Sep 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

It's our great pleasure to announce the creation of "ReScience" which is a peer-reviewed journal that targets computational research and encourages the explicit replication of already published research, promoting new and open-source implementations in order to ensure that the original research is reproducible.

Running a Code Retreat

Wed, 02 Sep 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I recently attended a scholarship retreat run by Google with about 40 other students. We spent one day doing a "code retreat" and I wanted to share my experience and what I learned about programming.

SSI Fellowship Applications Open

Wed, 02 Sep 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Fellowship Programme run by the Software Sustainability Institute funds researchers in exchange for their expertise and advice. The main goals of the Programme are encouraging best practice and gathering information about research software from all disciplines, and encouraging Fellows to develop their interests in the area of software sustainability (especially in their areas of work). The Programme also supports capacity building and policy development initiatives.

Better Teaching Practices

Tue, 01 Sep 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

It wasn't part of our original plan, but over time Software Carpentry has come to be about better teaching as much as it is about better computing. In aid of that, I would like to offer the following:

Three Graphs I Would Like to See

Mon, 31 Aug 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I spent part of the weekend chatting with a friend in Cambridge who used to be science editor at The Independent and now edits Scientific Computing World. During those conversations, I realized that there are three graphs I'd really like to see:

GSoC 2015 Finished

Sun, 30 Aug 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This year's edition of Google Summer of Code (GSoC) has now finished. We are very happy with the outcome of the work of the three students that worked under the NumFOCUS umbrella that we helped to coordinate.

2015 Post-Workshop Instructor Debriefing, Round 16 (morning)

Fri, 28 Aug 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The mentorship committee held their latest round of debriefing for instructors of recent workshops on Tuesday, August 18. This post highlights the themes discussed in both the morning and evening sessions. Our participants included multiple levels of experience, including new instructors and experienced instructors who were preparing to teach again soon. The most difficult aspects of their experiences are described below, as well as things that work well and may assist in ameliorating problems.

August 18 - 26, 2015: Instructors' Retreat, Applying Discounts, Adding Lessons, Be a Mentor, Undergraduate Training, and Improving RStudio.

Wed, 26 Aug 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Highlights

Do you want to meet other Software Carpentry instructors and share your training experience? Let us know when you'll be available for our first Virtual Instructors' Retreat. We've already heard back from 60 people!
How do you know if a fee waiver or discount will apply to your Software Carpentry Workshop? Read our new guidelines.
Mike Jackson wrote about how easy and rewarding it is to add lessons to the Software Carpentry repertoire. It's inspirational to see the power of openness and collaboration.

Contribute

Join our mentoring sub committee — it's a great opportunity to simultaneously learn and give back.
Share your ideas for a Maths undergraduate Python curriculum with Ian Hawke (and us).
RStudio wants to hear from you about how to improve their software for training purposes. An interesting discussion is already under way.

Fee Waivers and Discounted Fees

Mon, 24 Aug 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A couple of weeks ago the Steering Committee met in person and one of the topics discussed was fee waivers and discounted fees for workshops and instructor training. Continue reading for our conclusions.

Virtual Instructors Retreat

Mon, 24 Aug 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Mentoring sub-committee is planning a Virtual Instructors Retreat this fall. The goal of the event is to build community between instructors, as well as give us all a chance to practice our teaching and give and receive feedback from each other. This will be a day-long event where we meet in person with our local community, where possible, and remotely where meeting in person is not possible. World-wide, all groups will also interact with each other via internet conferencing for specific portions of the event.

We want as many of our instructors as possible to be able to participate, so please help us to do this by filling out this Doodle poll.

Also, we are interested in your ideas about such an event. Please feel free to mail any ideas, comments or questions to the Mentoring sub-committee.

Join the Mentoring Subcommittee

Sat, 22 Aug 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Mentoring Subcommittee is seeking for new members to help with the debriefing sessions and future activities. If you are interested in join us, please send a email to mentoring@lists.software-carpentry.org.

What Is ORCID?

Sat, 22 Aug 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Part of being a successful researcher lies in the ability to stand out from your peers, which can be done through making and being acknowledge for valuable and original contributions. Once acknowledged for one discovery this can then act as a springboard to allow your peers to identify your other scholarly contributions, or alternatively identify potential for future collaboration, or be used as a proof of your research skills when applying for further funding. In short, making your work and accomplishments known is crucial to success in academia.

Yet whilst so many functions of the academic process hang on the concept of citations and as such the ability to identify the researchers behind a piece of work, the actual means of identifying a researcher is not without its problems. For example how to identify the discoveries and related work of a specific "John Smith" after coming across one of the author's particularly informative publications? How do we keep up to date with a researcher's publications if they change their name? How do we keep track of a successful researcher who works across a number of institutions over the course of their career or who engages in work across a range of disciplines?

Feedback on Math with Python for Undergraduates

Fri, 21 Aug 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Ian Hawke (who put together these notebooks on testing numerical code) is now putting together some Jupyter Notebooks to teach Python to first-year undergraduates in mathematics. He would be grateful for feedback, and we'd be grateful if you could give him some: we'll learn a lot about what we should teach from seeing what you think he should.

Experiences Adding a Lesson on Make

Thu, 20 Aug 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

In June I added a lesson on Automation and Make. In this blog post, I describe how the lesson evolved, my experiences in porting it into the Software Carpentry lesson template, and the community's response...

Improving RStudio as a Teaching Tool

Thu, 20 Aug 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

On Twitter, the RStudio support team requested suggestions for how to make RStudio better as a teaching tool. So I've started an issue on their support site for instructors to chime in with ideas. Go ahead and let them know what would make teaching with RStudio easier and better!

Stickers

Tue, 18 Aug 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Want to dress up your laptop? Software Carpentry stickers are now available from Sticker Mule. All proceeds will go to the Software Carpentry Foundation.

August 3 - 17, 2015: Data Carpentry Funded, Citations, Improving Our Lessons, and Lab Data Management.

Mon, 17 Aug 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Highlights

Fantastic news from our sibling organisation, Data Carpentry, who received USD 750,000 in funding from the Gordon and Betty Moore Foundation.

Contribute

Do you have suggestions for the citation format of our lessons? Should all contributers be named? Please participate in this discussion to help us move forward with making our lessons citeable.
How can we improve the next version of our Python 3 lesson? Contribute to the discussion before September when content of the next release will be decided upon.
Meanwhile, a few posts focussing on preparing for and teaching various lessons have appeared over the past months. You can help improve what and how we teach by sharing your experience in terms of content, format, and preparation for teaching at a Software Carpentry workshop.
Have you designed and built a data management system for your lab or project? Please share your experience to help us rethink how we teach databases and their integration with other tools.

Science Track at PyCon UK 2015

Mon, 17 Aug 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Today, researchers in the sciences, humanities and arts, all use code as an everyday part of their work. Often such code is written using the popular Python programming language. Thanks to generous funding from the Software Sustainability Institute, PyCon UK will have a track for scientists and other researchers who want to improve their coding skills, learn from colleagues, and discover new ways in which Python and its community can support their work.

Prepping for the Python Lesson

Sun, 16 Aug 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Inspired in part by Byron Smith's post about trimming our standard Python lesson, Christina Koch has written a post of her own about preparing to teach that lesson. She organizes her discussion around the motivating question that opens the lesson: "We have to accomplish a task (reading in data, analyzing and plotting it) by writing a program. How can we be smart about it?" It's a good read, as it shows how an experienced instructor thinks about (re-)designing teaching material. I hope her conclusions will feed into our discussion of how to revise the lesson (which we're going to decide in September).

Checking What We Teach

Fri, 14 Aug 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Back in May, Jonathan Klaasen wrote a post about setting up a lab data management system. After re-reading it, I think it's a good reality check on what we teach about databases: from what I can tell, we cover most of the information management needs that Jonathan touches on. I also think it could be a great motivating example for a lesson on databases, and on how to combine them with other tools like shell scripts. If you've ever built or used something like what Jonathan describes, I'd be grateful for comments describing where your setup is the same and where it's different.

Teaching in Bali

Fri, 14 Aug 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

On July 2015, I was fortunate to receive an invitation to the first international workshop of the Diversity of the Indo-Pacific Network (DIPnet), which explains the "Bali" part, in order to teach R and the Unix command line during the first two days. The weeklong workshop was co-hosted by the Hawai'i Institute of Marine Biology and Indonesia Biodiversity Research Center (IBRC), and lead by Dr. Eric Crandall from CSU Monterey Bay. The theme of the workshop was "Molecular Ecology and Bioinformatics in Developing Countries", which featured lectures and labs developed by the invited participants in their area of expertise. For this, a two-day software carpentry workshop was very much needed to begin the week and prepare all participants for the computational sessions coming up in the week.

Data Carpentry Receives Grant from the Moore Foundation

Thu, 13 Aug 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Reposted from the Data Carpentry blog.

We are extremely pleased to announce that Data Carpentry has received $750,000 in funding from the Gordon and Betty Moore Foundation.

Trimming the Python Lesson

Thu, 13 Aug 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are close to releasing a new version of our standard Python lesson that uses Python 3 instead of Python 2. As soon as that's done, we're going to warm up this discussion about fixing that lesson. One new contribution to that is this post by Byron Smith about how he trimmed the existing lesson. If you have other experiences or comments, please add them to this GitHub issue so that we can make decisions in September about what to include in Version 5.4 in November.

Publishing, Metadata, and Being Ahead of the Curve

Wed, 12 Aug 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As described in earlier posts, we are publishing our Version 5.3 lessons through Zenodo to make it easier for people to cite them. We're getting closer, but there are still a few bumps in front of us, and we would like our readers to help us figure out what to do next.

2015 Post-Workshop Instructor Debriefing, Round 15

Fri, 07 Aug 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The most recent installation of instructor debriefings by the mentoring subcommittee was held on August 4 to discuss recently completed workshops. We were joined by new instructors as well as a number of very experienced instructors (some of whom also maintain lesson repos), who all taught recently or are preparing to teach workshops. We highlight below a few of the main points from our discussions, including interesting new ideas, things that worked well, and things that were difficult.

A Workshop for Undergraduates at UC Berkeley

Thu, 06 Aug 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We ran a workshop for undergraduates at UC Berkeley on July 9-10, 2015. This was a quickly planned workshop from concept to completion (~19 days). I was excited to run a workshop geared towards undergraduates, especially those involved in research projects in the summertime.

2015 Post-Workshop Instructor Debriefing, Round 14

Tue, 04 Aug 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

On July 21 the mentorship team ran the 14th round of instructor debriefing session and received feedback from workshops at the NIH, UC Berkeley and UC Davis.

July 23 - August 2, 2015: Online Survey Still Open, Recorded Lessons Available, and Another WiSE Workshop Coming Up.

Sun, 02 Aug 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Highlights

Complete the online SWOT survey before 5 August to help steer Software Carpentry in the right direction over the next few years.
Software Carpentry lessons from SciPy 2015 were recorded and is available in YouTube.

Opportunities

We're developing a template to provide solutions to challenges in a sensible way. Please share your ideas for a template with us.

Events

Another workshops for women in science and engineering will be held at UC Davis on August 17-18, 2015.

SciPy 2015 Workshop Videos

Wed, 29 Jul 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Software Carpentry was pleased to present a full two-day workshop during the recent SciPy Conference tutorials. The entire conference was recorded, including all sessions of our workshop:

The course materials are accessible via the workshop webpage and GitHub repo.

The course would not have been possible without the help of several Software Carpentry members: Azalee Bostroem, Matt Davis, Jess Hamrick, Ted Hart, Katy Huff, Thomas Kluyver, Jens Nielsen, April Wright, and Elizabeth Seiver. Many thanks to these talented folks and to the SciPy organizers for inviting us!

Solution for the Challenges

Tue, 28 Jul 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

One of the enhancement that we have in mind for the next release of our lessons is to provide the solution for the challenges. Having the solutions will help instructors speed up when preparing the lesson for workshops and learners reading the lessons. Before we start adding the solutions will be great to have one template for how it should be write. If you have some suggestions, please add it at this issue on GitHub.

A Pair of Workshops

Thu, 23 Jul 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Do you know your options for software licensing? Have you heard of new funders' requirements for software sharing? This workshop in Cambridge (UK) on Monday, September 14, is your chance to get expert advice on these and other questions about software licensing. Speakers include Neil Chue Hong, the Director of the Software Sustainability Institute, and Shoaib Sufi, the SSI's Community Leader.

The event is open to everyone and you are welcome to bring your questions to the workshop. There will lots of opportunities to discuss your queries at the sessions and during the dedicated networking lunch. For more information, please see the workshop's website.

There is also a workshop on Monday, October 19, at the Natural History Museum in London on getting credit for software. This workshop will explore what contribution software can and should make for academic reputational credit; i.e., how can the production of software tools and applications contribute to career advancement in the academic research setting for both researchers who build software as part of their research and developers who build tools and support research.

July 07 - 22, 2015: Hiring an Executive Director, Strategic Planning, SWC-inspired Book, Open Research Repository, AMY version 0.6, and New Team Members.

Thu, 23 Jul 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Highlights

Software Carpentry is entering the next phase and will be hiring a new Executive Director. Please apply if you are passionate about the mission of the Software Carpentry Foundation!
Do you want to help steer Software Carpentry in the right direction over the next five years? Help us by completing a short anonymous SWOT survey.

Changes

Two new coordinators, Maneesha Sane and Katarzyna Zaczek, have joined the Software Carpentry team, and we're saying goodbye to Amy Brown.
We've changed the workshop administration fees. Please read the post for details.

Resources

"Effective Computation in Physics" written by two Software Carpentry instructors, Anthony Scopatz and Katy Huff, is now available.
An Open Research Glossary generated through a crowd-sourcing effort with a number of named contributors is now available.
AMY version 0.6 has been released with a long list of new features and some fixes.

WiSE Workshop at UC Davis Aug 17-18

Thu, 23 Jul 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are pleased to announce that the latest in our series of workshops for women in science and engineering will be held at UC Davis on August 17-18, 2015. This workshop is aimed at those in science, engineering, medicine, or related fields who identify as women, female, or on the non-binary spectrum. Our instructors are experienced with creating safe spaces for those on the trans* spectrum, and one openly identifies as a trans woman; ensuring that we maintain an inclusive environment is a top priority for this workshop. For more information, or to register, please see the workshop website.

Changes to Workshop Administration Fees

Mon, 20 Jul 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

After discussion with our Advisory Council and Data Carpentry, we have agreed to make some changes to the administration fee we charge for workshops that we help organize in order to reflect the real cost of our staff's time and our overheads, and to reflect the value of the training. Please see the workshop request page for the updated pricing information.

Welcome Maneesha and Katarzyna

Mon, 20 Jul 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are very pleased to announce that Maneesha Sane (pronounced "sah-nay") will be joining us in August as our new Program Coordinator. Having coordinated and managed public events and mentoring programs for several organizations in the Philadelphia area, Maneesha retrained as a software developer. She has been managing events and matching mentors to learners for over a decade, and we are looking forward to working with her.

2015 Post-Workshop Instructor Debriefing, Round 13

Sat, 18 Jul 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Last week the mentorship team ran the 13th round of instructor debriefing session and received feedbacks from the workshops at Brigham Young University (check Belinda's post about it), Johns Hopkins University, Notre Dame University, Pennsylvania State University, the Jackson Laboratory, the University of Queensland, the University of Hawaii at Manoa, and the University of Toronto - Women in Science and Engineering (check Pauline's post).

AMY Version 0.6

Sat, 18 Jul 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Yesterday was the release day for AMY v0.6.. Here's what's new and what's changed in this release.

Help Software Carpentry's Strategic Planning

Sat, 18 Jul 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The SCF Steering Committee is undertaking a strategic planning process that will help us to identify the SCF's working priorities for short- and long-term success, based on input from you.

Your input is crucial—we are beginning the process by asking that you identify key Strengths, Weaknesses, Opportunities, and Threats (SWOT) facing SCF over the next five years via this anonymous survey. We will review the survey results, summarize key issues, and develop a summary and plan that addresses the issues raised.

To help, please fill in the SWOT Survey. Note that the survey asks for 3 answers for each strengths, weaknesses, opportunities, and threats but you may fill out more or less as you see fit. Please fill out the survey only once and by August 5, 2015.

Top 10 Myths about Teaching CS

Sat, 18 Jul 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Mark Guzdial (whose blog has been a frequent inspiration) recently wrote an article title Top 10 Myths about Teaching Computer Science:

The lack of women in Computer Science is just like all the other STEM fields.
To get more women in CS, we need more female CS faculty.
A good CS teacher is a good lecturer.
Clickers and the like are an add-on for a good teacher
Student evaluations are the best way to evaluate teaching.
Good teachers personalize education for students' learning styles.
High schools just can't teach CS well, so they shouldn't do it at all.
The real problem is to get more CS curriculum out into the hands of teachers.
All I need to do to be a good CS teacher is model good software development practice, because my job is to produce excellent software engineers.
Some people are just born to program.

The Open Research Glossary

Thu, 16 Jul 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

It's been knocking around for a while now, but this week saw a big new release of the Open Research Glossary, a crowdsourced glossary of terms, acronyms, tools and concepts in and around open science. There's a lot of jargon in this area and it's often a barrier to understanding for the uninitiated. For example: 'green', 'gold', 'diamond' and 'hybrid' open access are shorthand terms which aren't always fully understood.

The alphabet soup of acronyms are even worse! Take SHERPA/RoMEO for instance: it's a brilliant resource for checking author self-archiving rights relative to publisher-imposed restrictions and embargoes, but the name 'SHERPA/RoMEO' doesn't exactly make that clear. This glossary aims to elucidate all the need-to-know terms in open scholarship.

Software-Carpentry-Inspired Book: Released and On Sale!

Tue, 14 Jul 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We're excited to announce the official release of "Effective Computation In Physics". This book was written by two Software Carpentry instructors, Anthony Scopatz and myself, Katy Huff. We were enormously inspired by the vision and work of the Software Carpentry community and expanded on that vision in the book to create a "field guide to research in Python." While examples and more advanced content are presented in the context of research in the physical sciences, the majority of the book will be useful to all researchers doing scientific computation. In book form, we were able to dive in and expand on best practices more deeply and more extensively than is possible in a workshop. We're extremely proud to have created "SWC in a book" as instructor Daniel Chen recently described it.

You (or your students and colleagues) can even get a 50% discount if you grab it before July 17th using the code: WKPYDP.

Teaching with Jupyter

Tue, 14 Jul 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We had a great birds of a feather session at the SciPy conference last week and decided to create a mailing list specifically for instructors who are interested in using the Jupyter Notebook for teaching. The aim of the mailing list is to provide a place for instructors to share materials, strategies, advice, etc. on teaching with the notebook and the logistics that are involved with that. If you're interested in joining the mailing list as well, you can add yourself at https://groups.google.com/forum/#!forum/jupyter-education

Congratulations to Project Jupyter

Tue, 07 Jul 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This is wonderful news: the Helmsley Charitable Trust, the Alfred P. Sloan Foundation, and the Gordon and Betty Moore Foundation have just pledged $6M over three years to Project Jupyter (formerly known as the IPython Notebook). Fernando Perez of University of California, Berkeley and Lawrence Berkeley National Laboratory and Brian Granger of California Polytechnic University, San Luis Obispo will lead the project. Congratulations to them, their team, and their community—this is well deserved, and will help scientists all over the world in ways we cannot yet even imagine.

What I Learned in Brisbane

Tue, 07 Jul 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Not a single negative was recorded about the people either teaching or helping out at the Brisbane Software Carpentry bootcamp last Thursday and Friday (2-3 July). There were a lot of positive comments though:

The workshop is great with passionate instructors
All the helpers in the room are very helpful and fun
Excellent support from helpers
Instructors readily available to help especially when falling behind or [we] need help understanding codes

Hiring a New Executive Director for Software Carpentry

Mon, 06 Jul 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Software Carpentry Foundation seeks to hire a new Executive Director to build relations with partners, oversee the development of our curriculum, be Software Carpentry's spokesperson, and work with the Steering Committee to set our future direction. The successful candidate will initially be co-Executive Director, and will job-share with the current Executive Director during a brief transition period, after which they will become the new ED.

To apply, please send email to {{site.contact}} by July 31, 2015 with "Co-Executive Director" in the subject line, and include:

A brief resume or CV (approximately two pages).
A brief statement (approximately two pages) of what you would hope to accomplish in your first year as Executive Director. Please also include a paragraph about any work you may have done with Software Carpentry in the past and another about your experience working with other volunteer organizations.

We will begin interviews immediately after July 31, and hope to have someone in place no later than the end of August.

June 29 - July 06, 2015: Research Software Engineers, Not Changing Lesson Build Tools, and Moving to Python3.

Mon, 06 Jul 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Highlights

Do you know what a Research Software Engineer is or why we desperately need to recognise the role RSEs are playing in research? Read about the history of RSEs and a fellowship programme available for RSEs in the UK.
Version 5.4 of our lessons will be released at the end of November, rather than mid-August, and lesson build tools will remain unchanged until then. Most importantly we'll be changing the Python lessons to run on Python 3.

Contribute

Are you looking for other ways to contribute to Software Carpentry? Visit our Projects page to see an exciting list of opportunities for you to get involved.

Our Next Big Step

Mon, 06 Jul 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

With Software Carpentry's rapid growth over the past couple of years, the combined responsibilities of being the Executive Directory and running the instructor training program have become more than a single person can manage. And after five years of working to grow Software Carpentry into the world-wide community it has become, I'd like to spend more time with my family.

The Software Carpentry Foundation is therefore hiring a new Executive Director. I will transition to running instructor training so that the new ED can devote themselves to building relations with partners, overseeing the development of our curriculum, being Software Carpentry's spokesperson, and working with the Steering Committee to set our future direction. Our new hire will initially be co-Executive Director, and will job-share with me during a brief transition period, after which they will become the new ED.

This is the next logical step in Software Carpentry's evolution, and one that we have been working toward for more than a year. As with the election of the Steering Committee in January, it's a sign that Software Carpentry is here to stay, and nothing could make me prouder.

Pushing Back

Wed, 01 Jul 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A week ago, we posted a proposal to use Jekyll to build our lessons rather than Pandoc. The immediate reaction was almost uniformly positive, but in the days since, people have pushed back on two fronts:

What is a Research Software Engineer?

Mon, 29 Jun 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

By now, many people in the UK (well, many of the sort who read this blog) will have heard the term Research Software Engineer, but what exactly is an RSE, and what effect will the creation of this title have? To understand, we need to go back to the Software Sustainability Institute's Collaborations Workshop in early 2012 (summarized in these blog posts and others). Those discussions led to this position paper at Digital Research 2012, whose authors argued that:

June 17-28, 2015: A Lesson on Make, AMY 0.4 Released, Opportunities to Contribute, Practical Tips for Running Workshops, and Appointing a Program Coordinator.

Sun, 28 Jun 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Vacancies

Software Carpentry is hiring a Program Coordinator. Please see the blog post for more information.

Highlights

A new lesson on Make has been added to the Software Carpentry repertoire.
AMY 0.4 was released. New features are listed in the blog post.

Contribute

Would you like to contribute by recycling previously submitted assessment excercises and adding the best ones to our lessons? It can now easily be done by visiting the newly created repo of submitted MCQs and exercises.
Learner assessment has been receiving a lot of attention over the past months. A first draft of the new feedback survey is now available. We're looking forward to your comments/contributions.
Should we change our lesson templates to make use of Jekyll rather than Pandoc? Please contribute to the discussion to help us make an informed decision.
Greg Wilson summarised lessons learned from teaching instructors. There is again an opportunity for you to contribute to the direction instructor training takes in future. Let us hear from you?

Useful Tips

Splitting the terminal window allows the instructor to display recent commands while continueing with the lesson at the same time. Read the post by Raniere Silva to see how it's done.

2015 Post-Workshop Instructor Debriefing, Round 12

Fri, 26 Jun 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The mentoring subcommmittee hosted instructor debriefings on 23 June 2015 to discuss recently completed workshops. We are delighted that so many new instructors are joining us at these sessions as a way to prepare for upcoming workshops, and welcome anyone else interested to attend as well. Below we highlight a few discussion points from our sessions, including issues with lesson pacing and Python installation, as well as tips on using the etherpad and GitHub organizations. A more in-depth synopsis of a recent workshop can be found in this fantastic post on Raniere Silva's blog.

Training Lessons

Fri, 26 Jun 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I wrote about our experiments with the format of instructor training back in May. At that time, we had run the class as:

a multi-week online class,
an in-person two- or three-day class, and
a mixed mode with the trainees physically together for two days with the trainer coming in via teleconference.

We have since tried the mixed mode twice with the trainees at three different sites (three universities in Arizona for one run, and universities in Cape Town, Sheffield, and Ann Arbor for the other). We've also gathered a lot of feedback on what people want from instructor training and what its prerequisites should be. Here's what we've learned.

Workshop at CERN

Thu, 25 Jun 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

At the beginning of June Rémi Emonet, Kwasi Kwakwa, and Chelsea Chisholm ran a workshop at CERN. Rémi has just posted a review. It went well, and there are a lot of good ideas in his write-up—from using a whiteboard for diagrams to the IPython Notebook's successor (Jupyter) and some semi-improvised intermediate material.

Using Jekyll for Lessons

Wed, 24 Jun 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A recurring complaint about our lesson template is that it requires authors to commit generated HTML files to their repositories as well as their Markdown source files. This is necessary because we use Pandoc to convert Markdown to HTML, but GitHub will only run Jekyll.

There were a bunch of reasons for using Pandoc instead of Jekyll, but it is now clear that the simplicity of only committing Markdown—i.e., of using GitHub pages the way they're meant to be used—is more important. We have therefore created a prototype of a Jekyll-based template (which is rendered here). The most important changes are:

Another Good Workshop in Brazil

Tue, 23 Jun 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The indefatigable Raniere Silva has just posted a description of a workshop at the University of Ceará that he and Dani Ushizima just finished teaching. It went well, and there are a lot of good ideas in his write-up — please check it out.

Assessing Our Learners Part I

Tue, 23 Jun 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Three weeks ago, Jason Williams, Jeramia Ory, and Daniel Chen met at the New York Public Library to work out an initial survey to assess our learners. Greg Wilson and Katerena Kuksenok joined virtually to provide feedback. The goal was to take the comments from the various initial GitHub issues and create a draft of an assessment survey for everyone to provide input. Our first draft is up, so please provide feedback at {{site.github_url}}/assessment/issues/6.

Program Coordinator Position Available

Mon, 22 Jun 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Software Carpentry has grown and grown again since our re-launch in 2010. We are now helping thousands of scientists every year, and while many of our partners and instructors are now organizing workshops on their own, a lot of details still need to be sorted out to keep the whole show on the road.

We therefore wish to hire a Program Coordinator to manage our day-to-day operations. This paid position will initially be part-time, but we expect that it will convert to full-time after a probationary period if funding allows. The successful candidate does not need to be either a programmer or a scientist, but must be well-organized, and can be located anywhere with reliable Internet access. The full description is included below; to apply, please email {{site.contact}} with "Program Coordinator position" in the subject line and a resume (either attached as PDF, or a link to something online). And please help us spread the word: we're a fun bunch to work with, and this would be a chance for someone to help a lot of scientists get more done in less time, and with less pain.

Finally, I'd like to take this opportunity to thank Arliss Collins for all her hard work over the past year and a half. She is moving on to other duties now that our relationship with the Mozilla Science Lab has ended, but we couldn't have gotten through the past eighteen months without her. I'd also like to thank Amy Brown, who has come back to keep things going while we search for someone permanent, and welcome Kasia Zaczek, who is about to start handling workshops for us in Europe on behalf of Cyfronet in the same way that Giacomo Peru and Aleksandra Pawlik have been handling them in the UK on behalf of the SSI. Here's hoping that one day, somewhere, we can all get together for a group photo...

Recycling Training Course Material

Sun, 21 Jun 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Over the past two years, more than 200 people have written multiple choice questions and other assessment exercises as part of Software Carpentry instructor training. Many of these are very good, some are excellent, and they all deserve a second look to see whether they should be integrated into our lessons.

Research-Based Course Design

Sun, 21 Jun 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I've written before about the breadth and depth of Juha Sorva's work on computing education. His latest contribution is a paper co-authored with Otto Seppälä titled "Research-Based Design of the First Weeks of CS1". In it, they tie the specifics of the new intro programming class at Aalto University directly back to CS education research. More specifically (from their abstract):

Splitting the Shell Window

Sun, 21 Jun 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Raniere Silva has written a short post about a trick he found (via Kate Hertweck) for splitting the terminal window when teaching the shell so that recent commands stay visible at the top. It's a clever idea; we would welcome feedback from other instructors who have tried it or similar things. And if you have tricks of your own that you'd like to share, please let us know—we'd be happy to feature them here.

2015 Post-Workshop Instructor Debriefing, Round 11

Fri, 19 Jun 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We held our 11th round of instructor debriefing last week where we discussed the Software Carpentry workshops held at the University of Oslo, University of Connecticut, University of Campinas, University of the Basque Country, Lawrence Berkeley Lab, Berkeley Institute for Data Science, Murdoch Childrens Research Institute, and Oklahoma State University, as well as a Data Carpentry workshop at the National Data Integrity Conference at Colorado State University. At this debriefing we also had 3 new instructors join us (who have yet to teach a workshop but will be teaching one in the near future) to gain an insight about what works (and what doesn't) at our workshops out in the wild.

Amy Version 0.4

Fri, 19 Jun 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Today's the deadline for AMY v0.4. It contains a bunch of usability fixes, so all our admins should be happy :-)

Adding a Lesson on Make

Thu, 18 Jun 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are very pleased to announce the addition of a lesson on automation and Make, which was created by the SSI's Mike Jackson and Steve Crouch. The repository contains everything you need to teach it, and pull requests are very welcome.

Software Development Practices in Academia

Thu, 18 Jun 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Derek Groen, Xiaohu Guo, James Grogan, Ulf Schiller, and James Osborne have just submitted a paper to arXiv.org titled "Software development practices in academia: a case study comparison". From the abstract:

Academic software development practices often differ from those of commercial development settings, yet only limited research has been conducted on assessing software development practises in academia. Here we present a case study of software development practices in four open-source scientific codes over a period of nine years, characterizing the evolution of their respective development teams, their scientific productivity, and the adoption (or discontinuation) of specific software engineering practises as the team size changes. We show that the transient nature of the development team results in the adoption of different development strategies. We relate measures of publication output to accumulated numbers of developers and find that for the projects considered the time-scale for returns on expended development effort is approximately three years. We discuss the implications of our findings for evaluating the performance of research software development, and in general any computationally oriented scientific project.

Why I Am Not Excited About Julia

Thu, 18 Jun 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

If you hang out in scientific programming circles, you're probably heard of Julia by now. If you don't, or you haven't, it is:

Get More Done in Less Time

Wed, 17 Jun 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Over the past year, Alexandra Simperler has interviewed participants in Software Carpentry workshops to find out what impact we've actually had on their work. Her results are now available on arXiv.org:

The aim of this study was to investigate if participants of Software Carpentry (SC) get more done in less time. We asked 32 questions to assess 24 former participants to analyse if SC gave them the computing skills to accomplish this. Our research shows that time was already saved during the workshop as it could shorten the learning process of new skills. A majority of participants were able to use these new skills straight away and thus could speed up their day to day work.

Like Jory Schossau's study, Alexandra's work shows that workshop participants believe we're making their lives better.

June 10-16, 2015: Software Carpentry is Saving Time, Lessons Version 5.4, Greg's Time, Our Project List, and a Lesson on Reproducible Research.

Tue, 16 Jun 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Highlights

Software Carpentry get more done in less time is the latest publication about Software Carpentry and the impact it's having on research. It reports research done by Alexandra Simperler and is available at arXiv.
Version 5.4 of our lessons is currently in the pipeline and due for release in the middle of August. Please join the discussions and participate in addressing new and existing issues to help us reach our next milestone.
Over the last few months a lot has improved in the way Software Carpentry operates. Greg Wilson revisited some changes and the impact on how his time is spent six months after election of the Steering Committee.
Do you want to know how you can help to move Software Carpentry forward? Take a look at our project list or get in touch. There are loads of opportunities for people with a wide variety of skills.

Resources

Titus Brown posted a lesson on reproducible research on YouTube.

A Lesson on Reproducible Computational Analysis

Mon, 15 Jun 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Titus Brown has recorded a two-hour lesson on reproducible computational analysis and posted it to YouTube. Many thanks to the folks at ICER for making it available.

Updating the Project List

Mon, 15 Jun 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Updating my description of where my time goes made me realize that our list of things we need help with had fallen out of date. The highlights are below; please see the projects page for details, and get in touch if you'd like to help.

Routinely Unique

Sun, 14 Jun 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Back in April, Jeffrey Chang wrote wrote an article for Nature in which he pleaded for those who analyze bioinformatics data to be recognized as creative collaborators in need of career paths. In it, he observed:

To give greater support to researchers, our centre set out to develop a series of standardized services. We documented the projects that we took on over 18 months. Forty-six of them required 151 data-analysis tasks. No project was identical, and we were surprised at how common one-off requests were. There were a few routine procedures that many people wanted, such as finding genes expressed in a disease. But 79% of techniques applied to fewer than 20% of the projects. In other words, most researchers came to the bioinformatics core seeking customized analysis, not a standardized package.

Where the Time Goes (Version 2)

Sun, 14 Jun 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Last November, I wrote a post about where my time was going. A lot has changed since then, including my workload, so here's an update:

Running a Remote Workshop in South Africa

Sat, 13 Jun 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Last month we ran a workshop at Stellenbosch University in South Africa. The workshop instructors were Laurent Gatto (from the UK - coming in as remote instructor), and two local instructors - David Merand and Anelda van der Walt. We had four helpers in the room and 32 participants.

Warming Up for Version 5.4

Fri, 12 Jun 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

It's time to start thinking about what should be in Version 5.4 of our lessons (which we plan to release in the middle of August to get us through to the end of the year). We have opened a discussion ticket for each of the core lessons on GitHub; please add your thoughts there.

Teaching at NIH

Thu, 11 Jun 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Reposted from Fan's blog.

I just finished my two day software carpentry workshop held at National Cancer Institute (NCI), Rockville, MD. The course went for 2 days, from June 9th to June 10th and taught by Jonathan Guyer, Adina Howe, and I. I was in charge of the unix shell and version control (git) sessions (Also many thanks to Dilip Banerjee, Lynn Young for the help!). Personally, I think the course went really well but there are definitely things I could adjust and improve in the future. Below is the run downs.

Why We Can't Have Nice Things

Thu, 11 Jun 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

In the beginning, there were tables: rows upon rows, with columns separated by commas or tabs or something more exotic. They were elegant but limited, so programmers said, "Let there be XML!" And lo, there came a great wailing and gnashing of teeth, for who among us can truly comprehend external (parsed) general entity declarations and the encoding thereof?

Call for Chapter Proposals: Software Engineering for Science

Wed, 10 Jun 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

With the continuing increase in the importance and prevalence of software developed in support of science, there is a need to gather a set of best practices and case studies to serve as a standard reference book. We are producing a peer-reviewed, edited book to address this need. The book will be composed of chapters related to one of three topics that address the important needs. The book will have three sections, each related to one of those topics. The outline below enumerates those sections along with examples of the types of chapters that would fit within those sections. We solicit proposals from interested authors. Chapter proposals should fit into one of the following book sections.

2015 Post-Workshop Instructor Debriefing, Round 10

Tue, 09 Jun 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The mentorship team ran the 10th round of instructor debriefing session on May 26. Thanks to David Dotson, Daniel Chen, Sahar Rahmani, Christina Koch, David Merand and Laurent Gatto for feedback on their workshops. Rémi Emonet and Fan Yang, instructors at upcoming workshops, also attended.

May 29 - June 9, 2015: New Lesson About Data, SWC at ScipPy 2015, Updating our Lesson Templates, and Amy 0.3.

Tue, 09 Jun 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Highlights

Our latest addition to the Software Carpentry curriculum focuses on using Python to work with data on the web.
We've created an ambitious list of proposed improvements to our lesson templates and aim to have it implemented by September. Please add your comments or help us improve the templates.
Amy 0.3 has been released and include several improvements that will make administrators' lives easier.

Events

One of the aims of running a Software Carpentry workshop at SciPy 2015 is to provide Python novices with the basics ahead of the conference. There are still spaces available for the workshop.

A Remote Workshop at the University of Campinas

Sun, 07 Jun 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Raniere Silva has posted a summary of a workshop at the University of Campinas at which Jennifer Shelton, Maneesha Sane, and Natalie Robinson taught over the web. 23 learners from 3 countries enjoyed lessons and pizza, and we learned a few more things about how to teach remotely.

Amy Version 0.3

Sun, 07 Jun 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Version 0.3 of Amy, the web application we're building to manage workshops, has just been released. Among the improvements are:

many UI upgrades
autocompletion
password and permissions management
an easier way to award badges
and many others

It's already making our admins' lives better, and there are lots more improvements in the works. For details, please see Piotr Banaszkiewicz's recent blog post.

An Update on Publishing Our Lessons

Sun, 07 Jun 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Just a quick update on publishing our lessons: we prepared the release on schedule, uploaded it to Zenodo, and then discovered that we couldn't actually specify editors through their web interface, only authors, even though "editor" is an allowed value in the metadata. We also couldn't specify "lesson" as a type: our choices were "publication" (which is pretty generic) or "presentation" (which isn't quite right). Figshare and COS wouldn't let us do this either, but Zenodo is now working on it. They hope to release it by the end of June, at which point we'll wrap this up.

Teaching at Monsanto

Sun, 07 Jun 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As we discussed in April, the Steering Committee decided to run a small number of workshops for companies this year in order to see how our material would work for them, and whether they'd be willing to pay a higher administration fee that we could then use to underwrite workshops for people who might not otherwise be able to host one. The first of these workshops was held at Monsanto, and seems to have gone well: experience reports from two of the instructors are included below, and we hope to repeat the experiment for other companies in the coming months. (If you know a firm that would like our help, please do introduce us.)

Updating the Lesson Template

Sun, 07 Jun 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A couple of weeks ago, we asked our lesson maintainers what changes they would like to see in our lesson template based on their experiences getting Version 5.3 ready for publication. Their comments are summarized below; it's an ambitious list, but I think we can do most or all of this between now and September. If there are other things you think we should change in the way we structure lessons (rather than in particular lessons themselves), please add comments, and if you'd like to dive into one of these, please checkout the current lesson-template issues and create a pull request against the lesson-template repository.

Software Carpentry at SciPy 2015

Fri, 05 Jun 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This year at SciPy 2015 Software Carpentry is excited to be running a full two-day workshop during the tutorials that precede the main conference. We'll be covering our standard topics of shell, introductory Python, and version control with Git. We'll also be teaching a unit introducing the basic libraries of scientific Python. Attendees need not have any experience with the topics we cover.

We have our usual goals of helping scientists do more in less time with less pain, but at SciPy we also have a bonus goal of easing Python novices into the conference. A technical conference can present a steep learning curve to people unfamiliar with the field and we hope that a Software Carpentry workshop beforehand will give attendees a boost before wading into the conference deluge by exposing them to basic Python concepts and fundamental scientific Python libraries.

There is still plenty of room in the Software Carpentry tutorial, so if this seems useful to you please register with SciPy. And if you know anyone who would benefit from a workshop and would be interested in the conference please share this with them. Hope to see you in Austin!

Teaching Biocomputing at UT

Wed, 03 Jun 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Becca Tarvin has just posted an article about her experiences teaching biocomputing at the University of Texas. Like last month's report on teaching geoscientists, it's a welcome chance to compare what Software Carpentry does with what's possible in more conventional academic settings. As Becca says in summary:

From student surveys we note that students benefit greatly from our online resources, including cheatsheets, markdown lessons, and example code, and they like live-coding more than PowerPoint presentations. Although it's difficult to keep attendance up over the semester, the students that attend/use the online resources greatly improve their coding abilities. We are working to improve the accessibility of our course to all levels of programmers by offering additional resources including Open Coding Hour and by creating an online forum.

Working With Data on the Web

Mon, 01 Jun 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We have just added a new short lesson called Working With Data on the Web to our repertoire. If you would like to help us improve it, please fork it on GitHub and send us comments or pull requests.

Workshop at OU Libraries

Mon, 01 Jun 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

During 19-20 May 2015, I taught a Software Carpentry workshop at the University of Oklahoma (OU) at their fantastic Bizzell Memorial Libary with Jonah Duckles, Logan Cox, and Jeremiah Lant. Jonah and I taught in one room, and Logan and Jeremiah in the other (about 30 students in each room). Tuesday morning was the Shell with Jonah, Tuesday PM and Wednesday AM was Python with me (except the command line section which Jonah covered), and Wednesday PM was git with Jonah. This was my first time teaching the workshop so I decided to write up my observations and student feedback.

May 19 - 28, 2015: New Learner Assessments, Remote Instructor Training, Coding for Librarians, and Evolution of a Geoscience Computing Course.

Thu, 28 May 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Highlights

Software Carpentry instructors are encouraged to help develop new assessments for learners by suggesting questions that should be asked of learners.
A two-day remote instructor course will be trialed across three sites on June 8-9. Watch this space for feedback about the experience.

Resources

Andromeda Yelton has produced a wonderful resource for librarians who code (or want to code) in the form of six articles titled "Coding for Librarians: Learning by Example".
Christian Jacobs, Gerard Gorman, and Lorraine Craig's paper on the evolution of their geoscience computing course and the influence of Software Carpentry is available from arxiv.org.

A Few Articles on Education

Mon, 25 May 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Over the past year, I've come to realize that Software Carpentry will only work if knowledge flows in several directions. Scientists need to learn about software development, but software developers need to learn about science, too. In particular, they need to learn that it's possible to study software and programming scientifically, which is what motivated yesterday's post about my favorite papers from ICSE 2015.

And both groups need to learn about evidence-based teaching practices and the politics that education is embedded in (because without an understanding of the latter, no change is possible). While I don't have a snapshot like ICSE to offer, here are a few recent articles I've found illuminating:

ICSE 2015

Sun, 24 May 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Back when I was still trying to do science myself, my field of study was software engineering. The International Conference on Software Engineering is the big gathering for researchers in that area, and this year's has just wrapped up. Thanks to this Gist from Mike Hoye, I was able to browse the papers presented at ICSE and co-located workshops (like him, I'm outside the Great Paywall of Academia), and I've included titles and abstracts below from the ones I think readers of this blog might enjoy. They're only a fraction of what was presented, and I freely admit the sample is biased toward the things I understand and find interesting, but I hope they'll convince you that people are doing solid empirical studies in software engineering, and producing insights that we can and should act on.

Note: just over half of these papers (13 of 24) had an easily-findable version online. I'm not going to do the experiment, but I confidently predict that those 13 will be more widely read, and more influential, than the other 11.

Coding for Librarians

Fri, 22 May 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Andromeda Yelton (who has featured in this blog before) has written a set of six articles for Library Technology Reports titled Coding for Librarians: Learning by Example. From the introduction:

[This] draws from more than fifty interviews with librarians who have written code in the course of their work. Its goal is to help novice and intermediate programmers understand how programs work, how they can be useful in libraries, and how to learn more.

Three chapters discuss use cases for code in libraries. These include data import, export, and cleanup; expanded reporting capability; and patron-facing services such as improvements to catalog and LibGuide usability. Most of the programs discussed are short&mash;under a hundred lines—so that implementing or modifying them is within the reach of relatively novice programmers. Where possible, links to the code itself are provided. Several scripts are explained in depth.

Additional chapters focus on nontechnical aspects of library code. One chapter outlines political situations that have been faced by librarians who code and the solutions they have employed. Another chapter shares interviewees' advice on specific resources and strategies for learning to code.

Experiences with Geoscientists

Thu, 21 May 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Christian Jacobs, Gerard Gorman, and Lorraine Craig have written a paper titled "Experiences with efficient methodologies for teaching computer programming to geoscientists" that describes how their intro to computing course has changed over the last few years. It includes discussion of ideas they've borrowed from Software Carpentry, and some data on hwo students have responded. It's a good read, and we'd welcome more experience reports of this kind.

Plan to Assess Our Learners

Thu, 21 May 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The assessment subcommittee seeks to assess the effectiveness of the activities of the Software Carpentry Foundation (SCF). It met a few weeks ago and drafted an action plan on how to move forward to create a series of assessments for our learners. This action plan will be the basis on how further assessment tools will be developed.

Online Instructor Training Revisited

Tue, 19 May 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We have now run instructor training in three formats:

an in-person two- or three-day class,
a multi-week online class, and
a hybrid version in which the trainees are co-located, but the trainer comes in via the web.

We've also gathered a lot of feedback on what people want from instructor training and what its prerequisites should be. Based on all of that, we're going to try to combine the best features of everything we've done so far.

May 12 - 18, 2015: 79 New Instructors and Instructor Debriefing Round 9.

Mon, 18 May 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Highlights

79 new instructors have qualified since the last announcement made in March!

Instructor Tips

Changing your prompt in the terminal when teaching the Shell lessons could provide more screen real estate. "export PS1='$ '" will change your current terminal only.

New Members of the Team

Sat, 16 May 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

It's been several months since we last welcomed new instructors to the team. A lot of people have finished training since then, so please say hello to:

2015 Post-Workshop Instructor Debriefing, Round 9

Wed, 13 May 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This week the mentorship team ran the 9th round of instructor debriefing session. Thanks to Andrew MacDonald, Doug Latornell, Evan Morien, Ewan Barr, Isabell Kiral-Kornek, Jackie Milhans, Kara Woo, Karl Broman and Tiffany Timbers for the great feedback of the workshops at Northwestern University, Simon Fraser University, Swinburne University of Technology, University of Melbourne and Washington State University.

May 6 - 11, 2015: Lesson Prep for Publication, Capturing Instructors' Commands, and Instructor Debriefing.

Wed, 13 May 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Highlights

15 people in 5 countries are preparing the Software Carpentry lessons for publication.

Instructor Tips

Do you find that participants fall behind during Software Carpentry workshops because they can't follow commands entered into the terminal? You can use redirect and Dropbox to create a live document for participants to follow.

2015 Post-Workshop Instructor Debriefing, Round 8

Fri, 08 May 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The mentorship team met last week for a discussion with instructors who recently taught, including workshops at the National Center for Atmospheric Research (NCAR) and University of Texas at Arlington (the latter of which was taught by both authors of this post). Three important issues emerged during our discussion: recording the instructor's shell code, using example scripts to model increasing complexity in coding, and preparing instructors/helpers with answers to challenges.

April 28 - May 5, 2015: GSoC Projects, Nightly Rebuilds, Katy Huff, and a PhD Starter Kit.

Tue, 05 May 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Highlights

The Google Summer of Code students have been selected. Well done to Piotr Banaszkiewicz, Ian Henriksen, and Amit Jamadagni!
Nightly rebuilds of the Software Carpentry lessons version 5.3 are now available.

People

Meet Katy Huff - chair of the Steering Committee and a member of the finance subcommittee.

Resources

Achintya Rao wrote a fascinating PhD Starter Kit listing some best practices and useful tools postgraduate students could find useful on their journey towards graduation.

Events

Learn about Research in the Cloud in Feltham this July.

Achintya Rao's PhD Starter Kit

Sat, 02 May 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Achintya Rao started a PhD last January, and in response to a request for advice from a friend, wrote a PhD Starter Kit that lists useful tools and practices. Most of them involve software of one kind or another, and it's interesting to compare the list to what we teach: if nothing else, it tells me that we really do need to figure out what to teach people about publishing science in the 21st Century.

Research in the Cloud in London

Sat, 02 May 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Mark Stillwell and others are running a Research in the Cloud workshop in Feltham (near London) on July 15-17. Along with the standard Software Carpentry curriculum, they'll teach modules on cloud computing, including deployment, configuration, and management of virtual machines. Please see his blog post for details, or the workshop web site to register.

GSoC Projects for 2015

Fri, 01 May 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We're very pleased to announce that three students will be working on Google Summer of Code (GSoC) projects under the NumFOCUS umbrella that we are helping to coordinate.

Getting to Know: Katy Huff

Thu, 30 Apr 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This is the second in a series of posts about our contributors. We're posting these so our community can get to know each other better. If you'd like to be profiled, or you'd like to nominate another member, send an email to communications@lists.software-carpentry.org.

This profile is of Katy Huff, a member of our steering committee, who is also on the finance subcommittee. We already know a little about Katy from her election nomination post; now, read about her history with Software Carpentry and her surprising backup career plan.

April 21 - 27, 2015: The People Behind Software Carpentry, Debating Scientific Software, Learning Objects, and Ally Skills Workshops.

Mon, 27 Apr 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Highlights

Get to know the people behind Software Carpentry. First up: Matt Davis.

Conversations

A lively conversation about scientific software has been taking place on Titus Brown's blog, culminating (for now) in this post on popping the open source/open science bubble.

Recommendations

We'd like to encourage our instructors and lesson contributors to read about the paradox of learning objects as discussed by David Wiley.
Watch the video of the recent Ally Skills workshop offered by the Ada Initiative at PyCon 2015.

Getting to Know: Matt Davis

Mon, 27 Apr 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The subject of the first of our "Getting to Know" series of contributor profiles is Matt Davis, a long-time Software Carpentry team member. Matt is the vice-chair of our Steering Committee, and the Software Carpentry Foundation liason for the Lesson Organization and Development subcommittee.

Ada Initiative's Ally Skills Workshop

Sat, 25 Apr 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Ada Initiative ran their Ally Skills workshop at PyCon 2015, and by all accounts it was useful and thought-provoking. They don't do an online version, but you can watch this video of a workshop at the Wikimedia Foundation. Recommended.

Van Lindberg's Keynote: Say Thanks

Sat, 25 Apr 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Van Lindberg, the chair of the Python Software Foundation, gave a really insightful keynote at PyCon 2015 last week. In a nutshell:

The PSF's greatest challenge is that it's short of time: it has one full-time and three part-time employees.
What can you do to help?
- Say thanks to people for what they're doing.
- Raise up and mentor others.
- Persevere when trying something new. (Bonus points if you help someone out who doesn't look like you.)
How can a new organization grow up to be the PSF?
- Don't rush it. (In particular, don't put too much process in place too soon.)
- Default to openness.
- Build a culture of service.

There's lots more, and you should watch the whole thing, but I think these are good guidelines for Software Carpentry (and most other things, too).

The Paradox of Learning Objects

Wed, 22 Apr 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Warren Code recently forwarded this post by David Wiley, a serial innovator in open education and educational reform. In it, he recapitulates the history of "learning objects" and the paradox at the core of the idea of remixing and reusing teaching material. Since Software Carpentry is (sort of) trying to do exactly that, I think everyone who's currently teaching for us or helping us meet our first publication deadline should look it over.

April 13 - 20, 2015: A DOI for Software Carpentry Lessons, Good Enough Scientific Computing Practices, Code Reviews, and Library Carpentry

Tue, 21 Apr 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Highlights

Publication of our lessons to obtain DOIs is planned for May 14th. Please help us tidy lessons up or get in touch if you have any experience in publishing lessons built in GitHub.

Conversations

Let's write a follow-up to our article "Best Practices in Scientific Computing" called "Good Enough Practices in Scientific Computing".
Should or could Software Carpentry teach participants how to do code reviews? What are your thoughts?

Events

Library Carpentry is a new program piloted by James Baker. The first Library Carpentry event will be hosted in London in November 2015.

Learning in Both Directions

Tue, 21 Apr 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We have spent a lot of time thinking about how to assess the impact that Software Carpentry is having. We've done some small studies and collected a few testimonials, but it's been small potatoes compared to the 5000 people we taught last year alone.

After some back and forth with a colleague whose work I have admired for years, though, I realize that I've been trying to do this the wrong way. My training as an engineer taught me that only controlled, quantitative experiments were "real" science—that as Ernest Rutherford said, it's either physics or stamp collecting. I now understand that there are other rigorous ways to generate actionable insights, some of which are better suited to our needs than something like randomized control trials. More than that, I finally understand what one of my first teachers told me:

Teaching only works well when the teacher is also learning.

AAS Reflections

Sat, 18 Apr 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We just finished^* a workshop at the American Astronomical Society. I was lucky to recruit 3 instructors (in addition to myself) - Matt Davis, Erik Bray, and Phil Rosenfield. We also had Pauline Barmby volunteer as a helper (and on the fly instructor). While I wrote this blog post, you will see comments inserted by the instructors.

Close Enough Redux

Fri, 17 Apr 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Back in October, we explained why we don't teach testing in Software Carpentry workshops. In response, Ian Hawke has put together a really nice series of articles about how he would test a small numerical program. It's great content, and it also shows yet again how Jupyter (formerly the IPython Notebook) is changing the way scientists create and share ideas.

Invitation to Millions of Compute Hours: Announcing the Open Science Grid User School

Fri, 17 Apr 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

If you could access thousands or even millions of hours of computing, how would it transform your research? What discoveries might you make?

Each year the NSF-funded Open Science Grid (OSG) selects a group of 25-30 students to attend the OSG User School, a week-long dive into high-throughput computing approaches, technologies, and skills, within a larger context of computational research design that students can take into their future careers as researchers. Students across the country from nearly any research discipline are invited to apply, and selected applicants will obtain direct access to the OSG beyond the duration of the school.

Library Carpentry

Fri, 17 Apr 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We wrote about the digital skills classes at the British Library last October. We were therefore very pleased to see that James Baker, a Software Sustainability Institute Fellow, is piloting a new program called Library Carpentry. The first run will take place in November 2015 at the Centre for Information Science at City University London; the program will consist of four three-hour sessions, each for 40-50 participants. The announcement has more details, including a call for participants and another for volunteers. Please check them out, and lend a hand if you can.

Publishing Our Lessons

Fri, 17 Apr 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Digital Object Identifiers (DOIs) are one of the building blocks of academic bibliography systems. It's now possible to get a DOI for a GitHub repository (or more accurately, for the state of a GitHub repository at a particular point in time). We are going to use this to publish a citable version of our core lessons.

Korean Translation of Software Carpentry - version 5.2

Thu, 16 Apr 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Based on Software Carpentry version 5.2, we translated all the lessons in the form of pdf, html, mobi, epub, and azw3. It took almost 6 months: the first half spent on mostly lessons, and the second half spent on remainings. While translating and building various ebook formats, the original ebook production software of Software Carpentry seemed not to have upfront architectural concerns in terms of Chinese, Japanese, and Korean (aka, CJK)characters problems. Also, we are wondering how to generate various ebook formats from the current developing lessons.

Quality Is Free - Getting There Isn't

Wed, 15 Apr 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Worried about the rising tide of retractions, Nature Biotechnology recently announced that, "Its peer reviewers will now be asked to assess the availability of documentation and algorithms used in computational analyses, not just the description of the work. The journal is also exploring whether peer reviewers can test complex code..." That's a welcome step in theory, but I worry about how it will play out in practice. Scientists already complain about how much time they spend reviewing papers: reviewing code as well will take even more time, particularly if:

2015 Post-Workshop Instructor Debriefing, Round 7

Tue, 14 Apr 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The mentorship team held our latest round of post-workshop debriefing sessions for instructors who taught recently. Instructors from workshops at Clemson University, University of Miami, University of Melbourne, University of Oklahoma Libraries, Harvard School of Public Health, and Weill Cornell Medical College joined us for our discussions, as well as a few new instructors who will be teaching in coming weeks. Here's a recap of common themes and highlights:

April 6 - 13, 2015: The Steering Committee, Workshops for Companies, and a Discussion of WiSE Events

Mon, 13 Apr 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Highlights

A summary of the steering committee's activities has been provided by Karin Lagesen. Her post includes links to board meeting minutes and an overview of the newly-formed subcommittees.
Software Carpentry will be running five workshops for companies. Please get in touch if you are aware of companies that may be interested to participate in this pilot program.

Conversations

What are your feelings about women-only events to address the gender gap in STEM? Read about the experiences and observations from instructors during a recent WiSE workshop.

Good Enough Practices in Scientific Computing

Mon, 13 Apr 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

April Wright recently wrote a blog post about the reproducibility of a paper she recently submitted. In it, she said:

How to Send a Pull Request to the Lesson Template

Mon, 13 Apr 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

At the end of last year we split out lessons to have one Git repository per topic that we teach. To ensure that all lessons have the same look we use one template for our lessons. If you want to create a new lesson from scratch we already have a nice step-by-step guide but what about if you want to contribute to the template itself? In this post I will explain how I wrote my pull request for our template.

The Future Then and Now

Mon, 13 Apr 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Jon Udell's Internet Groupware for Scientific Collaboration taught me how to think about the web. He started work on an update a couple of months ago, and it has now been published by PLOS. What strikes me upon re-reading the first is how far we've come; upon reading the second, is how far we still have to go to make all of this normal. As William Gibson said, "The future is already here—it's just not very evenly distributed."

Reflections Following a Women in Science Workshop

Sun, 12 Apr 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The number of women pursuing careers in STEM fields has increased over the last few decades, yet, there are still major gender gaps in the areas of mathematics, computer science, engineering and physics (Hill, Corbettt and St Rose, 2010). In the US, women make up close to 50% of the work force yet the percentage of STEM jobs occupied by women is only 25% (Beede, Julian, Langdon, McKittrick, Khan, and Doms, 2011). Part of this gap may be linked to biases and stereotypes that suggest "boys are better than girls" at math. These stereotypes can affect a girl's performance on tests and her perceived skill at math, ultimately influencing her decision to pursue a career and education in fields that require a foundation in mathematics and computing (Shapiro and Williams, 2011; Gunderson, Ramirez, Levine, and Beilock, 2012).

How does this relate to Software Carpentry? Software Carpentry's mission is "to teach researchers basic lab skills for scientific computing." Given that female participants may experience an added anxiety related to gender biases when learning computing skills, Software Carpentry instructors should be aware of these stereotypes and how they might affect their learners. Further, workshops with at least one female instructor can counteract these stereotypes by showing that women are skillful in computing tasks.

But how much intervention is necessary and could extra support for women actually be a detriment? Following a Data and Software Carpentry workshop for Women in Science at the University of British Columbia, we have thought about the advantages and disadvantages of holding a workshop for women only. We have posed the following question:

"Is a women-only Software Carpentry workshop helpful or harmful for increasing the number of women in STEM?" A few points in each category are given below.

The Steering Committee has Landed!

Sun, 12 Apr 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Steering Committe has now had its first few meeetings. Minutes from them are up, please go have a look if you are wondering what we are up to!

A Project Inception Deck for Research Coding

Sat, 11 Apr 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I've never seen the point of comparing programmers to ninjas or samurai, but the people who do so often have good ideas. One that I particularly like is the Agile Inception Deck from Jonathan Rasmusson's The Agile Samurai, which sets out a ten-step process for making sure that everyone involved in a new project is actually trying to build the same thing. The ten steps are:

Workshops for Companies

Thu, 09 Apr 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

At our March 12, 2015 meeting, the Software Carpentry Foundation Steering Committee discussed whether and under what terms to provide workshops to for-profit corporations. There were no objections to the idea of doing workshops for corporations, but it is something new to Software Carpentry. We've decided to run a pilot program encompassing five corporate workshops so we can learn more about working with corporations. The pilot program will allow us to gauge corporate interest and collect feedback from instructors, coordinators, and others about how things go.

March 31 - April 6, 2015: A Lab Meeting, a LinkedIn Group, Two New Capstones, and Ideas for Instructors

Mon, 06 Apr 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Highlights

A summary of last week's lab meeting is now available. Please take a moment to read through the highlights and notes.
Software Carpentry and Data Carpentry instructors are invited to join our newly-created LinkedIn group.

Resources

Do you need to create a brand new lesson repository? The new process is outlined with an example - please try it out and send us your comments.
Damien Irving created a capstone example specifically for oceanographers.
A biomedical engineering MATLAB capstone was made available by Isa Kiko.

Technical Challenges

Git on Mac OS X 10.8: Instructors can find or propose solutions to the "Lazy symbol" error when installing git on old Macs (Mac OS X 10.8)

April 2015 Lab Meeting

Fri, 03 Apr 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We held our second lab meeting of 2015 on April 1st, and had near-record turnout. Notes from the Etherpad are included below; the highlights are:

We hope to have a first-quarter financial report by the end of this month. The good news is that there's lots of interest in partnerships and affiliations; the bad is that we're not collecting admin fees from nearly as many workshops as we need to.
We will charge for-profit organizations four times as much for organizing workshops as we charge universities and other non-profits; the extra money will be used to underwrite workshops for places that otherwise might not be able to afford to host them.
We have created a LinkedIn group for Software Carpentry and Data Carpentry instructors — if you're a LinkedIn user, you're welcome to join. Leigh Sheneman and Lynne Williams have volunteered to moderate the group.
Peter van Heusden and Gabriel Devenyi have volunteered to help with system administration — our thanks to Jon Pipitone and David Rio for all their help over the past couple of years.
Noam Ross is putting together a lesson on how to get unstuck, and Christina Koch is managing work on some extra Unix shell material.
We hope to have the current lessons tidied up by the end of April so that we can give them DOIs, and thereby make it easier for everyone who has contributed to them to get proper credit for their work.

2015 Post-Workshop Instructor Debriefing, Round 6

Wed, 01 Apr 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We held our sixth round of post-workshop debriefing last week. We discussed the Software Carpentry workshops held at the University of Arkansas, Utah State University, the University of Waterloo and the first Software Carpentry workshop ever held in Korea at the Korea Radio Promotion Association (and yes we now have lesson translated in Korean)! We were also joined by a Data Carpentry instructor who taught in a recent workshop at Espoo, Finland.

March 23-30, 2015: A Lab Meeting, a Dataset, NGS Course at MSU, and Postdoc Positions at BIDS

Mon, 30 Mar 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Highlights

Please remember to join us for the next online lab meeting on April 1st at 10:00 and 19:00 Eastern time.

Resources

If you are looking for a teaching-ready dataset, take a look at Ethan Whites simplified version of the Portal Projects Database now availble in csv, json, and sqlite.

Events

A 2+1 week NGS course will be offered by Titus Brown and others from 10 - 21 August. Applications are now open.
EuroSciPy 2015 call for papers is open until 20 April. The conference will take place in Cambridge, UK on 26-30 August.

Opportunities

Berkley Institute for Data Science (BIDS) is inviting applications for Postdoctoral Researchers in Data Science. Apply before 20 April.

Teaching in Yangon

Wed, 25 Mar 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

On Sat 7 March I spent a full day teaching a Software Carpentry workshop at the University of Yangon with 23 archaeologists from the Department of Archaeology. The workshop is part of a training component of an archaeological research project funded by the Australian Research Council, the University of Washington and the University of Wollongong. The group included graduate students, tutors and lecturers. Archaeology in Myanmar has a strong art history flavour, partly due to its British colonial heritage (where archaeology and art history are often paired, compared to the where US archaeology is usually a sub-field of anthropology) but mostly due to the country's extreme isolation from the rest of the world, where archaeology has taken a scientific turn in recent decades. This isolation takes several forms: travel restrictions that make it difficult for locals to travel overseas, and until recently, for foreigners to visit; small library budgets that make it difficult for university libraries to keep their collections and subscriptions current; and slow and unreliable internet connectivity make browsing the web, watching videos, and downloading files a lengthy, uncertain and frustrating process. All of this meant that the group's familiarity with using computers for research was lower than what might be expected from a Western audience, and so we adapted the SWC materials to accommodate this. We knew we wouldn't get through as much as a typical workshop, but we had the advantage of everyone starting at an equivalent skill level, so the sticky notes all went up and down at much the same time and we had a pleasant and relaxed atmosphere.

Weekly Update: March 16 - March 22, 2015

Sun, 22 Mar 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Highlights

The next online lab meeting will take place on 1 April at 10:00 and 19:00 Eastern time. Remember to sign up.

Resources

Jenny Bryan contributed a Gapminder data package now available through CRAN which might be helpful in setting up for Software Carpentry workshops.
Data Carpentry developed new material for teaching dplyr using their ecological dataset.

Opportunities

Students can apply to participate in our Google Summer of Code projects until 19:00 UTC on 27 March 2015. We have several exciting projects available.
Insight Data Science launched a new seven week fellows program in Health Data Science.

April 2015 Lab Meeting

Fri, 20 Mar 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The next Software Carpentry online lab meeting will take place on Wednesday, April 1 (no, really) at 10:00 and 19:00 Eastern time. (As usual, we will hold the meeting twice to accommodate people in different time zones.) Please sign up on this Etherpad to let us know whether you'll be attending, and if so, and what time. We'll post an agenda next week; if there's anything you'd particularly like to discuss, please let us know.

And Now We Are Three

Tue, 17 Mar 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The four core topics that every Software Carpentry workshop is supposed to teach are automating tasks using the Unix shell, structured programming in Python, R, or MATLAB, version control using Git or Mercurial, and data management using SQL. In practice, many workshops omit the fourth, either because instructors want to put more time into the first three, or because they don't think SQL is relevant to their learners.

The Steering Committee has therefore voted to take SQL out of the core. This doesn't mean that it can't or shouldn't be taught: it's still useful for many researchers to know, and the best way we've found to introduce key ideas in data management like atomic values, keys, and how to handle missing information. However, if instructors and learners would rather cover something else, they can do so.

Weekly Update: March 7 - March 15, 2015

Tue, 17 Mar 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Conversations

Get some fantastic teaching tips from novice and seasoned trainers. Join the conversation by adding your tips.
Useful advice is given about what to pack when you teach. Your contributions can potentially help fellow trainers.
The Steering Committee has voted to remove SQL from the list of core topics for workshops. Instructors should still teach it if they think it's right for their audience, but they may also now use that time for more programming, testing, or other topics.

Workshop at iPlant

Tue, 17 Mar 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The iPlant Software Carpentry Workshop in February at the University of Arizona in Tucson was an awesome realization of iPlant's and BIO5's collaborative nature. Bringing together iPlant, BIO5, the UA and Software Carpentry, this workshop served a large group of students and staff from a wide variety of backgrounds and a wide array of interests. 53 participants registered within 36 hours of publicizing the workshop. Participant demographics were as follows:

Teaching Tips

Sun, 15 Mar 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Last week's post on what's in your bag generated so many useful comments that we'd like to follow it up with another: what tips do you have for new instructors? The ones we've collected so far are listed below; please tell us what else we should tell people who are about to teach for the first time (and what else we should remind experienced instructors about).

What Do People Want to Learn?

Sun, 15 Mar 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

In the planning phase of organizing a Software Carpentry workshop for my home department of Molecular Biology & Biochemistry I started to wonder what participants want to learn. I designed a short survey to answer this question, and from my small department, ~20% (30 people) filled it in. Here's what they said:

2015 Post-workshop Instructor Debriefing, Round 5

Fri, 13 Mar 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

At our fifth round of post-workshop debriefing this week, we discussed workshops held at the New York Academy of Sciences, the University of Oslo, and the University of British Columbia. This was a very instructive meeting with important lessons learned from the perspective of both new and veteran instructors. One of the key take-home lessons is that new instructors would benefit from attending an instructor debriefing prior to doing their first workshop.

What's In Your Bag?

Wed, 11 Mar 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

What do you have in your knapsack when you travel to teach a workshop? My list is:

Weekly Update: Feb 28 - March 6, 2015

Mon, 09 Mar 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Highlights:	Do you want to help shape the future of Software Carpentry? The Software Carpentry Foundation is now calling for volunteers to serve on various standing committees.
	NumFOCUS has been selected as a Google Summer of Code mentoring organisation. Students can apply between 16 and 27 March 2015.
	A Contributor Covenant was added to our lessons and other repositories to promote a harassment-free environment for contribution to Software Carpentry.
Resources:	An early release version of Bioinformatics Data Skills is available via O'Reilly. Vince Buffalo's book is highly recommended for both novice and experienced bioinformaticians.
Opportunities:	Scientific Software Engineer positions are available at the UK Met Office
Changes:	Daisie Huang will be taking over from Jess Hamrick to maintain our Git lessons with Ivan Gonzalez. Our thanks to Jess for all her help.

Shape The Future of Software Carpentry in an SCF Standing Subcommittee

Thu, 05 Mar 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Software Carpentry Foundation Steering Committee was elected to pursue ambitious goals driven by the voices and expertise in our community. We are therefore thrilled to announce a set of community-driven subcommittees aimed at initiatives essential to the Software Carpentry Foundation.

Today, we are asking for representatives from the community to support this effort and shape the future of Software Carpentry. Please step forward to volunteer for the standing committee most important to you. Committee members will be asked to dedicate a few hours per month. To join us, please email the SCF Steering Committee with your name and a few sentences about your interest.

Funding Software Carpentry Workshops

Wed, 04 Mar 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Software Carpentry workshops are taught by volunteers, but hosts need to fund instructor travel, accommodation as well as our administrative fees of $750-$1250. While these costs are low, it can sometimes be a challenge to get funding for your first workshop, especially if others are unfamiliar with SWC. Here are some successful strategies that hosts have used to fund workshops:

Workshop in Krakow

Wed, 04 Mar 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Last weekend we (Paulina Lach and Piotr Banaszkiewicz, both from AGH; Klemens Noga from Cyfronet and myself from Cztery Bity organized a workshop in Kraków. We decided to do itourselves, as a group of certified Software Carpentry instructors, which was simply a faster and more effective solution than finding an institution which would act as a host. However, we received significant support from a few organisations. The PhD Students' Association at the Jagiellonian University offerred the venue. Thanks to the support provided by ACC Cyfronet AGH and Cztery Bity we were able to offer small catering. We were also supported by great helpers: Iwona Grelowska, Jakub Kruczek and Tomasz Jonak, all from AGH and Marcin Klimek, from Cztery Bity.

NumFOCUS Accepted as Google Summer of Code Mentoring Organization

Tue, 03 Mar 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A few weeks ago we announced that we would help NumFOCUS apply to Google Summer of Code (GSoC) as a mentoring organization. Yesterday the list of mentoring organization was announced and NumFOCUS had been selected.

The Most Viewed PLOS Biology Paper of 2014

Tue, 03 Mar 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We were very pleased to learn that the most viewed article in PLOS Biology in 2014 was "Best Practices for Scientific Computing", written by Dhavide Aruliah, C. Titus Brown, Neil Chue Hong, Matt Davis, Tommy Guy, Steven Haddock, Katy Huff, Ian Mitchell, Mark Plumbley, Ben Waugh, Ethan White, Paul Wilson, and yours truly. According to Stavroula Kousta, the article has been viewed more than 66,000 times.

Adding a Contributor Covenant

Mon, 02 Mar 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Software Carpentry Foundation's Steering Committee has voted to add a Contributor Covenant to our lessons and other repositories. Like the Code of Conduct for our workshops, the Contributor Covenant's aim is to ensure that participation in Software Carpentry is a harassment-free experience for everyone, regardless of level of experience, gender, gender identity and expression, sexual orientation, disability, personal appearance, body size, race, ethnicity, age, or religion.

Ten More Instructors

Mon, 02 Mar 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

It's a pleasure to welcome another ten instructors from the Southern Hemisphere to our team:

The First Software Carpentry in Korea

Mon, 02 Mar 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Last week, xwMOOC xwMOOC ran the first Software Carpentry workshop in Seoul. The workshop delivered not only an introduction to Unix Shell, Git and GitHub, Python and SQL but also computer science unplugged, rur-ple, python for informatics and cloud basics. Listening to attendees feedback, attendees concerns such as software business, all about what start-ups should know, how software business environment looks like and so on are provided during three days workshops.

Weekly Update: Feb 21 - Feb 27, 2015

Sun, 01 Mar 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Conversations:	How do you fund your SWC workshops? Please join the conversation.
	Should we remove old material when adding new in order to keep the lessons manageable? Should people have to teach the standard material a couple of times before introducing their own? Please contribute your ideas.
Resources:	Remember to sign up for Software Carpentry's mailing lists. They are rich resources for scientific computing.
	Great R cheat sheets available at RStudio include sheets for dplyr, tidyr, Shiny, and R Markdown.
	A preview release of RStudio v0.99.315 is now available for testing and feedback. New features include for example the removal of the 1K row limit in Data Viewer.
	IPython 3.0 has been released - congratulations to the whole team.
	So you think you can data? The rOpenSci Hackathon is going to host a friendly data challenge between Hadley Wickham and Wes McKinney - please follow this GitHub issue for news.

Quotable:	"This is @swcarpentry: try to teach enough in two days that things people find on the internet make a little more sense." @jiffyclub

Eleven New Instructors

Sat, 28 Feb 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

It's a pleasure to welcome another eleven instructors to our team:

2015 Post-workshop Instructor Debriefing, Round 4

Fri, 27 Feb 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We had our fourth post-workshop debriefing of the year this week, in which we discussed recent workshops at Jagiellonian University in Krakow, Poland, and Michigan State University. All four instructors from the Krakow workshop attended the debriefing and we had a fairly in-depth discussion about that workshop

Improving Instruction

Fri, 27 Feb 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

It's been quite a year for Software Carpentry instructor training: we had a great session at UC Davis (see also this excellent post from Rayna Harris) and another at the Research Bazaar kickoff in Melbourne. We've also started the twelfth round of online instructor training with 75 participants, making it our largest yet.

All of this has led to a flurry of activity in our GitHub repositories. The comments and pull requests are very welcome, but we need to keep three things in mind:

We already have more material than we can cover in two days.
We are not our learners.
No lesson survives its first presentation intact.

Wrong Is Useful: Lessons as Packages

Fri, 27 Feb 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

"What would Greg do? [pause] OK, now that we've ruled that out..."
— overheard

I wrote a post last July about using package managers like RPM, Homebrew, and Conda to track dependencies between lessons, so that a student could say something like conda install unit_testing and get a lesson on unit testing, along with the code, sample data, and other lessons it depends on. I also mused that it could help make research more reproducible: after all, a paper is just a lesson on something that's never been taught before.

Weekly Update: Feb 14 - Feb 20, 2015

Sun, 22 Feb 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Highlights:	We're looking for mentors and projects for Google Summer of Code: please get in touch if you have questions or suggestions.
	Research Bazaar was again a highlight on Twitter this week, and Damien Irving summarised plans for Software Carpentry in Australia and New Zealand.
Conversations:	An interesting conversation about "Library Carpentry" started by James Baker took place on Twitter.
New resources:	A new Browsercast tutorial "Creating a workshop website" was published. Please let us know if you have any comments.
	An early release of "Effective Computation in Physics" has been made available. The book is relevant to scientists of all kinds
Opportunities:	SciPy call for participation is open. Please submit your proposal for tutorials, talks, or posters.
	Call for Data Science Fellow applications through the Berkley Institute for Data Science is now open.

Applying to Google Summer of Code

Sat, 21 Feb 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

During last week's Steering Committee meeting we talked about applying to Google Summer of Code as a mentoring organization and decided to contact NumFOCUS to suggest that they applying as a umbrella organization for all the projects they support. NumFOCUS agreed, and for the past few days we have been racing to fill in the application.

Software Carpentry Set to Explode in Australia and New Zealand

Sat, 21 Feb 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Given the global success of Software Carpentry (10,000 learners and counting), it's easy to forget that the first two-day workshops were held only three short years ago. Excited by the potential of the project in those early days, Josh Madin (Macquarie University) and myself (University of Melbourne) pooled our funds and got Greg Wilson out to Australia to run the first ever workshops outside of Europe and North America. Since those initial Sydney and Melbourne workshops in February 2013, an additional 15 have been held around Australia and New Zealand and there are a dozen or so active local instructors.

Managing GitHub Notifications

Fri, 20 Feb 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

SWC has many GitHub repositories for lessons, websites, and workshops, and we have many conversations about our work take place through GitHub issue threads. Since GitHub creates e-mail notifications of repositories and issues you participate in, that means you can quickly start getting loads of e-mail.

While it's helpful to be kept abreast of developments on a project, if you get a flood of GitHub messages you won't see what's important. Here's a quick primer on fine-tuning your notifications so that you only see what you want, gleaned from this helpful conversation:

Workshop at the University at Albany, SUNY

Mon, 16 Feb 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We recently ran a workshop hosted by the University at Albany, SUNY's Department of Informatics. The workshop ran on the weekend of January 31 and February 1, 2015 with the following schedule:

Day 1 AM:	Automating tasks with the Unix shell (instructor: Thomas Guignard)
Day 1 PM:	Building programs with Python (instructor: Jeramia Ory)
Day 2 AM:	Version Control with Git (Jeramia)
Day 2 PM:	Managing data with SQL (Thomas)

Weekly Update: Feb 07 - Feb 13, 2015

Fri, 13 Feb 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Highlights:	NeSI (New Zealand) became a SWF affiliate.
	SWC instructor training at the Research Bazaar in Australia created a lot of activity on Twitter this week - 50 trainees attended the workshop.
Conversations:	Daniel Chen made his "Project Cookie Cutter" bash script available and triggered a conversation about this. Please get in touch with us via GitHub or email us if you use cookie cutter project templates or have comments.
Vacancies:	The National Ecological Observatory Network (NEON) is looking for a Science Educator/Evaluator in Boulder, Colorado.

NeSI Becomes Software Carpentry Affiliate

Thu, 12 Feb 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are very pleased to announce that New Zealand eScience Infrastructure (NeSI) has become an affiliate member of the Software Carpentry Foundation.

Online Scientific Collaboration: The Sequel

Thu, 12 Feb 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Jon Udell's Internet Groupware for Scientific Collaboration taught me how to think about the web. He's now revisiting that report, and would like our help. Details are below; please give him a shout if you can help.

Science Educator/Evaluator Position at NEON

Thu, 12 Feb 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The National Ecological Observatory Network (NEON) is looking for a Science Educator/Evaluator (SEE). This person will coordinate development and implementation of NEON's Education and Public Engagement (EDU) program/product evaluation plans. The program includes web-based educational resources, university student programs, and citizen science programs, as well as learning resources for university students and faculty. The SEE will coordinate the development of measures for assessing implementation and outcomes of existing programs and new programs as they are developed. The SEE will also work with scientists/science educators to develop learning modules that are pedagogically appropriate for undergraduate students and assessment tools that enable evaluation of the effectiveness of these learning modules. An important part of this effort will be coordination of collection and analysis of data to assess NEON EDU program impact on a variety of audiences (including university students and faculty, scientists, the general public, educators (formal and informal), and other program stakeholders). The SEE will work closely with NEON EDU management and with contractors hired to provide external evaluation of EDU program goals. The SEE will work out of NEON HQ offices in Boulder, Colorado.

For more information, please see the full advertisement.

2015 Post-workshop Instructor Debriefing, Round 3

Wed, 11 Feb 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We had our third post-workshop debriefing of the year yesterday, in which we discussed several recent workshops. The most important point was probably that workshops are running more smoothly today than they did a year ago, even for first-time instructors.

Cookie Cutter

Tue, 10 Feb 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I was first introduced to William Stafford Noble's paper "A Quick Guide to Organizing Computational Biology Projects" when Ivan Gonzalez and I taught Harvard last November. Noble describes how scientists in Computational Biology should set up their project folders so code, results, outputs, figures, and papers are all in easily understandable locations. He also writes about how one should run experiments (using driver scripts) to make workflows reproducible, readable, and understandable to others (and your future self).

Plot This

Mon, 09 Feb 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The most useful little guide to visualization I've ever found is the decision tree created by Andrew Abela, which you can find here. Do you want to show a comparison, a distribution, a relationship, or a composition? If it's a comparison, is it among items or over time? Each choice leads to a different kind of plot, and while you may not agree with all the choices, it makes the reasoning behind them concrete.

Weekly Update: Jan 31 - Feb 06, 2015

Mon, 09 Feb 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Highlights:	SWF gets 2 new affiliates (iPlant and Lab for Data Intensive Biology at UC Davis) and a new partner (University of Washington).
Conversations:	"...what matters most in teaching is the act itself - the verb": some observations on teaching and a book recommendation.
	Scientific coding and software engineering: what's the difference? A post by Daisie Huang: please add your comments.
	Rewarding Software Sharing by Mapping Scientific Software A post by Chris Bogart: please share your ideas.
Events:	SWC instructor training and Intro to Web Programming at Lawrence Berkeley Lab: March 2015
	rOpenSci Unconference: March 2015
	Software Sustainability Institute's Collaborations Workshop: March 2015
Other News:	Eight new instructors

Scientific Coding vs. Software Engineering

Sun, 08 Feb 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Daisie Huang recently wrote a great article for the Software Sustainability Institute's blog titled "Scientific coding and software engineering: what's the difference?". As a professional programmer who has become a scientist, rather than a scientist who's learned how to program, she has a fresh take on the differences between what those two groups do. Comments on her post would be very welcome.

Rewarding Software Sharing by Mapping Scientific Software

Sat, 07 Feb 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Sharing software you write with other scientists can magnify the impact of your research, but there can be a surprising amount of sometimes thankless extra work involved. I work with a group at Carnegie Mellon's Institute for Software Research who have been asking scientists what that extra work is, and what motivates them to do it—despite a sometimes uncertain link between that extra work and the ways many of them are evaluated in their jobs. We're looking at ways of measuring and mapping software and its impacts, in order to help scientists demonstrate the positive impact that their work on shared software has on science. We're running an experiment that you can help with.

2015 Post-workshop Instructor Debriefing, Round 2

Thu, 05 Feb 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Greg Wilson and I recently hosted a second post-workshop debriefing session for January to capture more experiences and lessons learned from instructors in the field. This meeting was attended by 10 instructors covering four recent workshops. We discussed how the workshops went, what worked, what didn't, and what could be improved.

Software Sustainability Institute's Collaborations Workshop 2015

Tue, 03 Feb 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Software Sustainability Institute's Collaborations Workshop 2015 (CW15), which will be held on March 25-27, 2015 in Oxford, UK, focusses on software, best practice and the social side of working past the boundaries of traditional disciplines and roles to accelerate research outcomes—or put differently, interdisciplinarity done right!

What will you learn by attending:

The social and technical sides of interdisciplinary working.
Examples of best practice of when it's been effective - and horror stories of when it has not.
Examples of techniques and technologies that could be applied from one discipline to another - discipline hopping.

University of Washington Becomes Software Carpentry Partner

Tue, 03 Feb 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are very pleased to announce that the eScience Institute at the University of Washington has become a partner of the Software Carpentry Foundation.

Welcome Our Newest Instructors

Tue, 03 Feb 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

It's a pleasure to welcome another eight instructors to our team:

Workshop in Illinois

Tue, 03 Feb 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Last week (29-30 January), Matthew Turk (NCSA), Ivan Gonzalez (Martinos Center), David LeBauer (UIUC, and a new SWC instructor), and I (Neal Davis, CSE, UIUC) taught a SWC workshop co-sponsored by NCSA and CSE. We introduced two innovations into the workshop model different what that used in prior events I've participated in:

iPlant Becomes Software Carpentry Affiliate

Mon, 02 Feb 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are very pleased to announce that the iPlant Collaborative has become an Affiliate Member of the Software Carpentry Foundation.

Lab for Data Intensive Biology at UC Davis Joins Software Carpentry as an Affiliate

Mon, 02 Feb 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are pleased to announce that the Laboratory for Data Intensive Biology at UC Davis has joined the Software Carpentry Foundation as an Affiliate Member for three years starting in 2015. "We've been long-term supporters of Software Carpentry, and Affiliate status lets us support the Software Carpentry Foundation in a tangible way," said Dr. C. Titus Brown, the lab director. "This status also gives us the opportunity to include Software Carpentry as part of a larger biological data science training program at UC Davis."

Nouns and Verbs

Mon, 02 Feb 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I've spoken and written many times about how puzzled I am that massive, open collaboration on lessons is so rare in the age of Wikipedia and open source software development. Hundreds of people have helped build the Wikipedia articles on Marvel Comics and the planet Mars, and hundreds more have helped build things like the Django web programming framework; why then are teachers still writing all their own slides and handouts?

As is frequently the case, the answer might be that I've been asking the wrong question. When I brought this up last week in an online call organized by the Open Knowledge Foundation, Phil Barker said:

"One difference between Wikipedia and OER [open educational resources] is that editing Wikipedia is about moving to consensus on facts, [but] editing teaching material is about tailoring it to specific local requirements (different teaching styles, different students, different curriculum standards)."

Our First Workshop in South Korea

Mon, 02 Feb 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are very pleased to announce that our first workshop in South Korea will be taking place in Seoul later this month. Many thanks to Kwangchun (Victor) Lee for organizing and teaching, and to the Korea Radio Promotion Association for hosting.

rOpenSci Unconference in March 2015

Mon, 02 Feb 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The good folks at rOpenSci have just announced that their second annual unconference (which will actually be more like an unhackathon) is happening at GitHub's headquarters in San Francisco in March 2015. They're also hosting a Data Science Social on March 26 - please see their site for details.

Weekly Update: Jan 24-30, 2015

Mon, 02 Feb 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Highlights:	New steering committee selected
	We have now taught ten thousand people
Conversations:	Keeping momentum going after workshops
	Factors and formulae in R lessons
Other News:	First new instructors of 2015
	Our first workshop in South Korea

Workshops in March at Lawrence Berkeley Lab

Mon, 02 Feb 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I will be teaching a two-day Software Carpentry instructor training course on March 10-11, 2015 at Lawrence Berkeley Lab, and a one-day course on web programming on March 13, 2015. Details are given below; the instructor training is open to LBL staff and to graduate students and staff associated with the Berkeley Institute for Data Science, while the web programming class is reserved for LBL staff alone.

Announcing 2015 Steering Committee

Sat, 31 Jan 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The election for the Software Carpentry Foundation's Steering Committee is now complete. 122 ballots were exercised out of 179 mailed out, and our new committee is:

Interim Steering Committee Meeting: Dec 16, 2014

Fri, 30 Jan 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Software Carpentry Foundation Interim Board Meeting: Dec 16, 2014

Data Carpentry Genomics and Assessment Hackathon

Wed, 28 Jan 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

If you're working or interested in genomics or assessment, we hope you'll consider applying for our upcoming Data Carpentry Genomics and Assessment hackathon. We're very excited about this event and the opportunity to develop lessons targeting genomics researchers and build assessment into the Data Carpentry curriculum. Travel support is available, so please apply to participate! It's a short application, and the deadline is this Friday, January 30th. If you have any questions about the event, please let us know.

Dates: March 23-25, 2015
Location: Cold Spring Harbor Labs, NY
Call for Participation
Application

Cast Your Vote

Mon, 26 Jan 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Voting is now open for Software Carpentry's new Steering Committee. If you are a member, you should have received a ballot by email from elections@electionbuddy.com. If you did not, please check your spam folder; if it is not there, please get in touch and we will sort it out as quickly as we can.

The Other Ninety Percent

Sat, 24 Jan 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Ninety percent or more of learning a skill takes place outside formal lessons as people try things out for themselves and turn attention into habit. This works best if a mentor is on hand to answer questions and provide feedback, but our workshop format doesn't lend itself to that: in most cases, our instructors are back on a plane (or back in their own lab) as soon as they're done teaching, so our learners have to make sense of what they've just been shown on their own. Based on follow-up discussions and Jory Schossau's work, here are things they stumble over in order of increasing pain:

Welcome Our First New Instructors of 2015

Sat, 24 Jan 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

One of the best parts of this job is welcoming new instructors to our team. Many of this year's first group have already helped with workshops, and I hope to have a chance to teach with all of them before too long.

Call for ELIXIR Node Coordinators

Thu, 22 Jan 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

One of the 2015 ELIXIR Pilot Projects focuses on supporting training in collaboration with the Data and Software Carpentry initiatives. The pilot "Working up and building the foundation for Data Carpentry and Software Carpentry within ELIXIR" includes a series of initial events across different ELIXIR Nodes. In order to achieve the goal of running Data and Software Carpentry (DC/SWC) workshops, promoting the DC/SWC teaching model, developing and sharing of the training materials, it would be useful if not essential, that each ELIXIR Node has a DC/SWC Coordinator.

Improving the Balance

Thu, 22 Jan 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Jennifer Martin's recent article "Ten Simple Rules to Achieve Conference Speaker Gender Balance" reminded me that while we've been reporting the number of workshops we've run, and the number of people who've attended, we haven't reported the gender balance among our instructors in a while. I did a quick check, and the result was sobering. From a high of almost 30% eighteen months ago, our instructor pool is now only 18% female. It seems that as we grow, we are regressing to computing's unfortunate mean: from a higher of 38% in the early 1980s, the proportion of computer science degrees awarded to women has dropped to 15-18%, and CS is the only STEM discipline where the gender balance has actually been getting worse over the past few decades.

University College London Becomes Software Carpentry Affiliate

Thu, 22 Jan 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are very pleased to announce that University College London has become an affiliate of the Software Carpentry Foundation. We are grateful for their support, and look forward to working with them more closely.

Post-Workshop Instructor Debriefing, Round 1

Wed, 21 Jan 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

In the first two weeks of January 2015, 14 Software Carpentry workshops were held at various locations. Every workshops brings its share of lessons learned and new experiences, particularly with new instructors coming online. On January 14, Greg Wilson hosted a post-workshop debriefing meeting attended by 21 instructors from 10 recent workshops. We got together to discuss how the workshops went, what worked, what didn't, and what could be improved.

Workshops in Oxford

Wed, 21 Jan 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Wellcome Trust Centre for Human Genetics at the University of Oxford hosted its first Software Carpentry workshop on 13th and 14th January. Instructor and co-organizer Philip Fowler has blogged about it, and included both photographs and some bubble charts of the feedback he received.

Separately, James Allen has blogged about the first and second days of a separate workshop that he ran at Oxford for people in atmostpheric, oceanic, and planetary physics. This was James' first time teaching for us, and his summary of things that he thinks could be improved includes some ideas that we'll fold back into the main notes.

Feedback from Workshop at UFSC

Tue, 20 Jan 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Last December we ran another R-based SWC workshop at University of Santa Catarina (Florianòpolis, SC, Brazil). The primary target audience encompassed grad students from the Biology Faculty who had some experience in programming and R. Raniere Silva (an undergrad in Applied Maths at UNICAMP, Campinas, SP, Brazil) co-taught the workshop with me and the Ecology grad students Renato Morais Araùjo and Juliano Bogoni (Department of Ecology and Zoology) acted as our hosts and in situ organizers.

Orwell, Dickens, and How We'll Know We're Done

Sat, 17 Jan 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I started working on a short capstone example last month to show learners how to get a badly-formatted reference list out of an Excel spreadsheet and into a relational database so that it would be easy to answer questions like, "Who has co-authored papers with whom?" I'd like to work up another capstone as well, but there's a problem: I can't actually do it myself for reasons that are both technical and political.

2015 Election: Adina Howe

Fri, 16 Jan 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Hi! My name is Adina Howe, and I thank you for considering my qualifications to serve on the Steering Committee of the Software Carpentry Foundation.

Software Carpentry has played a major role in my professional development, and I would argue played an integral role in my ability to fulfill my dreams as becoming a tenure-track professor. In 2008, I participated in a Software Carpentry workshop at Michigan State University as a postdoc. At the time, my research required some computational prowess that I did not possess. SWC provided me a foundation, resources, and mentors to help me improve my skills and discuss my frustration and eventually success! This impact (and great instruction and mentorship) insprired me to join as one of the first round of instructors (2011-ish). For more details, please feel free to peruse my complete vitae.

2015 Election: Ivan Gonzalez

Fri, 16 Jan 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I've been an instructor since September 2013 and have taught 10 workshops so far. I'm finishing a master's in Science Communication and work at the Martinos Center for Biomedical Imaging. I would propose three things to the new Committee: improve our outreach and communication strategies, expand Software Carpentry to other regions and languages, and strengthen the mentoring program for new instructors.

2015 Election: Jonah Duckles

Fri, 16 Jan 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Hi, I'm Jonah Duckles, I have been active in teaching Software Carpentry workshops (seven so far) and have found a joy in teaching and learning this curriculum that has made me passionate about the mission of the organization. In my job, I work to spread knowledge, skills and competency in wide ranging areas of scholarly computing. Serving on the Software Carpentry Steering Committee would be a natural extension of my current job, dovetailing well into my current responsibilities and I'm excited to be considered.

2015 Election: Tim Cerino

Fri, 16 Jan 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

On the first day of my first programming job, my boss handed me 700 lines of spaghetti code written in PL/1 and asked me to "fix it." On paper, I was a Research Assistant at an Economic Policy think-tank. I had taken a few Computer Science classes and had coded for fun as a kid and in college. But at that moment I felt that I had entered the world of "real" coding. (Ultimately the "fix" was to completely discard the code and redo everything using proper testable modular code, but I digress...)

2015 Election: John Blischak

Thu, 15 Jan 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I feel I would be a good addition to the Steering Committee of the Software Carpentry Foundation because 1) I am a scientist that had to learn programming on the job, just like our target audience, 2) my experience performing many roles within SWC, most importantly as the maintainer of the R lesson materials, and 3) my goal to increase the collaborative aspect of our lesson development process.

2015 Election: Jason Williams

Wed, 14 Jan 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

"You can trust in Jason, he's your friend!" — Decent People

I can't recall ever running in an election - rule by fiat is what "just feels right" to me. I have on occasion thought about running for office but I usually lack ambition (although I'm attracted to power). The fantasy of being the candidate people vote for even though he is clearly against their interest just makes my pedipalps tingle. So, when I learned about the Software Carpentry Foundation election, I thought, "let's try that!"

2015 Election: Jeramia Ory

Wed, 14 Jan 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

At SciPy 2013, I had the good fortune to attend a tutorial lead by Matt Davis and Katy Huff. As an undergraduate educator in the sciences, I was immediately impressed with the people and excited to try their approach with my students. Matt encouraged me to complete instructor training, which I did in April of 2014.

2015 Election: Karin Lagesen

Wed, 14 Jan 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I have been involved with Software Carpentry since I attended a workshop in Oslo, Norway, in 2012. I started out as a student, then went through the second round of instructor training, before being an instructor at my first workshop in 2013. Since then, I have organized one workshop and been an instructor at five others. I have also been involved with revamping the pre-assessment forms. This work has been very fulfilling and challenging on very many levels, and I now want to expand on my efforts by volunteering for the steering committee.

2015 Election: Sheldon McKay

Wed, 14 Jan 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I started my career as a molecular biologist working in the data-intensive field of functional genomics, enventually becoming a full-time bioinformatician. With the perspective of a person who transitioned from the wet lab to informatics, I have often served as a liaison between researchers and software developers and truly enjoy empowering scientists by teaching them computing skills to accelerate their research. Over the past decade, I have contributed to a variety of outreach and training efforts in scientific computing. When I learned about Software Carpentry, its mission resonated with me and I wanted to get involved. I completed instructor training in May 2014 and have been an instructor at six workshops since then and continue to do about one workshop every six weeks. I also serve as a volunteer topic maintainer for SQL. I am very proud to be associated with the Software Carpentry Foundation and have a lot of experience to offer. I would like to contribute more to our ongoing success by becoming a member of the steering committee.

Language Wars and Others

Wed, 14 Jan 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We often get asked, "Why do you teach [X]? Why don't you teach [Y]?" where X and Y are random permutations of Perl, Python, R, MATLAB, Julia, C++, and Javascript, or equally random permutations of different version control systems or text editors. There are three answers:

Practical Computing for Biologists (and Other Scientists)

Wed, 14 Jan 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are big fans of Steve Haddock and Casey Dunn's book Practical Computing for Biologists, which covers everything we do and more (and is suitable for all kinds of scientists, not just biologists). We were therefore very pleased to learn that they are running a course at Friday Harbor Laboratories this summer. The outline is below, and applications are due on February 1st.

2015 Election: Aleksandra Pawlik

Tue, 13 Jan 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

My first degree was in Computer Science. When I started the university I had very little idea about programming. I struggled a lot. I graduated very discouraged and convinced that I just couldn't do it. But then eight years later Greg asked me if I'd like to be a Software Carpentry instructor. I finished the instructor training and I realized that had I been taught differently, maybe now I'd have a great career as a software developer. But then, I wouldn't be writing this blog post.

Introducing Software Carpentry

Tue, 13 Jan 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As we said in December, we're planning to use the latest incarnation of Browsercast as a web-native alternative to screencasting for recording and presenting our slideshows. Katy Huff has just recorded the first of these, which you can watch on GitHub. We'd be very grateful for feedback both on its content and on Browsercast in general.

Thanks to RStudio

Tue, 13 Jan 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Our thanks to RStudio for their generous donation to help us run Software Carpentry workshops. Inspired by the innovations of R users in science, education, and industry, RStudio develops free and open tools for the R community. These include the RStudio development environment as well as the shiny, ggvis, and dplyr packages (among many others). RStudio also offers enterprise-ready professional products to make it easier for teams to scale and share work.

2015 Election: Raniere Silva

Mon, 12 Jan 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A year and a half ago I read the announcement of the Mozilla Science Lab by Mark Surman and that way I discovered Software Carpentry. Months later, I was participating in the 6th round of Instructor Training, and since then I have:

taught at all 6 workshops held in Brazil,
improved the lessons and helped new instructors do so too,
suggested changes to the new layout of our lessons and helped in the transition, and
publicized Software Carpentry.

Volunteering with Software Carpentry is one of the things that I'm really proud of doing and I'd be honored be a member of the SCF Steering Committee.

Instructor Training at UC Davis

Mon, 12 Jan 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

On Tuesday and Wednesday of last week, we ran a live instructor training class at UC Davis. Over 40 people from all across the country got a lightning introduction to the basics of educational psychology and instructional design, and had a chance to hear how and what we teach.

January 2015 Lab Meeting

Mon, 12 Jan 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We will hold our next online lab meeting at 11:00 and again at 19:00 Eastern time on Thursday, January 22, 2015. People who are standing for election will have a chance to talk about what they hope to do and to answer questions from other members, and we'll present updates on our finances and on our new workshop and lesson templates. If you're planning to attend, please add yourself to this Etherpad. We look forward to seeing you then.

2015 Election: Katy Huff

Fri, 09 Jan 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Software Carpentry has been part of my life for over six years, I began as an instructor, organizer, and curriculum developer and have been honored to grow as a researcher alongside Software Carpentry as it evolved over that time. In these years, I have organized and taught over a dozen workshops and have, for the last four months, served as an interim Steering Committee member. I have also made an effort to expand the reach of Software Carpentry as a co-author of both the Best Practices paper and a new O'Reilly book, "Effective Computation in Physics: Field Guide to Research in Python." Of course, none of this pays the bills. For that, I am a nuclear engineer...

2015 Election: Matt Davis

Fri, 09 Jan 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

In January 2012 I had the good luck to be in one of Greg Wilson's first workshops after he rebooted Software Carpentry in the current two-day format. I ended up as a helper in that workshop, and a month later I was in Toronto teaching Python as Software Carpentry's first volunteer instructor. Over the past three years I have:

taught at more workshops than I can remember
helped transition our materials from SVN to GitHub
written lessons and ipythonblocks
recruited and mentored new instructors
publicized Software Carpentry at conferences

Volunteering with Software Carpentry has been one of the most rewarding things I've ever done and I'd be honored to continue helping as a member of the SCF Steering Committee.

Research Software Engineer Position at the Oxford e-Research Centre

Tue, 06 Jan 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Research Software Engineer position available at the Oxford e-Research Centre, University of Oxford, UK (Java, Python, Web-app, Database, Semantic-web).

2015 Election Nominations

Mon, 05 Jan 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As we announced last year, an election will be held on January 26-30 for the seven positions on the Steering Committee of the Software Carpentry Foundation. If you are a qualified instructor who has taught at least twice in the past two years, or have done a significant chunk of non-teaching work for Software Carpentry, you can both stand for election and vote. We strongly urge you to consider standing: if you're willing and able to commit to giving the Foundation 3 hours a week, you'll help thousands of scientists. It'll be fun, too: few things in life are more satisfying than working with a dedicated bunch of people to make something useful happen.

In order to stand for election, you must write a blog post to introduce yourself to the community by Friday, January 16. This post must be around 500 words long, can be written in any format (e.g. question and answer, paragraph text), and must be titled, "2015 Election: Your Name". It should explain:

what your background is,
what your previous involvement with Software Carpentry has been, and most importantly
what you will do as a member of the Steering Committee to contribute to the growth and success of Software Carpentry.

2015 Election: Damien Irving

Mon, 05 Jan 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Damien Irving has withdrawn from the election in order to focus on completing his thesis. We're grateful for his work on the interim Steering Committee, and wish him the best of luck with his PhD.

The Future and Funding of Science

Sun, 04 Jan 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I was talking with friends over the holiday about the future of science and how it might one day be funded. Since it'll be ten years before I'm proven wrong, it seems like a good topic with which to start the new year.

Projects, Projects, Projects

Sun, 28 Dec 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We have updated our projects page with links to:

things we're building ourselves, and
things that our members are building.

The first list includes the templates for lessons and workshop websites, a a tool for managing workshops (for which we're using Django), the latest version of Browsercast, and more. The second list has everything from active papers to utilities for simulating vacuum thermionic energy conversion devices. If you'd like to help with the first, or if you're a member and would like your project listed in the second, please get in touch.

Welcome Aboard

Tue, 23 Dec 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A lot of people qualified as instructors this fall and winter, thanks in part to the live sessions we ran in Charlottesville, Norwich, and Seattle. They join the 86 other people who received their badge this year; we look forward to seeing them all run workshops before 2015 is over.

Interim Steering Committee Meeting: Dec 2, 2014

Fri, 19 Dec 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Software Carpentry Foundation Interim Board Meeting: Dec 2, 2014

All I Want for Christmas is a Pull Request...

Thu, 18 Dec 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As we said back in October, we're splitting the existing lesson repository into smaller and more manageable pieces. To do that, we have defined a new template for lessons, and have been extracting the history of the existing material from the current repository. (We wanted to get the entire history of each lesson so that people would receive credit for the work they've done.) The second step has taken longer than planned, but we now have all of the core novice lessons in repositories of their own:

Standing for Election

Thu, 18 Dec 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

From 26-30 January, an election will be held for the seven vacant positions on the inaugural Steering Committee of the Software Carpentry Foundation. This will be one of the biggest steps in the project's journey from two guys staying up until 3:00 am fourteen years ago to write lessons on Perl for scientists at Los Alamos to a mature open project run by the volunteers it belongs to.

If you are a qualified instructor who has taught at least twice in the past two years, or have done a significant chunk of non-teaching work for Software Carpentry, you can both stand for election and vote. We strongly urge you to consider standing: if you're willing and able to commit to giving the Foundation 3 hours a week, you'll help thousands of scientists get more done in less time and with less pain. It'll be fun, too: few things in life are as rewarding as building something, and our members are building something extraordinary.

In order to stand for election, you must write a blog post to introduce yourself to the community by Friday, January 16 (i.e., a full week before the start of the election). This post:

must be around 500 words long,
can be written in any format (e.g. question and answer, paragraph text), and
must be titled, "2015 Election: Your Name"

You can submit your post as a pull request to this website's repository or by email. It should explain:

what your background is,
what your previous involvement with Software Carpentry has been, and most importantly
what you will do as a member of the Steering Committee to contribute to the growth and success of Software Carpentry.

The last point is the most important. If you have experience managing money, we need a Treasurer; if your passion is helping new instructors or figuring out how well we're doing, we need people to lead mentorship and assessment, while if you come from a part of the world that hasn't seen much Software Carpentry activity yet, you might want to take the lead in getting us going there.

(Actually, the point that's really most important is that everyone will still be very welcome to volunteer in other ways, and that doing so will be as valuable as ever. We will still need topic maintainers, help with the website, and many other things, and I hope that having more people coordinating things will actually make it easier for you all to lend a hand.)

If seven or fewer nominations are received, those people who nominated will be automatically appointed to the Steering Committee and no formal election will be held. Vacancies on the Steering Committee can be filled at any time at the Committee's discretion. The regular positions on the Steering Committee (Chair, Vice-Chair, Secretary, Treasurer and then any others the Committee feels it needs) will be decided by a vote at the Committee's first meeting.

Guidelines for Extracting History

Mon, 15 Dec 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As discussed previously, we are currently extracting individual lessons from the bc repository to make them more modular, which will ease use, contribution, and maintenance. This post presents some guidelines for extracting individual lessons and how to contribute to lessons in the meantime.

Who Are We?

Mon, 15 Dec 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

For the last three years, I've been storing information about instructors, workshops, and other things in a small SQLite database so that I can look things up and generate statistics when I need to. I can't publish it, since it contains personal identifying information, but since I had to write a script to migrate the data to the tool we're building to manage workshops, it only took another few minutes to create a partly-redacted version of the data. ("Partly" because someone who was really keen could work backward workshop URLs to instructors' names, cross-reference, and recover the names of some fraction of our instructors. Since that information is all public anyway, though, I don't think I've introduced any new risks.)

The SQL source for the database is here; with it, you can regenerate the database using:

Feedback from the MSc Clinical Bioninformatics Workshop

Sat, 13 Dec 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Early in November we put on Software Carpentry for the students of the MSc course in Clinical Bioinformatics run by the University of Manchester, UK and the British National Health Service (NHS). The course combines academic curriculum with work-based programme. The students (who already are qualified professionals) are based at various clinical units in the UK and meet only a few times to attend short intense training sessions. The instructors at the Software Carpentry workshop were Aleksandra Nenadic, who taught for the first time, and myself; we were helped by Mike Cornell and Andy Brass.

Feedback from WiSE in Krakow

Sat, 13 Dec 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A week ago Krakow in Poland hosted the Software Carpentry workshop for Women in Science and Engineering. The registration filled up within 72 hours which clearly shows the need for such events.

Results of Software Sustainability Institute Survey

Sat, 13 Dec 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Software Sustainability Institute's recent survey of researchers at top UK universities is out. Headlines figure are:

92% of academics use research software
69% say that their research would not be practical without it
56% develop their own software (worryingly, 21% have no training in software development)
70% of male researchers develop their own software, and only 30% of female researchers do

For the full story, see this post on their blog.

UC Berkeley Postdoctoral Position in Nuclear Engineering

Sat, 13 Dec 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Department of Nuclear Engineering at the University of California, Berkeley is searching for one, possibly two high-caliber researchers to work with Prof. Rachel Slaybaugh's group in the area of computational neutronics. Fields of highest relevance are Computer Science, Applied Mathematics, and Nuclear Engineering.

Prof. Slaybaugh's group researches methods and algorithms for solving the Boltzmann transport equation more effectively. These methods are often inspired by the physics of the problem at hand, developments in computer hardware, or both. Ongoing work involves deterministic solution methods, Monte Carlo methods, and hybrid methods in which deterministic solutions are used to accelerate Monte Carlo solutions. Potential Projects include Angle-informed Hybrid Methods, Deterministic "Plug-and-Play" Research Environment Creation, Improving Multigroup Cross Sections for Hybrid Methods and Monte Carlo on Graphical Processing Units

We seek a candidate who can start this position as early as January 2015 but the appointment start date is flexible. The initial appointment is 100% time for one year, with the possibility of renewal for a second year, dependent upon job performance and funding. Starting salaries are in the range of $50,000 to $60,000 per year and commensurate with qualifications and experience.

For more information, please see the full description

UCL Research Software Dashboard Developer

Sat, 13 Dec 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The University College London Research Software Development Initiative is seeking a full-stack web developer to work on its Research Software Dashboard project from January 2015 to July 2015.

This is a new project, starting from scratch, to develop software to curate, promote, and manage the University's wide portfolio of cutting-edge scientific and scholarly software.

The project will provide an overview of the research software output of the college for scientists, managers, funders, investors and clients, including both open-source software and software being commercialised through the university's business and consulting arms. It will integrate with the University's code management infrastructure, based on GitHub Enterprise, software testing infrastructure based on Jenkins, and commercial software sales platform e-Lucid.

Software is an increasingly important scholarly output for research alongside publications, and this project will help retain UCL's leadership in this important aspect of twenty-first century research.

Those interested in being involved in this important project on a freelance or contractor basis should get in touch with James Hetherington (j.hetherington@ucl.ac.uk) for more information. For more information, please see the full description.

Templates: We Live, We Learn

Tue, 09 Dec 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We have partially converted four of our core lessons to the new lesson template, and are making a few tweaks as a result. The most important of these is to move the source files for a lesson's web pages out of a pages sub-directory and into the root directory, so that (for example) the first topic in a lesson will be ./01-intro.md rather than ./pages/01-intro.md.

Doing this means that the Markdown source files will be in the same directory as the HTML pages compiled from them. That's generally considered a bad thing, since (a) it makes it harder for people to tell source files from generated files and (b) the chances of accidentally deleting source files when you just meant to delete generated files goes up significantly. We think it's the right choice in this case because:

Software Carpentry Returns to Edinburgh

Mon, 08 Dec 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Last week, EPCC's ARCHER training team ran another Software Carpentry workshop here in Edinburgh, on 3rd and 4th of December. The workshop provided attendees with an introduction to version control and Git, building programs with Python, automating tasks with Make, and how (and how much) to test programs. These were set within the context of best practices for scientific computing.

Announcing the Lesson Validator

Fri, 05 Dec 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Thanks to the work of Andy Boughton and Raniere Silva, we now have a validator that determines whether lessons fit the new lesson template. This post explains what it does, where there is room for improvement, and how you can help.

Google to Support WiSE Poland

Fri, 05 Dec 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are happy to announce that Google will support the Software Carpentry workshop for Women in Science and Engineering which will take place this weekend in Krakow! We are grateful for their support, and look forward to the event; to which registration filled up within 72 hours!

New Accessibility Guidelines

Fri, 05 Dec 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Software Carpentry values the participation of every member of the scientific community and want all attendees to have an enjoyable and fulfilling experience. Thanks to Pauline Barmby, we now have an accessibility checklist for instructors and workshop hosts. Please have a look and send us fixes and suggestions for improvements.

An Advanced Short Course in Leeds

Thu, 04 Dec 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Helped by Aaron O'Leary, Peter Willetts, Marlene Mengoni and Jo Leng, Martin Callaghan, Devasena Inupakutika and I recently delivered a modified Software Carpentry workshop at the University of Leeds. Aimed at environmental scientists from across the UK and funded by NERC, the course included an extra day where small groups got to develop tools for environmental data analysis.

Cape Town South Africa Workshop

Thu, 04 Dec 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Last week we ran the biggest software carpentry workshop in South Africa to date. We had more than 80 participants in two rooms, each using the novice Python material. Pre-workshop assessment surveys showed that we'd have a mostly novice crew, but with a few outliers on the upper end. We accepted slightly higher than 80 enrollment cap and encouraged those with more advanced skills to evaluate their interest in becoming SWC workshop instructors themselves and help us out as helpers.

International Workshop on Software Engineering for High Performance Computing in Science

Thu, 04 Dec 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The 2015 International Workshop on Software Engineering for High Performance Computing in Science is being held in conjunction with the International Conference on Software Engineering in Florence, Italy in May 2015. Topics to be covered include:

Our First Election

Wed, 03 Dec 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Software Carpentry Foundation's bylaws state that the Steering Committee must be elected annually. With a new year upon us, the interim committee has decided that the first such election will take place in the week of Jan 26-30, 2015. Here are a few key points:

Every qualified instructor who has taught at least twice in the past two calendar years is automatically a member of the SCF, as is anyone who has done 30 days or more work for the SCF in the past calendar year, and anyone who has, in the opinion of the Steering Committee, made a significant contribution in the past year.
All SCF members may both stand for election and vote in it. We'll explain how to do both after the next interim committee meeting on December 16.
As discussed in an earlier post, members of the Steering Committee are expected to volunteer at least 2-3 hours a week to help mentor our instructors, conduct assessment, oversee major changes to the curriculum, manage our finances, and so on.

Plans for 2015: Instructor Training

Wed, 03 Dec 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Instructor training has been going well: it looks like more than 50% of people who participated this summer and fall will complete and start teaching for us, which is a new record. What's even better is that we have 136 people signed up for the next online course. Even if only half of them complete, it will enlarge the pool of instructors by more than a third, and broaden our coverage both geographically and across disciplines.

The question now is when that course should start. Teaching a group that size will take two full days a week, and in the short term, I need to focus on building partnerships, getting grant applications out the door, and overseeing election of our Steering Committee.

Plans for 2015: Lessons

Wed, 03 Dec 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The previous three posts in this set looked at instructor training, workshop organization, and the twin challenges of mentorship and assessment. In this final one, I'll summarize the state of the changes we're making to our curriculum.

We described the new template for lessons back in October, and since then, a handful of people have been working to improve it and to extract our existing lessons from the 'bc' repository and convert them to the new format. The first step has taken longer than planned: we want to be sure we get the entire history of each lesson, so that people receive credit for the work they've done, and thats proving to be a slog.

The good news is, you can now see what the result will look like. Our novice lesson on SQL now lives in this repository, and you can view its rendered form in this GitHub pages site. There's still a lot to do—the learning objectives need to be cleaned up, the challenges all need meaningful names, and there's clearly lots of scope for improving the lesson's appearance—but the pieces are there.

Plans for 2015: Mentorship and Assessment

Wed, 03 Dec 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The previous posts in this set looked at instructor training and workshop organization. In this one, I'd like to look at mentorship and assessment, which are two of the biggest challenges we need need to address in the coming year, and are good examples of the kinds of tasks that Steering Committee members will be asked to take on.

Azalee Bostroem did a great job summarizing why we need to mentor our new instructors (and keep those who've been with us a while up to date with changes to our lessons and procedures). The question is, how? Or more precisely, where will the hours come from? I said in September that I would organize a weekly meeting for instructors from recent and upcoming workshops. Only one took place, and saying "we just need to try harder" won't make the necessary hours appear.

The same is true of assessment. Jory Schossau has done valuable service analyzing survey data and interviewing bootcamp participants, and Daniel Chen and others are working hard to revise the instructors' post-workshop questionnaire, but despite several attempts, we haven't found anyone willing to fund a systematic look at what we're actually doing and what impact it's having. Once again, we can either say "we need to try harder" or come up with an alternative plan.

Plans for 2015: Workshop Organization

Wed, 03 Dec 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As I write this, 228 people have been certified to teach Software Carpentry workshops, of whom 136 have taught in the past twelve months. That's amazing, but as I said in a previous post, growth in one part of a pipeline inevitably turns another into a bottleneck. In our case, that bottleneck is organizational: Arliss Collins and Giacomo Peru are stretched to the limit handling requests and lining up instructors, and we're still not keeping up with demand.

This kind of work can't be done by a volunteers in bits and pieces: the startup overheads are significant, as is the context required to manage each conversation, and there are time when mail really needs to be answered right now, not when it's convenient. We are therefore asking our Partners to organize workshops in their region, and hope that others will shoulder some of the burden as well.

We're also trying once again to build a tool to simplify workshop management. I have started a simple Django application called Amy to keep track of who wants a workshop, who can teach what, who our learners have been, and so on. The data model is mostly done, and I've included a dump of our existing database with personal information redacted to aid further development and testing.

All Amy can do now is display information. What we want to do is add and edit new data. If you'd like to help, please fork the project and send us pull requests—we'd be grateful for your help.

Software Carpentry Foundation: Governance

Wed, 03 Dec 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I am pleased to announce that our interim Steering Committee has adopted bylaws for the Software Carpentry Foundation, which is the final step in us becoming an independent organization. The authoritative version is stored in this public GitHub repository, along with other key documents that will be outlined in the next couple of blog posts. In brief, the SCF has four parts:

the Membership, which includes active instructors and others contributing directly to the project;
the Steering Committee, which is elected by and from the membership and is the SCF's primary decision-making body;
the Advisory Council, which includes representatives from our major partners, and
the Executive Director, who is an employee of the SCF responsible for overseeing its daily operations. The Executive Director answers to the Steering Committee, but is not a member of it.

I have accepted the position of interim Executive Director, and the first elections for the Steering Committee will be held early in 2015. These bylaws, and the agreement with NumFOCUS that enables us to operate as a 501(c)3 non-profit, are the structure we need to run a successful independent foundation.

The next priority is fundraising: I've been without a salary since my contract with Mozilla ended in October, which clearly isn't sustainable. The next post in this series describes the different ways in which organizations can support us; we're close to signing deals with several groups, so I'm confident we'll start the new year with our finances in order.

Software Carpentry Foundation: Organizational Membership

Wed, 03 Dec 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As we said in the previous post in this series, our interim Steering Committee has adopted bylaws for the Software Carpentry Foundation. They have also agreed on four tiers of membership so that universities, companies, government labs, and other entities can help support and guide our work. The authoritative version is stored in this public GitHub repository, but in brief, the tiers are:

Partners, who make a significant long-term contribution to organizing and delivering workshops;
Affiliates, who are organizing workshops and helping with admin, but not at the same level;
Sponsors, who underwrite the cost of particular workshops that would otherwise not have backers; and
Donors, who simply wish to make cash or in-kind contributions to help with general operations.

Each membership tier is described below. The Steering Committee and Advisory Council will revisit them in a year's time to ensure they're meeting everyone's needs, and we will always be willing to discuss other arrangements. Please mail {{site.board_inquiries}} if you would like more information, or would like to start a discussion about how you could help. And of course, individual donations to the cause will always be welcome.

Software Carpentry Foundation: Workshops

Wed, 03 Dec 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As we said in the previous posts in this series, our interim Steering Committee has adopted bylaws for the Software Carpentry Foundation and a plan for organizational membership. Those memberships won't cover all of our central costs (such as instructor training), so we are going to start charging a fee for each workshop we help organize rather than just asking people to make a donation. In exchange, we will match hosts with instructors, handle registration, manage assessment, follow up to make sure people's travel expenses have been paid, and all the other running around that needs to happen behind the scenes.

One thing that won't change is that anyone who wants to organize and run a workshop on their own will always be free to do so without charge provided they satisfy a few simple conditions. In fact, we strongly encourage groups to get to the point where they can do this regularly, and to share their experiences with the community so that we can all help teach good lab practices for scientific computing.

Summarizing the News

Wed, 03 Dec 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We made several big announcements this morning, so here's a short summary to guide you through them all:

The bylaws for the Software Carpentry Foundation.
Our organizational membership scheme.
New rules for organizing and running workshops (including fees).
Plans for instructor training (which also explains why we're delaying the start of the next online course by a month).
Why we need more admin support and the tool we have started building to support that.
Plans for mentorship and assessment, which are two of the things that elected members of our Steering Committee will be asked to take on...
An update on our new lesson format (and links to some examples).
The date of our first election.

We'll talk about all of this at the Dec 4 lab meeting (which takes place at 10:00 and again at 19:00, both times Eastern — see this Etherpad for connection details). Hope to see you there...

What About MOOCs?

Tue, 02 Dec 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We frequently get asked whether Software Carpentry would work as a MOOC. The answer is that I think it can work well if it's what Siemens and Downes actually had in mind when they coined the term. They didn't people watching videos and then doing robo-graded exercises; instead, their connectivist model of learning assumed that participants would use the internet to collaborate in exploring ideas, rather than as a faster form of television.

I'm definitely excited about the Siemens and Downes kind of MOOC. In particular, I believe that instructors who don't have time to teach a full workshop might give us an hour a week to help people via one-to-one or one-to-few sessions via Skype and screen sharing. There was a lot of enthusiasm among the instructors for this when we tried it in the spring of 2012; that experiment wound down because we lacked critical mass, but we're five times larger now, and I think it would be worth trying again.

The most interesting question for me is where this fits. Should we start people off this way? Should people do the first day in person (so that we can get them through software setup and configuration issues), then do the rest online? Should this be used as the "day 3" follow-on that everyone keeps asking for? We'd like to try all of this and more; if you'd like to help, please let us know.

Goalposts for the Digital Humanities

Mon, 01 Dec 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As a follow-on to last month's post about courses at the British Library, I asked some people who are teaching digital humanists where their goalposts are, i.e., what they think the minimum someone in the humanities should know about working with computers and digital data. The brief responses were interesting:

Reminder: Lab Meeting on Thursday

Mon, 01 Dec 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Just a reminder that our last online lab meeting for 2014 will be held at 10:00 am Eastern time on Thursday, December 4, and will be repeated at 7:00 pm Eastern on the same day. We will be making several big announcements, so if you're planning to attend, please add yourself to the Etherpad at https://etherpad.mozilla.org/swc-labmeeting-2014-11 so that we have an idea of numbers. We look forward to seeing you Thursday.

How to Manage Confidential Data

Thu, 27 Nov 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A couple of days ago, a member of our community wrote:

I am very interested in working on improving research analytics in government using the principles developed by Software Carpentry. A key stumbling block I run into with my peers is their skepticism about these principles being able to protect data that is confidential or restricted use.

Translating Software Carpentry into Korean

Tue, 25 Nov 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This is pretty amazing: a group has translated the core Software Carpentry lessons into Korean:

Introduction: http://statkclee.github.io/xwmooc-sc/intro.html
The Unix Shell: http://statkclee.github.io/xwmooc-sc/novice/shell/
Version Control with Git: http://statkclee.github.io/xwmooc-sc/novice/git/
Programming with Python: http://statkclee.github.io/xwmooc-sc/novice/python/
Programming with R: http://statkclee.github.io/xwmooc-sc/novice/r/
Using Databases and SQL: http://statkclee.github.io/xwmooc-sc/novice/sql/
Extras: http://statkclee.github.io/xwmooc-sc/novice/extras/

It's a remarkable and encouraging achievement—congratulations and thanks to the team that did the work:

Victor KC Lee (이광춘): Translation Lead
Jungsu Han (한정수): Translator
Chungyeong Moon (문춘경): Illustrator
Ri Jeong Kim (김이정): Graphic Designer
Hwan Beom Kang (강환범): Support

We still need to figure out how to manage this: Gabriel Devenyi's post outlined our options, but someone will need to take the lead to set something up so that versions in various languages can stay in sync.

Congratulations to Data Carpentry

Mon, 24 Nov 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

In case you missed the announcement last week, our sibling organization Data Carpentry has received funding from the Moore Foundation to support its activities and growth, and Dr. Tracy Teal (a long-time contributor to Software Carpentry, and one of the founders of Data Carpentry) has accepted a position as Data Carpentry's project lead. We're very excited by both developments, and are looking forward to continuing to work with them.

Announcing WiSE Krakow!

Sun, 23 Nov 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

After running several Software Carpentry workshops for women in science and engineering in North America, we are moving east: on 6th and 7th December we will run a WiSE workshop in Krakow, Poland.

Like previous WiSE workshops, this one is open to women at all stages of their research careers, from graduate students, post-docs, and faculty to staff scientists at hospitals and in the public, private, and non-profit sectors. The instructors at the workshop will be Paulina Lach of Krakow's University of Science and Technology AGH and Aleksandra Pawlik, the Software Sustainability Institute's Training Leader. The helpers are Agnieszka Celińska (Pedagogical University of Krakow), Iwona Grelowska (University of Science and Technology AGH), Patrycja Radaczyńska (Creativestyle), and Anna Ślimak (Lunar Logic).

Instructor Training Stats

Sat, 22 Nov 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Software Carpentry Foundation's board asked me for some stats on instructor training, and I thought other people would find them interesting as well:

Adding a Projects Page

Thu, 20 Nov 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

It's long overdue, but we've finally started a page to showcase the projects that our instructors are involved in. If you'd like to add one, please send us a pull request or mail us for instructions.

(Turns out, we do pretty cool stuff...)

The New Instructor Post-Assessment Questionnaire

Wed, 19 Nov 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As mentioned two weeks ago, a group led by Daniel Chen have been revising the post-workshop survey for instructors so that we can get a better idea of who's actually teaching what, how long it's taking, and how it's going. The latest draft can be viewed here, and is included below. We'd be grateful for one last round of feedback—in particular, we're concerned about (a) the overall length and (b) how closely the headings are tied to our current curriculum. Please let us know what you think in comments on this post.

Close Enough for Scientific Work

Tue, 18 Nov 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The discussion around last month's post "Why We Don't Teach Testing (Even Though We'd Like To)" has been one of the most interesting in Software Carpentry's history. Inspired by that, and by discussion at WSSSPE 2.0, we are launching a collaborative book project called Close Enough for Scientific Work in which scientists will show one another how they test their software.

Contributions should be aimed at sophomores in science and engineering, and each should be sized to fit a one-hour lecture. While the format of each entry will vary according to its content, we expect most will follow this template:

Interim Board Meeting: Nov 18, 2014

Tue, 18 Nov 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Software Carpentry Foundation Interim Board Meeting: Nov 18, 2014

Lessons, the Repository Split, and Translations

Sat, 15 Nov 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Keeping on the roll of the posts about the repo split, templates, and metadata, Software Carpentry now needs to consider how to handle translated lessons. The core Software Carpentry lessons have be translated by bilingual instructors into Korean produced by Victor KC Lee (and friends) and Spanish by Francisco Navarro (and friends). With the upcoming repo split, I think its a good time to examine the various options of how we might handle translations generally.

Announcing November 2014 Lab Meeting

Fri, 14 Nov 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Our last online lab meeting for 2014 will be held at 10:00 am Eastern time on Thursday, December 4, and will be repeated at 7:00 pm Eastern on the same day. (We had originally planned to have it on November 27, but that would conflict with American Thanksgiving.) If you're planning to attend, please add yourself to the Etherpad at https://etherpad.mozilla.org/swc-labmeeting-2014-11 so that we have an idea of numbers. We look forward to seeing you in a few weeks.

And yes, we're calling it the November lab meeting even though it's in December, because it just wouldn't be programming without an off-by-one error...

Workshop Summary: Gerstein Science Information Centre, University of Toronto

Fri, 14 Nov 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Last week we ran a Python-based SWC workshop at the University of Toronto's Gerstein Science Information Centre. The advertised audience was graduate students, post-docs and other researchers in science, engineering and medicine at the University of Toronto.

The workshop had about 20 learners. Most were grad students or postdocs in life sciences. The pre-workshop assessment said that about three-quarters of learners had some programming experience with about half of those having some Python; maybe half of the total had some familiarity with the command line. Almost none had experience with version control or SQL. The instructors were Pauline Barmby, Greg Wilson, and Thomas Guignard; helpers were Sahar Rahmani, Luke Johnston, Daiva Nielsen, Tom Wright, and Fei Xu. Our host was Erica Lenton.

Replacing the Teaching Blog

Thu, 13 Nov 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We've been using a WordPress blog to manage instructor training for the past two and a half years, and while it's served us well, it's clear that we've outgrown it. It's also clear (to me at least) that we should get instructors started on GitHub earlier, so replacing WordPress with a GitHub-backed blog seems like it would serve a dual purpose.

Our requirements are:

Why It Matters

Tue, 11 Nov 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Sometimes I forget that it isn't obvious why scientists ought to learn to program—or why anyone else ought to. Being more productive, getting a better job... Those are all good reasons, but as Bret Victor points out, if we focus on those, we risk losing sight of what matters most.

To explain that, I have to quote Matthew Crawford's thought-provoking (and sometimes infuriating) Shop Class as Soulcraft:

Workshop at GeoSim (Potsdam)

Tue, 11 Nov 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I had never taught a Software Carpentry workshop before (though I have taught an Advanced Scientific Programming in Python many times), so when the hosts for this one asked if I would be willing to do a full-week workshop, I said they were crazy ;) They convinced me to do it, but I said in that case I could only do that if I can recycle the materials I use for our summer school. They agreed, and I managed to convince a buddy in Berlin to help me with tutoring.

Workshop at the University of Virginia

Tue, 11 Nov 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We pulled off our day-long data analysis bootcamp with hardly a hiccup yesterday. The schedule looked something like this:

AM pt 1:	intro to AWS & intro to Unix Shell
AM pt 2:	data analysis in Unix: alignment, quantitation of RNA-seq data
PM pt 1:	intro to R
PM pt 2:	data analysis in R: QC, differential expression, etc.

Ongoing Learning with User Groups

Mon, 10 Nov 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

For the past two years I've run the UC Davis R User's Group (D-RUG). In this post, I'll (1) outline the structure of D-RUG, (2) summarize some lessons learned, and (3) discuss how such users' groups could act to support and complement SWC's workshops.

Amdahl's Law and Software Carpentry

Fri, 07 Nov 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Amdahl's Law says that the speedup you can get by parallelizing a computation is limited by how much of the computational can't be sped up. For example, if 10% of a program's run time is inherently sequential, then even if you have an infinite number of processors, you can't speed it up more than 10 times.

The same rule applies to organizations like Software Carpentry. We now have almost 200 certified instructors; even working in pairs, and each teaching only once a year, that's enough to run two workshops a week, and we're training more all the time. But someone has to train them, and match them with workshops, and design new templates for lessons, and talk to potential sponsors, and those central activities are now limiting what we can accomplish.

Instructor Training at TGAC

Thu, 06 Nov 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Hayley London has written a great summary of the instructor training at TGAC in Norwich in October. Aleksandra Pawlik, Bill Mills, and I enjoyed meeting everyone, and I look forward to returning soon.

Why Institutional Partnerships Make So Much Sense

Thu, 06 Nov 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

When I first got involved in Software Carpentry, I liked nothing more than to lament the fact that universities don't teach their undergraduates (or early post-graduates) fundamental programming skills. It was one of the main reasons I felt so compelled to help out. By running our two-day workshops, I felt that we were basically picking up the slack until such time that universities woke up to themselves and started teaching this stuff. At this point, I felt that (like many non-profit organisations striving to make the world a better place) Software Carpentry would have achieved its mission by essentially making itself redundant.

Fast forward a couple of years and I'm now part of a new University of Melbourne department tasked with the job of teaching programming fundamentals to post-graduates. To my surprise, I've found that in this progressive new world we actually need Software Carpentry more than ever. The interim board of the Software Carpentry Foundation is seeking to form partnerships with organisations like my own, so I thought this was an opportune time to share our experiences.

You Should Read Juha Sorva's Thesis

Thu, 06 Nov 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

If you really want to dig deeper into educational research and how it applies to teaching programming, you should grab a copy of Juha Sorva's PhD thesis. The UUhistle system he built is interesting, and the research he did with it is thought-provoking, but what's really great is the summary of educational research in the first third of the thesis. I've extracted that part with his permission, and I can't recommend it highly enough.

A 'Joel Test' for Grassroots Programming Groups

Tue, 04 Nov 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Back during the first dot-com bubble, Joel Spolsky wrote an article titled "The Joel Test: 12 Steps to Better Code" that listed 12 questions you can ask to estimate the maturity of a software development team:

An R Workshop at the University of Sydney

Tue, 04 Nov 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Last week we ran an R-based SWC bootcamp at University of Sydney. The primary target audience encompassed grad students from the Department of Psychology who had little to no experience in programming. While some had previous limited experience with programming languages (e.g. MATLAB or R), most of them had been exposed only to statistical software such as SPSS.

Interim Board Meeting: Nov 4, 2014

Tue, 04 Nov 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Software Carpentry Foundation Interim Board Meeting: Nov 4, 2014

Software Carpentry Foundation: FAQ

Mon, 03 Nov 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As we announced two weeks ago, we are setting up an independent foundation to manage Software Carpentry's continued growth. This week, we passed an important milestone when we signed a fiscal sponsorship agreement with NumFOCUS. There are still lots of details to sort out, but here are brief answers to some of the questions we've received so far:

Revamping the Instructor Survey

Sun, 02 Nov 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

In September, Greg Wilson wrote a blog post on "building better teachers" that was inspired in part by an issue brought up by John Blischak on reporting time spent on lessons. In order for us capture this information, we decided to revamp our post-workshop instructor survey.

Particle Physicists Pulling Themselves From The Swamp

Fri, 31 Oct 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

What does it mean to work on a modern particle physics experiment like ATLAS (wikipedia, public) or CMS (wikipedia, public) at the Large Hadron Collider in the 21st century? It's fun, it's collaborating with great and interesting people, it's challenging, it's making you enjoy finding things out, it's what I always wanted to do. Also: it is painful, discouraging, and tends to suck the life out of a young mind. Confused? Let's rewind...

Why We Don't Teach Testing (Even Though We'd Like To)

Thu, 30 Oct 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

If you haven't been following Lorena Barba's course on numerical methods in Python, you should. It's a great example of how to use emerging tools to teach more effectively, and if we ever run Software Carpentry online again, we'll do it her way. Yesterday, though, when she posted this notebook, I tweeted, "Beautiful... but where are the unit tests?" In the wake of the discussion that followed, I'd like to explain why we no longer require people to teach testing as part of the Software Carpentry core, and then ask you all a favor.

Pandoc and Building Pages

Wed, 29 Oct 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Long-time readers of this blog and our discussion list will know that I'm unhappy with the choices we have for formatting our lessons. Thanks to a tweet from Karl Broman, I may have an answer. It's outlined below, and I'd be grateful for comments on usability and feasibility.

Why Software Matters

Tue, 28 Oct 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Why does software matter to scientists? It may seem obvious to people who read this blog, but that's like saying that the answer to, "Why opera?" is obvious to the sort of person who pays a month's rent to get a decent seat at Covent Garden. Why does software matter? And why does it matter whether it's written well?

British Library Courses

Mon, 27 Oct 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I had a chance to catch up with James Baker at the British Library on Friday, and discovered that they're running an amazing series of short classes on digital skills for librarians. With his permission, I've posted their outline below, along with a few excerpts from their FAQ. Some of it is site-specific, but I think a lot would be relevant elsewhere. (Programming in Libraries was developed alongside writing these lessons for the Programming Historian and tweaked for the library audience; interested readers should also check out the practical elements of Managing Personal Digital Research Information work through this public wiki.) If you'd like more information, please mail digitalresearch@bl.uk.

Lost in Space

Mon, 27 Oct 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

You probably haven't seen the 1998 movie Lost in Space, or if you have, you've suppressed the memory—it was awful. But I do know one guy who enjoyed it. His name was Joe, and he had worked on the software used to create its special effects. Ten minutes into the film he took out his Walkman (a primitive form of iPod), put on his headphones, and spent the next two hours head-bobbing to a mix of Bob Marley and Smashing Pumpkins.

A New Lesson Template, Version 2

Thu, 23 Oct 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Update: this post now includes feedback from participants in the instructor training session run at TGAC on Oct 22-23, 2014. Please see the bottom of this page for their comments.

Thanks to everyone for their feedback on the first draft of our new template for lessons. The major suggestions were:

Presenting the Novice R Materials and Future Plans for the SWC R Community

Mon, 20 Oct 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Approximately seven months after our initial meeting, the SWC R community has developed the first set of R lessons for use both in workshops and for self-directed learning from the SWC website. These novice R lessons are a translation of the current novice Python lessons.

Num Wrongs Plus Plus

Fri, 17 Oct 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I was teaching Git to a room of roughly 25 students on day 2 of a Software Carpentry workshop and we ran into a problem that feels like a case study on the reason it's hard to move science to safer practices.

Welcome More New Instructors

Thu, 16 Oct 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are very pleased to welcome another new batch of instructors to our team:

A Research Software Petition

Wed, 15 Oct 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

"We must accept that software is fundamental to research, or we will lose our ability to make groundbreaking discoveries." If you agree—and I hope you do—then please take a moment to add your name to this petition posted by the Software Sustainability Institute, and then help to spread the word by blogging, tweeting, and telling your friends.

A New Template for Lessons

Tue, 14 Oct 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Note: this post has been superseded by this one. Please post comments and feedback there.

We blogged two weeks ago about a new template for workshop websites. It's now time to start thinking about what lessons will look like: as we said at the last lab meeting, we're going to break the current lesson repository into smaller and more manageable pieces, but we need to decide what those pieces will look like first. The post below is our current thoughts; comments and/or follow-on posts about alternatives like those already written by Gabriel Devenyi and Raniere Silva would be very welcome.

A Self-Recorded Workshop

Tue, 14 Oct 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Among the many great lessons contained in Greg Wilson's recent post on building better teachers, perhaps one of the most important was that in order to improve our collective teaching standards, we really need to see each other in action:

Interim Board Meeting: Oct 14, 2014

Tue, 14 Oct 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Software Carpentry Foundation Interim Board Meeting: Oct 14, 2014

Of Templates and Metadata

Tue, 14 Oct 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As an appendix to the splitting the repository post, Greg recently posted a straw man template for how lessons might be structured after the repo split. He followed up after with more details. There a lot of good ideas there on how we can encourage good structure for lessons and help learners and instructors alike going forward.

Yet Another Template for Lessons

Tue, 14 Oct 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

After the splitting the repository post, Gabriel Devenyi and Greg Wilson wrote some suggestions for how the new lessons repositories should look like (see Gabriel's post about metadata and Greg's post about overall file structure). From my experience at the Mozilla Science Lab sprint I don't like Gabriel's preq metadata because I don't believe it helps very much. I also don't like Greg's proposal to duplicate some files in every Git repository, so here are some changes that I suggest.

Announcing the Creation of the Software Carpentry Foundation

Wed, 08 Oct 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

In order to foster Software Carpentry's continued growth, we are pleased to announce that we are creating an independent Software Carpentry Foundation (SCF). Like other non-profit open source foundations, it will decide Software Carpentry's overall scope and direction, manage its finances, and hold its intellectual property.

In order to work through the details have assembled an interim board drawn from a wide cross-section of our community:

Jenny Bryan (University of British Columbia)
Neil Chue Hong (University of Edinburgh / Software Sustainability Institute)
Carole Goble (University of Manchester / ELIXIR UK)
Josh Greenberg (Sloan Foundation (non-voting))
Katy Huff (University of California Berkeley)
Damien Irving (University of Melbourne / Research Platforms)
Adam Stone (Lawrence Berkeley National Laboratory)
Tracy Teal (Michigan State University / Data Carpentry)
Kaitlin Thaney (Mozilla Science Lab)
Greg Wilson (Software Carpentry)

This group's mandate is to draft the SCF's initial bylaws and get the foundation legal standing, then arrange the transition to the first permanent board some time early in 2015. Until then, we will continue to do what we have always done: teach scientists and engineers how to use computers to do more research in less time and with less pain.

ARCHER Software Carpentry workshop at The University of Edinburgh

Tue, 07 Oct 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

ARCHER, the UK's national supercomputing service, offers training in software development and high-performance computing to scientists and researchers across the UK. As part of our training service we are running a 2 day Software Carpentry workshop at EPCC, The University of Edinburgh, UK, on 3-4 December.

Ideas to Improve Instructor Training

Sun, 05 Oct 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Have you ever learned something new and then had it appear in other areas of your life? After a summer at SWC thinking about how to train better instructors (and how to be a better teacher myself) I get to try discussion-based teaching this quarter at UC Davis.

A New Template for Workshop Websites

Sat, 04 Oct 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The first step in reorganizing the bc repository is making it easier (much easier) for people to create websites for workshops. The current instructions are almost 3000 words long, and even experienced GitHub users find the process daunting, so we're going to simplify things as much as we can, even if that means not doing things the "right" way.

A Reproducible Science Hackathon

Sat, 04 Oct 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

NESCent is organizing a "Reproducible Science Hackathon" where participants will develop material and tools for teaching and facilitating a broad adoption of reproducible science. The deadline for application is next Friday (Oct 10th); see the full announcement for details.

Browsercast

Sat, 04 Oct 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As we mentioned back in April, Gabriel Ivanica spent the summer working on a Google Summer of Code project called Browsercast. His goal was to build a web-native alternative to screencasts; if you'd like to see the result, head over to the demo page, click on the eye icon in the upper left, and press the "play" button. I hope to use it, or something descended from it, to build searchable, accessible narrations of our lessons that will play well on everything from desktops through tablets to mobile phones. If you'd like to help, please fork the project on GitHub and dive in.

Congratulations to the Moore Investigators

Sat, 04 Oct 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Gordon and Betty Moore Foundation has just announced $21 million in grants to fourteen investigators in the emerging field of data-driven discovery. Among the recipients are Titus Brown and Ethan White, who have both been key contributors to Software Carpentry. Congratulations to them and to the other award winners—we look forward to working with them in the years to come.

Studying Impact

Sat, 04 Oct 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I am a Software Sustainability Institute Fellow, and am using my fellowship to work on making software training better. My main interest is in teaching computational chemistry software and making it accessible to a wide range of scientists. Like Software Carpentry, the courses I teach are one-off teaching events: basically, we have a maximum of two days to help your professional development.

Together with Greg Wilson, I am conducting research to analyse what impact Software Carpentry workshop have had on participants with respect to their professional skills. In particular, I would like to interview past participants in Software Carpentry workshops to analyse the impact those workshops have had on them. We cannot reimburse you for your time, but if you are willing to take part in an interview, your experience will help us make Software Carpentry better. The interviews can be done via telephone or Skype and can be arranged at a convenient time; we have ethics approval for this study, and every participant will be informed what exactly we do with their data.

Please contact me by email if you wish to take part, and pass on this request to other workshop alumni.

Thanks in advance,
Alexandra Simperler

Welcome Our New Instructors

Fri, 03 Oct 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are very pleased to welcome a new batch of instructors to our team:

Interim Board Meeting: Sep 30, 2014

Tue, 30 Sep 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Software Carpentry Foundation Interim Board Meeting: Sep 30, 2014

Splitting the Repository

Mon, 29 Sep 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

United Airlines messed up my travel again last weekend, so I finally had a chance to think some more about how Software Carpentry works and how we can make it work better. Having topic maintainers is one improvement; another, which was discussed at this month's lab meeting, is to break the bc repository that holds our lessons and workshop home pages into smaller and more manageable pieces.

UCOSP as a Model

Sun, 28 Sep 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Software Carpentry's two-day workshops are just one of many ways to teach people practical skills. Term-long group projects are another model that I'm very fond of, and earlier this year, the four people who've been running the UCOSP program in Canada wrote a paper about what they've learned. Some of the lessons rhyme with what we've learned from Software Carpentry, but other insights are new. If you know of papers describing lessons learned from other innovative teaching projects, pointers in the comments section would be very welcome.

September 2014 Lab Meeting Report

Fri, 26 Sep 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

After a two-month break for a sprint and some holidays, we held another monthly lab meeting this week. About 50 people showed up to talk about issues large and small; the key points are below.

Feedback from Imperial College London

Wed, 24 Sep 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

On 16-17 September, EPCC's ARCHER headed down to Imperial College London to run a Software Carpentry bootcamp. My colleague Arno Proeme made his instructor debut, covering version control and Git and good programming practice, while I covered shell hints and tips, automation and Make, and testing.

Learning Goals

Tue, 23 Sep 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A few weeks ago, Greg Wilson asked me how to better express the learning objectives listed in the Software Carpentry lessons. My main concerns with the existing goals are that they focus too much on specific skills, rather than attitude changes, and is that they are generally stated as lesson descriptions (more like instructor goals or a lesson outline) rather than as learning goals for participants.

A Proposal for Topic Maintainers

Thu, 18 Sep 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I can lift ten pounds. I can even still lift a hundred pounds, though my back won't thank me, but I can't lift a thousand pounds. Similarly, while I was able to review changes to our lessons when only half a dozen people were contributing, there are now almost sixty pull requests queued up, some of which have been waiting for attention for several months.

The solution we'd like to try, which is borrowed from other open source projects, is to give specific people authority to maintain specific parts of our core material. These maintainers will not be responsible for developing lessons themselves, though they may of course do so. Instead, their job will be to keep issues and pull requests moving by reviewing, managing discussion, merging into the master branch, and so on.

How to Prepare for the Data Incubator

Wed, 17 Sep 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

At The Data Incubator, we receive thousands of applications to join our data science fellowship. Our admissions bar is very high and we are often asked, "What can I do to prepare for the fellowship application process?"

Here are five important skills to develop and some resources on how to help you develop them. While we don't expect our applicants to possess all of these skills, most applicants already have a strong background in many of them.

Interim Board Meeting: Sep 16, 2014

Tue, 16 Sep 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Software Carpentry Foundation Interim Board Meeting: Sep 16, 2014

September 2014 Lab Meeting

Fri, 12 Sep 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

After taking a break in July for our sprint, and in August because people were on holiday, doing fieldwork, or both, we will resume our monthly online lab meetings this month. As always, we'll hold the meeting twice to accommodate time zones, childcare commitments, and the like; the first round will be 18:30-19:30 Eastern time on Thursday, Sept 25, and the second will be 11:00-12:00 Eastern time on Friday, Sept 26. We'll post connection details and the agenda closer to the time; if you have anything you'd like included, please send us mail.

Videos from Stanford

Fri, 12 Sep 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Stanford University has posted videos from the Software Carpentry workshop held there in August featuring Azalee Bostroem, Chris Lonnen, Dani Traphagen, and Chris Beitel. These are a great place to start if you'd like to do a little informal jugyokenkyu.

Our thanks to Amy Hodge and Stanford University for making these available.

Further Thoughts on Building Better Teachers

Wed, 10 Sep 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

On Greg's recommendation, I just finished reading Building a Better Teacher - it was an interesting book, and a well-written story. It turns out that I completely agree with Greg's assessment of the book's lessons and the challenges facing Software Carpentry, but I very much disagree with his proposed solutions!

More Thoughts on Better Teachers

Wed, 10 Sep 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Over the last few weeks I read Greg's blog post on Building Better Teachers and watched his Scipy keynote and I've been thinking a lot about how can instructors share their knowledge.

Software Carpentry Workshop at Universidade of São Paulo

Tue, 09 Sep 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Last week Alex Viana and I taught two Software Carpentry workshops at the Universidade São Paulo, the biggest public university in Brazil. In both workshops we took them through Bash, Git and Python over the course of two days.

An Update on Upcoming Bootcamps

Sat, 06 Sep 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

So it turns out we have more events coming up than I realized:

Building Better Teachers

Thu, 04 Sep 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Some books are intrinsically great. (I've read Going Postal half a dozen times, and enjoyed it just as much at each encounter.) Other books feel great because they hit you at the right time. Elizabeth Green's Building a Better Teacher is one of those: in a little over 300 pages, she takes a bunch of things I've been worrying about for the last two years and assembles them into a coherent whole. The end result is a road map of sorts for making our teaching more effective; the problem is, I don't know if it's a path we can actually follow.

Instructor Training at UC Davis in January 2015

Wed, 03 Sep 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Thanks to support from Prof. Titus Brown, I will be running an intensive in-person version of our instructor training course at UC Davis on January 6-7, 2015. This will be followed on January 8 by a third day of activities focusing on teaching computation and biology; for details, please see Titus's post and keep an eye on this blog for more information as well.

Nature Interview with Kaitlin Thaney

Wed, 03 Sep 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Nature has just published a short interview about Software Carpentry with Kaitlin Thaney, the director of the Mozilla Science Lab, and two bootcamp attendees (Harvard's Rebecca Perry and USNIDA's Brian Sadacca). As always, version control seems to have been one of their big take-aways, but as Kaitlin says, two days isn't enough for people to become proficient: our bootcamps really are just the first step.

Open Source Comes to Campus

Wed, 03 Sep 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Open Source Comes to Campus helps the next generation of open source contributors get started by teaching one-day workshops aimed at undergraduate and graduate students from all disciplines. This workshop introduces students to the tools and culture of open source development. We familiarize students with tools such as IRC, issue trackers and version control; talk plainly about aspects of open source that might be confusing or intimidating; introduce them to open source professionals who can talk about career and educational opportunities; and help them make their first contributions to open source projects.

OSCTC will be hosting events this fall at over a dozen schools, including the City College of San Francisco, Bucknell University, the University of Victoria, Hartnell College, the University of Washington, and the University of Connecticut. We're always looking for volunteers to mentor at our events, and if you want to bring an event to your school, please get in touch.

Fall 2014 Bootcamps (So Far)

Sat, 30 Aug 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Our calendar for the fall is filling up pretty quickly : we've added the following bootcamps in the last couple of weeks, and have more on the way. If you'd like us to run one where you are, please fill in this form and we'll get the ball rolling.

Software Carpentry workshop at Universidade Federal do Rio Grande do Sul

Sat, 30 Aug 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

At the end of this week Alex Viana and I taught a Software Carpentry workshop at the Universidade Federal do Rio Grande do Sul. We had around 30 learners of very different experience (from high school students to seniors professors of the university) and took them through Bash, Git and Python over the course of two days.

Software Carpentry at Cambridge University

Fri, 29 Aug 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

At the beginning of this week Thomas Kluyver and I taught a Software Carpentry bootcamp at the University of Cambridge. We had around 35 learners of very different experience and took them through Bash, Git and Python over the course of two days.

The New MATLAB Teaching Materials

Fri, 29 Aug 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

When Software Carpentry started running bootcamps back in 2012, Python was used exclusively for the programming lessons. While these lessons were as language agnostic as possible (i.e. the materials focused on transferrable programming concepts as opposed to specifics of the Python language), people soon expressed an interest in running bootcamps using other languages. R very quickly established itself as a regular alternative to Python, but it wasn't until early 2014 that the first ever Matlab bootcamp was held (see here and here for blog posts about the event; the official event page is here).

Software Carpentry at Brazilian Open Science Conference

Sat, 23 Aug 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Last week, Raniere Silva and Alex Viana attended at Brazilian Open Science Conference where they ran a Git course for Software Carpentry and Alex gave a talk about Software Carpentry (like the one that Damien Irving gave at PyCon Australia). Our workshop was a great time and from the feedbacks every student like it. One of the students said:

Very didactic, the instructors showed much care, dedication and attention with the students.

and another one

Very good, from the basic to complex in a easy way.

The Fifth ANGUS Course

Thu, 21 Aug 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Titus Brown recently blogged a summary of the fifth run of the Analyzing Next Generation Sequencing (ANGUS) course at Michigan State. It includes some interesting observations on what's working and what needs to be improved, and some thoughts on assessment—he'd welcome feedback.

Conversations About Teaching

Mon, 18 Aug 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Over the last few days, there have been four related discussion threads on the Software Carpentry mailing lists about what we use, what we teach, and how we teach it. Together, they highlight what we're doing well and where we need to do better.

A MOOC on Practical Numerical Methods with Python

Thu, 14 Aug 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As announced at SciPy'14 last month Prof. Lorena Barba will be teaching a MOOC titled Practical Numerical Methods with Python this fall, and the course site is now open for registration. Having worked through her excellent 12 Steps to Navier Stokes notebooks, I think this will be a great course, and I urge everyone interested in the subject to check it out.

News from Australia

Wed, 13 Aug 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Damien Irving summed up Software Carpentry activities in Australia in a short talk at PyCon Australia last week, and talked a bit as well about lessons learned. Next stops: Perth and Darwin!

Three Bootcamps for Librarians

Wed, 13 Aug 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Over the past month and a half I have been fortunate to (co-)lead three bootcamps targeted at librarians, a relatively new audience for software carpentry. These bootcamps have been located in Edmonton, Toronto, and New York, and in the same way that the term "librarian" is very broad encompassing many different disciplines and skill sets, the three bootcamps had their share of similarities and important differences. I'll start with similarities and then discuss each bootcamp's differences.

Inessential Weirdness in Software Carpentry

Mon, 11 Aug 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Sumana Harihareswara, of the Wikimedia Foundation, has started compiling a list of inessential weirdness in open source. (It was inspired by this article, which made me blush: I was a howler more than once in my youth.) My question is, what inessential weirdnesses are we guilty of that aren't already in Sumana's list?

UCL Research Software Development is Hiring

Mon, 11 Aug 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The University College London Research Software Development team is hiring.

The Research Software Engineer AGM and Hackday

Fri, 08 Aug 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Research Software Engineers community was founded to support the people who develop the software used in research. If you want be a part of the community, come to our AGM and hackday which takes place on 15-16 September at King's College London. It's a free event thanks to our sponsor Maudsley Digital.

Sustainability

Mon, 04 Aug 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I took part in a meeting about sustainable scientific software at last month's SciPy conference. Much of it was taken up with discussion of getting scientists recognition for building software, but there was also some interesting debate about what "sustainability" actually means. After talking to a few people in the software engineering research community, I'd like to propose that:

The Real Purpose of Sprints

Mon, 04 Aug 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

"The real purpose of sprints isn't the code, it's getting to know people..." This is what Greg Wilson told me after the first-ever Mozilla Science Lab global sprint. Indeed, over the course of two days, dozens of people all over the world met virtually and in person to develop and improve Software Carpentry teaching materials, infrastructure and a number of MSL-related projects.

Summer Sprint Summary

Tue, 29 Jul 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Last week, the Mozilla Science Lab hosted its first-ever global sprint. Dozens of people joined in from 18 cities (and a few from home) to work for two days on a wide variety of projects, ranging from new lessons to tools for mining the scientific literature. The MSL blog has all the details, and will have a series of posts about what was done over the next couple of weeks, but here are a few Software Carpentry-specific highlights:

Feedback from Cranfield

Mon, 28 Jul 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

On 21-23 July, EPCC's ARCHER training team visited a sun-drenched Cranfield University to run a Software Carpentry bootcamp and Introduction to Scientific Programming in Python. The 3 days combined a traditional bootcamp with a new day course providing an Introduction to Scientific Programming in Python.

GSoC Projects at Summer Sprint

Mon, 21 Jul 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As you may know, the Mozilla Science Lab's first-ever two-day sprint is in the next few days and all GSoC projects related with Mozilla Science Lab will be part of it. For more information about the plans for the GSoC projects continue reading this post.

SciPy 2014 Talks and Lessons

Mon, 21 Jul 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Talks from SciPy 2014 are now online, and include several from people associated with Software Carpentry. I particularly enjoyed Lorena Barba's keynote, in which she discussed how she's using the IPython Notebook in a flipped classroom. (I gave a talk too, but if you're a regular reader of this blog, you'll have heard most of it already.)

Summer Sprint FAQ

Mon, 21 Jul 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Mozilla Science Lab's first-ever two-day sprint is less than three weeks away, so here's a short FAQ to tell you who can take part and how.

Using a Package Manager for Lessons and Papers

Mon, 21 Jul 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I've been musing for a couple of years now about ways in which we could re-purpose off-the-shelf software engineering tools and techniques to serve the needs of teachers. One theme, which I touched on in my SciPy 2014 talk, is to get people to patch shared learning materials in the way they patch Wikipedia articles and open source code. Another is to use package managers like RPM, Homebrew, and Conda to track dependencies between lessons, so that I could say something like conda install suffragette_movement and get a lesson on the struggle for women's voting rights, along with the other lessons and materials it depends on (or updates and links to those other lessons if I already have some of them installed).

Bootcamps in Cyprus and Jordan

Thu, 17 Jul 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Last month, we held two bootcamps in Cyrpus and Jordan organised by the LinkSCEEM project. Both camps were part of larger workshops, with the final two days of the workshop in Cyprus focussed on the OpenMP and OpenACC APIs for shared-memory multi-processing and accelerators. It was held June 10-13 with 35 participants coming from 6 different countries (Cyprus, Greece, Lebanon, Jordan, Israel, and Egypt) with all the non-local participants being funded by the project.

ARCHER Software Carpentry Bootcamp at Imperial College London

Fri, 11 Jul 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

ARCHER, the UK's new national supercomputing service, offers training in software development and high-performance computing to scientists and researchers across the UK. As part of our training service we are running a 2 day Software Carpentry bootcamp at Imperial College London, UK, on 16-17 September.

Translating Software Carpentry into Portuguese

Tue, 08 Jul 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are very pleased to announce that we will run a half day workshop/advertising session at Open Science, Open Issues, an open science conference at Rio de Janeiro, and for that we will translate our Git lesson into Portuguese. If you If you would like to help, please head over to this GitHub repository, which has instructions and contact information.

Our First High School Workshop at Rockefeller University

Mon, 07 Jul 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Camille Avestruz, Ivan Gonzalez, Timothy Cerino, and Daniel Chen all had the great opportunity to teach Software Carpentry's first zero-entry workshop to high school students. We were able to teach at Rockefeller thanks to the scientific foresight of Jeanne Garbarino and the rest of the Rockefeller team along with Arliss Collins, Greg Wilson and the SWC team. Lastly, thanks to Gabriel Perez-Giz for volunteering his time to help during the workshop.

The main goal of this workshop was to expose tomorrow's scientists to scientific computing as early as possible. For example, as genomics data for biology continues to grow, we are beginning to see a shift of biologists from the pipetter to the data scientist. Our goal was not to teach everyone all the skills needed so they can dive into retrieving, cleaning, and analyzing genomics or astronomy data the next day, but rather show them what is possible with computers, and expose them for the first time that the GUI may not always be the best tool for the job; and give some foundation of knowledge and concepts for perpetual self learning.

We followed the traditional SWC workshop materials, adapting the pace as needed. Bash, Python, and Git were covered.

Scientific Groupware Revisited

Sat, 05 Jul 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

16 years ago, Jon Udell wrote a white paper titled "Internet Groupware for Scientific Collaboration" that profoundly changed how I thought about the web. In many ways, it seems as futuristic today as it did then: despite all the technological advances of the last decade and a half, a two-way web built on top of a universal canvas is still mostly a dream.

Udell and others are now working on Thali, an attempt to create a truly distributed web. It, and projects like Ward Cunningham's Smallest Federated Wiki, are a more truly open model for science than the de facto centralization typified by Google, Facebook, Twitter, and GitHub, but it's not clear we'll choose the long-term robustness of the former over the short-term convenience of the latter.

I asked Jon if I could re-post IGSC here. He said yes, but on reflection, I realized that doing so would run counter to the spirit of what he's been trying to tell us. Instead, I urge you to click on this link and marvel at the everyday miracle that follows.

Feedback from the bootcamp at Istituto Nazionale di Fisica Nucleare in Pisa

Mon, 30 Jun 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

At the beginning of June (3-6) Software Carpentry supported the leaning tower of Pisa. Well, no really. We actually supported the students at Istituto Nazionale di Fisica Nucleare at the University of Pisa in becomming more effective using their computational skills. The instructors were Rémi Emonet and Aleksandra Pawlik. The main organisers and hosts were Chiara Roda and Luca Baldini. Needless to say, running a bootcamp early June in Tuscany does have its perks. The memories of gelati and focaccine di ceci are still very vivid. How can you not love being a Software Carpentry instructor?

Our IUSE Proposal Was Rejected

Fri, 27 Jun 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We got word a few days ago that our proposal to the NSF's Improving Undergraduate STEM Education program had been rejected. The panel summary agreed that software training is a good idea, but were not convinced about our plans to shift from training grad students to undergrads. In particular, they were not convinced that Unix-based workshops would be best for undergrads, and felt that not being embedded in the regular curriculum was a weakness. These are fair criticisms:

Most undergraduates only use GUIs and cannot navigate the terminal to save their lives. The perception that Software Carpentry is Unix-specific therefore hurt the proposal, and showed that we didn't clearly explain our focus on underlying concepts.
The fact that we wouldn't be part of the regular curriculum is more difficult to address. The fact is, it really is hard to get undergrads to do things that interfere with earning grades, just as it's often hard to get grad students to do anything that doesn't immediately lead to a publication. Some of their concern about impact also seemed to be due to us concentrating on REU students, a self-selected bunch that already are motivated.

At the same time, though, the panel did not connect the results from years of Software Carpentry workshops and the expected impact of this effort. This indicates that the proposal did not effectively communicate how well our experience to date has laid the groundwork for efforts like this.

We're obviously disappointed by this rejection, but we've some useful lessons, and we hope that they will inspire others to put forward proposals of their own.

Summary of June 2014 Lab Meeting

Fri, 27 Jun 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

At our monthly lab meeting yesterday (June 26), we discussed a wide range of topics (though only about half as much as was on the agenda). Details notes and votes are below; the key points are:

We would like to do what we can to help the careers of people who volunteer for Software Carpentry. In particular, we'd like to find a way to highlight people who have taught lots of bootcamps, contributed lots of material, or both, without discouraging people who haven't, or getting into a situation in which bootcamp hosts ask, "Why aren't you sending your most experienced people to us?" We've opened a GitHub issue for people who'd like to discuss this topic, and we'd be grateful if you could share experiences with other systems and suggestions about what we could do.
We will have a meeting on Thursday, July 3 to discuss plans for the July 22-23 sprint. The time and connection details will be posted here on July 2.
People were either neutral or in favor of switching to Python 3. There was concern about compatibility, but it seems that most scientific packages are now Py3-compatible (and that there are some that only work with Py3, and not with Py2). In practice, this will mean:
- changing what people have on their machine when they leave the bootcamp, and
- minor modifications to our lessons (e.g., print as a function).
By default, Mercurial launches a GUI diff/merge tool when there's a conflict rather than displaying a textual diff. Novices seem to find side-by-side merge easier to work with, which raises the question: should we use a graphical tool for diff and merge when teaching Git as well? Again, most people were either neutral or in favor, subject to ease of installation. We'll try this at a couple of workshops in the coming months and report back.
Should we install Matt Davis's ipythonblocks module by default? Nobody was opposed, but it turned out that only a handful of people actually use it in teaching. We'll poll our instructors to find out more about usage before making a final decision.
In order to make it easy for people who want to teach regular expressions to do so, we'll integrate the software that runs regexpal.com (which ironically is down at the time of this writing) into the bc repo so that instructors can create a page in their bootcamp site where learners can play around.
Jonah Duckles has created a pull request that would change the way installation instructions are managed. Instead of editing _includes/setup.html, instructors would add a list of topics to the header of index.html that Jekyll's templating logic would use to control what is and isn't displayed. Most people were in favor, so we'll merge this once the PR is finished.
Many people are still not happy with the way our lesson materials are organized, or with what pre-requisite knowledge we actually expect. We will organize a meeting for next Thursday (July 3) to talk about options, then open a discussion issue on GitHub if we can come up with concrete alternatives.
Raniere Silva has been working over the last few weeks to generate EPUB and PDF from our materials. The goal is to allow anyone who wants to produce book-form notes to do so for use in a regular class, to give to learners after a bootcamp, or for self-study. He's been making good progress, but there are still lots of small problems (particularly with the PDF). We'll be hacking on this at the sprint.

Reminder: Lab Meeting Tomorrow

Wed, 25 Jun 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Our monthly online lab meetings will take place tomorrow (Thursday, June 26) at 10:00 am and 7:00 pm Eastern time. Please have a look at this Etherpad to see what we'll be discussing and voting on. We look forward to seeing you there.

Translating Software Carpentry into Spanish

Sun, 22 Jun 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are very pleased to announce that a group of volunteers have begun translating our lessons into Spanish. If you would like to help, please head over to this GitHub repository, which has instructions and contact information. Our thanks to Fran Navarro, David Perez-Suarez, Javier J. Gutiérrez, Iván González, Daniel Domene y Zuri Bauer, and Jorge Bernabé for organizing this, and to everyone else for lending a hand.

Help Us Build an Admin Tool for Bootcamps

Fri, 20 Jun 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

It takes a lot of work behind the scenes to match instructors to bootcamps, keep track of who taught (or learned) where, and manage our mailing lists. Our current cobbled-together tools are at their breaking point, so Atul Varma, a developer at Mozilla, put together a simple browser-on-the-desktop app to help administrators find instructors based on geography and skills. We'd like many other features—in particular, bulk upload of attendance lists from bootcamps—so if you would like to help us out, please clone this repository and send us fixes and additions.

Many thanks to Atul for getting us started—we're looking forward to not grepping for instructors any more :-).

Reflections on Claremont

Thu, 19 Jun 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Claremont Colleges bootcamp just wrapped a couple of hours ago, and judging from my pile of happy sticky note responses, I think we nailed it pretty good. Things were a bit fast on the first day of intro shell and Python, but I think we hit something pitch perfect for day two on intermediate Python and Git. As this Python-heavy workshop progressed, some questions started coming into focus from student responses, discussions with instructors and organizers, and examinations of just how to teach this ascendent skill in scientific computing. To be clear: I have never taught our Python material, and I consider myself a fairly casual Pythonista. This is a debate that the masters need to consider, but I'm hoping here to use my agnostic outsider's perspective to help set the conversation up.

Engineering-Focused Bootcamp

Wed, 18 Jun 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Summary: In advance of a bootcamp to be held for a Midwestern university's mechanical engineering department this fall, I've started reworking the traditional Software Carpentry (SWC) curriculum, and will be creating new teaching materials to accompany the workshop. Content changes are being made to accommodate the instructional needs of engineering students at the sponsoring university, while remaining consistent with the basic tenets and goals of the SWC movement. Comments and suggestions from the community on this matter are welcomed and encouraged, particularly in the area of PowerShell scripting (at which I'm currently a novice.) I'm also looking for instructors that might be willing to help teach such a bootcamp in the Detroit area this fall.

Teaching Support IT Job at UCL Physics and Astronomy

Wed, 18 Jun 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We're looking for someone with IT skills, and interests in teaching and physics, to work with us at the Department of Physics and Astronomy at UCL (University College London).

Other responsibilities may include maintaining and developing an IPython notebook server for teaching, and working with teaching staff to apply other innovative tools.

The application deadline is midnight on Sunday 6th July. Applicants must already have the right to work in the UK.

Reminder: June 2014 Lab Meeting

Tue, 17 Jun 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The next Software Carpentry lab meeting will take place next week at 10:00 am and 7:00 pm Eastern time on Thursday, June 26. (My apologies to people in Australia: I promised we'd hold the evening meeting first this time around, but unfortunately that slot conflicts with instructor training.) Our agenda is up on this Etherpad; we'll be discussing plans for the July sprint and voting on several pull requests and proposals. If you'd like to cast a few votes, please take a few minutes before the meeting to look through the items that are on the agenda.

A Double Handful of Bootcamps

Sun, 15 Jun 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The next couple of days are going to be busy, with bootcamps in Amman and Toronto, and at Duke, UC Davis, and Claremont Colleges. The week after will be just a crowded: we'll be at the universities of Reading and Notre Dame, the Canadian Applied and Industrial Mathematics Society, Rockefeller University in New York, and an event for women in science and engineering in Philadelphia. As always, we're grateful to our hosts and instructors for making all of this possible.

A Success Story from KAUST

Sun, 15 Jun 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Our ACM student chapter at King Abdullah University of Science and Technology (KAUST) is active in supporting the diverse research community at our university. We decided to offer Python tutorials to our community by the end of 2013. Due to no information on the background and interests of KAUST community, we crafted a short survey to help us in tailoring our tutorials. The following image shows the summary of the responses we got:

An Update on Our Sprint Plans

Sun, 15 Jun 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Plans for our two-day sprint in July are coming together: full details are being compiled on our Etherpad, but we hope the summary below will give you a taste of what we're hoping to do.

Fixing 14 Repositories

Sun, 15 Jun 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We cloned our workshop repository at the beginning of the workshop at Spelman to give everyone an identical directory tree and contents to explore, and to distribute the sample data files. On the first day of our workshop, about a third of the learners (all running Windows) could not clone the workshop repository: git clone was failing for them with the error:

Announcing June 2014 Lab Meeting

Mon, 09 Jun 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The next of our monthly online lab meetings will take place on Thursday, June 26, at 10:00 am and 7:00 pm Eastern time. (As always, we'll hold it twice to accommodate people's work schedules and time zones.) We will update this Etherpad with agenda items as the date approaches.

ARCHER Software Carpentry Bootcamp and Introduction to Scientific Programming in Python

Mon, 09 Jun 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

ARCHER, the UK's new national supercomputing service, offers training in software development and high-performance computing to scientists and researchers across the UK. As part of our training service we are running a 3 day Software Carpentry bootcamp and Introduction to Scientific Programming in Python at Cranfield University, UK, on 21-23 July.

Planning Our Summer Sprint

Mon, 09 Jun 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As a warm-up for our lab meeting on June 26, we have some more information to share about our two-day sprint in July. Please see our Etherpad for details, including more about sites and projects.

Registration Open for Instructor Training in Norwich in October

Mon, 09 Jun 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As we announced last week, we are running a two-day version of our instructor training course at The Genome Analysis Centre in Norwich, UK, on October 22-23, 2014. Registration is now open, and we look hope to see you there.

Collected Links

Thu, 05 Jun 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A lot of people have been writing about Software Carpentry on other blogs of late. Here, in no particular order, are a few of their posts:

Damien Irving: A vision for data analysis in the weather and climate sciences
Deb Paul: Tales from A Data Carpentry Workshop: In Demand
Tracy Teal: Inaugural Data Carpentry Workshop
Oxana Sachenkova: Software Carpentry at SciLifeLab, Sweden
Rob Davey: From the workshop to the workstation: Software Carpentry training from a bioinformatics perspective
Stephen Turner: Collaborative lesson development with GitHub
Karin Lagesen: The one where I went to Sweden
Damien Irving (again): Software Carpentry for Biomedical Imaging

Instructor Training in Norwich, October 2014

Thu, 05 Jun 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are pleased to announce that we will be running a two-day version of our instructor training course at The Genome Analysis Centre in Norwich, UK, on October 22-23, 2014. Registration details will be available shortly, but if you are interested in taking part, please mark your calendar and keep an eye on this blog, or the announcement page, for details.

Our thanks to the good folks at TGAC for making this possible—we look forward to an engaging couple of days.

Introducing Scientists to Testing and Code Review

Thu, 05 Jun 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As part of our two-day sprint in July, the khmer project will be offering a mentored open source contributathon. This will provide an opportunity for people interested in trying out the "GitHub flow" model, in which contributions are submitted for review using a pull request. Since the project has lots of unit tests and fairly high code coverage, people can also see how testing and code coverage interact with software development in practice. For more information, please see Titus Brown's blog post.

Keeping the Bootcamp Fun Alive!

Thu, 05 Jun 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

In the later part of 2013, I started the process of co-hosting a Bootcamp with Peter August and Judith Swift both from the University of Rhode Island (URI). During our discussions it became quite clear that as fantastic as the bootcamps are, it would be even better if we could extend the opportunities for learning. We tossed around several ideas but eventually decided we would try a special topics course at URI. Thus, Scientific Computing and Programming for Coastal Resource Managment was born. We opened the course up to any of the bootcamp participants who had an interest; official enrollment in the course was not required. In the end we had about 10 participants who attendend regularly, including a few students who enrolled for credit. Pete August (URI), Adam Smith (URI), and myself co-taught the course.

Research Computing Facilitator Jobs in Wisconsin

Thu, 05 Jun 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Advanced Computing Infrastructure at the University of Wisconsin-Madison seeks to hire two Research Computing Facilitators (RCFs). The ideal RCF will apply a background of academic research and the use of computation to assist campus researchers in enhancing their research endeavors through the use of on- and off-campus compute resources. Additionally, RCFs should possess the interpersonal skills to interface with researchers (faculty, graduate students, etc.) and campus technical staff from a variety of organizations and compute-related services, including the campus's large-scale computing center. Primary activities include consultations with researchers and the production of education/outreach materials distributed on the web and in person. Experience teaching research computing topics, such as those in Software Carpentry bootcamps, will be a plus. Please see the full description for more details. All activities will be bolstered by participation in the NSF ACI-REF program, where a primary goal is to build an inter-campus network of facilitators and technical staff.

Learning to Teach Never Ends

Wed, 28 May 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A month ago I took part in the first face-to-face Software Carpentry Instructors Training run by Greg Wilson from Software Carpentry and Warren Code from University of British Columbia. Unlike most of 40 participants, I had already completed the instructors course which Greg regularly runs online. My primary aim to attend the face-to-face training was to observe and learn how to run it because the plan is that the Software Sustainability Institute (which I work for) will support running such training in the future. So I attended the training in Toronto as a certified instructor and also as a Software Carpentry co-admin in the UK. Wearing different hats allowed me to look at the event from different perspectives and this is probably why it took me so long to write this post.

Teaching Librarians in Montreal

Wed, 28 May 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Preston Holmes, Jessica Hamrick, Luke Lee, and I helped deliver a Software Carpentry bootcamp during the PyCon sprints in Montreal in April 2014. The audience consisted of roughly 35 librarians coming mostly from the Montreal area.

Announcing Two More WiSE Bootcamps

Mon, 26 May 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Thanks to generous support from Waterfront International and Stevens Capital Management, we are pleased to announce that we are running two bootcamps for women in science and engineering this summer: one in Philadelphia on June 24-25, and the other in Toronto on July 8-9. Both bootcamps are open to female graduate students, post-docs, and faculty in science, engineering, and medicine, particularly those interested in analyzing large data, and build on previous WiSE bootcamps in Boston in 2013 and Berkeley earlier this year.

Data Science Study Invitation

Sat, 24 May 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Katie Kuksenok, a graduate student at the University of Washington, is interviewing academic researchers who do data science to explore the barriers and challenges they face. If you would be willing to take 20-30 minutes to be interviewed, please get in touch: among other things, her findings will be used to help improve curriculum for projects like Software Carpentry.

Summary of May 2014 Lab Meeting

Sat, 24 May 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Our monthly online lab meeting took place this past Thursday (May 22), and for the first time it included voting on pull requests and other issues. All the notes from the Etherpad are included below, but the high points are:

The Mozilla Science Lab is hiring a developer and a community manager.
People and projects that would like to take part in our global sprint on July 22-23 are invited to sign up on this Etherpad. We won't just work on Software Carpentry curriculum and tooling: related projects are welcome to use this opportunity to bring their communities together as well.
Arliss Collins and others will try to simplify the workflow for creating and managing bootcamps. (Right now, we rely on five different online systems, and most of the administrators we work with at host institutions can't make heads or tails of them.) If you would like to help, please let us know.
We will add people who want to be helpers at bootcamps to our instructors mailing list rather than creating yet another list for reaching them.
We voted on the following pull requests:
- Extra lessons will go under existing directories (e.g., novice/git/) rather than in a top-level extras directory.
- We won't try to standardize usage of "parameter" and "argument", since most instructors use them idiosyncratically and/or interchangeably.
- We will incorporate the new lessons on Mercurial as soon as they're done.
- We will merge the lessons on scikit-learn, Python string formatting, common Python error messages, and setting up SSH keys for GitHub.
- We won't include the lesson on tmux—people felt it was too specialized—and will ask that the lesson on text data mining in the shell be re-worked.
We also voted on the following proposals for new lessons:
- Using Excel properly, using Make to manage data pipelines, and regular expressions were all approved.
- The draft lesson on creating and syndicating data on the web was deferred (only a few people had looked at it).
- People liked the idea of lessons on statistics with Pandas and managing geospatial data, but we will need volunteers to take the lead.

Our next lab meeting, on June 26, will primarily be devoted to planning for our July 22-23 sprint. We look forward to seeing lots of you at both.

Lab Meeting Reminder

Wed, 21 May 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Our monthly lab meeting will take place on Thursday, May 22 at 10:00 am and again at 7:00 pm, Eastern time. There's a lot to discuss, so please take a moment beforehand to go through the lab meeting Etherpad and see if there are any pull requests or enhancement proposals that you are particularly interested in.

We look forward to seeing you online.

A Lot of Bootcamps in the Works

Tue, 20 May 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Last week, we ran our first bootcamp in Brazil—many thanks to Raniere Silva for this writeup about what worked and what didn't. We'll visit several other new countries over the next three months, as well as running a bunch of other events—we'll post again as registration opens up for these:

Behind the Scenes

Tue, 20 May 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A lot of people work behind the scenes to organize bootcamps, keep our website going (and readable), and generally make it possble for the rest of us to do what we do. I'd therefore like to offer some long-overdue thanks to Amy Brown, Arliss Collins, Abby Cabunoc, Ivan Gonzalez, Jon Pipitone, David Rio, and Raniere Silva for all their hard work: we wouldn't have come this far without them, and we're very grateful for everything they've done.

Job Openings at the Mozilla Science Lab

Wed, 14 May 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Mozilla Science Lab is looking for a community manager to build and scale existing community outreach efforts, and for a developer to lead technical prototyping efforts and engage with our community about technical projects. Possible office locations for these positions include Brooklyn, Toronto, London, Vancouver, and San Francisco, but we will consider remote working opportunities for the right candidate. For more information, or to apply, please see these postings.

Our First Data Carpentry Workshop

Wed, 14 May 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Update: for more information on Data Carpentry, please see their web site.

On May 8 and 9, 2014, 4 instructors, 4 assistants, and 27 learners filed into the largest meeting space at the National Evolutionary Synthesis Center (NESCent) for the inaugural Data Carpentry bootcamp. Data Carpentry is modeled on Software Carpentry, but focuses on tools and practices for more productively managing and manipulating data. The inaugural group of learners for this bootcamp was very diverse. They included graduate students, postdocs, faculty and staff, from three of the largest local research universities (Duke University, University of North Carolina, and North Carolina State University). Over 55% of the attendees were women and research areas ranged from evolutionary biology and ecology to microbial ecology, fungal phylogenomics, marine biology, and environmental engineering. One participant was even a library scientist from Duke Library.

Technical Training Officer (LVT Training Officer) position at TGAC

Wed, 14 May 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Genome Analysis Centre (TGAC) based at Norwich, UK is looking for a Linux and Virtualisation Technical Training Officer who will join the Training and Outreach Team.

Agenda for This Month's Lab Meeting

Tue, 13 May 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The next Software Carpentry lab meeting will be held at 10:00 Eastern on Thursday, May 22, and repeated at 19:00 Eastern on the same day. (You can use this site to translate those times into your time zone.) We will be voting on several small additions to the lesson material—please see the Etherpad for the list—and discussing plans for future additions, and the multi-site sprint in July. Conference call details will be on the Etherpad on Thursday morning; please add other agenda items to the pad, and we look forward to seeing you all then.

A Training Veteran Weighs In

Thu, 08 May 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I'm not sure if I qualify as a "instructional training veteran", but having participated in both the online Software Carpentry (SWC) instructor training, and two versions of the Instructional Skills Workshop (ISW) (on which the latest "live" SWC training was modeled) at the University of British Columbia, I'd like to share a few comments comparing my experience with both and what I've valued from each. Feel free to add your own impressions of the SWC or other instructional training in the comments at the end.

Assessment Results: First Batch

Thu, 08 May 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As many of our instructors and participants know Software Carpentry has been giving surveys for a number of months at every possible workshop. This takes a surprising amount of coordination and attention, much of which we owe to our wonderful two administrators: Amy Brown and Arliss Collins. The surveys are being administered before and after each workshop in an effort to gather difference data which could answer questions such as "How useful was this workshop to participants?", "Who benefits most from these workshops?", or "What ways could we improve the workshop?" We can also use this information to ask about the professional demographics such as "What fields are most represented by our participants?" and "What's the typical amount of programming experience?" both of which could help inform a tailored workshop experience to better suit the audience.

Knocking on the Future's Door

Thu, 08 May 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Once again I feel like I'm knocking on the future's door but nobody's answering. The task we set ourselves seemed simple: produce a nicely-formatted PDF of the Version 5 lessons to give learners as a reference (and to print as a book to give instructors when they finish their training). Fifty years after the creation of the first computer typesetting systems, you'd think this would be easy. It's not, and the reasons why highlight yet again why so many scientists would rather keep playing the kazoo than learn to play the violin.

A Multi-Site Sprint in July

Mon, 05 May 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We'll be holding our first-ever global sprint on July 22-23, 2014. This event will be modeled on Random Hacks of Kindness: people will work with friends and colleagues at sites around the globe, then hand off to participants west of them as their days end and others' begin. We will set up video conferencing between the various locations and a show-and-tell at the end (and yes, there will be stickers and t-shirts).

We have booked space for the sprint at the Mozilla offices in Paris, London, Toronto, Vancouver, and San Francisco. If you aren't in one of those cities, but are willing to help organize in your area, please add yourself to this Etherpad. We'll hash out the what and how at the next lab meeting—it's a community event, so we'd like the community to choose what to sprint on—but please get the date in your calendar: it just wouldn't be a party without you.

Playing the Kazoo

Mon, 05 May 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Yesterday, Matt Davis quoted Peter Wang as saying, "A violin is to a kazoo as Python is to Excel." To which I replied, "Exactly: anyone who wants to make music can play a kazoo right away without days of training." The difference between these two points of view lies at the heart of Software Carpentry. As I said in a post two years ago:

How to Improve Instructor Training

Fri, 02 May 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We ran a three-day intensive version of our instructor training course in Toronto earlier this week. The 40 attendees seemed to find it useful, and I'm very grateful to UBC's Warren Code for co-teaching, and to the University of Toronto's Jennifer Campbell for her presentation on MOOCs and flipped classrooms, but there are a few things we'll do differently next time around. They're listed below in no particular order; comments from attendees (and everyone else, particularly people who've been through the online instructor training) would be very welcome.

Wise as Athena...

Thu, 01 May 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Katy Huff has written an article for the Berkeley Science Review about our recent bootcamp for women in science and engineering in California. It's a good summary of what Software Carpentry is all about as well—please check it out.

PyCon 2014 Videos

Sun, 27 Apr 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A double handful of people associated with Software Carpentry gave talks at PyCon 2014 in Montreal two weeks ago. Thanks to Sheila Miguez and Will Kahn-Greene (who run the excellent PyVideo site), you can now view their presentations:

April 2014 Lab Meeting

Fri, 25 Apr 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We held another online lab meeting yesterday, which covered a fairly wide range of topics. The notes are below; comments about what we missed, and suggestions for the next lab meeting, are very welcome.

Position: Systems Integration Developer at UW-Madison

Thu, 24 Apr 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Center for High Throughput Computing (CHTC) at the University of Wisconsin-Madison provides compute resources and services to campus researchers at UW-Madison, and also develops the HTCondor computational scheduling software, which is used all over the world. We are seeking to hire a Systems Integration Developer, who will help us to accelerate the work of researchers through computational middleware scripting and the design of workflow tools. Please see the online posting at http://www.ohr.wisc.edu/WebListing/Unclassified/PVLSummary.aspx?pvl_num=78987 for details.

Import Lesson

Wed, 23 Apr 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Lorena Barba has done it again. Having created a wonderful 12-step introduction to the Navier-Stokes equations using the IPython Notebook, she has now published AeroPython, which teaches the use of potential flow for aerodynamic analysis via the panel method. You don't have to know what that is to appreciate the beauty of what she has built—I certainly don't, at least not yet—but I'd like to explore something that isn't in those notebooks. Before doing that, though, I need to introduce an acronym that never caught on.

Math Authoring Gap and MathUI

Wed, 23 Apr 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

In a previous post Greg Wilson wrote about the writing gap

between people who value the "I can see what I'm doing" of Microsoft Word and those who care more about the "I can tell what I did" of version control.

For STEM (science, technology, engineering and math) folks there is also the gap related to math expressions (you can use LaTeX to deal with math expressions but for beginners who spent around 20 years using Microsoft Word moving to LaTeX can be challenging).

Do you know that some people are working to solve the gap with math expressions?

Mr. Biczo Was Right

Wed, 23 Apr 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I didn't have nearly enough time to enjoy everything that was going on at PyCon 2014 last week. One event I particularly regret missing was a sprint organized in part by the folks at Scrapinghub. They got a bunch of people to write little scrapers to go through old conference websites, pull out speakers' names, and run them through a gender identification library in order to plot changes in gender balance per conference over time. The code is all available on GitHub, and Gayane Petrosyan (currently at the Hacker School in New York City) has started plotting some of the results.

GSoC Projects for 2014

Tue, 22 Apr 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We're very pleased to announce that two students will be working on Google Summer of Code projects we put forward, and that one of our regular contributors has received a GSoC as well. The projects are:

Office Hours for Code as a Research Object

Tue, 22 Apr 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Looking to learn more about how to get a DOI for your code? Interested in the underlying technology used to hook up GitHub (a code hosting service) and figshare (an open data repository)? Join us for our online office hours on Thursday, April 24. We'll be shedding light on the technical build done as part of our "code as a research object" project.

We'll be holding two sessions, one at 11 am ET for those in North America, Europe, and Africa, and another at 4 pm ET for our Australasian colleagues. These sessions are open to all, and are your chance to ask your implementation questions, gain a better understanding of how the prototype works, and peek under the hood at the codebase.

Have a question? Add it to the etherpad, where you can also find useful background on the project as well as dial-in information. We hope you'll join us.

Software Carpentry bootcamp at GARNet

Tue, 22 Apr 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The second week of April the University of Warwick in the UK hosted its first Software Carpentry bootcamp. The bootcamp was organised by GARNet which is a UK-based research network for the UK Arabidopsis and wider plant research community. GARNet facilitates collaboration and interaction between different researchers and supports skills development.

Workshops at SESYNC

Sat, 19 Apr 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Applications are now open for the 2014 Computational Summer Institute at the National Socio-Environment Synthesis Center (SESYNC). Small teams of researchers are invited to apply for spaces at the one-week institute which will be held July 7-11 in Annapolis, Maryland. The workshop will offer participants hands-on training in managing the lifecycle of their data and code with a focus on using open source tools, including R. Topics will include

Changing the Channel

Fri, 18 Apr 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A lot of open source projects use an antiquated-but-reliable chat system called IRC for long-running conversations. We've had an IRC channel for Software Carpentry for a while, but it's never had very much traffic. It also hasn't done anything to help people get involved in the open science community at large. We are therefore retiring it, and encouraging people—all kinds of people, not just instructors and bootcamp alumni—to join us on the Mozilla Science Lab IRC channel instead. We hope this will become a place where people can swap tips on teaching programming to scientists, discuss new forms of publication, seek help on specific research problems, and everything in between.

Workshop at University of Southern Denmark, Odense

Fri, 18 Apr 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The workshop at University of Southern Denmark, Odense was my first workshop for Software Carpentry, the same is true for Luis as well. We had taught before, Luis more than me, but it was still exciting for me to see how this workshop would go given that we never taught together before. While I taught bash, regular expressions and Git, Luis taught Python, unit testing and SQL. Initially, make was also on that list, but due to time constraints was kicked out to make room for a second Git lecture.

Do Not Be Worried

Wed, 16 Apr 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Sage advice for our instructors and learners from an eight-year-old:

Summarizing Our Instructors' Skills

Tue, 15 Apr 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We've been asking bootcamp participants to tell us about themselves for a while now, so it seems only fair to share some information about our instructors. First, of the 82 instructors who responded to our survey two weeks ago, how many are comfortable teaching which topic to novices, to intermediates, or not at all?

Bridging the Writing Gap

Sun, 06 Apr 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A few months ago, we had an interesting discussion about what Software Carpentry should teach about writing and publishing in the 21st Century. One thing that came through loud and clear was the gulf between people who value the "I can see what I'm doing" of Microsoft Word and those who care more about the "I can tell what I did" of version control¹.

Sites like Authorea, writeLaTeX, and ShareLaTeX are trying to bridge that gulf by giving people a WYSIWYG authoring tool in the browser that uses LaTeX as a storage format. This is pretty exciting: since it (potentially) allows collaborators to interact in whichever mode they prefer, it allows people to transition from one to the other instead of requiring them to make a great leap sideways.

Both sites currently allow people to save work on site or in Dropbox. It would be very cool if they also allowed people to save work in online version control repositories such as GitHub. Someone who isn't comfortable with version control could simply select "Save..." to push their changes, while someone who's already mastered pull requests and merging could interact that way, so that once again, the system could help people transition gradually from one mode to the other.

Does Continuous Publication Require Continuous Attention?

Sat, 05 Apr 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I read this post by Martin Fenner a couple of weeks ago. His thesis is that scientific publication is still very much a manual process, which makes publications relatively infrequent (and fairly painful) events. Instead, we ought to strive for continuous delivery: production of the "paper" (including release of associated code and data) should be fully automated so that authors can ship whenever they want with relatively little effort.

Continuous delivery is popular among software developers, who frequently argue it's more efficient using diagrams like this:

Summary of March 2014 Meeting to Discuss Novice R Material

Fri, 04 Apr 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Last week we had our first meeting to discuss the development of the novice R material. Our goal was to determine our plan for collaboratively creating the lessons.

Uniting the Narrative

Fri, 04 Apr 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Victoria workshop that just wrapped a few hours ago was my first stab at teaching SWC live, and I have to say - few things feel better in teaching than the tiny victory of having material that is right on pitch. I've had the lurking notion for years that the hilariously overshot lectures ('I need to cover the syllabus in half time so I can go on sabbatical'), conference talks ('One slide per 30 seconds for a solid hour seems reasonable') and lab presentations ('A 5 by 5 grid of histograms on one slide will communicate ALL the science!') were the tragic offspring of time constraints and one-upmanship, and in desperate need of a pan-academia slide deck disarmament treaty. I always pushed my students to strip it down, just hit the highlights and leave the details for the paper, because I always suspected that that little bit of entertainer's savvy was how we could really reach people: not by fact-bombing the audience into a glassy torpor, but by electrifying their curiosity just enough that the energy that propelled them forward was their own.

Announcing NBDiff

Sun, 30 Mar 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I am very pleased to announce the first release of NBDiff, a tool for diffing and merging IPython Notebooks that is now available on GitHub and PyPI. NBDiff was created by a team of students at Concordia University in Montréal; our thanks and congratulations to Shurouq Abusalah, Tavish Armstrong, Marwa Malti, Lina Nouh, Boris Pipev, Selena Sachdeva, and Richard Tang. For more information, see the NBDiff web site.

Updating Our Checklists

Sat, 29 Mar 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We have updated our checklists for bootcamp administrators, hosts, instructors (and lead instructors), and helpers. Further improvements are welcome, and we'd appreciate comments on this post suggesting items we could put in a checklist for learners.

Announcing Our Next Lab Meeting

Thu, 27 Mar 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Our next online lab meeting will be held on Thursday, April 24, 2014. As always, we'll run it twice to accommodate different time zones; please keep an eye on this blog, our Twitter account, and the mailing lists for connection details.

Building a Minimal Online Presence

Thu, 27 Mar 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Titus Brown and Ethan White led a half-hour discussion today of the minimal online presence people in academia ought to have. Notes from the call are below; to make a point-form story even pointier, please make yourself findable and shareable.

Changing Our Core Curriculum

Thu, 27 Mar 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We currently say that people must teach task automation, structured programming, version control, and unit testing in order to use the Software Carpentry name. However, many bootcamps either don't teach unit testing at all, or only cover it briefly, while many of us teach data management using SQL, which also seems like a core lab skill for scientific computing. We asked instructors this week whether we should formalize the change, and they responded as follows:

One of Our Inspirations

Thu, 27 Mar 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Last week's post about our original logo reminded me that I'd never blogged about one of the books that inspired Software Carpentry in the first place: Rubin Landau and Paul Fink's A Scientist's and Engineer's Guide to Workstations and Supercomputers: Coping with Unix, RISC, Vectors, and Programming. You'd have add a few zeroes to the speeds, bandwidths, and data sizes they discuss to bring it up to date, but other than that, it's held up surprisingly (or depressingly) well.

Not on the Shelves

Sun, 23 Mar 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Every few years, I indulge in a bit of sympathetic magic by writing reviews of books that don't actually exist in the hope that it will inspire someone to write them. Previous versions written in 1997, 2003, and 2009 led to Beautiful Code, Making Software, The Architecture of Open Source Applications, and a few others books as well. I'd welcome comments on what isn't in this list that you really wish you could read.

What Tools Do You Use to Get Your Job Done?

Sun, 23 Mar 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Some of you may have heard of usesthis.com. In this site people are interviewed about the tools they use to get their job done. The format is very straight forward, participants have to answer these questions:

Empirical Software Engineering Papers

Wed, 19 Mar 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

When I teach scientists programming, I frequently cite empirical studies in software engineering to back up my claims about various tools and practices making people more productive. No good, short survey of those papers exists—writing one has been on my to-do list for several years—but I hope the pointers below will be a useful substitute.

Data Science Workshops in Seattle

Tue, 18 Mar 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Via Sumana Harihareswara: the Community Data Science Workshops are a series of project-based workshops being held at the University of Washington for anyone interested in learning how to use programming and data science tools to ask and answer questions about online communities like Wikipedia, Twitter, free and open source software, and civic media.

The workshops are for people with no previous programming experience. The goal is to bring together both researchers and academics as well as participants and leaders in online communities. The workshops will all be free of charge. Participants from outside UW are encouraged to apply.

There will be three workshops held from 9am-4pm on three Saturdays in April and May. Each session will involve a period for lecture and technical demonstrations in the morning. This will be followed by a lunch graciously provided by the eSciences Institute at UW. The rest of the day will be followed by group work on programming and data science projects supported by more experienced mentors.

For more information, see the full announcement.

Our Original Logo

Tue, 18 Mar 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

How old is Software Carpentry? So old that the only surviving copy of our original logo is an unanimated GIF:

PyCon is Just a Month Away

Mon, 17 Mar 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

With PyCon just under a month away, there are a few things we want to make sure everyone knows about:

A Letter from John von Neumann

Fri, 14 Mar 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Plans to standardize and publish codes of various groups have been made in the past, and they have not been very successful so far. The difficulty is that most people who have been active in this field seem to believe that it is easier to write a new code than to understand an old one. This is probably exaggerated, but it is certainly true that the process of understanding a code practically involves redoing it de novo. The situation is not very unlike the one which existed in formal logics over a long initial period, where every new author invented a new symbolism. It took several decades until a few of these found wider acceptance, at least within limited groups. In the case of computing machine codes, the situation is even more difficult, since all logics refer, at least ideally, to the same substratum, whereas the machine codes frequently refer to physically different machines.

I think, nevertheless, that if a competent mathematician...is interested in working on this problem, he ought to be encouraged. The task may turn out to be a thankless one, but it is always possible that a competent and energetic man will in the end come up with something useful. My personal doubts are limited to the near future, and even if it will take a non-trivial number of years to produce something, there is no harm in starting early.

— John von Neumann, in a letter to Marston Morse, April 23, 1952

Collaborative Lesson Development - Why Not?

Fri, 14 Mar 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A few weeks ago, Greg Wilson asked me:

Why is there so little open, collaborative development of lesson plans and curricula? Is there something that makes teaching different from coding (e.g., open source software) and from writing (e.g., Wikipedia)?

A dozen emails later, I can't claim that we're much closer to a definitive answer, but we have come up with a working hypothesis. We'd be very interested in feedback.

Everything Old is New Again

Fri, 14 Mar 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Yesterday, the New York Times R&D Lab announced streamtools, a web-based graphical tool for working with streams of data. It lets users create dataflow systems to re-mix and process the streams they believe most data will consist of in the near future.

Streamtools looks nice, but I do have some questions:

Software Carpentry at TGAC

Fri, 14 Mar 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

In February The Genome Analysis Centre (TGAC) in Norwich hosted their first Software Carpentry bootcamp. TGAC is keen to intensively develop their training programme and facilities and Vicky Schneider who leads the Scientific Training, Education & Learning Programme strongly supports Software Carpentry, we are looking forward to more bootcamps in Norwich, UK.

You and Jimi Hendrix

Fri, 14 Mar 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I had a discussion a couple of weeks ago about software development tools and processes with some undergraduate students I'm mentoring. They asked why I'm so finicky about putting things under version control, writing unit tests, and creating tickets to keep track of what still needs to be done. The short answer is, because that's what Jimi Hendrix would have done.

John Hunter Technology Fellowship 2014

Wed, 12 Mar 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The John Hunter Technology Fellowship aims to bridge the gap between academia and real-world, open-source scientific computing projects by providing a capstone experience for individuals coming from a scientific educational background. The program consists of a 6 month project-based training program for postdoctoral scientists or senior graduate students. Fellows work on scientific computing open source projects under the guidance of mentors who are leading scientists and software engineers. The aim of the Fellowship is to enable Fellows to develop the skills needed to contribute to cutting-edge open source software projects while at the same time advancing or supporting the research program they and their mentor are involved in.

The Fellow receives an award of $33,000, which is meant to support the Fellow for 6 months of full-time work; The Fellowship can be started in the July 2014 - Jan 2015 time frame, and applications are due May 15, 2014. For more information, and to apply, please see the full announcement

Anatole France, Updated

Fri, 07 Mar 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Anatole France (1844-1924)

Then

The law, in its majestic equality, forbids the rich and poor alike to sleep under bridges, to beg in the streets, and to steal bread.

Now

The Internet, in its majestic equality, allows every scientist to analyze massive data sets using web services and cloud computing.

Reproducibility Workshop at XSEDE

Fri, 07 Mar 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The reproducibility@XSEDE workshop is a full-day event scheduled for Monday, July 14, 2014 in Atlanta, Georgia. The workshop will take place in conjunction with XSEDE14, the annual conference of the Extreme Science and Engineering Discovery Environment, and will feature an interactive, open-ended, discussion-oriented agenda focused on reproducibility in large-scale computational science. We hope to help promote a culture of reproducibility within the broad community of stakeholders connected to computation-enabled research, and expect our work to lead to recommendations address to this community.

This event will build on the 2009 Yale Data and Code Sharing Roundtable, which culminated in a declaration "demanding a resolution to the credibility crisis from the lack of reproducible research in computational science". To find out more, please see the workshop website, or send questions to reproducibility@xsede.org.

A Workshop for Librarians at PyCon

Tue, 04 Mar 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are very pleased to announce that we will be running a two-day workshop on programming skills for librarians during the PyCon sprints in Montreal on April 14 and 15, 2014. The workshop will introduce participants to core skills for managing and manipulating data, including task automation, version control, modular programming, and databases. The event is open to everyone, not just to PyCon attendees; please see this page for more information, register here, mail us if you have any questions, and help spread the word to your friends and colleagues.

Learn How to Teach People to Program

Tue, 04 Mar 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are very pleased to announce that we will be running two one-day workshops on how to teach programming during the PyCon sprints in Montreal on April 14 and 15, 2014. The workshops will be led by Greg Wilson, who will show participants how to use basic concepts from educational psychology and instructional design when teaching beginners how to code. The event is open to everyone, not just to PyCon attendees; please register here, or mail us if you have any questions.

SSI Collaborations Workshop and Hackday

Mon, 03 Mar 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Software Sustainability Institute is running the annual Collaborations Workshop (CW14) on 26-28 March in Oxford. The event brings together researchers, software developers, managers and funders to explore important ideas in software and research and to plant the seed of interdisciplinary collaborations. The theme this year is "Software in your reproducible research" and we also have a hackday!

Summary of Feb 2014 Lab Meeting

Mon, 03 Mar 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We held a lab meeting last week. 48 people attended in two sessions (to accommodate different time zones); the main points are summarized below.

Software Carpentry on the CBC

Sat, 01 Mar 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Greg Wilson was recently interviewed by CBC Radio's "Spark" program: you can listen to the interview online, or catch it live this Sunday at 1:00 pm.

The Open Scoop Challenge

Tue, 25 Feb 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Sooner or later, in every discussion of open science, someone will say there's a risk of being scooped if you share your code or data with your peers. I think this is pure FUD: in all the years I've worked with scientists, I have never met anyone this has actually happened to.

But absence of proof is not proof of absence, so I'd like to issue a challenge. If someone has ever published a result you were going to by taking advantage of software or data that you made publicly available, I'll send you a Software Carpentry t-shirt. You'll need to provide specifics, but I won't share those details with anyone without your permission. Please mail me if you'd like to chat.

Software Carpentry: the University Course

Mon, 24 Feb 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I recently submitted my final assignment for the Specialist Certificate in Teaching for Graduate Researchers. For this final task, we were asked to design a curriculum document for a university subject that we would like to see taught in the future. The first thing that came to mind was Software Carpentry, so here's my take on what a bootcamp would look like, if it were to become an actual university course.

Lab Meeting (Feb 2014)

Sun, 23 Feb 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We're having our first lab meeting of 2014 this week. As usual, we'll hold it twice to accommodate different time zones: once at 20:00 Eastern on Wednesday, February 26, and again at 11:00 Eastern on Thursday, February 27. Connection details will be posted on the meeting's Etherpad, and everyone is welcome both to attend and to add agenda items to the 'pad beforehand.

From Training to Engagement

Fri, 21 Feb 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I was interviewed about Software Carpentry earlier this week, and the interviewer's second question was, "Don't scientists all learn how to program these days as part of their education?" The answer, even today, is "no": the average scientist might know more about calculus and statistics than someone who did a degree in marketing or graphic design, but she probably doesn't know any more about how to build software and share data on the web.

Brent Gorda and I started Software Carpentry in 1998 to fix that. Our goal was to teach our colleagues the equivalent of basic lab skills for scientific computing so that they could get more done in less time and with less pain. As the project grew, I realized that this problem wasn't specific to scientists: almost everyone who uses the web spends hours or days doing things that a few simple programs could do for them faster and more reliably.

Lessons Learned Has Been Published

Wed, 19 Feb 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I'm pleased to announce that "Software Carpentry: Lessons Learned" has been published on F1000Research. A paper like this is necessarily incomplete, and so is any acknowledgments list, but I'd like to thank the following for their feedback:

Our Biggest Event Ever

Thu, 13 Feb 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

If you have friends or colleagues in or near Montreal, please let them know that we're running our largest bootcamp ever on April 14-15 in conjunction with PyCon 2014. Material will include our usual topics:

the Unix shell (and how to automate repetitive tasks);
Git and GitHub (and how to use version control to track and share work);
Python or R (and how to grow a program in a structured, testable, reusable way); and
databases (and the difference between structured and unstructured data).

but we'll split people between several rooms based on how much they already know so that each group can move at its own pace.

Register now!

rOpenSci Hackathon

Wed, 12 Feb 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The rOpenSci team has been cranking out a large number of software tools over the past several months. As regular blog readers are aware, our software packages provide programmatic access to a diverse and extensive trove of scientific data. More recently we've expanded our efforts to build more general purpose and cross-domain tools. These include tools for reading, writing, integrating and publishing data, a unit testing platform for data, and a mapping engine that can visualize various kinds of spatial data. Many of our projects are inspired by ad hoc discussions with other scientists and software developers both online (often on Twitter and GitHub) and offline. Several of these folks are now regular contributors to the project. To foster more such collaborations and drive new software innovations, we are excited to announce our first developer meeting next month at GitHub's headquarters in San Francisco. This meeting is made possible by support from the Alfred P. Sloan foundation and GitHub.

An Online Peer Instruction Tool

Sun, 09 Feb 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Peer instruction is a teaching technique originally developed by Eric Mazur and colleagues in the early 1990s. Study after study has shown that peer instructions works better than conventional lecturing, but to the best of my knowledge, no online learning platform directly supports peer instruction. I'd like to fix that, and I hope some of our readers would like to help.

Wrapping Up Round 7 (and a Reminder About Instructor Training)

Sun, 09 Feb 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We had the wrap-up meetings for Round 7 of instructor training this past Thursday. The full summary is on the training blog, but the key points are:

We need to include a lesson early in the course on the structure of the materials in our repository, and how to configure things for a workshop. We'll do a trial run on Thursday, February 20, in the usual time slots (10:00, 14:00, and 19:00 Eastern); please have a look at the Etherpad for connection details on the day.
People want a chance to watch more experienced teachers in action. One possibility would be to set up something like SmarterCookie or Edthena, where teachers upload videos of themselves in action so that other people can comment on what they do and how they do it. We will think about ways and means and see what we can set up in the next couple of weeks.

Our thanks once again to everyone who stayed with us through what turned out to be a longer-than-usual run of the course: I look forward to teaching with you all some day soon.

Finally, there are still seats available for the live three-day version of the instructor training course, which will run in Toronto on April 28-30, 2014. If you'd like to learn more about teaching in general, and teaching programming in specific, please sign up.

Keeping Track of Problems

Wed, 05 Feb 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Setup and configuration is often the biggest challenge bootcamp participants face, particularly on Windows. Our installation instructions are pretty good, but to make them better, Justin Kitzes has started building a wiki page on GitHub to keep track of what's gone wrong and what fixes have worked. Many thanks to Justin for getting the ball rolling—contributions would be very welcome.

Workshops at the Data Science Centers

Thu, 30 Jan 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are pleased to announce that we will be running workshops simultaneously at the data science centers at UC Berkeley, New York University, and the University of Washington on March 17-18, 2014. These workshops will introduce participants to the skills they should master before they tackle anything with "cloud" or "peta" in its name, including task automation, version control, modular programming, and structured data management. Novice and intermediate learners alike are welcome; for more information on topics and other details, please mail us or check out the registration pages for the three sites:

Teaching Online (Sort Of) in 2014

Tue, 28 Jan 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We have tried several times to teach Software Carpentry online, but the results have always been disappointing. We are therefore going to try something different this spring. Instead of running a purely online course for several hours a week, we are going to run regular two-day workshops in four selected cities with a mix of in-person and remote instructors.

Workshop for Women in Science and Engineering: April 14-15 at LBL

Tue, 28 Jan 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We're very pleased to announce that registration is now open for our second workshop for Women In Science and Engineering, which will be held at Lawrence Berkeley National Laboratory on April 14-15, 2014. We have a stellar roster of instructors and room for over 100 learners, so please sign up early and pass the word on to your friends and colleagues.

Workshops at PyCon in Montreal This April

Tue, 28 Jan 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are pleased to announce that registration is now open for our largest event ever: on April 14-15, 2014, in association with PyCon 2014 in Montreal, we will run three Software Carpentry workshops in parallel, along with a master class in next-generation sequencing for bioinformaticians and an introduction to R for Python programmers. (Note that these workshops are during the two days after PyCon, so you can attend both.)

Research Transparency Job at UC Berkeley

Sun, 26 Jan 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Interested in improving the standards of rigor in empirical social science research? Eager to collaborate with leading economists, political scientists and psychologists to promote research transparency? Wishing to stay abreast of new advances in empirical research methods and transparency software development? The Berkeley Initiative for Transparency in the Social Sciences (BITSS) is looking for a Program Associate to support the initiative's evaluation and outreach efforts. The candidate is expected to engage actively with social science researchers to raise awareness of new and emerging tools for research transparency. If this sounds like fun, apply now.

Feedback from the First MATLAB Bootcamp

Fri, 24 Jan 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Jan 28: Michael Croucher has also written a good post about what worked and what didn't at this bootcamp.

Last week Software Carpentry paired up with MATLAB to run a bootcamp at the University of Manchester. Apart from shell and version control, the remaining core topics (verification and testing, modular programming with algorithm design) were taught using MATLAB.

Why Not a MOOC?

Sun, 19 Jan 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We got mail yesterday asking us whether we were going to run Software Carpentry as a MOOC. The short answer is, "No." The full answer has several parts:

Feedback from the First Cambridge R Bootcamp

Sat, 18 Jan 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The first R Software Carpentry bootcamp in Cambridge took place on January 7 and 8, 2014. Hosted by the Centre for Mathematical Sciences, the bootcamp was organised by two SSI Fellows, Stephen Eglen and Laurent Gatto. There were 24 participants and a long waiting list of those who wanted to learn R, version control and make.

Wrapping Up in Narrangasett

Sat, 18 Jan 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I've just came back from the Software Carpentry bootcamp at Narrangasett, RI, held at the Coastal Institute of the University of Rhode Island and the US Enviromental Protection Agency. I taught with Patrick Fuller and Jeff Hollister, who was also the organizer. The local helpers were Betty Kreakie and Bryan Milstead.

The bootcamp was R-based and lasted two and a half days. The people at the Coastal Institute are using this bootcamp as the starting point for a class on "Computing for Natural Resources" during the next semester, so the extra half-day was used to talk about this coming class and to add a lesson on data visualization using R. Besides the standard syllabus, we extended a bit the lessons on R, added a short lesson on databases, and made a longer than usual session on testing.

Introducing the Image Novice Module

Thu, 16 Jan 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The image novice module provides a simple image manipulation interface for beginners that allows for easy loading, manipulating, and saving of image files. It has been developed collaboratively with Greg Wilson, Tony Yu, and Stephan van der Walt. Below, I show how to use the module for some basic image manipulation.

From 0 to 1 to 10

Wed, 15 Jan 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Andromeda Yelton has written a great blog post about her experiences teaching a bootcamp for librarians in Toronto earlier this week. It includes this insightful observation:

Publishing on the Web

Wed, 15 Jan 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Back in December, I asked a question on the Software Carpentry issue track: What should we teach about writing/publishing papers in a webby world? It led to a lively discussion about how the web is changing scientific publishing, the tools people can use to take advantage of those changes, and the gap between what's possible and what publishers (and senior academics) will actually accept. Many commenters dove immediately into the details of specific tools, but Justin Kitzes nailed it when he said:

Code Review, Round 2

Tue, 14 Jan 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Short version: we will launch a second pilot study of code review for scientific software in February 2014, in which experienced mentors will teach small groups of scientists how and when to do reviews. We are currently looking for both mentors and groups to take part.

It's Not Just the Tools that Differ Between the Two Cultures

Fri, 10 Jan 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

In the conversation that followed Philip Guo's blog post on the two cultures of computing, most on focused comparing the tools used by those two cultures, as discussed by Philip to create a contract. Some even wondered if Software Carpentry instructors should perhaps become more versed in MS Excel to meet potential students where they are. In so doing, we lost the fact that even with the same tools, the two cultures would likely use them differently. It's not about the tools, but about the users.

Job Opportunity at University College London

Thu, 09 Jan 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The University College London Research Software Development Team is recruiting a research software developer. Their ideal candidate will have experience creating and maintaining research software in one or more fields, and will "design, extend, refactor and maintain scientific software accross all subject areas". Sound like you? Have a look at James Hetherington's post about the position. Good luck!

'Best Practices' Has Been Published

Tue, 07 Jan 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I'm very pleased to announce that our paper Best Practices for Scientific Computing has been published in PLOS Biology. It presents 24 specific things scientists can do to get more done in less time with less pain:

Mental Models and Vicious Circles

Tue, 07 Jan 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A few days ago, Konrad Hinsen asked this on our discussion list:

Is anyone aware of teaching methods that aim at either developing or verifying the students' mental model of some non-trivial procedure? If so, have they been used for teaching the use of computers?

Test-Driven Development in Scientific Computing

Tue, 07 Jan 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Aziz Nanthaamornphong and Dr. Jeffrey Carver at the University of Alabama are conducting a survey of software development practices among computational scientists. This survey seeks to understand test-driven development in computational science and engineering domains, and should take approximately 15 minutes to complete. The survey can be found at:

http://goo.gl/9T2BIq

This survey study has been approved by the University of Alabama Institutional Review Board (IRB); for questions contact Jeffrey Carver.

We Need More of These

Tue, 07 Jan 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Michael Nygard's Release It! is one of my favorite books about the practical side of computing—I gave it five stars when it first came out, and while it's too advanced for most of the scientists we want to teach, there's something from it that I'd like to share: the diagram titled "Interaction of Patterns and Antipatterns" from page 43:

Introducing Arliss Collins

Mon, 06 Jan 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The incredible expansion of Software Carpentry over the last few months has been exciting and a little overwhelming, so we're pleased to introduce the newest member of the Mozilla Science Lab team, Arliss Collins. Arliss's background is in engineering and project management, and she'll be working with me, Kaitlin and Greg on bootcamp management, infrastructure development and maintenance, and communications.

Our Store Is Open

Tue, 31 Dec 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Show your Software Carpentry pride with a shirt, sticker, button, or coffee mug from our CafePress store. And Happy Hogmanay: may 2014 be kind to you and yours.

Tools, Conversations, and Cultures

Tue, 31 Dec 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Philip Guo is best known these days for The Ph.D. Grind, but I first met him through his Online Python Tutor. He helped teach a bootcamp for librarians last August, and recently wrote a blog post about the two cultures of computing that he encountered there. On one side he sees users who treat software as a tool for getting things done; on the other, he sees programmers who hold conversations with their software. The former use stand-alone GUIs backed by binary file formats (like Word and Excel), while the latter use command-line interfaces and text (like the shell, LaTeX, and Python). He writes:

Catch and Hold

Fri, 27 Dec 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I'm a big fan of Mark Guzdial's work on computing education. Last week, he tweeted this:

Google is made of people who succeed in current CS teaching model. Hard for them to realize that it's wrong for most people. #GoogleCSFirst

Oxford, One Year On

Wed, 25 Dec 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Philip Fowler recently posted a nice study of the impact last year's Software Carpentry bootcamp at Oxford had on its participants. The key finding for me is that 10 of the 13 respondents to his survey agreed or strongly agreed with the statement, "A year on, the workshop really encouraged me to change how I do my research." If you'd like to help us gather similar information from participants at other bootcamps, please get in touch.

Andromeda's Advice

Thu, 19 Dec 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We're very pleased to announce that Andromeda Yelton will be coming to Toronto in mid-January to help teach a bootcamp for librarians. Her advice on how to do this is online, along with her reflections on what she's learned herself. There's lots of good stuff in both, and we're looking forward to lots of new ideas.

So How Is Instructor Training Going?

Thu, 19 Dec 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We recently wrapped up the latest round of instructor training, which makes this a good time to look at how well the program is doing. Here are stats on the first six rounds:

Code as a Research Object

Wed, 11 Dec 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Mozilla Science Lab's newest project extends our existing work around code as a research object, exploring how we can better integrate code and scientific software into the scholarly workflow. The project will build and test a bridge to allow users to push code from their GitHub repository to figshare, providing a Digital Object Identifier for the code. We will also be working on a best practice standard (like MIAME for code) so that each research object has sufficient documentation to make it possible to meaningfully use.

For more information, please see the full post on the MSL website. And please join us for our December 12 community call, where we'll be talking with Ed Lazowska about the newly-announced data science centers, and with Arfon Smith and Mark Hahnel about this project.

Release 2013.11

Tue, 10 Dec 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are pleased to announce the release of v2013.11 of our lesson materials. This release represents nearly 2 years of collaborative lesson development from 54 contributors, and features lessons for using the Unix shell to automate repetitive tasks, modular programming in Python and R, personal and collaborative version control with Git, unit testing, and working with databases. Contributors include:

There Ought to Be a Badge

Tue, 10 Dec 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I feel like there should be a Software Carpentry badge for drawing one of these for someone for the first time.
— Amy Brown on Twitter

Mozilla Science Lab Community Call for December 2013

Mon, 09 Dec 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The last community call of the season for the Mozilla Science Lab will take place this Thursday, December 12. The call is open to the public and will start at 11 am ET. Call in details can be found on the call etherpad and on the wiki.

News from the SSI

Mon, 09 Dec 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Two announcements from the Software Sustainability Institute caught our eye this week:

Their next Collaborations Workshop will be held on March 26-27, 2014, in Oxford. This annual gathering brings together researchers, software developers, managers, funders and more to explore important ideas in software and research and to plant the seeds of interdisciplinary collaborations.
Sixteen new SSI Fellows have been selected: congratulations to them all!

Advanced Python for Biologists

Thu, 05 Dec 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

When I was writing Python for Biologists (see my previous guest post) I was extremely ruthless about leaving material out. I wanted the book to cover pretty much the same set of language features that I teach in my instructor-led introductory programming courses, which meant I had to limit the scope of the book to what an average, motivated beginner could take in in a full-time week of study. This restriction meant that, while I think the overall balance of material in Python for Biologists is still the most useful set for novice programmers, there are big topics that didn't make the cut. There's no mention of object-oriented or functional programming, nothing about Python's extremely elegant comprehension syntax, and nothing on recursion. As I was writing, I made a list of all these omissions, with the promise to myself that I would write about them some day.

Feedback from Edinburgh

Thu, 05 Dec 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

On the 3rd of December, Software Carpentry returned to Edinburgh with EPCC hosting a bootcamp as part of its involvement in both the PRACE Advanced Training Centre and The Software Sustainability Institute.

Two to the Fifth New Instructors

Thu, 05 Dec 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are very pleased to welcome 32 new instructors from six different countries to the Software Carpentry team: we look forward to working with them all in 2014 and beyond.

Joshua Ainsley
Camille Avestruz
Philipp Bayer
Nichole Bennett
Cliburn Chan
Emily Davenport
Neal Davis
Gabriel A. Devenyi

Jonah Duckles
Jordan Fish
Julian Garcia
Molly Gibson
Ivan Gonzalez
Joshua Herr
James Hetherington
Chris Holdgraf

Damien Irving
Ted Kirkpatrick
Christina Koch
Igor Kozlov
Luke Lee
Matthew Lightman
Yuxi Luo
David Perez-Suarez

Bill Rowell
Martin Schilling
Raniere Silva
Rachel Slaybaugh
Shoaib Sufi
Gayathri Swaminathan
Amanda Whitlock
April Wright

Software and Research Session at AGU 2013

Mon, 02 Dec 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A Town Hall Session on "Software and Research" proposed by four Software Sustainability Institute Fellows will be held at the American Geophysical Union (AGU) Fall Meeting in San Francisco. If you are around, come and join the session on Thursday, 12 December 2013, at 12:30 PM in Moscone West Room 2004.

WiSE Bootcamp at Lawrence Berkeley National Laboratory

Fri, 29 Nov 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are very pleased to announce that our second bootcamp for women in science and engineering will be held at Lawrence Berkeley National Laboratory on April 14-15, 2014. This event is a sequel to our first WiSE bootcamp in Boston this past June, and we hope to once again fill three rooms with over 120 learners. If you are interested, please mark the dates in your calendar—registration will open in January.

DiRAC Driving Test Comes to Edinburgh

Thu, 28 Nov 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

My colleague Mike Jackson recently posted about the DiRAC driving test. DiRAC is the UK's integrated supercomputing facility for theoretical modelling and HPC-based research in particle physics, astronomy and cosmology and is used by numerous researchers with diverse backgrounds. Whilst much of their work is very different, one commonality is that it often requires in-depth technical and software engineering techniques. The idea of the driving test was therefore to ensure that all users have the required knowledge for effective use of the consortium's machines.

Things I Wish Someone Had Told Me About Scientific Computing

Tue, 26 Nov 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

When I set out on my path into neuroscience, the idea of scientific computing was not even something that came up in passing. However, with little warning, I started spending my days at the computer writing code to analyse brain data.

Cleaning neuroimaging data is a multi-step process and I wanted to bring out the most from the data. To do that, I thought, would require using the best bits from a slew of neuroimaging preprocessing tools. Some were better at removing extraneous noise, while others were better at image morphing. But entering the commands one by one was tedious and error-prone and the formats for the different functions were not always compatible. There had to be a better way. So, writing some kind of code became my only option.

Registration Now Open for Instructor Training Course

Mon, 25 Nov 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As announced last week, we are offering a live version of our instructor training course on April 28-30, 2014, at the Mozilla office in Toronto. Registration is now open, and costs only $80 (though participants must cover their own travel and accommodation). Over the course of three days, 36 participants will be introduced to ideas from educational psychology and instructional design, and shown how to use them to teach programming to scientists (and everyone else). It will be the ninth offering of the course, but the first done live, and we hope to see many of you there.

Centre for Doctoral Training at Southampton

Sun, 24 Nov 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Hot on the heels of the announcement of three new data science centers in the US comes this: £350 million for PhD training centers in the United Kingdom. One of them, at the University of Southampton, will focus on next-generational computational modelling, and we're looking forward to working with Hans Fangohr, Ian Hawke, Seth Bullock, and everyone else involved. Congratulations!

The Art of Cold Calling (Updated)

Thu, 21 Nov 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This is an update to an earlier post about how to approach potential bootcamp hosts.

Newly-trained instructors have asked how we go about approaching potential workshop hosts. The short answer is, however we can. The longer answer is, we collect names from journal articles, our Twitter followers, people we bump into at conference, or (increasingly) people who've been through bootcamps, then send emails like the one below. But there's an art to it:

How Software Carpentry Helped Me Write a Paper

Tue, 19 Nov 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A paper I wrote titled Increasing the Efficiency of a Thermionic Engine Using a Negative Electron Affinity Collector was recently published in the Journal of Applied Physics. I found that applying some of the best practices advocated by the Software Carpentry project eliminated much of the drudgery and tedium which leads to manually induced error in developing the software and composing the manuscript. As a result, I was able to focus more on the scientific problems instead of on tedious bookkeeping-type issues.

What to Say at a Bootcamp, After It's All Said and Done

Tue, 19 Nov 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I'm going to be teaching my first ever bootcamp in the coming days. My biggest fear (and there are many!) is that the participants will walk out of the room on the final afternoon and go straight back to their old habits, never taking the time to incorporate what they've learned into their daily workflow. In an attempt to avoid this eventuality, I've planned a rousing concluding address to explain why the content taught at a Software Carpentry bootcamp is so important. It goes something like this...

Moving Forward with Assessment: Interviews

Sun, 17 Nov 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We would like to quantify realistic impact after the dust settles from a bootcamp. In particular, we'd like to know how (or whether) Software Carpentry has improved scientists' workflow three months or more after the bootcamp, on the theory that if people are still using things then, they must have found them useful. Our most recent steps toward this goal include obtaining Internal Review Board for Human Subjects Research (IRB) approval through Michigan State University and beginning to recruit volunteers for interviewing at the three month mark.

Thanks from Woods Hole

Sun, 17 Nov 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We ran a bootcamp last week at Woods Hole Oceanographic Institute. It seems to have gone well:

I just want to say that Will and Ross are rock stars. They may have started something big here in Woods Hole, a sort of revolution. They have taught us how to step back from our machines, think about what we are doing, and join forces with our neighbors, We will return to our respective projects on Monday morning in a more efficient and collaborative way. Thank you Software Carpentry.
— James Manning

Citing Us In Your CV

Sat, 16 Nov 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

People volunteer to teach Software Carpentry bootcamps for many reasons. We're always grateful that they do, and would like them to get all the credit they deserve. Here are some of the ways instructors currently mention us in their CVs:

Creating a Forum

Sat, 16 Nov 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Every few months, someone asks why we don't have some kind of forum or bulletin board for Software Carpentry. The answer is that we've tried setting one up, but there's never been enough traffic to keep it going.

We're a lot larger now than we used to be, though, so we decided at this week's lab meeting to try again starting in January. Half a dozen people have volunteered to be stewards, and we've opened an issue on GitHub to discuss the features people want, experiences with other forums, and so on. Please add comments to that issue to let us know what you think.

Instructor Training in Three Days

Sat, 16 Nov 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Update: see this post for information on registration.

We are pleased to announce that we will be running a live version of our online instructor training course on April 28-30, 2014, at the Mozilla office in Toronto. Over those three days, 30 participants will be introduced to ideas from educational psychology and instructional design, and shown how to use them to teach programming to scientists (and others). The course will be free, though we will require a deposit to discourage no-shows and attendees will need to cover their own travel and accommodation costs. Registration will start in January—please keep an eye on this blog for news.

Workshop at SIGCSE 2014

Sat, 16 Nov 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are very pleased to announce that Fernando Pérez, Peter Norvig, and Greg Wilson will be running a three-hour workshop on teaching with the IPython Notebook at SIGCSE 2014 in Atlanta on March 5, 2014. We'll assemble notes in this GitHub repository, and we look forward to lots of stimulating discussion.

Data Science Centers at UCB, UW, and NYU

Wed, 13 Nov 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Yesterday, the Gordon and Betty Moore Foundation and the Alfred P. Sloan Foundation (who are Software Carpentry's main backer) announced $37.8M to support new data science centers at UC Berkeley, the University of Washington, and New York University. As UCB's Fernando Perez says in his blog post, it's taken a lot of work to get this off the ground, but the impact will be enormous.

We hope to help. Drew Conway's Venn Diagram shows "Hacking Skills" as one of the three principal components of data science:

Women in Tech Workshop at PyData NYC

Wed, 13 Nov 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Software Carpentry has always focused on scientists at the graduate level and above, but getting people into the pipeline is even more important as getting them through it. Last week, thanks to backing from NumFOCUS and others, a group of volunteers helped do just that by running Python workshops for high school girls in conjunction with PyData NYC. Judging from Julia Evans' writeup, it was a great success. We look forward to seeing some of their alumni in our classes some day...

Report on the PLOS/Mozilla Code Review Pilot

Tue, 12 Nov 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Marian Petre and I have written a report on the pilot study of scientific code review done jointly by PLOS and Mozilla; you can read this summary on the Mozilla Science Lab blog, or grab the full report from arXiv. Our thanks to everyone who participated; we hope to be able to do a longer and more in-depth pilot early in 2014.

Our First Bootcamp in Adelaide

Fri, 08 Nov 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We ran our first bootcamp at the University of Adelaide from the 24th to the 26th of September 2013. Teachers and helpers came from all over the place: Ben Morris from the University of North Carolina acted as Principal Instructor, along with Jerico Revote from Monash University, Diego Barneche from Macquarie University, Philipp Bayer from the University of Queensland, Nicholas Crouch from Flinders University, and Nathan Watson-Haigh from the Australian Centre for Plant Functional Genomics (ACPFG), who also acted as the main organizer (thanks for that!).

November 2013 Lab Meeting

Sat, 02 Nov 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The first Mozilla Science Lab meeting is happening at 11:00 am Eastern time on Thursday, November 14; its agenda and connection details are on this Etherpad, and you're all invited.

Our monthly lab meeting will start immediately after the MSL meeting (hopefully around noon Eastern time, but it depends how long the MSL meeting runs). As always, we'll run our meeting again at 7:00 pm on the same day to accommodate different time zones, and our agenda is on this Etherpad. We look forward to seeing you then!

Reorganizing

Sat, 02 Nov 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

TL;DR: We're going to:

split our lesson materials for absolute beginners and people with some previous experience;
build some tools to manage those materials;
update and clarify the guidelines for instructor training, Software Carpentry's scope, making contributions, and use of our name and logo; and
use GitHub issues and blog posts to manage discussion of details rather than our mailing list.

Software Carpentry's Scope

Sat, 02 Nov 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

To help people figure out what's in and what out as we're reorganizing our material, I've drafted an explicit description of our scope inspired in part by AstroPy's idea of an "affiliated" package, which is, "...an astronomy-related Python package that is not part of the astropy core package, but has requested to be included as part of the Astropy project's community."

Domain-specific tools for machine learning and computational fluid dynamics aren't built into LAPACK or NumPy: they're stored in separate repositories and released in their own packages. Similarly, domain-specific lessons don't belong in Software Carpentry. Instead, they should build on our lessons, just as computer vision libraries build on matrix and image libraries. Our examples should be drawn from biology, physics, economics, and similar disciplines in order to appeal to our audience, but everything we include must be accessible to anyone who's done freshman science.

The State of Open Science

Thu, 17 Oct 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As part of the biannual Mozilla Summit a couple of weeks ago, John Jensen presented a report on the health of the open web. The details were fascinating, and are summed up in a graphic showing how the various layers of openness that we all depend on are doing. I've reproduced it on the left, and summarized my view of how open science is doing by those same measures on the right:

Open Web
User Choice & Control	Social Activity	Economic Activity	Trust
Diversity of Services
Innovation
Content Freedom
Interoperability
Access

Open Science
User Choice & Control	Social Activity	Economic Activity	Trust
Diversity of Services
Innovation
Content Freedom
Interoperability
Access

You Keep Using That Word

Thu, 17 Oct 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The first Workshop on Sustainable Software for Science: Practice and Experiences (WSSSPE) is going to be held in conjunction with Supercomputing 2013 on Sunday, November 17. After reading the papers on the workshop site, my reaction was:

Curriculum Design

Mon, 14 Oct 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I spoke with three different people about curriculum design last week, so this seems like a good time to summarize what I know about doing that properly, how we've actually been doing it for Software Carpentry, why the two are different, and what we hope to do in future. To set the stage, I need to talk about the medical program at McMaster University in Ontario.

Python for Biologists

Mon, 14 Oct 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Programmers, at least as found in scientific research institutions, generally fall into two groups. There are people with a computer science background, for whom programming is a way of Thinking About The World, and people with a scientific background, for whom programming is a way of Getting Things Done.

Enrolment Figures (Fall 2013)

Wed, 09 Oct 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Our total enrolment (or enrollment, if you're American) continues to grow steadily: if we stay on the curve below, we'll have our four thousandth learner by the end of this year.

A new UK administrator for Software Carpentry

Tue, 08 Oct 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are pleased to welcome Luke Tweddle of EPCC, The University of Edinburgh, to Software Carpentry. Luke is taking over as UK administrator from Mike Jackson on behalf of The Software Sustainability Institute. In his role, Luke will help UK researchers organise bootcamps for their research groups, institutions and communities; help the local organisers of bootcamps create and customize content; attract instructors and helpers; manage registration; advise on publicity; and provide support in all aspects of organising a bootcamp. If you'd like help organising a bootcamp, or if you're interested in becoming an instructor or helper, please get in touch.

October 2013 Lab Meeting

Fri, 04 Oct 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

25 people attended our monthly (ish) online lab meeting yesterday, during which we announced and discussed a wide variety of topics. The detailed summary is below the fold, but the key action items are:

Please take a moment to read Kaitlin Thaney's post about what's going on with the Mozilla Science Lab. We hope to announce some new projects at MozFest in London in a couple of weeks, and more shortly after that.
There's been a flurry of pull requests and comments on lesson material in the past couple of weeks, which is great news for our sustainability. To keep the momentum going, please either send at least one pull request on the teaching materials, or make at least one comment on someone else's pull request, by Monday Oct 14.
In aid of this, please get in touch if you're willing to spend a bit of time in the next couple of weeks helping instructors-in-training learn a bit more about GitHub and pull requests.
Finally, please each introduce us to one new potential bootcamp host: we have a bunch of new instructors, so we're now able to take on more load.

Our Biggest Bootcamp Ever at PyCon 2014

Fri, 04 Oct 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Thanks to the generous support of the Python Software Foundation, we will be running our biggest bootcamp ever in conjunction with PyCon 2014 in Montreal on April 14-15, 2014. We will have two rooms with seating for 50 people and two more with seating for 80, so our plan is to run three regular bootcamps for people of varying skill levels, and one more "master class" for scientists who are ready to learn more advanced topics.

Steven Koenig: What I've Learned

Tue, 01 Oct 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I have been a PhD student at Technische Universität München, Germany since May 2012. My research interest is biopolymers: how to produce them, and what to do with them. Where computers come into play? Literally everywhere. And what I need Software Carpentry for? Literally everything.

Bootcamp Student Composition by Domain

Mon, 30 Sep 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Recently I was challenged to show that computational science is important to biologists, and, more importantly, that biologists care about learning how to develop quality software. I am not a biologist. While I find the subject fascinating I cannot make any broad claims on the topic or hope to speak for all biologists. So I turned to Software Carpentry.

My experience teaching Software Carpentry and related bootcamps has been that biologists are always a significant portion of the student population. In order to demonstrate this I asked to have access to anonymized answers to part of the pre-assessment survey. Specifically, I was interested in the "Which of the following most closely aligns with your primary discipline?" question. With a big assist from Amy Brown I obtained the data I was looking for.

The Future: Today

Fri, 27 Sep 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Two papers appeared on my radar this week that give a taste of the kind of science we want everyone to be able to do in five years' time. The first, by Aide et al, is "Real-time bioacoustics monitoring and automated species identification"; the second, by Omberg et al, is "Enabling transparent and collaborative computational analysis of 12 tumor types within The Cancer Genome Atlas". They're both excellent pieces of 21st Century science, and they caught my eye because:

They're open access: I was able to read them on the off chance that they'd be interesting.
They show how a few scientists with serious software skills can accelerate the research of hundreds of others, provided those others have a few skills themselves.

Code and Data for the Social Sciences

Thu, 26 Sep 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Matthew Gentzkow and Jesse Shapiro have written an excellent guide "Code and Data in the Social Sciences". It's short (only 38 pages), very readable, and full of practical advice for scientists of all stripes:

Michael Hansen: What I've Learned

Wed, 25 Sep 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

My first exposure to Software Carpentry was during a bootcamp at Indiana University in March of 2012. I volunteered to be a helper, and found myself really enjoying helping motivated students learn how to incorporate the material into their own research. I've since volunteered as an instructor at a handful of other bootcamps (including one at IU again this past summer). In this post, I'll briefly discuss some "lessons learned" from Software Carpentry.

How Much Testing Is Enough?

Tue, 24 Sep 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

When we teach testing, we are sometimes asked, "How much is enough?" The short answer is, "It depends." To help construct a longer answer, we'd like to find out how much testing scientists do in other contexts. For example, the Cosmic Origins Spectrograph (COS) and the Space Telescope Imaging Spectrograph (STIS) are two of the instruments on the Hubble Space Telescope. During Cycle 21 (which starts this month, and runs for about a year), they will spend roughly 25% and 67% of their time respectively on calibration:

SSI Fellowship Programme 2014

Tue, 24 Sep 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Software Sustainability Institute is offering Fellowships with £3000 funding for travel, collaboration and running events, including Software Carpentry bootcamps. The Fellowship Programme 2014 recognises outstanding UK-based researchers who use and develop software. Online applications are open until September 27th, 2013 at 5:00 PM BST.

Lex Nederbragt: What I've Learned

Sat, 21 Sep 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I am a biologist with no formal training in Computational Science. A couple of years ago, the increasing size of my data forced me to stop using Excel, and switch to the Unix command line and scripting for my analyses. In the process, I learned what I needed by mostly by just doing it, but also from books, websites, and of course google.

Almost exactly one year ago, I attended a Software Carpentry bootcamp. I had heard about Software Carpentry and its bootcamps through twitter, started following their blog and became convinced that this was something I wanted to attend. At some point, I fired off an email to Software Carpentry asking what it would take to have a bootcamp at our university, the University of Oslo in Norway. The answer came down to 'get us a room and advertise the event, and we'll provide teachers'. This, in fact, was what happened, and as teachers, we got Mr Software Carpentry himself, Greg Wilson, who taught together with a local teacher (Hans Petter Langtangen).

Sarah Supp: What I've Learned

Sat, 21 Sep 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I have to admit, that when I first decided I was going to learn programming skills, it was for academic survival. I was decidedly not excited about taking a course. As a graduate student, I was lucky that Utah State University had a Programming for Biologists class, taught by Ethan White. Most universities still lack the infrastructure to teach these skills to scientists, outside of signing up for courses in the computer science department. Within the first week, I learned that programming is really like playing a series of logic games, and that it could actually be quite fun. Aside from practical skills, the most important thing I learned was not to be intimidated by computational problems.

Software Skills and Hummingbird Diversity

Sat, 21 Sep 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I am an ecologist. Traditionally, many ecologists spend part of the year outdoors, in the field, to collect data, and then spend the rest of the year analyzing that data and writing papers. With new sensors that passively collect audio, weather, or geolocation data, even a single field season can yield massive amounts of data. Add to that the pressing need for more long-term data (aggregating multiple years) and the problem of managing and analyzing the data becomes increasingly difficult for scientists without basic computational skill sets or strong computational collaborators.

Scientific Computing at AIMS

Sun, 15 Sep 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Close on the heels of another bootcamp in Madison, WI, for the past two weeks I had the pleasure of teaching a 30 hour scientific computing course at the African Institute for Mathematical Sciences (AIMS) in Cape Town, South Africa. The course was an expansion and refactor of Software Carpentry material to fit the AIMS need. You can find the repository of source material here.

I could gush about how I fell in love with Cape Town, the unexpected preponderance of vegan cafes in Muizenberg, the sea, the sand, the mountains, and la dolce vita I am otherwise used to from Santa Barbara. I will not. AIMS is about the students. Let's examine them after a brief tour of AIMS itself.

Konrad Hinsen: What I've Learned

Wed, 11 Sep 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

All of my research has been based on computation, starting with my Master's thesis (1989) on the dynamics of colloidal suspensions. Back then, computational physics concentrated on simple systems, and scientists usually wrote their own simple software from scratch. When I started to work on biomolecular simulations five years later, I discovered the sad state of non-trivial scientific software, written by generations of PhD students and postdocs who learned Fortran "on the job" and had the prime goal of getting their research project done with a minimum of programming effort.

Diego Barneche: What I've Learned

Mon, 09 Sep 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I have been a PhD student @ Macquarie University since late 2011. I'm broadly interested in global patterns of biological diversity, macroevolution, statistical and mathematical modelling as well as programming for science (mostly R based but hoping to get into Python very soon). At present I'm focused on quantitative niche-based approaches that can better bridge macroecology and macroevolution, mainly through theoretical approaches such as the Metabolic Theory of Ecology and stochastic population dynamics. I've recently become an instructor for the Software Carpentry group which taught me how to combine teaching, which is an old passion, and improving my programming skills and writing better scientific code, which is something newer.

Introducing Citation Files

Mon, 02 Sep 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Robin Wilson, of the University of Southampton, recently posted a note on the Software Sustainability Institute's blog about CITATION files. In brief, he (and we) would like to encourage scientific programmers to put a plain text file called CITATION in the root directory of each project, and to use it to tell readers how best to cite that software. The example Robin gives is:

Our Plan for the Science Lab

Mon, 02 Sep 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Kaitlin Thaney, the recently-appointed Director of the Mozilla Science Lab, recently posted some thoughts about the lab's mission and plans. Long story short, its focus is to improve communication and interoperability in open science—paraphrasing William Gibson, the future is already here, but the pieces don't play nicely with each other yet, so the lab is going to help make and strengthen social and technical connections. If you have thoughts on how to accelerate this, we'd like to hear from you.

Share Your Code With the Molecular Ecologist Blog

Mon, 02 Sep 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Almost everyone has written or needed code to accomplish a task. Whether it is to automate a process or create the perfect figure, these scripts are useful, yet do not always merit publication. However, sharing these more widely would save the rest of the community from repeating the effort.

To help get all this code into the public sphere, the Molecular Ecologist blog is starting a new series of posts: anyone can send in a piece describing their code, and we'll put it up on the blog. The code itself will be placed on our GitHub page for easier access.

Submissions can be sent to molecularecologist@gmail.com and viewed at https://github.com/TheMolecularEcologist. Or if you are already on GitHub, join our organization to submit and maintain your code in that manner. More detailed information can be found on the blog itself.

Teaching Librarians at Harvard

Mon, 02 Sep 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Last weekend, we ran a bootcamp at the Harvard-Smithsonian Center for Astronomy aimed primarily at librarians. It was a bit of a new venture for us, and there were some hiccups (some things were taught too fast, some instructors used too much jargon), but overall it seems to have gone well, and we hope to do more like this soon.

Gerry Walden: Bootcamp...for Librarians!
Philip Guo: Teaching Librarians Programming
Matthew Rutley: Teaching Python at Harvard with Software Carpentry

Jonathan Dursi Joins Compute Canada

Fri, 30 Aug 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Jonathan Dursi, a long-time supporter of Software Carpentry and one of our first instructors, has been named Interim Chief Technology Officer for Compute Canada. Congratulations!

Open Access Button Hackathon is Sept 7-8

Fri, 30 Aug 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Millions of people a day are denied access to the research they both need and paid for because of paywalls. It doesn't have to be like this, but we need your help to change it. We're making a tool to help change the system called the Open Access Button. The button is a browser-based tool which tracks every time someone is denied access to a paper. We then display this, along with the person's location, profession, and story on a real time, worldwide, interactive map of the problem.

It gets better though. While creating pressure to open up scholarly and scientific research, we help people work within the current broken system by helping them get access to the paper they need. We started building a prototype at the BMJ Hack Weekend, and came third place. But we're not finished yet and our launch is coming up fast! To help build it we're hosting a hackathon on September 7-8 of September in London. If you're a developer, have an eye for design or both we'd love to see you.

If you're not in the UK, you can join in from anywhere in the world—just sign up. And if you want any more information about the project, email us at oabutton@gmail.com or read more about the project.

August 2013 Lab Meeting

Fri, 23 Aug 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We held another of our semi-regular online lab meetings yesterday, complete with a theme song. We ran the meeting twice to accommodate different time zones; those present were:

Instructor Training Statistics

Fri, 23 Aug 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We started running an online training course for Software Carpentry instructors last fall. It's been going pretty well:

72 people signed up for the four rounds we ran between September 2012 and March 2013; 41 of them completed.
39 enrolled in the fifth round that started in May 2013; we expect one third to complete it over the next couple of weeks.
23 enrolled in the sixth round that started this month.

An average completion rate of 40% is pretty low compared to most university courses, but it compares favorably with the 5-10% completion rates of most MOOCs, and 9 of the 29 who dropped out of the first four rounds re-enrolled in a later round. Almost everyone who tells us why they dropped out blames real life getting in the way, but there's undoubtedly sampling bias in that: some people just disappear and stop responding to email.

The curriculum is now pretty stable (though we'll continue to tweak it based on feedback from learners and lab meetings), and feedback from people who've gone through it has mostly been positive. If you'd like to take part in a future round, please get in touch.

Two Cheers for GitHub

Fri, 23 Aug 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

There's been a micro-flurry of excitement in the scientific world about GitHub's recent announcement that they will now render tabular text files (i.e., CSV and TSV) on their site. Coming on the heels of their support for GeoJSON, this is still more evidence that they're serious about becoming a platform for working with data.

So why only two cheers? Because they're going about it in the wrong way, and as a consequence, they'll deliver a lot less value than they could, a lot later than they could.

What Makes Good Code Good at INTECOL13

Fri, 23 Aug 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

On 21st August, I attended INTECOL13 in London's Docklands. The 11th International Congress of Ecology, hosted by the British Ecological Society and INTECOL, is one of the world's premier conference for ecologists.

At the invitation of Matthew Smith from the BES Computational Ecology Specialist Interest Group and Microsoft Research, and Greg Wilson from Software Carpentry, I ran the latest instalment of The Software Sustainability Institute's "what makes good code good" discussions...

Video Interviews from SESYNC Workshop

Thu, 22 Aug 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The folks at SESYNC interviewed Titus Brown, Kaitlin Thaney, and Greg Wilson while they were in town for a workshop a few weeks ago, and the video is now up on their web site. It's a nice, short summary of the problem we're trying to solve and the approaches we're taking.

Creating Assessments and What to do With the Data

Tue, 20 Aug 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Our new assessment questionnaires are up for your review, so I thought I'd talk a little bit about the work behind the scenes, and some lessons learned. Over the past couple months I've had the opportunity to interview bootcamp participants who attended the Michigan State University Software Carpentry Bootcamp a little over one year ago, May of 2012. The goal for interviews were two-fold: develop assessment questionnaires, and gather data.

August 2013 Lab Meeting

Mon, 19 Aug 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We will have an online lab meeting on Thursday, August 22, 2013 to discuss changes to the website, plans for the coming few months, and common questions that really ought to be answered in the FAQ or somewhere else. Everyone is welcome to join in; to do so, head over to this Etherpad at either 11:00 or 19:00 Toronto time, and hop on the conference call described there as well. (We're holding the same meeting twice to accommodate different time zones; please try to sign in a few minutes before the top of the hour so that we can start more or less on time. Please also check the World Clock Meeting Planner to see when the times listed are for you.)

Bootcamp Questionnaires

Mon, 19 Aug 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Thanks to some hard work by Jory Schossau and others, we have developed three standard questionnaires for bootcamp participants:

Pre-assessment for Learners
Post-assessment for Learners
Post-assessment for Instructors

We hope to deploy these questionnaires starting in September, so we would be very grateful for feedback. Do you understand what we're asking? Do the responses provided include the answer you would give? As an instructor, would you find it useful to know these things about your learners before a bootcamp started? And what have we missed entirely?

Our New Operations Guide

Sun, 18 Aug 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Amy Brown, our indefatigable administrator, has put together some checklists for people involved in organizing and running bootcamps:

Summary of Host Survey

Wed, 14 Aug 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Earlier this summer, we asked people who had hosted Software Carpentry bootcamps to give us some retrospective feedback. Their answers are summarized below, and we'll be adjusting our operations in response.

How much time did you put into organizing your bootcamp?

Answers were split evenly between "a day or two" and "several days".

What was the total cost?

With the exception of our Australian double-header (where travel costs were clearly an outlier), the answers were in the $700-3000 range.

Would you like to host another one?

22 said "yes", 3 said "no", and 6 can't decide yet.

How do you feel about asking bootcamps to contribute $1500 toward central overheads?

I support this approach	12
I support this approach, but not at this price	5
I am neutral	7
I am opposed to this approach	3

What level and format would you prefer?

Complete beginners / 2 days	7
Complete beginners / 3 days	5
Intermediates / 2 days	13
Intermediates / 3 days	2
Other	3

What We Cover in Instructor Training

Tue, 13 Aug 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The sixth round of our online training class for Software Carpentry instructors is kicking off this week with 28 participants. Our main text is How Learning Works, an up-to-date summary of research results in educational psychology and instructional design, along with the chapter Mark Guzdial wrote for Making Software explaining why teaching programming is hard. Over the next 14 weeks, participants will work through both, and do the following:

Report on the Indiana Bootcamp

Fri, 26 Jul 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Mike Hansen, Jeff Shelton, and Aleksandra Pawlik posted a summary of their recent bootcamp at Indiana University on the Software Sustainability Institute blog, which we've reposted below. It includes some reflections on what works and what doesn't when using IPython Notebooks for teaching, and similarly with virtual machines.

Miscellaneous Videos

Sat, 20 Jul 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

People have been talking about Software Carpentry in a bunch of different venues recently. Katy Huff and Matt Davis did a tutorial at SciPy'13 that was captured in these three videos; Damien Irving gave a talk at PyCon Australia; Luis Figueira talked about learning how to learn; and Ariel Rokem and Shreyas Cholia talked about their experiences teaching in New Zealand in a recent Mozilla community call. (You need to enter a name—any will do—on the launch page for the recording, and then scroll ahead manually to 00:04:54 to hear them because the Flash player won't let me hyperlink directly to a particular time mark. The irony of both hasn't escaped us...)

The Fourteenth Anniversary

Fri, 19 Jul 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Fourteen years ago this October, I had the good fortune to attend a one-day workshop titled Open Source/Open Science at Brookhaven National Laboratory. A lot of the tools we work with have changed since then, but the central message remains the same: when it comes to science, the opposite of "open" isn't "closed" — it's "broken".

Welcome Our New Instructors

Fri, 19 Jul 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We're very pleased to welcome a dozen new instructors to the Software Carpentry team:

Diego Barneche
John Blischak
Daniel Falster
Rich FitzJohn
Steven Koenig
Ben Morris
Randy Olson
Karthik Ram
Jory Schossau
Will Trimble
Bogdan Vera
Jens von der Linden

Data Science Workflows

Thu, 18 Jul 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Half a dozen of us got together yesterday morning to chat about The Bad Data Handbook, what the curriculum for a Software Carpentry-style bootcamp for data scientists ought to be, and a bunch of other things. The Etherpad is unfortunately down right now, so you can't read David Flanders' excellent notes, but as an outcome, I'd like to ask you all to do a bit of homework.

DiRAC Driving Test Ready to Roll

Thu, 18 Jul 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Since June 2012, The Software Sustainability Institute and Software Carpentry have been working with the DiRAC consortium to develop a "driving test" or basic software skills aptitude test. The test is now ready to be rolled out across DiRAC.

Feedback from Bath

Thu, 18 Jul 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This week saw Software Carpentry head to Bath in England's South-West. The bootcamp was organised by Alex Chartier, as part of his role as a fellow of The Software Sustainability Institute. Chris Woods from the University of Bristol, joined me as co-instructor, making his bootcamp instructing debut.

Biological Computing User Stories

Wed, 17 Jul 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Three years ago, when we rebooted Software Carpentry, we wrote some brief descriptions of our intended audience and how we thought we could help them. One of the exercises we at the SESYNC meeting last week (described briefly here) was to write some similar before-and-after stories for biologists who do computing. The descriptions we got are listed below; some of my takeaways are:

Pretty much everyone's starting point is Excel...
The data hand-off problem is as important as the data processing problem.

The stories themselves are listed below—we'd like to hear what resonates and what's missing.

Computational Competence for Biologists

Tue, 16 Jul 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

On July 8 and 9, I had the pleasure of taking part in a two-day workshop at SESYNC to discuss what we ought to teach biologists about computing. It was a relatively small meeting, but the participants spanned the range from computer scientists and systems engineers to bioinformaticians, field biologists, and a few odd ducks like me.

eResearch New Zealand 2013 Bootcamp Roundup

Sun, 14 Jul 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

New Zealand is an amazing place - incredibly beautiful, with a very friendly vibe. We (Ariel Rokem and Shreyas Cholia) had the privilege of traveling all the way from California, to run a bootcamp down in Christchurch, as part of the eResearch NZ 2013 conference.

The Oslo Bootcamp

Wed, 10 Jul 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Karin Lagesen writes:

The bootcamp took place at the University Library at the main campus of the University of Oslo. We had set a limit at 40 people, and we had a total of 32 attendees signing up. Sign-up was in the beginning quite slow, and we were actually at one time wondering whether we would have to cancel the event. But, as we got into June sign-ups picked up. We had decided to ask people to pay $20 to sign up (which was subsequently converted into coffee) and this might have caused people to not sign up until they were sure they could join. We ended up with 27 actual attendees the first day and 25 on the second day. The attendees were for the most part related to biology in some way—not strange considering we had advertised it in connection with the Galaxy 2013 community conference and on several bioinformatics related email lists. Also, most of them were either doing their PhDs or working on a post-doc. Both Lex and I feared lots of issues with installation, but even though the pre-bootcamp survey showed that almost half of the attendees were bringing Windows laptops things went surprisingly smoothly.

Workshop for e-Infrastructure Trainers

Sat, 06 Jul 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Simon Hettrick, of the Software Sustainability Institute, recently posted this:

If you have an interest in training researchers how to use computing, software or data, you should attend the workshop for e-Infrastructure trainers on 14 August 2013 at the newly opened Hartree Centre.

For more information, please see the full announcement.

Sloan Foundation Proposal Round 2

Fri, 05 Jul 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

From our "better late than never" department: we submitted a proposal to the Sloan Foundation last August to create what we're now calling the Mozilla Science Lab. As regular readers already know, it was accepted; we're now more than six months into the project, so in keeping with our own earlier practice, and with the examples of ImpactStory, rOpenSci, NISO, and Dr. Holly Blik, I'm posting the proposal here. I hope it's useful to other people seeking funding for ventures like ours, and I'd be happy to answer questions.

WiSE Bootcamp Roundup

Fri, 05 Jul 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

It's been over a week since our first bootcamp for women in science and engineering wrapped up in Boston, and feedback has been coming in pretty steadily. In no particular order:

After WISE

Thu, 27 Jun 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Our first bootcamp for women in science and engineering is just wrapping up in Boston. It's gone very well—we'll be posting a summary soon, but in the meanwhile, here are a few things attendees can do as follow-up:

Software Carpentry: Lessons Learned

Thu, 20 Jun 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

With contributions from:

Azalee Bostroem (Space Telescope Science Institute)
Chris Cannam (Queen Mary, University of London)
Stephen Crouch (Software Sustainability Institute)
Matt Davis (Space Telescope Science Institute)
Luis Figueira (Queen Mary, University of London)
Richard "Tommy" Guy (Wave Accounting)
Edmund Hart (University of British Columbia)
Neil Chue Hong (Software Sustainability Institute)
Katy Huff (University of Wisconsin)

Michael Jackson (Edinburgh Parallel Computing Centre)
W. Trevor King (Drexel University)
Justin Kitzes (University of California, Berkeley)
Stephen McGough (University of Newcastle)
Lex Nederbragt (University of Oslo)
Tracy Teal (Michigan State University)
Ben Waugh (University College London)
Lynne J. Williams (Rotman Research Institute)
Ethan White (Utah State University)

Abstract

Over the last 15 years, Software Carpentry has evolved from a week-long training course at the US national laboratories into a worldwide volunteer effort to raise standards in scientific computing. This article explains what we have learned along the way the challenges we now face, and our plans for the future.

The Twelve Bar Blues of Open Science

Wed, 19 Jun 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Most musicians can play along with a twelve-bar blues once they know what the key and tempo are. Many kinds of scientific work are equally well structured: the results aren't predictable—it wouldn't be research if they were—but the equipment setup, sample preparation, note-taking, statistics, and write-up are (mostly) structured in ways that other scientists are familiar with. This lets them pick up each other's projects more quickly; it also gives scientists more time to do what's unique about a particular project because they don't have to spend time thinking about the things that aren't.

UMass Amherst Bootcamp: Perspective from a Helper

Tue, 18 Jun 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Background

A few weeks ago, I helped out with the Software Carpentry bootcamp at UMass Amherst. It was my first time being a helper. Previously, my exposure to Software Carpentry was the MOOC style online Spring 2011 class, which I took during physics graduate school. I was among the 5-10% of students to finish the course. The low completion rate was one of the reasons Software Carpentry switched to doing live, on-site bootcamps.

A couple years later, I got in touch with Greg Wilson and he mentioned that there was a bootcamp coming up in my area. He invited me to join as a helper, I agreed, and I suddenly found myself on several mailing lists. I had no idea what to expect, but the bootcamp turned out to be quite fun and I met many interesting people—including a core Python developer.

Post bootcamp, I had many thoughts about the logistics and teaching. The staff invited me to write these up in a blog post.

Salk Institute Feedback

Mon, 17 Jun 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Preston Holmes recently posted a detailed analysis of his experiences helping out at our Salk Institute bootcamp. It contains a lot of insights, and is well worth reading.

Bootcamp in Bristol, September 12-13, 2013

Sat, 15 Jun 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Our bootcamp at the University of Bristol on September 12-13, 2013, is now open for registration. Please register here if you would like to take part.

Announcing the Mozilla Science Lab

Fri, 14 Jun 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We're very excited to announce the launch of the Mozilla Science Lab, a new initiative that will help researchers around the world use the open web to shape science's future. The Science Lab will foster dialog between the open web community and scientific researchers. Together they'll share ideas, tools, and best practices for using next-generation web solutions to solve real problems in science, and explore ways to make research more agile and collaborative.

June 2013 Lab Meeting

Thu, 13 Jun 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Earlier this week, Software Carpentry had its first online lab meeting since October 2012. In attendance were:

North Carolina Bootcamps

Sun, 09 Jun 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Last month, we held two Software Carpentry workshops back-to-back in Durham, North Carolina, one at the National Evolutionary Synthesis Center and a second organized by Cliburn Chan for the biostats group at Duke just three days later. These workshops were taught by Jenny Bryan, Elliott Hauser, and myself with Karen Cranston teaching as well at NESCent. The workshops were both somewhat atypical: NESCent was a smaller workshop (only 11 participants) who work together and share an interest in evolutionary biology, and at both NESCent and Duke we presented in R rather than Python. Overall, both workshops were very successful, and I learned a lot, both about R and teaching a Software Carpentry workshop.

Running Bootcamps

Fri, 07 Jun 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

It's been eighteen months since we started running two-day bootcamps. We've grown rapidly:

Thoughts on an Advanced Bootcamp at Boulder

Fri, 07 Jun 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Alex Viana and myself recently wrapped up a bootcamp at the National Center for Atmosphere Research (NCAR) in Boulder, CO. It was a lesson in the advantages of being light on your feet when it comes to teaching for Software Carpentry. Our normal audience tends to be scientists with varying degrees of familiarity with the tools that we teach. This time we taught about 20 people all of whom were experience software engineers, some of whom had probably been programming longer than Alex or I had been alive. We had planned on starting the morning with some shell and then moving into Git. A quick survey of the room found that everyone was well versed in the shell so we abandoned that lesson all together and went straight to teaching Git.

Amsterdam Bootcamp

Thu, 06 Jun 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

What started out as an unexpected proposal from Greg became very much alive on May 2nd, as 40 physicists poured into the classroom at VU University Amsterdam and started up their laptops. After introducing the teachers (Stefano Cozzini and Justin Ely) and the helpers, we were off: the very first Software Carpentry bootcamp in the Netherlands!

In the months leading up to the bootcamp, I had been surprised at the sense of timeliness the bootcamp generated. People here clearly see the need for computational training, most importantly young scientists themselves—available tickets were mostly gone within a week.

Feedback and Experiences from Southampton

Wed, 05 Jun 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I have recently organised a bootcamp at the University of Southampton, as part of my Software Sustainability Institute fellowship. I hadn't actually attended a bootcamp before (I was meant to, but unfortunately had to cancel) so it was a little nerve-wracking doing all of the organising - it all worked out well in the end though.

From a Helper to an Instructor

Sun, 02 Jun 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This blog post is actually a bit overdue as the first time I instructed at a Software Carpentry bootcamp was over a month ago in Manchester. However, as now I had an opportunity to teach twice (recently in Krakow), this gives me more credit to share my experience of taking a step up from being a helper to being an instructor.

Software Carpentry at INTECOL13

Sun, 02 Jun 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

On 21st August 2013, we'll be attending INTECOL13 in London's Docklands. The 11th International Congress of Ecology, hosted by the British Ecological Society and INTECOL, is one of the world's premier conference for ecologists. At the invitation of Matthew Smith of the BES Computational Ecology Specialist Interest Group and Microsoft Research, Software Carpentry and The Software Sustainability Institute have a talk and a session.

Krakow Bootcamp Experience

Thu, 30 May 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

On the weekend of 18th and 19th May Krakow saw the first Software Carpentry bootcamp. The main organiser was Damian Marchewka from the PhD students' association at the Jagiellonian University. Karin Lagesen came from Oslo to co-teach with me, and four local helpers: Eryk Ciepiela, Maciej Czuchry, Klemens Noga from ACC Cyfronet and Leszek Tarkowski from Infotraining provided great support throughout the whole bootcamp. There were 28 attendees - most of them postgraduate students and several faculty members. They represented a range of disciplines from mathematics and theoretical physics to biology, genetics and medicine.

The Great Licenceathon

Thu, 30 May 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Software Sustainability Institute's Simon Hettrick writes:

Intellectual Property (IP) is a thorny issue in academia, generally because people don't know who owns what. There's a lot of hearsay, rumour and—frankly—misunderstanding about IP ownership, so we've decided to run an experiment. And we're going to need some guinea pigs…
…
To get a better understanding of how academia deals with IP ownership, we're running The Great Licenceathon. This will see people at different organisations working out who owns their IP. They'll then blog about their experiences and we'll pick through those blogs to see whether there's some common lessons we can learn.
…
If you want to know who owns your IP, and you have time to ask a few questions at your organisation, then please get in touch.

What Does Done Look Like?

Sun, 26 May 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

After recent posts about where we are, our infrastructure, and our plans for the summer break, it seems like a good time to raise our sights and ask how we want Software Carpentry to run once it's all grown up, and what the practice of science will look like when every grad student in the world goes through our training. (Pause for maniacal supervillain laugh.)

What Does Victory Look Like?

Sun, 26 May 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A lot of changes are happening to science as I write this. Crowdsourcing, open reviews, citeable data, new ways to measure contributions, automatically tracking the provenance of every calculation—each has emerged from its chrysalis and is just waiting for its wings to dry so that it can take flight.

Our job is to give scientists the skills they need to nurture these ideas. For the last post in this series, I'd therefore like to talk about what victory will look like—i.e., how scientists' lives will be different if and when the things we teach become as routine as doing statistics. My vision is simple:

Our Infrastructure

Sat, 25 May 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As described in these posts from May, and October 2012 (which build on this one from April 2012), we use a complicated collection of tools to manage our work:

Browsercast

Fri, 24 May 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

David Wolever (a freelance developer here in Toronto) has released the first version of BrowserCast, an IPython Notebook plugin that allows you to synchronize a voice-over soundtrack with a step-by-step reveal of a notebook, thereby creating a screencast-style presentation in the browser. He has created two traditional (video) screencasts showing why BrowserCast is cool and how to create a presentation with BrowserCast, but if you really want to get a feel for it, you can use the bookmarklet that lets you give it a try with your own notebooks, or grab the plugin from GitHub and take it for a test drive.

Feedback from the Oxford DTCs

Fri, 24 May 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Having helped at two previous bootcamps (Oxford last year and UCL last month), a couple of weeks ago I had my first attempt at organising one at the University of Oxford, primarily aimed at students from the Doctoral Training Centres. Mario Antonioletti and Shoaib Sufi from the Software Sustainability Institute kindly did most of the instructing (I only ran two sessions), and Mario has already written a blog post with his perspective. Here I want to report the feedback we received from attendees and give some of my thoughts on the experience.

Planning for the Break

Fri, 24 May 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We've had a busy nine months: 55 bootcamps since we restarted bootcamps last September, with 19 more in the next two months. We're taking a break then to catch our breath: we only have one event scheduled between mid-July and the end of August, which will give us time to reorganize our online materials and figure out what we're going to do in 2013-14. The list below outlines things that I'd like to see done during that break. I'm sure you have others; I'd be grateful if you'd add them as comments, along with your thoughts on things we shouldn't do, or should do differently.

Where We Are (More or Less)

Fri, 24 May 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

In January 2012, John Cook posted this to his widely-read blog:

In a review of linear programming solvers from 1987 to 2002, Bob Bixby says that solvers benefited as much from algorithm improvements as from Moore's law: "Three orders of magnitude in machine speed and three orders of magnitude in algorithmic speed add up to six orders of magnitude in solving power. A model that might have taken a year to solve 10 years ago can now solve in less than 30 seconds."

Experiences with the Oxford DTCs

Fri, 17 May 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Mario Antonioletti has posted his experiences on being a first-time instructor at our bootcamp for the Oxford doctoral training centres, our second in Oxford, last week.

Wrapping Up at UC Davis

Fri, 17 May 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Jenna Lang has posted a great wrap-up on the bootcamp at UC Davis — with Python cookies!

Announcing Hack4ac

Thu, 16 May 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Hack4ac is a one-day hackathon in London, England, on July 6. Its goals are:

Demonstrate the value of the CC-BY licence within academia. We are interested in supporting innovations around and on top of the literature.
Reach out to academics who are keen to learn or improve their programming skills to better their research. We're especially interested in academics who have never coded before.

It looks like fun—if you're in the area and interested in getting involved, they'd be happy to have you there.

Stanford Bootcamp Recap

Thu, 16 May 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

On May 6th-7th, we hosted a bootcamp at Stanford University. Participants were students, post-docs and staff affiliated with the Center for Neurobiological and Cognitive Imaging and the Neuroscience Graduate Program and both units provided support for the workshop. Bob Dougherty, the research director at the CNI, helped raise a substantial portion of the funds to support the bootcamp (coffee!) and Prof. Miriam Goodman, from the Department of Molecular and Cellular Physiology, helped to get the neuroscience program on board and to secure additional funding from that program. Prof. Goodman was a participant in a SWC bootcamp at Berkeley last year and was eager to get SWC to Stanford. Instructors were Paul Ivanov, Bernhard Konrad and myself.

A Mention in Science Careers

Tue, 14 May 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Vijee Venkatraman has written a good article for Science Careers titled "When All Science Becomes Data Science", which mentions Software Carpentry.

Git vs. Subversion and Feedback in General

Fri, 10 May 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Software Carpentry's mission is to help scientists teach other scientists how to be better programmers. If we want to do that successfully, we need to be scientists ourselves. In particular, we need to base what we teach on evidence, not anecdotes or personal preferences.

For example: we taught Git at the Toronto bootcamp last week, and once again I think our learners would have absorbed more if we'd taught Subversion. Why? Well, take a look at this diagram by Oliver Steele (which I found on this page written by Nick Quaranto):

Make It Easier to (Re)use Your Data

Fri, 03 May 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Software Carpentry has focused on computing for most of its 14 years (primarily because that's what I'm most familiar with) but it's increasingly clear that we need to tackle other parts of the research cycle. One is the new ideas clustered around publication, discovery and metrics, which I'll discuss in a future post. The other is data management; we only touch on the topic right now, but it's as important to most scientists as crunching numbers, and how best to do it is changing rapidly. Luckily, a few of our friends have written a guide for the perplexed:

More Detailed Feeback from Melbourne

Fri, 03 May 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The hosts of our February bootcamp at the AMOS conference in Melbourne have collected some more detailed feedback from participants. I'm pleased that two thirds thought the content was just right, and even more pleased that 83% thought version control "must be taught".

A Rational Computing Process: How and Why to Fake It

Thu, 02 May 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Parnas and Clement's 1986 paper "A Rational Design Process: How and Why to Fake It" [1] is one of the most widely read in the history of software engineering. In it, they argued that designing software according to some particular process isn't what matters; what does is creating documentation after the fact to make it look as though a rational process was followed so that other people can retrace the designers' thinking without heroic effort. Acknowledging the messiness of reality wasn't new: a century ago, Poincaré wrote that most mathematicians figured out the proof after they had figured out the answer, and everyone knows that the description of the experiment that's put in the paper is almost never how the experiment was actually done [2]:

Translucent Badges

Thu, 02 May 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Digital badges are a hot meme right now. They let anyone, anywhere, issue credentials that are finer-grained than degree certificates or driver's licenses. Want people to know that you can change the oil in a car? There's a badge for that. Or that you can speak conversational Frisian? There's a badge for that too. And "backpack" sites make it easy to aggregate badges, including the ones that show you've created content for Software Carpentry, that you're qualified to teach it, or that you've helped to organize and run a bootcamp.

Pre-Assessment Results

Tue, 30 Apr 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Of the 29 people who responded to a brief questionnaire before a recent bootcamp, we have:

18	graduate students
4	postdocs
4	staff
1	faculty member
1	general public
1	special student

They describe their current expertise as:

1	master
10	have written a few programs
17	have written a little code
1	has no programming experience

Here's how they thought they could do on some simple tasks:

Write a short program to read a file containing columns of numbers separated by commas, average the non-negative values in the second and fifth columns, and print the results.

11 could do it easily

15 could probably struggle through

3 wouldn't know where to start
In a directory with 1000 text files, create a list of all files that contain the word "Drosophila", and redirect the output to a file called results.txt.

12 could do it easily

10 could struggle through

7 wouldn't know where to start
Check out a working copy of a project from version control, add a file called paper.txt, and commit the change.

3 could do it easily

4 could struggle through

22 wouldn't know where to start
A tab-delimited file has two columns: the date, and the highest temperature on that day. Produce a graph showing the average highest temperature for each month.

19 could do it easily

0 could struggle through

10 wouldn't know where to start

An Update on Cumulative Enrolment

Mon, 29 Apr 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

It's been a busy few months, and the next three promise to be busier still. Somewhere in there we helped our two thousandth learner, and if everything goes well, we'll reach 2500 by mid-summer.

Many thanks to the entire team for their hard work.

Sound Software Competition

Sat, 27 Apr 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are very excited to announce the first competition for SoundSoftware.ac.uk prizes for reproducibility in audio and music research. Our goal is to promote the development and release of sustainable and reusable software and datasets alongside published research.

In a few of our recent papers and presentations we've seen that much audio and music research work is published without the accompanying software implementations. One reason is a lack of self-confidence - the fear that one's code is not good enough to share. (You can read more about this in our ICASSP 2012 paper, Towards Software Reuse in Audio and Music Research and in the Institute's post Haters Gonna Hate - why you shouldn't be ashamed of releasing your code). We believe that one way of countering this fear is to promote the idea that sharing is a worthwhile goal in itself.

Bootcamp Recap: Middle East and South Africa

Wed, 24 Apr 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Although running a Software Carpentry Bootcamp is a rewarding experience on its own, sometimes we get the opportunity to get to travel somewhere really interesting as volunteer educators. This March, I put on bootcamps at the American University of Beirut in Lebanon, King Abdullah University of Science and Technology in Saudi Arabia, and Stellenbosch University in South Africa. For all three bootcamps, I taught as primary instructor, with a total of approximately 12 hours of instruction spread out over 2 days. I adapted content from Software Carpentry as well as previous classes and lectures I've taught in scientific computing, reproducibility, and software development.

Manchester Once Again

Wed, 24 Apr 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Mike Jackson has posted a summary of Software Carpentry's second bootcamp in Manchester this month. We're hoping to visit again before the year's end—please keep an eye on this blog for announcements.

Software Carpentry at SciPy 2013

Tue, 23 Apr 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Several members of Software Carpentry's vast network will be traveling to Austin, TX in June for SciPy 2013. We are especially excited that Katy Huff and Matt Davis will be teaching a tutorial titled “Version Control and Unit Testing for Scientific Software.” The tutorial is aimed at beginners and the only prerequisite experience will be basic use of Python and the shell. Matt and Katy will cover why testing and version control are good ideas for working scientists and show how to get started with each. Here's the tutorial abstract:

Feedback from Arizona

Fri, 19 Apr 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Julie Messier, a plant ecologist and doctoral candidate at the University of Arizona who organized the recent bootcamp there, has posted about the bootcamp on her blog.

Spreadsheets, Retractions, and Bias

Fri, 19 Apr 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Just in case there's any misunderstanding:

I'm not suggesting that scientists should use Excel.

Now, with that out of the way… Guy Deutscher's wonderful book Through the Language Glass devotes several pages to the Matses people of South America:

Their language…compels them to make distinctions of mind-blowing subtlety whenever they report events… [Verbs have] a system of distinctions that linguists call "evidentiality", and as it happens, the Matses system…is the most elaborate that has ever been reported… Whenever Matses speakers use a verb, they are obliged to specify—like the finickiest of lawyers—exactly how they came to know about the facts they are reporting… There are separate verbal forms depending on whether you are reporting direct experience (you saw someone passing by with your own eyes), something inferred from evidence (you saw footprints on the sand), conjecture (people always pass by at that time of day), or hearsay (your neighbor told you he had seen someone passing by). If a statement is reported with the incorrect evidentiality form, it is considered a lie.

I thought about this in the wake of reactions to two reports this week of errors in scientific work. The first was the discovery by Herndon and others of mistakes in Reinhart & Rogoff's widely-quoted analysis of the relationship between debt and growth—mistakes that were due in part to bugs in an Excel spreadsheet. Quite a few of the scientists I follow on Twitter and elsewhere responded by saying, "See? I told you scientists shouldn't use Excel!"

Feedback from the EGI Forum

Tue, 16 Apr 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

At last week's EGI Forum in Manchester, we ran a day of bootcamp "highlights". These were 1.5 hour taster sessions drawn from bootcamp sessions. Our sessions covered,

Using version control to record provenance and collaborate more easily.
Using testing to help ensure your software, and results, are correct.
Data management using a NoSQL database to manage your data more easily.

We had 15-20 attendees for each session, mostly from a computer science or software development background, and in systems support or technical roles. The interests of the attendees were on the specific technologies covered in each session (Git, Python and nosetests, MongoDB) rather than the concepts underlying these, as most attendees were already familiar with these.

Feedback from UC Berkeley

Tue, 16 Apr 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Last weekend we held a bootcamp at the University of California, Berkeley, that was targeted for environmental scientists and ecologists. We had a great group of attendees (31 registered, 28 at start, and 24 at end) and instructors/helpers, and, as a major bonus, had no major technical or logistical emergencies to deal with (but see below for some minor ones).

Before the workshop, we had participants fill out a brief pre-workshop survey that we set up on Google Drive. The idea was to gather some initial impressions that would help us tailor the workshop content. We learned a few particularly useful tidbits from this exercise. First, about half the attendees had tried Python before and half had never used it. Second, nearly a third had used some form of version control before (big surprise), and half of these had used Git. Third, the most requested topics, by far, were related to importing and manipulating different types of data and to integrating Python with R and/or ArcGIS.

A Bootcamp in Toronto May 9-10, 2013

Mon, 08 Apr 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are offering a bootcamp at Mozilla's Toronto office on May 9-10, 2013. Please register here if you would like to take part.

Evaluation Revisited

Mon, 08 Apr 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Caitlyn Pickens (a graduate student at Michigan State who is interning with us this summer) has just posted an article about the validity of different kinds of assessment. Caitlyn's goal is to build some tools we can use to gauge the impact Software Carpentry is having—like almost everything involving human beings, it turns out to be more complicated than it first appears, and her post helps lay out the background knowledge needed to understand why.

Installation Revisited

Mon, 08 Apr 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Regular readers (and anyone who has attended a bootcamp) will know that getting learners' laptops set up is the single biggest headache we have. Titus Brown has just posted a discussion of what goes wrong, and some ways we could try to improve things. If you have suggestions to contribute, please add comments there.

Announcing a Bootcamp for Women in Science and Engineering

Sun, 07 Apr 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

On June 24-25, 2013, Software Carpentry will run a computing skills bootcamp in Boston for women in science, engineering, and medicine. With three rooms and six instructors, it will be one of the biggest events we've ever done.

bootcamps alternate short tutorials with hands-on practical exercises. Learners are taught tools and concepts they can use immediately to increase their productivity and improve confidence in their results. Topics covered include the Unix shell, version control, basic Python programming, testing, and debugging—the core skills needed to write, test and manage research software.

An Image Analysis Success Story

Fri, 05 Apr 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I am a alumni of the bootcamp in Paris in June 2012. I have been working on a paper on digital image analysis for experimental studies in ecology and evolution working with small-scale model organisms such as aquatic microbes, insects or fish. Besides explaining the principles of image analysis, we provide scripts in Python, R and ImageJ so people can directly use the methods. The paper has now been published, and I'd like to share our success with the Software Carpentry community.

Connecting Bootcamp Content to Motivation and Best Practices

Wed, 03 Apr 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As our next Software Carpentry bootcamp quickly approaches, we have been struggling with many of the usual questions of what to teach and what not to teach. We expect a broad array of skills and experience (pre-assessment is yet to be completed), and want to make sure that all the learners leave with a clear motivation to build on what they learn.

Using the IPython Notebook as a Teaching Tool

Sun, 24 Mar 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I had a fruitful discussion with Jon Pipitone today about using the IPython Notebook for teaching. Long story short, there are several possible approaches, but we can see problems with each. To set the stage, here are the two "pure" models most widely used for teaching programming today:

Frozen: the instructor embeds snippets of code in slides (which can be in LaTeX, PowerPoint, HTML, or some other format).

Advantages:

Easy to interleave code and commentary.
Easy to draw on top of code to highlight, make connections with commentary, etc.
Easy for other people to re-use.
Easy to add presenters' notes.

Disadvantages:

The code isn't immediately executable, so some combination of manual checking and tooling has to be used to ensure that the output shown is up-to-date with the code, that the code shown is up-to-date with any hand-out files, etc.
It's really easy for instructors to race through slides too fast for learners can follow.
"Watch and listen" is a passive learning model, and the more passive learners are, the less they learn.

Live Coding: the instructor types into an interpreter as she's teaching; the only thing on the screen is her code and its output, and everything else is delivered verbally.

Advantages:

Allows responsive improvisation: the instructor can answer "what if?" questions much more easily.
Constrains the speed of presentation (somewhat).
Facilitates "sideways knowledge transfer", e.g., learners can pick up keyboard shortcuts and other "hows" of coding, etc.
Learners learn more if they are typing in code as they follow along.

Disadvantages:

Learners now have to type and watch at the same time; the former often distracts from the latter (particularly if they make typing mistakes that they can't spot themselves, so that they wind up with a triple burden of watching, typing, and debugging simultaneously).
Learners walk away with just the code, not what was said about it, and code alone can be hard to re-understand.
It discourages the use of diagrams (instructors can't doodle directly in the Notebook the way they would on a whiteboard, and "let me import this now" is clumsy).
There's no obvious place to store the presenters' guide.

With practice, preparation, and the right A/V setup, instructors can use a hybrid model:

Cumulative Enrollment

Sun, 17 Mar 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

One measure of how well we're doing is the number of people we've helped. Here's what the last 15 months have looked like:

Testing Image Processing

Sun, 17 Mar 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Testing has always been part of Software Carpentry, but it's also always been one of our weak spots. We explain that testing can't possibly uncover all the mistakes in a piece of software, but is useful anyway, then talk about unit testing and test-driven development. Separately, in the extended program design example, we demonstrate how to refactor code to make it more testable.

What we don't do is show people how to test the science-y bits of scientific software. More specifically, our current material doesn't contain a single example showing how to check the correctness of something that does floating-point. You won't find much mention of this in books and articles aimed at mainstream programmers either: most just say, "Oh, round-off," then tell you to use an almostEquals assertion with a tolerance, without telling you how to decide what the tolerance should be, or what to do when your result is a vector or matrix rather than a single scalar value.

Snowstorms and Blackouts in Virginia

Fri, 15 Mar 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Despite inclement weather, last week's bootcamp at the University of Virginia went well. According to Steve Crouch, 33 researchers spent two days learning some useful computing skills from Steve, Carlos Anderson, and Ben Morris. Many thanks to Stephen Turner for hosting it—we hope to be back soon.

New Camps Coming Up

Thu, 14 Mar 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We have arranged plenty of bootcamps in the last few weeks; here's what's coming up:

Site	Dates	Registration Status
American University of Beirut	Mar 20–21	not yet open
King Abdullah University of Science and Technology	Mar 24–25	not yet open
University College London	Apr 4 and 8	not yet open
EGI Forum, Manchester	Apr 11	open
University of Wisconsin - Madison	Apr 29–30	not yet open
University of Oxford	May 9–10	restricted
NESCent	May 16–17	not yet open
Duke University	May 20–21	not yet open
University of Massachusetts Amherst	May 23–24	not yet open
University of Alberta	May 30–31	open
Tufts University	June 3–4	not yet open
University of Southampton	June 3–4	restricted
University of Oklahoma	July 1–2	restricted
Indiana University	July 11–12	not yet open
Utah State University	July 16–17	not yet open
University of Notre Dame	July 18–19	not yet open
University of Iowa	September 5–6	not yet open

And of course we're always looking for more eager researchers who want to organize a camp at their institution. If you're interested, please contact us for more information about what's involved.

Second Round at Lawrence Berkeley

Wed, 13 Mar 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Last week we finished up our second workshop at Lawrence Berkeley Lab. This time around, we tried a more free-form version of the feedback exercise in which students live-typed their feedback into an Etherpad at the end of each day. This went very well, as it gave us time to gather unique comments, get a rough idea of the amount of support for each comment, and to respond to some of the comments in real time. We asked three questions: what was good, what was bad/confusing, and what we didn't do or talk about that we should have.

A New Testing Framework for MATLAB

Tue, 12 Mar 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Steve Eddins just announced a new unit testing framework for MATLAB. Based in part on lessons learned from his earlier mUnit framework, it has everything you'd want from a modern framework: setup and teardown, analysis, and a bag full of helpful assertions. If you're using MATLAB, you should definitely check it out.

First Round at Lawrence Berkeley

Tue, 05 Mar 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We just wrapped up a two-day workshop at the Lawrence Berkeley National Laboratory, and another is due to start tomorrow. Here's what worked and what didn't from the first one:

Good

Bad

Doing examples ourselves
Liked Python examples
Interactive
Greg kinda knows this stuff
Like the history lessons etc.
Went through a lot of things
Liked IPython Notebook
Like intro to Git
Intro to shell
Examples are incremental
Good mix of theory and hands-on
Sticky notes
Everything is open source
Liked keeping functions small
Liked red-green-refactor

Little more structure please
Lost with how to use Git
Wanted a cheat sheet
Wanted more Python
Not a break Monday aft.
Not clear how to apply all tips to all things
Wanted more structure Monday aft. (and I was teaching it)
Wanted cookies in the afternoon
Too many things at once
Poor choice of room
Need a book or handouts
Didn't see Python interact with R and MATLAB
More specific to specific groups
Bringing Python back to the shell etc.
How to save terminal session/notebook/etc.

Many thanks to Shreyas Cholia, Adam Stone, Nina Lucido, Geoff Oxberry, Matthew Brett, Paul Ivanov, and all our learners for making it a success.

Teaching with ipythonblocks at UW

Sat, 02 Mar 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The recent University of Washington bootcamp was my first chance to try teaching Python with ipythonblocks, and it was a great success!

The students reported that the immediate feedback from ipythonblocks was a great way to see what their code was doing. We were able to move quickly through topics like for-loops and if-statements in relative comfort, and even cover slicing with ease.

Alternative Teaching Models

Fri, 01 Mar 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Our two-day bootcamps are working well, but that doesn't mean they're the best—or only—way to teach basic computing skills to scientists. We've been kicking around other ideas recently, and we'd welcome your input as well.

Become part of a regular university course.

Use the first few days of a course to teach Software Carpentry stuff, and the rest to teach "computational methods in [XYZ]".
Counter-argument: SWC doesn't really fit a traditional lecture format, but can fit with in-class projects and discussions, follow-me tutorials, etc.
And any course that includes SWC will have to be a fairly special and overly interactive course.

We have experimented at Columbia with running a two-day bootcamp at the start of a regular course, followed by 2 hours/week for the remainder of the semester—we should be able to report back on how well it works by April or May.

Feedback from UW Room B

Fri, 01 Mar 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Doug Latornell and Matt Davis taught in "Room B" at the recent University of Washington bootcamp. It was a great group of very enthusiastic students! Thanks also to our very capable helpers Jake Vanderplas and David Leen!

Here's our traditional good/bad feedback:

Good	Bad
Learning about Python and NumPy, especially the practice exercises.	Too much shell stuff (already shell proficient).
We talked about best practice issues in software development.	Difficult to keep up with instructor when switching between screens.
Great holistic crash course on Python and git. Did well with a range of expertise the room.	Too much typing while trying to listen.
Overall great intro to many great topics.	Would have preferred basic Python scripting instead of NumPy intro.
The one-on-one help and the exercises.	matplotlib and NumPy were a little dry.
Pace was quick, almost too fast, but not quite!	The Python part was rushed and we didn't have time to see all of the code.
Intuitive approach.	Command line tutorial was slow (but that was probably necessary for some).
Intro to IPython Notebook.	A bit too fast in some content.
Very well structured. Good collection of topics for introduction. Staff very responsive.	NumPy section could use more exercises/interaction.
Very helpful and useful.	Wanted to learn about SciPy.
The existence of Software Carpentry and the integrated exercises.	The software engineering felt too rushed -- if I made an error in my code I easily fell behind and was lost.
	Wanted just Python.

Washington Went Well

Fri, 01 Mar 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Last week, we put on the largest Software Carpentry bootcamp ever at the University of Washington: three rooms, six instructors, ten helpers, and 93 students spent two days together learning about Python, Git, the Unix shell, and a bunch of other useful things. Many thanks to Prof. Bill Howe for making it all possible—we look forward to coming back soon.

A Bootcamp for Women in Science and Engineering

Wed, 27 Feb 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This summer NumFOCUS and Software Carpentry are hosting a software skills bootcamp for women in science and engineering in Boston. We are seeking sponsorship from companies and others who would like to help raise the $6000 needed to make this happen. We'd appreciate your support, and would be grateful if you'd help spread the word.

Workshop for High-Energy Physics at UCL, Part 2

Wed, 27 Feb 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Three months after part 1, the second half of the UCL Software Carpentry workshop for High-Energy Physics took place on Friday 15th February 2013.

The experimental format with two sessions three months apart might have worked better if, as envisaged, we had managed to get an online discussion going in the gap. Unfortunately this did not work out. Despite several attempts at chasing up the participants, no-one registered or posted anything on the course web site. After talking to some of the students, I think their interest was just not great enough to overcome the bureaucracy involved in registering on the UCL Moodle server and the tendency to focus on their other (assessed) courses and research projects. Attendance was down from 65% to 30% (6 of 20 students), although those who did attend were clearly motivated, worked hard and asked thoughtful questions. With two helpers (James Hetherington and HEP postgrad Sam Cook) and myself as instructor, we had a student:staff ratio of 2:1.

Expanding Our Bootcamp Types

Fri, 15 Feb 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Prompted by a recent email I gave some thought to our plans to expand the types of bootcamps we offer, especially when it comes to a sort of "next level" bootcamp and whether we might offer bootcamps that introduce traditional software developers to the world of scientific programming:

There is significant discussion on the best way to expand Software Carpentry. Longer bootcamps, advanced bootcamps, etc. At some point we need to start actually experimenting with these things, but we have many constraints, most notably a severe shortage of person-hours to devote to developing curriculum and staffing events. Scientists, it turns out, are very busy people!

Wrapping Up in Melbourne

Fri, 15 Feb 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Two long but useful days at Melbourne Uni have just wound down, and five dozen meteorologists and climate scientists have given us feedback on a well-run bootcamp:

Good

Bad

splitting big problems into small pieces
SQLite is easy
functions instead of loops
Python was easy
naming functions after pets
hands-on application at the same time as instructor
morning tea
dictionaries
explaining why, rather than just what
swapping variables
using our own computers
copy/paste onto Etherpad
arrays in NumPy
sticky notes
a bit of motivational speaking
location (I'm not from Melbourne, eh)
Subversion FTW!
Git + BitBucket
Python Notebook
hey, there's useful stuff online
helpers
red/green/refactor
really practical exampels of test-driven development
Greg tells stories
lot of participation
talk to other people during the breaks
regular expressions
cheap!
seeing how programmers think
realizing how much room for growth I have
scared in the first ten minutes...
the material is free
peer instruction

WiFi
didn't do any meteorological data sets
the jokes
viewing angle
installing stuff
couldn't use some advanced stuff (I'm a beginner)
coffee
running right after conference
squeezing four half days into two full days (maybe try 3?)
not enough exercises
not long enough
content could have been more focused
didn't do NetCDF files
a bit of motivational speaking
didn't see comparison of Python with R
lot of spread of material/ability
longer breaks (I'm so tired)
databases irrelevant to me
IPython/NumPy confusing
wasn't in New Zealand
not enough time on version control
wasn't in Fiji
hated SQL
Greg speaks waaaaaaaaay too fast
learned a lot about Vi
hopping
pretty intense, dude
took a long time to restart after breaks
more on plotting
lighting in the room is uneven
more about linking Python to other things
...after first ten minutes, it was OK

More News from the UK

Thu, 14 Feb 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Two pieces of news from the UK:

The folks at Queen Mary, University of London just ran a two-day bootcamp for students in Media and Arts Technology, and will be running another such event soon.
Mike Jackson at the Software Sustainability Institute has posted some quick alternatives to bootcamps. If you'd like help with or advice on these, please get in touch.

Registration for Amsterdam Bootcamp is Open

Thu, 14 Feb 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Registration is now open for the upcoming bootcamp for physicists at VU University in Amsterdam.

Second Dry-Run of DiRAC Driver's License Exam

Wed, 13 Feb 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Back in August we did an alpha test of our driver's licence for DiRAC in conjunction with The Software Sustainability Institute. In the spirit of iterative development, we revised our test in light of our experiences and two weeks ago we did a 2nd dry-run. Four researchers based at University College London kindly agreed to take part as examinees, Dr. Jeremy Yates of DiRAC made the local arrangements, and James Hetherington of UCL, a newly-appointed fellow of The Software Sustainability Institute, provided local assistance.

Partnering with the SSI

Tue, 12 Feb 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are very pleased to announce that the Software Sustainability Institute has agreed to coordinate Software Carpentry activities in the UK. The SSI became involved in Software Carpentry in 2011 when they started developing online lectures for us on advanced shell techniques and file management in Python. In April 2012, they participated in the first general UK bootcamp at University College London; a fortnight later, in conjunction with the Digital Institute at Newcastle University and SoundSoftware, they delivered the first bootcamp to be run entirely by UK tutors. They have since delivered the majority of bootcamps run in the UK and been instrumental in helping us grow in Europe.

Correctness Isn't Compelling

Mon, 11 Feb 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The final report from the ICERM workshop on Reproducibility in Computational and Experimental Mathematics is now available, and its appearance has prompted me to explain why we don't put more emphasis on reproducibility in Software Carpentry. Long story short, it's because scientists don't care because they're not rewarded for doing so. Here's the math:

Assume five million scientific papers were published in the decade 1990–2000. (The actual number depends on what you include as a "paper", but our reasoning holds.)
Of those, perhaps a hundred have been retracted because of honest computational irreproducibility ("honest", because fraud isn't part of this argument).
That means the odds that a scientist will have to retract a particular paper because someone noticed that her calculations couldn't be reproduced are one in fifty thousand.
So if the average paper takes eight months to produce, and scientists work six-day weeks, that means it's only worth spending 115 extra seconds per paper on reproducibility as insurance.

Different assumptions and models will naturally produce different answers, but won't change the conclusion: given the system we have today, investing extra time to make work reproducible as insurance against error isn't economical. RR's advocates may respond, "That's why we're trying to change the system," but chicken-and-egg traps are notoriously difficult to break out of: if people don't care about the reproducibility of their own work, they're unlikely to check it when reviewing other people's work, and around and around we go. Trying to get them to be an early adopter of new practices (which aren't yet rewarded consistently by their peer group) is therefore a very hard sell.

UBC Went Well

Mon, 11 Feb 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Ted Hart and Ethan White just wrapped up a bootcamp at the University of British Columbia, and by all accounts it went very well:

Good

Bad

learn terminal
explanations are clear
SQLite is awesome
learn some Python
following along on screen
enthusiasm
content was relevant
practical software
SQLite is very helpful in data management
Etherpad
SQLite
SQLite: could use it tomorrow
like lots of problems with partners
iPython & SQLite
liked different levels as beginner so could get help
Etherpad ++ for collaborative note taking
current & up to date
Git
helpers and red light green light
pace was good
great intro to lots of methods (handled volume of info well)
free and open source software
good overview of things to improve workflow
collaborative notetaking and chating and small problems
Git from the command line (Linux guy)
Git & SQL, can use and share now
hands on problems well balanced with learning
good job balancing info and level variation

better examples of why Python is cool
disecting existing code
want to be inspired by an example of something cool
but not far enough to implement (longer)
cool projects
cheat sheets
brief one line explanation of everything before class & tailored workshops to specific disciplines
too hard to keep up
installation is much more difficult than R
fewer things so that we can actually use it
new to programming more exercises
not as clear how to use shell so less useful
shell: examples of why we should care, e.g., working across files
need to know more about iPython
lot of information, too much at times (fun stuff was at end)
didn't get to interconnecting the pieces (collaborative Git project) [group repo to play with]
group by OS or other similar interests
would have like to use Git from terminal
where are things coming from and going to
iPython notebook disconnected from scripting workflow
wanted SQL to R/Python (would have come back for another)
more time to integrate everything
why Python: spend a few minutes explaining why we chose it and what the alternatives are
expected it to be more advanced and focused on integration (more info)
"here's what you're going to learn" at beginning
not far enough in Python to know where to go next
some of the one and one help would have been good to share with the group
more difficult problems more collaboratively

Macquarie Went Well

Fri, 08 Feb 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Last week's bootcamp at Macquarie University went well: I fumbled the introduction to Python on Friday morning (which I haven't done in a long time), but with a bit of help from Eli Bressert and great crew of helpers, we got the afternoon back on track. Here's the usual good vs. bad feedback:

Good

Bad

Understanding the syntax of Python
The stories
The flow of material
Cygwin
Linking programming to thinking
Understanding size of manageable tasks
Version control
Sticky notes
Embedding provenance in files
Database concepts
Learning good work habits
Coding with loops
Free!
Let people make mistakes and then correcting
Interactive nature of the learning experience
I got to show my work to real human beings!

Installation
The stories
Python introduction (Greg screwed up)
Some programs don't work on some people's computers
Air conditioning
Struggling to switch between languages
Bit more practice with coding
Python in Cygwin sucks
"You should only work eight hours a day" — yeah, right
Not enough on databases
Couldn't see the screen
Discovering R isn't as good as I thought it was
Not having enough examples

We Have a Facebook Page

Wed, 06 Feb 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Software Carpentry finally has a Facebook page. Please drop by!

A Short Report from Tuebingen

Sun, 03 Feb 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Luis Figueira has posted a short summary of last week's bootcamp at the Max Planck Institute in Tuebingen, Germany. Judging by the number of people who have contacted us since to ask about volunteering at the next one, it appears to have gone very well.

Features and Scope in Open Courseware

Sun, 03 Feb 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A couple of weeks ago, Brian Granger (one of the core developers of IPython) posted some thoughts on features and scope in open source software. In it, he enumerates some of the risks associated with constantly adding new features to a piece of software (open source or otherwise):

Additional complexity in the code base (which makes future work more difficult).
Increased "surface area" for bugs (the more features there are, the more places a bug might be lurking).
Increased documentation and support time.
It forces developers to specialize, which makes "big picture" thinking harder.
Increased testing effort.
A more complex user experience. (Microsoft Word, anyone?)
Opportunity costs: time spent working on X is time not spent working on Y.

He then enumerates some things projects can do to throttle growth down to manageable levels:

The Missing Side of the Triangle

Sun, 03 Feb 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A few weeks ago, John Cook posted the following:

In a review of linear programming solvers from 1987 to 2002, Bob Bixby says that solvers benefited as much from algorithm improvements as from Moore's law.

Three orders of magnitude in machine speed and three orders of magnitude in algorithmic speed add up to six orders of magnitude in solving power. A model that might have taken a year to solve 10 years ago can now solve in less than 30 seconds.

A Short Report from Utah State

Sat, 02 Feb 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

(The first in an occasional series about other ways people are teaching computing to scientists.)

My "Advanced Programming and Database Management for Biologists" course at Utah State is based almost entirely on Software Carpentry (though the full curriculum is obviously not just what we teach in two days). It is a blended classroom: for about 2/3 of the semester the students view SWC material before class, at the beginning of class I provide 5-10 minutes of additional introduction to the material or answer questions the students had after viewing the videos, and then they spend the rest of class working on problems related to the material. They also do an independent project related to their research and the last 1/3 of the semester involves primarily me helping individual students work on their research projects (plus I present a set of more advanced or discipline relevant topics that they choose by voting on, e.g., the GPU peta cloud). It's been fairly popular for a rather nerdy graduate class and scores off the charts.

A Bunch of Bootcamps

Fri, 01 Feb 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We've had a busy month: we've run bootcamps at the Max Planck Institute in Tübingen, the Technische Universiteit München, Mozilla's office in Toronto, McGill University, Columbia University (a double-header that's still running today), and the universities of Waterloo, Chicago, and British Columbia. We've also booked a bunch more, some of which are now open for registration:

Site Dates Registration

University of British Columbia February 4-5 Open

Macquarie University February 7-8 Full

AMOS Conference (Melbourne) February 14-15 Full

University of Washington February 25-26 Full

Lawrence Berkeley National Laboratory March 4-5 & 6-7 For LBL staff

University of Virginia March 7-8 Coming soon

Utah State University March 23-24 Open

University of Arizona April 4-5 Open

University of Manchester April 18-19 Open

Telecom ParisTech April 20-21 Open

Vrije Universiteit (Amsterdam) May 2-3 Coming soon

GEOMAR (Kiel) May 6-7 For members of ISOS, IMAP and the SFB754

Howard Hughes Medical Institute (Virginia) May 6-7 For HHMI staff

Stanford University May 6-7 Coming soon

University of California Berkeley May 6-7 Coming soon (in R!)

Lawrence Berkeley National Laboratory May 9-10 For LBL staff

University of California Davis May 13-14 Coming soon

University of Oslo July 3-4 Open

University of Bath July 15-16 Coming soon

And we're hoping to finalize arrangements with the following sites real soon now:

Next-Generation Sequencing Course 2013

Fri, 01 Feb 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Analyzing Next-Generation Sequencing Data

http://bioinformatics.msu.edu/ngs-summer-course-2013

June 10th–20th, 2013
Kellogg Biological Station, MSU
Course sponsor: NIH.

Instructors: Dr. C. Titus Brown, Dr. Ian Dworkin, and Dr. Istvan Albert.

Course Description

This intensive two week summer course will introduce attendees with a strong biology background to the practice of analyzing short-read sequencing data from Illumina and other next-gen platforms. The first week will introduce students to computational thinking and large-scale data analysis on UNIX platforms. The second week will focus on mapping, assembly, and analysis of short-read data for resequencing, ChIP-seq, and RNAseq.

A Bootcamp at Mozilla

Wed, 30 Jan 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We ran a two-day bootcamp at Mozilla's office in Toronto last week for people from local research hospitals. It seemed to go well: only 28 of the 37 who'd registered showed up, but with a couple of exceptions because of scheduling conflicts, everyone who showed up on day 1 came back on day 2. Many thanks to Dhavide Aruliah, Yele Bonilla, Mike Conley, Gabriel Devenyi, Fan Dong, and Blake Winton for helping out.

Teaching R at UBC

Wed, 30 Jan 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This January we ran a second R-based bootcamp at the University of British Columbia over three days. It was taught by myself, Rick White, Bernhard Konrad, Davor Cubric and Jenny Bryan. The first day was a week before the last two days, and covered a basic introduction to R. The last two days was an advanced R workshop / Software Carpentry. We covered the following topics:

Novelty, Efficiency, and Trust

Mon, 28 Jan 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I've spent a lot of time trying to figure out what the "big picture" is for Software Carpentry. What are the best practices every scientist should master? What are the principles of computational thinking? How exactly are we helping people? My latest attempt to put this all in a nutshell has three strands: novelty, efficiency, and trust:

By improving scientists' computing skills, we're helping them do things they couldn't do before.

We're also showing them how to do things with less effort.

Finally, scientists can have more confidence in results (both their own and others') that are produced our way.

The second of these, efficiency, is what sets us apart from other "computing for scientists" initiatives. On on side, supercomputing and big data have opened up entirely new kinds of science, just as radio telescopes and PCR did. On another, advocates of open access and reproducible research are changing the way science is done, and by doing so, making it easier for people to check and build on each other's work. Our goal is to help them implement all of these things without superhuman effort—to do with ten minutes of scripting what used to take an hour of mousing around in a spreadsheet.

Visualizing Nuclear Fuel Inventories

Thu, 24 Jan 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I was working on characterizing the used nuclear fuel in the United States and needed to calculate the isotopic inventories for 174,000 fuel assemblies. This seemed like an overwhelming project and decided to see if using a database could help me. With the help of the online lessons from Software Carpentry I was able to write a Python script that would run the needed information and then store the results in a Sqlite database. I then used an open source tool RapidMiner to visual that data:

How to Become an Instructor

Wed, 23 Jan 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As we've mentioned elsewhere, our instructors are volunteers who donate their time because it's fun, because it makes the world a better place, because they learn things themselves from teaching, and because it's good for their careers. But how did they become instructors? And how can you become one too?

Starting with the first question, 21 of our 31 instructors are people who figured out how to do hygienic computational science before we met them (or in some cases, taught us some of what we now teach). The other 10 started as learners: they attended a bootcamp, then volunteered to help with another, and graduated from that to teaching. That's how we see the pool of instructors growing in future, so one of my main jobs this year is to regularize that.

Record and Playback in the IPython Notebook

Tue, 22 Jan 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We decided last fall to start teaching Python using the IPython Notebook rather than a plain old command-line interpreter. It will take us a few months to complete the transition, but one of the things that we're hoping to do in the meanwhile is build a record-and-playback plugin for the Notebook. We think this will be more useful than the screencasts we currently have because:

Online Office Hours

Mon, 21 Jan 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Starting this week we're trying something new: one-on-one online help sessions from instructors for anyone who has taken a bootcamp with us.

Bootcamps are great for introducing scientists to useful technology and new ideas about how to automate tasks and organise code. Our bootcamps try as much as possible to be relevant to scientists' daily workflow, but they're only two day long events; what happens when participants go back to "real life" and start to fit what they've learned into their daily work?

University of Chicago in January

Wed, 16 Jan 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This last Saturday and Sunday, Katy Huff and I had the pleasure of organizing a Software Carpentry workshop at the University of Chicago. We had 77 people attend in total and by the end of the day on Sunday we remained 45+ strong. (Frankly, who doesn't want to listen to me rant about documentation at 4 pm on a Sunday afternoon?!) This weekend was unique in that we had a host of first time instructors:

Montreal in January

Mon, 14 Jan 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This weekend was the first time I had instructed at a SWC bootcamp, and I'd only observed one other (Greg Wilson in Halifax in July 2012). By the new standards for instructors Greg's trying to put in place I would probably not even qualify, but hey, we're in a growth phase right now; we take a few chances, eh? I felt my inexperience acutely, but despite that I think I can say that the bootcamp went well, thanks to a great group of attendees and helpers and my co-instructor, Jessica McKellar. The site organizers were Alex Demarsh and Rolina van Gaalen, and the other helpers were Greg Ward, Julia Evans and Jonathan Villemaire-Krajden.

Teaching Commercially

Fri, 11 Jan 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A couple of people have contacted us recently to ask about running Software Carpentry bootcamps for companies. Our material is all Creative Commons licensed, so anyone who wants to use it for corporate training can do so, and doesn't need our permission. What does need our permission is using our name and logo, since they're trademarked. We're happy to give that permission if we've certified the instructor [1] and have a chance to double-check the content [2]. It would be great if starving grad students could help pay their bills from this, in the way that many programmers earn part or all of their living from open source software, so if you'd like to give it a try, please get in touch.

PLoS Ad for Software Carpentry

Thu, 10 Jan 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The folks at PLoS Computational Biology kindly placed an ad for Software Carpentry on their home page today—thanks!

The Art of Cold Calling

Sat, 05 Jan 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

An updated version of this post is now available.

A few people have asked how we go about approaching potential workshop hosts. The short answer is, however we can; the longer answer is, we collect names any way we can—from journal articles, watching who follows us on Twitter, just bumping into people—and send emails like the one below to people who are publishing there. A few things to note:

Advice From a Newbie No More

Fri, 04 Jan 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Adina Chuang-Howe recently wrote a great blog post titled "Advice from newbie to newbie", in which she gives some advice to her younger and not-yet-computationally-proficient self:

Commit (and stop whining)

Use the right tool for the job

Take Command

Install and use a text editor

Learn to manipulate files

Shortcuts!

Learn a programming language

Become addicted to print statements (I'd actually say "...to a debugger", but never mind)

Do something relevant

Start automating

and most importantly of all: "You can do it!" The full article is great advice to anyone who's getting into computing, scientific or otherwise, and well worth reading.

Why We Teach

Fri, 04 Jan 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Data Sharing and Management Snafu in 3 Short Acts: you have to laugh, because otherwise you'd cry...

Computer Science Curricula 2013

Sun, 23 Dec 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Following a roughly 10 year cycle, the ACM and IEEE Computer Society jointly sponsor the development of a Computing Curricula volume on Computer Science. These volumes have helped to set international curricular guidelines for undergraduate programs in computing. Planning for the next volume in the series began in the summer of 2010, and the Version 0.8 (Ironman) draft was published a couple of months ago. It divides topics into:

Code of Conduct

Fri, 21 Dec 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Software Carpentry is dedicated to providing a harassment-free learning experience for everyone, regardless of gender, sexual orientation, disability, physical appearance, body size, race, religion, or choice of text editor. Following the example of the Python Software Foundation, PyCon Canada, and other community members, we have therefore decided to adopt a code of conduct to make it clear what kinds of language and behavior are and are not acceptable at our events. If you'd like to know more, we couldn't possibly write anything better than this blog post by the Python Software Foundation's Jesse Noller.

Sample Data Management Plans

Fri, 21 Dec 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Neil Chue Hong recently pointed us at DataONE's Data Management Planning page, which has two really useful things:

a link to DMPTool, which will help you build a data management plan for your project; and

five sample data management plans, all written for the NSF, which people can use as reference models.

Data management is something we deliberately left out of "Best Practices" paper—it would have increased the paper's length by half again—but what we should put together is some sample code management plans. Any volunteers?

Minutes from 2012-12-19

Wed, 19 Dec 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Our second all-hands meeting was held on Wednesday, December 19, from 12:00-13:10 Eastern time.

Actions

Dhavide Aruliah

Co-author bullet-points and cheat sheet for Unix shell

Co-author bullet-points and cheat sheet for basic Python

Amy Brown

Proof-read Code of Conduct

Add Code of Conduct to bootcamp "how-to"

Create graphic for "Participant" badge

Create PDF certificates for "Participant", "Instructor", and "Organizer"

Add wording to web site to clarify what does/doesn't qualify for use of "Software Carpentry" name and logo

Add wording to web site asking people to let us know if they're using our material, and requiring them to check with us if they're using our name/logo

Keep track of who's talking to what sites about bootcamps

Get explicit retroactive signoff on contributions

Neil Chue Hong

Contact OSS-Watch regarding enforcing trademarks for our material

Matt Davis

Arrange bootcamp at Cal Poly

Arrange bootcamp at San Diego State

matplotlib material

Ted Hart

Co-author bullet-points and cheat sheet for regular expressions

Arrange bootcamp at ESA conf.

Bernhard Konrad

Co-author bullet-points and cheat sheet for testing

Emily Jane McTavish

Co-author bullet-points and cheat sheet for regular expressions

Create ticket to arrange bootcamp at U. Kansas

Geoff Oxberry

Create ticket to arrange bootcamp at U. Delaware

Cait Pickens

Co-author bullet-points and cheat sheet for Git

Co-author bullet-points and cheat sheet for basic Python

Karthik Ram

Co-author bullet-points and cheat sheet for Git

Joon Ro

matplotlib material

Alex Viana

Co-author bullet-points and cheat sheet for SQL

Ben Waugh

Co-author bullet-points and cheat sheet for testing

Ethan White

Co-author bullet-points and cheat sheet for SQL

Lynne Williams

Co-author bullet-points and cheat sheet for Unix shell

Greg Wilson

Publish Code of Conduct to web site

Retroactively issue "Participant" badges for 2012 bootcamp attendees

Issue "Instructor" and "Organizer" badges

Finish Instructor's Guide by end of Jan 2013

All

Check locally to ensure that it's OK to ask bootcamp participants to agree Code of Conduct

Claim ticket for "Arrange bootcamp at [site]"

Introduce us to one more potential bootcamp site for 2013

Volunteer to help with online office hours

Dhavide Aruliah

Erik Bray

Shreyas Cholia

Matt Davis

Ross Dickson

Steve Haddock

Mike Hansen

Geoff Oxberry

Cait Pickens

Alex Viana

Ben Waugh

Ethan White

Send Greg a picture of yourself in a Software Carpentry t-shirt

Roll Call

You've Shown Me the C, Now Where's the Python?

Sun, 16 Dec 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The W3C's Provenance Working Group recently published a new draft of their proposed standard for tracking provenance on the web. It's pretty dense stuff: even the primer, which uses the word "intuitive" five times to describe itself, is hard to follow if you've never been immersed in Dublin Core, TURTLE, and the like. That isn't a criticism—this stuff is intrinsically hard—but I think most scientists won't be able to see the forest for the trees.

Lorena Barba's Reproducibility PI Manifesto

Sat, 15 Dec 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Prof. Lorena Barba, of Boston University, presented an excellent manifesto at the ICERM meeting on reproducible research:

I will teach my graduate students about reproducibility.

All our research code (and writing) is under version control.

We will always carry out verification and validation.

For main results in a paper, we will share data, plotting script & figure under CC-BY.

We will upload the preprint to arXiv at the time of submission of a paper.

We will release code at the time of submission of a paper.

We will add a "Reproducibility" declaration at the end of each paper.

I will keep an up-to-date web presence.

So say we all.

Three Non-trivial Use Cases for Git

Sat, 15 Dec 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Today's guest post comes from Nicolas Limare.

Here are three problems I encountered while introducing fellow grad students and post-docs to Git. These are situations where Git seems not to provide the solution we need, either because it requires the addition of an external service provider, unusual configuration steps not adapted to beginners, or simply because the functionality is missing in Git. Or maybe did I just ignore the perfect command or option which would have answered all my wishes?

Two R Workshops at UBC in 2013

Thu, 13 Dec 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We're pleased to announce two workshops in 2013 at the University of British Columbia:

Beginners R
Date: January 11, 1 Day
Venue: MSL 101
Format: 4×1.5 hour blocks

R Basics, workspace, working directory, projects in RStudio

Importing, exporting and cleaning data.

Data manipulation, aggregation and indexing.

Working with a statistical model object.

All of the above will include lattice graphics and the usefulness of data frames, and will focus on code not the mouse.

Feedback from Edinburgh

Wed, 12 Dec 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This week saw the first bootcamp to be held at The University of Edinburgh. The bootcamp was organised by the The Software Sustainability Institute and EPCC as part of the PRACE Advanced Training Centre. We had 36 researchers from geosciences, astronomy, biology, statistics, chemistry and mathematics and based at 8 institutions across the UK. Azalee and myself led the workshop and we had 7 helpers including Nancy Giang of the University of Dundee who we recruited after she attended the Newcastle bootcamp back in October, and David Jones of the Climate Code Foundation. We also had as helper/observers Mike Mineter of The University of Edinburgh and Norman Gray of The University of Glasgow also came along to help and to see what a bootcamp is like, with a view to running bootcamps for the physics and geosciences communities.

IPython Funding: Hurray!

Wed, 12 Dec 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Via email:

We are proud to announce that we've received funding from the Alfred P. Sloan Foundation that will support IPython development for the next two years. Thanks to this $1.15M grant, Brian Granger and Fernando Perez will be working roughly 3/4 of their time on IPython, Min Ragan-Kelley will be the lead project engineer, and Paul Ivanov and Thomas Kluyver will work as postdocs on the project. Furthermore, Matthew Brett and JB Poline of Nipy fame will be working part-time on the development of notebooks for applied statistics in collaboration with Jonathan Taylor from Stanford.

Some of the Things We've Learned About Teaching Git

Tue, 11 Dec 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We've tried teaching Git six or seven times now; here's what some of the instructors have learned.

Instructor #1

I spent too much time explaining the "stage." It's an important concept that sets git apart from other VCSes, but for beginning students it's probably too advanced.

I wish I had used a more realistic project with several files and a history of logs. Creating a new project live is useful in teaching how to set up a git repository, but then I'm limited to working with a small history.

I left the GitHub discussion until the end, and because we were tight for time I was not able to really get into it. I wish I had started to talk about GitHub early on, maybe as soon as their first commit.

Instructor #2

I should have left out the part where folks set up their Git SSH keys. It's just not worth the confusion and only helps folks by letting them avoid typing in their passwords. Since more people understand passwords than SSH keys, just skip the SSH key set up part.

Also, I think I could have crammed in more content if the examples early on had been better/more time efficient. I would have liked to cover "git commit -amend," "git reset", "git merge --no-ff", various workflows, the importance of keeping your main branches clean and working in side branches, etc.

I also would have liked to end the whole thing with a more elaborate (though straightforward, ideally) exercise that folks work on by themselves, encounter conflicts, etc. As it stands in my notes, they only really encounter a conflict once.

Talking about local version control all the way through branches was good for me, but, as Instructor #1 suggests, it was helpful to introduce github immediately after branches and as soon as remotes come up.

Instructor #3

Don't pretend it's SVN or anything, just teach git as Git.

Use lots and lots of graphics and demonstrations, especially demonstrations of opposing cases, e.g., here's a pull where everything went great, here's a pull with a conflict, or here's a push that goes smoothly, and here's what happens when your local copy is out of date.

Things Are Going Well in Texas on Ada Lovelace's Birthday

Mon, 10 Dec 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

What To Work On In 2013

Sun, 09 Dec 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Votes are in and tallied: here's what you'd like us to work on in 2013.

Creating a Task List

Sat, 08 Dec 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.
We're going to start using GitHub to manage our content, and as a corollary of that, we're going to use its issue tracker to manage the things we're working on. I've already put a handful of medium-to-large items in it; suggestions would be welcome, particularly if accompanied by, "...and I'll do the work." :-)

Moving Up and Moving Down

Wed, 05 Dec 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Our existing workshops/material are aimed at people who know enough to write a hundred-line script, but don't yet use version control or do any systematic testing. Many scientists either haven't gotten this far, or are well past it, and we'd like to start helping them. We think that means we need:

"Software Carpentry for Complete Beginners", which would only cover basic programming and version control, and

"Advanced Software Carpentry", which would cover parallel computing, integration with legacy code written in C/C++ and Fortran, and so on.

The challenge, as always, is resources: doing this requires people to put together the content and teach half a dozen times to tune it and help show others how to teach it as well. It's a big commitment, but the potential rewards are pretty big too. If you're willing to commit a hundred hours or so to help make the world a better place and give your career a boost, please get in touch.

Our First Hackathon

Wed, 05 Dec 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We'd like to invite our friends to a week-long hackathon at Mozilla's office in Toronto in June to:

create new teaching material

give each other feedback on teaching (we'll run a workshop during the week)

build new tools (like extensions for the IPython Notebook)

get to know each other

We will have a small budget to help with travel and accommodation, but unfortunately most people will have to fund themselves. On the other hand, Toronto's beautiful that time of year, and there are lots of interesting places a visiting scientist could give a talk. If you're interested, please stay tuned for dates (which we should have in January).

Six Years Later

Wed, 05 Dec 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I'm giving a short talk next week at an ICERM workshop on Reproducibility in Computational and Experimental Mathematics, which has prompted me to look back at a talk I gave six years ago at SciPy'06. The slides from that talk are included below; I'm now trying to decide whether I'm pleased or depressed by how little has changed.

Sustainability

Wed, 05 Dec 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Software Carpentry wouldn't exist without support from the Sloan Foundation, Mozilla, and a lot of other supporters, but that support won't last forever. Over the next two years, we need to find a way to make this project self-sustaining; to achieve that, we're currently doing the following:

Use a "host pays expenses" model so that our budget doesn't have to grow as we scale up.
Status: mostly done.

Make contribution easier, so that people who want to help can do so with little or no overhead.
Status: we should finish migrating to GitHub by the end of the year, and using IPython Notebooks for teaching will help too, but the world still (desperately) needs a version control-friendly format for general teaching materials.

Demonstrate career value to instructors, so that people can keep teaching for us even when they're crunched by other commitments.
Status: we're adding testimonials all the time, and hope to have some bigger stories by the middle of 2013.

Assess our impact in ways scientists will find credible.
Status: we're currently exploring ways to do this, and would like to hear from anyone who'd like to help.

Lobby funding agencies and journals to ask hard questions about software (or about software development processes). In the long term, we need to convert twenty percent of scientists to our way of thinking, but we can accelerate that by focusing on the right twenty percent.
Status: we haven't started this yet.

Get scientists to include training in budgets and schedules.
Status: this is the longest shot of all, because it depends on funding agencies being amenable to the idea. In practice, it may have to wait until we reach our 20% target...

If you can think of other things we can do to ensure Software Carpentry's long-term viability, we'd like to hear from you.

Who Can Run a Software Carpentry Workshop?

Wed, 05 Dec 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A couple of people have mailed us in the past week to ask if they can use our materials and run a workshop on their own. The short answer is "yes"; the longer answer is "yes please" :-) All of our lessons are covered by the Creative Commons - Attribution 3.0 Unported licensed (often abbreviated "CC-BY"), which in brief means that you can share or remix stuff however you want, as long as you cite us as the original source (e.g., by providing a link back to this site).

Why Be an Instructor

Wed, 05 Dec 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Our instructors are all volunteers—bootcamp hosts cover their travel and accommodation costs, but they're not paid for their time. So why do they do it?

Make the world a better place. As I say in a lot of my talks, the two things we need to get through the next hundred years are more science and more courage. I don't know if we can do much about the latter, but we can sure help a lot with the former.

Make their own lives better. Most of the time, we try to have astronomers teach astronomers, ecologists teach ecologists, and so on. These are people whose tools instructors might one day want to use themselves, so by making them more clueful, instructors are helping themselves.

It's fun. How could it not be? You get to stand up and look smart in front of a bunch of smart people who actually want to be there, and you don't have to mark anything afterward.

Build a reputation. Showing up to run a really useful workshop is a great way for people to introduce themselves to places they'd eventually like to work, and a great way to make contact with potential collaborators. This is probably the most important reason from Software Carpentry's point of view, since it's what makes our model sustainable.

Get practice teaching. We're doing more every year to train instructors, and giving them chances to teach online as well—both of which are useful for people with academic careers in mind.

Help get people of diverse backgrounds into the pipeline. Computer Science is 12-15% female, and that figure has been dropping since the 1980s. From what I've seen, there's a similar gender skew among computationally-oriented people in other sciences, which if left alone will be self-perpetuating. Some of our instructors are involved in part because they want to break that cycle and be a public example of a competent, confident woman programmer.

Teaching forces you to learn new things, or learn old things in more detail than you already know. See for example "Graduate Students' Teaching Experiences Improve Their Methodological Research Skills"

The more you know, the less you have to write yourself. Putting a grant application together? Have a site review coming up? We probably have slides for that... :-)

And why do people do online office hours? For some, the primary incentive is a little bit of guilt: we can't give everyone all the help they need during a two-day workshop, and online follow-up is a way to make up for that. For others, it's a chance to pick up and practice some skills that are increasingly in demand: everyone believes that a lot of teaching is going to move online in the coming years, and this is a way to figure out how to do it.

See You at PyCon 2013

Tue, 04 Dec 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The list of talks for PyCon 2013 has been posted, and we're very pleased to see three by members of our team:

Titus Brown: Awesome Big Data Algorithms

Matt Davis: Teaching with the IPython Notebook

Jessica McKellar: How the Internet Works

There will also be the first-ever Python Education Summit and a whole lot more—hope to see you there.

European Grid Infrastructure is Organizing a Software Carpentry Workshop

Sat, 01 Dec 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The European Grid Infrastructure project is putting together a Software Carpentry workshop for Manchester in April-early details are in their post, and we'll post more details ourselves as they evolve.

Good News About Software Carpentry (and More)

Fri, 30 Nov 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We're pleased to announce that Mozilla and Software Carpentry will be continuing and expanding our work to empower scientists with computer science and webmaking skills. We have been working with the Sloan Foundation to teach scientists the concepts, tools and techniques they need to use computers and the web to accelerate their research. In 2013-14, the program will offer an expanded set of workshops, online tutorials, and a new peer-to-peer mentoring program.

Alpha Testing Ideas for the IPython Notebook

Tue, 27 Nov 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Before I post any of these ideas on the IPython Notebook wiki, I'd be grateful for feedback from our learners and instructors. Most of these (maybe all of them?) would be done as extensions, rather than as patches to IPyNB itself. Please let us know what you think; I'll revise and then cross-post to the IPyNB folks.

Our Users

Anna teaches scientists how to program. She usually does this by coding live in front of a small class (20-40 people), but also wants to create recordings for people to view outside class.

Titus Brown on the Scripps Institute Bootcamp

Sun, 25 Nov 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Hot on the heels of Cait Pickens' posts about our workshop at the Scripps Institute comes this post-mortem from Titus Brown. Again, there's lots of good analysis, and many useful recommendations—particularly concerning installation and setup headaches (which continue to be our second-biggest problem).

Cait Pickens on the Scripps Institute Bootcamp

Sat, 24 Nov 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Cait Pickens (a grad student from Michigan State University) has written a great series of posts about the bootcamp she helped teach at the Scripps Institute a couple of weeks ago:

Preparation

Day 1, Part 1

Day 1, Part 2

Day 2, Part 1

Day 2, Part 2

Day 2, Part 3

She also has a post describing how they used HipChat in the classroom, which is almost what I think we need to do peer instruction online. There's lots of good stuff to think about here—recommended.

The Tool (I Think) We Need To Do Peer Instruction Online

Mon, 19 Nov 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Clay Shirky's recent essays "Napster, Udacity, and the Academy" has attracted a fair bit of attention. I've written about some of the things it doesn't discuss on my personal blog, but here, I'd like to use it as a jumping off point for a description of a tool I'd like someone to write for us. Given that my last request produced working code in just a few hours, I'm hoping one of you will wow me again :-)

Who Wants To Build a Faded Example Tool for the IPython Notebook?

Mon, 19 Nov 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

While I'm asking people to write code for me, here's a small addition to the IPython Notebook that someone should be able to knock off in an hour. The starting points are cognitive load theory and the idea of faded examples: long story short, one good way to teach is to show learners one example in detail, then ask them to fill in the ever-larger blanks in a series of similar partly-worked-out examples. Here's one that Ethan White did for our online study group:

Updating Our Reading List

Sat, 17 Nov 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We're planning to launch an update to this web site in the next few days, and as part of that, we're revisiting some of our content. For example, we'd like to shorten and update our recommended reading list—the current short list is below, and we'd welcome suggestions for additions. However, if you'd like to put something in, please suggest something to take out (so that the list doesn't become a Sunday stew).

Making a Difference at LBL

Fri, 16 Nov 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Without any doubt, the best thing about teaching for Software Carpentry is making a difference for someone and getting to hear about it. One of our students at the Lawrence Berkeley Lab, nuclear engineer Dr. Bethany Goldblum, was impressed enough by Katy Huff's presentation of git that she wanted her team to start using version control.

Bethany's description of her team's past code management practices is typical of academic projects, with many people modifying their own copy of a code until that "resulted in over 20 versions of the codes, each with bugs of its own." On their current project "a postdoc is in the process of developing the code, but now every time he makes a change, he uploads it to a different directory on our analysis machine and we just have multiple copies of almost the same code floating around. Unless one is careful, they may not know which code is the most recent version. I believe there is some fear there to undo anything that the original developer put in place."

Matt Davis's Great Californian Adventure

Fri, 16 Nov 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Matt Davis (who works at the Space Telescope Science Institute, and has become one of our regular instructors) taught three bootcamps in eight days back in October. He has finally recovered enough to write a description of what he did, how it went, and what he learned. The two biggest lessons are:

Try to get to know your audience beforehand by talking to organizers and/or using surveys.

Try to plan bootcamps so that the students come from similar backgrounds and, if possible, the same department or lab.

We Apologize for the Interruption in Our Service

Fri, 16 Nov 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Our hosting service, Dreamhost, is relocating various domains to new servers. As a result, service at this site is suffering from longer-than-usual lag (90 seconds or more to load the home page, worse for pages with embedded videos). We've filed a problem report, and will try to get things back on track as quickly as possible. We will also start looking for a new hosting plan: slow-and-slower has become all too common here :-(

Who Wants To Write a Little Code?

Fri, 16 Nov 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We have always steered away from building libraries to use in teaching—we want to show people the "real" stuff, and we can't afford to maintain things. The IPython Notebook has us rethinking that: now that we can display images inline, it would be wonderful if we could leverage Mark Guzdial and Barbara Ericson's work, and teach basic Python using simple image manipulation for examples (rather than text munging). Their research found that using images led to higher retention: more students stuck around, and the ones who did, remembered and used more of what they'd learned.

This Is What We Do

Thu, 15 Nov 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Day 1 of the workshop at the Scripps Institute: a room full of biologists learning how to do better science faster by building things the right way.

A Mostly Successful Decade

Wed, 14 Nov 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A video of Fernando Perez's keynote at PyCon Canada, titled "Science And Python: retrospective of a (mostly) successful decade", has been posted. If you'd like to see what the IPython Notebook can do, and how it fits into a broader picture, it's well worth watching.
{% assign video_title = "Science And Python: retrospective of a (mostly) successful decade" %} {% assign video_slug = "F4rFuIb1Ie4" %} {% assign video_time = "1:00:58" %} {% include youtube %}

FOSDEM 2013

Wed, 14 Nov 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Via Sylwester Arabas (one of the organizers):

A day-long session ("devroom") on Free/Libre and Open Source Software (FLOSS) for scientists will be held during the next FOSDEM conference, Brussels, 2-3 February 2013 (http://fosdem.org/2013). We aim at having a dozen or two short talks introducing projects, advertising brand new features of established tools, discussing issues relevant to the development of software for scientific computing, and touching on the interdependence of FLOSS and open science.

Workshop for High-Energy Physics at UCL

Wed, 14 Nov 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Following the success of the open bootcamp at UCL in April/May this year, we decided to have a go at introducing some Software Carpentry to the first-year postgraduate students in my own research group, High-Energy Physics (HEP). There is already a longstanding University of London intercollegiate lecture programme for these students, including some programming in C++ and Python. The new workshops aim to help participants consolidate and extend their existing Python knowledge, as well as learning about more general best practice in software development, by following an extended worked example using version control and test-driven development.

Pre-Assessment

Tue, 13 Nov 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

One of the recurring problems in our bootcamps is that at any point, about 1/4 of people are lost, and 1/4 are bored. The only fix we can think of is to let people self-assess before arrival to determine whether this is the right thing for them or not (and to let instructors know more about who they're going to be teaching). We've used variations on the questionnaire below a couple of times with useful results; we'd appreciate feedback on:

Web 4 Science

Tue, 13 Nov 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Titus Brown (a long-time supporter of, and contributor to, this project) has a series of posts on his blog about using the web for science. As he's well ahead of the curve in doing this himself, I thought readers of this blog would be interested:

opening remarks

the awesomeness we're experiencing

the challenges ahead

strategizing for the future

tech wanted

There's lots here to chew on—I'm sure he'd welcome comments from our readers.

More Oxford feedback

Wed, 07 Nov 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Following on Phil's analysis of the feedback from the first Software Carpentry workshop in Oxford, here are the round-the-room comments from the attendees, helpers and instructors, good and bad...

Good

Learned something.

Mercurial, very useful yet simple. Can apply straight away.

Practical at end seemed useful way of tieing everything together.

Doing things in synch with lecturers.

Scientific approach to programming.

Whole approach.

Good overview of the key concepts and foundation to learn more.

Focussed.

Course as a whole is good.

Thorough - basic to advanced - and see new ways to do basics.

Good practice, can take forward.

Best practice.

Really good overview and intro to things not heard of before but would be very useful (version control, MDAnalysis).

Make and use for things other than C.

Other instructors chipping in on current instructor was good.

Loved it, especially version control and make.

Bad

Keeping up with the instructors typing on the projectors, so slow the pace.

Resources and case-studies from the simple examples to complex examples.

Would like more of a chance to apply what was learned with the instructors around, or at least feedback on a plan.

Most sessions ran over.

Problems with later aspects of Python.

Balance between beginner and advanced - always wasting someones time.

Knowing simpler Python commands might've helped beforehand.

Sometimes a bit too fast.

More time for exercises.

Target the course a bit more to their area.

Helps to start with, but good to have more courses and more on each area.

More time to work on problems and work through case-studies.

Downloading all the files for every exercise.

Days away from work.

More Tips

Tue, 06 Nov 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Philip Fowler, who recently hosted a bootcamp at Oxford, has written a 5-point guide for people who are thinking about doing it themselves—we hope you'll find it useful.

An Administrative Note

Mon, 05 Nov 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

In order to maintain focus, I'm moving discussion of general education & technology issues over to my personal blog. I will continue posting items specifically related to teaching computational competence to scientists here.

Winter School on Reproducible Research

Sun, 04 Nov 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

It is a pleasure to announce that the 13th Geilo Winter School will take place January 20th-25th, 2013 at Dr. Holms Hotel, Geilo, Norway. As you can see from the attached flyer, the Winter School will give an introduction to reproducible science and modern techniques for scientific software development, a topic we hope many of you find interesting. The aim of the winter school is that participants will be able to apply the learned techniques to make their own research reproducible, and topics that will be covered include reproducible research, verification and validation, software testing, and continuous integration. More information is available on the winter school webpages, and participants are also encouraged to present their own research in a poster.

How to Help at a Bootcamp

Sat, 03 Nov 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Aleksandra Pawlik has written a five-point guide to helping out with Software Carpentry bootcamps:

Familiarity is not enough.

You're not just there to troubleshoot, you're there to troubleshoot fast.

Expect the unexpected.

Be proactive.

IT all-rounder.

This, and Katy Huff's "How to Help at a Bootcamp", are tremendously valuable: if you have tips of your own to add, please send 'em in.

Charging and Being Charged

Thu, 01 Nov 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As noted in the minutes from our meeting on Monday, one of the many things we want to improve about our workshops is the attendance rate. While we get near-complete turnout for some, we have 50% no-shows for others, which is disappointing for instructors, and unfair to people left on the waiting list.

One idea is to charge $20 or so to help people decide if they're serious about taking part or not. However, we've been told in the past that as soon as we start charging, some universities will start to charge us for space, network access, and so on. I agree with their reasons—it would be chaos if random people could use their facilities for profit and not give any back to the university—but in some cases, what they'd charge us would far exceed what we'd get from learners.

Oxford Wrap-Up (with charts!)

Thu, 01 Nov 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The first Software Carpentry workshop in Oxford has wrapped up, and by all accounts was a big success. Philip Fowler, the host, has posted about running the workshop ("everyone sat up a bit straighter during version control...") and also done an analysis of what participants think they learned. In addition, Aron Ahmadia has given us some detailed feedback on how we could improve things the next time 'round. Our thanks to everyone involved—we're looking forward to a return visit soon.

Pelican Guts: on content management for Software Carpentry

Thu, 01 Nov 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This morning I spent some time talking over the content management problem for the Software Carpentry website with Greg. To recap: we'd like to make it easier for anyone to contribute and manage the website content (lecture pages, new bootcamps, events, etc). For example, there are plenty of bits of site building that could be automated so that contributing a new lecture or topic doesn'thave to involve wiring in links to a table of contents, forward and backward links to other lectures/topics, and so on.

A List of Bioinformatics Courses

Tue, 30 Oct 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Titus Brown is compiling a list of bioinformatics courses. If you know one that should be added, he'd enjoy hearing from you.

Minutes from 2012-10-29 All-Hands Meeting

Tue, 30 Oct 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We held our first online all-hands meeting yesterday (Monday, October 29), and despite Hurricane Sandy, 28 people were able to attend. Minutes from the meeting are given below; we have a lot to do, but we're very excited to be doing it.
In attendance:

Aron Ahmadia

Carlos Anderson

Azalee Bostroem

Erik Bray

Steve Crouch

Matt Davis

Ross Dickson

Justin Ely

Julia Gustavsen

Tommy Guy

Steve Haddock

Ted Hart

Konrad Hinsen

Katy Huff

Emily Jane McTavish

Trevor King

Justin Kitzes

Ian Langmore

Ian Mitchell

Aleksandra Pawlik

Ariel Rokem

Anthony Scopatz

Chang She

Joshua Smith

Laura Tremblay-Boyer

Ben Waugh

Ethan White

Greg Wilson

Minutes

Attendance and no-shows

Attendance ranges from less than 50% to over 100% of those who register

Former case is disappointing (to instructors) and unfair (to those left on the wait list)

Options:

Have people to register in groups/teams

Worked well at UC London and elsewhere

Difficult to implement on EventBrite

Try out team signup at upcoming workshops

Try a registration charge (e.g., $20) which we keep, return after attendance, or give to charity

Institutions may start charging us for space if any money changes hands

Find out if we can charge for attendance without being charged in turn

Returning money to attendees is difficult (getting credit card number and only charging for no-show is equally fraught)

What charity would we donate to? (Software Carpentry isn't a non-profit)

Licensing

Existing content is CC-BY / MIT licensed

Do not want to have to manage content with multiple licenses

Need sign-off from contributors

Linux Submitting Patches guide and GitHub's Contributing Guidelines are both examples we can copy.

Incorporate licensing agreement into web site

Get retroactive signoff for existing material

Workshop/participant level mismatch

Most common complaints about workshops are "too fast" and "too slow"

describes our target

Telling people what we expect as background hasn't worked

People who don't know enough to absorb what we teach show up in the hopes that they'll get something, and are then lost

People who know much of what we're teaching show up in the hopes of learning something new, and are bored

Having people send us material for evaluation (e.g., code sample) doesn't scale

"I think the people in my field able to produce a code example are way ahead of the group that needs the most help."

Use a self-assessment?

People will still undershoot/overshoot and show up anyway

Pre-tests scare away some of the people we most want to help

Ask them to work through a free online course before coming to us?

MOOCs work best for people who already understand concepts and want to add information...

...which isn't our audience...

...and many people who are willing to try a two-day workshop probably won't commit to a full course (up front)

Experiment with pre-assessments for upcoming workshops

Make completion of the pre-assessment a condition for getting a ticket

Impact Assessment

We don't know what impact we're having (but funders would really like to)

Collect names/email addresses of actual workshop participants to contact later

Design lightweight post-workshop instrument for use 3 months after

Migrating the Software Carpentry repository to GitHub

Find examples of people doing collaborative course development using Git

Need to reach consensus on one-big-repo vs. lots-of-little-repos

Produce short A-or-B position statements on Git organization by Nov 7 for vote

Online Lessons

Broadcast video tutorials didn't work well

Forums hosted at Software Carpentry didn't do well in 2010

SciComp Stack Exchange isn't intended for novice questions about shell, version control, etc.

Local learning groups (once-a-week lunchtime sessions) seemed to work well

And when video tutorials did work, it was partly because participants were co-located, e.g., watching a single screen in a small group and talking amongst themselves out of band

Experiment with online office hours

Create "Software Carpentry" tag on Stack Exchange

Re-launch our own forums

Given how many workshops we're running, and how closely they're scheduled, maybe this time there would be critical mass

Targeting Specific Audiences

Our mission is not to teach Python (or Bash, or Subversion), but to teach scientists how to think like programmers:

to grow programs in a structured, repeatable way (create and combine lots of little tools)

to manage and share what they build (version control, readable code, provenance)

to be confident that it's right (defensive programming, testing, debugging)

to automate, automate, automate (build system, and the very idea of programming)

and to do this using open source tools as far as possible.

Using R instead of Python fits this mission (see Justin Kitzes' summary)

Teaching MPI doesn't

What about teaching these concepts in C or Fortran to people who already speak the language?

Try using R instead of Python

Try using C or Fortran without teaching the language itself

Workshop Sprint in 2013

Initial idea: bring Europeans over to help run a bunch of workshops in North American in late March 2013, then send North Americans over to help teach another bunch in late June

Good for publicity

A good way for people to meet each other and build ties

Are (some) people able to take 2-3 weeks to do this? Yes.

Particularly if they give academic talks at the same time as teaching workshops, which everyone should.

Does the timing work? Exams, people leaving to work at field stations, etc.

Find out who's available when for workshop sprint

Find budget for workshop sprint

Content Sprint in 2013

Independently of the workshop sprint, bring people together for a one-week sprint on content

Or possibly 2-3 sites (e.g., London, Toronto, Mountain View) connected by virtual presence

Find out who's available when for content sprint

Find budget for content sprint

Deferred

IPython Notebooks experience report

Git experience report

What to do about Windows?

Badging

Actions

Try out team signup at upcoming workshops. [pending new workshops]

Find out if we can charge for attendance without being charged in turn. [all]

Incorporate licensing agreement into web site. [GVW]

Get retroactive signoff for existing material. [GVW]

Experiment with pre-assessments for upcoming workshops. [pending design and new workshops]

Collect names/email addresses of actual workshop participants to contact later. [all]

Design lightweight post-workshop instrument for use 3 months after. [WC, MH]

Find examples of people doing collaborative course development using Git. [any?]

Produce short A-or-B position statements on Git organization by Nov 7 for vote. [KH + AA, MD, TG, TK, CS, JS]

Experiment with online office hours. [KH, others?]

Create "Software Carpentry" tag on Stack Exchange. [volunteer?]

Re-launch our own forums. [deferred]

Try using R instead of Python. [TH, JK, LTB]

Try using C or Fortran without teaching the language itself. [AA, KH]

Find out who's available when for workshop sprint. [GVW]

Find budget for workshop sprint. [GVW]

Find out who's available when for content sprint. [GVW]

Find budget for content sprint. [GVW]

Note: GVW would really appreciate a volunteer to tackle #15-18.

AA Aron Ahmadia

WC Warren Code

MD Matt Davis

TG Tommy Guy

MH Mike Hansen

TH Ted Hart

KH Konrad Hinsen

KH Katy Huff

JK Justin Kitzes

TK Trevor King

CS Chang She

JS Joshua Smith

LTB Laura Tremblay-Boyer

GVW Greg Wilson

Position Available: Director, Webmaking Science Lab, Mozilla

Mon, 29 Oct 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Mozilla is seeking a dynamic individual to drive the vision and product strategy for its Webmaking Science Lab. This is a key strategic role reporting to the Executive Director. This individual will lead the charge in reaching out to the scientific community, helping to establish a vibrant community of scientists, developers, designers, and other individuals working to advance the practice of science on the web. A passion for open science, a keen understanding of the web as both a technology stack and culture, experience with existing web-based science toolsets, and enthusiasm for community-driven technology development are the initial key qualifications. As a senior member of the Mozilla organization, the successful candidate will be responsible for gathering partners, inspiring a team, building a community, shaping a vision, and leading the development and delivery of new products for conducting science both on and like the web.

Usability Testing and Instructional Design

Sun, 28 Oct 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This is a story in several parts.

1. From Guido van Rossum's intermittently-updated blog about the history and design of Python:

Python's first and foremost influence was ABC, a language designed in the early 1980s [that] was meant to be a teaching language, a replacement for BASIC, and a language and environment for personal computing. It was designed by first doing a task analysis of...programming...and then doing several iterations that included serious user testing... ABC's authors did invent the use of the colon that separates the lead-in clause from the indented block. After early user testing without the colon, it was discovered that the meaning of the indentation was unclear to beginners being taught the first steps of programming. The addition of the colon clarified it significantly: the colon somehow draws attention to what follows and ties the phrases before and after it together in just the right way.

This story and others like it were something of a revelation to me when I first encountered Python in the late 1990s. Usability testing of programming languages? Huh—why isn't everyone doing that?

Why This Is Hard (Part Deux)

Sat, 27 Oct 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I pointed a grad student at the IPython tutorial and made some notes as she started to work through it. (She has used the vanilla Python interpreter before, but is still a novice.) Here's what happened.

The first thing the tutorial mentions is tab completion, which she's used in other tools, so she tried that:

>>> x = 'hello' >>> x.<TAB> # shows methods of the string --- cool!

Does it work directly with strings?

Counting to Five (or, A Plan for Online Tutorials and What's Wrong With It)

Fri, 26 Oct 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

My daughter is five and a half years old, and some time in the last couple of weeks, she made a small cognitive breakthrough. It used to be that if I asked her, "What's five plus three?", she would carefully count five fingers, then count three, then go back and count them all together to get eight. She doesn't do that any more; instead, she holds up five fingers, then counts, "Six... seven... eight!" to get the answer. It may sound like a small thing, but it's not: the idea that five is always five, even if you didn't count it this time around, is such a big one that we forget it ever had to be learned.

Mozilla Web Literacies White Paper

Fri, 26 Oct 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Mozilla Foundation (which has been our home for the last eight months) have posted a white paper on web literacies that outlines their vision for the web:

...the world's largest public resource, the operating system of the future, and (we believe) one of the greatest drivers of happiness and human flourishing the world has ever seen.

The skills/competencies/literacies grid gives a condensed overview of how beginners and intermediates could explore, create, connect, and protect with and on the web. Over the coming months, we'll be working to find ways of connecting this with scientists. If you'd like to help, please get in touch.

Two Self-Assessments

Fri, 26 Oct 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A recurring problem with our workshops is the diversity of learners' backgrounds. Below are the results of two pre-workshop self-assessments (on different groups), one phrased in terms of content, the other in terms of specific tasks. The consistency is better than I expected: broadly speaking, it seems that people's impression of how much they know about X is more or less in line with their belief about whether they could do something that requires understanding X. The next step will be to follow up a few weeks after the workshop to see if they actually can do X.

Prime Numbers, Biologists, and Data Visualization

Thu, 25 Oct 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A couple of days ago, Titus Brown posted a question from Randy Olson: are there relevant, non-abstract problems (for biologists) that you can solve with minimal programming skills? I.e., what's the biology equivalent to traditional intro CS problems like "calculate the primes".

The answer has three parts:

Calculating primes, finding the longest line in a file, etc., are lousy introductory problems because they're completely valueless to most of the intended audience. You have to already believe you really, really want to learn how to program, and be good at delayed gratification, to get through them to the good stuff. (This isn't just true in computing: see the discussion in How Learning Works of the impact of motivation on learning.)

Guzdial, Ericson, et al's work at Georgia Tech showed pretty conclusively that a media-first approach to computing has better outcomes, i.e., if people are manipulating images (and audio and video) right from day 1, they'll retain more of what they learn, and more students will stay in the program. (But note: you probably can't use an industrial-strength image manipulation library like PIL for intro teaching, at least not directly: you need something that assumes less background knowledge both in domain and skills.)

Robbins, Senseman, and Pate have been piloting a "visualization first" programming course for biologists (see this paper if you can—apologies for the paywall). While it's early days, it seems that this should have the benefits of "media first" for scientists. For example, a programming exercise could be "draw the curves in red if the end point is less than the starting point, and in blue otherwise".

We haven't reorganized our intro material around this idea yet, partly because of inertia, but partly because of the installation headaches of getting visualization working on N platforms in a two-day workshop. I'm very keen to try it out, though, particularly if the IPython Notebook really does make simple visualization simple to do on all major platforms.

Feedback from Newcastle

Wed, 24 Oct 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This week the Digital Institute at Newcastle University with The Software Sustainability Institute ran a second bootcamp at Newcastle University. This bootcamp was run for attendees from the North East of England, from whom there was demand after the first Newcastle bootcamp, back in May.

Feedback from the attendees was good though some wanted more time or detail on specific subjects. The most popular comment was the desire to have the slides and instructor scripts available so that attendees could catch up if they fell behind the instructor. The other major challenge, again, was Cygwin. We may want to provide a VM for attendees who typically use Linux but have a Windows laptop.

25 Questions

Tue, 23 Oct 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We've tried several times to define Software Carpentry's aims and content in terms of the questions that researchers ask (see for example our competence matrix and this post mapping eight questions to our current content). Our latest attempt is crowd-sourced with a small 'c': the people who are taking part in the first round of our online study group for instructors and would-be instructors put together their own lists of questions, along with the answers they'd expect from novice, intermediate, and expert scientific programmers. I've consolidated those lists into 25 questions, which I'd like to boil back down to no more than 10. If you'd like to take a crack at doing that, please post your suggestion as a comment, and I'll send you a Software Carpentry t-shirt if we use it.

Key Points

Tue, 23 Oct 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

On the flight back from Vancouver yesterday, I finally did what I should have done eight months ago and compiled the key points from our core lesson content. The results are presented below, broken down by lesson and topic; going forward, we're going to use something like this as a basis for defining what Software Carpentry is, and what workshop attendees can expect to learn.

Twenty Percent

Tue, 23 Oct 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I realized a couple of days ago that I'd never blogged about what Software Carpentry needs to accomplish in order to change the practice of science fundamentally and permanently. In a nutshell, we need to convert a fifth of scientists to our way of thinking. Once we do that, the odds are better than 50-50 that every time someone sends a paper out for review, at least one reviewer will ask hard questions about how the computational work was done. I get that number by assuming:

Excel Isn't Intrinsically Evil

Mon, 22 Oct 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Excel and other spreadsheets aren't intrinsically evil, but like any power tool, they can easily take off fingers when used carelessly. From Neil Saunders' blog:

...a text download of 450K methylation from the Cancer Genome Atlas project reveals that Excel has had its evil way with the data at some point. Gene names such as MAR1, DEC1, OCT4 and SEPT9 are now reformatted as dates.

As he goes on to say, "Despair at the quality of public data, fears about reproducibility in science. Must be Monday."

Feedback from UC Berkeley

Mon, 22 Oct 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We just finished up our UC Berkeley bootcamp with me, Katy Huff, and Justin Kitzes teaching. Here's the feedback:

Good Bad

first day good with no programming experience couldn't run the VM on underpowered computer

good general intro to basics of Python and VC don't see why Python is better than R

liked the intro/advanced Python split would like a longer bootcamp with more focus on Python

good git introduction didn't want to learn Python

IPython notebook helped me learn would like to give anonymous feedback

got to see real software engineering didn't catch some important details that held things up

good coverage of Python + software engineering have better setup instructions, test scripts

wanted to learn Python Python instruction was too slow/basic

liked simple examples got left behind sometimes

IPython/shell interaction awesome not sure how to integrate git into workflow, have practical demos, case studies

good job squeezing a lot of content in hard to type/listen/watch all at once

introduced scientific Python would be nice to have a data cleaning example, workflow example

liked the showcase demos of matplotlib/IPython features could have sped up introductory topics

combination of topics was good have more content summaries, workflow visualizations

liked intro Python exercises reading suggestions to follow up bootcamp

understand the basics of version control wanted to see object oriented programming/calling C code

Unfortunately I missed a pre-workshop memo that most of the attendees would be experienced R users and I could have sped up/skipped some of the introductory programming material in favor of more scientific Python. The second day material on version control and software engineering still seemed to hit home, though, so we didn't totally waste their time. In future workshops I'll try to do a better job getting attendee profiles beforehand.

Getting Credit

Mon, 22 Oct 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A recurring theme in our discussion with scientists is how hard it is to get academic credit for building software, but there are some hopeful signs. As Carole Goble pointed out to me a couple of weeks ago, Nucleic Acids Researchruns two special issues every year: one devoted to web servers, and one to databases. As the introduction to the former makes clear, the journal's definition of "web servers" is pretty broad:

Why Teaching People to Program Is Hard

Sun, 21 Oct 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Update: it's clear from comments that I explained myself poorly in this post. We don't ever teach by starting with a big example like the one below—we start with basic arithmetic, then assignment, then lists and loops, and so on. (See our Python lectures for details.) What I was trying to show was that by the time we reach a realistic example, students have to interleave those concepts in a very fine-grained way, i.e., if they'd been taking notes in the order in which we taught things, they'd have to flip back and forth through those notes constantly in order to make sense of things. That "cognitive assembly" is an extra burden on novices.

Feedback from Lawrence Berkeley National Lab

Sat, 20 Oct 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Katy Huff, Justin Kitzes, and I wrapped up our LBL workshop yesterday. We had ~25 participants with a broad range of backgrounds and levels of experience. Here's our traditional table of good/bad feedback:

Good Bad

can use python better don't know how to apply git to work

covered lots of material covered too much material

can use git better now group felt too large for number of helpers

covered from basics to advanced programming felt unprepared for git instruction

helpers were knowledgeable and nice couldn't run VM, add system reqs

motivated version control well python level was too easy

started with basics, had good help need well defined goals for exercises

got exposed to a lot group people by experience

all the material is available on GitHub would have liked a more detailed description of course

learned enough to get started didn't get any emails on the waitlist

have hope getting info ahead of time

good overview for later whole thing too fast

format - work through code live topics jumped around

testing cases maybe too language (Python) dependent, no objects

self-consistent complete package burnt out at end of first day (fast)

inspirational 9-4:30 draining, more breaks between topics

great intro to vocabulary, how programmers think didn't expect to program at end, was right

gained confidence from seeing experts switch morning git with afternoon python on 2nd day

seeing github, advice on resources not clear that mac users needed to install software ahead, why certain steps

had to do exercises, version control in beginner class some hard to follow along, relies on previous success with commands

instructors :-D, with real-world experience more relevant examples to audience, day to day data/problems

demo writing, troubleshooting, testing very valuable what was xcode for?

falling behind in python - 2 tiered exercises

documentation needs comments

is there help for release management?

advanced math?

third day - integrative exercise, case study - start to finish

follow up project, final suggested homework assignment

The good news is that they liked our material and they liked us (and seemed to think we knew what we were talking about). The bad news is a lot of them had a hard time keeping up and a couple complained it was too slow. These are typical complaints of our shotgun approach to teaching to a random sampling of scientists. We'd like to do more discipline specific bootcamps where we can tailor things better. Another typical complaint was they they aren't sure how to apply what they've learned to their research. We're still working on that one.

I Screwed Up (or, Why Automation Isn't Always a Good Thing)

Wed, 17 Oct 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A couple of months ago, a group of astronomers asked us to run a workshop at Caltech in conjunction with their annual get-together: they'd cover travel costs for two instructors, and seats at the workshop would be reserved for them. Which would have been fine, except when I pushed the button to create the workshop, I also created an EventBrite signup page and a Software Carpentry bootcamp page with open enrollment. As a result, I had to send email to 15 people last night to say that there isn't actually room.

Rebuilding Redux

Fri, 12 Oct 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The time has come to replace our creaking combination of web tools with something that will let us do more for more people while spending less time switching between browser tabs. To recap an earlier post, we currently use:

WordPress for our blog, web site pages, and comments;

YouTube to host our videos (links to which are embedded in the WordPress pages);

Subversion (administered via the Dreamhost control panel) to manage our slides, code examples, and other files;

Mailman (administered the same way) to manage mailing lists;

a Google Map to show the locations of past and future bootcamps;

a Google Calendar to show their dates;

Google Analytics to track web site traffic;

EventBrite to handle event signup; and

OpenBadges to keep track of who's accomplished what (currently in beta).

We also use Vidyo for video conferencing (primarily tutorials), and run Etherpad beside it so that people have a text channel for asking questions, posting links and code fragments, taking minutes, and so on. As I pointed out in that aforementioned post, this means admins need at least six different logins to get things done, and need to copy a lot of information from place to place. For example, whenever we commit to a new bootcamp, someone needs to:

Why We Teach Version Control

Fri, 12 Oct 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Dark Matter, Public Health, and Scientific Computing

Wed, 10 Oct 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This is the text of a talk given at the 8th IEEE International Conference on eScience, October 10, 2012. The slides are also available.

Back in March, Scott Hanselman wrote a blog post titled Dark Matter Developers: The Unseen 99% that crystallized something I'd been thinking about for a while. In it, he said:

[We] hypothesize that there is another kind of developer than the ones we meet all the time. We call them Dark Matter Developers. They don't read a lot of blogs, they never write blogs, they don't go to user groups, they don't tweet or facebook, and you don't often see them at large conferences... [A]s one of the loud-online-pushing-things-forward 1%, I might think I need to find these Dark Matter Developers and explain to them how they need to get online! Join the community! Get a blog, start changing stuff, mix it up! But...those dark matter 99% have a lot to teach us about GETTING STUFF DONE... They aren't chasing the latest beta or pushing any limits, they are just producing.

Purdue

Wed, 10 Oct 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Here's the feedback we received for our workshop in Purdue this week. We discussed the shell and Python on day 1, and then git and debugging/testing/documentation on day 2.

Good

Interactivity (technology and people)

Day 2 topics (version control, testing and documentation)

Hints about what's out there

Overall breadth

Hands-on

VirtualBox and using a Virtual Machine

Topics — git and shell

Online materials, breadth

Testing

Python

Good depth, overview

Documenting the bootcamp online

Interesting content

Good for beginners

Examples on day 1 (shell, Python)

Organization

Tips about what not to do

Bad

Too basic for some

Jump up in difficulty on day 2

VirtualBox problems

Wanted Windows-specific examples

Wanted C++ examples

Wanted advantages/disadvantages of Python data structures

Python subtleties not covered for beginners (when to use a colon)

Too fast to follow sometimes (need to pause before executing commands)

Wanted specific skills (i.e., Python)

Wanted to know more about Valgrind

Version control (too in depth — branches)

git command-line was overwhelming (Github was good)

More exercises, homework?

Detailed schedule (so we can cherry-pick topics)

Too fast sometimes

Slides

No printed documentation for notes

What about SVN users?

Convergent Evolution

Fri, 05 Oct 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Earlier this week, a Google search turned up a Software Carpentry workshop at the ICHEC in Dublin—which was surprising, because we didn't know they were running one. It turns out they'd reinvented the term independently, though their "carpentry" was a lot more advanced than ours—according to ICHEC's Simon Wong:

Half of the workshop was on software development concept and techniques but specifically aimed at deployment on the cloud, e.g. test-driven development, code management, SQL vs NoSQL databases. The rest of the workshop was more getting the attendees familiarised with various Amazon Web Services (e.g. Elastic MapReduce)

UCL Researchers to Get Help with Software Development

Fri, 05 Oct 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

University College London (UCL) has hired a team leader to start a three-person team for Research Software Development. James Hetherington will spend a lot of his time in the next few months talking to those of us working in various areas of research, so we will get to provide some input into the team's activities. However, we do know what they will be offering advice and consultation on software development best practices, including the fundamentals as well as the headline-grabbing HPC work, and hosting version control and issue tracking services.

Transitioning to the IPython Notebook

Thu, 04 Oct 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Here at Software Carpentry we've been teaching Python for a little while. We teach Python in our bootcamps and we have a library of instructional videos on Python topics including basics, dictionaries, regular expressions, and object oriented programming.

As usual, though, we're not satisfied with the status quo. For both our bootcamps and our online material we're in the process of switching to the IPython Notebook. The Notebook lets the user write and execute Python code and see the results, all inside their favorite web browser.

Wanted: An Entry-Level Provenance Library

Thu, 04 Oct 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

One of the reason we keep teaching Subversion is that it allows us to show students a simple but useful trick. If you add the following to a text file:

$Revision: $

and then tell Subversion to set the "Revision" keyword on that file, the next time you commit it, Subversion will automatically update the text to:

$Revision: 423$

or whatever the revision number actually is. This is handy if you're mailing files around, and want people to be able to tell exactly which revision they have, but what makes it really useful is this:

Best Practices for Scientific Computing

Wed, 03 Oct 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The following pre-print is now available on arXiv:

Best Practices for Scientific Computing

D.A. Aruliah^a, C. Titus Brown^b, Neil P. Chue Hong^c, Matt Davis^d, Richard T. Guy^e, Steven H.D. Haddock^f, Katy Huff^g, Ian Mitchell^h, Mark Plumbleyⁱ, Ben Waugh^j, Ethan P. White^k, Greg Wilson^l, and Paul Wilson^g.

^aUniversity of Ontario Institute of Technology, ^bMichigan State University, ^cSoftware Sustainability Institute, ^dSpace Telescope Science Institute, ^eUniversity of Toronto, ^fMonterey Bay Aquarium Research Institute, ^gUniversity of Wisconsin, ^hUniversity of British Columbia, ⁱQueen Mary University London, ^jUniversity College London, ^kUtah State University, and ^lSoftware Carpentry.

How to Help at a Bootcamp

Tue, 02 Oct 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

When we lead a bootcamp of more than, say, a dozen people, extra helpers are essential to making everything go smoothly. As our guide to running a bootcamp says:

It's ideal to have one helper for every five attendees. Recruit helpers locally, from your own and neighbouring institutions.

Helpers should have some knowledge of the material covered in the bootcamp. Ideally, a helper will already have attended a bootcamp and would like to become an instructor themselves.

What Would You Like in an Instructor's Guide?

Mon, 01 Oct 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

One of our goals for the next three months is to create an instructor's guide for Software Carpentry to help people teach this material more effectively: something akin to the SSI's "How to run a Software Carpentry bootcamp", but larger, and focused on the content. For example, when we teach the Unix shell (which is normally the morning of the first day), we actually don't introduce many commands—last time I actually checked, I only showed learners pwd, mkdir, cd, ls, rm, mv, cat, head, tail, wc, cut, and history. Instead, our real aims are to give them what they need to get through subsequent programming exercises in Python (i.e., creating data files, moving program files around, etc.), and to:

Oslo and Columbia

Sun, 30 Sep 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Here's some feedback from participants in our Oslo workshop:

Good

Discovered a dishwasher

Test-driven development

Lots of anecdotes

Power plugs and coffee!

Jokes (no, really)

Reminded me why I should use version control

Inspirational

Discovering the web page

Not slaughtered by Vikings (yet)

Good teaching skills

Free!

Bad

Lunch at twelve, please

Didn't cover SQL

Need to show code and output side by side

Couldn't keep up with typing

Greg speaks too fast

Not good context for some examples

Matrix programming lacked exercises and examples

More exercises please

34 registered, 15 attended

Tell us more about installation

Lack of key point summaries

Too short

And from our Columbia University workshop in New York City (see also this great writeup from the course instructor, Dr. Rachel Schutt):

The Real Hard Work

Sun, 30 Sep 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I spent a couple of thought-provoking hours at Codecademy's office in New York on Thursday, during which my host said, "It's so different from Silicon Valley, where a lack of experience is considered an asset." The next day, while teaching at Columbia, I used one of my favorite sayings, "A week of hard work can sometimes save you an hour of thought," and someone in the audience piped up, "Or an hour of reading." The two comments helped crystallize something that I've been thinking about for several months now about what's hard for creative people to do, and what implications that has.

Workshop at the University of Newcastle in October

Sat, 29 Sep 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The University of Newcastle is running a Software Carpentry bootcamp on October 22-23, 2012. For more information, or to register, please see their web site.

How to Run a Bootcamp (new and improved)

Thu, 27 Sep 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The folks at the Software Sustainability Institute have written a guide to running a Software Carpentry bootcamp that includes a lot more detail than our own. It's a great resource—if you have suggestions for improvements or additions, please send 'em to us.

Computational Thinking and Ice Floating in Bathtubs

Wed, 26 Sep 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I've been thinking about what "computational thinking" actually means ever since a workshop at Microsoft in 2007, where I found that everyone's definition seemed to boil down to, "Whatever I've been pushing for the last decade, but with a new name." My ideas keep evolving, but Lewis Epstein's wonderful book Thinking Physics remains their foundation. In it, he presents dozens of problems in freshman physics that can be solved without calculation, so long as you have a solid grasp of basic Newtonian mechanics. For example: put a large piece of ice in a bathtub, then fill the bathtub exactly to the rim. When the ice melts, will the water overflow, will the water level go down, or will it stay the same? The answer is that it will stay the same, because the ice displaced its own weight in water, so when it melts, it exactly fills the "hole" it created. But that's not the real question; the real question is, imagine you put a large cake of ice in the tub, then put a small iron weight on the ice cake, and then filled the tub to the rim. This time, when the ice melts and the iron weight sinks to the bottom of the tub, what happens to the water level?

Feedback and wrap-up from York

Thu, 20 Sep 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The SoundSoftware project arranged a Software Carpentry bootcamp for the week preceding DAFx'2012, the conference on Digital Audio Effects, at the University of York.

This bootcamp featured two days of non-subject-specific Software Carpentry material, presented by Greg, and an additional day of material specific to audio and music researchers, presented by Adam and Becky of Codasign. Many thanks to Greg, Adam, and Becky for their hard work.

Why This Stuff Is Hard To Teach

Thu, 20 Sep 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

If we get funding to continue our work (we hope to find out in a month), one of the first things we want to do is put together an introduction to web programming for scientists. As I've remarked many times before, we won't try to teach people how to build web applications: all we can do in the time we have, starting from what they know, is teach them how to create security holes. What we would like to show them is how to pull data off the web and post data of their own for others to consume, but even that turns out to be a lot harder than it should be.

Post-Mortem on the NGS Course

Tue, 18 Sep 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Titus Brown has posted a post-mortem on his Next Generation Sequencing course, and the independent assessment that was done of it. As with Jorge Aranda's assessment of Software Carpentry, the results were very positive—what we and other like-minded people are doing really does make a difference.

What's In Your Stack?

Tue, 18 Sep 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As a long-delayed follow-up to a conversation with Travis Oliphant: based on our experience, researchers who are computational novices want more than numerical computing and visualization tools. They want a complete stack—an end-to-end, nachos-to-cheesecake [1] solution to their basic computing needs, including an editor, a debugger, a version control system, blogging tools, data management tools, some way to create presentations and papers, and so on. My personal stack includes:

Systematic Curriculum Design

Sun, 16 Sep 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Executive summary: we'd appreciate your help organizing and motivating our material better.

One of the good things about traveling is that it gives me time to think. One of the bad things about thinking is that every time I do, I wind up with more work than I had when I started. For example, to organize and motivate our content, I'm using eight questions that scientists frequently ask:

Number Crunching with Python: DC Python Workshop

Thu, 13 Sep 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

On November 11, 2012 I'll be giving a workshop in Washington, D.C. on number crunching with Python. The event is organized by DC Python and Data Science DC, who were kind enough to ask me to teach. The focus of the workshop will be on core scientific Python packages like NumPy, SciPy, matplotlib, IPython, and, with the help of Skipper Seabold, pandas.

The workshop filled up fast but there is a wait list you can join in case people drop out or we find a bigger venue. Details are at http://meetup.dcpython.org/events/81931062/.

Patterns Wanted

Wed, 12 Sep 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

At some point or other, most programmers have encountered the idea of design patterns in software, and many (including myself) have been zealous about them, at least for a while. They haven't actually revolutionized either the practice of software development or the way we teach it, but becoming familiar with them is to programmers what learning the Beatles' greatest hits is to musicians.

The Software Is Open (even if the interviews aren't)

Wed, 12 Sep 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

FLOSS for Science, a site devoted to free/libre/open source software for scientific computing, has just published two volumes of interviews with people in the community. The content doesn't appear to be open, but it's only $9.99 for the pair.

How Quickly Do Workshops Fill Up?

Thu, 06 Sep 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I had to compile some data on signups anyway, so here's a graph showing cumulative registration per workshop over time (counting from the day registration opened) for past workshops:

and for upcoming workshops (excluding two that just opened up):

Doing a bit more analysis, this spring's workshops filled up a lot faster than the ones we're running this fall. I think the main reason is that those ones were organized and advertised while school was on, while the upcoming series have been set up over the summer. We'll repeat the analysis in a few months to see if that hypothesis holds.

Final Results of Demographic Survey

Tue, 04 Sep 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

192 people have now responded to the demographic survey we reported two weeks ago. Results are shown below; I'll hold off posting breakdown by discipline until I find a better (standard) classification scheme. Long story short, you are grad students, age 25-35, 28% female, with no clear preference for Linux, Mac OS X, or Windows. No idea what kind of music you listen to, though, or where you stand on the crucial "cave men versus astronauts" question—we'll include those in next year's survey.

Free As In Pretty Much Whatever You Want

Tue, 04 Sep 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A couple of different people have asked us recently whether they can use our materials in their courses. The answer is an emphatic "yes": all of our slides, essays, posts, and what-not are covered by the Creative Commons — Attribution license, and all of our code is covered by the open source MIT License, so basically, you can do whatever you want, whenever you want, without seeking special permission. (And yes, this means you can charge for your courses, even though we don't charge for ours.) The only requirement (imposed by the CC-BY license) is that you cite us as the source, and of course we'd be grateful if you'd point us at your material and/or send us any changes you make.

Not Really Disjoint

Tue, 04 Sep 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The twinned discussions in bioinformatics about openness and software quality are heating up. A recent salvo on Gas Stations Without Pumps is titled "Accountable research software", and one statement in particular caught my eye:

The rapid prototyping skills needed for research programming and the careful specification, error checking, and testing needed for software engineering are almost completely disjoint.

I might agree that careful specification isn't needed for research programming, but error checking and testing definitely are. In fact, if we've learned anything from the agile movement in the last 15 years, it's that the more improvisatory your development process is, the more important careful craftsmanship is as well—unless, of course, you don't care whether your programs are producing correct answers or not.

Lifted by the Audience

Sun, 02 Sep 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I spent Thursday and Friday recording most of the material we've been using in workshops for the past six months [1]. One thing that kept bugging me was how flat and uninspiring it was to talk to a camera: I had forgotten just how much a live audience energizes me. I also realized how much I rely on people's questions, and the expressions on their faces, to keep me on track: it took me almost 15 minutes to remember that I needed to explain that Python uses indentation to show nesting, whereas I always do that at the right time in a live class because someone either asks or looks puzzled.

A Problem With Badges

Wed, 29 Aug 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We issued the first learning badges for Software Carpentry a couple of weeks ago, and in doing so, uncovered a significant flaw in the design of existing badging systems. My intent was to give a bunch of people "Instructor" badges; by mistake, I set the pulldown to "Organizer", then clicked "Issue". As a result, two dozen people got the wrong kind of badge—and there doesn't appear to be any way for me to undo that. According to Dave Lester, who created the WPBadger plugin for WordPress that I'm using:

Linking Forward From a Bibliography?

Wed, 29 Aug 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This web site's bibliography lists 116 papers related in some way to the practice of scientific computing. I'd like to know:

what related papers I've missed, and

when a new related paper appears.

It seems to me that I ought to be able to upload the entire bibliography into some tool, then say, "Find everything that isn't in this list that cites things that are, minus everything in this other list over here that I've already looked at and decided aren't relevant." Going a step further, I ought to be able to prime some kind of robot to do this check automatically every week or so, and mail me when there's a hit so that I can either read it or add it to the exclusion list.

Please Help the Hunter Family

Wed, 29 Aug 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are very sad to report that John Hunter, the principal author of the widely-used Python plotting package Matplotlib, is losing his battle with cancer. If you would like to help his family in this difficult time, please see Stefan van der Walt's post, or make a donation to the memorial fund set up to help with his children's education.

An Interview with Titus Brown

Mon, 27 Aug 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The folks at Simply Statistics interviewed Prof. Titus Brown earlier this month; there's lots of good stuff in the post for people interested in doing computational science (or any kind of science) better.

An Updated List of Upcoming Workshops

Tue, 21 Aug 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Here's an update to our upcoming workshop list:

Venue Start Date Instructor(s)

DAFX Conference (York, UK) 2012-09-13 Greg Wilson

University of Oslo 2012-09-17 Hans Petter Langtangen, Greg Wilson

Purdue University 2012-10-08 Michael Hansen, Anthony Scopatz, Jeff Shelton

Lawrence Berkeley National Laboratory 2012-10-17 Matt Davis, Katy Huff

University of British Columbia 2012-10-18 Laura Trembley-Boyer, Ian Mitchell, Peter Rawsthorne, Greg Wilson

University of California Berkeley 2012-10-20 Matt Davis, Katy Huff

Caltech 2012-10-23 Matt Davis

Oxford 2012-10-30 Stephen Crouch, Mike Jackson

Scripps Institute 2012-11-15 Titus Brown, Tracy Teal

University of North Carolina 2012-11-28 Jason Pell, Ethan White, Greg Wilson

University of Edinburgh 2012-12-04 TBD

University of Texas (Austin) 2012-12-10 Rosangela Canino-Koning, Katy Huff, Anthony Scopatz

University of Chicago 2013-01-12 Katy Huff, Anthony Scopatz

Technical University Munich 2013-01-22 TBD

Macquarie University 2013-01-31 Greg Wilson

Virginia Tech 2013-01-31 Tommy Guy

AMOS Conference (Melbourne) 2013-02-14 Greg Wilson

We're currently trying to settle on dates (and instructors) for:

What We Talk About When We Talk About Software Carpentry

Mon, 20 Aug 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Who Are You?

Fri, 17 Aug 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We asked participants in this year's workshops to tell us a bit about themselves. So far, 121 have done so, and their responses are summarized below.

Age

<20 0.0%

20-25 18.3%

26-30 31.7%

31-35 25.0%

36-40 9.2%

41-45 4.2%

46-50 4.2%

>50 7.5%

Gender

Female 26.9%

Male 73.1%

Occupation

Undergrad 2.6%

Grad student 55.7%

Post-doc 13.9%

Faculty 9.6%

Support 13.0%

Industry 5.2%

Preferred Platforms

Linux 54.2%

Mac OS 50.0%

Windows 35.8%

Platforms sum to more than 100% because multiple responses were allowed; I was surprised by the number of Linux responses, since we only ever had a handful of Linux laptops in the room, but looking at the responses in more detail, it appears that many of the people who use Mac laptops use Linux servers for production runs.

Alpha Test of Driver's License Exam

Thu, 16 Aug 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As we announced back in June, we're working with the Software Sustainability Institute to create a "driver's license" exam for the DiRAC supercomputing facility. Mike Jackson at the SSI alpha tested our exam on four people last week; the exam itself, and his comments, are below. We would be very grateful for feedback from the community on the scope and level of the questions.

Exam

You have been given the URL of a version control repository, along with a username and a password for it. You have one hour to complete the following tasks. If at any point you would like clarification, please do not hesitate to ask. If at any point you are unable to complete a task, you may also ask for help so that you can proceed to the next task, but doing so will be considered equivalent to not completing that task.

A Question and Answer Matrix for Software Carpentry

Tue, 14 Aug 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Following up on yesterday's post about applying educational principles to this course, here's a not-yet-completed Q&A matrix for this course. The section headings are questions people ask (or equivalently, tasks they want to perform). The headings underneath are the major topics Software Carpentry covers, and below each of those is my attempt to relate those topics to the questions. "TBD" means "I haven't written it yet", while "N/A" means "I can't think of any relationship." This matrix is going to be the basis of our next big reorganization of material (which should start this fall), so we would be very grateful for your input:

Applying Pedagogical Principles in This Course

Tue, 14 Aug 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I've talked a bit about instructional design and educational principles in this blog in the past; here's a concrete example of how we try to apply those ideas. As well as teaching the basics of the Bash shell and Python, we also (try to) teach people that every programming system must have:

Individual things. A typical "thing" in the shell is a file; in Python, it's a number or string.

Groups of things. In the shell, these are the lines in a file, or the files that match a pattern like *.dat. In Python, they're lists and dictionaries.

Commands that operate on things, like addition or the cp command.

Ways to repeat commands, either explicitly (like for loops in the shell and Python) or implicitly (like wc *.dat).

Ways to make choices, like if statements in Python or file-matching patterns in the shell. (We're not going to try to teach people how to do conditionals in the shell.)

Ways to create chunks, like functions and shell scripts.

Ways to combine those chunks, like function composition and pipes.

These ideas are connected: groups of things and ways to repeat commands tie together naturally, as do creating functions and combining them. Those connections:

Interview about Software Carpentry (and Education)

Tue, 14 Aug 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Chris Gammell and Jeff Shelton interviewed me a couple of weeks ago for their Engineering Commons podcast series. The interview turned out to mostly be about education, online and otherwise—I hope readers will find it interesting.

That Was Quick

Wed, 01 Aug 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Registration for our October 2012 workshop at UC Berkeley opened mid-afternoon yesterday. We were sold out and putting people on the waitlist in less than four hours.

We're Going to Be Busy

Wed, 01 Aug 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We're going to be busy—and these are just the ones we have confirmed.

Sep 13-14, 2012: DAFX Conference (York, UK)

Sep 17-18, 2012: University of Oslo

Oct 8-9, 2012: Purdue University

Oct 17-18, 2012: Lawrence Berkeley National Laboratory

Oct 18-19, 2012: University of British Columbia

Oct 20-21, 2012: University of California Berkeley

Oct 23-24, 2012: Caltech

Oct 29-30, 2012: Oxford

Nov 15-16, 2012: Scripps Institute

Nov 28-29, 2012: University of North Carolina

Dec 4-5, 2012: University of Edinburgh

Dec 10-11, 2012: University of Texas (Austin)

Jan 19-20, 2013: University of Chicago

Jan 22-23, 2013: Technical University Munich

Feb 15-14, 2013: AMOS Conference (Melbourne)

Record and Playback

Mon, 30 Jul 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The biggest bottleneck Software Carpentry faces right now is a shortage of experienced instructors. To help fix that, we are going to record a complete presentation of our core two-day material so that people who want to teach it themselves can see how we say things, as well as what we say [1, 2].

As soon as we say "record", though, we have to ask, what exactly are we recording? Audio and video of a presenter in front of a whiteboard? Sure—that helps humanize the presentation. But what about the presenter's desktop? Viewers definitely need to see it, but should they see an MP4 in which the text on the presenter's screen appears as colored pixels arranged in the shapes of characters, or should we record the characters directly? I think the latter is by far the best option, since:

Software Carpentry Needs You!

Sat, 28 Jul 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Software Carpentry needs you! If you found our workshops, tutorials, or online material useful, there are many ways you could help us grow:

Send us a testimonial to add to this page, or write a longer one that we can post on our blog. (We show these to potential sponsors as evidence that we're doing good things, and to people who are thinking about organizing or attending workshops to give them an idea of the benefits.)

IPython Notebook + Towtruck + Etherpad + Slide Drive = Win

Sun, 22 Jul 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Executive summary: the tool I want for teaching programming doesn't yet exist, but could be built by combining the IPython Notebook, Towtruck, Etherpad, and Slide Drive. This would be a better web-based tool for teaching programming than anything currently available, and would alleviate the most critical bottleneck programming-for-everyone efforts face: the shortage of competent tutors.

A lot of people say harsh things about PowerPoint, but as I've discussed before, the alternatives are all unsatisfying as well. HTML5 slideshows don't allow authors to mix text and graphics free-hand the way people do on a whiteboard; instead, the two have to be segregated into blocks. Screencasts are unsatisfying too: they don't display first-class content [1], and, like slideshows, aren't interactively hackable.

How Robust Is Your Programming Language?

Sat, 21 Jul 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

One of the biggest problems in teaching novices how to program is that most programming systems are not robust. A car can go quite a long distance on a slightly-flat tire, and people can live for years with just one kidney or half a liver, but get one character out of place in a program, and boof—it's game over. And spotting that one character can be very hard, particularly if you're a novice and are learning mostly through copy-paste-and-tweak.

Software Carpentry in Paris !

Sat, 21 Jul 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A couple of weeks ago, we organized the first Software Carpentry Bootcamp in Paris. We were lucky enough to be able to host it at INRIA's research center, Place d'Italie. The focus of this bootcamp was Python, with 3 interventions on python: beginner's introduction, by Christophe Combelles, numerical computing with numpy by Konrad Hinsen and the python scientific ecosystem by Alexandre Gramfort, and an introduction to DCVS with git, by myself.

Workshop wrap up from the Rutherford Appleton Laboratory

Thu, 19 Jul 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Last week, Software Carpentry hosted a bootcamp at the Costener's House, a converted monastery owned by the Rutherford Appleton Laboratory in Oxfordshire, England. Fifteen participants took part, with experience ranging from "gap year" (about to enter University) to people with years of professional development experience.

One of the things that stood out at this workshop was the more advanced background of many of the participants, which allowed us to be more advanced with the material. The topics that seemed most popular were Software Engineering and Testing. From my perspective, it seemed like the hardest part of the workshop was the exercise we designed for the workshop, which was a scaled down version of the recommendation engine we use as an example in the Matrix Computation portion of our material. We did not use numpy, so placed a greater emphasis on preparing the data for further computation. While it took longer than I anticipated to explain and motivate the exercise, the payoff was that we could use the same problem on day 2 to introduce testing and databases. One of the difficulties in a 2-way workshop is avoiding the feeling of jumping topics every hour, and carrying this example through all of our Python work seemed to help tie the topics together.

Wrapping Up in Halifax

Tue, 17 Jul 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Things went pretty well here in Halifax—thanks to the local helpers, and to Justin Ely from STScI for coming up to teach. Next stop, Scarborough!

Good Bad

Pedagogy

Work more efficiently (shell)

Support staff

pair computing / working in groups on problems

Version Control / logging

Planning workflow methods

provenance

data processing

website

citations

passing on good habits

good teaching tools

presentation style

post-its

stories

scheduling

transfer concepts to science

"practical" code practices were very portable

too fast/too short

more on code organization

more handouts / outline

svn binary examples / how to create repository

how do i organize repository?

wanted more background on everything

testing too confusing

incomplete system requirements for examples

stories

greg works too fast / code disappears on run

how to put all together?

public relations/ getting the work out

introduce everyone and helpers

Some people were wait-listed and people didn't show up

use teaching tools more

contrived examples

Wrapping Up in Boston

Tue, 10 Jul 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We just wrapped up a two-day workshop in Boston with learners from several universities. It seems to have gone pretty well; we look forward to coming back soon. Many thanks once again to Jessica McKellar, Geraldine van der Auwera, and Gus Muench for setting it up, and to Jessica for teaching yesterday afternoon.

Good Bad

learning shell

databases

running text editor from shell

"I feel relatively functional"

integrating different tools on the shell

learning software testing

unit testing

instructor cares about learning how we learn

emphasized conceptual aspects

shell scripting

having ways to learn more about each subject

database + integration with shell

Python programming

nice length

version control for the win!

modularization for the win!

Greg's anecdotes

liked the level of teaching

well integrated with web site

liked specific recommendations (provenance)

couldn't keep up

not enough coffee in the room

already familiar with some stuff

Greg types too fast then covers up the window

sometimes ducks questions

not having room information ahead of time

software installation list please

setup on Windows

afternoons too fast

didn't know what level to expect coming in

wanted more Python

want final products

how to speed up

9:00 is early for MIT people

wanted to learn more about paths and shell stuff

Greg's anecdotes

explanation of Unix organization

where to find help

more assistants around

one-page primer

Independent Assessment of the Past Six Months

Thu, 05 Jul 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As many of you know, Dr. Jorge Aranda has been doing an independent assessment of Software Carpentry's effectiveness over the past six months. His brief was to tell us whether we were having an impact on scientists' lives, and if so, what kind, and how we could do better. He just submitted his final report (PDF), and we would welcome your thoughts.

A Supercomputing Driver's License

Wed, 27 Jun 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Software Carpentry and the Software Sustainability Institute are working together to develop a "driver's licence" for researchers who wish to use the DiRAC integrated supercomputing facility. The aim is to assess whether a researcher has the programming skills needed to use DiRAC productively, and if not, give them feedback on what further training they need. The hour-long test, which will include writing, coding, and interactive elements, will focus on fundamental skills including version control, unit testing, writing maintainable code, using Makefiles, and code review. Trials will begin this summer, and the completed test will be presented to DiRAC in September 2012.

Fortran Format Statements and Regular Expressions

Wed, 27 Jun 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

One of our learners has a large number of Fortran format statements embedded in a program he's inherited, and would like to come up with equivalent regular expressions in order to parse the program's data with tools written in other languages. If you know of a tool to translate one into the other, or a Fortran-to-regexp dictionary, we'd welcome a pointer.

Where We Are (June 2012 edition)

Wed, 27 Jun 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Last week, a double dozen friends and colleagues gathered physically and virtually to review what we've done and where we should go next. I'll postpone discussion of the biggest "what next" items to separate posts, but here's the status report itself.

Successes

We've reached a lot of people

We have delivered 12 workshops to 376 learners in 4 countries (curriculum)

Numbers will be 18 workshops, 520 learners, and 5 countries by the end of July

Have actually lost count of the number of online tutorials we've run...

What we're doing is helping

96% of participants happy with training

Significant improvement in computational understanding

We get mail

Scientists want more

17 workshops in the pipeline

2-4 new contacts/month

We can scale

20 co-instructors

7 have gone on away missions

5 have run online tutorials

We can become self-sustaining

Have raised $25K in other funding

Shifting to "host pays costs" model for workshops

Will try to count Software Carpentry training toward NSF requirements for training in Responsible Conduct of Research

Working with the Software Sustainability Institute to create a driver's license for the DiRAC supercomputing facility

Room for Improvement

Very little new video content (because very little demand)

But some interesting experiments using video for formative assessment

No learner-led challenges

Most grad students feel their research is challenging enough

Having more mentors, from more disciplines, may help us help more people with their research problems

Badging delayed pending a software release (mid-July)

Initially badge instructors rather than learners (greater immediate value to recipient)

Never-ending installation/setup/configuration difficulties

Particularly for Windows users (20-40% of our learners)

All technical solutions have proven to be unsatisfactory in different ways

Some material is going stale and/or not as useful as we thought (e.g., Subversion, databases)

Will update/replace

Pacing continues to be a problem

20% find it too fast while 20% find it too slow

More instructors will allow us to stream people appropriately

Online tutorials have had mixed success

"Only" 10-20% of learners still participating after 3 weeks

Which is actually higher retention than many online courses

Partly because of technical difficulties and time pressures

But online instruction remains less effective than in-person instruction with today's technology

Workshop participants are predominantly male (even when drawing from disciplines that are gender-balanced)

Now in contact with WiSE/diversity groups in the US to try to arrange workshops for their members

Still don't know what to teach scientists about the web

Handling Variant Configuration Files

Tue, 26 Jun 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

One of our learners came to us with a problem last week. The program she uses depends on some complex configuration files, which she'd like to store in version control. However, a couple of parameters change depending on the machine the program is running on. She doesn't want to check those changes into version control over and over again; what should she do?

Pessimism and Doom

Tue, 26 Jun 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

According to recent research, an absence of optimism plays a large role in keeping people trapped in poverty:

This hopelessness manifests itself in many ways. One is a sort of pathological conservatism, where people forgo even feasible things with potentially large benefits for fear of losing the little they already possess.

The parallels with scientific computing practically jump off the page, don't they?

Two Posts on Scientific Workflows

Tue, 26 Jun 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Carly Strasser recently posted two articles on informal and formal scientific workflows, both of which are likely to be interesting to readers of this blog.

If You Want to Teach, Isn't It Only Fair to Learn a Few Things First?

Mon, 25 Jun 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Carla, a high school student, is doing a class project comparing climate change in the Northern and Southern hemispheres. She wants to see whether the gap between the average annual temperatures in Canada and Australia increased during the Twentieth Century. The raw data she needs is available online; all she needs to do (for some value of "all") is get it, load it into her program, do her calculations, and create an HTML to display her results for other students to see.

Feedback from Johns Hopkins

Wed, 20 Jun 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Today we wrapped up our 2-day bootcamp at Johns Hopkins University in Baltimore. We had a pretty small group of about 12 each day (out of 20 signed up). In a really pleasant surprise we had students come from Brooklyn, NY and Virginia, plus two students who commuted up from Sigma Space in Lanham, MD. Overall feedback seemed quite positive:

Good Bad

advanced bash (find, pipes)

git and GitHub are cool

all the bootcamp material is online (at GitHub)

intro to Python

free

intro to shell

saw some advanced Python

testing/debugging

branching in git

bootcamp was local

examples relevant to scientists

watching how someone else works

we covered everything we planned (from Matt)

show/use more IPython features (from someone who already uses it)

shell went too slow, Python went too fast

projector small/dim (+fonts small)

had to take time off work (about half said they would have come on the weekend.)

would like domain specific Python examples

would like a third day (majority would have come for another day)

hard if you miss the first day

want more NumPy, SciPy, etc.

get left behind if you make a mistake

don't see complexity when writing examples from scratch, could provide some pre-written code

more step-by-step instructions online

wanted to see professional workflow with IPython notebook (from Josh)

could use pre-class surveys to tailor lessons

Big thanks to the other instructors Joshua, Sasha, and Mike!

A Busy Week (And Schwag!)

Mon, 18 Jun 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We have a busy week coming up:

Monday: Start of a two-day workshop at Johns Hopkins University

Online tutorial for participants in the London workshop

Tuesday: Online tutorial for participants in the Edmonton workshop

Wednesday: Online tutorial for participants in the Newcastle workshop

And another one for participants in the Vancouver workshop

And one more for participants in the Michigan State workshop

Thursday: Combined tutorial for participants in previous workshops

Friday: All-day strategy meeting in Toronto

In amongst all of this, we're revising a paper on recommend practices for scientific computing, summarizing our assessment of the impact our workshops have had, filling in a few more blanks in our instructors' guide, and talking to four sites about running workshops in the fall.

Pretty Well Sums It Up

Fri, 15 Jun 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Speaking at a conference in the UK today, Andrew Eland (of Google) said:

Think of the things scientists won't build because they're not exposed to computing in school.

+1 to that.

This Week's Tutorials

Fri, 15 Jun 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We ran five and a half online tutorials this week: one each for the workshop participants from UCL, Newcastle, UBC, and Edmonton, one for students combined from five previous workshops, and a make-up mini-tutorial for students from Plymouth who'd had A/V problems. The topics included:

creating simple HTML pages (both by hand and programmatically)

Python dictionaries (a.k.a. "hashes" or "maps" in other languages)

the Pandas statistical package

People who would like to review should look at the first half of the Version 3 notes on HTML (slides 1-17), the Version 4 videos on sets and dictionaries, and—well, we don't have anything on Pandas yet, but its creator has a book that you can preview.

All Entries for the Executable Paper Grand Challenge

Thu, 14 Jun 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I don't know how I missed this, but at last year's International Conference on Computational Science, two dozen different groups presented papers in response to Elsevier's Executable Paper challenge. PDFs of their work are all online, and links are included below. As I've said before, one of our goals is to give researchers the skills they need to do this kind of thing; looking through these papers, we've still got a ways to go.

First Workshop on Maintainable Software Practices in e-Science

Sun, 10 Jun 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

First Workshop on Maintainable Software Practices in e-Science

9 October 2012
Co-located with 8th IEEE International Conference on eScience, Chicago

This workshop will focus on the issues relating to the development and maintenance of software that can endure past the limited periods of defined project durations and project funding, and go beyond software engineering best practice to address aspects of cultural, organisational and policy change. By bringing together all those with an interest in ensuring the longer term development and use of software for research, including researchers, developers, research computing specialists, software engineers, infrastructure providers, facilitators, and funders, the goal of this workshop is to understand what software practices can be successfully applied and which lead to long-term improvements in the development of software for e-Science.

But the Greatest of These Is...

Fri, 08 Jun 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

In her EAGE keynote earlier today, Victoria Stodden talked about the central role of geophysics in the reproducible research movement. After discussing the problem, she identified five interlocking solutions:

Tools

Intellectual property barriers

Funding agency policy / federal regulation

Journal policy

Institutional expectations

I think these are all important, but I think they are all less important than something which doesn't appear anywhere in her slides:

We Get Mail

Fri, 08 Jun 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Hi Greg and Adina!

Two days ago I filled out the Software Carpentry survey which asked me, among other things, how much time I had saved using my new Software Carpentry skills. I filled out the I had not really saved that many hours yet (but that I was approaching computational problems in a different way and began implementing version control!).

However, that number of hours changed dramatically later that evening! In preparation for a talk I will give at a conference in a few weeks I was re-doing some analyses. I came to one which was very painful to me because it required using 3 different python scripts for several different files and repeating this for a range of numbers (so we're talking running 3 scripts about 20 times). I did not find this particular analysis very useful or informative, but it really makes my supervisor happy so I thought I should at least try to be a silver (or bronze) student and give it a whirl.

Ten Simple Rules

Thu, 07 Jun 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The "Ten Simple Rules" series being run in PLoS Computational Biology has a lot of useful gems. Written by editor-in-chief Philip Bourne and others, the entire collection to date is available as a single PDF, but for those who prefer bite-sized reading, here are ten simple rules for:

...Starting a Company (PDF)

...Getting Involved in Your Scientific Community (PDF)

...Teaching Bioinformatics at the High School Level (PDF)

...Developing a Short Bioinformatics Training Course (PDF)

...Getting Help from Online Scientific Communities (PDF)

...Building and Maintaining a Scientific Reputation (PDF)

...Providing a Scientific Web Resource (PDF)

...Getting Ahead as a Computational Biologist in Academia (PDF)

...Editing Wikipedia (PDF)

...Organizing a Virtual Conference—Anywhere (PDF)

...Chairing a Scientific Session (PDF)

...Choosing between Industry and Academia (PDF)

...Combining Teaching and Research (PDF)

...Organizing a Scientific Meeting (PDF)

...Aspiring Scientists in a Low-Income Country (PDF)

...Graduate Students (PDF)

...Doing Your Best Research, According to Hamming (PDF)

...a Good Poster Presentation (PDF)

...Making Good Oral Presentations (PDF)

...a Successful Collaboration (PDF)

...Selecting a Postdoctoral Position (PDF)

...Reviewers (PDF)

...Getting Grants (PDF)

...Getting Published (PDF)

So, what would your ten simple rules for good computational science be?

Tutorial: NumPy, SciPy, and matplotlib

Thu, 07 Jun 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Today I did a toy data analysis of some annual temperature data in Australia and Canada over the last ~100 years. The goal of the exercise was to demonstrate loading data, inspecting it, and fitting trends. My last tutorial didn't involve any real data so this week we wanted to change that.

Like my previous tutorial I used the IPython HTML notebook to present. I hadn't been planning to use pylab mode or inline plots but some issues with my matplotlib forced me in that direction. There's an awkward situation here because I actually recommend people not use the pylab interface to matplotlib because the behind-the-scenes magic can cause problems (difficult to debug problems), but for doing demos the inline plots are really the way to go. The obvious upside is that the plots I made as part of the tutorial are embedded in the notebook for you to see now.

Software Carpentry: The E-Book Version?

Mon, 04 Jun 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

If I could send email two years into the past, I'd tell myself to spend a lot less time making videos for Software Carpentry, and a lot more time exploring interactive formats. One that I've been following recently is the IPython Notebook, a "live" lab notebook that allows scientists to mix, share, and explore text, code, graphs, and the like. Another, which I've just come across through this post on Mark Guzdial's ever-informative blog, is an interactive version of "How to Think Like a Computer Scientist". As Brad Miller says in his description:

What Skills Are Required to Implement Open Access?

Mon, 04 Jun 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Late last night, local time, the Access2Research petition at whitehouse.gov got 25,000 signatures, which means that the White House is committed to responding. As someone who no longer reviews for journals that aren't OA, I obviously think that's great news, but it once again raises the question: what concepts and skills do researchers need in order to implement open access? Jon Udell has written at length about some things (most recently here), but to sharpen up the discussion:

Git tutorial links

Sun, 03 Jun 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I promised to post some links related to the Git tutorial I taught on May 24, 2012. Most of my material was based on this excellent online book (free): http://git-scm.com/book/ch1-3.html. Here's another resource with help on specific and common Git commands: http://gitref.org/.

Someone asked the differences between Git and Mercurial, but because I haven't used Mercurial I wasn't able to answer. I found these links that I hope are useful: http://www.wikivs.com/wiki/Git_vs_Mercurial and http://www.rockstarprogrammer.org/post/2008/apr/06/differences-between-mercurial-and-git/. Also, here's a video of Linus Torvalds himself (creator of the Linux kernel and Git) promoting Git (and putting down everything that is not Git):

Introduction to NumPy Tutorial

Fri, 01 Jun 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Today I did a tutorial moving quickly through the basic usage of NumPy, the essential library for doing numeric computing with Python. We covered building arrays; indexing; array math; NumPy's element-wise functions; array attributes and methods; random numbers; masked arrays; and array comparison.

I presented using the IPython HTML Notebook. I enjoyed it because I never had to switch between a terminal and text editor, but I wonder what the people viewing thought. A nice feature of the notebook is that I can export it, both as a PDF and in the .ipynb format importable by IPython. The PDF is here, and the .ipynb file here.

Dictionaries are a Scientist's Friend

Thu, 31 May 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

[Code]

Today I would like to share with other scientists the power of dictionaries. I recently learned about this data structure during a Software Carpentry bootcamp that Greg Wilson and Ethan White organized. Greg and Jason Pell (from Michigan State) visited Utah State University for a two day bootcamp where I am a postdoc studying patterns of biodiversity. During that time we covered a whole slew of topics, but today I wanted to demonstrate how dictionaries helped to significantly speed up my Python code.

A Poster for the Software Carpentry Workshop at INRIA

Wed, 30 May 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

SoundSoftware 2012: Workshop on Software and Data for Audio and Music Research

Wed, 30 May 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The SoundSoftware project — a UK based project aiming towards software sustainability in the Audio and Music Research community — is organising a one day workshop on "Software and Data for Audio and Music Research". The aim of this workshop is to discuss issues such as robust software development for audio and music research, reproducible research, management of research data, and open access.

How to Run a Bootcamp

Tue, 29 May 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

With so many people setting up and running Software Carpentry workshops, I thought it was time to put together a more complete how-to. If I've forgotten anything, please let me know. (Later: please also see this PLoS paper"Ten Simple Rules for Developing a Short Bioinformatics Training Course".)

Assumptions

bootcamp is 2-3 days long

For 20-50 learners...

...who are typically grad students in science and engineering

There is 10:1 ratio of helpers to learners (or better)

Goals

What to Read If You're Teaching Software Carpentry

Sun, 27 May 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

One of our goals—in fact, our biggest goal—is to grow the pool of people who can teach Software Carpentry workshops, so that we can reach more researchers in future. We've learned that having scientists teach scientists works better than having professional programmers (or computer science grad students) teach scientists, but that raises a question: what do we teach the average neuropsychologist or geophysicist about software engineering to prepare her for teaching our stuff? Ideally, we'd like them all to have read a double dozen books, and to have ten or more years of experience in industry, but that's not going to happen. Instead, we've settled on a very short list:

Feedback from the University of British Columbia

Thu, 24 May 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Aaaand that's a wrap in Vancouver, ladies and gentlemen: our workshop at the University of British Columbia seems to have gone well:

Good Bad

clarity

data management

sticky notes

workflow

overview/coverage

resources

coffee/snacks

exercises

how to use programming correctly

learning terminal

testing

learning science

Python

helpers

one step further

version control

modules on website

anecdotes

general philosophy (organic)

not enough Python

too fast (esp 2nd day)

hands-on (second day)

needed warmup primer

Greg's jokes

resources (overwhelming)

text editor

software install issues

too short

crossover confusion

too much on data management

communication prior to course (software install)

screen real estate

more version control

why better than MATLAB

mystify setup

That's my last road trip until mid-July—it's good to have ended this round on a high note. My thanks again to Ian Mitchell, Adina Chuang Howe, and everyone else who helped or attended.

No CT Without PL

Thu, 24 May 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

In a blog post earlier today, Mark Guzdial argues that computational thinking requires learning with a programming language. Unlike many such claims and counter-claims, his is based on a wealth of research, most recently an excellent dissertation by Juha Sorva. I strongly agree with Mark's position: our real goal in Software Carpentry is to teach computational thinking, but the only way to do that successfully is to teach some basic programming skills and use those to convey larger ideas. If we're successful in getting follow-on funding at the end of our current grant, we're going to work hard to re-cast everything in this way.

Spot the Workshops

Thu, 24 May 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Alone and Misunderstood

Wed, 23 May 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Jeffrey Mirel and Simona Goldin's recent article in The Atlantic titled "Alone in the Classroom" initially struck a chord with me, particularly when they said, "A recent study by Scholastic and the Gates Foundation found that teachers spend only about 3 percent of their teaching day collaborating with colleagues. The majority of American teachers plan, teach, and examine their practice alone." But then Mirel and Goldin blew it by saying:

Responsible Conduct

Wed, 23 May 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Titus Brown, Ethan White, and I have been talking about what Responsible Conduct of Research (RCR) standards would look like in computational science. Titus has posted a four-point summary; we would welcome your input over on his blog.

Citing Versions

Tue, 22 May 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We got mail yesterday from a workshop participant saying, "My question is how does one show in a research paper that the underlying data and the software is version controlled?" Cameron Neylon's answer, slightly edited, was:

My approach in an idea world would be to have all of my data (or links to it) under version control along with the code. When the version to be used for the publication is clear I would give it a tag (I'm a Git user but there is similarly functionality in all version control systems) and then push that to an online repository. You can then give a link or reference to the appropriate repository version online. If you don't want to put your main repository online then you could just put up the version from the publication.

An Exercise With Matplotlib and Numpy

Mon, 21 May 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

For this tutorial, we'll be plotting some weather data from a site call Weather Underground. You can download temperature readings and weather events for your local area in a comma-separated file.

I've put weather data for Bloomington, IN in a file called weather.csv. Each row is one day, and there are columns for min/mean/max temperature, dew point, wind speed, etc. We'll be plotting temperature and weather event data (e.g., rain, snow).

Being More Systematic About Publicity

Mon, 21 May 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Several people have suggested that we need to be more systematic about publicizing workshops and other events: blogging and tweeting reaches people who already know about us, but doesn't reach those who don't. If you know of mailing lists and/or news aggregators aimed at researchers who might be interested in what we do, please mail pointers to {{site.contact}}.

What's Wrong With All This?

Sun, 20 May 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Titus Brown doesn't like this web site. He's OK with the content (I think), but he finds it awkward to use, and while I don't feel as strongly as he does, I accept that we have outgrown WordPress. The question is, what should we use instead? We need a lot more than just a blog and some static web pages, but learning management systems like Moodle weren't built with our ad hoc model in mind (they're really teaching administration systems), and newer tools like P2PU feel like a step backward. I started thinking about requirements for a replacement back in April, but got distracted. Here's a longer look.

Space at Upcoming Events

Sat, 19 May 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Here's how registration is going for upcoming events:

University of British Columbia May 22-23 39/40

Johns Hopkins University June 18-19 7/20

Paris June 28-29 9/25

Boston July 9-10 23/40

University of Waterloo July 12-13 1/40

Halifax July 16-17 8/40

University of Toronto (Scarborough) July 19-20 14/40

If you'd like to join us, there's still plenty of space—and if you have friends who could use some training in basic software skills, please point them our way.

Feedback from Alberta

Fri, 18 May 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Our two-day workshop at the University of Alberta wound up a couple of hours ago. We had quite a few no-shows this time (which was annoying, given how many people were waitlisted), but those who did come seemed to get a lot out of it:

Good Bad

Room

Mix of talking & doing

Stickies

Version control

Hands on

Link on online video

Python

Clear speaking

Computer in lab (using linux)

Automatic versioning

Programming in windows in Cygwin

Philosophy

Discussion of productivity

Good reading suggestions

Functional programming

Overall workflow

I feel more competent (morale boost)

Researched anectodes, backed with data

Website

TDD

Instructor's body language

Helpers

Coffee hard

Need more projectors

Having to keep stickies

No testing

Not enough depth

Not convinced about version control

Too fast on day 1, too slow on day 2

Need levels

Came late

Not enough Python

No lunch

No time for notes

More version control

Too short break

No shows

Pace (a little fast)

Supervisor wasn't here (need to convince her)

Where is the code (dropbox?)

Bad chairs

Windows alienation

Mailing list

Making DB (no info)

Many thanks to Rose, Neil, and Paul for making it possible.

The Most Important Scientific Result Published in the Last Year

Fri, 18 May 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

J.M. Wicherts, M. Bakker, and D. Molenaar: "Willingness to Share Research Data Is Related to the Strength of the Evidence and the Quality of Reporting of Statistical Results". PLoS ONE, 6(11): e26828, 2011, doi:10.1371/journal.pone.0026828.

Background

The widespread reluctance to share published research data is often hypothesized to be due to the authors' fear that reanalysis may expose errors in their work or may produce conclusions that contradict their own. However, these hypotheses have not previously been studied systematically.

Halifax in July

Thu, 17 May 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We have just added another workshop to the summer's list, this one at Saint Mary's University in Halifax, Nova Scotia, on July 16-17. Please let friends and colleagues know—I look forward to meeting them.

And One More: Johns Hopkins in June

Wed, 16 May 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We're pleased to announce that we will be running a two-day bootcamp at Johns Hopkins University in Baltimore on June 18-19, 2012. We only have space for 20 participants, so please register early.

Feedback from Newcastle upon Tyne

Tue, 15 May 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This week's Newcastle bootcamp, organised by the Digital Institute at Newcastle University with the Software Sustainability Institute and SoundSoftware, was the first Software Carpentry bootcamp run entirely locally in the UK. For the organisers it was a slightly nervous experience, hoping we could get the material to hold together in presentation without Greg's experience at hand.

Feedback from the learners was generally good on the material, the venue and the structure. The most common complaint was that it was hard to follow along at times, and I think there are several areas where we'll be able to improve the "flow" for future events.

Fooling the Internet

Tue, 15 May 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A recent article in The Atlantic titled, "How the Professor Who Fooled Wikipedia Got Caught By Reddit" describes how GMU's Prof. T. Mills Kelly has had students fake history online, and how their most recent effort unraveled. There's lots to think about here regarding what scientists should know about using the web, trusting it, and making it their own...

Two Bootcamps in Ontario in July

Tue, 15 May 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are pleased to announce that we will be running two bootcamps in Ontario in July: one at the University of Waterloo on July 12-13, and another at the University of Toronto (Scarborough) on July 19-20. If you'd like to take part, please sign up, and please let friends and colleagues know about them as well.

Solution to Indented List Problem

Mon, 14 May 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Last week's homework was to convert a two-level bullet-point list like this:

* A * B * 1 * 2 * C * 3

into an HTML list like this:

<ul> <li>A</li> <li>B <ul> <li>1</li> <li>2</li> </ul> </li> <li>C <ul> <li>3</li> </ul> </li> </ul>

so it would display like this:

A

B

1

2

C

3

My solution is shown in the video below; the code follows.

Feedback from Michigan State

Sat, 12 May 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Our workshop at Michigan State University this week was three days long instead of two, and included two topics (Git and the IPython notebook) that we haven't tried before. Feedback was generally positive, but we've got lots to work on for next time as well.

Good Bad

Using history

Ending with general theory

Pen and paper database design

Version control was useful

Good practice in software

Concise and module programming in Python

Console segment

Smooth flow between Bash and Python

Challenging and flowed nicely

Futher reading material

Desktop setup

Instructor teaching style

Permission to spend less time coding

iPython notebook looks great

Paired programming model

Git script (tutorials used were available)

Legal issues (opensource)

Good for beginners

Free course (and food!)

Variety

Practical (reality-based)

Overview of DB options

Testing

Better ways to do things

Somewhat static seating created helpful partners

Typing speed is too fast

Class time chunks too long

Why iPython

Need more 'why'

Curriculum

Advanced Git bounced

Too much switching screens

Some things failed

Beverages included only caffeine

Need snacks at breaks

Lacked connection between course material and applicability

Tuesday way too long

Wanted a cheat sheet

Not enough exercises

How to create DB

Anti-Windows bigotry

Next day install at end of day

Some concepts skipped

Don't know where to start (registrationg etc.)

Inappropriate room size

Breadth

PPT for CS

Run My Code

Fri, 11 May 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

RunMyCode is a web site and service intended to support reproducible research (initially in computational economics). Authors create companion web sites for papers that include the software they used; other people can then re-run their models, and (crucially) play with parameters, using cloud-based instances of those environments. They only support MATLAB, R, and SAS right now, but are hoping to add more tools soon. It's a cool idea, and we'd welcome your impressions.

Fish and Bugs

Thu, 10 May 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The May/June 2012 issue of Washington Monthly has an article by Alison Fairbrother titled "A Fish Story". Near the top, it says, "In 2009, a routine methodological upgrade at NOAA—and the subsequent discovery of a few lines of faulty computer code—forced the start of a profound shift in the ASMFC's estimates of menhaden stocks." A few pages later, we get more details:

Bootcamp in Boston, July 9-10

Wed, 09 May 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are pleased to announce that we will be running a bootcamp on July 9 and 10 in Boston—please see its page for details (some of which we're still working out). We have room for 40 participants, so please register early. (And if you can, register with friends: we are finding that people get a lot more out of this training if they're learning with their labmates and other collaborators.)

The Architecture of Open Source Applications: Volume 2

Tue, 08 May 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are very pleased to announce that The Architecture of Open Source Applications: Volume 2 is now available from Lulu. A PDF version will go on sale in the next few days, and e-book will become available as soon as we can produce it. Many thanks to everyone who contributed, and to the indefatigable Amy Brown for pulling it all together. As always, all royalties will go directly to Amnesty International, so if you buy a copy, you'll be helping to make the world a better place.

An Exercise With Functions and Plotting

Sun, 06 May 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Let's say you have a text file called workout.csv that contains information about your workouts for the month of March:

# date, kind of workout, distance (miles), time (min) "2012, Mar-01", run, 2, 25 "2012, Mar-03", bike, 10, 55 "2012, Mar-06", bike, 5, 20 "2012, Mar-09", run, 3, 42 "2012, Mar-10", skateboarding, 2, 10 # Broke my leg :( "2012, Mar-11", Wii, 0, 60 "2012, Mar-12", Wii, 0, 60 "2012, Mar-13", Wii, 0, 60 "2012, Mar-14", Wii, 0, 60

It's a common-separated value (CSV) file, but contains comments and blank lines. The first line (a comment) describes the fields in this file, which are (from left to right) the date of your workout, the kind of workout, how many miles you traveled, and how many minutes you spent.

UCL Bootcamp: Version Control Wrap-Up

Fri, 04 May 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

For the bootcamp at UCL, we tried using Mercurial (with EasyMercurial) instead of Subversion in the version control segment.

You can see the plan for the segment on this EasyMercurial project page. Briefly, we opened with a few plain slides about the purpose of version control, followed by a hands-on example in three parts (working by yourself, working by yourself with an online remote repository, and working with others). We started at the beginning and got as far as "hg bisect", but did not cover branching.

The Good and the Bad of It

Wed, 02 May 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We've come to the end of a hot, slightly sweaty, two days of learning in UCL and have just done a straw poll of participants of the good and bad points of this bootcamp. Something that was trialled at UCL was asking participants to sign up in small (3-4 people) teams drawn from their local research group. In total, we had a little over 40 learners and 10 helpers. We also taught the version control portion using EasyMercurial (thanks to Chris Cannam from SoundSoftware).

Better Across the Pond?

Mon, 30 Apr 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I'm sitting in a packed room helping out at the UCL bootcamp, the first of a series to be run in the United Kingdom (Newcastle is next, and there are plans to run additional workshops in Oxford, RAL, Glasgow, Edinburgh and Bristol). I'm thinking about two things: why would you design a 60 seater lecture theatre with only one window for ventilation and why is Software Carpentry so popular in the UK?

Stop Me If You've Heard This One

Sat, 28 Apr 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I used to tell this joke:

An engineer says, "Theory approximates reality."
A mathematician says, "Reality approximates theory."
A sociologist says, "Would you like fries with that?"

Skip forward ten years. It was the early 2000s, just after the first dot-com bubble burst, and I started noticing that all of a sudden, programmers were taking design and graphic designers seriously. Overnight, it seemed, companies had started paying designers competitively and giving them real authority. Somehow, nerds like me who had made jokes about people with "soft" skills (and boy, isn't that term revealing) had come to realize just how valuable and difficult those skills were.

Solution to Sets and Dictionaries Exercise

Thu, 26 Apr 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Last week, I posted an exercise on working with sets and dictionaries that also included a fair bit of file I/O and string manipulation. My solution is below, in four parts, along with the code produced in each. If someone would like to re-do the file parsing using regular expressions, I'd be happy to post that as well.

import sys #-------------------- def parse_pair(pair): ''' Parse an atom-count pair. If the count is missing, assume that the count value is 1. ''' if '*' not in pair: return pair, 1 atom, count = pair.split('*') count = int(count) return atom, count #-------------------- def parse_molecule(text): ''' Get a single molecule description from a text string. ''' name, formula_text = text.split(':') name = name.strip() pairs = formula_text.strip().split('.') formula = {} for p in pairs: atom, count = parse_pair(p) assert atom not in formula, \ 'Already seen atom %s in text %s' % (atom, text) formula[atom] = count return name, formula #-------------------- def read_molecules(reader): ''' Read molecules from a molecule file, returning a dictionary of {name : formula} pairs. ''' result = {} for line in reader: line = line.strip() if (not line) or line.startswith('#'): continue name, formula = parse_molecule(line) assert name not in result, \ 'Already seen %s!' % name result[name] = formula return result #-------------------- print read_molecules(sys.stdin)

Part 2

An Exercise With Sets and Dictionaries

Fri, 20 Apr 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

You are working for a nanotechnology company that prides itself on manufacturing some of the finest molecules in the world. Your job is to rewrite parts of their ordering system, which keeps track of what molecules they can actually make. Before trying this exercise, please review:

Introduction

Storage

Dictionaries

Examples

Nanotech Example

Submit your work by mailing Greg:

your final program,

the input file(s) you used to test it, and

a shell script that runs all of your tests.

1. Reading

Three Years Later

Thu, 19 Apr 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

It's not putting it too strongly to say that Software Carpentry changed my life.

That's where Software Carpentry came in. I had just about given up on learning to program, resigning myself to a lifetime of GUI-clicking and begging for help whenever more was needed. Then I got an email from a friend, forwarding some information about the course. Hallelujah! Finally, a course for beginners like me, which didn't assume any prior knowledge but also didn't talk down. The things I learned have opened countless doors for me. I can write my own code and understand code written by others, and I'm not afraid to learn new programming languages as needed. I even took a postdoc in a highly computational lab, which I would never have had the nerve to do if it weren't for Software Carpentry. Given my background and interests, I'll probably never be one of the really expert programmers in my field, but I don't need to be. I know more than enough to function independently, and more importantly, I know how to ask the experts for help, and I can understand their answers.

Behind the Scenes (or, the Ethics of Cultivating Discontent)

Wed, 18 Apr 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A lot goes on behind the scenes here at software-carpentry.org:

The site itself is WordPress with a partly-customized theme. We use the blog for topics like this and pages (over a hundred of them) for lecture topics. We used to use Trac to manage work items, but nobody kept it up-to-date; these days, we use a WordPress to-do list plugin for the same purpose, and with as little result.

Our videos are hosted on YouTube—we used to store them locally, but performance improved a lot when we offloaded.

We manage our mailing lists and version control repositories through the Dreamhost control panel, which actually delegates mailing list management to Mailman.

The calendar and map are hosted by Google.

We do event registration through EventBrite.

We currently use BlueJeans and Skype for web conferencing, but it's been plagued with both technical and social difficulties: people need to have the right Skype client for their OS, and there are the usual problems with unmuted microphones, unintelligible audio, feedback loops, and so on. Forget flying cars: I'll believe the future has arrived when we can make this work...

This analysis leaves me feeling a bit conflicted. When I think about what we should teach researchers about the web, I have three requirements:

Where Next?

Wed, 18 Apr 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Do you know someone who'd be helped by a Software Carpentry bootcamp? Or can you introduce us to someone who'd help us organize one in a new venue? If so, please introduce us: we'd like to start planning for the fall, and we're always keen to make new friends.

GitHub for Education

Tue, 17 Apr 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

In my experience, most teachers don't develop courses from scratch. Instead, they take whatever material is at hand, modify it to meet their needs, and then—well, that's usually where they stop. Unlike open source software developers, they usually don't give it back to the community in any explicit way. Instead, the next person who needs a starting point has to stumble over it in a Google search, and the original creator may never know that someone improved upon what they did.

Halfway Home

Tue, 17 Apr 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We're half-way through our current round of work, so it's time to start thinking about what we've accomplished, what we've learned, and what we'd like to do next. Here's what I think we now know:

Our training makes scientists more productive.

We can prove it.

Our methods scale.

We can become self-sustaining in 2-3 years.

In more detail:

1. Our training makes scientists more productive.

In Search of Prior Arguments

Tue, 17 Apr 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A faculty member whose research involves building some fairly complex scientific software would like to make all his work open source. He is repeatedly having to justify this choice to funding agencies and his dean, whose objections include:

concern for sensitive information being released (anything involving pollution has the potential to become a political football)

concern for misuse by naive users undermining the reputation of the tool

concern for missing value in licensing the IP

I know lots of other people have had to overcome these and other objections; what I'm looking for is a published (and preferably peer-reviewed) refutation of them that he and others can cite, preferably one that is specific to open source in science (rather than in general). Pointers would be very welcome.

Utah State University Wrap-Up

Mon, 16 Apr 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Our bootcamp at Utah State University finished earlier today—many thanks once again to Ethan White and his friends for hosting us. Here's what the students thought:

Good Bad

keeping history

break work into pieces

integrating different things

good for different levels of knowledge

liked the pace

good tech support from helpers

download data from the web

liked the escalation

using up-to-date languages

emphasis on commonality of patterns

liked flexibility

talked about formatting/style

everything consistent

Greg's stories

hearing from multiple people

organized around simple, general structural things

liked structure around data processing

machine/software setup

hard to keep up with Greg's typing

wind tunnel

no beer

assumed knowledge we didn't have

less lecture, more time on examples/practicals

rushed

wide range of applicability

wanted more complicated examples

some things glossed over

don't understand strengths/weaknesses of languages

would have liked time to make better examples

some places were "show" not "do"

didn't cover testing

Greg's stories

couldn't tell if people were with us or not

hard to keep track of what tool we were in

coordination across people

Next stop: London!

Data Munging with Regular Expressions

Sun, 15 Apr 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Indiana U's Mike Hansen has written a blog post explaining how he used regular expressions to rename and fix some MATLAB files. Even if you don't use MATLAB, you should find lots of useful stuff in it. Thanks, Mike.

We're Neutral (but Not Really)

Sat, 14 Apr 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

From Wikipedia:

Open science is the umbrella term of the movement to make scientific research, data and dissemination accessible to all levels of an inquiring society, amateur or professional. It encompasses practices such as publishing open research, campaigning for open access, encouraging scientists to practice open notebook science, and generally making it easier to publish and communicate scientific knowledge.

The term reproducible research...refers to the idea that the ultimate product of research is the paper along with the full computational environment used to produce the results in the paper such as the code, data, etc. necessary for reproduction of the results and building upon the research.

Solution to Data Merging with Dictionaries

Thu, 12 Apr 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This week's tutorial problem was to merge the data from a set of input files to show how often different species were observed on different dates. The shell pipeline, Python code, and two sample input files follow the video.
{% assign video_title="Data Merging with Dictionaries" %} {% assign video_slug="S-dqlYWs4S0" %} {% assign video_time="00:09:18" %} {% include youtube %}
shell command

grep -h -v '#' *.txt | sort | uniq -c

merge.py

Video Update

Thu, 12 Apr 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Back in February, we asked people to make short screencasts of themselves solving a simple programming problem. The submissions convinced us that it's a good idea, but so many people ran into so many problems that we're taking a step back and trying to write better instructions, select a simpler (more stable, more reliable) set of tools, etc. Some people's screen recording software stopped recording at 10 minutes without giving any signal; others produced files that play properly on some platforms, but have no audio on others; and screen resolution problems have made text unreadable in several cases. I didn't think this would be so hard in the early 21st Century, but then, I say that a lot...We'll pick this up again as soon as we can; in the meantime, our thanks (again) to everyone who has contributed so far.

Straw Man for Web Programming

Tue, 10 Apr 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Last week, I asked what we should teach researchers about the web. I think that I have an answer, and that the easiest way to describe it is by describing what we want learners to be able to build when we're done. So, imagine you are studying changes in rainfall due to climate change in North America. As part of that work, you're comparing results from your simulation with historical data from Environment Canada. Since your calculations may be useful to other scientists, you want to share them on the web. You are therefore going to build a command-line tool so that:

A Future Student

Mon, 09 Apr 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I don't know what we'll be teaching in 20 years, but I hope Tristan Davis will be there to learn.

On Crossing Australia (or, Further Thoughts on What to Teach Researchers about the Web)

Fri, 06 Apr 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A while back, I blogged about Bret Victor's "Inventing on Principle" talk at CUSEC'12, which is an inspiring vision of what programming could be. Ned Gulley's thoughts on it are more nuanced (and more interesting):

Victor's real power is his ability to rapidly create and deploy these tools. In a twinkling he can size up a task that is worth studying, put a box around it and spin a tool. He does this so effortlessly, with such mesmerizing legerdemain, that we lose sight of this meta-skill. What Victor was really doing in his talk was illustrating the power of tool spinning, the rapid creation of customized, context-sensitive, insight-generating tools...

Titus Brown Finds a Theme

Fri, 06 Apr 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Titus Brown just posted "Big Data Biology — why efficiency matters", in which he explains the academic, practical, and algorithmic reasons why efficient computation is good for science. His opinions are all grounded in his extensive personal experience working in the overlap between biology and computing, and, combined with a couple of things Michelle Levesque showed off at our Oakland workshop last week, have me thinking that we ought to include half an hour on performance analysis and tuning in the Software Carpentry core. Damn you, Titus—I thought I had this curriculum figured out :-)

Lessons Learned at the University Of Chicago

Thu, 05 Apr 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Software Carpentry brought a bootcamp to the University of Chicago with collaboration from the FLASH Center at the University of Chicago's Computational Institute and The Hacker Within . The instructors were Milad Fatenejad, Katy Huff, Anthony Scopatz, and Joshua R. Smith.

Space constraints at U. Chicago meant that only 50 empty seats could be secured for two contiguous days on campus. But, the room was lovely!

Solution to Data Checking Problem

Wed, 04 Apr 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I finally had a chance this morning to record my solution to the final exercise I set the learners from the Space Telescope Science Institute. It demonstrates how to build a little program in Python that checks the consistency of some experimental data; along the way, it uses file I/O, functions, assert statements, lists of lists, and on-the-fly unpacking. As always, feedback on the content and format would be greatly appreciated. (And if anyone would like to post their own solution, either as plain code or as a video, please let me know so that I can include it here.)

A Four-Day Curriculum

Tue, 03 Apr 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

In response to my weekend post about what we teach in two days, Steve Haddock sent me this link to a four-day course based on the excellent book Practical Computing for Biologists that he co-wrote with Casey Dunn. It's worth reading and pondering in full, but in brief, their curriculum (in half-day chunks) is:

Installing software and working with text editors.

Using a command-line shell to create pipelines.

Python.

More Python.

Scientific Python (and a bit on graphics).

More on Python graphics, bioinformatics, and generating web pages with Sphinx.

Relational databases.

Images.

The delight is in the details, of course, but the three things that jump out are (a) having half a day at the start to get things set up, (b) the domain-specific material (scientific Python, bioinformatics, making graphs, etc.), and (c) the absence of version control. What doesn't show up here, but was clear from Steve's email, is that they also spend much more time doing hands-on exercises than we do in a two-day course. If we cut out databases, so that we were only covering the shell, (very) basic Python, and version control, we could do more hands-on work, but I don't know if narrower-but-deeper is a good trade-off or not. As I said two years ago in a slightly different context, the problem is never figuring out what should be in a course; the problem is always to figure out what should be left out.

Solution to the First Image Processing Homework

Tue, 03 Apr 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We had technical issues during yesterday's online tutorial (again), so I have recorded the solution to the image processing exercise we gave the students in Indiana and Toronto. I'd be very grateful for feedback (not just from students): is this a useful way to present ideas? Or should we put our effort into debugging [name of web conferencing software goes here]?
{% assign video_title="Solution to First Image Processing Homework" %} {% assign video_slug="zwl1W4PjaMI" %} {% assign video_time="00:16:25" %} {% include youtube %}
My final code is:

Upcoming Events for Webmaking Instructors

Tue, 03 Apr 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Software Carpentry has had several homes over the years. Right now, it's part of a larger effort by the Mozilla Foundation to teach people—all kinds of people—how to make the web their own. And as part of that, Mozilla is running events over the next couple of months in Boston, San Francisco, Toronto, and London to bring together people who are trying to teach this stuff. The details are on Michelle Levesque's blog; we hope you'll be able to join us.

Sending Email Back in Time

Sun, 01 Apr 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We're about to release the second volume of The Architecture of Open Source Applications, which has indirectly prompted a bit of soul-searching on my part. When we invited people to contribute, we asked them to give us the one-hour whiteboard talk they'd give to a new developer being brought onto the project. We also asked them to sum up what they'd learned. "Imagine you could send a brief email back in time," we told them. "What would you say to your younger self?" Which of course begs the question, what would I tell the me of 2010 about Software Carpentry?

What to Teach Researchers About the Web

Sun, 01 Apr 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

One reason I'm reflecting on what I've learned in the last two years is a question that is back on the top of my work pile: what should we teach researchers about the web? Partly, it's a priority because I'm currently embedded in Mozilla; their mandate is to defend and extend the open web, and their educational efforts are all aimed at that, so I ought to be doing something too. The real reason, though, is that a lot of things have brought this into sharper focus recently:

Maintaining Momentum

Fri, 30 Mar 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

For a variety of reasons (which is my way of saying "I don't know why" :-), Software Carpentry has proven really popular in the UK. We have close to 200 people signed up for 40 seats at our London workshop, and over 160 for 40 seats at Newcastle, with more coming in all the time. We also now have groups from three other universities, and a couple along disciplinary lines, asking if we can run workshops for them. However, the funding that has supported this round of work will run out at the end of June, and the earliest we could receive more support from the same source is mid-October. Our challenge now is therefore to find ways of supporting them in the short term so that momentum isn't lost. Here are some ideas; we'd welcome more.

What We Teach in Two Days

Fri, 30 Mar 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This week's workshops at MBARI and NERSC both had more lecturing and less hands-on practical work than either I or the students would have liked, but when we're trying to squeeze so many things into two days, that's probably unavoidable. We hope that the online tutorials we're going to run over the next few weeks will make up for that by giving learners a chance to practice their skills at leisure.

Wrapping Up in Oakland

Fri, 30 Mar 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We've wrapped up the workshop in Oakland for folks from NERSC, Berkeley, and Stanford. More later (when I'm home and have slept), but here's the students' feedback. Many thanks to Shreyas Cholia for organizing the workshop, to Michelle Levesque for helping to teach it, and to Elango Cheran and Jorge Aranda for helping out.

Good Bad

software strategy

Python intro

Subversion good!

anecdotes (no, really)

speed of typing easy to follow

connected whole course to pipeline model

liked emphasis on programming hygiene

liked worked examples

incorporated pedagogy

agnostic about languages

programming philosophy

live coding (not PowerPoint)

liked learning about SVN

learned that some tools exist

humor

online resources

similar environments

engagement on first day

whole atmosphere of class

relating dev to science

having people around

learning terminology/theory

comprehensive set of tools

big picture view

four topics picked were good

free!

room was too warm

Greg didn't say SVN has branching

need more use cases from audience

too short

Michelle types too fast

first three hours slow

first three hours too fast

Python: too much basic software discussion

needed more time to talk to neighbors

mention the next step

more depth

enough rope to hang themselves

too few interactive examples

room was too small, too

more MATLAB

advanced students could come later

more examples to work on our own

engagement on second day

more instruction about setup/install

hands-on with HDF5 etc.

too fast

too diverse levels

already knew lots of this

didn't talk about Silicon Valley

wanted a before+after test

healthier snacks

Bootcamp in Paris June 28-29, 2012

Wed, 28 Mar 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Nelle Varoquaux and others are organizing a Software Carpentry bootcamp at INRIA Paris, which will run June 28-29, 2012. We'll post more details as we have them, and look forward to seeing you there.

Wrapping Up MBARI Workshop

Wed, 28 Mar 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Steve "Jellyfish" Haddock and Greg Wilson taught a two-day workshop at MBARI this week. It seemed to go well: feedback is below.

Good Bad

used specific, concrete examples

hands on

good breadth

knowing the tools exist

seeing the Power of Pipes

seeing how to use version control for traceability

insistence that prior experience wasn't necessary

the Power of Python (!!!)

liked the Power of Databases

saw background of databases

now understand how much more efficient is possible

stories are entertaining

starting from scratch helpful

databases may not be completely useless

liked having things online after day 1

great motivational speaker

emphasis on commonality/interoperability

liked provenance

web site

linked coding/learning

introduced things to use *with* Python

content was fairly conceptual

clean code from the beginning

drawing parallels between Python and MATLAB

including version numbers (provenance)

forced to learn my Mac

being allowed to talk to neighbors

"programming is meaningless" is good

prepared pedagogy, not material

I feel like I'm a better programmer

great going over Unix stuff

some things were too fast (esp. day 2)

instructor wanders

more practice time

instructor types too fast

wanted two screens at once

switching between Python and shell confusing

more time on switching between screens

follow instructor or take notes but not both

handout please!

not enough hands-on practice

not knowing how to get data into database

can't keep track with multiple Python files

didn't explain regular expressions

lack of hands-on

more data manipulation

not enough time

shorter, more frequent breaks

download folder of examples

can't do it all in two days

confused with version control

accelerated toward the end

don't know how to use database in research

advance materials

show up early for setup

say "meaningless" first

sys.argv etc. not covered

Object-Oriented Programming in Fortran 2003

Fri, 23 Mar 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Damian Rouson is teaching a class in Berkeley March 26-28, 2012, and again April 10-12, on object-oriented programming in Fortran 2003. The March class is full, but there are still seats available for April. Fortran is still the workhorse of scientific computing, and recent dialects like 2003 have a lot of powerful features—it's well worth checking out.

And While We're Stuck Here With 21 Seconds Worth of Music to Fill...

Sun, 18 Mar 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Scholarship in the Age of the Internetatron:
{% assign video_title="Scholarship in the Age of the Internetatron" %} {% assign video_slug="8omwsw8ibzw" %} {% assign video_time="00:02:42" %} {% include youtube %}

The Dark Matter of Computational Science

Sun, 18 Mar 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Scott Hanselman's recent post "Dark Matter Developers" got me thinking once again about what Software Carpentry is about. He says:

[We] hypothesize that there is another kind of developer than the ones we meet all the time. We call them Dark Matter Developers. They don't read a lot of blogs, they never write blogs, they don't go to user groups, they don't tweet or facebook, and you don't often see them at large conferences... Where are [they]? Probably getting work done.

Thank You, Enthought

Fri, 16 Mar 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are pleased to announce that Enthought has generously given us a grant to support some of the travel costs associated with our bootcamps. Along with hosting an earlier version of our material for several years and preparing a rock-solid Python distribution for scientific computing, this is one more good deed we'd like to thank them for.

Enthought, Inc is delighted to provide funding to Software Carpentry to support educating scientists in basic computing skills. Enthought was founded with a mission to improve the way scientific computing is accomplished, which we do through our support of open source initiatives, applications we build for customers, and through a goal we share with Software Carpentry: teaching scientists computing skills that improve their productivity and advances their contributions to science.

Wrapping Up the STScI Course

Fri, 16 Mar 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The online portion of our work with learners at the Space Telescope Science Institute wound up today. 6 of the 14 people who took part in the on-site workshop submitted "graduation exercise" videos, and three more sent apologies (time pressure, technical difficulties, etc.), which I think is a pretty good completion rate. As always, we finished up by asking everyone to give us one good and one bad thing about the class:

First Homework for Indiana Students (and a few from Ontario)

Thu, 15 Mar 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We ran our first online tutorial yesterday for students from our recent bootcamp in Indiana (plus a few from Toronto). Despite some technical difficulties with web conferencing, we managed to get through some basic image processing. The learners all now have homework for next week (shown below); considering that most of them have little prior programming experience, and only a half day of Python, it's pretty cool that they're able to do things like this.

Ask the CompuScienceGeek?

Wed, 14 Mar 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Titus Brown recently blogged a question about how to organize the files used in a computational science pipeline: by project, by paper, separately, or something else. Rather than answering that, I fastened onto a throw-away comment:

I think we need an "ask the compusciencegeek" service...

My earlier idea of having a reserved area on Stack Overflow for novice questions didn't lead anywhere, but I've enjoyed the Software Sustainability Institute's "Ask Steve" column. Should we start something similar here? Or direct people at Steve, and syndicate his answers here? What questions do you have about using computers in your research that we could answer?

How We're Doing

Wed, 14 Mar 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The first online session for our Indiana University students will run tomorrow afternoon (assuming we can get web conferencing to work—why is this still so hard in 2012?). Meanwhile, our other workshops are filling up quickly:

Site Start Ticketed Space

Monterey Bay Aquarium Research Institute March 26 17 40

Lawrence Berkeley National Laboratory March 28 40 40

University of Chicago April 2 116 150

Utah State University April 14 26 50

University College London April 30 49 50

Michigan State University May 7 7 40

Newcastle University May 14 9 40

University of British Columbia May 22 40 40

and we are now discussing more workshops at other sites: Purdue, Oklahoma, Paris, Melbourne, and possibly ones in South America and Africa as well. If you'd like us to help organize one where you are, please get in touch.

Where Next for the Next-Gen Course (and Software Carpentry)?

Wed, 14 Mar 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Titus Brown's next-generation sequencing course has been a great success: so great, in fact, that he's overwhelmed with applications for this year's run. That has made him think aloud (or rather, ablog) about where to take the course next. His options include:

Maintain the status quo (but the effort/reward ratio for him isn't sustainable).

Run it for pay (which would put it out of reach for most learners).

Flip the course, using videos for training and live sessions for tutorials (see below for discussion).

Franchise, i.e., train other people to deliver the course wherever they are (see below as well).

Merge with Software Carpentry.

I'm all in favor of the last option: I would really like to see Software Carpentry become a hub through which other people offer, find, and/or run "software for scientists" training (both live and online). In practice, this would mean:

The IPython Notebook

Mon, 12 Mar 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Titus Brown has created a short video showing what the IPython Notebook can do. The short answer is, a lot: it's an interactive notes-plus-graphics-plus-live-coding tool that runs in the browser. (The video doesn't really get going 'til the 2:50 mark, so you may want to skip ahead to then.) The question isn't whether we want to using this in Software Carpentry, but when. Installing it is (in my experience) still painful, but it holds tremendous potential. If we can get to the point where 90% of our intended audience can "just run it", and the other 10% need only a couple of minutes' help, we'll begin the switch.

The Trieste Workshop, One Week Later

Mon, 12 Mar 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Katy Huff and I are back from Trieste, Italy, where we were instructors in the Advanced School for Scientific Computer at the ICTP. This was a different sort of workshop in many ways. First of all, it was 2 weeks long and the students were from all over the globe. Countries represented include Russia, Bangladesh, China, India, Pakistan, Albania, Iran, Palestine, Ghana, Nigeria, Chad, South Africa, Serbia, Romania, Ukraine, Argentina, Ecuador, and Colombia. The level of effort they showed was astounding: between our exercises and the project they brought from home, many students were in the computer labs until 11:00 at night.

Our Indiana U Workshop Went Well

Fri, 09 Mar 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Our two-day bootcamp at Indiana U. on March 7 and 8 went well—while less than half of the people who registered actually showed up (more on that in a moment), everyone who did attend the two days seemed to get a lot out of it. Special thanks are due to Prof. Andy Lumsdaine (host), Jennifer Richards (organizer), and Ben Martin, Randy Heiland, Joe Cottam, Mike Hansen, DongInn Kim, and Purdue's own Jeff Shelton for helping out.

What's the Model, Kenneth?

Fri, 09 Mar 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Over on rwxweb [1], Michelle Levesque has posted and dissected a diagram showing how various webbish skills depend on one another. It's an invaluable design aid, but it violates the third of Jon Udell's seven principles: it's presented as a bitmapped image, rather than as something a machine can easily parse and digest. I can't, for example, read the source and generate a two-column table showing "X depends on Y", or something like that. I wrote a bit back in January about what sort of data model we could use to represent things like that, but that's the wrong approach as well. What I really want is to discover that someone has already thought this through and created such a model, so that if I want to alter or extend Michelle's model, I can change the authoritative machine-readable representation [2], and she can merge (or at least diff) those changes. All I've found so far, though, are bits of database schema buried in things like Moodle and SCORM; there doesn't seem to be anything that a programmer would recognize as a model or format per se.

I Resemble That Remark

Wed, 07 Mar 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Titus Brown's recent post "Top 12 Reasons You Know You are a Big Data Biologist" may resonate with some readers of this blog. I particularly like the idea (in the comments) of using links in email to drive garbage collection of data sets.

Software Carpentry Meetup at PyCon

Wed, 07 Mar 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I'm not able to be at PyCon this year, but if Software Carpentry fans would like to get together, Shreyas Cholia is organizing a lunchtime meetup on Friday. Please drop him a line if you'd like to connect.

Happy People

Mon, 05 Mar 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Our workshop at the International Center for Theoretical Physics in Trieste has ended, and judging from the participants' smiles, it went well:

Thanks once again to Tommy Guy and Katy Huff for teaching, to Stefano Cozzini for inviting them, and to everyone who took part for all their hard work.

Help Us Write Assessment Questions

Mon, 05 Mar 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We have been asking people the questions below before the start of a bootcamp in order to get a handle on how much they already know. As part of evaluating the current round of work, we'd like to expand the list a little, and touch on a few more topics. Our rough categorization of how much someone should know about various things is in the competence matrix; ideally, we want another 3-4 questions on each topic. Our criteria are:

Open Education Week

Mon, 05 Mar 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Today marks the start of Open Education Week. To celebrate, please post a link here to your favorite online educational resource.

Programs as Experimental Apparatus

Mon, 05 Mar 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Suppose you have two photographs of a patch of the night sky:

and you want to know how different they are. The simplest way would be to see if any of the pixels' values differ, but that's pretty much guaranteed to return a "yes". A better measure is to see how many pixels differ by more than some threshold, but that raises two questions:

Performance Curves, Curriculum Design, and Trust

Sun, 04 Mar 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Suppose you have a processing pipeline with three stages:

Each stage takes one second to run; what's its overall performance? As Roger Hockney pointed out in the 1980s, that question isn't well formed. What we really need to ask is, how does its performance change as a function of the size of its input? It takes 3 seconds to process one piece of data, 4 to process two, 5 to process three, and so on. Inverting those numbers, its rate is 1/3 result per second for one piece of data, 2/4 = 1/2 result/sec for two, 3/5 for 3, etc. If we draw this curve, we get:

Inscight from Trieste

Thu, 01 Mar 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

In the latest Inscight podcast, the inimitable Katy Huff talks to several of the participants in the bootcamp she and Tommy Guy are leading at the International Centre for Theoretical Physics. It's great to hear so many people from all over the world learning so much.

Toronto Bootcamp February 2012: How We Did

Thu, 01 Mar 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Last week's bootcamp at the University of Toronto was not the most successful one we've ever done: quite a few registrants didn't show up, and based on feedback from the instructors, we tried to cover too much, too fast. That said, the learners who stuck with us all the way through to Friday afternoon came away knowing a lot of useful things (or at least knowing that those things existed). We'll gear back a bit for next week's workshop at Indiana U. and see how that goes.

ULP (or, This is tricky and perhaps profound)

Thu, 01 Mar 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Bruce Dawson's "Comparing Floating Point Numbers (2012 edition)" is, as he says of its subject, tricky, and perhaps profound—worthwhile reading for anyone pushing numbers around.

Worth Reading, Worth Watching

Wed, 29 Feb 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The PLoS Computational Biology article "Ten Simple Rules for Getting Help from Online Scientific Communities" has a lot of good advice.

Mark Guzdial's talk "Helping Everyone Create with Computing" is full of good ideas—see in particular the discussion of computational scientists starting around 06:15.

Reproducibility Redux

Tue, 28 Feb 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A recent editorial in Nature, and a longer article by Darrell Ince, Les Hatton, and John Graham-Cumming titled "The case for open computer programs", are just two signs of the growing pressure to raise standards around computational work in science. We can debate the exact meaning and value of reproducibility, but there's no arguing with the fact that if scientists want to do better work, they'll need better skills. That's the real long-term goal of Software Carpentry.

Badges (Finalized)

Mon, 27 Feb 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We have finalized our first set of Software Carpentry badges—with luck, we'll list the first set of recipients later this week.

for people who complete the core curriculum

for helping to teach (either workshops or online)

for organizing workshops

for creating (or fixing) content

Frustration (continued)

Mon, 27 Feb 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

It's been a frustrating couple of days. To recap, I want to convert our material from PowerPoint to HTML5 to make it easier for people to fork and merge, to make things easier to re-style, because it's an open format, and so on. David Seifried has welded an HTML5 audio player to Caleb Troughton's deck.js to create a display tool, which I'm very pleased with, but the content is killing me. Seriously. Here are three slides taken from our first episode on the Unix shell:

Fourth (or Sixth) Online Tutorial

Fri, 24 Feb 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

For the past four weeks, I've been meeting online with learners from the Space Telescope Science Institute to work through some Python topics and exercises. (We split the group in two for a couple of those sessions to accommodate different interests and levels.) I think it's been going pretty well: about half the people who took part in the two-day workshop in January are still with us, and we've just covered basic image processing with PIL. I'm looking forward to finding out whether the model scales to more groups with more people in the coming four months.

Trieste, Italy Workshop - Week 1

Fri, 24 Feb 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Last Sunday, Tommy Guy and Katy Huff flew to Trieste, a small city in northeastern Italy, to assist in teaching an Advanced School on Scientific Software Development at the International Center for Theoretical Physics (ICTP).

Stefano Cozzini, Graziano Giuliani, and Antun Balaz invited us to help them teach this two week workshop which is one in a series of workshops focused on high performance computing for scientific applications. The ICTP mission is to extend access to advanced scientific tools and education to scientists from developing countries. Undergraduate students, graduate students, and post-graduate scientists gathered in Italy from nations all over South America, Asia, the Middle East, and Africa.

Should We Relocate Our Repository?

Thu, 23 Feb 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Software Carpentry has been an open project since 2004: MIT License for the code, Creative Commons for everything else. All our stuff has lived in a public Subversion repository since then as well—it's at http://svn.software-carpentry.org/swc if you want to check it out. Today's question is, should we move that repo off our server and onto GitHub, BitBucket, or some other repo-hosting service? We want to make it easy for people to remix our content; as I said back in December, that means making it easy for them to fork, merge, and share. Would you be more likely to do this if our slides, learning plans, diagrams, and code samples were on some well-known hosting site rather than in Subversion on our own machine?

Granules of Research

Wed, 22 Feb 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Cameron Neylon recently posted an article titled "Github for science? Shouldn't we perhaps build TCP/IP first?" His argument is that the web's a good way to move text around, because it was built by programmers, and programmers work with text. It's not (yet) well suited to moving science around, because we don't (yet) have something as granular and portable as text for scientific ideas. Yes, any particular piece of research can be represented as text, but so can any image, or any audio stream, or anything else—it's the structure that adds meaning, and we haven't (yet) agreed on structures. Circling back to today's first post, part of what we're trying to do is give scientists the background to understand and take part in conversations like these...

Watch Me: Trial Run

Wed, 22 Feb 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A dozen people have come forward since I asked last week for volunteers to make short screencasts showing how they program. I just sent them a sample problem to work on to test things out (see below the fold); the videos they create won't be made public, but I hope it gives readers an idea of the scale of problems we're going to be looking at. If you have suggestions for interesting problems of a similar size, please add them as comments on this post.

What Deep Thoughts Look Like

Wed, 22 Feb 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Before writing yesterday's post about assessment, I should have explained what I mean by"fundamental concepts". I'll start with Lewis Epstein's wonderful book Thinking Physics:

Here's a typical problem from the book. Put a block of ice in a bathtub, then fill the bathtub to the brim with water, so that the block is floating freely. When the ice melts, will the water level go up (causing a spill), go down, or stay the same? Hm... well, the ice displaces its own weight of water, so when it melts, it exactly fills the "hole" it made, so the water level stays the same.

Assessment Redux

Tue, 21 Feb 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The single biggest challenge Software Carpentry faces right now is how to tell what impact it's having. This is only partly to satisfy funders—as I said back in December, if we don't know how to tell if we succeeded, we're going to fail. It would be (relatively) easy to put together a multiple-choice quiz to see how much people have learned about basic shell commands, the syntax of Python, and so on, but that would only address the shallowest aspects of learning. We're trying to impart some fundamental principles, and what we need is questions that will tell us whether people have internalized them. (As many studies have shown, it's possible to get a decent score on a quiz without actually understanding the subject matter.)

Badges (Mark 1)

Tue, 21 Feb 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

One of our key deliverables for the Sloan Foundation-funded work is a badging program built on top of Mozilla's Open Badges Initiative. Riffing on our new logo, Carri Han has designed three badges for us:

for people who have mastered our core content

for people who have organized and run workshops

for people who have created content

Please let us know what you think.

Hello from Trieste!

Tue, 21 Feb 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Day 1 of the Trieste bootcamp was a success! Katy and I covered the Bash shell and git. It was encouraging to see students in the lab after dinner working on their shell exercises. In general, the students are very enthusiastic. Later we'll try to list their home countries. So far, I've met people from Colombia, Mexico, Argentina, South Africa, Cameroon, Nigeria, Romania, Italy, Pakistan, Iran, India, and China. Their fields range from astronomy to nuclear physics to climate science.

Why *Not* Use Python

Tue, 21 Feb 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

When we started Software Carpentry back in the late 1990s, we used Perl as a teaching language instead of Python. At the time, it was a no-brainer: Perl had many more users, better documentation, and more libraries. We switched because we found ourselves explaining the same inconsistencies over and over again (as I've said many times since, every page of the O'Reilly Pocket Guide to Perl used one of the words "except", "unless", or "however" at least once). Python had fewer "buts": we saw right away that students were learning concepts more quickly, and they seemed to retain more as well.

A Flash (well, MP4) from the Past

Sun, 19 Feb 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

In July 2009, we held a one-day symposium on open science at the University of Toronto. I recently uploaded video from those talks to YouTube; the audio is a bit shaky, but I hope they're useful despite that. The talks are linked below.

Titus Brown: Choosing Infrastructure and Testing Tools for Scientific Software Projects

Cameron Neylon: A Web Native Research Record: Applying the Best of the Web to the Lab Notebook

How They See Us, Part N

Thu, 16 Feb 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This week's Ed-Tech Podcast from Steve Hargadon and Audrey Watters discusses Software Carpentry a bit around the 23:00 mark [1]. In answer to Hargadon's point about home schooling, and whether the way people learning programming even fits the notion of class, we have a couple of answers. First, most of the people we're trying to help don't know enough (yet) to know what to type into Google, how to recognize when they've stumbled upon an answer to their problem, or what to tag a question with on Stack Overflow. Some can climb that hill themselves; a handful can't, but most won't (see below), so one of our goals is to help them get from A to B so that they can get from B to Z.

Analyzing Next-Generation Sequencing Data

Wed, 15 Feb 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Analyzing Next-Generation Sequencing Data

http://bioinformatics.msu.edu/ngs-summer-course-2012

June 4th — June 15th, 2012
Kellogg Biological Station, MSU
Course sponsor: NIH.

Instructors: Dr. C. Titus Brown, Dr. Ian Dworkin, and Dr. Istvan Albert.

Board of advisors: Dr. Kevin White; Dr. Paul Sternberg; Dr. Rich Lenski; Dr. Robin Buell; Dr. Jim Tiedje; Dr. Lincoln Stein

Applications are being accepted through March 1st (midnight Pacific)!

And Speaking of New...

Wed, 15 Feb 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

...check out Bret Victor's talk at CUSEC 2012—jump in around the 7:00 mark and watch for a couple of minutes. You'll want to go back and watch the whole thing...

Slide Drive

Wed, 15 Feb 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Speaking of new kinds of content (which I've been doing a lot), David Seifreid has built a working prototype of a new slideshow tool that combines deck.js with an HTML5 audio player. You can check out a demo or grab the source from https://github.com/dseif/slide-drive. Slides are pure HTML like this:

<section popcorn-slideshow="24"> <h2>Solution</h2> <p>Short intensive workshops</p> <div> Our solution combines short, intensive workshops... </div> <div popcorn-slideshow="27"> <p>Plus self-paced online instruction</p> <div> ...with self-paced online instruction. </div> </div> </section>

which combines slide pages and transcripts in a single file suitable for diffing and merging. (Images are still in external files, but I can live with that.) You can pause the slideshow at any point to select and copy the content (something you definitely can't do with a video), and we'll add support for translations into other languages and so on.

Watch Me: Volunteers Wanted

Wed, 15 Feb 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Back in 2007, Jon Udell observed that screencasts facilitate accidental knowledge transfer in a way that more traditional media don't. As I said yesterday, we'd therefore like to start recording short screencasts of programmers thinking aloud as they solve small problems using their preferred tools. The aim is to show learners how to program—what order to write things in, how to debug, when and how much to test, and so on. Everything will be covered by the same Creative Commons license as our other material, and made freely available for remixing and other use.

New Kinds of Content

Tue, 14 Feb 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Mark Guzdial, whose blog on CS education is always interesting, recently posted about using worksheets to help people learn to write programs. As he says, research going back 30 years shows that reading and writing skills develop independently; there's also a ton of research showing that partially-worked examples are a very effective (possibly the most effective) way to teach people new skills.

Which immediately suggests two questions:

Stack Underflow?

Tue, 14 Feb 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Circling back to an earlier post, one of the challenges free-range learners face is where to get help after their workshop or short course is over. Google isn't much help, since they don't yet know enough to know what to look for, and Q&A sites like Stack Overflow can be pretty intimidating: most novices don't want to bother people and/or don't want to look clueless in public (and getting "That's a stupid thing to ask, newbie" doesn't help).

Advertising Flyer

Mon, 13 Feb 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I've put together a one-page flyer to advertise the course (each site would customize the "Local info" space at the bottom). Please let us know what you think...

Formatting Revisited

Mon, 13 Feb 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

David Seifreid has been working for the past week to combine deck.js, a Javascript-plus-CSS slideshow package, with the AudioJS audio player, so that we can re-do slides as pure HTML5 (instead of using PNGs exported from PowerPoint). At the same time, I'm trying to turn the course notes into long-form prose (what used to be called a "book") for people who prefer reading at leisure. How should all this content be managed? My previous post on 21st Century teaching formats described what I'd eventually like, but the tool I want doesn't exist yet, so what can be done now?

How Many Legs Does Science Have?

Mon, 13 Feb 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Back in 2010, Moshe Vardi wrote an opinion piece titled "Science Has Only Two Legs", in which he argued that computational science is just another form of experimental science, and that programs should be held to the same standards as other pieces of experimental apparatus. It's an interesting counterpoint to recent excitement around "big data" (as evidenced in works like The Fourth Paradigm), and, I think, directly relevant to what we're trying to teach. What are your views?

Our New Look

Mon, 13 Feb 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We have a new logo:

which is also available in stacked form for coffee mugs (thanks, Steve):

We also have some merchandise, and the first batch of laptop stickers is on its way :-)

Pre-Workshop Questionnaire

Sun, 12 Feb 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I've been asking people to fill in the short questionnaire below before our workshops in order to give us a better idea of what they want and what they already know. How do you score, and how could we improve the questions?

Roughly what fraction of your working time do you spend creating, modifying, or testing software?

What operating system will you be using in this class?

Linux

Mac OS X

Windows

Do you understand the following Unix shell commands well enough to explain them to someone else?

ls data/*.txt

find ~ -name '*.py' -print

Do you understand the following Subversion commands well enough to explain them to someone else?

svn update

svn diff -r 1723

Do you understand the following Python statements well enough to explain them to someone else?

{'east' : 5, 'west' : 11}

__init__(self, *args)

Do you understand the following testing concepts well enough to explain them to someone else?

fixture

mock object

Do you understand the following Make terms and commands well enough to explain them to someone else?

dependency

cp $< $@

Do you understand the following SQL terms and commands well enough to explain them to someone else?

select * from data where data.left < data.right;

inner join

What do you hope to gain from this workshop? What could we do to make it most useful for you?

Advanced Scientific Programming in Python

Fri, 10 Feb 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Advanced Scientific Programming in Python

A Summer School by the G-Node and the Institute of Experimental and Applied Physics, Christian-Albrechts-Universität zu Kiel

Scientists spend more and more time writing, maintaining, and debugging software. While techniques for doing this efficiently have evolved, only few scientists actually use them. As a result, instead of doing their research, they spend far too much time writing deficient code and reinventing the wheel. In this course we will present a selection of advanced programming techniques, incorporating theoretical lectures and practical exercises tailored to the needs of a programming scientist. New skills will be tested in a real programming project: we will team up to develop an entertaining scientific computer game.

Audrey Watters on Software Carpentry

Fri, 10 Feb 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Audrey Watters is a prolific, insightful writer on all things related to technology and education. I recently asked her to take a look at this course, and tell us what sorts of things might be worth trying. Her first post on the subject asks some questions about educating end-user programmers in general; your thoughts would be welcome too.

Comparing Software Carpentry to CS Principles

Thu, 09 Feb 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A lot of new educational initiatives in computing have sprung up in the past couple of years, ranging from Mozilla's Hackasaurus to the UK rethinking its grade-school curriculum. One of the biggest is the "Computer Science: Principles" project in the United States, which, with backing from the National Science Foundation, is trying to re-define the high school Advanced Placement course. They have some well-grounded ideas about what should be taught and why; how does Software Carpentry compare?

Multiple Pitches

Thu, 09 Feb 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Based on the feedback we've had on my first attempt [no longer online] at "Software Carpentry in 90 Seconds", it seems as though we should actually create three videos:

one aimed at potential participants, i.e., students, workshop organizers, and content creators (since we're hoping these will largely be the same people);

one for potential sponsors, who are both funding bodies and the senior scientists who set strategic direction for departments and disciplines; and

some "talking head" testimonials from former participants.

Does this sound like the right breakdown?

Why We Don't Teach Parallel Computing in Software Carpentry

Tue, 07 Feb 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Konrad Hinsen recently wrote a blog post that explains why teaching parallel computing with Python is hard. To make a long story short, Python's multiprocessing module can fail on simple problems in a whole bunch of ways that require fairly advanced understanding to diagnose and repair—and that's even before you try doing things on Windows.

First Online Tutorial

Fri, 03 Feb 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Our first online tutorial with the folks at the Space Telescope Science Institute via Skype, and I think it worked well. Our setup was:

The students got together in a meeting room. Each student brought their own laptop.

One extra laptop was connected to the projector; its webcam was pointed at the room, so that I could see the students, and its microphone (mostly) picked up their voices.

I shared my desktop, so that instead of seeing me, the students could see what I was viewing and typing.

I (mostly) used a full-screen terminal window, white on black, with an 18-point font, switching back and forth between my editor and running my evolving program on the command line.

That was pretty much it, and as I said, I think it worked as well as live coding in the classroom works as a lecturing technique (which is pretty well). There were a few times when I wanted to see what was on their screens, and going forward, we're going to have to find a way to do that. Overall, though, I think that using Skype for connecting, and native desktop tools for everything else, works better for small groups than things like Elluminate (now part of Blackboard) that try to do it all in one. I know it won't scale to dozens of people, but this will certainly get us through the next six months. If anyone has tips to share, they'd be very welcome.

We're Going to Be Busy

Fri, 03 Feb 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Here are the bootcamps we have lined up for the next few months:

International Center for Theoretical Physics, Trieste, February 18 — March 2

University of Toronto, February 23-24

Indiana University, March 7-8

Monterey Bay Aquarium Research Institute, March 26-27

Lawrence Berkeley National Laboratory, March 28-29

University of Chicago, April 2-3

Utah State University, April 14-15

University College London, April 30 — May 1

Michigan State University, May 7-9

Newcastle University, May 14-15

University of Alberta, May 16-17

University of British Columbia, May 22-23

We could use help with all of them, both in person as they're running, and online as participants follow up with self-paced material. If you'd like to help out, please get in touch.

Software Carpentry in a Minute and a Half

Thu, 02 Feb 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I've recorded a first draft of the quick introduction I mentioned yesterday. Feedback would be very welcome.

Where To Host Q+A and Discussion?

Thu, 02 Feb 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

People have questions and want answers, or ideas and complaints they want to share. Right now, the only ways for them to do this on our site are:

Mail us.

Add a comment to a page or blog post.

Um... that's it.

We experimented with forums last year, but they never reached critical mass (and have since filled up with spam). What should we do going forward? Should we try to resurrect those forums? Set up a Q&A mailing list? Or direct everyone to the computational science area at StackExchange? The pros and cons as I see it are:

Re-doing the Three-Minute Pitch

Wed, 01 Feb 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

It's time to revise Software Carpentry's three-minute pitch. Here's what I think I need to say; as always, comments would be welcome.

Opening slide: large logo, the title "Computing Skills for Scientists and Engineers", and a small block at the bottom with the date and license.

Our mission is

to help scientists and engineers be more productive...

...by teaching them basic computing skills

The problem is

Reorganizing This Web Site

Tue, 31 Jan 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

It's time to reorganize this web site. Here's my plan; comments would be welcome. In particular, WordPress might not be the right tool to use going forward, but I'm not sure what else would be as easy to set up and maintain.

Overall design: the logo and "Software Carpentry" always appear at the top, along with a Javascript site menu; pages are fixed width (and all of that width is taken up with content—no sidebars); the footer contains a copyright notice and links to half a dozen pages (About, Contact, License, etc.). These footer links are redundant, since those links are in the Javascript menus as well, but this makes them more visible.

Learners and Their Needs

Sun, 29 Jan 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I originally wrote these descriptions as part of a post on formats for learning material. I'm finding them useful in other contexts as well, so I'm re-posting them separately. Our description of our audience describes four scientific users in more detail.

Zuzel likes textbooks. More specifically, she likes prose that she can read and re-read at leisure. She'll use a tablet, but would rather listen to music while she learns than to someone lecturing.

Yeleina prefers interactive learning. She wants to see things evolve on the whiteboard or on the screen; recordings of live coding sessions with voiceover are OK, but slide after slide of bullet points puts her to sleep. She also wants to be able to share ideas about learning content with her peers.

Xanthe is a surfer, not a diver—she wants to skim the pages that Giggle searches turn up and piece things together herself. Bullet points and brief sentences work well for her, particularly in areas she's already familiar with.

Wafiya teaches programming at a city library. She needs to remix content created by other educators to meet her learners' needs, but doesn't have a lot of time to do so. She also needs to be able to find content that fits into her (individualized and group) learning plans.

Veronique is a programmer who is passionate about teaching. She spends several hours a week writing short tutorials, answering questions in Stuck Underflow and other online forums, and occasionally recording screencasts. She'd like to do more with less effort (she finds today's tools frustrating), to make the content she's creating more useful, and to get more feedback from its users.

Ursula's preferences are more constrained than Zuzel's, Yeleina's, or Xanthe's because she is visually impaired. Her main assistive aid, a screen reader, can only "see" text (captions in PNG or JPEG images don't count), and becomes confused when pages are modified in place by Javascript.

Tahura is an assistant professor at a medium-sized university. She has to teach three undergraduate courses each year, one of which is always "Introduction to Scientific Programming". While she'd like to experiment with new teaching methods, her priority right now is to get tenure, so she (reluctantly) sticks to the university's traditional format: three hour-long lectures a week, one homework exercise every three weeks, a midterm, and a final exam.

Terminology

Sun, 29 Jan 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Before going further with the redesign of the Software Carpentry curriculum, I need to define a few terms and their relationships. These definitions refer to another post on learners and their needs, which you may want to read first.

A concept is an atom of learning. It can be a fact (e.g., what a call stack is), a technique (e.g., how to pass parameters to a function), or a rule (e.g., when to copy data).

An association is any connection between two concepts. Obviously, all concepts in a topic should be strongly associated.

A dependency is a prerequisite association between two concepts. Circular dependencies may exist (you can't understand A unless you understand B, which depends on C, which in turn depends on A), but we'll work hard to avoid such cycles, since every particular learner encounters concepts in some sequential order.

A topic is a set of related concepts; for example, the topics "Parameters" includes all of the concepts described above. A concept may be part of several topics, but for learning purposes, we would like its first appearance for an individual learner to be gentler than subsequent appearances for that same learner.

A lesson is an atom of teaching. It will typically comprise a small number (1-4) of closed-related concepts, and be a few minutes long. Where possible, it should be followed immediately by some kind of reinforcement (discussed below). A concept may appear in several lessons (same idea, different contexts).

A tutorial is a specific sequence of lessons on related topics; for example, the tutorial "Functions" would include lessons on parameters, returning values, and so on. Tutorials aren't strictly necessary for free-range learners—they can go through lessons in any order that respects dependencies—but we must define them:

to give learners like Zuzel the larger narrative arcs they need (learners like Xanthe only need lessons); and

to give instructors like Tahura, who are working in traditional classroom settings, guidance on how to fill an hour-long lecture.

Reinforcement is something done to help people absorb a concept, such as further examples, class discussion, quizzes, or exercises. All four are problematic:

Further examples are based on guesses of what learners might not have understood correctly; such guesses are often wrong.

Class discussion is hard to implement for asynchronous, self-directed learners.

Multiple choice quizzes can mislead or frustrate learners (most questions have more than one right answer), or fail to clear up learners' misconceptions ("You're wrong, but I won't tell you why").

Long-answer quizzes and exercises require a human assessor's attention (which is also hard to implement for asynchronous learners).

Assessmentis anything done to determine how well something else has been done. "How well" is important: assessment will not always (or even usually) be black-and-white. Specific kinds are:

formative assessment, which provides short-term diagnostic feedback to learners and teachers;

summative assessment, which demonstrates attainment of some level; and

evaluative assessment, which tell us how well the teaching is working.

Formative assessment is a kind of reinforcement; summative and evaluative are (usually) not.

The diagram below shows these relationships. I've drawn most relationships as many-to-many, but in practice, we hope that most will actually be one-to-many, i.e., that concepts will fit neatly into topics, and topics into lessons. The one-to-one relationship between summative assessments and topics is also a bit misleading: what it's meant to imply is that any particular learner will eventually demonstrate mastery of some topic (e.g., Yeleina will show that she understands functions). This is the smallest "chunk" for which we'd contemplate awarding a badge, though as discussed earlier, we're going to start with something coarser-grained.

Never Mind the Content, What About the Format?

Thu, 26 Jan 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I'm still gnawing on the problem of how to construct content for 21st Century learning—or, more prosaically, what I should use to build the next version of Software Carpentry. My starting point is the need to serve several different kinds of users [1], whose descriptions I have moved to a separate post on learners and their needs.

Textbook: big blocks of prose in some narrative order, with pictures, either printed or electronic, read at the learner's pace, alone.

Zuzel likes this.

Yeleina doesn't.

Xanthe uses content out of order via the index or search bar.

Wafiya remixes content from several textbooks to create lessons (by photocopying, merging PDFs, or whatever). Like Zuzel, she has read content in order, but like Xanthe, she mainly uses the index now.

Veronique has thought about writing one, but (a) doesn't think she has that much to say about any single topic, and (b) is put off by the effort that would be required.

Note: the comments below about the difficulty of copying, pasting, and altering also apply to electronic textbooks, as do the proposed remedies.

Static slideshow: a page-by-page dump of a PowerPoint deck, possibly accompanied by a transcript of what the lecturer would say when delivering it.

Zuzel uses this as if it were a badly-written textbook, with the transcript as the prose and the slides as diagrams.

Yeleina finds it distracting to switch attention back and forth from slides to transcript.

Xanthe searches the transcript to find what she wants, then curses because her search engine can't "see" the text in the slides. She also hates the fact that she can't copy and paste the code in the slides (since they're PNGs embedded in a web page).

Wafiya remixes this content like any other. She's too polite to curse, but she finds it tedious to re-type the code that's shown in the slides (but isn't duplicated as text in the accompanying transcript). She's also finds it wearying to have to re-do diagrams: since the slides are PNGs, it's difficult for her to copy part of a slide, move its elements around, and add a few of her own.

Veronique doesn't create material in this format because she thinks it's old-fashioned and not useful.

Note: source code can be made available as copy-and-pasteable text directly in the page, or for download; diagrams can similarly be made available as SVGs to facilitate remixing. Doing either currently requires considerable extra work on the part of content creators.

Voice-over slideshow screencast: a video recording of the slides (as they would appear on screen in a lecture) with someone speaking over them, and subtitles.

Zuzel ignores the video and reads the transcript as if it were a static slideshow. If a transcript isn't available, she (reluctantly) watches the video.

Yeleina prefers this to a static slideshow, but prefers the doodling screencast described below even more.

Xanthe hits the "back" button as soon as she realizes it's a video (unless there's a transcript, in which case she curses because she can't copy and paste code out of a video).

Wafiya directs students like Yeleina to these, but finds them harder to remix than other formats.

Veronique thinks this format is also old-fashioned and not useful.

Note: I'm assuming the subtitles are duplicated as a transcript, or available in some other searchable form. I'm also assuming that code is available for donwload or duplicated in the page for coying and pasting, though all of this requires extra work.

Voice-over doodling screencast: a Khan Academy-style recording of someone doodling on a tablet or coding live.

Zuzel treats this like a slideshow screencast.

Yeleina likes this format a lot, particularly if she can add comments at specific points and see her peers' comments.

Xanthe has mixed feelings: she dislikes explanations delivered this way, but frequently watches "how to" videos, since they're more likely to be accurate and complete than written descriptions.

Wafiya treats these like slideshow screencasts.

Veronique creates these fairly regularly: they're easy to do, and easy to re-do when systems change or she discovers a mistake.

Note: I'm making the same assumptions about transcripts, code, and diagrams as above.

Recorded whiteboard lecture: someone with a camera has recorded someone giving a lecture in a lecture hall, and spliced that with whatever was on the lecturer's screen.

Zuzel treats this like any other screencast.

Yeleina prefers this to doodling screencasts because she can see the speaker's body language.

Xanthe treats these like any other screencast, i.e., she'll use it if there's a searchable transcript and things to copy and paste, or if it's a recording of a live "how to" coding session.

Wafiya treats these like slideshow screencasts.

Veronique doesn't create these, partly because of the setup required, but also because she doesn't think seeing her adds value—the lesson's supposed to be about the stuff.

Note: I'm assuming an electronic whiteboard, since video of someone writing on an actual whiteboard is usually illegible.

Radio drama: a voice-only podcast-style presentation.

Zuzel ignores the audio and reads the text transcript.

Ditto for Yeleina.

Ditto for Xanthe.

Ditto for Wafiya.

Veronique doesn't create these.

Note: but for Ursula, who is blind, this is the only format—all the others fold into it. She doesn't need code samples as text for copying and pasting: she needs them so that her screen reader can tell her what's that code contains.

Star Wars: high-quality video with custom animations, cut scenes, and other special effects.

Zuzel watches these sometimes, but doesn't learn any more from them than she would from a slideshow.

Yeleina enjoys these, which means she pays more attention to them, which means she learns more (but no more than she'd learn from an engaging lecturer).

Xanthe doesn't see the point. Unless something blows up.

Wafiya likes their high production values, and remixes the special effects segments frequently.

Veronique can't afford to produce this kind of material.

Note: again, I'm making assumptions about transcripts, copy-and-pasting, etc.

Write your own adventureexploration: typically a set of connected ideas or challenges with explicit dependency information (i.e., you should/must learn A and B before tackling C).

Zuzel finds the lack of narrative difficult.

Yeleina enjoys these if each node in the graph is in one of her preferred formats. She enjoys them even more if she is exploring with peers.

Xanthe ignores the ordering and searches for what (she thinks) she needs. If content is locked down—i.e., if the system won't let her see or search C until she's "completed" A and B—she writes an angry tweet and moves on.

Wafiya likes this format for several reasons, but only if everything is always visible. First, it tells her how other teachers think ideas connect (something that is missing or out-of-band for other delivery formats). Second, it's easy to remix: again, providing it's open, she can reorder things as she thinks best for particular learners.

Veronique would like to do this, but has discovered that creating the metadata about dependencies and recommended paths is as hard as writing a textbook.

Wander aroundexploration: lots of little snippets, but no explicit dependency information.

Zuzel finds this even more difficult.

Yeleina likes this less than the "write your own adventure" format: she thinks it's no different than just using Giggle to find things.

Xanthe likes this because it's just like using Giggle searches. In fact, she uses every other format as if it were this one.

Wafiya feels the same way as Yeleina: she likes having stuff to remix, but she has to do that remixing before this material is useful to those of her students who aren't as independent as Xanthe.

Veronique creates content like this almost without realizing it by answering questions at Stuck Underflow.

Jam session: a bunch of learners in a room working through material simultaneously.

Zuzel doesn't like it: it's too noisy for her to concentrate, and she can't go back at her own pace to review.

Yeleina thinks this is the best... thing... ever.

Xanthe is Giggling for information as soon as the presenter tells people what the topic is, but will stop and watch carefully when the presenter is typing live on screen.

Wafiya can only book space to do this occasionally, and even then, she doesn't enjoy improv teaching.

Veronique enjoys doing this—she's volunteers with a local free-range learning group—but can only find time once every couple of months.

Note: in theory this can be combined with any of the formats above. In practice, it's almost always short, live lectures interspersed with hands-on practical work.

Personal tutoring: one-to-one instruction, a.k.a. "pair learning".

Zuzel doesn't mind this, but she really does prefer books...

Yeleina actually prefers jam sessions, since they tend to be more lively.

Xanthe likes having someone available to answer questions on demand, but is happy Giggling on her own for most of what she needs.

Wafiya wishes she could do this with every one of her learners, but there simply aren't enough hours in the day. The personalized lesson plans she draws up are the closest approximation she can manage.

Veronique does this a lot, but is frustrated that it doesn't scale—she really wants to help more than one person at a time.

I'm sure some of the above is inconsistent or just plain wrong, but here are my takeaways:

Our Long Tail

Thu, 26 Jan 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Frequency a page is viewed (as a percentage of total views) vs. pages (ordered by frequency); data taken over the last 90 days. I guess there's something to this "long tail" stuff after all.

The Big Picture

Wed, 25 Jan 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I'm trying to be systematic about re-designing the core curriculum of Software Carpentry. So far, I've identified 11 common questions:

Q01: How can I write a simple program?

Q02: How can I make the program I've written easier to reuse?

Q03: How can I reuse code that other people have written?

Q04: How can I share my work with other people?

Q05: How can I keep track of what I've done?

Q06: How can I tell if my program is working correctly?

Q07: How can I find and fix bugs when it isn't?

Q08: How can I get data into my program?

Q09: How can I manage my data?

Q10: How can I automate this task?

Q11: How can I make my program faster?

whose answers depend on three fundamental principles:

Badging

Tue, 24 Jan 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

One of the things we need to do in the next six months along with running workshops and updating our online content is to create some sort of badging to recognize people's skills and contributions. As we said in the proposal to the Sloan Foundation, "A badge program will provide near-term incentives for both learning and mentoring; a framework to support viral, peer-driven engagement with the program; and facilitate recognition by partner institutions and potential employers."

Take Out Agile, and Add...What?

Tue, 24 Jan 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Based on the feedback we've received so far (both as comments and by email), it looks like we should take development methodologies (i.e., agile development) out of the core curriculum and replace it with two hours on:

Nothing: there's already too much in the core.

Spreadsheets: because many scientists use them badly.

NumPy and/or Pandas: because many of them are crunching matrices/doing stats.

Visualization: which in practice would mean the basics of matplotlib.

Image manipulation: because it's fun as well as useful, and lets us talk about binary vs. text data.

I am quite arbitrarily limiting options to those five. Please cast your vote (one vote, not three out of five) in comments. We'd be grateful if you could include a brief explanation as well.

Test-Driven Public Speaking

Tue, 24 Jan 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Once again, Cameron Neylon explains things much better than I ever could:

"The impact factor of a journal is a better predictor of the chances of a paper being retracted than...of the number of citations."

Revising the Curriculum

Mon, 23 Jan 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I've been thinking some more about what the foundation and core of Software Carpentry actually are (and not just because Jon Pipitone keeps pestering me to do so). My last attempt had a foundation of seven principles and dozen topics in the core. I think I can slim that down even further; in fact, I think three big principles form the foundation of computational thinking:

The First Bootcamp of 2012

Fri, 20 Jan 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We just wrapped up the first bootcamp of 2012 at the Space Telescope Science Institute. 14 scientists with a wide variety of computational backgrounds spent two days learning about testing, version control, program structure, the basics of Python, and the psychology of learning and programming. We're following up with 6 weeks of online material, partly because that's what fits everyone's schedules, and partly to see whether a blended approach works better than either strategy on its own.

Why Is This Hard?

Sun, 15 Jan 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I've been teaching scientists to program since 1998 (or 1986, if you want to start with my first lunch-and-learn for grad students in physics at the University of Edinburgh). Technology has advanced by leaps and bounds in that time, but I don't think it's any easier than it used to be to get basic software skills into people's heads. What makes it hard?

The What, Why, and How of Bootcamps

Fri, 13 Jan 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We've just added a single-page description of the two-day bootcamps we're planning to run in the next six months. In brief, their aim is to ensure that people have a few core skills, so that they can tackle our online material productively, and to help them get past startup hurdles such as software installation. If you have questions, comments, or suggestions, please add them to that page; if you'd us to help you organize and run a bootcamp, please get in touch.

Sloan Foundation Grant to Software Carpentry and Mozilla

Wed, 11 Jan 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are very pleased to announce that the Sloan Foundation has generously agreed to fund six months of work by Software Carpentry and the Mozilla Foundation. The proposal we submitted, which outlines what we're going to try to do, is included below—it's a lot of work, but we're very excited to have the opportunity to move Software Carpentry forward. Details are below the fold...

Settings Our Sights a Little Bit Lower

Wed, 04 Jan 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A couple of days ago, I posted replies to some of the comments that people had made on my posts about Software Carpentry's future. To recap, I want SC to:

offer learning materials so that people can work through them on their own;

be a repository where people can evolve those materials;

coordinate people who are organizing live workshops, offering technical support [1], etc.; and

coordinate a distributed research program to evaluate Software Carpentry's effectiveness.

It would be easier to achieve these four goals if we had merge-friendly formats for learning materials (both micro and macro), a large team of core content developers, and stable long-term financial support. Since we don't have any of those unicorns, what should we try to do in the next six months?

Some Responses to Some Comments

Sat, 31 Dec 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Several people have written some useful comments on my recent "where are we going?" posts. It's exactly the kind of feedback I was after, so here are my answers.

Goal #1: helping thousands of people each year.

You propose two very broad ideas for what this would mean: a) community "co-learning" initiatives like Hacker Within, presumably using Software Carpentry content, or somehow organised by SWC? and b) more people contributing to SWC content, as well as supporting others online. In essence, the vision is for both offline and online co-learning communities to exist. It doesn't sounds like you envision SWC as an authorative source for instruction or community, but it is to be a hub of some sort, right?

The Fire Last Time

Sat, 31 Dec 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Back in November, Justin Reich wrote a post titled "Will Free Benefit the Rich?" (re-posted as "Open Educational Resources Expand Educational Inequalities"). In it, he outlines two possible futures. In scenario 1, everyone benefits from free online resources in ways that narrow the gap between the rich and the poor. In scenario 2, everyone benefits, but the well-off benefit more, so the gap widens. What caused a fuss was his claim that the empirical data we have so far supports scenario 2: in the year of Occupy, saying that something is going to increase inequality is going to stir up strong emotions.

Fork, Merge, and Share

Fri, 30 Dec 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As George Pòlya said, sometimes the best way to solve a problem is to solve a more general one. In that spirit, this post was originally going to be about the mechanics of helping thousands of people a year (which is the first of our five-year goals). After getting feedback from a few people on early drafts, though, it has morphed into a discussion of something that I hope you'll find more interesting [1].

Yet Another Survey

Thu, 29 Dec 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Prakash Prabhu and others recently published "A Survey of the Practice of Computational Science" based on information from 114 researchers at Princeton University. The emphasis is different from that of the survey Hannay and others (including me) did in 2008-09, but the findings are broadly similar.

Organizing Instruction and Study to Improve Student Learning

Sat, 24 Dec 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I had breakfast a couple of days ago with Jon Pipitone, a former student who has helped out with Software Carpentry off and on in the past. When we discussed my post summarizing what I've learned so far about online education, he had several questions and suggestions (thanks, Jon). I'm still digesting everything he said, but there is one point I'd like to act on now.

What Success Looks Like Five Years Out

Sat, 24 Dec 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Having talked about what I've learned and how well our teaching measures up, I'd like to explore what success would actually look like for Software Carpentry. Our long-term objective is to make productive and reliable computing practices the norm in science and engineering. My model is the way statistics became a "normal" part of most scientists' training in the 20th Century. Most psychologists and geologists are emphatically not statisticians, but most of them know (or at least have known, at some point) what a t-test is, what a correlation coefficient means, and how to tell when they're out of their depth. Equally, most scientists should know how to pipe a few simple filters together to clean up their data, how to use version control to track and share their work, whether their computational results are as trustworthy as their experimental results [1], and when to go and talk to either a real numerical analyst or a professional software developer.

It Just Keeps On Hurting

Tue, 20 Dec 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I received email a few days ago from someone who had just found this site (reproduced here with permission):

I am working on getting myself set up to do scientific programming in Python on a MacBook Pro. I plan to use mySql and Pentaho. I am new to open source, Mac, and Python. Where I've run into the most problems is getting a working environment established with all the tools in place. For instance, I ran into a huge problem trying to install mysqldb, the Python/MySQL interface package. Trying to resolve that, I've encountered a bewildering array of alternative ways to do things, each accompanied by strong opinions and each entailing a different group of packages. One example being package managers: easy_install, pip, various brews, macports, etc. I don't really care about some absolute right way—I just need working, consistent tool set.

What I've Learned So Far

Tue, 20 Dec 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I worked on Software Carpentry full-time for a year starting in May 2010. In that time I created over 140 PowerPoint decks, about 12 hours of video, and taught the course six times (twice online and four times in person). I also learned a few things that I hope other people will find useful:

Online teaching isn't as effective as in-person teaching.

Today's tools suck.

If content is king, then community is queen, pope, emperor, and president-for-life.

If you don't know how to tell if you succeeded, you're going to fail.

If you don't take the time to explore what other people have done, you deserve to fail.

Online teaching isn't as effective as in-person teaching. "I talk but don't listen, you listen but don't speak" is a lousy teaching model, but that's all recorded video is. Ask-and-answer is much more effective, particularly when you can see the baffled looks on students' faces, but no present-day web-based technology can deliver that for more than a handful of students at a time. In my gloomier moments, I fear there's some sort of conservation law at work: what we gain in reach, we lose in effectiveness. However the experiments we did with chat sessions seemed to work well; we should have experimented with that more, and with desktop sharing as well.

New Features in Excel for Scientists

Tue, 13 Dec 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Carly Strasser, of the Digital Curation for Excel Project, has posted some ideas about new features that could be added to Excel to make it more useful for scientists. In brief, the list is:

Generate metadata.

Generate a data citation for the data file.

Check the spreadsheet for export compatibility.

Link to archive services.

Feedback and suggestions would be welcome...

How to Teach Webcraft and Programming to Free-Range Students

Wed, 07 Dec 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I will be running a P2PU course starting in January on teaching free-range learners how to program and build stuff on the web. The blurb is below; anyone who wants to can sign up to follow along or take part (we expect it will require 3-4 hours/week from mid-January to some time in April). I'm not an expert on these subjects by any means, but I've learned a few things from running Software Carpentry that I think are worth sharing, and hope that this course will give me a chance to learn more. If you're interested in teaching scientists how to do things with computers, please come and join us.

Three Short Thoughts

Tue, 29 Nov 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A BBC article title "Coding — the New Latin" resonated: Latin was the language of learned discourse in the formative years of modern science, but not something most people spoke day-to-day. I think that's a good model for computing in the sciences; like statistics, it requires familiarity, not expertise.

Jorge Aranda's review of Codermetrics talks about the limitations to quantification in software engineering. I've said before that we need to measure how much time is lost due to poor computing skills in order to get people to take this kind of training seriously, but I'm very conscious of just how much measurement can't tell us.

Another thought-provoking post by Cameron Neylon asks what it's reasonable to expect from scientists, and their software. I'm sure he'd enjoy hearing what you think...

Building a Bibliography

Fri, 25 Nov 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

With help from several of our regular readers, we have assembled a bibliography of research related to software engineering and computational science to go with our recommended reading list for students. We hope you find it useful, and we would welcome corrections and additions.

Knowledge of the Second Kind

Sat, 19 Nov 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Over the last three years, a group of students has quietly been converting snacks and enthusiasm into scientists who can program.

The Hacker Within is a student club at the University of Wisconsin — Madison which came about when a number of nuclear engineering graduate students needed a forum to exchange tools and share best practices for their increasingly software intensive research. The success that followed provides an example of an educational model that has fostered necessary software skills among science and engineering graduate students.

Accessible to All?

Fri, 18 Nov 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I just posted an article on my personal blog about the (in)accessibility of online educational material (including Software Carpentry's). As I said there, there aren't any easy answers, but if we do find funding to keep this project going, I'd like to find ways to make our content easier for everyone to use.

Quantifying Installation Costs

Fri, 18 Nov 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A few months ago, I tried to quantify the cost of poor software skills. A recent post from Adam Klein gives is a good excuse to try to do something similar for the cost of installing software. In his post, Klein describes the 17 steps he went through to set up a Python data hacking environment on a new machine. If we assume that each step has a 5% chance of failing for some reason (packages have moved on, the compiler isn't exactly the same version as Klein was using, whatever), then the chances of the whole process working are (1-.05)¹⁷, or roughly 42%. In other words, his process will fail to work the first time for over half of the people who try it. In some cases, they'll be able to figure out why, fix the problem, and move on, but in many others they won't—as I said earlier this week on my personal blog, we've taken something that may or may not be intrinsically hard (programming), and made it much harder by burying under layer upon layer of grief. The end result is that when a scientist sits down to try something new, s/he has no way of knowing whether it will take an hour, a day, or forever. It's hard to build a career on top of that kind of uncertainty...

Show Me the Data

Fri, 18 Nov 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I got mail from a colleague at a prominent US university yesterday saying (in part, and elided to protect the guilty):

...the graduate student representative to the curriculum committee reported that the students did not want a scientific computing course, that they would instead figure it out themselves.... How does one respond to statements like this...that have...basically frozen skill levels? The options I see are formal ("in curriculum") training, bootcamps and /workshops, and letting them "figure it out themselves". Are there arguments about the successes of each?

Clearing Up Code

Mon, 14 Nov 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The November/December 2011 issue of IEEE Software has a good article by the Climate Code Foundation's Nick Barnes and David Jones titled "Clear Climate Code: Rewriting Legacy Science Software for Clarity". In it, they describe how and why they rewrote a program used to calculate and compare global surface temperatures based on historical data. The original had been attacked by climate change denialists, first because it wasn't publicly available, and then because it was tangled and hard to run. Their rewrite produced something smaller, faster, and much easier to understand; most importantly, though, it validated the results of the initial program.

Surviving the Tsunami

Mon, 14 Nov 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The October 2011 issue of ACM Queue features an article by Bruce Berriman and Steven Groom titled "How Will Astronomy Archives Survive the Data Tsunami?" The figures are scary: astronomers already have a petabyte of publicy-available data, and are adding half a petabyte per year, a rate which will increase dramatically as new instruments come online. The only way to avoid this all becoming write-only is to bet on emerging technologies, from general-purpose GPUs to cloud computing. The problem, of course, is that "emerging" usually means "flaky", both because the tools haven't had time to mature, and because we, their users, don't have the years of experience needed to know how best to use them. (As far as I'm concerned, we're still trying to figure out how best to use object-oriented programming in science, and we've been at it for thirty years...)

Successful Bootcamp

Fri, 11 Nov 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Our 2011 Software Carpentry bootcamp, hosted with help by The Hacker Within and Scinet, was a huge success. We hosted 25 students for two very full days of hands on introductions to Python, The Shell, Nose tests, SVN, and sqlite.

So what did we learn? We're still waiting on participant feedback, but a few things come to mind.

First, wow, how did we screw software installation up so very badly? Most of our technical problems can from inconsistencies between various Python builds. Some students installed numpy separately from python and ran into version mismatches. The demo packages were missing from some people's iPython installations (including my own!) That doesn't even mention Cygwin. We got by thanks to the help of the wonderful volunteers who were able to sort out most of the problems, but it's hard to pitch a produce that is that difficult to get up and running. Imagine if we tried to publish papers as inaccessible as our code.

The Best vs. the Good

Tue, 08 Nov 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Cameron Neylon recently posted another thought-provoking piece, this one titled, "Building the perfect data repository...or the one that might get used". In it, he talks about why big institutional efforts to create scientific data repositories have mostly failed to take off, and points at simpler grassroots efforts that scientists might actually adopt because they immediately and obviously solve problems that scientists actually realize they have. One that he points to is DataStage, "a secure personalized 'local' file management environment for use at the research group level, appearing as a mapped drive on the user's PC." Another is If This Then That, which is the simplest "dataflow" tool you could imagine, and which I've been using regularly since it burst on the scene a couple of months ago.

The Ladder of Abstraction and the Future of Online Teaching

Tue, 08 Nov 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Up and Down the Ladder of Abstraction is one of the most thought-provoking things to hit the web in a long time. Its author, Bret Victor, doesn't just talk about the design process—he shows us what a great interactive tutorial ought to look like. (For a shorter, simpler, but equally inspiring example, have a look at the home page for his Tangle project, and ask yourself what learning would be like if learners could play with every diagram and quantitative statement in their "textbooks" that way...)

Nirvana on Monday Night

Sun, 06 Nov 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are running a two-day bootcamp at the University of Toronto tomorrow and Tuesday (Nov 7-8). As part of that, we'll be heading to Nirvana, at 434 College Street (just east of Bathurst) around 6:00 pm on Monday for pints, good food, and conversation. You're all welcome to join us.

Research Without Walls

Sat, 22 Oct 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Open science, reproducible research, and better computational skills aren't necessarily connected, but in my experience, people who care about one usually care about the others as well. In that light, I just signed the "Research Without Walls" pledge: effective today,I will assist in the peer review process (as a reviewer, board/committee member, chair, editor, etc.) only for conferences, journals, and other publication venues that make all accepted publications available to the public for free via the web. If you believe that sharing ideas is the heart and soul of science, please sign up as well.

Slides from Hans-Martin

Fri, 21 Oct 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Hans-Martin von Gaudecker has posted the slides for his course "Effective programming practices for economists". They're a nice complement to the ones on this site—thanks, HM.

American Scientist Article on Empirical Studies of Software Engineering

Wed, 19 Oct 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

An article that Jorge Aranda and I wrote for American Scientist about empirical studies of software engineering is now up on the web. We hope it's a good introduction to the area, and we look forward to your feedback. If you'd like to know more about the area, please check out our joint "neat results" blog at http://www.neverworkintheory.org/.

The Science Code Manifesto's Five C's

Fri, 14 Oct 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Science Code Manifesto comprises five core principles:

Code: All source code written specifically to process data for a published paper must be available to the reviewers and readers of the paper.

Copyright: The copyright ownership and license of any released source code must be clearly stated.

Citation: Researchers who use or adapt science source code in their research must credit the code's creators in resulting publications.

Credit: Software contributions must be included in systems of scientific assessment, credit, and recognition.

Curation: Source code must remain available, linked to related materials, for the useful lifetime of the publication.

In a way, Software Carpentry's goal is to give people the skills they need to do these things right: to create code that other people can read, to share and evolve that code in maintainable ways, and to create code that's useful in the first place. If you agree with their principles, please take a moment to endorse them, and please help make them a reality.

Updating to HTML 5

Fri, 14 Oct 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are looking for a volunteer to update our episodes on HTML to the new HTML 5 standard. The changes are (probably) fairly minor, and as always, we'll help with post-production (editing of audio, etc.). If you're interested, please get in touch.

Four New Episodes on Databases Using Microsoft Access

Fri, 07 Oct 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Thanks to the indefatigable Ethan White, we have just added four more episodes on using databases with Microsoft Access:

aggregating data

combining data with 'join'

nested queries

handling missing data

We hope you enjoy them.

Revamping This Site

Wed, 05 Oct 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We have been using WordPress as a content management system (CMS) for Software Carpentry since the launch of Version 4 in July 2010, and while nothing in particular is broken, I'm increasingly dissatisfied with its interface and performance. Looking at other online tutorial sites, there are a lot of things we could add; I'd be grateful for your feedback on which you would actually use, and what we might have missed. Our feature list is:

2011 Software Carpentry Bootcamp Sold Out!

Tue, 04 Oct 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Software Carpentry is teaming up with The Hacker Within to offer a 2 day bootcamp for scientific programmers on November 7-8. Forty tickets and ten faculty observer slots filled up within two weeks of announcing the workshop on campus.

So what are we going to cover? From what we know, the participants are a diverse group. They include Industrial Engineers, Physicists, Psychologists, and many other departments on campus. One thing they all deal with is data, and often lots of it. Data analysts need a place to store data, so we'll introduce databases using SQLite and version control with Subversion. Data analysts also need programming tools to reformat, clean, and analyze their input. We will spend over half of the time introducing Python, and we'll also introduce the shell.

Plus Ca Change...

Thu, 22 Sep 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Once again, Compute Canada has sent out a document for "review" without leaving time for people to provide meaningful input. And once again, it's all about "big iron", as if scientists were somehow, magically, going to acquire the basic skills needed to write modular code, test it, maintain it, and—oh, what's the use? I've known for years that the only way we're going to fix this is to show a younger generation of scientists that doing things the right way also makes them more productive, and then wait for them to have enough seniority to fix our backward-looking institutions.

I'm Not Normally Lost for Words

Tue, 20 Sep 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I mentioned last week that I'm trying to put together a lecture on packaging and installation. It's proving harder than I expected: I'm not normally lost for words, but I'm struggling to get these ones to come together. My goal is to help people understand what happens when they install software on a computer, so that they can:

diagnose and fix problems when things go wrong;

understand why the things that are in packaging tools are there, and how to use them.

After a few false starts, I think the best way to do this will be challenge/response, i.e., trace the evolution of a typical scientific software package from a single Python file shared by email to a mix of Python and C that plays nicely with distutils2. Steps I've identified are:

The Simplest Web That Could Possibly Work

Sat, 17 Sep 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A new web tool called If This Then That generated a flurry of interest this week (see for example Scott Hanselman's blog post). Simply put, IFTTT lets you connect web services together using if-then rules. It's as simple as:

Click on "this" to start.

Select "Twitter", and tick off boxes to create a rule (such as "when Alan Turing tweets").

Click on "that".

Select what you want to happen (e.g., echo the tweet to your own Facebook page).

There is no step 5.

That's the beauty of it: there is no step 5. It knows how to work with all sorts of services, from Instapaper and Pinboard to weather reports (yes, weather reports), and there is no step 5. Just as Twitter stripped blogs down to their bare essentials, IFTTT takes graphical workflow tools like Yahoo! Pipes and says, "What's the simplest version of this idea that could possibly be useful?"

Progress Of A Sort

Tue, 13 Sep 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As I mentioned a few months ago, I'm going to turn Software Carpentry into a book. Here's the present status:

Chapter Word Count

Introduction 768

Spreadsheets 3924

Subversion 6405

Python 6345

Interlude: What Is Text? 1071

Functions and Libraries 9409

Interlude: Boolean Logic 921

Case Study: Invasion Percolation 8929

Interlude: How Are Numbers Stored? 1828

Testing 4549

Error Handling 1529

The Shell 12919

Make 5668

Interlude: Provenance 1810

Sets and Dictionaries 7627

Case Study: Phylogenetic Trees 1689

Systems Programming 60

Interlude: Configuring Programs 2146

Numerical Programming 6193

Multimedia Programming 4125

Steganography 1657

Installation 135

HTML and XML 6404

Databases 10676

Regular Expressions 8037

Object-Oriented Programming 4002

Building Desktop GUIs 137

Interlude: Persistence 5984

Web Programming 9639

Security 1285

Performance 8043

Parallel Programming 931

Software Engineering 6391

Epilog 873

Acknowledgments 202

Glossary 12074

Bibliography 3378

Total 167,763

167,673 words might seem like a lot, but based on past experience, I think I'm about halfway to a readable book—editing is always as much work as writing stuff in the first place.

What Happens When You Install Something?

Thu, 08 Sep 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The most frustrating part of this course is always getting things set up on students' machines. (Yes, we could give them all a Linux VM with everything installed properly, but if they're going back to Windows or Mac OS when the course is over, that's what they want to learn on.) Setting up a development environment is often just as frustrating for professionals (it took many of us half a day to get Lernanta running on our machines a few weeks ago), so I think we'd be doing Software Carpentry students a favor if we taught them what actually happens when you install software. Knowing that would help them debug things when they go wrong (and they always go wrong), and also help them understand why packaging their own work to share with others is as complicated as it is. My first cut at a syllabus for such a lecture is:

Where is the Puck Going to Be?

Mon, 05 Sep 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Looking at the schedule for Science Online London 2011 makes me feel that Software Carpentry is showing people how to solve yesterday's computational problems—that it's answering the questions people had (or should have had) in 1995, when desktop applications were the only game in town, and when computers were primarily used for calculating, rather than for sharing. They still are, and I think that version control and regular expressions and what-not are still the rock on which more novel things are built, but a lot of other things are taking shape in the fog. You can call it "open science" or "science 2.0" or whatever you want, but its focus is on sharing information more effectively by changing publication models, enabling reproducibility, making data findable, encouraging scientists to use groupware (sorry—we're supposed to call it "social media" these days), and so on.

Teaching Security to Scientists

Fri, 02 Sep 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Thanks to everyone for their suggestions regarding what we should teach about computer security if we only have one hour (the usual constraint for topics in this course). The outline below is based in part on the lecture on security from Version 3 of this course, in part on Rick Wash's excellent study of folks models of computer security, and in part on mistakes I've seen (or made) myself in the past five years. Feedback would be very welcome, but remember: we're teaching scientists and engineers who are programming as a way to do science, not as an end in itself.

Renting Cycles Has Never Been Easier (For Some Definition of 'Easier')

Thu, 01 Sep 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This video, titled "Building an AWS Cluster in 10 Minutes", is both inspiring and depressing: inspiring, because you really can set up a cheap supercomputer for almost no money in just a few minutes, and depressing, because it's effectively out of reach for most scientists, who won't understand the terms being thrown around (certainly not well enough to get themselves out of trouble if anything goes wrong). The good news is, Titus Brown's course has shown that scientists can get from zero to cluster in just two (full-time) weeks.

Demos Reinforce Errors, and Confusion is Good

Wed, 17 Aug 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Mark Guzdial has posted a good summary of this year's ICER keynote by physics education guru Eric Mazur, in which he reported the results of several recent experiments. The most important for us are:

Giving people a demo of something actually results in them understanding it less well, because they fit what they've seen into their preconceptions (which are then reinforced). Guzdial interprets this to mean that CS educators need to do more live coding.

Students like teachers who clarify things, but students who are confused are actually more likely to learn and understand.

Students' self-reported understanding of a topic has no relation to their actual understanding of it (which highlights once again the fact that self-assessment is useless).

There are obvious implications for this course...

Introducing Programming a Different Way

Mon, 08 Aug 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Our quick introduction to Python is the module I'm least happy with, so I've been thinking about how to re-design it. I've included a new outline below; comments would be very welcome.

Programming is what you do when you can't find an off-the-shelf tool to do what you want

Scientists' needs are so specialized that there often isn't such a tool

Even if it exists, the cost of finding it may be too high (can't Google by semantics)

Why is programming hard to teach/learn?

Computing in Physics 101: What We're Doing Wrong

Thu, 04 Aug 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Mark Guzdial and his colleagues do top-notch research on computing education—that's "teaching people computing", not "using computers to teach people", though for obvious reasons, the two frequently overlap. He recently wrote three blog posts that I think everyon pushing for more computing in the classroom should read. In them, he describes the results of Daniel Caballero's PhD research, in which he compared first-year physics students doing a traditional course with ones doing an equivalent course which included a large programming component. His findings were:

Software Carpentry in HPCWire

Fri, 22 Jul 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

HPCWire has run an interview with me about Software Carpentry (a follow-up to one they did several years ago). Regular readers will have seen the main points before, but I hope it's a good summary of the current state of play, and of what's wrong with equating "scientific computing" and "high performance computing".

And Speaking of Titus Brown...

Wed, 20 Jul 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Michigan State's Titus Brown recently ran his course on analyzing next-generation sequencing data for the second time. Judging from his report, it was just as successful as last year's. Congrats!

How Much Do You Need?

Wed, 20 Jul 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Michigan State's Titus Brown has posted a good discussion of what kind of computing hardware you need to do bioinformatics, and why. Long story short, it's about $100K/year (US).

Material from Newcastle Workshop Now Available

Wed, 20 Jul 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Back in June, we mentioned a workshop on computing skills at the University of Newcastle. It reportedly went very well, and presentations and other materials are now available. Many thanks to Elizabeth Petrie, the other organizers, and the presenters.

The Case of Abinit

Wed, 20 Jul 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The latest issue of Computing in Science & Engineering has a good article by Yann Pouillon, Jean-Michel Beuken, Thierry Deutsch, Marc Torrent, and Xavier Gonze titled "Organizing Software Growth and Distributed Development: the Case of Abinit" (unfortunately behind a pay wall). It describes the infrastructure and practices used to manage Abinit, a half-million line open-source program. It's a lot more engineering than most teams need, but it provides a lot of insight into what small projects might grow into. If anyone would like to do a similar paper about what a smaller-scale team uses, it would be a welcome counterpoint.

Architecture of Open Source Applications Webinars Tuesday July 13 and 20

Mon, 11 Jul 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Smart Bear Software is hosting two online panel discussions about The Architecture of Open Source Applications, at 1:00 pm EST on Wednesday, July 13, and again at the same time (with different panelists) a week later. You can sign up on their site; we look forward to seeing/hearing from lots of you.

Stanford Course Went Well

Sun, 10 Jul 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Prof. Risa Wechsler, along with Alex Ji and Zahan Malkani, recently ran a short course at Stanford based in part on Software Carpentry called Physics 91S1: Practical Computing for Scientists. The course notes and handouts are available online, and the open source bits of code are on GitHub. We hope you find them useful...

Reproducible Computational Geophysics

Wed, 06 Jul 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A summary of a recent workshop on Open Software Tools for Reproducible Computational Geophysics is now online. Lots of interesting stuff, and once again, the things we teach in this course are prerequisites for doing/using/understanding much of it.

Mentioned in Nature Methods

Fri, 01 Jul 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A recent article in Nature Methods by Jeffrey Perkel titled "Coding your way out of a problem" makes mention of Software Carpentry. Elsewhere, Mike Croucher has written a nice article called "In defense of inefficient scientific code", which makes many of the same points we've been making here.

It Will Never Work in Theory

Wed, 29 Jun 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Inspired in part by Lambda the Ultimate, which reports on what's new in programming language research, Jorge Aranda and I have started a new blog called "It Will Never Work in Theory" to bring you the latest results in empirical studies of software engineering. The first posts discuss:

Rahman and Devanbu's "Ownership, Experience, and Defects: A Fine-Grained Study of Authorship", which found that code worked on by one developer (rather than many) is more often implicated in defects, but that a developer's experience with a particular file (rather than the project in general) reduces defect rates.

Stolee and Elbaum's "Refactoring Pipe-like Mashups for End-User Programmers", which applies the "code smells" meme to Yahoo! Pipes (and by implication shows that refactoring ideas can be applied to other end-user programming systems).

Mockus's "Organizational Volatility and its Effects on Software", which found that an influx of newcomers into a project doesn't increase fault rates (since they're usually given simple tasks to start with), but that organizational change can still account for about 20% of faults.

Our aim in starting this blog is to continue the work begun in Making Software: to let practitioners know what researchers have discovered, and what kinds of questions they can answer, and to give researchers feedback on what's useful, what isn't, and what they ought to look at next. We look forward to your feedback.

Michael Nielsen Talks About Open Science in San Francisco on June 29

Wed, 22 Jun 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As per his blog post, the inimitable [1] Michael Nielsen will be talking about "Why the net doesn't work for science—and how to fix it" next Wednesday in San Francisco. It's sure to be both informative and enjoyable—hope you can make it.

[1] Well, you try to imitate an Australian quantum physicist turned open science advocate...

Doing the Math

Mon, 20 Jun 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Let's do some math. Suppose that working through the Software Carpentry course takes the average scientist five full-time weeks. It doesn't matter whether that's one five-week marathon, or whether the time is spread out over several months; the cost is still roughly 10% of the scientist's annual salary (if you're thinking like an administrator) or 10% of their annual published output (if you're thinking like the scientist herself). How big a difference does it have to make to her productivity to be worthwhile?

Health Informatics Resources

Sat, 18 Jun 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Via William Hopper, a list of online healthcare informatics resources that might be of interest to some readers. If you have others, I'm sure he'd enjoy hearing from you.

New Episode: MATLAB Structs and Cell Arrays

Wed, 15 Jun 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The title says it all: thanks to the tireless Tommy Guy, we have a new episode on MATLAB structs and cell arrays.

A New Look

Tue, 14 Jun 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I'm fond of the Software Carpentry logo, but the blue-to-white color fade is difficult to print on coffee mugs, and impossible to embroider on shirts. Thanks to the talented Veronica Wong, we have a new one:

We'll be converting things over piece by piece as we rebuild the website over the summer.

Audio Processing in Python

Fri, 10 Jun 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Thanks to Becky Stewart, we now have a 12-minute episode on audio processing in Python. We hope you find it useful—as always, feedback is very welcome.

Practical Computing for Everyone (not just biologists)

Tue, 07 Jun 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Steven Haddock and Casey Dunn:
Practical Computing for Biologists.
Sinauer Associates, 2010, 0878933913.

My copy of Practical Computing for Biologists arrived last week, and I've been very impressed. It is a well-written, well-paced guide to basic computing skills for scientists and engineers of all stripes (not just biologists). Many of the topics will be familiar:

editing text files (including how to use regular expressions in an editor)

the Unix shell

basic Python programming (including debugging strategies)

relational databases

SSH

installing and configuring software

There are also a few that we don't cover, such as interacting with hardware, and some that are covered in more depth than we give them, like image manipulation. The pace is gentler than Software Carpentry, but the last couple of years have convinced me that's a good thing: I think Haddock & Dunn have it right for this target audience. And it's beautifully produced: full-color printing and great graphical design make this book a joy to read. If I ever do turn Software Carpentry into a book, I might skip the topics PCB covers and just tell people to go and buy it.

Programming for Scientists at Newcastle University: June 20, 2011

Sat, 04 Jun 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

From the announcement:

Programming is becoming an increasingly important part of scientific research, yet many scientists are self-taught programmers with little formal training. This means that we are often unfamiliar with simple tools that can make programming and dealing with data faster, more reliable and more reproducible. This event is a day-long workshop to develop awareness of the skills and tools that help make computing more efficient and provide results that are less prone to error. If you've ever thought "Surely there must be a better way to do this", then this is the event for you!

Five on Systems Programming

Thu, 02 Jun 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Thanks the Software Sustainability Institute's Mike Jackson, we now have five episodes on how to inspect and manipulate files and directories from inside a Python program—many thanks. If you would like to contribute to this project as well, please get in touch.

Workshop at CEF'11

Wed, 01 Jun 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I will be giving a one-day workshop on software skills for computational economists at the 17th International Conference on Computing in Economics and Finance (CEF 2011) in San Francisco on Tuesday, June 28. I'm going to talk about version control, tracking provenance, systematic testing, and key results in empirical software engineering; there will also be a guest lecture from Michael Nielsen about the future of scientific practice in a networked world.

'The Architecture of Open Source Applications' is Now Available

Mon, 23 May 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

It has been slightly over a year in the making, but it's finally here: The Architecture of Open Source Applications has been published. You can buy the book directly from Lulu.com at http://www.lulu.com/product/paperback/the-architecture-of-open-source-applications/15819207, or view the contents online at http://aosabook.org.

My thanks to all the people who contributed to the book, and especially to Amy Brown, my tireless and diligent co-editor. We hope you enjoy it.

More Interested in the Asides

Sat, 14 May 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

So how did the term go, you ask? Here's what traffic on this site looked like:

The three spikes in March where we had up to ten times our usual 250-a-day visitors were the articles on tuple spaces, literate programming, and graph layout—in other words, the articles that weren't about teaching scientists and engineers basic computational skills. As the man said, "So it goes."

Damn the Torpedoes (but I could use some help navigating)

Fri, 13 May 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Despite my own calculations (which were optimistic to begin with, and are more so now that I have a new job), I'm going to try to turn Software Carpentry into a book. Here's what's I'm planning to include, and its presesnt status; I would be very grateful for feedback, especially from alumni, about what doesn't need to be here, and what's missing. I think I'm still overshooting the knowledge and needs of far too many people; if you think so too, please let me know.

The Architecture of Open Source Applications

Fri, 06 May 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

My apologies for how quiet this site has been recently—finding a new job takes time [1], and we had to make one final push on the other project I've been working on. But I'm pleased to announce that The Architecture of Open Source Applications is now at the printer's—we're going to give the first copy a final going-over, then make it available for purchase (hopefully within two weeks). I'm very pleased with how it looks, and very grateful to my co-editor Amy Brown, the contributors, and the reviewers for all their hard work.

The Hacker Within at MSU in June

Tue, 03 May 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The University of Wisconsin's Hacker Within team are running a bootcamp at Michigan State University June 4-5 2011—see the announcement for details. Space is limited, so sign up now!

Managing Data

Mon, 02 May 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We have just posted a screencast on managing data written by Orion Buske, a graduate student in bioinformatics at the University of Toronto (and former Software Carpentry TA). We hope you enjoy it.

Chapters

Sat, 23 Apr 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I've converted three more Software Carpentry lectures to chapter format, bringing the total to four:

the shell

number crunching with NumPy

regular expressions

software engineering

It's all CC-A licensed, and yeah, the images could be a lot better: they're just screen captures from the original PowerPoint slides, because re-doing them as SVG felt even more like yak shaving than converting the text. Please let me know if this is useful.

In Praise of Street Fighting

Fri, 22 Apr 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Sanjoy Mahajan's Street-Fighting Mathematics is subtitled "The Art of Educated Guessing and Opportunistic Problem Solving". As the author says in the introduction, "Too much mathematical rigor teaches rigor mortis: the fear of making an unjustified leap even when it lands on a correct result." It's only 134 pages long (including the index), but it's packed full of practical ideas for tackling mathematical problems; in a sense, it aims to train people to do the kind of rough-and-ready calculations that David MacKay deploys so effectively in Sustainable Energy—Without the Hot Air.

Holding Up a Mirror

Mon, 18 Apr 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Cameron Neylon always has interesting things to say. In a recent talk, he commented on my tendency to idealize laboratory practice when contrasting it with the sorry state of computational work. Thought-provoking...

Prototyping

Mon, 11 Apr 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The numbers might not make sense, but I'm still curious about whether Software Carpentry would make sense as a book. To find out, I spent five hours converting the existing lecture on regular expressions to chapter format. I'd welcome feedback about how useful this is. In particular, without the highlighting and animation of PowerPoint slides and video, how much harder is it to understand what's going on? What could be done to make it easier? More pictures? (I really want to draw arrows to point at the tabs and spaces in the sample data files...) Fully hyperlinked audio? Side-by-side display of code and text (which would be difficult to squeeze in, but not impossible)? Please let me know...

By The Numbers

Sat, 09 Apr 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

So what next for Software Carpentry? One possibility is to turn the scripts for the episodes, the examples, and the diagrams in the slides into a book. Let's crunch some numbers to see how feasible that is:

Topics 24

...mostly done (say, 80%) 18

...not even started 6

Words per topic 5,000

Words to write 48,000

Words per day 1,500

Days required 32

Editing, diagrams, etc. ×2

Total days 64

Code complete July 15, 2011

That completion date assumes I start this coming Monday (April 11, 2011). The schedule assumes a 5-day rather than a 7-day week, but that's because the longer hours would actually be less productive (see Evan Robinson's "Why Crunch Mode Doesn't Work" for a summary of the research). 1500 finished words per day, and the factor of two for editing and diagrams, are both based on my experience with several previous books, so there's not much give there.

Harder Than It Should Be

Thu, 31 Mar 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Someone once said, "Chemistry is basically anything chemists will give each other awards for doing." Or something like that—Google doesn't find matches for that exact quote. Even if I've mangled it, the idea is sound: art is no more and no less than what great artists accept as being art.

So what is computer science? More particularly, what constitutes the core of computer science? What's the stuff that everyone who calls themselves a "computer scientist" should know, or at least have seen? One way to answer the question would be to look at what people are given prizes for, but that's turning out to be harder than I expected, and the reason highlights a gap in this course.

Using Bein

Thu, 31 Mar 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Many thanks to Frederick Ross for putting together a short screencast on Bein, a workflow manager and miniature laboratory information management system (LIMS) built in Python that fills the gap for the working scientist between the classical shell and big workflow managers like Galaxy and major LIMS systems like OpenBIS. Please have a look and see if it could help make your computational life easier.

Practical Computing for Scientists at Stanford

Wed, 30 Mar 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Prof. Risa Wechsler writes from Stanford:

We are teaching a course at Stanford this term inspired by Software Carpentry. The course, "Practical Computing for Scientists", will be a Student Initiated Course, led by two physics undergrads (former summer research students with my group)—the course will be targeted at physics undergrads with a bit of programming experience to prepare them for summer research, but we expect participation from undergrads from other fields and some grad students and postdocs as well. I am hopeful that we will be able to keep this going as a regular student-led course with richer material as it develops. We expect that we will use some of your materials as well as some materials developed here.

Spring 2011 Course Over

Wed, 30 Mar 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Spring 2011 course is now over! We had a wonderful time, learned a lot, and hope the same is true for everyone who participated in the course.

We would love it if you would take a minute or two and give us any comments, feedback, and suggestions you have regarding the course and your experience in it.

To start the ball rolling, Erin Osborne gave us some awesome feedback, posted with permission:

And I'm on a Horse

Sat, 26 Mar 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Patrick Mackenzie (whom I've never met) gave a good lightning talk at the Business of Software that sums up a lot of what we haven't done for Software Carpentry. Any reworking of the material really (really) has to be built around what you (the researchers) need, rather than what we (the programmers) know.

A Better Way to Teach Programming to Scientists

Thu, 24 Mar 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This year's SIGCSE conference on computer science education featured a very cool paper by Robbins, Senseman, and Pate, of the University of Texas at San Antonio, called "Teaching Biologists to Compute using Data Visualization". In it, they describe CS 1173: Data Analysis and Visualization using MATLAB, which introduces students in the life sciences to programming using a problem-first approach. Like the media-first approach that Mark Guzdial and his colleagues introduced at Georgia Tech, this rewards students right away—they don't have to wade through a "CS first" morass of data types and arcane rules about Boolean expressions for three or four weeks in order to get to the useful bits. Given another year, I'd have rewritten our intro to Python this way...

Our First Episode on Microsoft Access

Wed, 23 Mar 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We have just posted our first episode on using a database with Microsoft Access — many thanks to Utah State's Ethan White for creating it.

I'd Settle for 0.1%

Tue, 22 Mar 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

In a recent article about computational thinking, Carnegie-Mellon's Jeannette Wing says:

...every scientific directorate and office at the National Science Foundation participates in the Cyber-enabled Discovery and Innovation, or CDI, program, an initiative started four years ago with a fiscal year 2011 budget request of $100 million. CDI is in a nutshell "computational thinking for science and engineering."

0.1% of that would keep Software Carpentry going for another year... Or if I'm allowed to be a curmudgeon for a moment, hands up those people who believe that $100 million is going to do 1000 times more for science and engineering than another year of work on these materials? *sigh*

You'll Need a Large Screen

Tue, 22 Mar 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I've been working on a graph showing

the connections between the questions this course tries to address,

our answers to them,

the knowledge and skills needed to understand and apply those answers, and

the big concepts behind that knowledge and those skills.

The latest version is linked from the thumbnail below. I know it's tangled, but I hope it's the first step toward a roadmap for redesigning this course to serve your needs better. Feedback and assistance would both be very welcome.

Using a Debugger

Mon, 21 Mar 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We've just uploaded a new video showing how to use a debugger to track down a problem. Using a debugger instead of 'print' statements will save you a lot of time, and the skill transfers directly to pretty much any language. Please let us know what you think.

Videos of Autumn School Lectures

Mon, 21 Mar 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I'm pleased to announce that video recordings of the Software Carpentry lectures I gave in London last fall are now online at http://soundsoftware.ac.uk/autumnschool2010video. Please excuse the mustache—I don't know what I was thinking... Many thanks to Chris Cannam, Ivan Damnjanovic, Luis Figueira, and of course Mark Plumbley.

On a Personal Note...

Fri, 18 Mar 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Since money hasn't materialized for another year of full-time work on this course, I'm now looking for a job. My CV is up to date, and my interview shoes are polished, so if you know of anything meaningful and interesting in downtown Toronto [1], please give us a shout. I'd obviously prefer something related to scientific computing, Python, and/or education, but wherever I wind up, Software Carpentry will keep going. As always, if you'd like to help with that, please get in touch.

Questions and Answers

Thu, 17 Mar 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A first cut of a question-and-answer matrix for this course is now up for viewing. It doesn't include everything, and the answers aren't particularly helpful without a second matrix showing how they connect to specific lecture topics and big principles, but I hope it's a start. Please give feedback as comments on this post; we'll add more Q's and A's in the coming days.

Graph Layout, Models vs. Views, and Computational Thinking

Wed, 16 Mar 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Money for me to keep working full-time on Software Carpentry hasn't materialized, so as I've mentioned in a couple of recent posts, I'm trying to find a way to organize the course material more coherently in the hope that other people will (finally) start to contribute content as well. As part of that, I have spent the last two days trying to draw a graph showing the questions Software Carpentry seeks to answer, and the concepts that underpin the answers. It's been harder than I expected, and I think the reasons might give some insight into how a computational thinker thinks.

Next-Generation Sequencing Course at MSU

Wed, 16 Mar 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Analyzing Next-Generation Sequencing Data

June 6th — June 17th, 2011

Kellogg Biological Station, Michigan State University

Instructors: Dr. C. Titus Brown, Dr. Ian Dworkin, and Dr. Istvan Albert.

Applications must be received by March 25th for full consideration. See the full announcement at http://bioinformatics.msu.edu/ngs-summer-course-2011 for more information.

Course Description:

This intensive two week summer course will introduce students with a strong biology background to the practice of analyzing short-read sequencing data from Roche 454, Illumina GA2, ABI SOLiD, Pacific Biosciences, and other next-gen platforms. The first week will introduce students to computational thinking and large-scale data analysis on UNIX platforms. The second week will focus on mapping, assembly, and analysis of short-read data for resequencing, ChIP-seq, and RNAseq.

Call for Participation

Tue, 15 Mar 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Are you a computational scientist (but not a computer scientist) who develops scientific software? That is, do you use mathematical models to describe scientific processes and then implement these models in your software?

Is your main reason for software development to advance science?

Is your software used by a wider community of scientists (and maybe not only scientists)?

Would you be willing to spare about one hour of your time to participate in a PhD study?

Twenty Questions (Minus Two)

Tue, 15 Mar 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Following up on last week's musings about reorganizing the course, we've drawn up eighteen questions that we think cover the reasons people come to this course. That leaves us two short of the traditional twenty: what would you add to this list?

How can I automate this task?

How can I read/control hardware?

How can I control my program?

How do I count things?

How can I clean up this data?

How can I get insight into my data?

How do I track down bugs in my program?

How can I parse this legacy data file?

How can I save data so that I can read it later?

How can I tell if my program is working correctly?

How can I make this program easier to use?

How can I use remote machines?

How can I reuse this legacy program?

How can I share my work with others?

How can I make my program faster?

How can I make my code easier to reuse?

How can I keep track of my work?

How can I plug my code into a framework like Galaxy or Taverna?

What To Demand

Sat, 12 Mar 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Peter Norvig (formerly of NASA, now at Google) recently gave a talk titled "What to Demand From a Scientific Computing Language". It's a good talk (and not just because he explains why he's a fan of Python). I was a bit disappointed, though, by this list:

Shouldn't there be a version control system on this list? And some sort of provenance tool? OK, that's a trick question: there aren't any provenance tools in widespread use, but what about testing tools? There are lots of those, and there's even a name ("xUnit") for the whole JUnit-style family of tools across different languages. Saying "these aren't really core to scientific computing" is sort of like saying "disinfection isn't really core to surgery".

Musing About Reorganization

Fri, 11 Mar 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I'm increasingly unhappy with the organization of this course. On the off chance that funding materializes and we're able to undertake a major redesign, I'd like to explain why and ask for your input.

Right now, our lectures are broken into topics along lines a computer scientist would instantly recognize: basic programming, regular expressions, databases, and so on. That is not how members of our intended audience see things when they first come to us—if it was, they probably wouldn't need this course. They start with problems like:

Science Illustrated

Fri, 11 Mar 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Videos of lectures from a two-day symposium on scientific visualization called Science Illustrated are now online. Many thanks to Mubdi Rahman for organizing it.

Advanced Scientific Programming in Python

Wed, 09 Mar 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Advanced Scientific Programming in Python

A Summer School by the G-Node and the School of Psychology, University of St Andrews

Scientists spend more and more time writing, maintaining, and debugging software. While techniques for doing this efficiently have evolved, only few scientists actually use them. As a result, instead of doing their research, they spend far too much time writing deficient code and reinventing the wheel. In this course we will present a selection of advanced programming techniques, incorporating theoretical lectures and practical exercises tailored to the needs of a programming scientist. New skills will be tested in a real programming project: we will team up to develop an entertaining scientific computer game.

High Tech That Looks Low Tech

Wed, 09 Mar 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

These videos about climate change are great: they look low tech, with hand-drawn diagrams and low-fi narration, but I imagine a lot more work went into their animation than into our slideshows.

Literate Programming

Mon, 07 Mar 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Last week's post about the tuple space programming model was so popular that I thought readers might enjoy a discussion of another beautiful idea that failed: literate programming. Like Lisp and other toenail-based languages, it inspires a kind of passion in its fans that is normally reserved for gods, sports teams, and angsty rock bands. And, like them, it leaves everyone else wondering what the big deal is.

Tuple Spaces (or, Good Ideas Don't Always Win)

Tue, 01 Mar 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I've resisted adding a module on high-performance computing to this course for a lot of reasons: I think other things are more important, there's enough coverage elsewhere, the software is hard for novices to set up... But there's another reason, one that may not be as good, but still has a seat at the table. Deep down, the reason I'm reluctant to teach MPI (the de facto standard for parallel programming) is that there's a much better model out there, one that works on all kinds of hardware, is comprehensible to novices, and delivers good performance on a wide range of problems. Its name is tuple space, its most famous implementation is Linda, and unfortunately, for a lot of reasons that I still don't understand, it somehow became an "also ran" in parallel programming.

We Got a Mention in Comm. ACM

Fri, 25 Feb 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The March 2010 issue of Communications of the ACM has an article on grid computing that mentions Software Carpentry and Titus Brown's course at Michigan State—yay! Unfortunately, Software Carpentry didn't get a URL, and the bit.ly URL for Titus's course currently redirects to the wrong place.

Later: the bit.ly URL is apparently working again.

An Easy Place to Start: Systems Programming

Thu, 24 Feb 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As a follow-up to recent posts on how to contribute, what better looks like, and Elango Cheran's secure shell episode, here's a specific request: we would like three episodes on systems programming, by which we mean writing programs to:

manipulate files and directories (chdir, stat, etc.)

working with archives (tar, zip, etc.)

run other programs (the subprocess module in all its fearsome glory)

A typical use case is writing a Python script to run a compiled legacy application a few hundred times for slightly different parameter values, putting each run's output in a separate directory that's created on the fly, and so on. If you're interested, please get in touch — it's easier than you think.

Ask, And Ye Shall Receive

Wed, 23 Feb 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Thanks to Elango Cheran, we now have an episode on the secure shell—yay!

What Better Looks Like

Tue, 22 Feb 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Paradise is exactly like where you are right now... only much, much better.
— Laurie Anderson

It's hard to make things better if you don't know what "better" looks like, so in the wake of some email responses to last week's post on how to contribute, here are some thoughts on what we'd like Software Carpentry to look like when it's finished (or as finished as something like this ever is).

Three More Episodes on MATLAB

Sat, 19 Feb 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Three more episodes on MATLAB are now available, two on input and output, and one on plotting. Thanks to Tommy Guy for putting them together; as always, feedback is appreciated.

Mirroring Software Carpentry

Fri, 18 Feb 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As another followup to the recent post on how to contribute, I'd welcome pointers to places that might host or mirror Software Carpentry material. Efforts like MIT's OpenCourseware and CMU's Open Learning Initiative seem to host only the content they produce themselves; the Khan Academy seems open to contributions, though I don't see any, and then there's Academic Earth. Where else could we put our content to get it in front of more people?

Scientific Computing Podcast

Fri, 18 Feb 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

inSCIght is a podcast that focuses on scientific computing in all of its various forms. Every week we have four panelists engage head-to-head on poignant and interesting topics. The panelists are drawn from all across the scientific computing community. From embedded systems experts to very high level language gurus, from biologists to nuclear engineers, the hosts of inSCIght use computers to solve science and engineering problems everyday. This podcast throws people, ideas, and opinions into an audio-blender hoping to educate and entice each other and the world.

Reddit on Scientific Programming

Thu, 17 Feb 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Nature's article on scientific computing has attracted 75 comments on Reddit. Some interesting ideas being kicked around; please go ahead and add your own.

How to Contribute

Wed, 16 Feb 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Following up on yesterday's post, I have written a short guide explaining how you can help Software Carpentry improve and grow. Reporting bugs, suggesting changes, contributing content, or running the course yourself: they're all a lot easier than you think, and a great way to gain some experience with new kinds of teaching and learning that are becoming more important with every passing year.

I Want Their Software

Wed, 16 Feb 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This blog post about the Khan Academy might gush a bit, but the embedded video is a must-watch for anyone who cares about education. I really want their software...

First Four MATLAB Episodes

Tue, 15 Feb 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We have just posted the first four episodes of our introduction to MATLAB; more should be up this week and next. As always, feedback is very welcome.

Top Ten Why Nots

Tue, 15 Feb 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We've had quite a few firsts and successes in the past ten months, but our biggest failure continues to be the lack of contributions from users and educators: while quite a few people are using our material, only one person has (so far) volunteered to create material for us. If that doesn't change, Software Carpentry will stagnate as soon as I have to move on to other things (which, given my lack of success in raising another round of funding, will probably happen this spring).

Audio for Three Software Engineering Episodes

Mon, 14 Feb 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We've added audio (and a few images) for three of the episodes on software engineering (the introduction, agile development, and the principles of computational thinking). It's a slightly different format than most of our previous episodes, and we need to choose a better way of embedding MP3 audio in web pages, but we hope it's useful as is.

Two More Episodes on Spreadsheets

Mon, 14 Feb 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We have added two more short episodes on spreadsheets, covering pivot tables and named cell ranges. Please let us know if you find them useful, and what else we should cover to help researchers who use spreadsheets to work with data.

Updates to Spreadsheet Lecture

Fri, 11 Feb 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We have just posted updates to the first seven episodes of the lecture on spreadsheets: the new version uses Excel 2010, and the screencasts are larger to make things easier to see. Please let us know what you think.

What Computational Science Means to Me

Tue, 08 Feb 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

My latest attempt to define what "computational thinking" actually means is now on the web in draft form—comments would be very welcome. In brief, the eight principles are:

It's all just data.

Data doesn't mean anything on its own—it has to be interpreted.

Programming is about creating and composing abstractions.

Models are for computers, and views are for people.

Paranoia makes us productive.

Better algorithms always trump better hardware.

Automation is the key to acceleration.

The tool shapes the hand.

Scripts for Two More Software Engineering Episodes

Thu, 03 Feb 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We've posted the scripts for two more episodes on software engineering (but not the slides or videos, since they don't exist yet). I'm not happy with the discussion of agile: it feels like too much has been left out, but I'm not sure what to add without breaking our self-imposed 10-minute limit. I also don't know how much our audience needs to know: these episodes are meant to be end-of-course pointers to other material they could look at next, so we're after a big-picture guide to "should I dig deeper" rather than the details. As always, feedback is welcome...

Three Months, Two Spikes, One Conclusion

Wed, 02 Feb 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Here are the traffic stats for the last three months at software-carpentry.org:

November 2010

December 2010

January 2011

It's hard to make out what's happening because of the big spike in December, and the less prominent (but still significant) spike in November, so here's a summary:

Once these spikes are removed, the average number of distinct visitors is slowly increasing each month.

Posting a lecture on something popular boosts our readership dramatically, but only briefly, and only for that topic—most of those visitors don't stick around.

So what were the spikes? The first, in November, came when we posted our Python lectures. The second, in December, was Tommy Guy's episode on how to build a simple recommendation engine, which in turn was based on an example from Toby Segaran's excellent book Programming Collective Intelligence. That was more popular than anything else we've ever put on the site, which is both encouraging (if we build it, you will come) and disheartening (I have no idea what else to build that will be that popular).

First Episode on Software Engineering

Tue, 01 Feb 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I know we promised a lecture on high-performance computing, but today wound up going in a different direction: we have just posted an episode on empirical results in software engineering, which we hope will set the stage for discussion of how to manage larger teams and projects. We hope you enjoy hearing about some of the science behind the things this course teaches; as always, feedback is welcome.

A Competence Matrix for Software Carpentry

Mon, 31 Jan 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Last summer, we tried to create a concept map showing how the big ideas in Software Carpentry related to each other. We weren't satisfied with it, so we set it aside. With our first anniversary coming up in three months, though, it's time to revisit the idea of a high-level overview of the course. To that end, I have posted a first attempt at a competence matrix that organizes skills into levels. Please leave comments on that page telling me what's wrong or missing.

Bootcamp

Thu, 27 Jan 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A couple of weeks ago, I went to the University of Wisconsin — Madison to speak at a three-day software skills bootcamp run by The Hacker Within, a grassroots student group that's taking the same "teach what you know, learn what you don't" approach as the Peer-to-Peer University and other efforts. I was very impressed then, and am even more so now that I've had time to reflect on what I saw (and recover from my close encounter with Delta Airlines, whose motto appears to be, "We don't care because we don't have to").

Notes Toward a Lecture on High-Performance Computing

Thu, 27 Jan 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As I've said several times now, we're going to add a lecture on high-performance computing to the course. We have rough outlines of the content in the version control repository, and feedback would be very welcome.

Introduction

The Example Application

Improving Sequential Performance

Profiling

Parallelism Defined

Computer Architecture in Ten Minutes or Less

Task Farming

Basic MPI

Collective Operations in MPI

Research Study: How Do You Test Your MATLAB?

Thu, 27 Jan 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Have you ever wondered how scientists test their code? We have, and we'd like you to help us find out. If you use MATLAB, please have a look at our new research project—and please forward the link to other groups and lists where we might be able to recruit people. (Of course, we'd also welcome input from NumPy users...)

See also these posts from Steve Eddins, who built the MATLAB xUnit testing framework, and thanks in advance for your help.

Fighting Spam

Wed, 26 Jan 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We recently experienced a spate of spam and fake user accounts on the course discussion forums. In this post I'll briefly explain what we're doing to stop that from happening again, and how the technology works.

The course forums are powered by bbPress, the forum software written by the WordPress folks. When I first installed it I left it with the default configuration settings and without doing anything in particular to lock it down from spammers. Bad idea. When I checked this morning we had over fifty fake user accounts, and two spam postings.

Software Carpentry Sprint in July

Wed, 26 Jan 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Well this was a nice birthday present: the Python Software Foundation announced today that they will provide some support for a Software Carpentry sprint in Toronto in July. Thanks, folks—we'll post details about what, when, and where as soon as we have them.

The Case Against Peer Review

Wed, 26 Jan 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Cameron Neylon recently made the case against peer review once again; the dialog near his posting's end is too accurate to be funny. In this light, calls for peer review of scientific software are probably misplaced; we need a better mechanism than one that's already mostly failing.

Thinking Like the Web

Wed, 26 Jan 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Jon Udell's 1999 book Practical Internet Groupware was a revelation for me: it was the first coherent explanation I'd ever read of how the disparate collection of technologies and social conventions that we call "the web" fit together, and what the deeper patterns and concepts beneath them are. After a lot of further work and thought, Jon has condensed those ideas into seven principles—or as he puts it, "Seven Ways to Think Like the Web". These concepts are the most meaningful definition yet of what the phrase "computational thinking" actually means, and of what people who aren't programmers need to know in order to use the web effectively. As I said in the post on Tom Limoncelli's plea to software vendors, we'll know we're teaching the right things, the right way, when people who have done this course understand these principles and how to apply them.

Scientists Aren't Stupid: Software Is

Fri, 21 Jan 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Last night, Mike Bayer (@zzzeek) tweeted:

Why are "scientists", who are so dramatically smarter than me, such dumdums when it comes to basic programming skills: http://bit.ly/fWPtjW

Taavi Burns (@jaaaarel) replied:

Ask @gvwilson of @swcarpentry?

So here I am, and what I want to say is:

Scientists aren't stupid: software is.

Seriously. I spent an hour and a half last night trying and failing to get Thunderbird and Dreamhost's email to play nicely together. Today, I'm wrestling with the fact that Python's multiprocessing library gets lost in infinite recursion if you try to do something crazy like, oh, I don't know, leave out the if __name__ == '__main__' check (but only on Windows). In both cases, the parties involved can explain why it does what it does, and in both cases, I just... don't... care.

How to Cite Software Carpentry

Thu, 20 Jan 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

If you are citing Software Carpentry in papers or technical reports, the web site and the 2006 article in Computing in Science & Engineering are probably best; the other articles below provide other background that might also be useful. I hope to update the 2006 article some time this year, but a lot of other things are ahead of it in queue...

MIT Rethinking OpenCourseWare

Thu, 20 Jan 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

MIT's OpenCourseWare initiative was (and probably still is) the highest-profile "open and online" initiative in higher education. According to this article, they're rethinking their approach to help the self-directed learners who have emerged as their biggest users. I'm really excited by this, and by P2PU's peer-to-peer model, and all the others that are emerging (including the hybrid models we're trying out). I also think that asking "which model will win?" misses the point: each one is a best fit for a different combination of educator, content, and learner.

Making System Administrators' Lives Easier

Wed, 19 Jan 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Google's Thomas Limoncelli (author of Time Management for System Administrators) has an article in ACM Queue titled "A Plea to Software Vendors from Sysadmins—10 Do's and Don'ts". His list is:

Do have a "silent install" option.

Don't make the administrative interface a GUI.

Do create an API so that the system can be remotely administered.

Do have a configuration file that is an ASCII file, not a binary blob.

Do include a clearly defined method to restore all user data, a single user's data, and individual items.

Do instrument the system so that we can monitor more than just, "Is it up or down?"

Do tell us about security issues.

Do use the built-in system logging mechanism.

Don't scribble all over the disk.

Do publish documentation electronically on your Web site.

Every one of these applies equally to scientific software, big and small. We'll know we're teaching the right things, the right way, when people who have done this course know how to write software that obeys these rules and (more importantly) understands why they should.

Version Control and Newline Conventions

Wed, 19 Jan 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

There are two widely used conventions for representing the end of a line and the start of the next. Unix-like systems have traditionally used one character, \n (ASCII 10, "line feed"), while Microsoft DOS and Windows used a sequence \r\n (ASCII 13, "carriage return", \n). Being invisible, the difference of conventions usually goes unnoticed, specially because most modern software can correctly display text written with either one. However, there can be interoperability issues when using editors that follow different conventions.

Exercises for Shell Posted

Tue, 18 Jan 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Orion Buske has posted exercises to go with the lecture on the Unix shell. These will get you to do real things with real bio data; please post questions and comments in the forum as you work on them.

Demographics (part two)

Mon, 17 Jan 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Here are summaries (slightly edited) of what people taking the course do. There's quite a range...

Stem cell biology, high-throughput screening, high-content analysis.

My research aims to improve the resolution of ultrasonic images created during ultrasonic inspections of thin metal sheets and metal welds though the use of digital signal processing algorithms.

I am currently working on learning Django to create websites. I am also developing new course content in programming and scientific computing for a 4-year degree in GIS.

I do simulations of protein folding and assembly for naturally disordered proteins, the kind that are implicated in degenerative diseases like Alzheimer's and Parkinson's. I also use dimensionality reduction techniques to find clusters in the conformation spaces of the proteins.

I am working on computational simulation, trying to understand the basic rules underlying protein dynamics

I am currently working on modeling the accretion disks of cataclysmic variable stars. I also study High Mass XRay Binaries, as well as Low Mass XRay Binaries.

We build and maintain global databases of fisheries statistics and analyze this information to uncover ways by which we can manage global fishery resources more sustainably.

Use classical molecular dynamics running on clusters and supercomputers to simulate the dynamics of individual proteins from a variety of species. In particular I focus on proteins that are embedded in the cell membranes as these are involved in many different important physiological functions e.g. transport and cell signalling.

Technical solutions to health and safety problems

I am a molecular biologist and study how different genes affect cellular phenotypes. For example, I switch off specific genes (alone or in combination) using RNAi. Typically, I perform thousands of experiments using automated liquid handling stations (robots) and record the resulting phenotypes (typically as numbers). In addition, I study how gene expression is affected using next-generation sequencing techniques (RNAseq).

Chemical health and safety, mostly data analysis of biological monitoring (that's urine and blood samples); creating software tools to help non-mathematicians run mathematical models (check out http://www.advancedreachtool.com/); some PBPK modelling (systems of differential equations of how chemicals move through the body).

We study how information about sensory stimuli is transduced, transformed, and represented in the central nervous system. At present we focus on such processes in the early visual system.

I am currently working on the fluid dynamics of blood droplet impact using a simulation method called Smoothed Particle Hydrodynamics. I am currently writing my code in Matlab, but may need to move to C or C++ to increase the speed of the computations.

Developing algorithms for neuroimaging data analysis. This involves implementation of image processing algorithms as well as statistical tools.

I am using statistical downscaling, which involves different types of multi-variate regressions, to predict surface observations from information provided by course resolution global climate models. Currently I am working on a project in which we are using downscaling to make future projections of the severity and likelihood of wildfire in BC. I am using R almost exclusively for this project.

Running hydrologic models in a unix environment. Coding in R to analyze outputs including plotting and mapping. Querying data from netcdfs and adjusting code in a statistical downscaling model that can be in c, fortran, or shell.

On the bioinformatics front I am comparing the evolution of the overlapping regions of the 4 genes from Hepatitis B Virus. I have used python to cut, copy and curate the sequences downloaded from Genbank to get a useable and well annotated dataset. Quite a task considering the sorry state of the data in the HBV database.

I do research to understand how we recognize speech and how we learn to recognize speech. By speech recognition, I mean the mapping from the real-valued speech signal to abstract linguistic structure, like sentence structure and meaning. Methodology includes recording speakers of different languages, running psychological speech perception experiments on adults and infants, and computational modeling using machine learning algorithms.

My research is in the fields of hydrology and aquatic ecology. I use both field and model approaches to understand the linkages between physical and chemical hydrologic processes and the resulting impact on aquatic ecosystems.

At Census Bureau: Small area estimation for local government surveys. Ongoing: Statistical applications of specialized optimization methods; statistical modeling of systems of differential equations.

Working on a Brownian dynamics simulation of secreted protein mass transport from adherent mouse embryonic stem cell (mESC) culture with perfusion of laminar flow; culturing mESCs for self-renewal or differentiation induces the secretion of many different signalling proteins.

I'm working with a database of location and health information for 20,000 diabetes patients in California and trying to evaluate how/if their health condition is related to their neighborhood environments. I work with GIS to characterize their access to healthy and unhealthy food resources, parks, public transportation, and other environmental factors as the data becomes available.

Applied Statistics with focus on Statistical Genetics.

Numerical simulation of an impacting drop on to a solid surface with an open cavity which has a wide range of application such as optimizing the production cost of water repellent fabrics, predicting the quality of arc welding, improving printing quality... Long story short, application of CFD in simulation of multiphase fluids processes.

The physics for blood flight in blood pattern analysis. Ultimately we're trying to create a program that can recognize blood stains and draw the path of their flight back to their source.

Attention and working memory. Currently attempting to program a useful field of view task, and an experiment that tests memory for facial features.

I study stem cell bioengineering, but try to bring a computational twist into things. So I'm working on models of cell proliferations and differentiation, and starting to look at some optimization methods to help with the engineering aspects of many projects within the lab.

Investigation into the role of international seafood trade in the expansion of the world's marine fisheries.

There is a lack of surface markers used to classify early cardiac progenitor cell types during differentiation from mouse embryonic stem cells. My project is to use mass spectrometry and microarrays to identify surface markers (and other proteins) involved in the differentiation process. Using bioinformatics and potential cell-cell interaction modeling, I would like to mine the data to extract potentially biologically relevant data pertaining to this process.

I'm studying the effects of ocean acidification on marine organisms. This involves constructing a lab to manipulate and monitor water chemistry. Currently, I'm trying to understand if growing under different chemical conditions results in biological materials with different structural properties.

I'm working on the molecular evolution of the small subunit of the prokaryotic ribosome. This is a structural informatics project dealing with analyzing 2 and 3-dimensional molecular structures and implementing energy algorithms for ribosomal RNA molecules. My current stage of work involves wetlab RNA production and functional assays on RNA fragments produced in the previous stage of analysis.

Implicit Large-Eddy Simulation (ILES) and Direct Numerical Simulation (DNS) of transition to turbulence. Looking at both the physics of transition and the numerical methods require to model it accurately.

I am trying to develop a new way of stimulating paralyzed muscle. I am using a technology called surface Functional Electrical Stimulation, which uses electricity to contract skeletal muscle. This method is very rough and does not give precise joint torque vectors. I am looking for a way to use this technology such that the joint torque created by the contracting is precise and repeatable.

I'm currently studying the mechanisms by which neurons in the superior colliculus selectively respond to some visual stimuli but not others. What are the properties of these neurons and the input they receive that allow them to respond to specific sensory features?

My work involves statistical downscaling, a way of relating future projections made by climate models at the large scale to the scale of weather stations. Climate models work well for broad studies of the climate system but are poor at describing small scale effects such as the climates of cities because of their low resolution (grid cells with sides on the order of hundreds of kilometres). In downscaling, statistical functions are used to relate the low resolution model data to weather station observations during past times. These functions can then be used to relate future climate projections to the smaller scales, yielding projections with much greater detail.

Currently focusing on Application Security; identifying Cyber threats and mitigation strategies. Would like to become more proficient in programming and design to be able to develop my own Application Security tool set.

A central theme in my research is the development of advanced (model-based) process control methodologies on the basis of such observed data, with a particular focus on chemical and metallurgical systems. This involves extracting complex, nonlinear regularities and trends in measured plant data. Subsequently, models are derived for the purposes of, among other, nonlinear system identification, process diagnostics, as well as insights into underlying process behavior. To this end, a number of mathematical and computational techniques are used including: * Nonparametric learning methods such as kernel methods, ensemble methods and artificial neural networks * Nonlinear time series analysis * Multivariate statistics and other exploratory data analysis techniques.

I primarily do Computational Fluid Dynamics (CFD) modelling of combustion systems, ranging in size and application from pilot/experimental scale to industrial/power utility scale. The experimental scale work tends to be in new technologies, such as gasification and high pressure combustion, whereas the industrial scale work tends to look at design and/or operational issues in existing systems.

For my PhD, I investigated a type of piezoelectric ultrasonic motor. There were two parts to my research. The first part was the coupled axial-torsional vibration of pretwisted beams; I derived beam equations to model the structure and compared the predicted resonance frequencies with results from finite element method. In the second part I studied the nonlinear dynamical system of a disk bouncing on a vibrating platform as a model of the stator-rotor interaction of the motor.

My current area of interest is voting behavior and party strategies. I aim to combine two assumptions about voting behavior into one more general model. Such a heterogeneous voter model can tackle many empirical puzzles and explain political outcomes of democratic processes more accurate.

Supporting 300+ scientists in biotech research field.

I am doing research in statistical genetics. It is well known that genetic variants are related to many complex diseases. My work is to develop statistical methods for analysis this type of data.

Many of the most pressing issues in ecology require understanding how biological systems respond to global scale changes in climate and habitat. In order to understand these impacts it is necessary to study ecology at large scales. Research in my lab focuses on using quantitative macroecological approaches, including large ecological databases, advanced statistical methods, and theoretical modeling to understand broad scale ecological patterns.

Supporting clinical research data management at a commercial company. I'm about to enroll in a bioinformatics masters program with Johns Hopkins University and aspire to work on microbial ecological genomics (e.g. Human Microbiome Project etc).

I write image and statistical analysis software in Matlab, more of statistical analysis at the moment than image analysis.The SW is a GUI tool for users (mostly our team at the moment) to run statistical analysis on neuroimages image sets. The image sets are mostly PET, MRI, and fMRI studies.

Working on protein sequence analysis, phylogenetics, and evolutionary models.

I do modeling and theoretical work related to superconducting quantum bits. Previously, I modeled a particular design for a superconducting quantum bit. At present, I am trying to design a protocol to transfer the state of a superconducting quantum bit to another, via a superconducting transmission line.

I am working on methods for estimating the size of a genetic effect from large datasets in cases where the same dataset or same individuals is used for multiple purposes in sequence. Dataset sizes include 1M + variables, 2K + patients. Methods I'm using include bootstrap resampling, in which analysis of the dataset is repeated 10K-50K times.

My current research will provide musicians a system to compose and perform with a sound corpus by exploring a three-dimensional space by means of non-contact gestures. I am also planning to do research in automatic music recommendation and playlist generation.

I'm currently conducting psychiatric genetic research. This involves analyzing data from high-through DNA genotyping experiments.

The focus of my thesis is to develop an reinforcement learning (RL) agent for augmenting the supervisory controllers on an industrial grinding circuits. The majority of these circuits depend on expert systems (ES) for supervisory control, which are by in large more static than they are adaptive. My research is focused on augmenting these systems; specifically, to introduce online adaptability to ES using RL.

I am working on hydrologic modelling projects for several watersheds in British Columbia. This involves model set up, calibration (using multi-objective optimization tools), uncertaintly analysis. The models will be later forced with downscaled outputs from GCMs for future climate projections. I am also comparing the results with the RCM projections for the same catchment.

High throughput imaging system.

I am working on the connection between meteorological model output to hydrological model input for the purposes of water supply forecasting.

Resource modeling for healthcare systems.

I am a Political Scientist, working on policy processes in regional trade agreements. Statistically, I rely multivariate statistics, above all survival Models. In my most recent work I started to have a look at networks, too.

Clinical trail Epidemiology Study Genome Wide Association Study Data base management

CFD and Fluid-Structure Interaction studies of the repsonse of flexible bodies to various fluid loadings.

Solid state quantum computing, particularly superconducting qubits and doped diamond spin assemblies to build scalable quantum architecture.

I am working on Computational Fluid dynamics applied to reacting flows. Also interested in compressible flows over blunt body.

Customizing a Plone content management system for use in a parliamentary context, and using the Deliverance theming application to present two applications to the end user via a single unified user interface.

My research is focused on how neurons encode and decode odor information.

I am working on the background reduction for the PICASSO Experiment.

Human factors research. Statistical analysis on crash data. Explore driver distraction's effects on injury severities.

I am currently investigating different copy number algorithms by doing a large-scale evaluation of many different algorithms.

I am using bioinformatic methods to study microbial genomes and evaluate the effects of microbial community on ecosystem.

Bioinformatics support. I am responsible for the public accessible databases and maintaining and updating public tools like UCSC genome browser, Ensembl browser, Galaxy server for internal use.

I perform data analysis on high-throughput data from experiments designed to investigate the molecular mechanisms of how hypoxia (low oxygenation levels) in cancer tissues arises, and how cancer cells cope with this adversarial condition, with the hope that sufficient insight may eventually lead to novel therapeutic strategies. I work with both microarray and next-generation sequencing data.

I work in Statistical and Bioinformatics analysis of complex disease. My previous work focused on GWAS of longitudinal traits related to diabetes. Currently I work on sequence and expression analysis of head and neck cancer. My work involves a lot of figuring out how to work with massive datasets and complex algorithms.

I am a field ecologist switching to using digital evolution (Avida) for studies of evolutionary ecology. In my work, I look at questions related to the evolution of tolerance, group formation, and group stability, prerequisites for the evolution of sociality. My research includes developing experiments for in-silico testing of hypotheses of the evolution of cooperation and sociality which are based on in-situ observations of carnivore ecology.

Research on mesoscale variability in the California Current using quasi-Lagragian isobaric floats. The main focus is on westward transport off Central California coast, its kinematic characteristics and mechanisms. Besides Lagrangian data, we use satellite altimetry, model outputs, standard oceanographic data.

Housing market and monetary policy.

I have a wide range of research interests from understanding the effects of climate change to understanding anthropogenic effects on coastal systems. At the moment, I am working on developing models to help improve our understanding of the processes affecting marine snow and particle flux

I am developing techniques to perform deep sequencing on single cells. I am interested in how RNA transcripts partition asymmetrically after cell divisions critical in development. My research has a long-term goal of better understanding cell division processes that are typically mis-regulated in cancer cells.

I study phylogenetic and biogeographic relationships among terrestrial mammals. I am interested in how geography and evolutionary processes interact to produce and maintain biodiversity. I also do some research on phylogenetic methods and ways of analyzing geographically explicit DNA data.

'I'm studying how agricultural practices effect microbial communities and the impact that these changes have on greenhouse gas flux from soils. I'm using metagenomic data and a microbial ecology approach to determine the diversity and composition of these communities. I'm also developing approaches to analyze metagenomic data

Demographics (part one)

Sun, 16 Jan 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As near as we can tell, here's where this term's students are from. It's quite a mix...

BEACON Center for the Study of Evolution in Action, Michigan State University

Biostatistics Division, University of Toronto

Boston University

Centre for Addiction and Mental Health, University of Toronto

Department of Aerospace Science and Engineering, University of Toronto

Department of Biochemistry, University of Oxford

Department of Biochemistry, University of Toronto

Department of Economics, Queen's University

Department of Geography, University of British Columbia

Department of Mechanical Engineering, Polytechnic Institute of New York University

Department of Mechanical and Industrial Engineering, University of Toronto

Department of Microbiology and Molecular Genetics, Michigan State University

Department of Molecular Medicine and Haematology, University of the Witwatersrand

Department of Physics, Yale University

Department of Physiology, University of Wisconsin — Madison

Department of Process Engineering, University of Stellenbosch, South Africa

FTW Telecommunications Research Center, Vienna, Austria

Faculty of Science, University of Ontario Institute of Technology

Fisheries Centre, University of British Columbia

Friday Harbor Labs, University of Washington

Georgia Institute of Technology

German Cancer Research Center

Health and Safety Laboratory, UK

Howard Hughes Medical Institute

IMS, Inc.

Institute of Biomaterials and Biomedical Engineering, University of Toronto

Institute for Environmental and Spatial Analysis, Gainesville State College

James Franck Institude, University of Chicago

Janelia Farm Research Campus, Howard Hughes Medical Insititute

Lineberger Cancer Center, University of North Carolina

McGill University

Mechanobiology Institute, National of Singapore

Michigan State University

Nankai University

National Evolutionary Synthesis Center

National University of Singapore

Natural Resources Canada

Ontario Institute for Cancer Research

Pacific Climate Impacts Consortium, University of Victoria

Physics Department, California State University, Fresno

Predictek Inc.

SAIC

Samuel Lunenfeld Research Institute of Mount Sinai Hospital, Toronto

Stem Cell Bioengineering Lab, University of Toronto

US Census Bureau

United Nations Department of Economic and Social Affairs

Universit of North Carolina — Chapel Hill

University of British Columbia

University of California Berkeley

University of California Los Angeles

University of Georgia

University of Mannheim, Center for Doctoral Studies in Social Science

Utah State University

self employed

unemployed

Our Funding Pitch

Fri, 14 Jan 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A couple of people have recently asked, "How do you go about asking for money?" Applying for grants from NSERC, the NSF, and other agencies is something they understand (though it never actually worked for me), but cold-calling people, or emailing them out of the blue, isn't something most academics have ever done.

I'm now doing another round of this, and since this is supposed to be an open-everything project, I've posted the short pitch I use as a starting point when approaching someone new. I hope you find it useful, and of course, feedback is always welcome.

The Hacker Within

Fri, 14 Jan 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I got back to Toronto late last night from visiting The Hacker Within, a grassroots student organization at the University of Wisconsin — Madison that helps science and engineering students figure out how to use computing in their research. I think it's a great model for schools elsewhere to adopt, and despite truly awful "service" from Delta Airlines [1], it was worth going down to chat with Katy, Nico, Paul, and everyone else, and to see a large auditorium filled with people (80? more?) learning how version control, the shell, Doxygen, and other tools can make them more productive researchers. And as a bonus, I can now take "sleep in a frat house" off my list of 1000 things to do before I die... :-)

Slower Than Expected

Mon, 10 Jan 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We've started work on the high-performance computing lecture, and as part of that, we'd like to set a puzzle for our more advanced readers. We've written several versions of invasion percolation; the two of interest here use a list-based representation for the underlying grid, and a NumPy array. Run either program as:

python filename.py -g -n 31 -v 100

-g turns off the end-of-run graphical display; -n 31 tells the program to use a 31×31 grid; and -v 100 tells it to initialize cells to random values between 0 and 100. Try varying the size, compare the speeds of the two programs, see if you can figure out why one is faster than the other, and then tell us:

Software Carpentry in One Picture and Five Words

Mon, 10 Jan 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This 1995 Pirelli ad pretty much sums up what Software Carpentry is about:

The Spring 2011 Course Begins

Mon, 10 Jan 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The spring (or, if you'd rather, winter) run of the Software Carpentry course has started. We have 86 students currently registered. Here is the welcome email we sent out to the students late last week:

Hello,

Welcome to Software Carpentry. I'm Jon, one of the teaching assistants for this course, and I'm joined by TAs Orion Buske, Tommy Guy, Luis Zarrabeitia, and the course creator, Greg Wilson. In this email I'll tell you a bit about how the course is organised.

Funding (A Plea for Contacts)

Sun, 09 Jan 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I've been working full-time on Software Carpentry since the beginning of May 2010. As I said at the end of November, progress has been steady, but we're only at 60-70% of plan. I am therefore now looking for more funding so that I can continue developing material and delivering the course online this summer and fall. I would welcome pointers to potential backers, and introductions even more. (But please, only ones that are fairly focused: "have you tried the NSF?" or "there must be companies that need this stuff!" aren't actually very useful...)

First Half of Lecture on Object-Oriented Programming

Thu, 06 Jan 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The first four episodes of our lecture on object-oriented programming in Python are now available:

Introduction

Basics

Interfaces

Inheritance

There's (lots) more to come, but early feedback on these would be very welcome.

What I Learned From Software Carpentry

Thu, 06 Jan 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Anna Maidens, who works for the UK Met Office in Exeter and took part in our October 2010 Software Carpentry course there, did a presentation a few weeks ago to her team about what she learned. She has kindly given permission for us to post those slides; as always, we're eager to hear what other people have taken away from the course as well.

Software Carpentry Bootcamp Jan 12-14 in Madison

Fri, 31 Dec 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Registration is now open for a three-day Software Carpentry bootcamp Jan 12-14, 2011, at the University of Wisconsin — Madison, organized by the folks at The Hacker Within. I'll be speaking on the first morning, and hanging out the rest of the time trying to learn as much as I can. Look forward to seeing you there!

More Detailed Outline for HPC Lecture

Thu, 30 Dec 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I've added a more detailed outline for a lecture on high-performance computing to the site; feedback on the content and order would be very welcome, as would pointers to tools that are easy to teach with (as opposed to being powerful in the hands of experienced users).

Elimination

Mon, 27 Dec 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I'm working up another essay on software design, and would like to ask readers of this blog how they handle something that comes up when simulating interacting agents. If your program models the behavior of a flock of birds, it probably looks something like this:

# create birds birds = [] for i in range(num_birds): new_bird = Bird(...parameters...) birds.append(new_bird) # simulate movement for t in range(num_timesteps): for b in birds: b.move(birds) # need access to other birds to calculate forces

There's a flaw in this—or at least, something questionable. By the time you are moving the last bird for time t, every other bird is effectively at time t+1. There are many solutions, the simplest of which is to calculate each bird's new position in one loop, then update the bird in another:

Open Research Computation

Mon, 27 Dec 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

By now, many of you have (hopefully) seen the announcement of Open Research Computation, a new journal devoted to "...peer reviewed articles that describe the development, capacities, and uses of software designed for use by researchers in any field." The editorial board includes several friends of this course; as one of them, Titus Brown, observed in his blog:

...the problem with the online world for scientists [is] there's no real systematized incentive to any of this online stuff. And that makes it really tough. I'm going through Reappointment right now... Nowhere on there is there a place for "influential blog posts" — how would you measure that, anyway? Same with software — I listed my various software releases on the "scientific products" page of the form, and have since been asked to describe and discuss the impact of my software. Since I don't track downloads, and half or more of the software hasn't been published yet and can't easily be cited, and people don't seem to reliably cite open source software anyway, I'm not sure how to document the impact.

Local Subversion Repositories

Sun, 26 Dec 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A colleague in the UK who is going to teach Software Carpentry asked about setting up repositories. In particular, he doesn't have a server where he can create accounts and repos, so he was thinking of using Git or Mercurial, and having students host their repos on their own machines. That's not actually necessary: if you're going the locally-hosted route, and giving each student a separate repository, you can still use Subversion: just use the "file:" protocol for connecting instead of "http:" or "svn+ssh:". For example:

Extended Examples

Thu, 23 Dec 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We'd like to add more extended examples to this course, both because they're fun and because they're a good way to show how our topics relate to one another. Right now, we have:

an entire lecture on invasion percolation (which we'll shorten),

phylogenetic tree reconstruction, and

how to recommend papers to people.

We plan to add a simple N-body simulation (easy to do badly, motivates testing, visualization, and a bit of numerical analysis), but after that, we'd like to find some examples from psychology, linguistics, and other areas that are starting to do more computational work. Parsing text files full of patient records, putting the information in a database, doing some analysis using SQL, and visualizing the results is one possibility; what else would you recommend or like to see?

Compute Canada's 'Strategic' Plan Isn't

Tue, 21 Dec 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Last Friday—December 17—I received an email from Compute Canada. The emphasis is mine:

A Strategic Plan for Compute Canada was a key recommendation of the International Review Panel. This plan draws on suggestions from that panel as well as the information and discussions from the Town Hall Meetings held earlier this year. It has been many months in preparation and review by Compute Canada's committees and must be submitted to CFI before the end of December 2010... Your comments are invited and should be sent...by end of day Monday, December 20th.

Executable Papers

Mon, 20 Dec 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Elsevier is sponsoring an "Executable Paper Grand Challenge". If you have more than just ideas about the future of scholarly publication in computational science, it may be a good way to get them some press.

Building a Recommendation Engine with NumPy

Wed, 15 Dec 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Tommy Guy's explanation of how to build a recommendation engine in NumPy, based on an example from Toby Segaran's excellent book Programming Collective Intelligence, is now online.

Presents for the Holidays

Tue, 14 Dec 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Some of the best presents I have ever received have been recommendations: "Oh, you'd like this author," or, "You really should listen to this album." So, in the holiday spirit, please take a moment and share (in the comments) some pointers to computational resources that you think deserve to be better known. To get the ball rolling, I'm having fun playing with Ruffus, a simple Python library for constructing pipelined workflows. I'd also recommend Jonathan Weiner's Time, Love, Memory: A Great Biologist and His Quest for the Origins of Behavior. It's ostensibly a biography of Seymour Benzer (who spent his entire career exploring how genes determine behavior), but it's actually the best description I've ever read of how a successful long-lived research program actually works.

Slides for First Five OO Episodes Online

Mon, 13 Dec 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Slides for the first five episodes of our lecture on object-oriented programming are now online; we'd welcome feedback on the approach we're taking. If you like what you see, we'll try to have recordings up before the holiday break.

Performance and Parallelism

Fri, 10 Dec 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Some topics for a lecture on parallel programming:

how to measure/compare performance (raw speed, weak scaling, strong scaling, Amdahl's Law, response time vs. throughput)

the register/cache/RAM/virtual memory/local disk/remote storage hierarchy and the relative performance of each (order of magnitude)

in-processor pipelining (or, why branches reduce performance, and why vectorized operations are a good thing)

how that data-parallel model extends to distributed-memory systems, and what the limits of that model are

the shared-memory (threads and locks) model, its performance
limitations, deadlock, and race conditions

the pure task farm model, its map/reduce cousin, and their limitations

the actors model (processes with their own state communicating only through messages, as in MPI)

It's too much (each point should be an hour-long lecture in its own right, rather than 10-12 minutes of a larger lecture); what do we cut, and what's in there that doesn't need to be?

Winter 2011 Signup vs. Spam Filters

Fri, 10 Dec 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

If you asked to enrol in the Winter 2011 run of the course, you should have received an email request from us on Wednesday pointing you at a form we'd like you to fill in, and another message today clarifying one of the questions. You should also then have received a message saying, "Some people are reporting that they only got #2 of the above, presumably because their spam filters ate #1, so please check." If you didn't get Wednesday's message, or this morning's repeat, please contact us directly and we'll sort it out.

Where Are My Keys?

Thu, 09 Dec 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I was looking through some Python code a few days ago, and noticed that its author was using this:

if something in dict.keys(): dict[something] += 1

instead of:

if something in dict: dict[something] += 1

It seems like a small difference, but it's actually a very important one. The second form checks to see whether something is a key in the dictionary dict. The first form, on the other hand, creates a list of all the keys in the dictionary, then searches that list from start to finish to see if something is there. They both produce the right answer, but the fast form does it in constant time, while the slow form (the one with the call to dict.keys()) takes time proportional to the size of the dictionary. The really bad news is, the "error" is silent.

How Do You Manage a Terabyte?

Wed, 08 Dec 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This question has come up a couple of times, and I'd welcome feedback from readers. Suppose you have a large, but not enormous, amount of scientific data to manage: too much to easily keep a copy on every researcher's laptop, but not enough to justify buying special-purpose storage hardware or hiring a full-time sys admin. What do you do? Break it into pieces, compress them with gzip or its moral equivalent, put the chunks on a central server, and create an index so that people can download and uncompress what they need, when they need it? Or...or what? What do you do, and why?

Approaching Objects from a New Direction

Tue, 07 Dec 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We have just posted early drafts of slides for the first two episodes of our lecture on object-oriented programming. These take a "nuts and bolts" approach, showing more about how objects work than is usual in an introductory course in the hopes of dispelling some of the mystery and confusion that often surrounds them. Feedback would be very welcome...

Pins, Balls, and Arbitrary Decisions

Mon, 06 Dec 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We'd like to include more extended examples like invasion percolation in this course, but they're surprisingly hard to write. One that seems simple at first is a Galton box simulator. As the diagram below shows, this is simply a a vertical board with interleaved rows of pins. Small balls are dropped in at the top, and bounce left and right as they hit pins until they land in the boxes at the bottom. Since each bounce randomly goes left or right, the distribution of balls is binomial, which, in the limit, approximates a normal distribution.

Cast Your Votes

Thu, 02 Dec 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The solstitial holiday is approaching fast, but with a bit of luck, we should be able to get one more lecture up on the web before it arrives. Topics we could tackle are:

a survey of empirical results in software engineering,

building desktop GUIs,

handling XML, or

object-oriented programming.

(Web programming isn't on the list because it will take more time than we have.) If you have a preference, please cast your vote in a comment on this post...

Fall 2010: What Went Right, What Went Wrong

Thu, 02 Dec 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As we're getting close to wrapping up this semester's offering of Software Carpentry, we've been putting some thought into the things that worked and didn't work in terms of how we administered the course. Here's what we came up with:

What Worked Well

Version Control — Students really seemed to recognize the value of incorporating version control into their work practices. Of all the topics covered so far, we've received the most positive feedback about version control.

Midterm Quiz — This was useful for assessing the students' learning, and getting an idea of how many of the students were keeping up with the material.

Check-ins — About a month ago, we had each of the students meet with us on Skype for 10 minutes or so. Again, this helped us get an idea of how the students were doing with the course material, and also allowed the students to give us feedback on things they'd like to see changed/added to SWC.

TA Organization — Although we got off to a bit of a slow start, the Software Carpentry TAs managed to get organized in terms of the administrative aspects of the course. Division of labor, and explicitly assigning tasks to specific people seemed to work well.

Asking Students for their Problems — We spoke to a few students about the kinds of computer-related problems they are facing in their research. Next semester we hope to write weekly blog posts in which we will work through the solutions to the problems that the students are reporting.

They like us! — The feedback we've been getting from students indicates that they are enjoying the SWC course, and see the value of the material.

What Went Wrong

Peer to Peer

Thu, 02 Dec 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

One of the things we think didn't work well in this term's online run of the course was peer-to-peer discussion among students. Such discussion is one sign of a vibrant open source project or educational community, but our mailing list was almost silent except for announcements we posted ourselves, and other than a few bug reports, there were very few comments from students on the lecture pages. We think students will get more out of the course if they talk more amongst themselves for several reasons:

Prerequisites (or, When to Say No)

Thu, 02 Dec 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

How much should Software Carpentry assume students know before they start? Or to put a sharper point on it, how much should this course require students to know? To date, we have said that students should have at least one prior programming course, and have to understand loops, conditionals, functions, arrays, and simple file I/O: the kinds of things that are usually (but not always) covered in a CS-1 course, and hopefully remembered thereafter. To make this more concrete, we've said that students should be able to solve this problem in the language of their choice:

Programmer Competency Matrix

Thu, 02 Dec 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

There's no scientific research behind this tabulation of what programmers ought to know, and some of the categorizations are unactionably vague, but it's still a useful guide. Most of Software Carpentry is Level 1, with some Level 0 and Level 2 stuff blended in.

Red-R

Thu, 02 Dec 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

From a reader, a link to Red-R, a visual programming environment for R. The six-minute screencast on the documentation page shows what it's capable of. I don't know of any similar projects for Python—I'd appreciate pointers.

First Four Episodes on Multimedia

Tue, 30 Nov 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Our first four episodes on multimedia programming, which cover basic image processing, are now online—comments are welcome.

Winter 2011 Online Course Now Full

Mon, 29 Nov 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As of this morning, 102 people have signed up for our Winter 2011 offering—it will be the largest run of the course ever. Registration has now closed, but we will send out notices when we have dates for the next one.

Next Part of Persistence Essay Online

Fri, 26 Nov 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The second part of the discussion of persistence is now online. We're almost up to how Python's 'pickle' module works; should get there over the weekend or on Monday.

Hours So Far

Thu, 25 Nov 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The chart below shows cumulative hours on Software Carpentry since we started working on Version 4 in May. We're not as far along as I'd like, but progress has been pretty steady...

Four Episodes on Matrix Programming

Tue, 23 Nov 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Tommy Guy has recorded four introductory episodes on matrix programming with NumPy covering:

Introduction

Basic operations

Indexing

Linear algebra

Please have a look and let us know what you think.

Phylogenetic Trees

Tue, 23 Nov 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Elango Cheran has recorded an episodes for the sets & dictionaries lecture on reconstructing phylogenetic trees. It's a neat application of dictionaries to a problem that's usually explained as a matrix. Please check it out and let us know what you think.

Repository URL Change

Sun, 21 Nov 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We've been having problems hosting the course's Subversion repository at its current URL (http://software-carpentry.org/swc), and so we've moved it to:

http://svn.software-carpentry.org/swc

This URL points to the same repository as before. If you already have a working copy checked out you can switch it to use the new URL by running the following shell command in the top-level folder of your working copy:

Mid-term Quiz Results

Sat, 20 Nov 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A few weeks ago we gave a quiz to the SWC students on the first three topics of the course: Version Control, Spreadsheets, and Databases. We did this in addition to our student check-ins in order to get a sense of how the students were doing with the material. Half of the class (24 students) submitted their answers.

Results:
Everyone did really, really well! The majority of the mistakes were small SQL syntax errors, or other similarly minor things.

Now Annotated

Fri, 19 Nov 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We have added brief notes on the books in the course's bibliography—we hope you find them useful.

Summary of student check-ins

Thu, 18 Nov 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Over the last two weeks we've had short one-on-one conversations with the current students about how the course is going for them, and what they've enjoyed and haven't. We've heard from 30 students of the ~48 signed up in the course. For most students, their feedback was about the first three lectures (Version Control, Databases, Spreadsheets) and to some extent the Python lectures.

Making Software Screencast

Wed, 17 Nov 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A short screencast about Making Software is now up on Amazon. In each of the book's 30 chapters, leading researchers in software engineering explain what we actually know about code review, statistical prediction of software faults, how programmers communicate, whether using design patterns makes code better, the relative quality of open and closed source software, and a host of other topics. We hope you enjoy it.

New Section for Essays

Wed, 17 Nov 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

By popular request, we have moved the longer essay-style posts to a new section of this web site. We have also added a new essay on saving and loading data that also talks about metadata, registries, and the open/closed principle.

Ratios and Rework

Tue, 16 Nov 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

It's been six months and a bit since we started working on Version 4 of this course, so I'd like to share two things we've learned about creating online tutorial videos:

It takes a long time.

A lot of that time feels like it should be unnecessary.

Let's start with "a long time". It takes me 3-5 hours to prepare a good slide deck for an hour-long lecture that I'm going to deliver in person. A slide deck for a screencast takes 8-10, because I need to prepare more slides: since there isn't a lecturer making eye contact and pointing at things to keep viewers engaged, a slide deck for online use has to have many more small transitions.

Counting Things (Part 2)

Sun, 07 Nov 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This post has been moved.

Done In London

Fri, 05 Nov 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We wrapped up this week's class in London today: I think a lot of the students felt that they'd been in the wind tunnel most of the week, but the feedback was fairly positive:

Good

new content (hadn't seen before)

covered a lot (good to see things)

easy to apply / pragmatic

group exercises were fun / useful

Subversion

entertaining instructor (blah blah blah)

better understanding of best practices

persuasive arguments for best practices

nice to meet everybody!

accidental knowledge transfer (vertically and sideways)

Bad

Counting Things (Part 1)

Sun, 31 Oct 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This post has been moved.

Dubois on Maintaining Correctness

Sat, 30 Oct 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Something else I didn't get to at the Met Office last week:

Paul F. Dubois: "Maintaining Correctness in Scientific Programs". Computing in Science & Engineering, May-June 2005, http://doi.ieeecomputersociety.org/10.1109/MCSE.2005.54.

Please, please, please, if you're building scientific software of any kind, find a copy and read it, because it's the best explanation I've ever found of why the right way to do it is, well, the right way to do it. Paul ties together a banker's dozen good ideas under the banner defense in depth: no matter how scrupulous we are, any of these might fail, so we have others in place to catch the mistakes that creep through. To quote the list that opens the paper, they are:

Need Something to Debug

Sat, 30 Oct 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Following up on the previous post about the way Paul Dubois organizes good practices as "defense in depth", we would really like to include a lecture on debugging that does more than present a few general rules like "make sure you know what the right output is supposed to be" or "divide and conquer". More specifically, we need an example that contains a handful of qualitatively different bugs that can be found in qualitatively different ways: one by a code analyzer, another using coverage ("Wait, why isn't the 'else' ever being executed?"), a third by unit testing, a fourth by differential comparison of pre-change and post-change output, a fifth using breakpoints and single-stepping, and so on. If you have such an example, or are willing to help us construct one, please let us know.

Would You Prefer...

Sat, 30 Oct 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Here are three instructional videos that I enjoyed watching:

The Crisis of Credit Visualized

Math is Not Linear

Changing Education Paradigms

What they have in common:

They're informative.

They're fun to watch.

It must have taken a ton of work to create them.

So my questions are:

Do you like them more than what we're doing?

Are they "better enough" than what we're doing to justify the extra effort? We'd produce half as much material if we did this—would that be a good tradeoff?

Provenance (Or, What We Didn't Quite Get to at the Met Office)

Fri, 29 Oct 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This post has been moved.

Feedback at UKMO

Thu, 28 Oct 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I've just finished teaching a four-day version of the course at the UK Met Office in Exeter. I think it went reasonably well, even if we did version control last instead of first :-). Here's the feedback the students gave me:

Good

Most of what the instructor told us was probably true!

Liked the emphasis on the human aspects of software development

Liked the emphasis on evidence-based software engineering

Lots of details on how to do testing

Liked it when the instructor went off on tangents

Liked having pointers to more material

Instructor enthusiastic

Lots of stuff on course web site

Enjoyed learning a new language

Finally had someone explain why to make code readable

Nice mix of concepts and details

Modular course design allowed me to catch up when I got lost

Bad

How We've Helped

Thu, 28 Oct 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We have started to collect testimonials from former students; if you have a story to share, please send it in.

ComputerWorld Canada Educator of the Year

Wed, 27 Oct 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

ComputerWorld Canada has named Software Carpentry's Greg Wilson the 2010 IT Educator of the Year for "...recognizing the application of innovative techniques and development of new curriculum and delivery of programs that stimulate learning." Greg would like to thank Prof. Karen Reid and Prof. Marsha Chechik for the nomination; the students who have taken or worked on this course; and most especially, his mum and dad, who taught him the things that matter most.

Configuration Files

Sun, 24 Oct 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This post has been moved.

Slides Available as PDF and PPT

Thu, 21 Oct 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The home pages of the following lectures now have links to PDF and PowerPoint versions of the slides used in each episode:

Python

Sets and Dictionaries

Testing

The Unix Shell

Regular Expressions

Version Control

Make

Program Design

We haven't provided slides for the spreadsheets and databases lectures because they aren't slide-based. Please let us know if you find these useful, or if you notice any typos that we should fix.

Final Four Episodes of Python Lecture

Mon, 18 Oct 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The last four episodes of our lecture on Python are now available, covering:

libraries,

tuples,

slicing, and

what is text, anyway?

The last isn't really about Python, but it's something people trip over pretty quickly when they start using it. Hope you find them useful...

How Did You Find Us?

Mon, 18 Oct 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Hope you don't mind another administrative meta-question, but: how did you find this site?

A peer or colleague pointed you at us?

Your boss or supervisor did so?

A Google for one of the topics we cover?

A mention by someone else you follow?

A pointer in some kind of publication (American Scientist, Nature, Computing in Science & Engineering, etc.)

We contacted you at some point?

Other (please specify)?

Please add a comment and let us know.

Ratings Revised

Sun, 17 Oct 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We asked, you answered: here are the latest results from our survey of what topics you'd most like us to cover, with links to the ones that have been posted. A few notes:

N = 188 responses.

This is the first time data visualization has dropped out of the top 5.

It's also the first time that nerdish subjects like computational complexity and functional languages have placed anywhere near the top.

There's a noticeable mismatch between things we think people should know (like version control) and things people think they want (building desktop GUIs).

Conclusion: more data ≠ more insight.

Six More Python Episodes

Fri, 15 Oct 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We've added six more Python screencasts to the site:

basics

control flow

I/O

aliasing

functions

first-class functions

We hope you like them—please let us know.

Dexy

Thu, 14 Oct 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

From Ana Nelson, a neat new reproducible research tool called Dexy that takes the pipe-and-filter model to new heights (or extremes, depending on your point of view :-) ). There's a quick intro here, and a video of her recent talk about Dexy here. I'm already thinking about ways of combining this with consistency-checking tools so that if the descriptions of results get out of step with the results themselves, alarms will go off. Check it out, and let us know what you think—is it useful and approachable enough to include in this course?

Five Rules for Computational Scientists

Thu, 14 Oct 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Stepping back from the details for a moment, here are five rules every computational scientist should (try to) follow:

1. Version Control

Put every primary artifact (source code, raw data files, parameters, etc.) in a version control system so that you have a record of exactly what you did, and when. There's no need to store things you re-create, such as the graphs you generate from your data files, as long as you have the raw material archived and timestamped.

Nature Article on Scientific Programming

Thu, 14 Oct 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Nature has just published an article by Zeeya Merali titled "Computational science: ...Error" that looks at what's wrong with scientific computing today. Software Carpentry gets a mention, along with some of the scientists who are trying to raise awareness and standards. And in a companion article, Nick Barnes makes the case for scientists publishing their research software. Here's hoping both will be widely read.

Three Python Screencasts Up

Thu, 14 Oct 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Three Python screencasts narrated by Dominique Vuvan, an alumnus of the Summer 2009 run of the course, are now online: introduction (episode 1), lists (4), and strings (6). We'll be recording and filling in the other episodes just as soon as construction in the office slows down...

Three More Episodes on Spreadsheets

Wed, 13 Oct 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We have uploaded episodes 5, 6, and 7 of Jason Montojo's lecture on spreadsheets, which cover conditionals, pivot tables, and the basics of charting. Comments would be very welcome—and if you'd like to contribute material as well, please get in touch (we're particularly looking for help with data visualization).

Python Lecture Coming Online

Tue, 12 Oct 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Slides for the first few episodes of our lecture on Python are now online; recordings should follow tomorrow and Thursday, and slides for the rest of the lecture should be up by the weekend. I think the content is OK, but it feels like there's too much shoveling: we're throwing a lot of features at people without showing them how they'll make life better. I think there's inevitably some of that in an intro like this—people need to know a few things before any of them can be put to work—but I'd welcome suggestions on reorganizing the material.

Using Subversion from the Command Line

Tue, 05 Oct 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Ainsley Lawson, one of the TAs for the Fall 2010 online course, has recorded an 8-minute episode showing how to use Subversion from the command line. We hope you find it useful.

Aaaand We're Off!

Mon, 04 Oct 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Our first all-online offering of the course starts today: over 40 graduate students and researchers are taking part. We're going to use email, Skype, DimDim, and whatever else comes in handy to supplement the recorded lectures with tutoring and tech support—I'm looking forward to finding out how well all this Internewt stuff actually works :-)

Do You Use Software Carpentry?

Sun, 03 Oct 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

If you use any of the Software Carpentry material in courses of your own, please add a link in the comments section below: our sponsors would like to know what our reach is (i.e., the more links we get, the more likely it is that we'll be able to continue helping you :-) ).

What Questions Do You (Frequently) Ask?

Sun, 03 Oct 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As part of redesigning this site, we're finally putting together an FAQ. It's pretty brief right now: what questions have you had about Software Carpentry that we could add (and answer)?

Ten Short Papers Every Computational Scientist Should Read

Thu, 30 Sep 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

No, we don't have a list—not yet—but we'd like to. What short, readable papers or articles do you think every scientist doing computational work should read at some point in their career (preferably early in their career)? Paul Dubois' 2005 article in Computing in Science and Engineering on maintaining correctness in scientific programs is a favorite of mine; so is Evan Robinson's summary of research on the effects of overwork on productivity. (Actually, I think everyone should read Robinson's article, not just computational scientists...) What else should be on the list? To qualify, entries must be short (up to a few pages long), well written, of broad general interest, and have something important to say that's relevant to our target audience. Suggestions in the comments, please...

Tracking Utility and Impact

Thu, 30 Sep 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Mark Guzdial recently posted some interesting (and for us, slightly depressing) statistics about MIT Open Courseware. Long story short, it looks like that flagship effort isn't as widely used as many of us had hoped or believed. I can't find equivalent stats for the Khan Academy (a scrappy "agile" alternative to MOC that has been getting a lot of attention in the geek community), but even if I could, those numbers probably wouldn't answer my real question: who is this reaching, and what impact is it having? As Guzdial points out, asking users to self-report is subject to large sampling bias, as is googling for links back to lectures. Our sponsors want to know who's using our stuff, and whether it's helping them: how do we answer those questions?

A New Site Design

Tue, 28 Sep 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We've just made a few changes to the layout and look and feel of the Software Carpentry site.

The blog and new content is now accessible directly at http://software-carpentry.org/ Version 3.0 of the course material is still available at http://software-carpentry.org/3_0.

We've put all of the new content under the Topics menu, and shuffled a few of the other pages around to make things easier to find.

We've updated the WordPress theme to one that works better for embedded screencasts.

Let us know in the comments how you like the new configuration, and if you find any glitches.

Software Carpentry at UCSF

Thu, 23 Sep 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Via a comment from Scooter Morris: the University of California San Francisco is offering a "short course" variant of Software Carpentry under the course codes BMI-280/BMI-219. Their slides are online in S5 format, and include lots of useful stuff that aren't in the stock version. If you know of other offerings, please send pointers!

Response Has Been Overwhelming

Wed, 22 Sep 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I'm very pleased to announce that the Fall 2010 offering of this course to Ontario graduate students is now full: we'll be sending acceptance notices to applicants tonight and tomorrow. We hope to offer it again starting in January 2011, so if you didn't make it in this time, please check back in December.

I'm No Graphic Artist...

Tue, 21 Sep 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I'm not much of a graphic artist (in both my startups, I was told after a few weeks that I was never to work on the user interface again), so I'd appreciate your help designing a flyer/noticeboard poster for Software Carpentry. This PDF is what I have so far; anything more constructive than, "My eyes! My eyes! Aargh!" would be very welcome.

Your Favorite Running Examples?

Mon, 20 Sep 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I've been fond of invasion percolation since I first encountered it: the problem is simple to state, but its implementation brings up quite a variety of useful ideas, from testing in the face of randomness, to associative data structures, to the limits of parallelism. It's a good size, too: I can develop a first version in front of students in about 30 minutes, then branch off in various directions for almost as long as they'll listen to me.

Survey: Help Needed

Thu, 16 Sep 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Dr. Roscoe Bartlett (Sandia National Laboratory), Dr. Jeffrey Carver (University of Alabama), and Dr. Lorin Hochstein (University of Southern California) are conducting a survey of software development practices among computational scientists. This survey seeks to understand current software development practices and identify areas of need. The goal is to produce a report on the status of scientific software development which can serve as the basis for future work in this area. The survey should take approximately 15 minutes to complete.

Five Episodes on Make

Wed, 15 Sep 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The first five episodes of the lecture on Make have been recorded, edited, and posted. I still need to do one on using Make to track data provenance, but as I discovered when I started to write it, there are a few other topics I need to fill in first. I will do an episode on managing C/C++ compilation with Make first (probably after I return from teaching in Colorado next week). As always, comments are welcome.

Testing Scientific Software

Wed, 15 Sep 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Yesterday, Michael Feathers tweeted that, "The hardest bit of TDD [test-driven development] for ppl in scientific computation is that they often don't know what their intermediate results should be." I'd go further: if the average scientist or engineer knew what the output of their program was supposed to be, they wouldn't need to run the program. And if you don't know what answer is right, what do you compare your tests' output to?

Software Carpentry Offered Online in Fall 2010 (for Ontario students)

Tue, 14 Sep 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I'm pleased to announce that we'll be running Software Carpentry online October-December 2010 for up to 40 graduate students and researchers at Ontario universities. Please see the Fall 2010 page for details (as we get them), or contact us if you would like to enrol. The course is free, but non-credit.

Will America's Universities Go The Way Of Its Car Companies?

Mon, 13 Sep 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Two days before I flew south to speak at Michigan State University, I read an article in The Economist that asked the question in the title of this post. As it says (quoting US News & World Report), "If colleges were businesses, they would be ripe for hostile takeovers, complete with serious cost-cutting and painful reorganisations." I agree with many of their points: the universities I have worked at (six of them, on three continents) all suffered from the sedimentary buildup of red tape that you'd expect of a large organization that hasn't faced real competitive pressure in living memory, and struggled to achieve either of their core goals of fostering research and passing knowledge on. To quote a friend who's still in the system, "We're here to do research, they pay us to teach, we spend our time on administration."

And For My Next Trick...

Thu, 09 Sep 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

One of the things people have voted right up to the top of our poll is reproducible research. The phrase means different things to different people, but part of it must be accurately tracking the provenance of every bit of data a scientist touches: where it came from, what was done to it, what was done to the results, and so on. Once I've done the next episode of the Make lecture (on macros), I'd like to spend 10 minutes showing people how to do some of this by:

Slides for the First Four Episodes on Make

Wed, 08 Sep 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I don't like Make, but I'm resigned to teaching it: other tools have different, rather than lesser, flaws. I've posted slides for the first four episodes of the lecture about it, and hope to have scripts and screencasts up soon. I'd welcome feedback—in particular, if anyone knows a clean solution to the problem at the end of the fourth episode (getting summary data to depend on stats.py), please let me know.

Getting the Source

Mon, 06 Sep 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

In response to several queries: if you'd like to get the source for both the Version 3 material (the old static HTML pages) and the new Version 4 stuff, they're both in our Subversion repository at http://software-carpentry.org/swc. The whole world should have read permission; please let us know if you run into any problems.

And if you're minded to contribute material, we'd welcome help: please contact us by email to talk about what you could do.

Eight Episodes on the Unix Shell

Fri, 03 Sep 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I had planned to leave it out, but your votes count: the first eight screencasts of our lecture on the Unix shell are now online. Topics covered are:

Introduction (4:09)

Files and Directories (9:55)

Creating and Deleting (6:23)

Pipes and Filters (9:11)

Permissions (10:54)

Finding Things (9:22)

Job Control (5:37)

Variables (6:49)

I've already got a list of changes to make (thanks to feedback from IsaacG2—my apologies, but I didn't see your comments until after I'd recorded), but I hope this will be a good start. And for those who are interested, the eight episodes together took 36.5 hours to write, revise, record, and edit, for 62:20 of screen time. That's better than the 50:1 ratio I was clocking in June, but I need to squeeze it further if I'm going to finish everything on time.

Three More Sets of Slides

Wed, 01 Sep 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As promised in yesterday's post, the slides for three more episodes on the Unix shell are now online, covering how to find things, job control, and shell variables. Next up: public/private keys and SSH.

Five Episodes on the Shell (and Three to Come)

Tue, 31 Aug 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I've just posted the slides for the first five episodes of the lecture on the Unix shell, covering:

Introduction (or, why would anyone do this to themselves, really?)

Files and directories

Creating and deleting things

Pipes and filters

Permissions

Minor things I have yet to cover:

Drive letters on Windows, and how Cygwin hacks around them

The 'echo' command

(Briefly) Windows access control lists

Wildcards in filenames ('*' has been covered)

Major things (the remaining three episodes):

Four More Screencasts on Testing

Fri, 27 Aug 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

After far too many delays, the next four screencasts on testing are finally available, covering systematic unit testing with Nose, why you should test interfaces rather than implementations, why testing floating point is hard, and how to create and manage fixtures systematically. I hope you enjoy them—have a good weekend.

Another Update on What You Want

Thu, 26 Aug 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Responses have slowed, so here are the final scores for the topics we're including (or thinking about including) in this course. It looks like the Unix shell just might go back in...

2.51 Automating Repetitive Tasks

2.50 Reproducible Research

2.50 Data Visualization

2.47 Version Control

2.44 Performance Optimization

2.41 Data Structures

2.39 Testing and Quality Assurance

2.39 Coding Style

2.38 Basic Programming

2.35 Using the Unix Shell

2.35 Parallel Programming

2.35 Debugging with a Debugger

2.29 Computational Complexity

2.22 Object-Oriented Programming

2.20 Working in Teams/on Large Projects

2.19 Designing a Data Model

2.15 Refactoring

2.10 Matrix Algebra

2.09 Static and Dynamic Code Analysis Tools

2.07 Systems Programming

2.04 Integrating with C and Fortran

2.03 Design Patterns

2.01 Packaging Code for Release

1.95 Functional Languages

1.94 Handling Binary Data

1.82 Image Processing

1.74 Build a Desktop User Interface

1.73 XML

1.65 Create a Web Service

1.39 Geographic Information Systems

What Don't You Understand That You'd Like To?

Mon, 23 Aug 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

What don't you understand about computers and computing that you'd like to? I'm not talking about big stuff, like "How does it all work, anyway?" or, "How do I write a game that's as addictive as BubbleSpinner?" What don't you understand about smaller-scale stuff, like handwriting recognition or how Excel knows which cells to update when you change a number or how your machine decides that it's time to update Java and what it does next. Please let me know, or if you're feeling shy, point your friends at this page and have them ask for you—we have to use something as motivating examples for this course, and if we can explain something interesting while teaching you fundamental computing concepts, so much the better.

Slides and Scripts for the Next Two Episodes

Thu, 19 Aug 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I have posted slides and transcripts for the next two lectures on testing, which cover unit testing with Nose, and testing interfaces instead of implementations. I was hoping to record the screencasts today, but setting up a new 64-bit desktop machine took longer than expected—hopefully they'll be up early next week, along with episodes on floating point and setting up fixtures. Comments, as always, would be greatly appreciated.

43% Independent

Mon, 16 Aug 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Cecilia d'Oliveira and colleagues recently wrote an essay in Science about MIT's OpenCourseWare initiative, ten years after its inception. Among the stats:

OCW currently receives upwards of 1.5 million visits each month from 900,000 unique individuals. Students have grown to 42% of the audience, and educators and independent learners now constitute 9% and 43% of visitors, respectively. Twelve percent of educators responding to a March 2010 visitor survey indicated that they do incorporate OCW materials into their own content as anticipated, but educators more frequently use OCW for personal learning (37%), to adopt new teaching methods (18%), and as a reference for their students (16%). Students were largely expected to use the site as a supplement to materials they received in their own classes, a use identified by 40% of students. Just over 43%, however, indicated that they also use OCW for personal learning beyond the scope of their formal studies, and a further 12% use it as an aid in planning their course of study. Independent learners use OCW in a variety of personal (41%) and professional (50%) contexts, including home-schooling children and keeping up on developments in their professional field. 66% of visitors indicate they are mostly or completely successful at meeting their educational goals for visiting the site.

Interview with Cameron Neylon

Thu, 12 Aug 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Today's interview is with Cameron Neylon, a noted advocate of open science.

Tell us a bit about your organization and its goals.

I work for the UK Science and Technology Facilities Council. We are a research funder but although we provide some direct funding our main role is to build and run or subscribe to large scale research infrastructure on behalf of UK scientists. For instance we run telescopes, pay the UK subscription to CERN, as well as supporting and running synchrotrons, neutron sources, high powered lasers, microfabrication facilities and large scale computing infrastructure.

An Answer That Most Students Won't Understand

Thu, 05 Aug 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Two days ago, I asked how to generates tests from tables of fixtures using Nose:

...does Nose already have a tool for running through a table of fixtures and expected results? My hand-rolled version is:

Tests = ( # R1 R2 Expected ( ((0, 0), (0, 0)), ((0, 0), (0, 0)), None ), ( ((0, 0), (0, 0)), ((0, 0), (1, 1)), None ), ( ((0, 0), (1, 1)), ((0, 0), (1, 1)), ((0, 0), (1, 1)) ), ( ((0, 3), (2, 5)), ((1, 0), (2, 4)), ((1, 3), (2, 4)) ) ) def test_table(): for (R1, R2, expected) in Tests: yield run_it, R1, R2, expected def run_it(R1, R2, expected): assert overlap(R1, R2) == expected

which is simple enough if students already understand generators and function application, but hell to explain if they don't—and they won't.

Software Carpentry for Audio and Music Researchers

Thu, 05 Aug 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I will be teaching a version of Software Carpentry tailored for audio and music researchers at Queen Mary University in London from November 1st to 5th, 2010. From the full announcement:

We are seeking nominations for a small number (up to 15) of UK-based PhD students or early career researchers to attend the Autumn School. The SoundSoftware.ac.uk project will pay for reasonable travel and accommodation costs for attendees.

A Question About Nose

Tue, 03 Aug 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I'm putting together an episode of the testing lecture to introduce unit testing frameworks. In the past, I've used unittest, but colleagues have had good experiences with Nose, which doesn't require students to understand classes and methods in order to write tests. As an example, they're going to test a function that finds the overlap between fields in aerial photographs of Saskatchewan (where it's safe to assume that everything is a rectangle :-) ).

Open Source, Open Science in 1999

Tue, 03 Aug 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A long time ago, in a galaxy far, far away... Actually, it was 1999, and the venue was Brookhaven National Laboratory in New York—it just feels like a different time and place. The event was a conference called "Open Source, Open Science", and Stephen Adler, Tom Throwe, and Sean McCorkle have very kindly resurrected its web site at http://openscience.bnl.gov/ for those interested in the history of an idea whose time may finally have come. It's interesting to see what has changed in the intervening decade, and what hasn't: supercomputing is still disproportionately represented, but I think the open side has won the debate that was the subject of the main panel discussion, "Overcoming the obstacles...in using and contributing to Open Source technologies."

Interview with Sergey Fomel

Mon, 02 Aug 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Sergey Fomel is a professor at the University of Texas at Austin, and the leader of the Madagascar project.

Tell us a bit about your organization and its goals.

I work at the University of Texas at Austin, with a joint appointment between the Bureau of Economic Geology and the Department of Geological Sciences. My group conducts research on computational geophysics, with applications to petroleum exploration.

Interview with Davor Cubranic

Sat, 31 Jul 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Today's interview is with Dr. Davor Cubranic, a statistician who lives and works in Vancouver, B.C. Davor recently ran a workshop for faculty and grad students in statistics that covered many of the same ideas as Software Carpentry.

Tell us a bit about your organization and its goals.

I work in the Department of Statistics of a large research university. Our goals are production of research papers, often in collaboration with researchers in other departments, such as life sciences, engineering, or forestry.

A Little Bit of Javascript

Fri, 30 Jul 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As I've mentioned before, one actionable finding in educational research is that faded examples—ones in which progressively less of the solution is shown to students as they progress—are a very effective teaching tool. I've been thinking about how to add them to this course, and have an idea I'd like to try out. It requires more Javascript than I know, though, so I'm hoping someone who reads this blog will be willing to write it for us. (And in general, if anyone wants to help produce material for this course, please get in touch: we're looking for scripts, slides, voiceovers, examples, artwork, and everything else that an open education project needs.)

Stats for July

Fri, 30 Jul 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Visitors

Page Views

Survey Update

Thu, 29 Jul 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Here's an update on responses to the survey I posted a couple of weeks ago. 172 people have responded at this point; it's encouraging that priorities are relatively stable as numbers increase.

Education

77.3% Graduate degree

22.1% Undergraduate degree

0.6% High school

Field

41.0% Computer Science

30.1% Earth Sciences

28.9% Physics

25.4% Mathematics and Statistics

11.0% Microbiology

9.2% Biomedical Engineering

6.9% Macrobiology

5.2% Medicine and Health Care

5.2% Electrical Engineering

5.2% Astronomy

4.6% Mechanical Engineering

4.6% Aerospace Engineering

4.0% Chemical Engineering

2.9% Psychology

2.3% Economics

2.3% Business/Finance

1.2% Linguistics

1.2% Civil Engineering

0.6% Social Sciences

0.6% Arts and Humanities

Role

44.8% Academic Researcher

32.8% Software Developer

16.7% Graduate Student

16.7% Government Research Scientist

10.3% Engineer

9.8% Manager/Supervisor

8.6% System Administrator

3.4% Teacher

2.9% Industrial Research Scientist

1.1% Undergraduate student

1.1% Laboratory Technician

Priorities

2.51 Automating Repetitive Tasks

2.50 Reproducible Research

2.49 Data Visualization

2.46 Version Control

2.43 Performance Optimization

2.41 Data Structures

2.41 Coding Style

2.38 Basic Programming

2.37 Testing and Quality Assurance

2.35 Parallel Programming

2.34 Debugging with a Debugger

2.33 Using the Unix Shell

2.29 Computational Complexity

2.21 Object-Oriented Programming

2.21 Designing a Data Model

2.19 Working in Teams/on Large Projects

2.14 Refactoring

2.10 Static and Dynamic Code Analysis Tools

2.09 Matrix Algebra

2.06 Systems Programming

2.06 Integrating with C and Fortran

2.03 Design Patterns

2.01 Packaging Code for Release

1.95 Functional Languages

1.93 Handling Binary Data

1.80 Image Processing

1.77 Introduction

1.75 Build a Desktop User Interface

1.73 XML

1.64 Create a Web Service

1.39 Geographic Information Systems

Two More Episodes on Version Control

Thu, 29 Jul 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The third and fourth episodes of our lecture on version control are now online. These explain how to handle conflicts from concurrent edits, and how to roll back changes. As with the second episode on basic workflow, they use a mix of slides, screen recording, and sound effects.

The next episode is supposed to explain how to create a repository, but I'm still trying to figure out what to show people. A repo on the same machine that's being used for development is better than nothing, but that doesn't help people share work with colleagues. On the other hand, creating a repo on a server somewhere requires at least basic knowledge of the shell: even if someone is willing to type in a password for each interaction (so that they don't need to know about public/private keypairs), they'll need to know enough to SSH in to the server and run "svnadmin create reponame". There are web-based control panels for creating and managing repositories, and we could just require them to ask their friendly neighborhood sys admin to set that up, but it's just enough of a stumbling block to, well, be a stumbling block. Suggestions would be welcome...

Mark Guzdial on Software Carpentry

Wed, 28 Jul 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Mark Guzdial, a leading researcher in computing education, blogged a few days ago about the Texas Advanced Computing Center's training program for computational scientists, and asked, "Given the importance of computational science, what do all scientists and engineers need to know about high-performance computing?" As you might expect, I replied to say that the question was almost always premature: we should first ask what scientists and engineers need to know about computing in general before tackling HPC.

Second Lecture on Version Control

Mon, 26 Jul 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Our second lecture on version control is now on the web. It combines screen recording with static slides; please let us know whether the format works for you; in particular, can you follow what's happening on the desktop?

Introduction to Version Control

Sat, 24 Jul 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

It took a lot longer to put together than I expected, but I'm pleased with the result—the newest screencast explains what version control is, and why you'd want to use it, in four minutes and four seconds, including a wolf howl and a maniacal laugh. I hope you like it...

And in case you haven't been reading comments, I'd be very happy to include a parallel version control lecture based on a distributed version control system like Mercurial or Git if someone would like to create it. Software Carpentry is an open source project—if that's the itch you want to scratch, then please email me and we'll figure out how to make it happen.

Strictly Speaking, This Isn't Part of Testing

Fri, 23 Jul 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The second episode of the lecture on testing is now up. It covers exceptions, which strictly speaking aren't part of testing, but it seemed like a natural place to introduce them. As always, feedback is welcome...

First Episode of Testing Lecture

Thu, 22 Jul 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The first episode of the lecture on testing is now online. We're going to be showing people how to use Nose, so I've tried to motivate the idea of a unit testing framework—I'd appreciate feedback on whether it works or not.

And yes, I am working on the version control lecture, but I've been stumbling over minor technical glitches, and I'm afraid that if I try recording today, I'll start using uncivilized language. Hopefully I'll get to it tomorrow...

Popular, Fast, or Usable: Pick One

Wed, 21 Jul 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

One of the core skills in any software developer's toolbox is automating repetitive tasks. We've been teaching this in Software Carpentry with Make for 13 years, and for 13 years, we've been looking for something better: something that has a less user-hostile syntax, requires less arcane knowledge of the shell, and at least pretends to do the same thing on different platforms. We briefly considered Ant (where "briefly" means "as long as it took to find the post by Ant's creator saying he would do it all differently next time"), but it's clear that tools like Rake and SCons, that embed build commands in a full-blown scripting language, are the future of build systems. Rake is based on Ruby, but SCons is based on Python (and actually has its roots in the original Software Carpentry project), so it should be a no-brainer, right?

A Note on Tools

Tue, 20 Jul 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

After a bit of experimenting, I've decided to use SmartSVN as a GUI client for Subversion in the version control lectures, rather than RapidSVN or the command line. My reasoning is:

Requiring people to learn the shell before they can start learning stuff they actually care about raises the drop-out rate noticeably. Requiring them to learn the shell before they can start learning something they don't yet realize they ought to use, like version control, raises it even more. That rules out the Subversion command line tools.

SmartSVN is dual-op (the stripped-down version is free, the pro version is not, neither is open source). In contrast, RapidSVN is pure open source, so it would be my first choice (in fact, it was)...

...except RapidSVN doesn't have a built-in diff/merge tool. They recommend SourceGear's DiffMerge, which is free (that's good) but users have to configure RapidSVN to use it (that's bad—very bad). On the Mac, for example, users have to copy a shell script into /usr/bin, then go into RapidSVN's preferences, set the path to the shell script, and get eight (!) command-line arguments exactly right, or things will fail in strange ways (like you can merge files, but when it tries to save your work, it doubles the dirname of the path to the file and then complains about not being able to create /Users/swc/repo/Users/swc/repo/filename).

If something can break for reasons that its intended users won't be able to understand, diagnose, and fix, then that something is itself fundamentally broken. In my opinion, the possibility of configuration hell with RapidSVN outweighs both its licensing and its support for "file://" URLs (which SmartSVN doesn't allow).

Five... Five... Five Scripts in One!

Tue, 20 Jul 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I've updated the script for the introductory episode on version control that I posted yesterday, and added four more covering basic operations, handling conflicts, rolling back changes (with a very brief mention of branching and merging), and setting up repositories for personal use (creating a shared repo requires more skills than students are likely to have, and should be left in the hands of sys admins). The five scripts are down below the break; comments before I start making up slides and recording would be very welcome.

Interview with STSci's Perry Greenfield

Tue, 20 Jul 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Today's interview is with Dr. Perry Greenfield of the Space Telescope Science Institute.

Tell us a bit about your organization and its goals.

We run the Hubble Space Telescope, starting with soliciting and gathering proposals from astronomers that want to use it, to scheduling the selected proposals, generating and uploading telescope commands, processing the downloaded data from raw telemetry to calibrated science images and spectra and providing the results in an archive to the astronomical community. The general goal is to maximize the scientific potential of HST. We are also helping develop the systems needed to run the next large space telescope planned, the James Webb Space Telescope. STScI employs approximately 500 people and is located on the Johns Hopkins University Campus in Baltimore, MD.

Interview with The Hackers Within

Tue, 20 Jul 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Today's interview is with Nicholas Preston, Katy Huff, and Milad Fatenejad of the The Hacker Within at the University of Wisconsin-Madison.

Tell us a bit about your organization and its goals.

The Hacker Within is a student-run, skill-sharing, interest group dedicated to scientific software development at the University of Wisconsin-Madison. We typically meet every week or two discuss programming and computation. The meetings are also a good place to ask questions, share ideas, and meet others who use computation in their research.

Script for Introduction to Version Control

Mon, 19 Jul 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

While I'm waiting for artwork, I'd be grateful for feedback on the script for the introductory episode on version control.

Hello, and welcome to the first episode of the Software Carpentry lecture on version control. In this episode, we will explain what version control is, how it works, and why you should use it.

Suppose you and a friend are working together on a paper.

You both want to edit the file at the same time—what should you do?

You could take turns, but then each of you would spend have the time waiting for the other.

Another option would be to go ahead and work simultaneously, then patch things up afterward.

But somehow, stuff always winds up getting lost, overwitten, or duplicated.

The right solution is to use a version control system.

This keeps the master copy of the file in a central repository, usually located on a server—a computer that is never used directly by people, but only by the applications serving them.

No-one ever edits the master copy directly.

Instead, you and your friend each have a working copy on your own computers.

You work independently, making whatever changes you want to your local copies.

As soon you are ready to share your changes, you commit your changes to the repository.

Your friend can then update her working copy to get those changes.

And of course, if your friend finishes her part first, she can commit, and then you can update.

But what if you and your friend want to make changes to the same part of the paper?

Old-fashioned version control systems prevented this from happening by locking the master copy. Everyone's working copy would normally be read-only. When someone wanted to start work on a file, the version control system would make her copy of that file writeable. When she was finished work, the version control system would copy her changes to the repository, then mark her copy as read-only once again.

Only one person at a time could have a writeable copy. This guaranteed that two or more people could never accidentally make changes to the same file at the same time...

...but it also meant that only person at a time could work on any given file.

This is essentially the "one at a time" strategy from the start of this episode, but with the version control system acting as the referee to prevent accidents.

In practice, locking like this isn't as restrictive as it sounds. If you and your friend repeatedly find that you're trying to edit the same file, the solution is to break your paper (or your program) into several smaller files, so that you can work simultaneously.

However, most of today's version control systems use a different strategy, one based on the old saying that it's easier to get forgiveness than permission. In these systems, nothing is ever locked—everyone is always allowed to edit the files in their working copy.

Sometimes, of course, you and your friend will make changes to the same part of the paper.

If your friend updates first first, her changes go into the repository as usual.

If you try to commit something that would overwrite her changes, the version control system will stop you...

...and highlight the conflict by marking the overlapping regions in your working copy.

It's up to you to edit the file to resolve the conflict. You can keep your changes, accept your friend's, or write something new that combines the two—it's up to you.

Once you have fixed things, you can go ahead and commit.

Experience shows that version control is better than mailing files back and forth for at least three reasons. First, it's hard (but not impossible) to accidentally overlook or overwrite someone's changes—the version control system highlights them for you automatically.

Second, there are no arguments about whose copy is the most up to date—the master copy is.

These features mean that version control is worth using even when you're the only person working on a particular set of files.

Because it's a more reliable way to move files around between the computers you use than copying things onto a USB or sending email to yourself.

More importantly, whether you're working on your own or in a group, version control allows you to look at or undo changes what you did weeks, months, or years ago.

This works because the version control system never actually overwrites the master copy in the repository.

Instead, everyone time someone commits a new version, the system saves it on top of the previous master copy, along with some information about when the change was made and who made it.

This means that you can always see what the file looked like last week before someone rewrote the analysis section while you were on holiday.

It also means that you can always fetch old versions of things, like the exact version of the program you used to produce the graph on page 5 of that paper that someone is now challenging.

Version control systems do have one important shortcoming, though. If you are working with plain text files, it's easy for the version control system to find and display differences, and to help you merge them.

Unfortunately, today's version control systems won't do this for images, MP3s, PDFs, and Microsoft Word or Excel files. These aren't stored as text—they use specialized binary data formats, and there usually aren't tools for finding, displaying, or merging the differences between them. In most cases, all the version system can do is say, "These files are different." That's better than nothing, but not by much.

Even with this limitation, version control is probably the most important concept in this entire course. It's not just because it facilitates sharing; version control also allows you to look at or undo changes what you did weeks, months, or years ago.

We'll talk more about using version control to make your research reproducible in a later lecture. In the next episode of this one, we'll look at the most popular open source version control system in use today, called Subversion.

The four episodes after this introduction will cover:

A Gentle Introduction

Sun, 18 Jul 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Via several routes, I've been pointed at MIT Open CourseWare's "A Gentle Introduction to Programming Using Python", as taught by Sarina Canelake in January 2010. An hour of lecture a day for 10 days (two weeks), plus two hours of lab per day, it is... "a gentle introduction to programming using Python for highly motivated students with little or no prior experience in programming computers". The pace is brisk:

An Interview with Hans Petter Langtangen

Sun, 18 Jul 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Our latest interview is with Hans Petter Langtangen, author of two books for scientists about Python (and a lot more).

Tell us a bit about your organization and its goals.

I am on 80% leave from the computer science department at the University of Oslo, to work at Simula Research Laboratory. Simula is primarily funded by the government, with a commitment to do long-term fundamental research in three areas: software engineering, networks and distributed systems, and scientific computing. My work is in the latter area. The funding is dependent on a successful scientific evaluation by an international panel every five years. About 120 people work at Simula today.

Clip Art

Fri, 16 Jul 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As per your votes, I'm about to start writing the lecture on version control. Our first attempt, two months ago, did a pretty good job of covering the basics, but the slides were downright boring. In particular, we used standard Microsoft clip art to illustrate two people sharing files via a server:

I'd like to use something more entertaining, like these animals from StudioFibonacci:

Survey Results

Thu, 15 Jul 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Here are the results of the survey that we announced a couple of days ago. I'm a bit surprised that so many computer scientists responded, and equally surprised by the popularity of "biomedical engineering" — who knew? The scores for various topics hold a few surprises as well: I would have predicted that something with the word "web" in it would have scored near the top of the list, rather than at the bottom.

Two New Episodes on Dictionaries

Wed, 14 Jul 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We have just posted the last two episodes of the lecture on sets and dictionaries, which introduce dictionaries and give examples of their use. Please let us know what you think about the pace and level of detail.

A Shorter Version of the Sets and Tuples Episode

Tue, 13 Jul 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

One of the many advantages of online delivery of course material is that it allows us to present content in several ways, tailored to different audiences. For example, some people want to understand how sets are stored in hashtables; others just want to know that you cannot put a list in a set, and should use a tuple instead. We now have episodes catering to both: the longer and more detailed one posted last week, and a shorter and less detailed one posted today. We'd welcome feedback: is this confusing, helpful, or (as is so often the case) a bit of both?

Interview: Andrew Lumsdaine of Indiana University

Tue, 13 Jul 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Today's interview is with Indiana University's Professor Andrew Lumsdaine.

Tell us a bit about your organization and its goals.

One of the reasons the School of Informatics and Computing exists is that computing of various kinds has become part of almost all academic disciplines. The school's goal is to educate students in computer-related areas, as well as traditional computer science, and it's important that the principles of software development be taught. The same holds for my research group, which works in HPC (where the 'P' means both "performance" and "productivity"). There's a huge need for reproducibility in CS research, both for the sake of sound science and also so that people actually can build on each other's work. In order for that to happen, we need some guarantees about quality and reusability, and improving basic skills is a necessary prerequisite.

Traffic

Tue, 13 Jul 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

In case you're interested, we seem to average slightly around 280 views per blog post right now. I'll post stats on page views (i.e., how many people are looking at the lectures themselves) at the end of this month.

Interview with Michigan State's Titus Brown

Sun, 11 Jul 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Today's interview is with Professor Titus Brown, from Michigan State University.

Tell us a bit about your organization and its goals.

Michigan State is one of the Big Ten and hence a really big place. We do a lot of research, in particular, and a lot of that research is in biology. The Gene Expression in Disease and Development focus group is a collection of molecular biologists at MSU that are interested in gene expression and methods (experimental, genomic, and bioinformatic) for understanding it.

HPC and Programmability

Sat, 10 Jul 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Via Andrew Lumsdaine, a pointer to an interesting article in Communications of the ACM by Eugene Loh titled "The Ideal HPC Programming Language". A few key quotes:

These programmability studies began with a focus on programming languages, but the focus quickly shifted to other topics. Existing languages—notably Fortran...—proved remarkably adequate. Programming challenges stem mostly from other factors.

The DARPA HPCS program...sponsored the development of new programming languages: Chapel from Cray, Fortress from Sun, and X10 from IBM. Proponents of those languages would show early on how rewriting familiar HPC benchmarks in the new languages could reduce source-code volume substantially—tenfold reductions were not surprising—but rewriting these benchmarks even in Fortran achieved similar source-code reductions and corresponding improvements in expressivity.

Which Topics Are Most Important to You?

Sat, 10 Jul 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We have created a survey to find out which of the topics we're planning to cover are most important to you. Please take three and a half minutes to fill it in; we'd be grateful as well if you could re-post the link, forward it to relevant mailing lists, etc.

http://www.surveymonkey.com/s/FM9YV9C

Thanks for your help!

Interview: SciNet's Daniel Gruner

Thu, 08 Jul 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Today's interview is with Dr. Daniel Gruner of SciNet.

Tell us a bit about your organization and its goals.

SciNet is a High Performance Computing (HPC) consortium. We provide hardware and software platforms for researchers who require HPC in their work. We are also enabling this research, in ways that had not even been thought of before. The enabling is partly due to us having the largest systems in Canada, and partly by helping people port their codes to run on these very large systems.

Two Episodes on Sets

Thu, 08 Jul 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I've just posted an introduction to sets in Python, and a brief look at how they're implemented (which you need to know in order to understand why you can't put a list in a set). The second of these has more theory in it than most of our episodes; I'd be grateful for feedback from non-computer scientists about how much sense it makes.

Using Science to Design This Course

Wed, 07 Jul 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

One of the many reasons I left the University of Toronto to work full-time revising this course was to explore two areas I've been curious about for years: educational technology, and the science of learning. With my daughter a year away from kindergarten, is no longer academic [1]. In this post and some that follow, I'd like to talk about what I'm finding and how it's influencing the design of this course. If you want to know more, please subscribe to Mark Guzdial's excellent and thought-provoking blog about computer science education.

Hubs, Spokes, and Gonzo Programming Skills

Tue, 06 Jul 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

In the aftermath of his (very successful) course on next-generation sequencing, Titus Brown has posted some thoughts on how to tie that material to Software Carpentry. To make a short story even shorter, he thinks Software Carpentry requires more knowledge on entry than most biologists have, and proposes a hub-and-spoke model as a solution. He also thinks we need a "Greg gets hit by a bus" plan. Please head over to his post and add comments there: April 2011 (the end date for the current round of funding) seems a long way off, but it isn't.

That's, Uh, Pretty Ambitious

Tue, 06 Jul 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The title of this post is taken from the reaction of the first person I showed this course schedule to. If we're really going to teach Software Carpentry in one university term (12-13 weeks), this is the pace we'll have to move at. What do you think? Is it crazy, or just nuts? And what, if anything, can be done about it?

The Violas of Programming

Tue, 29 Jun 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Orchestral musicians make jokes about violas and viola players. "What's a string quartet? A great violin player, a mediocre violin player, a bad violin player, and a cellist." Or, "What's the difference between a viola and a trampoline? You take off your shoes to jump on a trampoline." But violas are as essential as they are unglamorous: hardly anyone plays them as a lead or solo instrument, but string quartets just don't sound right without that third voice.

Last Episode on Sets and Dictionaries Posted First

Mon, 28 Jun 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

For our lecture on sets & dictionaries, I decided to start at the end with the wrap-up example on nanotechnology. I'll double back tomorrow and start filling in the pieces that this example relies on.

Four Down-What Next?

Sun, 27 Jun 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The regular expressions lecture has been promoted from the development area to the main site, where it joins the lectures on program design, databases, and spreadsheets. We have a lot of other topics still to cover: please let us know which you'd like to see first.

Final Episode of Regular Expressions Lecture Now Online

Fri, 25 Jun 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The fifth and final episode of our lecture on regular expressions is now online. In this episode, we work through a new problem—extracting bibliographic citations from papers—and introduce a few new tools along the way. Please let us know what you think.

Another Example of small-p Patterns

Thu, 24 Jun 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A couple of weeks ago, I asked whether people would find an exploration of ways to count things useful. The consensus was "yes", so I've started drawing up notes. While working on them, it occurred to me that "ways to persist things" might be just as interesting. Some of the approaches that could be discussed are:

Save a list of numbers (one per line).

Save a matrix using a header line to record dimensions (introducing the idea of metadata) and M values per line.

Save a matrix without metadata (which requires the length of the second and subsequent lines to be checked).

Save mixed atomic values (which gets hard when strings are allowed).

Saving records whose structure is known in advance (the hardest of the bunch, since this is the point where aliasing appears).

Saving in binary rather than ASCII.

Creating self-describing data (save the format at the top, then the data).

What other persistence patterns do you think would be worth explaining, and why?

Eric Lander on Genomics

Thu, 24 Jun 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This lecture by MIT's Eric Lander is a great overview of where genomics is, how we got here, and where we might be going. "Biology as an information science" is proving to be a very rich idea...

SIAM News Article About Software Carpentry

Thu, 24 Jun 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The June 2010 issue of SIAM News (published by the Society for Industrial and Applied Mathematics) has an article I wrote about Software Carpentry called "Get More Done with Less Pain". Here's hoping the course can live up to that billing :-)

Software Carpentry in Three and a Half Minutes

Wed, 23 Jun 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I had a chance to give a quick pitch for Software Carpentry yesterday, which I've since recorded and posted to our web site. I hope it's a useful overview of what we're trying to do and why; I'd be very grateful for ideas about improving it.

Note: a newer version of the pitch, titled "Software Carpentry in Ninety Seconds", is now available.

Software Developer: Audio and Digital Music

Wed, 23 Jun 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Centre for Digital Music (C4DM) at Queen Mary, University of London, is seeking an experienced Software Developer with a background and knowledge in Audio and Digital Music, to work on a new EPSRC-funded project "Sustainable Software for Digital Music and Audio Research". The aim of this project is to provide a Service to support the development and use of software, data and metadata to enable high quality research in the Audio and Digital Music research community. Please see the job posting for more information.

Episode 4 on Regular Expressions

Tue, 22 Jun 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The fourth episode of our lecture on regular expressions is now online. This one explores shortcuts and escape sequences; we'd be grateful for feedback on whether it works a single episode, or ought to be split in two.

A Little Bit of Theory

Mon, 21 Jun 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The third episode of our lecture on regular expressions is now on the web. This one describes how regular expressions are implemented using finite state machines: it's more theoretical (with a very small 't') than previous episodes, and we'd welcome feedback.

Interview with Microsoft's David Rich

Mon, 21 Jun 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The latest in our series of interviews with Software Carpentry sponsors is with Microsoft's David Rich.

Tell us a bit about your organization and its goals.

Microsoft has very recently expanded the group working on HPC (clusters and similar) to address a broader "technical computing" solution set. This includes areas such as parallel programming, and making it easier for scientists and engineers to apply computational power to their work.

Second Lecture on Regular Expressions

Sat, 19 Jun 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The second lecture on regular expressions has been posted to the development area. This one covers operators like '*' and '+', and introduces character sets. Please let us know what you think.

First Half of Spreadsheets Lecture Now Online

Fri, 18 Jun 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The first four episodes of Jason Montojo's lecture on spreadsheets are now online: topics covered include basic calculations, aggregation, controlling the display, sorting, and debugging. Please have a look and let us know what you think.

For World Cup Fans (and Everyone Else)

Fri, 18 Jun 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Mike Knell has posted a funny piece titled "If Sports Got Reported Like Science" [accessed via Internet Archive]. Perhaps I should have put quotes around "funny"...

Our First Few Exercises

Fri, 18 Jun 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Jon Pipitone has posted a few self-assessment exercises for the lecture episode on selecting data from a database. Please have a look and let us know what you think: would having little self-tests like this at the end of each episode help solidify your understanding of the material?

People You Don't Want On Your Team

Fri, 18 Jun 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Study after study has shown that the biggest causes of software project failure are over-optimistic scheduling and unstable requirements. I think the only reason team dynamics isn't in the #1 spot is that it's hard for an outsider to judge after the fact. In my experience, though, how well people work together is a lot more important than how smart they are individually: everyone in the engineering group of the first start-up I joined was very good at their job, but when you put us together, our IQs somehow canceled out.

Let's Try That Again

Thu, 17 Jun 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Thanks to everyone for their feedback on our first attempt at a lecture on regular expressions. Our second attempt [no longer online] is now up for your viewing pleasure: it's about ten minutes long, and uses data scrubbing to motivate the use of REs. As always, we'd be grateful for suggestions.

A Voice from the Back of the Room

Wed, 16 Jun 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Mark Guzdial recently posted another thought-provoking piece, this one about how teachers are biased toward assessing a class's progress by their interactions with its brightest students, and how important it is to assess understanding systematically to find out what's actually going on. We're still wrestling with this in Software Carpentry: what can we put on the web to help students determine whether they have understood a particular topic and are ready to move on?

Is Live Coding Worth It?

Wed, 16 Jun 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I put together an introductory lecture on regular expressions yesterday, but I'm not happy with the results, and I'd like your feedback to help me make it better.

One of the reasons we're moving from static HTML to videos is to show people how to program, rather than just the results of programming. As part of that, I want (or wanted—I may be changing my mind) to do all the programming examples live instead of showing snippets of code and snippets of output slide-style. Jon has already done that with the database lectures, but his "programs" were single-line SQL statements; this episode is the first time we've had longer examples. It was a pain to do: each time I made a typo inside a function definition, for example, I'd wind up with a dozen or more junk lines on screen (since the error wouldn't show up until the first time I called the function). I don't want to waste viewers' time on trivial errors, but there's no way in IDLE (or other IDEs) to erase the last few lines of interaction with the interpreter. I "solved" the problem by hitting return until the offending text had scrolled off the top of the screen, but it leaves a couple of ugly breaks in the video. If IDLE had a "clear screen" button, I might use that periodically, but (a) it doesn't, and (b) one of the reasons to code live is to keep recent context in view, which "clear screen" would erase.

Glossary and License Online

Tue, 15 Jun 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A glossary of terms used in this course is now online; corrections and suggestions for both additions and deletions would be very welcome. The licenses used for course content and code have also been posted—both are as open as we can make them.

Next-Gen Sequencing Course at MSU: It Went Well

Tue, 15 Jun 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Titus Brown has posted a summary of his course on next-generation sequencing data analysis for biologists with little or no previous training in computing. It went very well, and he's already trying to figure out how to do it again next year. If you can help, please give him a shout!

Later: Titus and his students used Amazon's web services in the course, which earned them a post on Amazon's blog.

Interview: Mark Plumbley at Queen Mary University of London

Mon, 14 Jun 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Today's interview is with Prof. Mark Plumbley, of the Department of Electronic Engineering & Computer Science at Queen Mary University of London.

Tell us a bit about your organization and its goals.

We are a new project "Sustainable Software for Digital Music and Audio Research", funded by the UK Engineering and Physical Sciences Research Council (EPSRC), and based at the Centre for Digital Music (C4DM) at Queen Mary University of London. The aim of the project is to provide a Service to support the development and use of software, data and metadata to enable high quality research in the Audio and Digital Music research community. It's really about getting research results—including robust software to implement that research—out to the people who should be able to use them, and then keep it working.

The Cowichan Problems

Sat, 12 Jun 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Back in the 1990s, as the first wave of euphoria about parallel computing was topping out, I had a crazy idea: why don't we actually try to measure, or at least compare, the usability of different parallel programming systems? I left the field before taking the idea very far, but with talk about clouds and GPUs growing louder by the day, I think the idea is worth revisiting. My proposal is below the cut; I'd be interested in feedback.

Counting Things

Fri, 11 Jun 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I re-read Robins et al's "Learning and Teaching Programming: A Review and Discussion" [1] and Eccles and Stacey's "Understanding the Parallel Programming" [2] this morning. One of the key takeaways in both is that novice programmers tend to classify problems according to problem domain or language features, while experienced programmers tend to classify them according to solution strategies. Where a beginner sees a problem in molecular biology or statistics, for example, an expert sees an opportunity to apply divide-and-conquer or accumulate-and-adjust.

Interview: David Jackson at the UK Met Office

Fri, 11 Jun 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The second of today's sponsor interviews is with David Jackson from the UK Met Office.

Tell us a bit about your organization and its goals.

You can find out more about us at http://www.metoffice.gov.uk, but broadly speaking, we:

predict the weather for tomorrow, next week, next season and beyond;

are a significant contributor to the global understanding of climate change;

are leading researchers of weather science;

provide forecasts for sporting events such as Wimbledon and Open Golf;

stand shoulder to shoulder with Armed Forces around the world;

help keep roads open and planes flying;

inform the decisions and policies of businesses and governments across the world;

help the National Health Service provide preventative healthcare.

Our group is focussed on research.

Interview: SHARCNET's Hugh Couchman

Fri, 11 Jun 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The first of our two sponsor interviews today is with SHARCNET's Prof. Hugh Couchman.

Tell us a bit about your organization and its goals.

SHARCNET provides high-performance computing (HPC) resources and services to researchers in Ontario and Canada. Our goal is to promote and facilitate the use of HPC techniques among researchers in all fields and to train a new generation of computationally-skilled individuals for the benefit of competitive research, the economy and society in Canada.

Thought for the Day

Fri, 11 Jun 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We talked of the education of children; and I asked him what he thought was the best way to teach them first. Johnson: "Sir, it is no matter what you teach them first, any more than what leg you shall put into your breeches first. Sir, you may stand disputing which is best to put in first, but in the mean time your breech is bare. Sir, while you are considering which of two things you should teach your child first, another boy has learnt them both."

Interview: Jim Graham of Scimatic

Thu, 10 Jun 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Over the next few weeks, I'll be interviewing the people whose sponsorship has made this course possible. First up is Scimatic's Jim Graham.

Tell us a bit about your organization and its goals.

Scimatic (www.scimatic.com) is a company that writes software for scientists. Historically, we've written software on contract for manufacturers of scientific equipment. We have also written our own software product, Samples, to help scientists keep their experiments organized.

Our Lecture on Databases is Now Online

Thu, 10 Jun 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Our introduction to the basics of SQL is now online: the episodes in this series include:

Introduction

Selecting Data

Filtering

Sorting

Aggregation

Joining Tables

Missing Data

Nested Queries

Please let us know what you think. And please also let us know what topic from the course outline you'd like us to tackle next.

The Big Picture (version 3)

Thu, 10 Jun 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Version 3 of the concept map describing what we mean by "computational thinking" incorporates feedback from several people—thanks for sending it. Please let us know what you think...

Reorganizing Content

Wed, 09 Jun 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

In anticipation of posting more lecture content, we have reorganized the pages on this site. If you look at the side menu, you'll see that "About/" and "Program Design/" now have sub-pages; we'll organize each lecture the same way to make it easier to follow along.

And if you're having trouble posting or viewing comments, please send us email— we think we've sorted it out, but hey, it's software...

Episode 11: Making It Fast

Tue, 08 Jun 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Today's episode is the last in the series on program design using invasion percolation as an example. In it, we look at how to make the program we've developed so far run zillions of times faster. It's the longest episode so far, and probably the most difficult, but it makes two very important points:

The biggest performance gains come from improvements to algorithms and data structures, not from tweaking loops and rearranging conditionals.

If you want to write a fast program, you should start by writing a simple one, test it, and then rewrite it piece by piece, re-testing as you go along.

Here's the complete list of episodes:

Testing Invasion Percolation

Mon, 07 Jun 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Finally, after all that refactoring, we get to test our invasion percolation program—although in fact, we spend more time discussing how to structure testing code than we do actually testing.

Introduction

The Grid

Aliasing

Randomness

Finding Neighbors

Resolving Ties

Assembling the Program

Bugs

Refactoring

Testing

The Big Picture (version 2)

Mon, 07 Jun 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Thanks to everyone who gave us feedback on the first version of a concept map for Software Carpentry. The second version is shown below, along with a list of terms that ought to be on it, but aren't. Where would you place them, and how would they connect to each other and to the concepts that are already linked?

Concept Map

Fri, 04 Jun 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

While re-designing the Software Carpentry course, I have realized that we rely over and over again on some underyling concepts that are hard to capture as lecture topics. I think these concepts are the heart of any useful definition of "computational thinking". The diagram below is a first attempt to capture what these concepts are and how they're related. (The list below the diagram summarizes the relationships in textual form for easier reading.) Suggestions for improvements would be very welcome...

Refactoring Invasion Percolation

Fri, 04 Jun 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Today's episode (the ninth in the lecture about designing an invasion percolation program) was supposed to be about testing, but wound up being about refactoring instead—which probably won't surprise anyone who's done any significant amount of programming. We'd be grateful for feedback on how easy or hard it is to follow along as the program is rearranged.

Introduction

The Grid

Aliasing

Randomness

Finding Neighbors

Resolving Ties

Assembling the Program

Bugs

Refactoring

Assembling a Program

Thu, 03 Jun 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The seventh episode of our lecture on program design is now online. In this one, we actually assemble a complete version of the program step by step using the pieces designed earlier. The final program still has one nasty bug, though, and I'll award a Software Carpentry mug to the first person who can find it (source code attached to this post). For reference, the first six episodes are listed below, along with another link to this one.

If You Want to Look Ahead...

Thu, 03 Jun 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Our previous post said that there was a bug in the first complete version of our invasion percolation program, and offered a Software Carpentry mug to the first person to find it. If you're the sort who prefers to look up the answers in the back, we've just posted the solution as episode 8 in the series on program design. For reference, here's the lecture so far:

Program Design: the Second Instalment

Tue, 01 Jun 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Having recorded and posted four episodes of a lecture on program design yesterday, I've managed another two today (highlighted below):

Introduction

The Grid

Aliasing

Randomness

Finding Neighbors

Resolving Ties

The remaining episodes are much longer, and will show how to assemble the program from the pieces seen so far, how to refactor the result to make it more testable, how to test it, how to speed it up, and how to build a GUI for it. Slides for the first two of these episodes are done; I hope to have everything up by the end of this week or early next. As always, feedback is greatly appreciated.

Who Reports On The Other 97 Per Cent?

Tue, 01 Jun 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The BBC has posted a nice graphic of the latest Top 500 list of supercomputers around the world. It's pretty impressive, particularly if you're a Linux fan (check out the display by operating system). However, nobody would think that a list of the world's 500 richest people told them much about the state of the economy; I'd be much more interested in statistics on how much computing power is employed by the 97% or so of scientists who don't use a supercomputer.

Program Design: the First Third

Mon, 31 May 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I spent last week making and re-making slides for a lecture on program design, and managed to record the first third of them today—the rest should follow tomorrow and Wednesday. We've taken on board your comments about pace and layout, and would be grateful for more feedback—please go ahead and post comments on this post to tell us what you think.

Introduction

The Grid

Aliasing

Randomness

Jim Graham on Reproducibility

Sat, 29 May 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

In response to Titus Brown's not-really-joking spoof of how most scientists manage their data, Scimatic's Jim Graham has asked, "What is reproducibility, anyway?" His main point is that if I re-run your code on your data, and get the same result that you got, I haven't actually added to the sum total of human knowledge. If I get an equivalent answer using different code, on the other hand, then our confidence in the answer is usefully increased. What do you think?

Teaching databases by example

Fri, 28 May 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Over the last two weeks I've been spending most of my Software Carpentry time working on the database unit. They began as a fairly straight-forward translation of the Software Carpentry 3.0 lecture notes with only a few changes to the sequencing of the topics. The plan for the unit is fairly simple: each screencast introducing only one or two new topics, and builds on the previous screencasts with as few as possible forward references to later topics. The topics themselves are really just the different language features (SELECT, WHERE, JOIN, etc..) presented as tasks you might want to perform, and illustrated by working with toy data.

Badges and Stars

Thu, 27 May 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Open Notebook Science folks have developed a set of badges to help people label their work, ranging from "All Content — Immediate Release" to "Selected Content — Delayed Release". Like the badges for various Creative Commons licenses, or Tim Berners-Lee's proposed five-star system for rating open data, the biggest benefit of this kind of categorization is that it encourages people to think more clearly about what they are (or aren't) doing, and why.

Archiving Experiments to Raise Scientific Standards

Tue, 25 May 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

An NSF Workshop on Archiving Experiments to Raise Scientific Standards begins today (May 25, 2010) at the University of Utah — the schedule has links to some of the presentations, and there's a live video feed as well. Topic areas range from networking and compilers to geophysics and the life sciences — it'll be interesting to see what comes out, and how it interacts with the Open Provenance work.

Evaluating Methods and Protocols

Wed, 19 May 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

From their home page:

ScienceCheck.org is the only site dedicated specifically to sharing objective evaluations of published experimental methods/protocols, from researchers with real-world experience.

Sadly, the "computer science" category (filed under "Other") is currently empty, and there isn't a "computational science" category as far as I can see. Still, it's a beginning...

We'll Know We've Succeeded If...

Tue, 18 May 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We will know the students taking this course have learned something if:

They understand why Titus Brown's Data Management Plan post is both funny and sad.

They have the knowledge, skills, and tools they need to do something about it.

Day 11: Slides

Mon, 17 May 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Today's screencast experiment [link no longer active] is narration over PowerPoint slides: it isn't animation per se, but we'd like your feedback on whether something like this is a good way to explain "big picture" concepts.

Day 10: Closed Captioning

Fri, 14 May 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Our latest screencast, on NULL values in SQL, is now online. [Original link no longer active; more recent screencast on NULL values.] Unlike its predecessors, this one has closed captions (as well as a transcript in the enclosing page). Please let us know what you think: are the captions helpful, or do you find them distracting?

Why Most Scientists Don't Like Computers

Fri, 14 May 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Psychology is fascinating: so much of what we think we know about people turns out not to be true, while so many everyday oddities turn out to have rational explanations (for some version of "rational"). For example, I've known for twenty-five years that most scientists dread the point in their work when they have to do something new with a computer. I've finally figured out why: it's because the yield-to-effort function is wildly discontinuous, and human beings hate that.

A Word (Or Three) From Our Sponsors

Thu, 13 May 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Starting next week, we're going to start posting interviews with the people who are sponsoring Software Carpentry. Questions we'll ask include:

What sort of work does your organization do?

What kind of people work there? What backgrounds and skills do they typically have when they join you, and where do they need help?

What do they find most frustrating when doing computational work?

How do you hope this course will help them?

How will you tell if it actually has (i.e., how will you evaluate this course's success or failure)?

What else would you like us to ask them?

Day 8: Exercises (with a screencast)

Thu, 13 May 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Today I experimented with creating a few simple exercises for the material covered in the first screencast on databases. The very last exercise includes a screencast that walks through the solution, and a potential pitfall along the way. My apologies for the crappy audio and choppiness in a few places — I was using my laptop microphone, and I tried to edit out a few clicks and pops.

Day 9: Programming

Thu, 13 May 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Our first screencast showing ~~a bit of Python programming~~ is now up for comments. [Link no longer active, more recent link to screencast on dictionaries.] It's deliberately low-tech: we used Emacs as an editor, and simply showed the program's textual output instead of stepping through it with a debugger. (Our next programming lectures will be IDE-based for comparison.) Please tell us what you think about the pace, watchability, etc.

Day 7: Mini-screencasts

Wed, 12 May 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Yesterday Jason and Greg experimented with breaking apart a 10 minute screencast on database grouping and aggregation into even more bite-sized pieces. We have two flavours today: ~~point form notes with links to the screencasts~~, or ~~point form notes with screencasts embedded~~. [Links no longer active, latest screencast on database aggregation as of January 2012.]

Feedback on this format would be really helpful. Are these chunks too small? Do you prefer embedding over linking, or vice versa? Also, we'd welcome any comments on the material itself, of course.

Day 6: Screencast With Point-Form Notes

Mon, 10 May 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Jason Montojo and I put together an introductory screencast about spreadsheets today. Unlike our previous screencasts, this one is accompanied by point-form slide-style notes instead of prose paragraphs. We'd be grateful for your feedback: are notes of this kind useful? Are they comprehensible without watching the screencast itself? If the notes were available in slide format (Keynote, PowerPoint, Impress, or plain old PDF) would they be enough for you to lecture with?

Why We're Self-Hosting

Mon, 10 May 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A couple of people have asked why we're planning to host and serve the content we're developing instead of using a combination of YouTube and Google Code, or one of the emerging online education services such as LearnHub or Supercool School. Part of the answer is longevity: while standards like SCORM are supposed to make e-learning content portable between different systems, in reality there is still a large degree of vendor lock-in, particularly with hosted services, and we don't want to find that our content has gone off-line because someone hasn't survived a market downturn. Hostable learning management systems like Moodle require more work to set up and administer, but are also more robust (at least for now).

Microsoft

Sun, 09 May 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are very pleased to announce that Microsoft Corporation has come on board as a sponsor of this project. Many thank to David Rich for making this happen!

Day 5: A Different Kind of Screencast

Fri, 07 May 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I spent most of today creating another screencast, this one explaining what "and", "or", and "not" mean to programmers and how they work. There are no live examples this time: instead, I illustrated the talk with some simple images. Setting aside the question of whether Venn diagrams are the best way to explain Boolean logic, do you prefer this to live desktop of the previous screencast? Do the jumps back and forth between sets and tables make sense, or are they confusing? Do the colors add value, or should we stick to black & white? And how about the pace—is it too fast, too slow, too repetitive, et cetera?

Day 4: First Preliminary Alpha Test Etc. Screencast

Thu, 06 May 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Jon Pipitone and I have created a screencast showing the basics (and I really mean "the basics") of pulling data out of a database using SQL. It took about 4.5 hours in total to produce 7 minutes of video:

Writing script: 1.5 hours

Doing 3 takes: 1.0 hours

Editing: 1.0 hours

Transcribing: 0.5 hours

Screenshots: 0.5 hours

This was the first time either of us had used Camtasia; I expect that we'll be able to cut the recording, editing, and screenshotting time in half with practice. Transcribing time is dictated by typing and editing speed, so it's unlikely to come down, and scripting time will probably be the same or greater for future topics (since some of them will require diagrams and other time-consuming prep). We're therefore looking at 3 hours of production for 5-7 minutes of screen time; if you figure a lecture is 50 minutes, and there are 25 of them in the course, that works out to... um... carry the six... 750 hours to get all of the existing material online. Pad by half to account for real-world effects, and that's 35-40 weeks — tight, but doable.

A Question About Documentation

Wed, 05 May 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

What kind of documentation do you use when you're programming? How useful do you find it? If have three minutes to fill in a very short survey on the topic (it's literally half a dozen questions), we'd be very grateful for your feedback.

Day 1: Shuffling Sticky Notes Around

Tue, 04 May 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Day 1 of Software Carpentry Version 4: Jason, Jon, and I spent the afternoon shuffling sticky notes around on a whiteboard, trying to map out key concepts for our databases lecture(s) and their dependencies. The result isn't all that different from what's in Version 3, but there was one important insight. J&J commented that many of the students in the winter run of the course had understood the material, but been unable to apply it to their own problems. Our tentative explanation is that in order to apply what they have learned about SQL to their own data, students have to (a) pre-process that data (e.g., read a CSV file, then parse the cells in certain columns to extract timing values embedded in other data) and (b) figure out how to store those values in a database (i.e., do some data modeling and schema design). Our lesson plan for Version 4 already has a slot for a lecture on modeling: maybe this should be given more emphasis?

Day 2: More Sticky Notes

Tue, 04 May 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Day 2. I feel I should say something like, "We traversed the lower ridge, and are camped in a small valley overlooked by the main peak—I can hear wallabies hooting in the distance," but (a) it hasn't been quite that dramatic and (b) I don't think wallabies hoot. In this reality, Jason, Jon, and I spent another afternoon mapping content: in particular, disentangling the mouse-clicking and data-analyzing aspects of the spreadsheet lecture. Tomorrow we hope to do our first pair of five-minute trial screencasts, just to see how well (or how poorly) our chosen tools work.

Setting Up a New Windows Machine

Mon, 03 May 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Setting up a new machine is never fun, but it's always interesting to compare what different people have in their toolkits. Here's what I have installed so far on the desktop machine I'll be using to do Software Carpentry course develpoment:

Windows XP (yes, I still use XP)

Cygwin (a collection of Unix emulation tools for Windows)

Adobe PDF Reader

Audacity (for sound editing)

MikTeX (an all-in-one LaTeX for Windows)

GIMP (the open source clone of Photoshop)

Google Talk (for chatting with my co-conspirators)

Inkscape (an open source vector drawing tool)

iTunes (this course brought to you by Beethoven, John Coltrane, and a variety of 80s bands)

MATLAB (I'm using R2008b)

MWSnap (for doing screen captures)

Microsoft Office 2007 (because most scientists who use anything, use Excel)

OpenOffice (but I'll mostly use MS Office)

Python 2.6 (because some of the packages I want don't exist for Python 3 yet)

R 2.11 (because after Python and MATLAB, R and Perl are our next target languages)

SciPy (which gives me almost everything I want that isn't in the standard Python install)

Skype (for talking to collaborators)

VLC (for viewing video files)

yED (for creating and editing simple bubble-and-arrow diagrams)

What's missing (so far) are the science-specific tools that I'll be adding as needed for particular topics—I'll blog about those as I install them. What's interesting is how many of these tools are free (as in beer) and open (as in modifiable): I'll be blogging soon about the distinction, and why I think both are important to science, as well.

T Minus One

Sun, 02 May 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Tomorrow (May 3, 2010) will be my first day of full-time work on Version 4 of this course (where "full-time" means "except for getting my daughter a new passport and taking her to the dentist for the first time"). I'm very pleased that Jason Montojo and Jon Pipitone will be joining me part-time for May and June — they helped organize and deliver the most recent run of the course at the University of Toronto, and with them on board, I'm confident that we'll be able to hit our first milestone.

Apologies for the Flurry of Re-Posts

Mon, 19 Apr 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Our apologies for the flood of re-posts that some of you may have seen over the weekend: apparently, adding a category to a post, or changing its existing category, makes some blog readers believe the whole post is new. We're sorry for any confusion or inconvenience the clutter may have caused.

File Sharing for Scientists

Fri, 16 Apr 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A scientist I recently met in Toronto had a problem: how to share large files with colleagues. Each file is a couple of hundred megabytes; dozens are produced each week, but each is only interesting for a couple of months; and there are confidentiality issues, so some kind of password protection is needed. Conventional file-sharing services like Dropbox aren't designed for data that size, so in the end she bought a domain and set up secure FTP.

Scimatic Sponsorship

Thu, 15 Apr 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We're very pleased to announce that Scimatic Software, a Toronto based company that specializes in the development of software for the scientific community, has come on board as a sponsor of this project. Many thanks to Jamie McQuay and Jim Graham!

More on Instructional Design

Mon, 12 Apr 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Like many programmers, I've learned most of what I know by poking around and breaking things. Quite naturally, that has led me to believe that this is the best way to learn—after all, if it worked for me, it has to be pretty good, right? But research says otherwise. Kirschner, Sweller, and Clark's paper, "Why Minimal Guidance During Instruction Does Not Work: An Analysis of the Failure of Constructivist, Discovery, Problem-Based, Experiential, and Inquiry-Based Teaching", was published in Educational Psychologist in 2006, but the whole text is available online.

Teaching Open Source

Mon, 12 Apr 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Over at opensource.com, Red Hat's Greg DeKoenigsberg has a post about a new collaboratively-authored textbook on open source software aimed squarely at undergrad courses. As Máirín Duffy points out in the first comment, it's very code-centric, but in my experience, that's the right approach: students won't be ready for discussion of design until they're proficient in coding [1]. I'm looking forward to borrowing lots from the book for Software Carpentry...

Measuring Science

Sun, 11 Apr 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Julia Lane, the director of the Science of Science & Innovation Policy program at the National Science Foundation, wrote an article for Nature a couple of weeks ago titled "Let's make science metrics more scientific". As the summary at the start says:

Existing metrics have known flaws

A reliable, open, joined-up data infrastructure is needed

Data should be collected on the full range of scientists' work

Social scientists and economists should be involved

The same points could be made about evaluating software developers (or any other kind of knowledge worker). The devil, as always, is in the details, and unfortunately I have to start doing evaluations before those details are worked out. Several of the supporters for this course need me to demonstrate its impact on the productivity of the scientists who take it (so that they can in turn justify their contribution to their funders). It isn't enough to ask students who have completed the course whether they think they know more about programming than they used to: ignoring the obvious problems of survivor bias and self-assessment, I would still have to demonstrate that making people better programmers also makes them better scientists. I believe it does, but belief is not evidence, and doesn't convey scale.

Platforms

Thu, 08 Apr 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

After Thursday's post-mortem on the latest offering of Software Carpentry at the Universitiy of Toronto, I had a chance to talk further with Jon Pipitone, who was one of the tutors (and who is just wrapping up an M.Sc. looking at code quality in climate models). We got onto the topic of infrastructure for Version 4, which needs to be settled quickly.

a way to deliver content to students, including text and images, audio/video, exercises (with solutions), sample data sets, and useful software;

a way for students to feed questions back to the course organizers (asynchronously through email and bulletin boards and/or synchronously through VoIP and desktop sharing);

a way for instructors (who may or may not be contributors) to respond to students;

a way for lay contributors (who may also be students) to offer new content, from pointing out typos to providing exercises or whole new lectures; and

a way for core contributors to manage and edit contributions, create some of their own, et cetera.

This description implies some social infrastructure, including:

Software Carpentry for Economists in Mannheim This Autumn

Thu, 08 Apr 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Hans-Martin von Gaudecker is planning to teach a Software Carpentry-style course for economists at Universität Mannheim this autumn — as his announcement says, "I think it is amazing that a profession obsessed with efficiency affords a very obvious inefficiency: Most researchers nowadays spend a fair share of their time programming, but hardly anyone has been taught to do that well." I'll post updates here as he sends them.

Feedback and Boundaries

Sun, 04 Apr 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Thanks to the initiative of Dominique Vuvan (who took Software Carpentry last summer), we ran a semi-formal version of the course from last November through to this past week for grad students in Psychology, Linguistics, and a few other disciplines at the University of Toronto. Weekly tutorials were offered in both Python and MATLAB by graduate teaching assistants from Computer Science, covering roughly half of the existing material.

Models To Imitate

Thu, 01 Apr 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

My father once told me that a week of hard work can sometimes save you an hour of thought. In that spirit, I've been looking for asynchronous online courses to imitate. I previously mentioned MIT's Open Courseware, CMU's Open Learning Initiative, and (closer to my scale) Saleem Khan's Khan Academy. Google Code University's lessons on programming languages are also on my radar—I'll blog more about them once I finish the Python material—but another model that I'm looking at closely is Teaching Open Source, a collaborative effort to get more open source into college and university courses. I first encountered them through POSSE (Professors' Open Source Summer Experience), which they describe as:

Simon Singh Wins (and So Does Science)

Thu, 01 Apr 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Simon Singh, the science journalist who was sued for libel by the British Chiropractic Association, has won the right to rely on the defense of "fair comment". (Full ruling linked from this Index on Censorship post.) Singh had pointed out that there's no evidence to back up BCA claims that their particular brand of pseudoscience could help with asthma and other ailments; it has taken him two years and £200,000 later to get this far, and it may be another two years before the matter is finally settled, but this is an important victory for everyone who believes in rational inquiry.

Periodic Table of Science Bloggers

Wed, 31 Mar 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

David Bradley has created a periodic table of science bloggers that regular readers might enjoy:

Formats

Tue, 30 Mar 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As I said in last week's announcement, and mentioned again in a later post, one of the main goals of this rewrite is to make it possible for students to do the course when and where they want to. That means recording audio and video, but much of the material will probably still be textual: code samples (obviously), lecture notes (for those who prefer skimming to viewing, or who want to teach the material locally), and exercises will still be words on a virtual page. And even the AV material will (probably) be accompanied by scripts or transcripts, depending on what turns out to work best.

What's Not on the Reading List

Mon, 29 Mar 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I mentioned yesterday that I maintain a list of books that haven't been written yet. Partly it's an exercise in sympathetic magic—if the reviews exist, maybe the books will follow—but it's also useful for organizing my thoughts about what a programmer's education should look like. Looking at the books I've matched to various topics in the Software Carpentry course outline, there are some distressing gaps:

Given that programmers spend upwards of 40% of their time debugging, there are very few books about it, and only one collection of exercises (Barr's Find the Bug).

There's a lot on higher-level programming techniques, but it's scattered across dozens of books as disparate as The Seasoned Schemer, Effective C++, and The Practice of Programming. I haven't read Perrotta's Metaprogramming Ruby yet, but it looks like it will be another rich source of ideas.

Material on systems programming—manipulating files and directories, running sub-processes, etc.—is equally scattered. The Art of Unix Programming includes all the right topics, but covers too much, in too much detail, at too low a level. Gift & Jones' Python for Unix and Linux System Administration has the same two faults (from Software Carpentry's point of view—I think both are excellent books in general), but uses a scripting language for examples, so it made the list.

Mark Guzdial and others have done excellent research showing the benefits of teaching programming using multimedia, i.e., showing students how to manipulate images, sound, and video as a way of explaining loops and conditionals. That's half of why the revised course outline includes image processing early on (the other halves being "it's fun" and "it's useful"). Once again, most of what I'm familiar with is either documentation for specific libraries, or textbooks on the theory of computer vision, but there are some promising titles in the MATLAB world that I need to explore further.

Performance. It's been 15 years since I first grumbled about this, and the situation hasn't improved. Most books on computer systems performance are really textbooks on queueing theory; of that family, Jain's Art of Computer Systems Performance Analysis is still head and shoulders above the crowd. Souders' High Performance Web Sites is the closest modern equivalent I've found to Bentley's classic Writing Efficient Programs, but neither is really appropriate for scientists, who need to think about disk I/O (biologists and their databases), pipelining and caching (climatologists with their differential equations), and garbage collection (everybody using a VM-based language). I had hoped that High Performance Python would fill this gap, but it seems to have been delayed indefinitely. (And yes, I've looked at Writing Efficient Ruby Code; it has some of what our students want, but not nearly enough.)

There are lots of books about data modeling, but all the ones I know focus exclusively on either the relational approach or object-oriented design, with a smattering that talk about XML, RDF, and so on. I haven't yet found something that compares and contrasts the three approaches; pointers would be welcome.

Web programming. There are (literally) thousands of books on the subject, but that's the problem: almost all treatments are book-length, and this course only has room for one or two lectures. It is possible to build a simple web service in that time, but only by (a) using a cookbook approach, rather than teaching students how things actually work, and (b) ignoring security issues completely. I'm not comfortable with the first, and flat-out refuse to do the second: if this course shows people how to write a simple CGI script that's vulnerable to SQL injection and cross-site scripting, then it's our fault when the students' machines are hacked. This gap is as much in the available libraries as in the books, but that doesn't make it any less pressing.

Given these gaps, I may drop one or two topics (such as performance and web programming) and either swap in one of the discarded topics or spend more time on some of the core material. I'm hoping neither will be necessary; as I said above, pointers to books in any language that are at the right level, and cover the right areas, would be very welcome.

Recommended Reading

Sun, 28 Mar 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I'm slightly obsessed with reading lists. (I even maintain a list of books that haven't been written yet, in the hope that it will inspire people to turn some of the entries from fantasy into reality.) Partly to give credit to all the people whose work inspired Software Carpentry, and partly to guide students who want to learn more than we can fit into a double dozen lectures, I have started a bibliography, and added links to relevant books to the lecture descriptions in the course outline. Pointers to other material would be very welcome; I will blog soon about areas that I feel are particularly lacking.

Instructional Design

Fri, 26 Mar 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As well as deciding on the format of the course, I have to re-shape its content. In contrast to e-learning, there seems to be a lot of solid material available on instructional design. The most useful guide I've found so far is Wiggins & McTighe's Understanding by Design. I was initially a bit put off by the micro-industry the authors have built around the book, but its step-by-step approach immediately felt right:

Online Delivery

Fri, 26 Mar 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As the announcement of Version 4 said, Software Carpentry is being redesigned so that it can be delivered in several ways. I want to support:

traditional classroom lectures, with someone at the front of the room talking over a series of slides and/or coding sessions to a captive audience;

students reading/viewing material on their own time, at their own pace, when and as they need it; and

hybrid models, in which students work through as much as they can on their own, then get help (face-to-face or over the web) when they hit roadblocks.

#1 isn't easy to do well, but the challenges involved are well understood. #2 and #3 are going to be a lot harder: it's new ground for me, and despite the fact that the Internet is older than many of my students, most of the educational establishment still thinks of it as "new" as well.

Software Carpentry Version 4 is a Go!

Thu, 25 Mar 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I am very excited to announce that I am going to work full-time on revising the Software Carpentry course from May 2010 to May 2011. This work has been made possible by the generosity of our members:

The School of Informatics and Computing at Indiana University

The Gene Expression in Disease and Development Focus Group at Michigan State University

MITACS

The Centre for Digital Music at Queen Mary University of London

SciNet

SHARCNET

The UK Met Office

I would also like to thank The MathWorks, the University of Toronto, the Python Software Foundation, and Los Alamos National Laboratory, whose support over the past 13 years has allowed us to help scientists use computers more productively.

Summer Course: Analyzing Next-Generation Sequencing Data

Thu, 25 Mar 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Analyzing Next-Generation Sequencing Data

May 31 — June 11th, 2010
Kellogg Biological Station, Michigan State University
CSE 891 s431 / MMG 890 s433, 2 cr

http://bioinformatics.msu.edu/ngs-summer-course-2010

Applications are due by midnight EST, April 9th, 2010.

Course sponsor: Gene Expression in Disease and Development Focus Group at Michigan State University.

Instructors: Dr. C. Titus Brown and Dr. Gregory V. Wilson

This intensive two week summer course will introduce students with a strong biology background to the practice of analyzing short-read sequencing data from the Illumina GA2 and other next-gen platforms. The first week will introduce students to computational thinking and large-scale data analysis on UNIX platforms. The second week will focus on mapping, assembly, and analysis of short-read data for resequencing, ChIP-seq, and RNAseq.

Now on Twitter

Tue, 23 Mar 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

You can now follow our progress at 'swcarpentry' on Twitter.

How Much Of This Should Scientists Understand?

Thu, 11 Mar 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Let's start with the problem description:

All of the Software Carpentry course material (including lecture notes, code samples, data files, and images) is stored in a Subversion repository. That's currently hosted at the University of Toronto, but I'd like to move it to the software-carpentry.org domain (along with this blog). However, software-carpentry.org is hosted with site5.com, who only provide one shell account per domain for cheap accounts like the one I bought.

Panton Principles

Sun, 28 Feb 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Via Cameron Neylon: the Panton Principles are guidelines for open data in science. In full:

Science is based on building on, reusing and openly criticising the published body of scientific knowledge.

For science to effectively function, and for society to reap the full benefits from scientific endeavours, it is crucial that science data be made open.

By open data in science we mean that it is freely available on the public internet permitting any user to download, copy, analyse, re-process, pass them to software or use them for any other purpose without financial, legal, or technical barriers other than those inseparable from gaining access to the internet itself. To this end data related to published science should be explicitly placed in the public domain.

Eighty Per Cent!

Thu, 25 Feb 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As of this morning, I have signed commitments for 4/5 of the money I need to spend a year working full-time on updating the Software Carpentry course. If you, or someone you know, would like to help me help scientists be more productive, please get in touch: I have only 64 days in which to find the $30K I still need.

BEACON Funded!

Mon, 22 Feb 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Congratulations to Titus Brown and others on the NSF's announcement that it will fund the BEACON (Bio/computational Evolution in Action Consortium) Science and Technology Center. BEACON "...BEACON is focused on studying the evolution of organization across multiple scales—from genomic and cellular, to multicellular, to inter-multicellular (a.k.a. social)—using techniques from experimental evolution, modeling, and digital life systems." Long story short, this means that Michigan State University and its partner institutions "...has money explicitly for supporting students doing really sexy interdisciplinary work combining computation and biology."

Two Views

Fri, 12 Feb 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Darrell Ince (who, as far as I know, hasn't worked directly with climate scientists or their software) wrote about ClimateGate for The Guardian. In response, Steve Easterbrook (who has) wrote a detailed rebuttal. Here's hoping the latter is as widely read as the former.

It Seems That Everyone Cares

Sun, 24 Jan 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Ars Technica isn't primarily a science site, but even they are now worried about reproducibility in computational science. I think it no longer matters how important this "crisis" actually is—sooner or later, major funding agencies are going to mandate adoption of something like the Open Provenance Model. Problem is, given the current skill set of the average scientist, that will almost certainly translate into burden without benefit.

Big Science == Big Skills Gap

Wed, 20 Jan 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Over on Nature News, Eric Hand's article "'Big science' spurs collaborative trend" is subtitled, "Complicated projects mean that science is becoming more globalized." It talks about the benefits of international collaboration, but what it doesn't say is that sharing ideas, results, procedures, and software requires skills that aren't part of the standard curriculum. One of the main goals of the rewrite of Software Carpentry is to teach scientists some of what they need to know in order to do what Hand describes without heroic effort. I'd be grateful for suggestions about topics and tools that ought to be on the list, but aren't.

Was Designed To, But Didn't

Mon, 18 Jan 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Michael Clarke has written a thoughtful post exploring why the web hasn't disrupted scientific publishing, even though it was designed to do exactly that.

Podcast with Jon Udell

Wed, 13 Jan 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Jon Udell recently interviewed me about Software Carpentry and related topics—the podcast is now on his web site.

Whatcha Gonna Do When They Come For You?

Wed, 13 Jan 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

First it was pharma companies withholding "unhelpful" data, then it was ClimateGate, and now there's this:

One of the founders of the controversial 'Baby Einstein' range of products is taking the University of Washington to court in an attempt to force the institution's scientists to release their raw data to him...William Clark...wants records relating to two studies published in 2004 and 2007. The latter found an "association between early viewing of baby DVDs/videos and poor language development" while the former suggested "efforts to limit television viewing in early childhood may be warranted".

How We Got Here, and Where We Are

Sun, 10 Jan 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I gave a talk in Santa Fe early in 1997 describing a set of articles I'd organized for the Summer 1996 and Fall 1996 issues of IEEE Computational Science and Engineering (now Computing in Science & Engineering on the subject, "What should computer scientists teach physical scientists and engineers?" After the talk, John Reynders (then the director of the Advanced Computing Lab at Los Alamos National Laboratory) challenged me to put my money where my mouth was and actually teach basic software development skills to working scientists.

New Challenges

Thu, 07 Jan 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As some of you already know, my contract with the University of Toronto runs out this spring, and I have decided not to seek renewal. I've learned a lot in this job, and had a chance to work with some great people, but it's time for new challenges.

What I'd most like to do next is spend a year working full-time on the Software Carpentry course—of all the things I've done, it's the one that I think has the most potential to make scientists' lives better. My goal is to raise approximately CDN$25,000 from each of half a dozen sponsors so that I can reorganize and revamp the content, add screencasts and video lectures, and generally drag it into the 21st Century. An abbreviated proposal is included below the cut—if you or anyone you know would be interested in discussing possibilities, please give me a shout.

Osmosis is Just a Fancy Name for Failure

Wed, 30 Dec 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

My last post linked to a PLoS paper by Dudley and Butte on developing effective bioinformatics programming skills. I asked, "How many hours do the authors think are needed to acquire these skills?" In response, Atul Butte said, "I think the ideal scenario is when one's research projects enable one to learn these skills, so that these skills get learned in a practical way outside the classroom too, while doing science," while Luis Pedro Coelho asked, "Does it matter over the long (or even medium) term? Isn't improving your skills even you if aren't being immediately productive what school is for?"

Dudley and Butte on Software Skills

Sun, 27 Dec 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Via Titus Brown, a new PLoS paper titled "A Quick Guide for Developing Effective Bioinformatics Programming Skills" by Joel Dudley and Atul Butte. Their recommendations are:

Programming languages

Embracing open source

Unix command-line skills

Keeping projects documented and manageable

Preserving source code with version control

Embracing parallel computing paradigms

Structuring data for speed and scalability

Understanding the capabilities of hardware

Embracing standards and interoperability

Put a high value on your time

I think all these things matter, but:

NSF Programs

Sat, 19 Dec 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I'd be interested in hearing from anyone who has enough direct experience of the following NSF programs to know whether they might be willing to support Software Carpentry:

Course, Curriculum, and Laboratory Improvement

Integrative Graduate Education and Research Traineeship

CISE Pathways to Revitalized Undergraduate Computing Education

Innovations in Engineering Education, Curriculum, and Infrastructure

Double Standards

Fri, 18 Dec 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Nicola Scafetta is refusing to release the software on which he bases his claims that the sun is responsible for much of terrestrial warming during the last century. I obviously think that scientists should be required to do this as a condition of publication; coming as this does on the heels of Climategate, it will be interesting to see if journals finally start pushing in that direction. It also highlights the need to add more material to this course to cover packaging for release and data provenance.

Why Opening Up (Probably) Wouldn't Help

Fri, 11 Dec 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Good post from Steve Easterbrook on why open-sourcing climate models probably wouldn't make a difference.

Thanks, Jamie

Sat, 28 Nov 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The warmup tutorials for our grassroots Software Carpentry course started this week, and we'd like to send a "thank you" to Jamie Winter at The MathWorks, who has provided students with temporary licenses for MATLAB. It's all been very last minute, and we're grateful to Jamie for pulling this off on such short notice.

Caesar's Wife

Thu, 26 Nov 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Improving the way scientists use computers isn't just about making them more productive: it's also essential to defending the integrity of their work. Stories like this one about a researcher struggling in vain for three years to replicate someone else's results can only undermine public confidence at a time when we need to make a lot of hard decisions in a hurry. Sadly, we have no one to blame but ourselves...

Tutorials Start This Week

Tue, 24 Nov 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

After a lot of hard work from Dominique and Jon, we're kicking off warmup tutorials for Software Carpentry this week. 65 students from Psychology, Linguistics, Chemical Engineering, and a couple of other departments will get three weeks of review on basic programming, then start the regular material in January. Our thanks to MITACS, the MathWorks, SciNet, and DCS for their support.

Serendipitous and Unexpected

Sun, 22 Nov 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Via Ryan Lilian:

Most research effort does not produce what is thought of as a traditionally publishable result. That doesn't mean, however, that nothing was gained by conducting the research. These results, whether they are failures or merely perplexing, can provide valuable insights into open problems and prevent other researchers from duplicating work. We started a journal that focuses on serendipitous (I have no idea why this worked) and unexpected (it seems like this technique should work on this problem but it doesn't) results. The goal of the journal is to provide a venue where ideas can flow and be debated.

Special Issue of Computing in Science and Engineering

Wed, 18 Nov 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A special issue of Computing in Science & Engineering that Andy Lumsdaine and I edited, devoted to software engineering in computational science, is now available. We'd like to thank everyone who contributed:

Report on the Second International Workshop on Software Engineering for CSE, by Jeffrey Carver (University of Alabama)

Managing Chaos: Lessons Learned Developing Software in the Life Sciences, by Sarah Killcoyne and John Boyle (Institute for Systems Biology)

Scientific Computing's Productivity Gridlock: How Software Engineering Can Help, by Stuart Faulk (University of Oregon), Eugene Loh and Michael L. Van De Vanter (Sun Microsystems), Susan Squires (Tactics), and Lawrence G. Votta, (Brincos)

Mutation Sensitivity Testing, by Daniel Hook (Engineering Seismology Group Solutions) and Diane Kelly (Royal Military College of Canada)

Automated Software Testing for MATLAB, by Steve Eddins (The MathWorks)

The libflame Library for Dense Matrix Computations, by Field G. Van Zee, Ernie Chan, and Robert A. van de Geijn (University of Texas at Austin), and Enrique S. Quintana-Ortí and Gregorio Quintana-Ortí (Universidad Jaime I de Castellón)

Engineering the Software for Understanding Climate Change, by Steve Easterbrook (University of Toronto) and Timothy Johns (Hadley Centre for Climate Prediction and Research)

Cloud Computing for Beginners

Sun, 15 Nov 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Ana Nelson has posted step-by-step instructions showing how to use Amazon's EC2 cloud computing platform to run simulations. There are still a lot of fiddly details, but the barriers to entry are getting lower all the time...

Packaging

Fri, 13 Nov 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Martijn Faassen has posted a nice history of packaging and distribution tools for Python. Yes, it's a topic only a geek could love, but anyone who wants to distribute software to other people needs to grapple with these issues. The question is, should these tools, the problems that motivate them, and the technology underlying them be part of this course? Or should something equivalent (and if so, what)?

Python in Science

Fri, 06 Nov 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Guido van Rossum just posted a description of a variety of scientific Python projects. The diversity is pretty impressive...

Our Target Audience

Sun, 01 Nov 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Some graduate students at the University of Toronto have asked us to run the course for them later this fall or during the winter. There's an obvious selection bias (if they were expert programmers, they wouldn't need this course), but I think they're pretty representative of scientists at their level:

01. Name 39/39 100%

02. Email address 39/39 100%

03. Level of study

MSc 12/39 30%

PhD 27/39 70%

04. Primary programming language

MATLAB 16/39 41%

Python 2/39 5%

Other 6/39 15%

None 15/39 39%

05. Knowledge of primary language

Don't know how to use it 28/39 72%

Understand basic commands 10/39 26%

Can program competently 1/39 2%

Expert 0/30 0%

06. What other languages do you know?

HTML 11/27 41%

R 4/27 15%

Other (VB, Java, Perl, etc.) 10/27 37%

No answer 12/39

07. Would you like pre-class tutorial on

programming basics (loops, files, if/else)?

Yes 36/39 92%

No 3/39 8%

08. Do you have a laptop?

Yes 39/39 100%

No 0/39 0%

09. Preferred OS

Windows XP 14/39 36%

Windows Vista 12/39 31%

Mac OS X 7/39 18%

Linux/Unix 9/30 0%

10. Do you have a MATLAB license?

Yes 9/39 23%

No 30/30 77%

11. Which topics are you interested in?

Databases 16/39 47%

Functions and Modules 14/39 41%

Debugging 10/39 29%

Image Processing 10/39 30%

Object-Oriented Programming 10/39 30%

Web Application Programming 9/39 26%

GUI Programming 8/39 23%

Web Client Programming 7/39 21%

Computational Complexity 6/39 18%

How Web Servers Work 6/39 18%

Regular Expressions 6/39 18%

XML 6/39 18%

Automated Builds 5/39 15%

Sets and Dictionaries 5/39 15%

Unix Shell Scripting 5/39 15%

Binary Data 3/39 9%

Empirical Software Engineering 3/39 9%

Quality Assurance 3/39 9%

Unit Testing 3/39 9%

Version Control 3/39 9%

Software Development Lifecycles 1/39 3%

Other (please specify) 10/39 30%

By Popular Request...

Fri, 30 Oct 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

...I have added a lecture on high performance computing to the revised outline for the course. Several people suggested it, and what's the point of asking for feedback if I don't listen?

Cryptography Isn't Security

Fri, 23 Oct 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

One topic that I've tried to include in this course a couple of times, without success, is security. I feel irresponsible not saying something about how to share safely, but I've never found something that (a) would fit into one hour, (b) wasn't platitudes, and (c) gave listeners something they could act on.

One reviewer suggested talking about public/private key pairs (to help people set up SSH), signing things digitally, and the like. I'm leery of going down that road, though, since it could easily leave people with a misplaced faith in technical solutions to security problems. As always, suggestions would be welcome...

Creating New Niches

Wed, 21 Oct 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

"Publish or perish" is the central credo of academic life: despite all the hoopla about the blogosphere and online what-not, the reality for most of us is that if our work doesn't get into a respected journal or conference, it doesn't count.

But what do you do if there isn't a home for your kind of work? People working in scientific computing have been struggling with this for at least a quarter century: while there are many places to submit the results of programs, there are very few places where you can publish a description of the program itself, even if building it took years and required one intellectual breakthrough after another. In contrast, if you design a new telescope, there are at least half a dozen places you could turn.

Should Modeling Be Part of This Course?

Wed, 21 Oct 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Jon Pipitone has a good description on his blog of work the grad students in our department are doing to translate work in climate change into software engineering terms. Their first step is to represent the ideas in MacKay's excellent Sustainable Energy Without the Hot Air in two of the graphical notations that computer scientists use for system design. I was initially very skeptical, but looking at their work so far, I'm quite impressed. My question is, would it be useful for scientists to know how to do this themselves? More specifically, is the lecture on data modeling that I've planned to include in Version 4.0 worthwhile or not?

Revised Plan

Fri, 16 Oct 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I've posted updates to the revised course outline. In particular, I have:

Moved testing earlier.

Clarified intent in a couple of places.

Made an list of things we're leaving out.

As always, feedback would be welcome. I'd also be grateful for pointers to places that might fund this work: as I've found in the past, many people think the course is a good idea, but it doesn't quite fit into their funding mandates :-(

Videos from Symposium Are Now Online

Thu, 08 Oct 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I have put video recordings of the guest talks given at our July 29 symposium on Science 2.0 — please click the titles of the talks on the index page. Thanks again to all of our speakers, to the MaRS Centre technicians for the raw recordings, and to Tanya Murdoch for editing.

Comments on Course Reorganization

Tue, 06 Oct 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I'm grateful to Lorin Hochstein for sending detailed feedback on my proposal to reorganize the course. His comments are below, with my replies and his counter-replies interspersed; more comments would be very welcome.

Content I think you could drop if you wanted to save time:

Read Data Directly From Hardware. I suspect that this would be relevant to only a small minority of your audience. Especially if you're teaching the course mostly in Python, because this is the sort of thing you should really do in C.

The Hacker Within

Mon, 05 Oct 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I did a web search for "software carpentry" a couple of weeks ago to update my impressions of how the material was being used, and stumbled across The Hacker Within,a self-help group at the University of Wisconsin for grad students in science who want to improve their computing skills. I had a great chat with the organizers by phone on Thursday, and hope to get there in the spring to meet them in person. It's always great to meet the people the course is designed to help; I'd welcome pointers to other groups if you have them.

A Strange Obsession

Tue, 29 Sep 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I just spent a few minutes browsing these slides from a talk given in Karlsruhe in September by Microsoft's Fabrizio Gagliardi. The talk's title was "Cloud Computing for Scientific Research", and it's chock-full of big: mega-this, peta-that, and isn't it all exciting? The only mention of anything at the desktop scale is on slide 19, which mentions a plugin to allow MATLAB to talk to the cloud, and Excel views of Azure data. Once again, I'm puzzled (and a bit disappointed) that the world's premier desktop software company has decided to ignore what most scientists care about most. I'm equally disappointed that there was nothing at all in these slides about improving scientists' skills, especially since Microsoft has invested so heavily in improving its own processes. Oh well...

Presentation, Presentation, Presentation

Thu, 24 Sep 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Right now, the Software Carpentry material is basically printed pages on the web. Each lecture is a linear HTML page: bullet point follows bullet point, interrupted only by code snippets, tables, and diagrams. If I'm going to update the content, I'd also like to update the presentation; the question is, "To what?" An audio recording of me talking over the slides would add some value, though I think that typing in what I would say would probably be more useful, since most people can read faster than I can speak, and audio still isn't googleable.

Grant Proposal

Tue, 22 Sep 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Rebuilding the Software Carpentry course is too big to do in anyone's spare time, so I've submitted a small funding application (included below). If you know any venues that might welcome a similar proposal, I'd be grateful for pointers.

A Self-Paced Software Skills Bootcamp for Research Scientists

Proposed Dates and Venue

Trial run to take place online in July 2010.

Type of Activity

Another Reason to Care About Provenance

Mon, 21 Sep 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A vice president at ETH is resigning today because of accusations of fraud. A key lab notebook is missing, retractions have been retracted, and according to ETH President Ralph Eichler, "there is now no legal way of finding out for sure who was responsible for the falsifications."

Question: if someone accused you of falsifying results, how well and how easily could you defend yourself? How long would it take you to pull together all the notes, data, and programs you used three years ago to produce the paper being challenged? And would you career survive? Even if you "won", you would probably lose weeks or months of research time.

Updated Outline for Revised Course

Fri, 18 Sep 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I have updated the description [no longer online] of how I plan/hope to reorganize the course. My thanks to everyone who commented on the earlier draft; I'd be very grateful for feedback on this one as well—I realize that some of the lectures are still hopelessly ambitious, but I hope it's at least a target to shoot at.

Partial Outline of New Version of Course

Tue, 15 Sep 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I have put a mostly-complete draft of my ideas for a rewrite of the course online. This version would start with specific problems, then backfill the tools and skills needed to solve them, but there are some big outstanding issues:

There still isn't anything on the computational thinking skills that Jon Udell identified. I don't know how to tackle these without relying more than I want to on closed-source offerings like Yahoo! Pipes that could disappear at a moment's notice.

I just can't see how to fit some topics—particularly object-oriented programming and web programming—into the available space (a maximum of two hours of lecture and four hours of lab for each). What are the minimum useful problem/lecture for each?

There are a few others as well (just search for 'TODO'), but those are the big ones. Any help you can give would be very welcome.

Job Opening: MITACS Scientific Coordinator

Fri, 11 Sep 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

MITACS (which provided funding for this summer's offering of the course, and which funds a lot of other mathematically-oriented work in Canada) has an opening for a scientific coordinator. Details are in the job posting; specific responsibilities include:

Promote MITACS programs within the scientific community and provide a point of contact for feedback and questions

Ensure quality, fairness and timeliness of scientific peer-review processes

Program management tasks such as:

Establish program strategy and action plans for new MITACS programs

Develop and monitor program budgets

Coordinate with other MITACS staff on program implementation and ongoing monitoring of program effectiveness

Work closely with the Communications and Government Relations Department on program promotion and feedback

Provide regular status updates, reports and program evaluation

Other duties as assigned from time to time

If you'd like to help shape the development of mathematics and science in Canada, this might be the job for you...

Two Links

Fri, 11 Sep 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Mendeley has been getting some good press—anyone have first-hand experiences to share?

A Nature special on data sharing. Bryn Nelson's piece on empty archives is particularly worth reading.

R for Programmers?

Sat, 05 Sep 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

What's the best introduction to the statistical language R for experienced programmers who are more interested in its support for objects, systems programming, and the like than they are in statistics?

Is The Future Waving At You?

Sun, 30 Aug 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Cameron Neylon has been playing with Google Wave, and he likes it. His presentation at Science Online in London in August explains why (you can also watch video, though sadly there's no soundtrack). He's even writing robots to automate some scientifically interesting tasks. Nature News liked Wave too, which (a) reminds me yet again of how prescient Jon Udell's "Internet Groupware for Scientific Collaboration" was ten years ago, and (b) makes me wonder (also again) how much this course should be re-thought.

How Important is Geospatial Data to You?

Wed, 26 Aug 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Software Carpentry currently teaches students how to manipulate text (using regular expressions), XML (using DOM), relational data (with SQL), and binary data. A decade ago, when we first put the course together, that covered everything I'd ever seen more than one or two scientists use. Today, though, an increasing number are using geospatial (map) data as well. How important is this to your work? If the answer is "very", what data do you work with, what do you do with it, and what would you like to be able to do?

Bad News and Good News

Mon, 24 Aug 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The bad news is, retractions of scientific papers have risen tenfold since 1990. The good news is, the rate has gone from 0.0007% to 0.007%. Going back to bad news, though, some estimates of how many papers ought to be retracted are around 1%, so we still have a long way to go.

Playing Safe

Mon, 24 Aug 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This thoughtful article from the New York Times asks whether the current grant system for funding research discourages researchers from taking risks. My personal experience undoubtedly biases me, but I tend to agree—the problem is coming up with something better.

Science and JoVE

Mon, 24 Aug 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Science and JoVE, the Journal of Visualized Experiments, have partnered to produce and publish scientific videos online. The aim is to enhance scientific articles published in Science through video demonstrations of experimental techniques. See the announcement for a link to the first joint work.

Who Owns Your Data?

Mon, 24 Aug 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Nature's "Great Beyond" blog reports another attempt to force climate scientists to release their data. I'm of at least two minds on this: I believe openness is absolutely crucial to science (and society as a whole), but I understand scientists' concern about being scooped, and equally their concern about having their work misrepresented or quoted out of context. Mostly, I come down on the side of openness—how about you?

The Delight Is In The Details, Too

Sun, 23 Aug 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

They say the devil is in the details, but so's the delight, because it's the details that determine whether something works or doesn't. So let's take a look at how to translate the last post's "big picture" into actual course content.

Every competent developer uses some kind of tool to automate tasks that involve dependencies. The best known is still Make, which compares the last-modified-times on files to see which ones are stale, and runs shell commands to bring them up to date. Ant, Rake, and whatever's built into your IDE all work basically the same way, and can all be used (with greater or less ease) to recompile software, re-run tests, prepare distributions, update web sites, and what have you.

The Big Picture

Fri, 21 Aug 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

One of the lessons we learned at Los Alamos National Laboratory in the 1990s and early 2000s is that most scientists don't actually want to learn how to program—they want solve scientific problems.To many,programming is a tax they have to pay in order to do their research.To the rest,it's something they really would find interesting,but they have a grant deadline coming up and a paper to finish.

It's Like Not Wearing Your Cleats in the House

Sat, 15 Aug 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Carl Zimmer, one of my favorite science writers, recently posted about three new books aimed at scientists: Unscientific America, Am I Making Myself Clear?, and Don't Be Such a Scientist. All three are aimed squarely at the biggest problem modern science faces—the inability of most scientists to explain themselves to non-specialists—and all three are now on my read-soon list. I don't think communication skills will ever be part of this course, but given the problems our planet faces, they damn well need to be part of every scientist's education.

You Can Do a Lot Without Programming

Sat, 15 Aug 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

From Cameron Neylon, a short video showing how to embed molecules from the ChemSpider service into a wiki page. I was surprised and impressed to discover during his visit to Toronto just how little programming Cameron does: mostly, he leverages his understanding of how information moves around the Internet to plumb existing tools and services together. This is part of (or dependent on) what Jeannette Wing calls "computational thinking", and one of the goals for the next revision of this course is to focus more on those kinds of skills.

American Scientist Article on How Scientists Use Computers

Thu, 06 Aug 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

American Scientist has just published a short article summarizing the results from the survey we did last year of how scientists actually use computers.

The Ice Cream Test

Tue, 04 Aug 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Extensive experimentation has uncovered a foolproof way to tell Aussies from Brits: simply observe their reaction to ice cream.

Aussie Brit

Guest Speakers' Slides Now Available

Mon, 03 Aug 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Our guest speakers' slides are now on the web; video will follow soon.

What *Is* Open Science?

Mon, 03 Aug 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Over at the OpenScience Project, Dan Gezelter is trying to define what "open science" means. He thinks there are four key points:

Transparency in experimental methodology, observation, and collection of data.

Public availability and reusability of scientific data.

Public accessibility and transparency of scientific communication.

Using web-based tools to facilitate scientific collaboration.

What changes need to be made to this course to prepare people to do these things effectively?

Next Steps

Sun, 02 Aug 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

It's clear from Friday's end-of-course review that the course needs shaking up. Before that starts, though, there's a higher-level question to answer: should the course notes be converted to a wiki to encourage contributions from others? It was always my hope that other people would contribute material, but in four years, only five ever have; perhaps wikification would change that.

Right now, the notes are stored as HTML pages in a Subversion repository and compiled by a little Python script to resolve cross-references, insert code samples, and so on. The advantages of this approach are:

Post-Mortem

Sat, 01 Aug 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Today was the last day of the course, so we spent the morning talking about what had gone well and what had not. The high and low points were:

Good

The course was fun.

The TAs were fantastic.

The format (one hour of lecture plus two hours of lab, twice a day) worked well.

Enjoyed the parts where the instructors programmed live.

Liked the emphasis on working practices that complement coding.

Liked the spread of topics, and the variability of things that are useful in all the different fields.

Liked the pair programming.

Welcomed exposure to standard libraries that weren't necessarily covered in the course.

Liked the pre-arrival questions about what people knew, were doing, and wanted from the course.

The examples were good.

So were the donuts.

Bad

Three weeks is too long.

Some of the later topics were not as useful.

Would have preferred to use standard libraries for the image processing lecture and exercises instead of simplified libraries.

Too little coverage of too many subjects.

The formatting of the slides leaves much to be desired.

Too many lectures ran over time (which was particularly hard in afternoon sessions).

Divided attention in FriendFeed is a problem.

The less applied stuff (e.g., computational complexity) wasn't as useful or as interesting.

Students weren't given enough time to work on their own projects.

Didn't feel encouraged to make suggestions or provide feedback.

Not enough on shell programming.

Too much shell programming.

A/V between Toronto and Edmonton was crude by modern standards.

Changes

More on object-oriented programming.

More feedback on the students' solutions to the exercises—they didn't get the equivalent of grading.

Put the exercises up before the class, so that students know what the lecture's going to be leading them to.

It's been a good three weeks—I enjoyed getting to know the students, and look forward to seeing what they do with what they've learned.

A Good Afternoon

Fri, 31 Jul 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Yesterday afternoon, the students and ninety other guests were treated to six engaging talks about Science 2.0 from Titus Brown, Cameron Neylon, Victoria Stodden, David Rich, Michael Nielsen, and Jon Udell. We'll post slides and video here as soon as we get them; until then, you can catch up on what happened in the FriendFeed room or by reading Steve Easterbrook's real-time blog of the event.

Our thanks once again to everyone who made the day possible:

Day[-2]

Fri, 31 Jul 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Today (Thursday) was the second-to-last of the course. It's been a long haul, but we hope a rewarding one. In the morning, the students had an hour-long overview of results from empirical studies of real-world software engineering; in the afternoon, we looked at how traditional and agile development processes are responses to those facts. Tomorrow morning, we'll spend an hour talking about what's gone right and what's gone wrong, then head out for a farewell lunch.

Every Day Is a Big Day...

Wed, 29 Jul 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

...but today is bigger than most: after lectures yesterday on server-side web programming and building GUIs, students will spend this morning exploring ways to apply what they've learned in the course to their own research problems. This afternoon, we have a stellar line up of speakers from 1:00 to 6:00 pm at the MaRS Centre to talk about how the web is changing the way science is done. We'll post video of the talks as soon as we can, but if you'd like to follow along in real time, we are:

Day 11 and Day 12

Tue, 28 Jul 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Yesterday (Monday) morning we covered the basics of handling binary data, including bit twiddling and the use of Python's struct module to pack and unpack binary representations of objects. The afternoon was a lightning introduction to how the web works: a simple socket example (just to show students the plumbing) was followed by a description of HTTP's requests and responses, then a look at urllib. Today is going to be devoted to the absolute bare bones of server-side programming, with GUI programming using Tkinter as a follow-up. "Fast paced" doesn't even touch it, but I hope students will come away with an idea of what's possible, and where to look for more information.

Where This Course Came From

Mon, 27 Jul 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I have created an Amazon.com Listmania list [no longer online] for the books that most directly influenced this course. Some topics aren't directly represented—there's nothing devoted to handling XML or binary data, for example, or on big-Oh complexity or web programming—but I hope what's there will be of use.

Martin Fenner on SciBarCamp

Sun, 26 Jul 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Nature's Martin Fenner has blogged a summary of what he heard and saw at SciBarCamp'09 in Palo Alto a couple of weeks ago. Cameron Neylon was there too—in fact, I'd be willing to bet that a healthy number of attendees posted their experiences in some form or another. Maybe the next challenge for scientific publishing is something that will aggregate and summarize disparate reports of such events?

Day 10 Done - and With It, Week 2

Fri, 24 Jul 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

XML in the morning, regular expressions in the afternoon—it's been a long week, but a productive one (I hope). We're going to shuffle some of the material around so that we can do binary data processing, web client and server programming, and GUI programming next week. The highlight, of course, will be the guest speakers on Wednesday afternoon. I hope the students have a restful weekend; I'm looking forward to the last lap.

Day 9

Fri, 24 Jul 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I meant to write this post yesterday, but we're watching Torchwood: Children of Earth, and, well, you know how it goes. We spent two sessions looking at the basics of SQL and relational databases, which almost every scientist is going to encounter at some point in their career. The morning exercises were obviously too easy—students completed them in a little over and hour—but the afternoon seemed to go at the right pace. This morning we're going to look at XML, and this afternoon at text processing with regular expressions.

Day 7: Lots More Objects

Wed, 22 Jul 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We took a detour from the notes yesterday morning and built some classes to represent matrices. One reason was to show students how to break a problem down into pieces and solve them one by one; another was to introduce the notions of encapsulation, polymorphism, and inheritance. Aran Donohue (one of the TAs here in Toronto) thinks that inheritance is over-taught, and taught too early—he may be right, but it's also hard to avoid, since so many libraries and frameworks (and tutorials, but that's a circular argument) expect familiarity with it.

Day 8: Getting It Right

Wed, 22 Jul 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This morning's lecture was a high-level look at software testing, with a long-ish detour into exceptions; this afternoon was an introduction to Python's unittest framework, with some handwaving about testing that functions throw the right kinds of exceptions, replacing files and other slow objects with mock equivalents, and how you decide what tests to write first. It felt like it went better than yesterday's lectures on object-oriented programming, which I think left some students a little bewildered. We're spending tomorrow looking at databases, which means I have to brush up on my SQL tonight...

Day 6: Theory and Practice

Tue, 21 Jul 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We started Week 2 with a morning of theory: Francois Pitt, a senior lecturer in U of T's Computer Science department, gave students a crash course in algorithmic complexity. It's not something they'll use every day (or even every week), but if they ever have a conversation with a computer scientist about making something go faster, the odds are good it'll come up.

Elsevier's Future, Version 0.1

Tue, 21 Jul 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Elsevier (a big scientific publisher) have unveiled their guess of what the future of scientific publishing looks like. Tabs? Got 'em. Audio? Yup. RSS feeds for updates, corrections, and comments? The raw data, complete with provenance information so other researchers can try to reproduce the results? Still waiting...

Day 5

Sun, 19 Jul 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Day 5 of the course: Paul Lu gave a lecture on Make in the morning, and students had a chance to work on problems of their choice in the afternoon. Of all the tools we teach in this course, Make is the least satisfying (at least to me): the basic concept is simple, and I wouldn't run a project without some automating repetitive tasks, but Make's syntax and limitations are repellant.

Quantum to Cosmos: October 15-25 in Waterloo

Sun, 19 Jul 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Perimeter Institute is organizing a ten-day science festival in Waterloo this October called "Quantum to Cosmos". It promises to take you from the strange subatomic world of the quantum to the outer reaches of the cosmic frontier. And for those who can't make it, all events will be streamed online live and on demand 24 hours a day. Hope to see you there!

Day 4

Fri, 17 Jul 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Today was all about the Unix shell: Karen Reid covered a wide range of topics, often using live examples. Some of the students had never used a command-line prompt before, so all of the material was new; others already knew their way around pretty well, but Karen's explanations filled in some gaps and made sense out of what had, for some of them, seemed like arbitrary magic.

Day 3

Thu, 16 Jul 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Day 3 of the course—we did sets and dictionaries in the morning (glossing over the handwaving in the notes about computational complexity, since Francois Pitt is going to cover that properly later), and then Paul Gries showed students some basic image processing. Students found the morning exercise challenging, but seemed to be getting lots out of it. The visit to the pub this evening was still quite welcome, though :-). Tomorrow, Karen Reid is introducing them to the Unix shell; we'll let you know how it goes.

Day 2

Wed, 15 Jul 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

After three Python lectures yesterday, we slowed down a bit today and looked at a pair of tools every professional programmer relies on: symbolic debuggers and version control systems. Paul Lu covered the former topic using WingIDE; Ken Bauer covered the latter using the SmartSVN GUI for Subversion. Working in pairs seems to agree with the students, though as is so often the case, installing and configuring tools is often more work than actually using them. We'll look at Python's sets and dictionaries tomorrow morning, then Paul Gries will give a quick introduction to image processing in the afternoon.

Aaaand They're Off!

Mon, 13 Jul 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We had our first lecture this morning, and the students are working on their first exercises right now in pairs. You can follow progress at http://friendfeed.com/softwarecarpentryjuly2009.

See You Monday!

Fri, 10 Jul 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We're all set up and ready to roll! We decided in the end not to record audio or video for this set of lectures, but we'll be posting updates to the lecture notes as we go along. I look forward to meeting our students on Monday...

Registration for July 29 Talks is Now Open

Sat, 04 Jul 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As part of this summer's run of the course, we have organized an afternoon of public talks from a stellar lineup of speakers. The event is free, but registration is required, and you can now sign up at http://science20.eventbrite.com/. The talks will run 1-6 pm on Wednesday, July 29, and will be followed by a wine and cheese. For details, including speaker bios and abstracts for their talks, see the "Guest Speakers" page.

Quality Control and Traceability

Mon, 29 Jun 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Nature News recently reported work by Gurt Vriend at other to clean up records in the widely-used Protein Data Bank (PDB). This is great news, but what's missing is a way to track forward from entries to see what already-published papers have relied on data that's now known to be incorrect. One of the many goals of this course is to give scientists the understanding they need to tackle this problem.

Ready for Proofreading

Mon, 29 Jun 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We're ready for feedback — if you check out the new slides, you'll see a little feedback bubble at the bottom of each topic. Clicking that will give you a chance to send us email to tell us about formatting glitches, factual errors, or anything else you'd like fixed. Please let us know what you think...

The Environmental e-Science Revolution

Mon, 29 Jun 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Steve Easterbrook has summarized some papers from a recent Royal Society workshop on how the web is changing the way environmental science is done. We'll be asking students in the Software Carpentry course how much of this they're doing, and how much they would like to.

Updating the License

Thu, 25 Jun 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

In response to several requests, we have updated the license on the course material: the course content is now covered by the Creative Commons Attribution license, while the example code is (still) covered by an open source MIT license. In plain English, this means that you can re-use course content however you want, as long as you give us credit.

Topics and Schedule

Wed, 24 Jun 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A draft schedule for the July 13-31 offering of the course is now available. We'd welcome your feedback: does this order make sense, are there topics we've included that you don't care about, have we left out anything really important (and if so, what should we drop to make room for it), etc.

Note: the draft schedule has been retired. Please see the course outline instead.

Another New Version of the Slides

Tue, 23 Jun 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A new(er) version of the slides has been posted at http://software-carpentry.org that includes styling changes courtesy of Ryan Feeley. There are still many minor formatting glitches; we'll fix them in the coming week, and post a schedule showing which lectures are going to be given when.

We've also updated the Guest Speakers page with bios and abstracts for the people who'll be talking at the MaRS Centre in Toronto on July 29. Talks will run 1-6 pm, and will be followed by a wine and cheese. The event is free, but will require advance registration—we'll post details here as soon as we have them.

And Speaking of Sightings...

Mon, 15 Jun 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A new (but very rough) version of the slides for the course is now up at http://software-carpentry.org. This uses Eric Meyer's S5 package for formatting and pagination (along with a short Python script to insert cross-references). We were planning to use LaTeX, but after messing around with packages, dependencies, and the like, HTML started to look at lot simpler. We still have a lot of reformatting to do (particularly with tables and code inclusions); if you'd like to help, or if you are a CSS expert and can help make the slides look good with Internet Explorer (right now they're styled for Firefox), please let us know.

Neylon's Head in the Clouds

Mon, 15 Jun 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Cameron Neylon (a guest speaker at this summer's offering of the course) is putting together a paper for the new BMC journal Automated Experimentation titled "Head in the Clouds: Re-imagining the experimental laboratory record for the web-based networked world". It's an excellent description of what a web-native lab notebook could/should look like, and much more besides.

Sightings

Mon, 15 Jun 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Peter Saffrey sent email from Glasgow last week to say that he'd run a one-day course based on Software Carpentry. If you see other uses, please let us know—we'd be happy to link to them.

Software Carpentry in Edmonton July 13-31

Tue, 02 Jun 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Registration is now open for students wishing to do Software Carpentry in Edmonton July 13-31. The course will be co-taught with its counterpart in Toronto, though the Edmonton edition is only open to graduate students and post-docs at Alberta institutions. For more information, please see the full announcement.

Two Spots Left in Toronto

Tue, 02 Jun 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

There are only two spots left for the Toronto offering of the course July 13-31. As announced earlier, graduate students from anywhere in Canada are welcome to aplply, and we can provide up to $1500 in support for travel and accommodation costs for out-of-town students. If you are interested, please contact us.

Big Code vs. Science 2.0

Mon, 01 Jun 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The original goal of this course was to give scientists and engineers the skills they needed to build large pieces of software without heroic effort. It's increasingly clear, though, that another goal is equally important: to help them take part in what's sometimes called "Science 2.0": the sharing of information through the Internet. Cameron Neylon's slides on capturing process are, like Jon Udell's on computational thinking, a good summary of the skills required—skills which overlap with, but are distinct from, those needed by software developers.

SECSE Workshop

Mon, 01 Jun 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The latest in a series of workshops on "Software Engineering for Computational Science and Engineering" was held in Vancouver on May 23, just after ICSE'09. Steve Easterbrook has written a good summary of what was discussed, and Jeffrey Carver's longer summary will appear in a future issue of Computing in Science and Engineering.

Error Handling

Tue, 12 May 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

One topic that isn't currently in the curriculum that I'd really like to add is detecting, handling, reporting, and recovering from errors. This makes up is 10-30% of the code in real applications, but dealing with errors is almost omitted from textbook examples and tutorials for the sake of clarity (Tanenbaum's Minix book being a laudable exception). I have asked elsewhere for someone to write an entire book on the subject; if anyone wants to take a crack at an hour-long lecture, please get in touch.

Links for Summer Interns

Mon, 11 May 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Our summer interns started today; our first job is to define exactly what they'll be working on this summer, so it seems like a good time to round up a few links on interesting topics. My apologies for those hidden behind paywalls...

Steve's Project Ideas

Social network analysis for scientists

Electronic lab notebooks

Reproducible Research

If I said, "I just got a really interesting result in the lab, but I didn't record the steps I took or the settings on the machine," no reputable journal would publish my paper. If I said, "I just got a really interesting computational result," most reviewers and editors wouldn't even ask if I'd archived my code and the parameters I used, or whether that code would run on someone else's machine. Reproducible research (RR) is the idea of making computational science as trustworthy as experimental science by creating tools and working practices that will allow scientists to re-create past results.

How Scientists Use Computers: Survey Part 2

Sat, 09 May 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Thank you once again for taking part in our Fall 2008 survey of how scientists use computers in their research. We will present a paper describing our findings at ICSE'09 in Vancouver on May 23, and will make the results public as soon after that as possible. There will also be an article in American Scientist magazine discussing what you've told us some time this summer.

Topics and Schedule Posted

Wed, 06 May 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We've put up a list of the topics we intend to cover, and the order in which we intend to cover them. It's very provisional; we'll update it regularly, and your comments would be very welcome.

Note: the topics page has been retired. Please see the course outline instead.

Entrance Requirements

Mon, 04 May 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Our rough guide to what students should know before taking this course is now on the prerequisites page. If you don't feel confident you know this material, but still want to take the course in July, please let us know: we're organizing some tutorial sessions in May and June.

Note: the prerequisites page has been retired. Please see the target audience page instead.

What If Scientists Didn't Compete?

Fri, 01 May 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Interesting summary in the New York Times of some collaborative science done by Dr. Sean Cutler. One of the goals of this course is to give scientists the skills they need to do things like this routinely. This raises a few questions:

Should one of the "graduation exercises" for the course be to write some kind of search tool (or a wrapper around several search tools)? Finding potential collaborators is sometimes the biggest challenge.

Should more emphasis be placed on sharing and merging data sets? Or on looking for inconsistencies in them?

What else do you need to be able to do in order to collaborate more effectively with your colleagues?

Empirical Software Engineering and Scientific Computing

Tue, 28 Apr 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The slides for my talk at the National Research Council on empirical software engineering and how scientists actually use computers are now up on SlideShare. The colors in some of the embedded images were messed up during upload, but the result should still be readable.

Firming Up Course Goals

Mon, 27 Apr 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The best way to design a course is to describe the things students will be able to do when it's over; the best way to do that is to specify graduation exercises. Ours are listed in the Goals page on this blog. We would be grateful for feedback: are these the things you want to be able to do? What did we forget? What could we take out to make room for things you care about more? Please leave your comments on the page itself; we'll update it regularly based on what you say.

Madagascar Course in Delft June 12-13

Mon, 27 Apr 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Via Victoria Stodden, a link to a course on reproducible research with Madagascar being run in Delft on June 12-13. Focus will be seismic research, but the ideas are more generally applicable.

What Supervisors Need To Know

Thu, 23 Apr 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I received an interesting email yesterday from a grad student who took this course the last time it was offered at the University of Toronto. It said in part:

My supervisor could better advise students doing computational work if they had more background knowledge. They are routinely faced with questions like:

Is a project possible, given the background of the student and the difficulty of the tasks?

How long should a project take, and what can be considered good progress?

What training should a student have?

How to manage collaboration between students, data archives, etc?

How to make sense of and build upon work done by previous students?

On a more personal note—I would enjoy my supervisor having a clearer idea of what I do.

Software Carpentry in Alberta

Wed, 08 Apr 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I'm very pleased to announce that thanks to generous support from Cybera, Software Carpentry will be offered at the University of Alberta in Edmonton this summer. The course will be co-taught with the offering at the University of Toronto from July 13 to 31. For more information, or to enrol, please contact Professor Paul Lu.

We've Started a FAQ

Wed, 08 Apr 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We have started a FAQ [no longer online] for the July 2009 offerings of the course in Edmonton and Toronto. Please let us know if you have any questions that it doesn't answer yet.

Cameron Neylon on the Three Opens

Fri, 03 Apr 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Cameron Neylon has another good post up, this one on open data, open source, and open process. Like many advocates of open science, he feels he has to choose between using open source software on one hand, and getting more science done on the other. I sympathize, especially since my colleagues and I have to choose what to use and not use in the July 2009 run of the Software Carpentry course.

Software Carpentry in Toronto July 13-31 2009

Wed, 01 Apr 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Thanks to a grant from MITACS, the University of Toronto will offer the Software Carpentry course as a condensed three-week bootcamp this summer from July 13-31, 2009. This course is an accelerated introduction to software development aimed at graduate students in science and engineering; its goal is to give them the tools and skills they need to use computers more effectively in their research. 16 spaces are available to students registered in full-time graduate programs in Canada; the fee for the course is $500, but grants of up to $1500 for students from outside the Greater Toronto Area are available to help offset travel and accommodation costs. If you wish to attend the course, or would like more information on content, schedule, prerequisites, eligibility, or other details, please contact Greg Wilson by email at {{site.contact}}. Please also subscribe to the new Software Carpentry blog at http://software-carpentry.org/blog/ for updates.

User Stories

Mon, 30 Mar 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

One of the tricks I teach my undergraduates is to create fictional personas to describe the intended users of a system—or in this case, a course. Here are three of the "people" I've had in mind while developing Software Carpentry to date; my goal is to update these stories to better reflect how scientists work today.

Bhargan Basepair

Bhargan Basepair received a B.Sc. in biochemistry five years ago. He has been working since then for Genes'R'Us, a biotech firm with labs in four countries. He did a Java programming course as a freshman, and a bioinformatics course using Perl as a senior.

Inference for R

Wed, 25 Mar 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Inference for R lets users nest the R statistical language in Word and Excel. It's a neat idea, and another example of the kind of bottom-up innovation that I predict will eventually lead to fully-fledged extensible programming systems. (If Bespin [merged into Ace, as of Jan 2011] made it easier to do things like this, I might understand all the excitement...)

Open Notebook Science Badges

Wed, 25 Mar 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I blogged last summer about creating a badging scheme for open science. Turns out it's been done: ONS Claims has badges for four flavors of open science. Two sets are available in various sizes, all variations on the themes below:

All content Selected content

Immediate release

Delayed release

Here's hoping they're widely adopted.

Legal Frameworks for Reproducible Research

Tue, 17 Mar 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

My grad students and I met Victoria Stodden for the first time yesterday, and had a great time talking about reproducible research, science 2.0, and most particularly the kind of legal/copyright frameworks needed to move science forward. She has two papers up that anyone interested in the subject should read: a short one that appeared in Computing in Science and Engineering titled The Legal Framework for Reproducible Research in the Sciences: Licensing and Copyright, anda longer one due out soon called Enabling Reproducible Research: Open Licensing For Scientific Innovation. If you have thoughts on the subject, I'm sure she'd enjoy hearing from you.

Open Science and Autism's False Prophets

Mon, 02 Mar 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Paul Offit has a new book out called Autism's False Prophets, which looks at how the myth about vaccination causing autism arose and spread. As his condensed essay on the subject says, it's a far from simple story, but one that has echoes in areas such as climate change where science and public policy must dine at the same table. I still haven't decided whether open science will make a difference to this or not: putting data, calculations, and papers freely available online in real time will help scientists talk to one another, but I don't think it will help them communicate with the other 99.9% of our species. Thoughts?

Das Kapital, Computational Thinking, and Productivity

Mon, 23 Feb 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Somewhere in The Age of Uncertainty, Galbraith wrote that what made Das Kapital and the Bible great books was that they were so large, and so full of contradictions, that everyone could find support in them for anything they wanted. I have felt the same way about the phrase "computational thinking" ever since I attended a workshop at Microsoft Research in September 2007. In one of the breakout sessions, six of us tried to operationalize our understanding of the term by coming up with questions for a quiz that could be given to someone to determine if he or she was thinking computationally. It quickly became clear that we meant very different things when we used those two words. It was also clear (to me at least) that this ambiguity was socially very useful, since (to switch metaphors) it allowed people to attend the same church while disagreeing on the nature of salvation. It's not a polite fiction per se, but rather a—um, damn, I don't know the word—a thing that no one looks at closely because doing so would cause discomfort or friction.

Computer Supported Collaborative Science

Wed, 18 Feb 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I've used the term "CSCS" a few times now; time to start groping toward a definition. "Computer supported collaborative science" (CSCS) is a specialization of computer supported collaborative work, which is the study of "how collaborative activities and their coordination can be supported by means of computer systems". Insert the word "scientific", and you have CSCS. More specifically, CSCS includes science 2.0, open notebook science, reproducible research, workflow & provenance, and other things modern computing technology can do to help scientists find and share information.

Open Science Panel at Columbia

Wed, 18 Feb 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Via Jon Pipitone: there's a panel discussion tomorrow at Columbia titled "Open Science: Good For Research, Good For Researchers?" Jean-Claude Bradley, Barry Canton, and Bora Zivkovic are all going to be there, and yes, video will be distributed. I'm looking forward to it—it'll be a lot of thinking on computer supported collaborative science in one place.

Enough Players to Hand Out Medals

Mon, 16 Feb 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I blogged last August about the first and second Provenance Challenge, in which the creators of systems for tracking scientific data and workflows were given sample problems, then asked to have their tools answer a variety of questions. (Results from the first were reported in Concurrency and Computation, but ironically, those articles are not openly available; the third challenge will kick off soon.) Chasing down one of those references again, I came across the Open Notebook Science Challenge, which "...calls upon people with access to materials and equipment to measure the solubility of compounds (aldehydes, amines and carboxylic acids are a priority) in organic solvents and report their findings using Open Notebook Science". This isn't quite the same thing as automatically tracking data provenance; instead, it is "...the practice of making the entire primary record of a research project publicly available online as it is recorded". There are lots of interesting research questions for computer scientists here, ranging from privacy and security issues to notification, peripheral awareness, ontological engineering, and more — for example, see Cameron Neylon's latest post synthesizing discussion about using OpenID to identify scientific researchers and their contributions.

Carl Zimmer's Readers' Reading List

Wed, 11 Feb 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Carl Zimmer (prolific and talented writer on biology and evolution) has posted a crowd-sourced reading list of great science writing. Lots of good stuff...

MTEST

Wed, 11 Feb 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Steve Eddins has posted an xUnit-style testing harness for MATLAB called MTEST on the MATLAB Central File Exchange. It's a nice piece of work, and I hope numerical programmers will make heavy use of it.

Python Textbooks for Biotech

Wed, 11 Feb 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Jan Erik Moström recently posted a request for textbook recommendations for teaching programming (using Python) to biotechnology students. He has now posted the responses [no longer online] he received — might be interesting to some readers of this blog.

Sharing Data Isn't That Easy

Fri, 06 Feb 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Interesting post from Systeme D called "ShareAlike considered harmful" (for geo data, anyway). I should give this to my students and ask them to think it through...

Cameron Neylon Says Interesting Things

Wed, 04 Feb 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This time, he has blogged about best practices for making scientific data available. I think this kind of thing will have a much bigger impact on scientists' productivity than any amount of parallel supercomputing, and that computer scientists could have a lot of impact by helping "real" scientists figure out how to do it better.

Communicate First, Standardize Second

Fri, 30 Jan 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

That quote from Jean-Claude Bradley is on slide 34 of Cameron Neylon's presentation "Open Access, Open Data. Open Research?" Very worthwhile...

Web Native Lab Notebooks

Tue, 27 Jan 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Good post from Cameron Neylon about the lab notebooks of the future: "The traditional paper notebook is to the fully integrated web based lab record as a card index is to Google."

A New Kind of Big Science

Fri, 23 Jan 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Via Michael Nielsen, a guest column by Aaron Hirsh at the NY Times on "A New Kind of Big Science":

There is another way to extend our scientific reach, and I believe it can also restore some of what is lost in the process of centralization. It has been called Citizen Science, and it involves the enlistment of large numbers of relatively untrained individuals in the collection of scientific data. To return to our architectural metaphor, if Big Science builds the high-rise yet higher, Citizen Science extends outward the community of villages.

I *Want* To Be A Number

Sat, 10 Jan 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Via Science in the Open's summary of the workshop on open science at PSB'09, a link to a paper explaining why scientists will want to identify themselves with unique serial numbers.

Time to Freshen It Up

Wed, 31 Dec 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I tell the students in my software engineering classes that the absolute value of code coverage in testing isn't as important as the trend: if you're testing a smaller percentage of your software as time goes by, you're headed for trouble. The same is true of site stats: I don't care much about the absolute number of visitors, but if the curve is down and to the right, it's time to give the site a polish, or retire it.

Things I'd Like To Finish In the Next 489 Days

Fri, 26 Dec 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

One of the things I teach my students is that the real purpose of a schedule is to tell you when to start cutting corners and dropping features. The ticker on my web site tells me I have 489 days left in my contract with the university; I signed up hoping to study ways of teaching second-stage novices [1] how to be better programmers, but after four failed attempts to get NSERC funding [2], it's time to lower my sights. Here are the things I'd like to finish off before my stint at U of T is over:

A Healthy Dose of Scepticism

Wed, 24 Dec 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Titus Brown's latest post (which opens with, "The latest hot idea for making a protein-protein interaction database leaves me lukewarm") should be read by every computer scientist who's "just trying to help":

...while tools can be helpeful, the fundamental problem is much more, well, fundamental: science is hard. Connecting the dots is hard. Thinking clearly about the problem and separating the wheat from the chaff, so to speak, is hard. I worry that for the majority of biologists, new tools are going to be more distracting than helpful. We need to build simpler, easier-to-use tools, not more complicated tools; we need to keep our focus on the goal (solving biological problems) and not just on intermediate stages like improving databases and building better prediction tools.

Google Pulls the Plug on Scientific Data Sharing Project

Fri, 19 Dec 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Google has decided not to launch its scientific data sharing service — another victim of the recession, I suppose. Bummer :-(

The National Academy Would Like to Hear From You

Fri, 19 Dec 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Via Carl Zimmer: the US National Academy of Sciences would like you to fill in a two-minute survey about what science topics you care about most. Current results are:

Three Reasons to Distrust Microarray Results

Wed, 10 Dec 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Interesting post:

...the paper actually demonstrated that is it possible to distinguish microarray experiments conducted on one day from experiments conducted another day. That is, batch effects from the lab were much larger than differences between patients who did and did not respond to therapy... As is so often the case, data were mislabeled. In fact, 3/4 of the samples were mislabeled.

Igor, Connect the Electrodes!

Sun, 30 Nov 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Software Carpentry course site is still getting a fair bit of traffic, although readership is definitely tailing off:

I'm hoping to run an intensive three-week version of the course in June 2009 in Toronto (details to follow); hope I can find time between now and then to finish wikifying the course notes, get the MATLAB material online, and generally freshen the site up.

SECSE'09 Call for Papers

Fri, 21 Nov 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Second International Workshop on Software Engineering for Computational Science and Engineering

Saturday, May 23, 2009

Co-located with ICSE 2009 — Vancouver, Canada

http://www.cs.ua.edu/~SECSE09

Overview

This workshop is concerned with the development of:

Scientific software applications, where the focus is on directly solving scientific problems. These applications include, but are not limited to, large parallel models/simulations of the physical world (high performance computing systems).

Applications that support scientific endeavors. Such applications include, but are not limited to, systems for managing and/or manipulating large amounts of data.

A particular software application might fit into both categories (for example, a weather forecasting system might both run climatology models and produce visualisations of big data sets) or just one (for example, nuclear simulations fit into the first category and laboratory information management software into the second). For brevity, we refer to both categories under the umbrella title of "Computational Science and Engineering (CS&E)".

Getting the Science Right-Or At Least, Less Wrong

Thu, 20 Nov 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Via The Great Beyond:

The US National Academy of Sciences has created an initiative that will link TV and movie directors with scientists and engineers to incorporate more accurate science content into entertainment.

Press release here, web site here. That would be a cool job...

Science Lessons for MPs

Mon, 17 Nov 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Via Nature: politicians from the UK Conservative Party will be required to take science lessons. On the one hand, kind of sad that they didn't learn the basics in grade school. On the other hand, yay!, and when will Canadian parties require the same?

What Sciences Are There?

Sun, 16 Nov 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Over 1900 people have already responded to our survey of how scientists use computers, and it still has two weeks left to run. Our next task will be to analyze the data we've collected, which (among other things) means coding people's free-form descriptions of their specialties so that we can talk about physicists and chemists as opposed to "this one person who's doing N-brane quantum foam approximations to multiversal steady-state thingummies".

One Good Survey Deserves Another

Tue, 04 Nov 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

While we're running our survey of ~~how scientists use computers~~ [link no longer active], the folks at MATLAB are asking their users a few questions too. If you use any MathWorks products, and have a few minutes, they'd be grateful for your help.

1731 People

Sun, 02 Nov 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

1731 people have completed our survey of how scientists use computers since it went online three weeks ago. That's pretty cool, but I'd like to double the number (at least). If you consider yourself a working scientist, and haven't taken the survey yet, please take a moment and do so. If you aren't a scientist, but know some, please pass on the link:
~~http://softwareresearch.ca/seg/SCS/scientific-computing-survey.html~~
[link no longer active]

Finding and Re-using Open Scientific Resources

Mon, 27 Oct 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Via Cameron Neylon, a workshop in London in November on finding and re-using open scientific resources. Wish I could go...

Surveying Scientists' Use of Computers

Wed, 15 Oct 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Computers are as important to modern scientists as test tubes, but we know surprisingly little about how scientists develop and use software in their research. To find out, the University of Toronto, Simula Research Laboratory, and the National Research Council of Canada have launched an online survey in conjunction with American Scientist magazine. If you have 20 minutes to take part, please go to:

~~http://softwareresearch.ca/seg/SCS/scientific-computing-survey.html~~ [link no longer active]

Science in the 21st Century

Thu, 11 Sep 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I'm at the "Science in the 21st Century" conference at the Perimeter Institute today. There are 32 people in the room right now: 23 are male and 9 are female, but only one is non-Caucasian, which pretty much matches the numbers in the picture from the conference dinner last night. That's about the same M/F ratio I see in science grad courses at U of T, but definitely not the ethnic distribution—wonder why? It can't just be a "seniority effect" — this is a pretty young crowd. We see the same thing at DemoCamp: non-Caucasians are often a majority n sci/tech classes and companies in the Greater Toronto Area, but definitely a minority on Tuesday nights. Thoughts? Michael Nielsen says that SciBarCamp was 50/50...

Science 2.0: the Future of Online Tools for Scientists

Thu, 04 Sep 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A pub night and panel with Timo Hannay, Cameron Neylon, and Michael Nielsen, hosted by Nature Network Toronto

What does the future hold for the way we do science? Are online repositories such as GenBank and the physics preprint ArXiv, or social tools such as Nature Network, about to change science profoundly? To find out, join Nature Network Toronto for an interactive panel discussion over drinks at the pub.

Bil Lewis Works With Biologists...

Fri, 22 Aug 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

...and occasionally finds it frustrating.

Data Provenance Challenge

Wed, 13 Aug 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

John's summary of our discussion about what to teach scientists about reproducible research if they already believe it's a good thing, and want to start doing it reminded me that I never posted about the Provenance Challenge. It has been run twice so far; each time, authors of tools to track the provenance (or lineage) of scientific data have to implemented some workflows, then answers questions about where data came from, what was done to it, and so on. The results of the first challenge are described system-by-system in these papers (sorry, but it's behind a wall — if you google for combinations of the authors' names, you can find PDF preprints). This is a very cool research area, and I hope one of my incoming grad students will want to do something with it.

SciFoo, eGY, and Splitting

Mon, 11 Aug 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

OpenWetware has posted notes from SciFoo. I'm sorry I missed it; looking forward to "Science in the 21st Century" even more than before.

Those notes pointed me at the Electronic Geophysical Year declaration. I agree in principle, but think that something like the Open Source Initiative's "certified open" badge would be more useful than UN-ish statements like, "Effort should be made to identify and rescue critical Earth system data and ensure persistent access to them."

We're very close to re-launching the Software Carpentry site as a wiki, and I'm wondering if I should move the science-and-computing thread out of this blog into a separate one at that site. Thoughts?

They're Breeding Like Rabbits

Fri, 01 Aug 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Cameron Neylon complains about the proliferation of networking sites, aggregators, and what-not for scientists. I think he's right: none of them will succeed until there's massive consolidation. Maybe LinkedIn or someone like that could offer a cheap-but-not-free service customized to scientists' need on top of its existing infrastructure?

Next Lecture?

Mon, 28 Jul 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Software Carpentry course currently contains the following lectures:

Introduction

The Unix Shell (2 lectures)

Version Control

Automated Builds

Basic Scripting (bool/int/float, for/while/if)

Strings, Lists, and Files

Functions and Libraries

Programming Style

Quality Assurance (basic testing)

Sets, Dictionaries, and Complexity

Debugging

Object-Oriented Programming (2 lectures)

Unit Testing (unittest — should switch this to nose)

Regular Expressions

Binary Data

XML

Relational Databases

Spreadsheets

Numerical Programming (the basics of NumPy)

Integration (subprocess+pipes and wrapping C functions)

Web Client Programming (HTTP request/response, URL encoding)

Web Server Programming (basic CGI processing)

Security (the weakest lecture of the bunch)

The Development Process (a mish-mash of sturdy and agile)

Teamware (introduces portals like DrProject)

Conclusion (various "where to look next" suggestions)

Between now and Christmas, I want to tidy them up, duplicate the examples in MATLAB, and add some of the content I wrote for "CSC301: Introduction to Software Engineering". Since I won't have time to do everything, I'd like your help prioritizing. Which of the following topics do you think is most important to add? And what have I forgotten entirely?

Badge of Reproducibility

Wed, 23 Jul 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Coming back to the badge meme from earlier this week, John Cook's new Reproducible Research blog pointed me at this page on the EPFL site advertising a paper called "What, Why and How of Reproducible Research in Signal Processing". Notice the "Reproducible Research" badge? The "add your evaluation" link takes you to a formlet that lets you choose between:

I have tested this code and it works

I have tested this code and it does not work (on my computer)

I have tested this code and was able to reproduce the results from the paper

I have tested this code and was unable to reproduce the results from the paper

It's a good start...

Quick Quiz to Measure What Scientists Know

Wed, 23 Jul 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Suppose you have a room full of scientists—hundreds of 'em—and want to find out how they actually use computers in their work. There isn't time to interview them individually, or to record their desktops during a typical working week, so you've decided to ask them to self-assess their understanding of some key terms on a scale of:

No idea what it is.

Use it/have used it infrequently.

Use it regularly.

Couldn't get through the day without it.

My list is below; what have I forgotten, and (more importantly) how would you criticize this assessment method?

Reviving the Software Carpentry Mailing List

Tue, 22 Jul 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Luke Petrolekas and I are thiiiiis close to having the Software Carpentry notes converted to a wiki. Once they are, I'm going to be working with Tina Yee to update them, do the examples in MATLAB as well as Python, and fix some longstanding bugs. I'm also going to resurrect the project's two mailing lists (one for occasional announcements, the other for people interested in developing new material and/or teaching the course). If you'd like to be on either or both, please let us know.

Badge of Honor?

Sat, 19 Jul 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I met up with Shirley Wu, Michael Nielsen, and a few other ISMB attendees yesterday to talk about what's variously called Science 2.0 or Open Science. It was pretty rushed (and not helped by the bar we wound up in), but it got me thinking about creating an "open science" badge that scientists could apply to their work. Right now, people are using a variety of terms in inconsistent ways; it sometimes takes a very close reading to figure out exactly what the mean. I'd really like to see the PSB workshop (or some other meeting like it) put a peg in the ground and say, "If you do the following things, you can put this 'open science' badge on your lab's web site, and put, 'This research is certified open.' in your papers." The W3C's familiar badges and the Open Source Initiative's certification of software licenses have done a lot to clarify discussion, and have given people standards to aspire to. Nine years after the "Open Source/Open Science" workshop at Brookhaven National Laboratory, maybe it's time to borrow those ideas and put them into practice.

Kevin's Been Busy

Tue, 01 Jul 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Kevin Brown has been busy — he's been coordinating, installing, maintaining, fixing, and figuring out how to use a new $20 million supercomputer for cancer research. No word on how much money will be spent training people how to use it effectively, but hey, I'm easy to reach... :-)

What a Proposal Looks Like

Fri, 13 Jun 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I got word earlier this week that The MathWorks (makers of MATLAB) had approved my request for funding to spruce up the Software Carpentry notes, and find out how scientists are actually using computers. I faxed a signed copy of the paperwork down to them today—with luck, work will start in a couple of weeks, and I'm very excited to have a chance to work with the NRC's Janice Singer on the survey. And since students (graduate and undergraduate alike) occasionally ask about how academic funding works, the text of the proposal is below the cut. It isn't quite the same as programming, but in the end it might be more useful...
In 200 words or less, describe your proposed project.

Faking Results

Fri, 06 Jun 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Via BoingBoing, a story about scientists Photoshopping experimental results. Sometimes it's outright fakery; sometimes they're just "cleaning up" or "correcting". Either way, it raises an interesting question: how often are people doing this with computational results? Without scientists' code, or any other way to reproduce their work, we'll probably never know.

Three Weeks and Change

Tue, 03 Jun 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Everyone's making good progress:

Ming and Bing have posted their second demo — next step is to do some serious design of the final product.

After correcting an earlier post of mine, Xuan has blogged a ~~fuller description~~ of what she and Edward are building.

Zeev Lieber posted a brief summary of what Dmitri and William did last term. Dmitri's still wrestling with GWT exceptions.

Jeff Balogh ~~shortens his build~~.

Matthew Basset sent his first data.

Joseph Yeung is ~~converting an EXE to an MMC~~.

Victoria Mui completes a subtask.

Daniel Servos likes Moodle plugins.

Qiyu Zhu can edit roles.

Nick Jamil provides more details.

Qi Yang bursts into song.

Oh, and it looks like I'm going to get a grant from The MathWorks to upgrade the Software Carpentry course and find out what scientists are actually doing with computers — more on that once I know whether Sadie and I have bought a house or not.

Programming and Scientific Education on Slashdot

Fri, 30 May 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Via Adam Goucher, a Slashdot thread about whether programming should be part of science education. 300+ comments and counting, almost all relating personal experiences.

Reminded of the Difference Once Again

Tue, 27 May 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Via Irving Reid, a great quote from Henrik Kniberg about how to tell when you're done:

So when a team member says that a story is Done and moves the story card into the Done column, a customer could run into the room at that moment and say "Great Lets go live now", and nobody in the team will say "no, but wait".

Ironically, Kniberg's post is called "Version Control for Multiple Agile Teams". Why ironic? Because I was talking yesterday to a professor in my department — young guy, smart, already has a better research and publication record than I'll ever have. I brought up version control (as in, "Why do less than 10% of faculty and grad students use it?"); his reply was, "Well, we're just going to back everything up once a day."

Interviewed by Jon Udell

Sun, 25 May 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Jon Udell has posted an interview he did with me last week at IT Conversations. The title is "High Performance Computing Considered Harmful", and slides from a talk of the same name that I gave in Austin are available.

Later: here's Jon's summary of the interview. Curmudgeonly? Moi? :-)

Why Don't We Do This?

Wed, 21 May 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Source Code for Biology and Medicine is a peer-reviewed journal from BioMed Central devoted to, well, source code for biology and medicine. It's been around for at least a couple of years; based on a quick scan of four of their most popular papers, they seem to cover everything from getting and installing software to its numerical properties, user interface, and typical applications. This is very cool—but why isn't there something like this for computer science? All I can think of are Software: Practice & Experience and The Journal of Systems and Software, both of which are one meta removed from SCBM.

But I Was Gone Less than 48 Hours!

Fri, 16 May 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I left Toronto for Austin mid-day Wednesday, and got back at midnight last night. Lots happened in the interim, so here's a linkandthoughtdump (which I bet actually is one word in German):

Gave a talk about Beautiful Code to the Austin Python Users' Group Wednesday at Enthought's swanky offices. (They're the kind folks who provide web hosting for the Software Carpentry course.) About 27 people in attendance, and good discussion afterward; was grateful to Travis Vaught and Sergey Fomel for rides from the airport and to the hotel respectively.

Gave another talk titled "HPC Considered Harmful" at the Texas Advanced Computing Center's Second Annual Scientific Software Days. I was a bit nervous about telling people at a supercomputing center that focusing on massive parallelism and peak performance is wrongheaded, but there were a lot of nodding heads.

I made lots of notes from two other talks that I want to follow up on at some point:

Robert van de Geijn's FLAME system lets you draw matrix operations, then automatically generates the corresponding high-performance code. It's a great example of a real high-level programming tool for scientists (and yet another special case of what a real extensible programming system would support).

Eric Jones (also from Enthought) talked about a tool they're building that watches changes to variables in Python programs, and automatically generates interactive plots of their values. It sounds simpler and less impressive than it actually is; I've asked him to put together a screencast, and I think you'll be wowed—I was. (Later: Steve Eddings from The Mathworks sent me a link about data linking in MATLAB, complete with a video tutorial.)

At roughly the same time, half a world away, Diomidis Spinellis presented a study comparing the code quality of Linux, Windows, OpenSolaris, and FreeBSD. Very cool work; wish I'd been at ICSE'08 to ask questions.

Meanwhile, Dmitri Vassiliev, who is continuing his work on SlashID this summer, has discovered that generated code is next-to-impossible to debug. Not to be a one-note symphony or anything, but I said in that same article about extensible programming systems that the real challenge is not extending notation, but creating extensible debugging tools so that those notations and high-level representations can be fixed when they break. Robert van de Geijn doesn't think FLAME needs a debugger; respectfully, I disagree.

Science in the Open has a plea to scientists to make their raw data available, motivated by yet another irreproducible result.

Kosta Zabashta has posted early thoughts about integrating IRC into DrProject. (Gray on black? Kosta...your design skills rival mine...) I need to tell him that DrProject's RPC module doesn't handle tickets because Jeff Balogh is going to replace the entire ticketing system with an extensible one this summer, using his Dojo Form Editor as a front end...

Elisabeth Hendrickson has thoughts on automating tests for legacy web applications. Students, take note.

Thanks to Nick Jamil and others, we have instructions for installing DrProject on Windows. Yay!

Everything old is new again, including Ada and the Bletchley Park Colossus.

And then there's this:

SE-CSE Workshop

Thu, 15 May 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

One of the downsides of being in Texas is that I couldn't attend the First International Workshop on Software Engineering for Computational Science and Engineering, which was held at ICSE'08 in Leipzig this week. Papers are here (I'll be reading them on the flight home); they look interesting, but the biggest thing for me is the change in the workshop's name—it used to be "Software Engineering for High-Performance Computing Applications" (~~2004~~, ~~2005~~, 2006, 2007), and I'm hoping the change of name reflects a genuine broadening of focus.

Those Who Will Not Learn From History...

Mon, 05 May 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

My article "Those Who Will Not Learn From History..." is in this month's Computing in Science and Engineering (and mirrored here).

SPOC

Mon, 14 Apr 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Regarding the idea of reproducible research, I stumbled over I-SPOC while looking through Google Summer of Code stuff. From their pitch:

The overall goal of this project is to build computational and social infrastructure to support the use of a new form of scientific communication called a SPOC (Scientific Paper with Open Communication). A SPOC combines a standard academic paper with open source computational models written in any publicly accessible computer language. SPOCs will (i) link computational results with the models that produce them, allowing independent verification and validation (ii) create incentives for cleaner, more transparent code and for the sharing of code (iii) enable others to extend and improve existing computational models and to verify model robustness (iv) bring computational models to life allowing faculty, students, and other scholars to see dynamic phenomena emerge and (v) have an enormous effect on the teaching of science.

Three Studies (Maybe Four)

Thu, 10 Apr 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We're in the thick of picking students and projects for Google Summer of Code, which has inspired some less-random-than-usual thoughts. Here are two studies I'd like to do (or see done):

What has happened to previous students? How many are still involved in open source? How many have gone on to {start a company, grad school, prison}? What do they think they learned from the program? How much of the software they wrote is still in use? Etc.

Every one of the 175 organizations blessed by Google this year is using the same web application for collecting and voting on projects. From what I can tell, they're all using it in different ways: +4 means something very different to the Python Software Foundation than it does to Eclipse or SWIG. They're also using a bewildering variety of other channels for communication: wikis, IRC, Skype chat sessions, mailing lists (the most popular), and so on. Why? Is this another reflection of Jorge Aranda's finding that every small development group evolves a different process, but all those processes "work" in some sense, or is it—actually, I don't have any competing hypotheses right now, but I'm sure there are some.

And while we're on the subject of studies, I just read Hochstein et al's paper "Experiments to Understand HPC Time to Development" (CT Watch Quarterly, 2(4A), November 2006). They watched a bunch of grad students at different universities develop some simple parallel applications using a variety of tools, and measured productivity as (relative speedup)/(relative effort), where relative speedup is (reference execution time)/(parallel execution time), and relative effort is (parallel effort)/(reference effort). The speedup measure is unproblematic, but as far as I can tell, they don't explain where their "reference effort" measure comes from. I suspect it's the effort required to build a serial solution to the problem, and that "parallel effort" is then the additional time required to parallelize; I've mailed the authors to ask, but haven't heard back yet.

Summer Plans for Software Carpentry

Wed, 02 Apr 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Software Carpentry site is still getting a lot of traffic, despite my neglect:

This summer, I'd like to:

Convert the site from static HTML pages to a wiki to make it easier for people to contribute content and fix bugs.

Translate the examples into MATLAB to make them accessible to a larger audience. (Yes, Python is still my favorite language, and yes, the Python versions will remain—I just want it to be possible for the average mechanical engineer to follow the discussion of testing without first having to learn a new programming language).

Add some of the material that I developed for CSC301: Introduction to Software Engineering, and some of what Titus Brown wrote for Intermediate and Advanced Software Carpentry.

The odds of all three happening are close to zero: my grad students are going to be in the middle of real research, we're hoping to have half a dozen or ten undergraduate interns as well, we have a "CS-1 in Python" book to finish for Pragmatic, and oh yeah, I'm getting married twice. If you want to help out, now would be a good time to raise your hand... :-)

The Retractions Just Keep Coming In

Wed, 02 Apr 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Via Titus Brown, yet another published result retracted because of a bug in code.

Meet the New Flaw

Mon, 31 Mar 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I was pretty excited when I heard that Microsoft was getting into scientific computing. As the world's biggest desktop software company, I figured they might understand that scientific computing and high-performance computing are not automatically the same thing, and that reliability and reproducibility are more important than peak performance. Turns out I was wrong: the workshop I attended last September was dominated by discussion of topics like GPU programming and computational grids that are still bleeding-edge computer science, rather than the nuts and bolts that would actually help most scientists be productive day-to-day, Microsoft's new HPC++ Computational Finance lab's site [since closed] has a lot on speed but nothing on correctness, et cetera.

Nice Quote

Wed, 26 Mar 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

An article about computational science in a scientific publication is not the scholarship itself, it is merely advertising of the scholarship. The actual scholarship is the complete software development environment and the complete set of instructions which generated the figures.

—David L. Donoho, WaveLab and Reproducible Research, 1995, p. 5.

(via Andrew Lumsdaine)

Survey: Silent Errors in Scientific Code

Fri, 07 Mar 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Posted on behalf of Daniel Hook and Diane Kelly:

We are members of a software research group from Queen's University who are investigating ideas and tools to assist with the development of scientific software. We are starting a project focused on finding silent or hidden errors in scientific code. (Silent errors are errors that don't result in a crash, an error message or an other obvious indicator of a problem.) To create a catalogue of common silent errors, we would like to hear debugging "war stories" from computational scientists. Using these stories we hope to provide improved code testing techniques specifically for scientists.

LearnHub Launches with Software Carpentry Front and Center

Thu, 06 Mar 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Toronto-area startup LearnHub.com has launched their new e-learning site with part of the Software Carpentry course as their featured offering. I'm looking forward to meeting a whole new batch of students... ;-)

Scientific Groupware Revisited

Tue, 26 Feb 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Nature would like "to hear from physicists what kind of tools would help in managing the ever-growing tide of information from, and the exciting possibilities of, the internet." (I presume scientists in lesser disciplines will be asked to chip in at a later date ;-). It will be interesting to see how much (if anything) has changed since Jon Udell wrote "Internet Groupware for Scientific Collaboration" eight years ago.

O'Reilly Creating a Web Version of Mathematica

Wed, 20 Feb 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

According to this press release, O'Reilly is partnering with Wolfram Research to create a Web 2.0 version of Mathematica that will "be browser accessible and, utilizing AJAX technologies, will emulate the desktop version of the software with remarkable fidelity". I'll be very interested to see how they handle equations (MathML isn't well supported) and grahpics (IE7 doesn't do SVG, so what, Flash? Silverlight?).

Grumpy Minds Think Alike

Thu, 14 Feb 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Ned Gulley pointed me at this talk by fellow-Mathworkser Steve Eddins that hits a lot of the same core ideas as Software Carpentry. Time to convert SC to a wiki, I think, and start pushing it again...

SciBarCamp in Toronto March 15-16

Sat, 02 Feb 2008 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

SciBarCamp is weekend unconference for scientists, artists, and technologists. The goal is to create connections between science, entrepreneurs and local businesses, and arts and culture. I think this is a great idea, and wish I could attend (I already have a plane ticket for SIGCSE in Portland); it's free, but there is only space for around 100 people, so please register by sending an email to Jen Dodd.

Python Supercomputing Statistics

Sun, 09 Dec 2007 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I have a major grant application and (hopefully final) revisions to my next children's book due on Friday, so of course I'm reading white papers about Python-friendly supercomputing from Interactive Supercomputing, a Boston-area firm that's about three years old. IS offers several kinds of parallelism for MATLAB, Python, R, SAS, and other high-level languages; I don't know if their tools are any easier to use than anyone else's, but they have an impressive team (including Russ Barbour, ex-Apollo, and Steve Reinhardt, ex-Cray).

The Burning Man of HPC

Fri, 26 Oct 2007 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

It's a small world: Andy Oram, my co-editor on Beautiful Code, just interviewed Brent Gorda, co-author of the first version of the Software Carpentry course, about the cluster challenge that's running at Supercomputing'07. Don't be fooled by the purple-on-black titles: this is a very cool idea. Students teams get fixed time (and fixed electrical power) to put together a cluster and run some benchmarks. Doing this earned you a PhD twenty years ago; ten years ago, it would have been a line item in a department's budget. It's now a contest for students, and I look forward to finding a cluster in the bottom of a cereal box before I die... ;-)

Doomed to Repeat It

Tue, 02 Oct 2007 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Those who cannot learn from history are doomed to repeat it. (Santayana)

I spent a day and a half last week at a workshop on computational science education. There were lots of smart people in attendance, all very passionate about the need to teach scientists how best to use the awesome power of a fully functional mothership—sorry, of modern hardware and software—but as the discussion went on, I grew more and more pessimistic. I'm old enough to remember the first wave of excitement about computational science in the 1980s; dozens of university-level programs were set up (mostly by physicists), and everyone involved was confident that the revolution would be unstoppable.

Another Sighting of Software Carpentry

Tue, 25 Sep 2007 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Tiziano Zito is teaching Software Carpentry in Berlin. I admit I'm biased, but I think that's pretty cool ;-) Traffic on the site is holding steady as well — wish I had time to go back and update the notes...

Openness and (the promise of) XML

Wed, 05 Sep 2007 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Over at O'Reilly, people are gloating about ISO's decision not to ratify Microsoft's OOXML document "standard". At the same time, Jon Udell is saying much more interesting things about what smart document formats could do for the sciences. To quote him quoting Clifford Lynch, "Scientific literature that is computed upon, not merely read by humans."

Random Survey about HPC

Fri, 31 Aug 2007 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This was just forwarded by a friend:

The High Performance Computing (HPC) community is initiating a study to develop education and training pathways in order to enable scientists and engineers to advance the pace of discovery by taking advantage of high performance computing (HPC) and grids.

In order to inform our study, we invite you to complete a survey about your experiences. We would like you to evaluate a list of categories of expertise according to how relevant they are for your work. We also would like to collect additional information about your training experiences.

How I'm Doing

Tue, 07 Aug 2007 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I wasn't happy with the two courses I taught this past winter — too many distractions, too little preparation. The feedback on the Software Carpentry course was therefore a pleasant surprise: I've heard second-hand that several of the Computer Science grad students were disappointed by its slow pace, but overall I did better than I expected. On a 1-5 scale:

Background required to successfully complete the course: 2.0

How easy to obtain details/background to supplement lecture material: 2.0

Did term work increase understanding: 3.7

Material was presented too slow/fast: 2.0

Material was too broad/specialized: 2.9

Workload was too light/heavy: 2.9

How well organized was the lecturer: 4.0 (no idea whose class they were in...)

How satisfied: 4.5

Overall rating: 4.2

The most common positive comments were that the course was practical and pragmatic, and that the collaborative projects were worthwhile. Negatives include the assignment being distributed and marked very late, not enough examples of what good programs actually look like, the course being slow for CS students, a lack of depth in some areas (particularly security), and my jokes being corny.

How Not to Collaborate

Tue, 31 Jul 2007 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I posted a note a while back about an upcoming workshop at Microsoft Research on computational education for scientists. If you read the call for papers, you'll discover that there aren't any instructions on how to submit material; nor is there any contact information, other than the generic "contact us" link the bottom corner that gives you nothing more than Microsoft's generic 1-800 number ("press 1 for Windows Vista support..."). After bouncing around for five minutes, I got a human being who told me that she couldn't give out any phone numbers, but I was welcome to fax my question to them...

Computational Education for Scientists

Wed, 18 Jul 2007 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Via Zachary Dodds: Microsoft is hosting a workshop on computational education for scientists in September. I'm going to try to be there—anyone else?

Win a Trip to Reno!

Thu, 05 Jul 2007 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Francis Sullivan is running a contest: describe what you'd do if someone gave you a desktop petaflop machine, and you could ~~win a trip to Supercomputing '07 in Reno~~. 1000 words, due by August 10.

Another Sighting of Software Carpentry

Wed, 04 Jul 2007 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Rick Wagner, a computational astrophysicist at UCSD, is teaching a course based on Software Carpentry. Yay!

Two Studies of ASCI (and no, that's not a typo)

Wed, 27 Jun 2007 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

After spending ten years helping scientists write programs for massively-parallel computers, I realized that what scientists really needed was to learn how to program, full stop. It took me another eight years to (a) get up to speed with the theory and practice of modern software engineering, (b) realize how big the gap between the two was, and (c) accept that the only people who are entitled to have an opinion about how we ought to be building software are the ones who are studying how well their favorite tools and practices actually work in the field. I no longer care what you're pushing—formal methods and cleanroom development or agile adhocracy and pair programming—unless you've gone the extra mile and collected data to show what effect it's actually having.

Software Carpentry at LLNL

Tue, 26 Jun 2007 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Titus Brown has posted about the first run of his Advanced Software Carpentry course at Lawrence Livermore National Laboratory. It was a bit bumpy, but seems to have gone well overall, and he has put a ton of useful material on the web.

Inspirational Videos

Wed, 20 Jun 2007 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This, on scientific computing and visualization, is pretty fricken cool.

Software Carpentry Screencasts by Chris Lasher

Wed, 20 Jun 2007 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Chris Lasher, a grad student at the University of Virginia Virginia Tech (sorry, sorry), has posted some screencasts about version control based on the Software Carpentry notes at ShowMeDo.com; a second series about using the shell should be up shortly. It's just plain cool to see the material picked up and carried forward in ways I wouldn't even have thought of a couple of years ago.

Nature Precedings

Mon, 18 Jun 2007 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Tim O'Reilly posted a note from Timo Hannay, of Nature magazine, about their new "Precedings", which:

giv[es] researchers a place to post documents such as preprints and presentations in a way that makes them globally visible and citable. Submissions are filtered by a team of curators to weed out obviously inappropriate material, but there's no peer-review so accepted contributions appear online very quickly — usually within a couple of hours.

Praising the Good

Mon, 11 Jun 2007 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I frequently gripe about how backward scientific software development is, so it's only fair that I give praise where praise is due. Steve Easterbrook pointed me at the UK Met Office's Flexible Configuration Management system, which combines code management using Subversion and Trac with a GNU Make-based build and an extraction system that pulls data out of the repository and builds it for particular platforms. This image shows how the three parts fit together; it ain't rocket science, but it's very professional, and a good model for others to follow.

Computational Scientists Still Don't Get It

Sat, 05 May 2007 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A workshop called "Software Issues in Computational Science and Engineering" is running in Uppsala, Sweden, this August. Here's their blurb:

Software for numerical computations faces multiple challenges. The software should be easy to use. Ideally, adaptation to new applications should be flexible, and extension to incorporate new numerical techniques straightforward. At the same time the software should execute extremely efficiently on various high-performance computing platforms. Accuracy and robustness are other key features. The overall challenge is to find ways to construct numerical software so that all these different goals are met simultaneously.

Titus Brown Teaching Software Carpentry

Mon, 02 Apr 2007 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Titus Brown will be teaching "Intermediate and Advanced Software Carpentry with Python" at Lawrence Livermore National Laboratory this spring, and is open to teaching it elsewhere after that. Meanwhile, Chris Lasher has put together some more debugging material for the course. As mentioned previously, they're planning to run a writing spree at SciPy'07; hope to see lots of people there.

Sign Error: Five Papers Retracted

Mon, 19 Mar 2007 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Via Genome Biology (free registration for trial access required), news that scientists from the Scripps Institute have to retract five papers published in various prestigious journals because of a sign error in a computer program. As Gregory Petsko says in the article:

Their mistake has consequences beyond the damage to the unfortunate young investigator and his team. For five years, other labs have been interpreting their biophysical and biochemical data in terms of the wrong structures. A number of scientists have been unable to publish their results because they seemed to contradict the published X-ray structures. I personally know of at least one investigator whose grant application was turned down for funding because his biochemical data did not agree with the structures. One could argue that an entire sub-field has been held back for years...

SciPy'07 Dates Announced

Sun, 11 Mar 2007 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

According to www.scipy.org, The SciPy 2007 Conference will be on August 16-17 this year; tutorials and sprints will run on the 14th, 15th, and 18th. I won't be able to attend (new baby), but I'd like to organize a half-day or one-day session to update and extend the Software Carpentry notes. Lots of modules need writing, both on Python-specific stuff and on general software engineering skills for scientists and engineers. I'd particularly like to see:

Reproducibility of Computational Results

Sat, 10 Mar 2007 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Via Titus Brown, a link to the Insight Journal, an open access online publication covering medical image processing. They have a very interesting process requirement: your source code must compile & be verifiable by an automatic system. I've been expecting something like this for a long time; glad to see it happening.

Later: and, via Gary Bader, Source Code for Biology and Medicine. Anyone know of a journal or journals like this for physics, chemistry, geology, and other non-life-science areas? Or (wistfully) computer science?

Software Carpentry Screencasts

Wed, 07 Feb 2007 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Chris Lasher, who audited the Software Carpentry course long-distance last time around, has put together some screencasts to go with the first few lectures:

The shell (screencast)

Subversion (screencast)

I think this is very cool — if you have feedback, or want to praise him (hint, hint), please drop him a line. I would also be interested in hearing how useful you find these — if there were twenty or thirty covering the whole course, would you actually watch them? And would they help you understand the course material?

Software Carpentry Usage in December

Thu, 18 Jan 2007 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Visits to the Software Carpentry site were down quite a bit in December; I had expected a drop because of the holiday, but not this large. On the bright side, 47 students have signed up for the course at U of T this term; about 1/3 are from Computer Science, while the rest span Civil Engineering, Ecology, Physics, Ecology, and the life sciences.

YouTube for Data

Tue, 05 Dec 2006 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

TechCrunch is running an interesting report on Swivel, which bills itself as "YouTube for data". Anyone and everyone can upload data sets to share with the rest of the world, link together, and what have you. It's going to be a great resource for teaching...

...but I'm less enthusiastic about it for science, unless its creators have some verification and validation magic up their sleeves that they're not yet talking about. Still, it's another sign of what Richard Dawkins called "the evolution of evolvability": every new kind of thing (segmented bodies, eyes, web services) gives evolution a new affordance on which to act. Taggable public upload is turning out to be a very mutable affordance...

Software Carpentry article in CiSE

Tue, 28 Nov 2006 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The November 2006 issue of Computing in Science and Engineering includes an article I wrote titled "Software Carpentry: Getting Scientists to Write Better Code by making Them More Productive". It's available to subscribers only on the magazine's web site, but they have kindly given me permission to post it here. I think it's out too late to bring November's stats up above October's, but I could be surprised.

Software Carpentry continues to grow

Thu, 02 Nov 2006 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Traffic on the Software Carpentry site was up again in October, after a dip in September:

The sys admin at Enthought also cleaned the comment spam out of the Trac they gave me to manage the project. I have two dozen more minor tickets to file from email I've received in the last few weeks (much of it from Germany). Still waiting for people to start contributing content — any volunteers?

Computational Result Retracted

Tue, 31 Oct 2006 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

From the latest Nature (Vol 443, No 7114, p 1013):

When a new, independent code is used for the calculations on which the conclusions of this Letter were based, the results reported for the evolution of obliquity cannot be reproduced. This code was written in the inertial frame and is more reliable than the one used in the Letter. In most runs, the obliquities can change by only a few degrees and attain large values in only a very few cases. In addition, the obliquity variation shown in the Supplementary Information, although correct, originates from changes to orbital inclination of the planet, and close encounters are not effective in causing large obliquities.

German Version of 'Bottleneck'

Thu, 26 Oct 2006 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A German version of my article "Where's the Real Bottleneck in Scientific Computing?" has just appeared in Spektrum magazine. Pay-per-view, unfortunately, but the Software Carpentry site has had a flurry of hits from .de domains.

Dec 1: I just received my copy in the mail—I sound so much...sterner...in German ;-)

Gregory V. Wilson ist Professor für Computerwissenschaft an der University of Toronto. Sein Kurs ist erhältlich unter www.swc.scipy.org/.

Oh My God It's Django!

Thu, 17 Aug 2006 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Guido just pronounced: Django is the web framework

Won't be part of the core, but will be as "standard" as PIL or NumPy

This was not what I expected the outcome of my talk would be, but hey, I'll take it ;-)

He hopes that Django and TurboGears will converge

Eric Jones: Enthought Tool Suite

Enthought does open source software and consulting for scientific computing with Python

One of the sponsors of this conference

Providing hosting for Software Carpentry

Walked through traits and other offerings

Chris Mueller: Synthetic Programming with Python

SciPy'06: First Morning

Thu, 17 Aug 2006 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Guido van Rossum's Keynote

Python 2.5 coming Real Soon (Sept 12)

Python 3000 is a brand-new revision of the language

Name chosen as a dig at Windows 2000, and so that it couldn't possibly be late

Fix design bugs dating from 1990-91 + get rid of deprecated features

First time Guido has allowed himself to be backward incompatible

Need process, but don't want to become C++ or the next Perl 6

Alpha early 2007, final a year later (early 2008)

Cares a lot about bringing users with him

Will go as far as 2.9 (run out of digits)

Changes:

New keywords allowed

dict.keys(), range(), zip() won't return lists

All strings Unicode; mutable 'bytes' data type

Binary file I/O redesign

Drop as an alias for !=

Etc.

See PEP 3099 for things that won't happen (e.g., programmable syntax)

Can't do perfect mechanical translation (dynamic languages)

Use pychecker-like tool to handle 80% of cases

Create instrumented Python 2.x that warns about "doomed" constructs

See PEP 3100 for the laundry list

Small points

Kill classic classes

Exceptions must derive from BaseException

int/int will return a float

Remove last differences between int and long

Absolute import by default

Kill sys.exc_type and friends

Kill dict.has_key, file.xreadlines()

Kill apply(), input(), buffer(), coerce()

Kill ancient library modules; more stdlib cleanup

exec becomes a funciton again

Kill `x` in favor of repr(x)

Change except clause syntax to exception E1, E2, E3 as err

Means "as" becomes a keyword

[f(x) for x in S] becoms sugar for list(f(x) for x in S)

General trend in Python away from lists toward more abstract structures

Kill raise E, arg in favor of raise E(arg)

zip becomes izip

lambda lives!

String types reform (bytes and str instead of str and unicode)

All data s either ibnary or text (conversions happen at I/O time)

Different APIs for binary and text streams

New standard I/O stack

C stdio has too many problems

Borrow from Java streams API (bleah)

Print becomes a function (boo)

See mailing list thread for justification

But I think that putting the output file at the end in print(x, y, file=z) is going to trip people up

Dict views instead of lists

dict.keys() and dict.items() return a set view

dict.views() will return a bag (multiset) view

Can delete from (but not add to) a view

Modifies the dict accordingly

Drop default implementations of comparison operators

<, <=, etc., currently compare by address — will raise TypeError

== and != should remain (useful)

Generic and overloaded functions (see his blog — running out of time)

Python sprints coming up (Aug 21-24)

Q&A

Py3K team is smaller than Perl6 — GvR optimistic that people will get the work done

Taking advantage of multicore?

GvR not a big fan of threads

Prefers loose coupling (one process per core)

Last attempt to get rid of the GIL slowed Python down by 2X

But neither Jython nor IronPython have a GIL

Will C-Python API change much?

Yup — just like the language

PyPy/type inference?

Python 4.0 or a sibling language

Travis Oliphant on the State of NumPy

SciPy and Software Carpentry

Wed, 16 Aug 2006 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I'm flying down to CalTech this evening to give a talk on Software Carpentry at SciPy'06. There's been a fair bit of traffic on the web site in the last couple of weeks, and I'm looking forward to hearing how else we can help scientists and engineers become more productive programmers.

Total visits: 4354 in 15 days, from 731 distinct domains (excluding obvious robots).

HPCWire Interview on Software Carpentry

Fri, 04 Aug 2006 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

HPCWire recently interviewed me about the Software Carpentry course. Coincidentally, I've started getting patches from new contributors (especially Ralph Corderoy, who did a very thorough review of the shell programming lectures). More would be welcome...

Design Patterns in Scientific Software

Sun, 30 Jul 2006 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This paper, via the OpenScience Project, is a couple of years old, but a good step in an interesting direction.

The Parallel Tools Platform

Thu, 20 Jul 2006 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Parallel Tools Platform is an open source extension to Eclipse for writing, running, and debugging large parallel programs. This tutorial gives an overview of what it can do (with lots of pretty pictures); this article from CiSE goes into more detail. The lack of decent development tools (particularly debuggers) was one of the reasons I left parallel computing a decade ago; PTP almost looks nice enough to tempt me back into the pool...

Software Carpentry 3.0

Fri, 14 Jul 2006 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I am pleased to announce the release of Software Carpentry 3.0 (Final). The web site is up, the mailing lists have been populated, all the tickets have been moved into the project's Trac, and I've submitted a proposal for a wrap-up talk to SciPy'06. It's taken 18 months to complete instead of the 10 I originally planned, but I have a green light to teach the course again this winter at the University of Toronto, and hope that others will pick the course up elsewhere.

Software Carpentry's new home

Sun, 25 Jun 2006 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Thanks to the folks at Enthought, the Software Carpentry course notes have a new home at http://www.swc.scipy.org. I'll move the wiki, bug tracker, and mailing lists there over the next couple of days as well. I hope the community will find the material useful — it's certainly been a lot of fun putting it together.

Revised Lecture on Teamware

Fri, 05 May 2006 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I've revised the Software Carpentry lecture on using team tools. I'd be grateful for feedback.

Software Carpentry 1111

Wed, 03 May 2006 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Revision 1111 of Software Carpentry just went into the repository. All the images are now there (thanks, Nick), along with code fragments and exercises (not as many as I'd like, but enough to get people started). Printed, it comes to 346 pages, but don't do this at home—the supposedly-transparent PNGs are still solid black when printed.

Things I'd like to do (or would like volunteers to contribute in the usual open source way) include:

Corrections Done

Fri, 28 Apr 2006 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

All the outstanding minor corrections to the Software Carpentry notes have now been made; there are still 21 diagrams outstanding, but they should be in by Sunday. My thanks to everyone who provided feedback—especially Adam Goucher and his very sharp eyes.

Zipf's Law of Feedback

Mon, 17 Apr 2006 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Zipf's Law says that frequency is inversely proportional to rank, i.e., the second most common word in a large body of text will occur half as many times as the most common. I have observed an even steeper curve for Software Carpentry feedback: of the 336 corrections I've received, 212 are from one person (Adam Goucher), 21 from Matthew Moelter, 12 from the next two people, and then we're down into the curve's long tail. Has anyone ever done similar stats on the volume or frequency of contributions to software projects?

341 Words

Sun, 09 Apr 2006 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The glossary for the Software Carpentry course now defines 341 terms. What may be more interesting (for those of you who have been following the course's development) is what I've taken out:

Code coverage and execution profiling: they really should be in the course, but don't fit into any of the existing lectures.

Date and time manipulation: it isn't part of software engineering per se, but like Unicode, floating-point roundoff, and a dozen other things, this is one of the subjects that everybody just ought to know about. Again, it doesn't fit neatly into any of the existing lectures.

Cross-site scripting, and a few other security-related terms: the security lecture has been completely revamped. It's much less ambitious, but (I hope) more informative.

Everything to do with UML: I've never used it outside of class, and have only ever worked with one person who did. I therefore feel like a bit of a fraud including it in a course on practical software development.

Things that I want to add (eventually):

New Security Lecture Up

Wed, 05 Apr 2006 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A new lecture on security is up. It has changed a lot from the hash I presented in the fall; feedback would be very, very welcome.

Integration and XML Lectures

Tue, 04 Apr 2006 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Hot on the heels of yesterday's lecture on integration (which desperately needs iron-willed editing) comes a rewrite of the XML lecture. This is a mix of HTML formatting rules and tags, and DOM; I'd appreciate knowing if you think it hangs together, and whether there needs to be more on how to process XML programmatically.

2020 Hype

Sun, 26 Mar 2006 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A report from Microsoft Research called 2020 Science got a lot of press this week: Nature seems to think it's the biggest story of the year so far, and The Economist gave it three full columns. Sadly, amidst the gush about how computers are revolutionizing science, no one mentions that most scientists have no idea how reliable their programs are—in fact, most scientists don't even know how they would figure that out [1,2]. If someone submitting a paper to Nature said, "We didn't calibrate the equipment, we didn't write down the settings, and we have no idea what the error bars on our graphs should be," their work would be bounced without a second thought. Unless computational scientists decide to live up to those standards, the "revolution" that 2020 Science describes will be a long time coming.

Web Server Programming Lecture Is Up

Mon, 06 Mar 2006 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The lecture on server-side programming is up: if you think it should be split into two parts, please let me know. That leaves only three to go (XML, security, and integration).

And by this time on Thursday, we'll be in Lima, Peru...

Client-Side Web Programming Lecture

Fri, 03 Mar 2006 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Software Carpentry lecture on client-side web programming is now up (sans diagrams). Comments and corrections welcome.

Last Two Lectures Are Up

Thu, 02 Mar 2006 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The last two lectures in the Software Carpentry course are up:

Teamware

Summary

(where "last" means "last in delivery order", not "last to be revised"). They're both fairly rough right now, so high-level feedback would be more useful than pointers to typos.

Database Lecture is Up

Thu, 23 Feb 2006 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The lecture on databases (actually an introduction to SQL) is now up. Comments and corrections welcome.

Second Lecture on Testing Now Online

Wed, 22 Feb 2006 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The second lecture on testing is now online. As always, comments and corrections are appreciated.

Second Lecture on Object-Oriented Programming

Tue, 21 Feb 2006 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The second lecture on object-oriented programming is now on the web. This describes operator overloading and static methods, and includes the design patterns material that was in the old design lecture (which has been removed—the general consensus was that it didn't work). As always, comments are welcome.

What Else for Software Carpentry?

Tue, 21 Feb 2006 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

16 lectures are now in place (more or less), which means I have 8 more to do. The syllabus shows what I've covered already; my current plans include:

unit testing

XML

SQL

more SQL

small-team development process

What do you think the other three should cover (keeping in mind that this is supposed to be a course on basic software engineering, rather than scientific programming)? Options include:

AAAS Annual Meeting 2006

Mon, 20 Feb 2006 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Wednesday, 11:10 p.m.: phone call from Air Can'tada saying that my Thursday morning flight to St Louis has been cancelled because of bad weather. Next available is 4:00 p.m. Friday afternoon—two and a half hours after my workshop is due to end. No, they can't help me find an alternative carrier. Expedia can, though, and by midnight, I have a ticket on Delta, via Cincinnati.

Data Lineage

Tue, 14 Feb 2006 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The January 2005 issue of ACM Computing Surveys (vol. 37, no. 1, if you prefer) has good review by Rajendra Bose and James Frew titled "Lineage Retrieval for Scientific Data Processing: A Survey". In it, they look at what scientists do to keep track of what data they have, where it came from, and what has been done to it. Some of my students last term were worrying about the same issues in the context of HL7 medical data. It seems like an ideal place for software engineers to apply their skills: I'd be interested in hearing from people who have home-grown or small-scale systems I could use as a starting point for a lecture in Software Carpentry.

Lecture on Binary Data

Tue, 14 Feb 2006 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Software Carpentry lecture on binary data is now up on the web. The content of this one has been fairly stable for a while, but that just means that all the bugs will be in the details—comments and corrections are greatly appreciated.

Regular Expressions Lecture is Up

Sun, 12 Feb 2006 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Software Carpentry lecture on regular expressions is now on the web. It's a bit of a jump from the previous lecture on design, but given how shaky the latter was, I needed the cuddly warm blanket of working on something I'm sure I understand. As always, comments and corrections to me, please.

Software Carpentry Design Lecture

Fri, 10 Feb 2006 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I've just posted a completely new lecture on object-oriented design ideas. I'd be very grateful for high-level comments: is it useful material? Do you think that relatively inexperienced programmers will get it? Etc.

First Lecture on Object-Oriented Programming Is Up

Mon, 06 Feb 2006 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I've split the lecture on object-oriented programming in two; the first one is now on the web (without diagrams). This ties with the rewrite of the style lecture for "most changes since the fall", so comments and corrections would be very welcome.

Eleven days 'til the AAAS workshop...

Debugging Lecture

Thu, 02 Feb 2006 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The debugging lecture is now up for comments and corrections. It still has the old art work, and the code sample for logging is pending, and it still feels like I'm waving my hands more than in other lectures, so suggestions would be particularly welcome.

Fourth Python Lecture for Software Carpentry

Sun, 29 Jan 2006 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The fourth lecture on Python for the Software Carpentry course is now up. It covers sets, dictionaries, and a little bit of algorithmic complexity. Comments and criticisms are very welcome.

And if that's too serious for year, check out the web site for the Waterfall 2006 conference ;-).

Quality Assurance Lecture Now Available

Tue, 24 Jan 2006 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

It used to be the introductory lecture on testing; now it's more of an introduction to quality assurance from a programmer's point of view. It's changed even more than the style lecture; comments and corrections would be very welcome. (And yes, I know, the diagrams are missing...)

Programming Style Lecture Has Been Revised

Mon, 23 Jan 2006 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Software Carpentry lecture on programming style has been revised. Comments on this would be particularly welcome, as it's substantially different from what I delivered last fall.

Third Software Carpentry Python Lecture on the Web

Mon, 23 Jan 2006 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The third lecture on Python for the Software Carpentry course has been posted; comments and criticisms are invited and appreciated.

Second Python Lecture Now on the Web

Wed, 18 Jan 2006 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The second revised lecture on Python for Software Carpentry is now on the web. As always, comments and corrections are welcome.

Intro Python Lecture Available

Sun, 15 Jan 2006 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The first Software Carpentry lecture on Python has been revised, and is up on the web. Comments and criticisms are very welcome.

Build Lecture Is Up

Wed, 11 Jan 2006 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The revised lecture on Make is now up for comments and feedback. Still no diagrams, I'm afraid...

Two More Revised Software Carpentry Lectures

Mon, 09 Jan 2006 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I have revised the second shell lecture and the lecture on version control for the Software Carpentry course, and added the necessary glossary definitions. As always, comments are greatly appreciated.

First Shell Lecture for Software Carpentry is Up

Wed, 04 Jan 2006 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The first lecture on using the shell is up on the new and improved Software Carpentry web site. Comments and feedback would be greatly appreciated.

Software Carpentry Introduction revised and on the web

Mon, 02 Jan 2006 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I have revised the Introduction to the Software Carpentry course. This was the easy one: no diagrams, no code fragments, no glossary entries; the rest will be harder. I'd be very grateful for feedback, comments, and corrections. (And yes, audio will follow, but not today—my nephews are playing Zombies downstairs, and juvenile shouts of, "Eat lead you undead scum!" aren't quite the background I have in mind.

$67 million a year

Wed, 28 Dec 2005 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The US Dept. of Energy has just announced the next round of funding for SciDAC, its flagship supercomputing program. US$67 million per year for three to five years. Supercomputing Online reports:

Research proposals funded under the SciDAC program will help create a comprehensive, scientific computing software infrastructure that integrates applied mathematics, computer science and computational science in the physical, biological and environmental sciences for scientific discovery on petascale computers.

New Year's Schedule for Software Carpentry

Tue, 27 Dec 2005 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I'm teaching a cut-down version of Software Carpentry at the IASSE in two and a half weeks. I'll have students half days for the weeks of January 16 and 23, and full days for the week of February 6. That's only 20 lectures (rather than 26), so the question is, what to cut? The answer has wider implications, since this will be the version of the course I take to the AAAS workshop.

Procrastination: One of the Few Things in Life Nicer Than Toast

Fri, 23 Dec 2005 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I finished rewriting the build system for the Software Carpentry course notes yesterday. Doing so was an extended form of procrastination: the system I built over the summer and used through the fall was adequate, but I wanted to clean a few things up, and then, well, I might as well make it easier for other instructors to add site-specific content, and make tables inclusions instead of inlining them, and mumble mumble mumble type type type...

Maintaining Correctness

Sun, 11 Dec 2005 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I'm re-thinking the lectures in the Software Carpentry course based on feedback from this term's students. I'm going to merge the three lectures on different development processes into one, and use the space that frees up to talk in more detail about programming style and software design—assuming, of course, I can think of something to say that isn't banal.

I also want to talk about the material in an article by Paul Dubois in the May/June 2005 issue of Computing in Science & Engineering called "Maintaining Correctness in Scientific Programs". Here are a few key lines from the introduction:

American Scientist article on Software Carpentry

Fri, 09 Dec 2005 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Jan/Feb 2006 issue of American Scientist, the magazine of Sigma Xi, contains an article on Software Carpentry.

Executive Version of Software Carpentry Course

Thu, 08 Dec 2005 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I'll be teaching a shortened version of the Software Carpentry course at the Institute for Advanced Studies in Software Engineering in Toronto early next year. For more information, or if you are interested in taking part, please see the IASSE's web site. The dates are:

January 16-20: one lecture per day, plus practical, with a quiz on Friday.

January 23-27: ditto.

February 6-10: two lectures and practical per day, with a short final exam on Friday afternoon.

I've got a lot to do between now and then...

Workshop at AAAS '06

Fri, 04 Nov 2005 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As I've mentioned before, I'm running a workshop on "Essential Software Skills for Research Scientists" at the Annual Meeting of the American Association for the Advancement of Science on Friday, February 17, in St Louis. I'm hoping to use the workshop to convince scientists, administrators, and policy makers that the things covered in the Software Carpentry course are essential to good science. If you know someone who ought to attend, please let them know, or pass them my contact information.

Software Carpentry at the AAAS

Wed, 21 Sep 2005 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I just received word that I'll be running a workshop on the aims, benefits, and curriculum of the Software Carpentry course at the Annual Meeting of the American Association for the Advancement of Science in St Louis on Friday, February 17. The AAAS AM is the biggest gathering of scientists in the world; I'm pretty excited.

Meanwhile, Michael Hoffman is teaching a short course on Python at the European Bioinformatics Institute in Cambridge, England, based on the course notes. 24 people are now enrolled in the course at the University of Toronto, 47 are auditing locally, and 21 others are sitting in from other locations. (These number don't include the students enrolled in the course at Indiana University, or the study group at CalTech.) Oh, and 38 students have already submitted Exercise 1, even though it isn't due until 5:00 Friday ;-).

Day 9

Tue, 20 Sep 2005 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Week two of Software Carpentry, and things are starting to settle down. There are now 93 (!) people signed up for the course:

U of T Elsewhere

Auditing 18 21

Enrolled 25

Unknown 29

Of those actually at the University of Toronto (as opposed to the
local hospitals—a few radiologists are sitting in) the breakdown
is:

16 Computer Science

14 Physics

9 Civil Engineering

Mechanical and Industrial Engineering

5 Biochemistry

Institute for Aerospace Studies

2 Institute of Medical Science

Mathematics

1 Astronomy

Biomaterials and Biomedical Engineering

Botany

Geology

Medical Biophysics

Zoology

Non-degree student

This week's lecture went much more smoothly than last week's, in part because we were in a larger room, with seating for everyone, and in part because the content was an introduction to Python, which I've given more times than I can count. There were still some glitches in the slides, though: a few things were out of order, and I really do need to choose more concrete examples. 33 students have completed the first exercise, which is already more than I'd expected—I'm feeling uncharacteristically optimistic right now ;-)

Software Carpentry: First Meeting

Wed, 14 Sep 2005 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The Toronto edition of the Software Carpentry course met for the first time on Monday. 51 people crammed into a room with seating for 34 (and no air conditioning, on an unseasonably warm day): medical biophysicists, computer scientists, civil engineers, and even a couple of faculty members.

I was pleased with the turnout, but less pleased with my lectures. I don't think my introduction to the shell made sense to anyone who didn't already know the content—it was far too long, and as Andy Lumsdaine (who's teaching from these same notes at Indiana) said, this stuff really does need to be interactive.

Software Carpentry at Indiana University

Mon, 22 Aug 2005 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Andy Lumsdaine is offering the Software Carpentry course at Indiana University starting on August 29, under the heading B649 Software Tools and Practices.

Software Carpentry course in Nature

Fri, 29 Jul 2005 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Nature has run a short blurb about the course I'm putting together on software development skills for scientists and engineers.

Software Carpentry notes are up

Fri, 08 Jul 2005 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I put an alpha version of the notes for the Software Carpentry course [1] on-line yesterday. Now, I'm looking for a way to convert them into a single PDF document, so that reviewers can download them in one shot. I've found some open source tools that don't do style sheets, and some commercial tools that'll do one page at a time, for money, but nothing so far that strikes me as being any better than "printing" with PDFCreator, one page at a time. Given that there are 31 pages, this is tedious enough that I'd like to find an alternative...

Python Software Foundation Grant

Thu, 30 Dec 2004 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Greg Wilson has been awared a grant from the Python Software Foundation to revamp his course on "Software Engineering for Scientists and Engineers", and put it under an open license, so that it can be used at other institutions. W00t!

11	could do it easily
15	could probably struggle through
3	wouldn't know where to start

12	could do it easily
10	could struggle through
7	wouldn't know where to start

Site	Dates	Registration
University of British Columbia	February 4-5	Open
Macquarie University	February 7-8	Full
AMOS Conference (Melbourne)	February 14-15	Full
University of Washington	February 25-26	Full
Lawrence Berkeley National Laboratory	March 4-5 & 6-7	For LBL staff
University of Virginia	March 7-8	Coming soon
Utah State University	March 23-24	Open
University of Arizona	April 4-5	Open
University of Manchester	April 18-19	Open
Telecom ParisTech	April 20-21	Open
Vrije Universiteit (Amsterdam)	May 2-3	Coming soon
GEOMAR (Kiel)	May 6-7	For members of ISOS, IMAP and the SFB754
Howard Hughes Medical Institute (Virginia)	May 6-7	For HHMI staff
Stanford University	May 6-7	Coming soon
University of California Berkeley	May 6-7	Coming soon (in R!)
Lawrence Berkeley National Laboratory	May 9-10	For LBL staff
University of California Davis	May 13-14	Coming soon
University of Oslo	July 3-4	Open
University of Bath	July 15-16	Coming soon

AA	Aron Ahmadia
WC	Warren Code
MD	Matt Davis
TG	Tommy Guy
MH	Mike Hansen
TH	Ted Hart
KH	Konrad Hinsen
KH	Katy Huff
JK	Justin Kitzes
TK	Trevor King
CS	Chang She
JS	Joshua Smith
LTB	Laura Tremblay-Boyer
GVW	Greg Wilson

Good	Bad
first day good with no programming experience	couldn't run the VM on underpowered computer
good general intro to basics of Python and VC	don't see why Python is better than R
liked the intro/advanced Python split	would like a longer bootcamp with more focus on Python
good git introduction	didn't want to learn Python
IPython notebook helped me learn	would like to give anonymous feedback
got to see real software engineering	didn't catch some important details that held things up
good coverage of Python + software engineering	have better setup instructions, test scripts
wanted to learn Python	Python instruction was too slow/basic
liked simple examples	got left behind sometimes
IPython/shell interaction awesome	not sure how to integrate git into workflow, have practical demos, case studies
good job squeezing a lot of content in	hard to type/listen/watch all at once
introduced scientific Python	would be nice to have a data cleaning example, workflow example
liked the showcase demos of matplotlib/IPython features	could have sped up introductory topics
combination of topics was good	have more content summaries, workflow visualizations
liked intro Python exercises	reading suggestions to follow up bootcamp
understand the basics of version control	wanted to see object oriented programming/calling C code

Good	Bad
can use python better	don't know how to apply git to work
covered lots of material	covered too much material
can use git better now	group felt too large for number of helpers
covered from basics to advanced programming	felt unprepared for git instruction
helpers were knowledgeable and nice	couldn't run VM, add system reqs
motivated version control well	python level was too easy
started with basics, had good help	need well defined goals for exercises
got exposed to a lot	group people by experience
all the material is available on GitHub	would have liked a more detailed description of course
learned enough to get started	didn't get any emails on the waitlist
have hope	getting info ahead of time
good overview for later	whole thing too fast
format - work through code live	topics jumped around
testing cases	maybe too language (Python) dependent, no objects
self-consistent complete package	burnt out at end of first day (fast)
inspirational	9-4:30 draining, more breaks between topics
great intro to vocabulary, how programmers think	didn't expect to program at end, was right
gained confidence from seeing experts	switch morning git with afternoon python on 2nd day
seeing github, advice on resources	not clear that mac users needed to install software ahead, why certain steps
had to do exercises, version control in beginner class	some hard to follow along, relies on previous success with commands
instructors :-D, with real-world experience	more relevant examples to audience, day to day data/problems
demo writing, troubleshooting, testing very valuable	what was xcode for?
	falling behind in python - 2 tiered exercises
	documentation needs comments
	is there help for release management?
	advanced math?
	third day - integrative exercise, case study - start to finish
	follow up project, final suggested homework assignment

Venue	Start Date	Instructor(s)
DAFX Conference (York, UK)	2012-09-13	Greg Wilson
University of Oslo	2012-09-17	Hans Petter Langtangen, Greg Wilson
Purdue University	2012-10-08	Michael Hansen, Anthony Scopatz, Jeff Shelton
Lawrence Berkeley National Laboratory	2012-10-17	Matt Davis, Katy Huff
University of British Columbia	2012-10-18	Laura Trembley-Boyer, Ian Mitchell, Peter Rawsthorne, Greg Wilson
University of California Berkeley	2012-10-20	Matt Davis, Katy Huff
Caltech	2012-10-23	Matt Davis
Oxford	2012-10-30	Stephen Crouch, Mike Jackson
Scripps Institute	2012-11-15	Titus Brown, Tracy Teal
University of North Carolina	2012-11-28	Jason Pell, Ethan White, Greg Wilson
University of Edinburgh	2012-12-04	TBD
University of Texas (Austin)	2012-12-10	Rosangela Canino-Koning, Katy Huff, Anthony Scopatz
University of Chicago	2013-01-12	Katy Huff, Anthony Scopatz
Technical University Munich	2013-01-22	TBD
Macquarie University	2013-01-31	Greg Wilson
Virginia Tech	2013-01-31	Tommy Guy
AMOS Conference (Melbourne)	2013-02-14	Greg Wilson

Age
<20	0.0%
20-25	18.3%
26-30	31.7%
31-35	25.0%
36-40	9.2%
41-45	4.2%
46-50	4.2%
>50	7.5%
Gender
Female	26.9%
Male	73.1%
Occupation
Undergrad	2.6%
Grad student	55.7%
Post-doc	13.9%
Faculty	9.6%
Support	13.0%
Industry	5.2%
Preferred Platforms
Linux	54.2%
Mac OS	50.0%
Windows	35.8%

Sep 13-14, 2012:	DAFX Conference (York, UK)
Sep 17-18, 2012:	University of Oslo
Oct 8-9, 2012:	Purdue University
Oct 17-18, 2012:	Lawrence Berkeley National Laboratory
Oct 18-19, 2012:	University of British Columbia
Oct 20-21, 2012:	University of California Berkeley
Oct 23-24, 2012:	Caltech
Oct 29-30, 2012:	Oxford
Nov 15-16, 2012:	Scripps Institute
Nov 28-29, 2012:	University of North Carolina
Dec 4-5, 2012:	University of Edinburgh
Dec 10-11, 2012:	University of Texas (Austin)
Jan 19-20, 2013:	University of Chicago
Jan 22-23, 2013:	Technical University Munich
Feb 15-14, 2013:	AMOS Conference (Melbourne)

Monday:	Start of a two-day workshop at Johns Hopkins University
	Online tutorial for participants in the London workshop
Tuesday:	Online tutorial for participants in the Edmonton workshop
Wednesday:	Online tutorial for participants in the Newcastle workshop
	And another one for participants in the Vancouver workshop
	And one more for participants in the Michigan State workshop
Thursday:	Combined tutorial for participants in previous workshops
Friday:	All-day strategy meeting in Toronto

Good	Bad
clarity data management sticky notes workflow overview/coverage resources coffee/snacks exercises how to use programming correctly learning terminal testing learning science Python helpers one step further version control modules on website anecdotes general philosophy (organic)	not enough Python too fast (esp 2nd day) hands-on (second day) needed warmup primer Greg's jokes resources (overwhelming) text editor software install issues too short crossover confusion too much on data management communication prior to course (software install) screen real estate more version control why better than MATLAB mystify setup

University of British Columbia	May 22-23	39/40
Johns Hopkins University	June 18-19	7/20
Paris	June 28-29	9/25
Boston	July 9-10	23/40
University of Waterloo	July 12-13	1/40
Halifax	July 16-17	8/40
University of Toronto (Scarborough)	July 19-20	14/40

Site	Start	Ticketed	Space
Monterey Bay Aquarium Research Institute	March 26	17	40
Lawrence Berkeley National Laboratory	March 28	40	40
University of Chicago	April 2	116	150
Utah State University	April 14	26	50
University College London	April 30	49	50
Michigan State University	May 7	7	40
Newcastle University	May 14	9	40
University of British Columbia	May 22	40	40

	for people who complete the core curriculum
	for helping to teach (either workshops or online)
	for organizing workshops
	for creating (or fixing) content

	for people who have mastered our core content
	for people who have organized and run workshops
	for people who have created content

Q01:	How can I write a simple program?
Q02:	How can I make the program I've written easier to reuse?
Q03:	How can I reuse code that other people have written?
Q04:	How can I share my work with other people?
Q05:	How can I keep track of what I've done?
Q06:	How can I tell if my program is working correctly?
Q07:	How can I find and fix bugs when it isn't?
Q08:	How can I get data into my program?
Q09:	How can I manage my data?
Q10:	How can I automate this task?
Q11:	How can I make my program faster?

Chapter	Word Count
Introduction	768
Spreadsheets	3924
Subversion	6405
Python	6345
Interlude: What Is Text?	1071
Functions and Libraries	9409
Interlude: Boolean Logic	921
Case Study: Invasion Percolation	8929
Interlude: How Are Numbers Stored?	1828
Testing	4549
Error Handling	1529
The Shell	12919
Make	5668
Interlude: Provenance	1810
Sets and Dictionaries	7627
Case Study: Phylogenetic Trees	1689
Systems Programming	60
Interlude: Configuring Programs	2146
Numerical Programming	6193
Multimedia Programming	4125
Steganography	1657
Installation	135
HTML and XML	6404
Databases	10676
Regular Expressions	8037
Object-Oriented Programming	4002
Building Desktop GUIs	137
Interlude: Persistence	5984
Web Programming	9639
Security	1285
Performance	8043
Parallel Programming	931
Software Engineering	6391
Epilog	873
Acknowledgments	202
Glossary	12074
Bibliography	3378
Total	167,763

Topics	24
...mostly done (say, 80%)	18
...not even started	6
Words per topic	5,000
Words to write	48,000
Words per day	1,500
Days required	32
Editing, diagrams, etc.	×2
Total days	64
Code complete	July 15, 2011

2.51	Automating Repetitive Tasks
2.50	Reproducible Research
2.50	Data Visualization
2.47	Version Control
2.44	Performance Optimization
2.41	Data Structures
2.39	Testing and Quality Assurance
2.39	Coding Style
2.38	Basic Programming
2.35	Using the Unix Shell
2.35	Parallel Programming
2.35	Debugging with a Debugger
2.29	Computational Complexity
2.22	Object-Oriented Programming
2.20	Working in Teams/on Large Projects
2.19	Designing a Data Model
2.15	Refactoring
2.10	Matrix Algebra
2.09	Static and Dynamic Code Analysis Tools
2.07	Systems Programming
2.04	Integrating with C and Fortran
2.03	Design Patterns
2.01	Packaging Code for Release
1.95	Functional Languages
1.94	Handling Binary Data
1.82	Image Processing
1.74	Build a Desktop User Interface
1.73	XML
1.65	Create a Web Service
1.39	Geographic Information Systems

Education
77.3%		Graduate degree
22.1%		Undergraduate degree
0.6%		High school
Field
41.0%		Computer Science
30.1%		Earth Sciences
28.9%		Physics
25.4%		Mathematics and Statistics
11.0%		Microbiology
9.2%		Biomedical Engineering
6.9%		Macrobiology
5.2%		Medicine and Health Care
5.2%		Electrical Engineering
5.2%		Astronomy
4.6%		Mechanical Engineering
4.6%		Aerospace Engineering
4.0%		Chemical Engineering
2.9%		Psychology
2.3%		Economics
2.3%		Business/Finance
1.2%		Linguistics
1.2%		Civil Engineering
0.6%		Social Sciences
0.6%		Arts and Humanities
Role
44.8%		Academic Researcher
32.8%		Software Developer
16.7%		Graduate Student
16.7%		Government Research Scientist
10.3%		Engineer
9.8%		Manager/Supervisor
8.6%		System Administrator
3.4%		Teacher
2.9%		Industrial Research Scientist
1.1%		Undergraduate student
1.1%		Laboratory Technician
Priorities
2.51		Automating Repetitive Tasks
2.50		Reproducible Research
2.49		Data Visualization
2.46		Version Control
2.43		Performance Optimization
2.41		Data Structures
2.41		Coding Style
2.38		Basic Programming
2.37		Testing and Quality Assurance
2.35		Parallel Programming
2.34		Debugging with a Debugger
2.33		Using the Unix Shell
2.29		Computational Complexity
2.21		Object-Oriented Programming
2.21		Designing a Data Model
2.19		Working in Teams/on Large Projects
2.14		Refactoring
2.10		Static and Dynamic Code Analysis Tools
2.09		Matrix Algebra
2.06		Systems Programming
2.06		Integrating with C and Fortran
2.03		Design Patterns
2.01		Packaging Code for Release
1.95		Functional Languages
1.93		Handling Binary Data
1.80		Image Processing
1.77		Introduction
1.75		Build a Desktop User Interface
1.73		XML
1.64		Create a Web Service
1.39		Geographic Information Systems

Software Carpentry on The Carpentries

Software Carpentry Governance Committee - Goals for 2026

Looking for New Committee Members!

Calling for new Lesson Program Governors

What is a Lesson Program Governor?

The Carpentries’ Websites Relaunched!

Almost Ready: The Carpentries’ new websites and Handbooks

Introduction

Lesson programs' websites content pared down

Translating Software Carpentry lessons into Ukrainian language: a status update

Lesson programs' blog posts migration

Changes to Software Carpentry's Lesson Program Governance Committee and their work

Welcoming the New Lesson Program Governance Committees

Calling for Volunteers to join a Lesson Program Governance Committee

Planning for Lesson Program Governance Committees

Where We Have Been

Running Newcastle University's First Online Software Carpentries Workshop

How it all started

Teaching Software Carpentry at the University of Botswana

Carpentry on a Chromebook

Want to Organise a Workshop on Image Processing?

Running workshops on limited budgets

Existing Funding Models

Software Carpentry: Considering the Future

Webinar with Rochelle Tractenberg: Debrief

Welcome to New Trainers

Revival of the African Task Force

Carpentries for Latin America

Carpentries para Latinoamerica

Mentoring Groups are Back!

Announcing the first joint Carpentries Bug BBQ

Call for Code of Conduct Committee Volunteers

Call for Contributions: Data Carpentry Ecology and Software Carpentry Curriculum Advisory Committees

First African Carpentries Instructor Training of 2018.

CarpentryCon - Hotel Accommodation Options

Hotel Accommodation for CarpentryCon 2018

Lesson Infrastructure Subcommittee 2018 February meeting

Software Carpentry and Data Carpentry merge

State of the State: Instructor Checkout

Library Carpentry Governance - An Update

Mentoring Groups Showcase their Accomplishments

My Favorite Tool - Docker

What kind of tool is it?

Why I like it

Valerie Aurora to Keynote at CarpentryCon 2018

Making the Case for CarpentryCon

Carpentry Champions Call

Unveiling Our New Logo

Carpentries Transition From Fiscally Sponsored Project to NumFOCUS Community Alliance Member

Be our Advocate

Foundations and Funders: Why Should You Sponsor CarpentryCon 2018?

Why sponsor CarpentryCon?

Why Should You Sponsor CarpentryCon 2018?

SPONSORS!

Why sponsor CarpentryCon?

Workshop Template Enhancement Proposal

How do instructors get placed at workshops?

My Favorite Tool - Midnight Commander

What kind of tool is it?

Why I like it

The Centrality of the Code of Conduct

Teaching Statistics in the 21st Century

Fourteen and Counting - People's Favorite Tools

My Favorite Tool - Twitter

Why do I like Twitter?

A Successful 2nd RSE Conference

My Favorite Tool - QGIS

My Favorite tool - A Coding Sandbox

Why do I like it?

Green Stickies for 2017: Report from the Happy Holidays Green Sticky Party Calls

Our 2017 Community Service Award winner: Anelda van der Walt

Announcing the 2018 Executive Council for the Carpentries

Challenges Assessing Data Science

My Favorite Tool - Todoist

Why do I like it?

Poster Competition Now Open

Design a Promotional Poster for CarpentryCon 2018

When Do Workshops Work? A Response to the 'Null Effects' paper from Feldon et al.

CarpentryCon 2018 - Website is Live!

Celebrate the Wins of 2017: Join the year's final community call

Writing script:	1.5 hours
Doing 3 takes:	1.0 hours
Editing:	1.0 hours
Transcribing:	0.5 hours
Screenshots:	0.5 hours

01.	Name	39/39	100%
02.	Email address	39/39	100%
03.	Level of study
	MSc	12/39	30%
	PhD	27/39	70%
04.	Primary programming language
	MATLAB	16/39	41%
	Python	2/39	5%
	Other	6/39	15%
	None	15/39	39%
05.	Knowledge of primary language
	Don't know how to use it	28/39	72%
	Understand basic commands	10/39	26%
	Can program competently	1/39	2%
	Expert	0/30	0%
06.	What other languages do you know?
	HTML	11/27	41%
	R	4/27	15%
	Other (VB, Java, Perl, etc.)	10/27	37%
	No answer	12/39
07.	Would you like pre-class tutorial on
	programming basics (loops, files, if/else)?
	Yes	36/39	92%
	No	3/39	8%
08.	Do you have a laptop?
	Yes	39/39	100%
	No	0/39	0%
09.	Preferred OS
	Windows XP	14/39	36%
	Windows Vista	12/39	31%
	Mac OS X	7/39	18%
	Linux/Unix	9/30	0%
10.	Do you have a MATLAB license?
	Yes	9/39	23%
	No	30/30	77%
11.	Which topics are you interested in?
	Databases	16/39	47%
	Functions and Modules	14/39	41%
	Debugging	10/39	29%
	Image Processing	10/39	30%
	Object-Oriented Programming	10/39	30%
	Web Application Programming	9/39	26%
	GUI Programming	8/39	23%
	Web Client Programming	7/39	21%
	Computational Complexity	6/39	18%
	How Web Servers Work	6/39	18%
	Regular Expressions	6/39	18%
	XML	6/39	18%
	Automated Builds	5/39	15%
	Sets and Dictionaries	5/39	15%
	Unix Shell Scripting	5/39	15%
	Binary Data	3/39	9%
	Empirical Software Engineering	3/39	9%
	Quality Assurance	3/39	9%
	Unit Testing	3/39	9%
	Version Control	3/39	9%
	Software Development Lifecycles	1/39	3%
	Other (please specify)	10/39	30%