Tooling on The Carpentries

Using RMarkdown with the new lesson template

Fri, 08 Jul 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Our lesson template is getting a face-lift. Actually, it is a lot more than that: all the internal mechanics are also affected.

What’s new with the template?

The lesson maintainers have developed a new template that comes with features that have been repeatedly requested such as an easy way to navigate among episodes within a lesson: there are now “previous” and “next” arrows in each episode.

So You Want to Make a Screencast

Wed, 13 Apr 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I am the author of [Learning Cython](http://shop.oreilly.com/product/0636920046813.do), a new series of videos teaching [Cython](http://cython.org/). There are over 70 videos, making up around five and a half hours of content. If you're involved in education, you may have considered making a screencast like these and I'd like to give a short overview of what that entails.

Publisher

You may be happy to self-produce and self-host, and I would be too except that I was approached by a publisher and made an offer to produce the videos for distribution on their site. How this happened is interesting: in 2015, on a whim, I decided to submit a Cython talk proposal for PyCon AU. My talk got accepted, and a few weeks before the conference the publisher contacted me and made the offer.

AMY release v1.5.1

Sun, 03 Apr 2016 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We’re very pleased to announce the newest (v1.5.1) release of AMY, our workshop management tool. This release is special because:

it contains more bugfixes than new features
we had a number of submissions from prospective Google Summer of Code 2016 (GSoC) students.

GSoC students that submitted fixes or new features to AMY:

Shubham Singh added “Notes” field to instructor profile update form
Nikhil Verma found and fixed “List duplicates” page error when no duplicates existed
Shubham Singh added new tag “hackaton”
Chris Medrela found and fixed 404 page for revisions that didn’t exist

Additionally Nikhil Verma is working on getting certificates to generate from within AMY, but this will probably land in v1.5.2.

The Morea Framework

Fri, 20 Nov 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I first met Philip Johnson, a professor of Computer Science at the University of Hawaii, through shared interests in empirical software engineering research and Google Summer of Code. He has recently been developed the Morea Framework for creating structured course websites using GitHub and Jekyll. "Morea" stands for "Modules, Outcomes, Readings, Experiences, and Assessments", which are the five main elements the framework supports. As you can see from the project gallery, it's much more structured than our lessons. It also requires more tooling—Morea Framework sites are built using custom Jekyll plugins, and the source relies much more heavily on include files than our template—and it's geared very strongly toward traditional semester-long courses.

AMY 1.0 Released

Sat, 10 Oct 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We are very pleased to announce the release of Version 1.0 of AMY, our online workshop administration tool. A complete list of features is included in this blog post, and this milestone describes what we're planning to do for Version 1.1. If you enjoy web programming in Django, you'd be welcome to join in.

AMY Version 0.6

Sat, 18 Jul 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Yesterday was the release day for AMY v0.6.. Here's what's new and what's changed in this release.

Teaching with Jupyter

Tue, 14 Jul 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We had a great birds of a feather session at the SciPy conference last week and decided to create a mailing list specifically for instructors who are interested in using the Jupyter Notebook for teaching. The aim of the mailing list is to provide a place for instructors to share materials, strategies, advice, etc. on teaching with the notebook and the logistics that are involved with that. If you're interested in joining the mailing list as well, you can add yourself at https://groups.google.com/forum/#!forum/jupyter-education

Congratulations to Project Jupyter

Tue, 07 Jul 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This is wonderful news: the Helmsley Charitable Trust, the Alfred P. Sloan Foundation, and the Gordon and Betty Moore Foundation have just pledged $6M over three years to Project Jupyter (formerly known as the IPython Notebook). Fernando Perez of University of California, Berkeley and Lawrence Berkeley National Laboratory and Brian Granger of California Polytechnic University, San Luis Obispo will lead the project. Congratulations to them, their team, and their community—this is well deserved, and will help scientists all over the world in ways we cannot yet even imagine.

Pushing Back

Wed, 01 Jul 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A week ago, we posted a proposal to use Jekyll to build our lessons rather than Pandoc. The immediate reaction was almost uniformly positive, but in the days since, people have pushed back on two fronts:

Using Jekyll for Lessons

Wed, 24 Jun 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A recurring complaint about our lesson template is that it requires authors to commit generated HTML files to their repositories as well as their Markdown source files. This is necessary because we use Pandoc to convert Markdown to HTML, but GitHub will only run Jekyll.

There were a bunch of reasons for using Pandoc instead of Jekyll, but it is now clear that the simplicity of only committing Markdown—i.e., of using GitHub pages the way they're meant to be used—is more important. We have therefore created a prototype of a Jekyll-based template (which is rendered here). The most important changes are:

Splitting the Shell Window

Sun, 21 Jun 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Raniere Silva has written a short post about a trick he found (via Kate Hertweck) for splitting the terminal window when teaching the shell so that recent commands stay visible at the top. It's a clever idea; we would welcome feedback from other instructors who have tried it or similar things. And if you have tricks of your own that you'd like to share, please let us know—we'd be happy to feature them here.

Amy Version 0.4

Fri, 19 Jun 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Today's the deadline for AMY v0.4. It contains a bunch of usability fixes, so all our admins should be happy :-)

Amy Version 0.3

Sun, 07 Jun 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Version 0.3 of Amy, the web application we're building to manage workshops, has just been released. Among the improvements are:

many UI upgrades
autocompletion
password and permissions management
an easier way to award badges
and many others

It's already making our admins' lives better, and there are lots more improvements in the works. For details, please see Piotr Banaszkiewicz's recent blog post.

Updating the Lesson Template

Sun, 07 Jun 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A couple of weeks ago, we asked our lesson maintainers what changes they would like to see in our lesson template based on their experiences getting Version 5.3 ready for publication. Their comments are summarized below; it's an ambitious list, but I think we can do most or all of this between now and September. If there are other things you think we should change in the way we structure lessons (rather than in particular lessons themselves), please add comments, and if you'd like to dive into one of these, please checkout the current lesson-template issues and create a pull request against the lesson-template repository.

How to Send a Pull Request to the Lesson Template

Mon, 13 Apr 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

At the end of last year we split out lessons to have one Git repository per topic that we teach. To ensure that all lessons have the same look we use one template for our lessons. If you want to create a new lesson from scratch we already have a nice step-by-step guide but what about if you want to contribute to the template itself? In this post I will explain how I wrote my pull request for our template.

Wrong Is Useful: Lessons as Packages

Fri, 27 Feb 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

"What would Greg do? [pause] OK, now that we've ruled that out..."
— overheard

I wrote a post last July about using package managers like RPM, Homebrew, and Conda to track dependencies between lessons, so that a student could say something like conda install unit_testing and get a lesson on unit testing, along with the code, sample data, and other lessons it depends on. I also mused that it could help make research more reproducible: after all, a paper is just a lesson on something that's never been taught before.

Managing GitHub Notifications

Fri, 20 Feb 2015 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

SWC has many GitHub repositories for lessons, websites, and workshops, and we have many conversations about our work take place through GitHub issue threads. Since GitHub creates e-mail notifications of repositories and issues you participate in, that means you can quickly start getting loads of e-mail.

While it's helpful to be kept abreast of developments on a project, if you get a flood of GitHub messages you won't see what's important. Here's a quick primer on fine-tuning your notifications so that you only see what you want, gleaned from this helpful conversation:

Guidelines for Extracting History

Mon, 15 Dec 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As discussed previously, we are currently extracting individual lessons from the bc repository to make them more modular, which will ease use, contribution, and maintenance. This post presents some guidelines for extracting individual lessons and how to contribute to lessons in the meantime.

Templates: We Live, We Learn

Tue, 09 Dec 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We have partially converted four of our core lessons to the new lesson template, and are making a few tweaks as a result. The most important of these is to move the source files for a lesson's web pages out of a pages sub-directory and into the root directory, so that (for example) the first topic in a lesson will be ./01-intro.md rather than ./pages/01-intro.md.

Doing this means that the Markdown source files will be in the same directory as the HTML pages compiled from them. That's generally considered a bad thing, since (a) it makes it harder for people to tell source files from generated files and (b) the chances of accidentally deleting source files when you just meant to delete generated files goes up significantly. We think it's the right choice in this case because:

Announcing the Lesson Validator

Fri, 05 Dec 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Thanks to the work of Andy Boughton and Raniere Silva, we now have a validator that determines whether lessons fit the new lesson template. This post explains what it does, where there is room for improvement, and how you can help.

Lessons, the Repository Split, and Translations

Sat, 15 Nov 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Keeping on the roll of the posts about the repo split, templates, and metadata, Software Carpentry now needs to consider how to handle translated lessons. The core Software Carpentry lessons have be translated by bilingual instructors into Korean produced by Victor KC Lee (and friends) and Spanish by Francisco Navarro (and friends). With the upcoming repo split, I think its a good time to examine the various options of how we might handle translations generally.

Pandoc and Building Pages

Wed, 29 Oct 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Long-time readers of this blog and our discussion list will know that I'm unhappy with the choices we have for formatting our lessons. Thanks to a tweet from Karl Broman, I may have an answer. It's outlined below, and I'd be grateful for comments on usability and feasibility.

A New Lesson Template, Version 2

Thu, 23 Oct 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Update: this post now includes feedback from participants in the instructor training session run at TGAC on Oct 22-23, 2014. Please see the bottom of this page for their comments.

Thanks to everyone for their feedback on the first draft of our new template for lessons. The major suggestions were:

A New Template for Lessons

Tue, 14 Oct 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Note: this post has been superseded by this one. Please post comments and feedback there.

We blogged two weeks ago about a new template for workshop websites. It's now time to start thinking about what lessons will look like: as we said at the last lab meeting, we're going to break the current lesson repository into smaller and more manageable pieces, but we need to decide what those pieces will look like first. The post below is our current thoughts; comments and/or follow-on posts about alternatives like those already written by Gabriel Devenyi and Raniere Silva would be very welcome.

Of Templates and Metadata

Tue, 14 Oct 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As an appendix to the splitting the repository post, Greg recently posted a straw man template for how lessons might be structured after the repo split. He followed up after with more details. There a lot of good ideas there on how we can encourage good structure for lessons and help learners and instructors alike going forward.

Yet Another Template for Lessons

Tue, 14 Oct 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

After the splitting the repository post, Gabriel Devenyi and Greg Wilson wrote some suggestions for how the new lessons repositories should look like (see Gabriel's post about metadata and Greg's post about overall file structure). From my experience at the Mozilla Science Lab sprint I don't like Gabriel's preq metadata because I don't believe it helps very much. I also don't like Greg's proposal to duplicate some files in every Git repository, so here are some changes that I suggest.

A New Template for Workshop Websites

Sat, 04 Oct 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The first step in reorganizing the bc repository is making it easier (much easier) for people to create websites for workshops. The current instructions are almost 3000 words long, and even experienced GitHub users find the process daunting, so we're going to simplify things as much as we can, even if that means not doing things the "right" way.

Browsercast

Sat, 04 Oct 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As we mentioned back in April, Gabriel Ivanica spent the summer working on a Google Summer of Code project called Browsercast. His goal was to build a web-native alternative to screencasts; if you'd like to see the result, head over to the demo page, click on the eye icon in the upper left, and press the "play" button. I hope to use it, or something descended from it, to build searchable, accessible narrations of our lessons that will play well on everything from desktops through tablets to mobile phones. If you'd like to help, please fork the project on GitHub and dive in.

Help Us Build an Admin Tool for Bootcamps

Fri, 20 Jun 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

It takes a lot of work behind the scenes to match instructors to bootcamps, keep track of who taught (or learned) where, and manage our mailing lists. Our current cobbled-together tools are at their breaking point, so Atul Varma, a developer at Mozilla, put together a simple browser-on-the-desktop app to help administrators find instructors based on geography and skills. We'd like many other features—in particular, bulk upload of attendance lists from bootcamps—so if you would like to help us out, please clone this repository and send us fixes and additions.

Many thanks to Atul for getting us started—we're looking forward to not grepping for instructors any more :-).

Fixing 14 Repositories

Sun, 15 Jun 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We cloned our workshop repository at the beginning of the workshop at Spelman to give everyone an identical directory tree and contents to explore, and to distribute the sample data files. On the first day of our workshop, about a third of the learners (all running Windows) could not clone the workshop repository: git clone was failing for them with the error:

Knocking on the Future's Door

Thu, 08 May 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Once again I feel like I'm knocking on the future's door but nobody's answering. The task we set ourselves seemed simple: produce a nicely-formatted PDF of the Version 5 lessons to give learners as a reference (and to print as a book to give instructors when they finish their training). Fifty years after the creation of the first computer typesetting systems, you'd think this would be easy. It's not, and the reasons why highlight yet again why so many scientists would rather keep playing the kazoo than learn to play the violin.

Math Authoring Gap and MathUI

Wed, 23 Apr 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

In a previous post Greg Wilson wrote about the writing gap

between people who value the "I can see what I'm doing" of Microsoft Word and those who care more about the "I can tell what I did" of version control.

For STEM (science, technology, engineering and math) folks there is also the gap related to math expressions (you can use LaTeX to deal with math expressions but for beginners who spent around 20 years using Microsoft Word moving to LaTeX can be challenging).

Do you know that some people are working to solve the gap with math expressions?

Bridging the Writing Gap

Sun, 06 Apr 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A few months ago, we had an interesting discussion about what Software Carpentry should teach about writing and publishing in the 21st Century. One thing that came through loud and clear was the gulf between people who value the "I can see what I'm doing" of Microsoft Word and those who care more about the "I can tell what I did" of version control¹.

Sites like Authorea, writeLaTeX, and ShareLaTeX are trying to bridge that gulf by giving people a WYSIWYG authoring tool in the browser that uses LaTeX as a storage format. This is pretty exciting: since it (potentially) allows collaborators to interact in whichever mode they prefer, it allows people to transition from one to the other instead of requiring them to make a great leap sideways.

Both sites currently allow people to save work on site or in Dropbox. It would be very cool if they also allowed people to save work in online version control repositories such as GitHub. Someone who isn't comfortable with version control could simply select "Save..." to push their changes, while someone who's already mastered pull requests and merging could interact that way, so that once again, the system could help people transition gradually from one mode to the other.

Announcing NBDiff

Sun, 30 Mar 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I am very pleased to announce the first release of NBDiff, a tool for diffing and merging IPython Notebooks that is now available on GitHub and PyPI. NBDiff was created by a team of students at Concordia University in Montréal; our thanks and congratulations to Shurouq Abusalah, Tavish Armstrong, Marwa Malti, Lina Nouh, Boris Pipev, Selena Sachdeva, and Richard Tang. For more information, see the NBDiff web site.

Everything Old is New Again

Fri, 14 Mar 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Yesterday, the New York Times R&D Lab announced streamtools, a web-based graphical tool for working with streams of data. It lets users create dataflow systems to re-mix and process the streams they believe most data will consist of in the near future.

Streamtools looks nice, but I do have some questions:

An Online Peer Instruction Tool

Sun, 09 Feb 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Peer instruction is a teaching technique originally developed by Eric Mazur and colleagues in the early 1990s. Study after study has shown that peer instructions works better than conventional lecturing, but to the best of my knowledge, no online learning platform directly supports peer instruction. I'd like to fix that, and I hope some of our readers would like to help.

Keeping Track of Problems

Wed, 05 Feb 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Setup and configuration is often the biggest challenge bootcamp participants face, particularly on Windows. Our installation instructions are pretty good, but to make them better, Justin Kitzes has started building a wiki page on GitHub to keep track of what's gone wrong and what fixes have worked. Many thanks to Justin for getting the ball rolling—contributions would be very welcome.

Introducing the Image Novice Module

Thu, 16 Jan 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The image novice module provides a simple image manipulation interface for beginners that allows for easy loading, manipulating, and saving of image files. It has been developed collaboratively with Greg Wilson, Tony Yu, and Stephan van der Walt. Below, I show how to use the module for some basic image manipulation.

Publishing on the Web

Wed, 15 Jan 2014 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Back in December, I asked a question on the Software Carpentry issue track: What should we teach about writing/publishing papers in a webby world? It led to a lively discussion about how the web is changing scientific publishing, the tools people can use to take advantage of those changes, and the gap between what's possible and what publishers (and senior academics) will actually accept. Many commenters dove immediately into the details of specific tools, but Justin Kitzes nailed it when he said:

Introducing Citation Files

Mon, 02 Sep 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Robin Wilson, of the University of Southampton, recently posted a note on the Software Sustainability Institute's blog about CITATION files. In brief, he (and we) would like to encourage scientific programmers to put a plain text file called CITATION in the root directory of each project, and to use it to tell readers how best to cite that software. The example Robin gives is:

Browsercast

Fri, 24 May 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

David Wolever (a freelance developer here in Toronto) has released the first version of BrowserCast, an IPython Notebook plugin that allows you to synchronize a voice-over soundtrack with a step-by-step reveal of a notebook, thereby creating a screencast-style presentation in the browser. He has created two traditional (video) screencasts showing why BrowserCast is cool and how to create a presentation with BrowserCast, but if you really want to get a feel for it, you can use the bookmarklet that lets you give it a try with your own notebooks, or grab the plugin from GitHub and take it for a test drive.

Translucent Badges

Thu, 02 May 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Digital badges are a hot meme right now. They let anyone, anywhere, issue credentials that are finer-grained than degree certificates or driver's licenses. Want people to know that you can change the oil in a car? There's a badge for that. Or that you can speak conversational Frisian? There's a badge for that too. And "backpack" sites make it easy to aggregate badges, including the ones that show you've created content for Software Carpentry, that you're qualified to teach it, or that you've helped to organize and run a bootcamp.

Installation Revisited

Mon, 08 Apr 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Regular readers (and anyone who has attended a bootcamp) will know that getting learners' laptops set up is the single biggest headache we have. Titus Brown has just posted a discussion of what goes wrong, and some ways we could try to improve things. If you have suggestions to contribute, please add comments there.

Using the IPython Notebook as a Teaching Tool

Sun, 24 Mar 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I had a fruitful discussion with Jon Pipitone today about using the IPython Notebook for teaching. Long story short, there are several possible approaches, but we can see problems with each. To set the stage, here are the two "pure" models most widely used for teaching programming today:

Frozen: the instructor embeds snippets of code in slides (which can be in LaTeX, PowerPoint, HTML, or some other format).

Advantages:

Easy to interleave code and commentary.
Easy to draw on top of code to highlight, make connections with commentary, etc.
Easy for other people to re-use.
Easy to add presenters' notes.

Disadvantages:

The code isn't immediately executable, so some combination of manual checking and tooling has to be used to ensure that the output shown is up-to-date with the code, that the code shown is up-to-date with any hand-out files, etc.
It's really easy for instructors to race through slides too fast for learners can follow.
"Watch and listen" is a passive learning model, and the more passive learners are, the less they learn.

Live Coding: the instructor types into an interpreter as she's teaching; the only thing on the screen is her code and its output, and everything else is delivered verbally.

Advantages:

Allows responsive improvisation: the instructor can answer "what if?" questions much more easily.
Constrains the speed of presentation (somewhat).
Facilitates "sideways knowledge transfer", e.g., learners can pick up keyboard shortcuts and other "hows" of coding, etc.
Learners learn more if they are typing in code as they follow along.

Disadvantages:

Learners now have to type and watch at the same time; the former often distracts from the latter (particularly if they make typing mistakes that they can't spot themselves, so that they wind up with a triple burden of watching, typing, and debugging simultaneously).
Learners walk away with just the code, not what was said about it, and code alone can be hard to re-understand.
It discourages the use of diagrams (instructors can't doodle directly in the Notebook the way they would on a whiteboard, and "let me import this now" is clumsy).
There's no obvious place to store the presenters' guide.

With practice, preparation, and the right A/V setup, instructors can use a hybrid model:

Teaching with ipythonblocks at UW

Sat, 02 Mar 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The recent University of Washington bootcamp was my first chance to try teaching Python with ipythonblocks, and it was a great success!

The students reported that the immediate feedback from ipythonblocks was a great way to see what their code was doing. We were able to move quickly through topics like for-loops and if-statements in relative comfort, and even cover slicing with ease.

Record and Playback in the IPython Notebook

Tue, 22 Jan 2013 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We decided last fall to start teaching Python using the IPython Notebook rather than a plain old command-line interpreter. It will take us a few months to complete the transition, but one of the things that we're hoping to do in the meanwhile is build a record-and-playback plugin for the Notebook. We think this will be more useful than the screencasts we currently have because:

You've Shown Me the C, Now Where's the Python?

Sun, 16 Dec 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The W3C's Provenance Working Group recently published a new draft of their proposed standard for tracking provenance on the web. It's pretty dense stuff: even the primer, which uses the word "intuitive" five times to describe itself, is hard to follow if you've never been immersed in Dublin Core, TURTLE, and the like. That isn't a criticism—this stuff is intrinsically hard—but I think most scientists won't be able to see the forest for the trees.

Alpha Testing Ideas for the IPython Notebook

Tue, 27 Nov 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Before I post any of these ideas on the IPython Notebook wiki, I'd be grateful for feedback from our learners and instructors. Most of these (maybe all of them?) would be done as extensions, rather than as patches to IPyNB itself. Please let us know what you think; I'll revise and then cross-post to the IPyNB folks.

Our Users

Anna teaches scientists how to program. She usually does this by coding live in front of a small class (20-40 people), but also wants to create recordings for people to view outside class.

The Tool (I Think) We Need To Do Peer Instruction Online

Mon, 19 Nov 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Clay Shirky's recent essays "Napster, Udacity, and the Academy" has attracted a fair bit of attention. I've written about some of the things it doesn't discuss on my personal blog, but here, I'd like to use it as a jumping off point for a description of a tool I'd like someone to write for us. Given that my last request produced working code in just a few hours, I'm hoping one of you will wow me again :-)

Who Wants To Build a Faded Example Tool for the IPython Notebook?

Mon, 19 Nov 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

While I'm asking people to write code for me, here's a small addition to the IPython Notebook that someone should be able to knock off in an hour. The starting points are cognitive load theory and the idea of faded examples: long story short, one good way to teach is to show learners one example in detail, then ask them to fill in the ever-larger blanks in a series of similar partly-worked-out examples. Here's one that Ethan White did for our online study group:

Who Wants To Write a Little Code?

Fri, 16 Nov 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We have always steered away from building libraries to use in teaching—we want to show people the "real" stuff, and we can't afford to maintain things. The IPython Notebook has us rethinking that: now that we can display images inline, it would be wonderful if we could leverage Mark Guzdial and Barbara Ericson's work, and teach basic Python using simple image manipulation for examples (rather than text munging). Their research found that using images led to higher retention: more students stuck around, and the ones who did, remembered and used more of what they'd learned.

A Mostly Successful Decade

Wed, 14 Nov 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

A video of Fernando Perez's keynote at PyCon Canada, titled "Science And Python: retrospective of a (mostly) successful decade", has been posted. If you'd like to see what the IPython Notebook can do, and how it fits into a broader picture, it's well worth watching.

{% assign video_title = "Science And Python: retrospective of a (mostly) successful decade" %} {% assign video_slug = "F4rFuIb1Ie4" %} {% assign video_time = "1:00:58" %} {% include youtube %}

Pelican Guts: on content management for Software Carpentry

Thu, 01 Nov 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

This morning I spent some time talking over the content management problem for the Software Carpentry website with Greg. To recap: we'd like to make it easier for anyone to contribute and manage the website content (lecture pages, new bootcamps, events, etc). For example, there are plenty of bits of site building that could be automated so that contributing a new lecture or topic doesn'thave to involve wiring in links to a table of contents, forward and backward links to other lectures/topics, and so on.

Counting to Five (or, A Plan for Online Tutorials and What's Wrong With It)

Fri, 26 Oct 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

My daughter is five and a half years old, and some time in the last couple of weeks, she made a small cognitive breakthrough. It used to be that if I asked her, "What's five plus three?", she would carefully count five fingers, then count three, then go back and count them all together to get eight. She doesn't do that any more; instead, she holds up five fingers, then counts, "Six... seven... eight!" to get the answer. It may sound like a small thing, but it's not: the idea that five is always five, even if you didn't count it this time around, is such a big one that we forget it ever had to be learned.

Rebuilding Redux

Fri, 12 Oct 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The time has come to replace our creaking combination of web tools with something that will let us do more for more people while spending less time switching between browser tabs. To recap an earlier post, we currently use:

WordPress for our blog, web site pages, and comments;
YouTube to host our videos (links to which are embedded in the WordPress pages);
Subversion (administered via the Dreamhost control panel) to manage our slides, code examples, and other files;
Mailman (administered the same way) to manage mailing lists;
a Google Map to show the locations of past and future bootcamps;
a Google Calendar to show their dates;
Google Analytics to track web site traffic;
EventBrite to handle event signup; and
OpenBadges to keep track of who's accomplished what (currently in beta).

We also use Vidyo for video conferencing (primarily tutorials), and run Etherpad beside it so that people have a text channel for asking questions, posting links and code fragments, taking minutes, and so on. As I pointed out in that aforementioned post, this means admins need at least six different logins to get things done, and need to copy a lot of information from place to place. For example, whenever we commit to a new bootcamp, someone needs to:

Record and Playback

Mon, 30 Jul 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

The biggest bottleneck Software Carpentry faces right now is a shortage of experienced instructors. To help fix that, we are going to record a complete presentation of our core two-day material so that people who want to teach it themselves can see how we say things, as well as what we say [1, 2].

As soon as we say "record", though, we have to ask, what exactly are we recording? Audio and video of a presenter in front of a whiteboard? Sure—that helps humanize the presentation. But what about the presenter's desktop? Viewers definitely need to see it, but should they see an MP4 in which the text on the presenter's screen appears as colored pixels arranged in the shapes of characters, or should we record the characters directly? I think the latter is by far the best option, since:

IPython Notebook + Towtruck + Etherpad + Slide Drive = Win

Sun, 22 Jul 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Executive summary: the tool I want for teaching programming doesn't yet exist, but could be built by combining the IPython Notebook, Towtruck, Etherpad, and Slide Drive. This would be a better web-based tool for teaching programming than anything currently available, and would alleviate the most critical bottleneck programming-for-everyone efforts face: the shortage of competent tutors.

A lot of people say harsh things about PowerPoint, but as I've discussed before, the alternatives are all unsatisfying as well. HTML5 slideshows don't allow authors to mix text and graphics free-hand the way people do on a whiteboard; instead, the two have to be segregated into blocks. Screencasts are unsatisfying too: they don't display first-class content [1], and, like slideshows, aren't interactively hackable.

How Robust Is Your Programming Language?

Sat, 21 Jul 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

One of the biggest problems in teaching novices how to program is that most programming systems are not robust. A car can go quite a long distance on a slightly-flat tire, and people can live for years with just one kidney or half a liver, but get one character out of place in a program, and boof—it's game over. And spotting that one character can be very hard, particularly if you're a novice and are learning mostly through copy-paste-and-tweak.

Software Carpentry: The E-Book Version?

Mon, 04 Jun 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

If I could send email two years into the past, I'd tell myself to spend a lot less time making videos for Software Carpentry, and a lot more time exploring interactive formats. One that I've been following recently is the IPython Notebook, a "live" lab notebook that allows scientists to mix, share, and explore text, code, graphs, and the like. Another, which I've just come across through this post on Mark Guzdial's ever-informative blog, is an interactive version of "How to Think Like a Computer Scientist". As Brad Miller says in his description:

What's Wrong With All This?

Sun, 20 May 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Titus Brown doesn't like this web site. He's OK with the content (I think), but he finds it awkward to use, and while I don't feel as strongly as he does, I accept that we have outgrown WordPress. The question is, what should we use instead? We need a lot more than just a blog and some static web pages, but learning management systems like Moodle weren't built with our ad hoc model in mind (they're really teaching administration systems), and newer tools like P2PU feel like a step backward. I started thinking about requirements for a replacement back in April, but got distracted. Here's a longer look.

Reorganizing This Web Site

Tue, 31 Jan 2012 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

It's time to reorganize this web site. Here's my plan; comments would be welcome. In particular, WordPress might not be the right tool to use going forward, but I'm not sure what else would be as easy to set up and maintain.

Overall design: the logo and "Software Carpentry" always appear at the top, along with a Javascript site menu; pages are fixed width (and all of that width is taken up with content—no sidebars); the footer contains a copyright notice and links to half a dozen pages (About, Contact, License, etc.). These footer links are redundant, since those links are in the Javascript menus as well, but this makes them more visible.

Revamping This Site

Wed, 05 Oct 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We have been using WordPress as a content management system (CMS) for Software Carpentry since the launch of Version 4 in July 2010, and while nothing in particular is broken, I'm increasingly dissatisfied with its interface and performance. Looking at other online tutorial sites, there are a lot of things we could add; I'd be grateful for your feedback on which you would actually use, and what we might have missed. Our feature list is:

A New Look

Tue, 14 Jun 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

I'm fond of the Software Carpentry logo, but the blue-to-white color fade is difficult to print on coffee mugs, and impossible to embroider on shirts. Thanks to the talented Veronica Wong, we have a new one:

We'll be converting things over piece by piece as we rebuild the website over the summer.

Fighting Spam

Wed, 26 Jan 2011 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We recently experienced a spate of spam and fake user accounts on the course discussion forums. In this post I'll briefly explain what we're doing to stop that from happening again, and how the technology works.

The course forums are powered by bbPress, the forum software written by the WordPress folks. When I first installed it I left it with the default configuration settings and without doing anything in particular to lock it down from spammers. Bad idea. When I checked this morning we had over fifty fake user accounts, and two spam postings.

Repository URL Change

Sun, 21 Nov 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

We've been having problems hosting the course's Subversion repository at its current URL (http://software-carpentry.org/swc), and so we've moved it to:

http://svn.software-carpentry.org/swc

This URL points to the same repository as before. If you already have a working copy checked out you can switch it to use the new URL by running the following shell command in the top-level folder of your working copy:

A Little Bit of Javascript

Fri, 30 Jul 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As I've mentioned before, one actionable finding in educational research is that faded examples—ones in which progressively less of the solution is shown to students as they progress—are a very effective teaching tool. I've been thinking about how to add them to this course, and have an idea I'd like to try out. It requires more Javascript than I know, though, so I'm hoping someone who reads this blog will be willing to write it for us. (And in general, if anyone wants to help produce material for this course, please get in touch: we're looking for scripts, slides, voiceovers, examples, artwork, and everything else that an open education project needs.)

Popular, Fast, or Usable: Pick One

Wed, 21 Jul 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

One of the core skills in any software developer's toolbox is automating repetitive tasks. We've been teaching this in Software Carpentry with Make for 13 years, and for 13 years, we've been looking for something better: something that has a less user-hostile syntax, requires less arcane knowledge of the shell, and at least pretends to do the same thing on different platforms. We briefly considered Ant (where "briefly" means "as long as it took to find the post by Ant's creator saying he would do it all differently next time"), but it's clear that tools like Rake and SCons, that embed build commands in a full-blown scripting language, are the future of build systems. Rake is based on Ruby, but SCons is based on Python (and actually has its roots in the original Software Carpentry project), so it should be a no-brainer, right?

A Note on Tools

Tue, 20 Jul 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

After a bit of experimenting, I've decided to use SmartSVN as a GUI client for Subversion in the version control lectures, rather than RapidSVN or the command line. My reasoning is:

Requiring people to learn the shell before they can start learning stuff they actually care about raises the drop-out rate noticeably. Requiring them to learn the shell before they can start learning something they don't yet realize they ought to use, like version control, raises it even more. That rules out the Subversion command line tools.
SmartSVN is dual-op (the stripped-down version is free, the pro version is not, neither is open source). In contrast, RapidSVN is pure open source, so it would be my first choice (in fact, it was)...
...except RapidSVN doesn't have a built-in diff/merge tool. They recommend SourceGear's DiffMerge, which is free (that's good) but users have to configure RapidSVN to use it (that's bad—very bad). On the Mac, for example, users have to copy a shell script into /usr/bin, then go into RapidSVN's preferences, set the path to the shell script, and get eight (!) command-line arguments exactly right, or things will fail in strange ways (like you can merge files, but when it tries to save your work, it doubles the dirname of the path to the file and then complains about not being able to create /Users/swc/repo/Users/swc/repo/filename).

If something can break for reasons that its intended users won't be able to understand, diagnose, and fix, then that something is itself fundamentally broken. In my opinion, the possibility of configuration hell with RapidSVN outweighs both its licensing and its support for "file://" URLs (which SmartSVN doesn't allow).

Clip Art

Fri, 16 Jul 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As per your votes, I'm about to start writing the lecture on version control. Our first attempt, two months ago, did a pretty good job of covering the basics, but the slides were downright boring. In particular, we used standard Microsoft clip art to illustrate two people sharing files via a server:

I'd like to use something more entertaining, like these animals from StudioFibonacci:

Setting Up a New Windows Machine

Mon, 03 May 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Setting up a new machine is never fun, but it's always interesting to compare what different people have in their toolkits. Here's what I have installed so far on the desktop machine I'll be using to do Software Carpentry course develpoment:

Windows XP (yes, I still use XP)
Cygwin (a collection of Unix emulation tools for Windows)
Adobe PDF Reader
Audacity (for sound editing)
MikTeX (an all-in-one LaTeX for Windows)
GIMP (the open source clone of Photoshop)
Google Talk (for chatting with my co-conspirators)
Inkscape (an open source vector drawing tool)
iTunes (this course brought to you by Beethoven, John Coltrane, and a variety of 80s bands)
MATLAB (I'm using R2008b)
MWSnap (for doing screen captures)
Microsoft Office 2007 (because most scientists who use anything, use Excel)
OpenOffice (but I'll mostly use MS Office)
Python 2.6 (because some of the packages I want don't exist for Python 3 yet)
R 2.11 (because after Python and MATLAB, R and Perl are our next target languages)
SciPy (which gives me almost everything I want that isn't in the standard Python install)
Skype (for talking to collaborators)
VLC (for viewing video files)
yED (for creating and editing simple bubble-and-arrow diagrams)

What's missing (so far) are the science-specific tools that I'll be adding as needed for particular topics—I'll blog about those as I install them. What's interesting is how many of these tools are free (as in beer) and open (as in modifiable): I'll be blogging soon about the distinction, and why I think both are important to science, as well.

More on Instructional Design

Mon, 12 Apr 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Like many programmers, I've learned most of what I know by poking around and breaking things. Quite naturally, that has led me to believe that this is the best way to learn—after all, if it worked for me, it has to be pretty good, right? But research says otherwise. Kirschner, Sweller, and Clark's paper, "Why Minimal Guidance During Instruction Does Not Work: An Analysis of the Failure of Constructivist, Discovery, Problem-Based, Experiential, and Inquiry-Based Teaching", was published in Educational Psychologist in 2006, but the whole text is available online.

Platforms

Thu, 08 Apr 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

After Thursday's post-mortem on the latest offering of Software Carpentry at the Universitiy of Toronto, I had a chance to talk further with Jon Pipitone, who was one of the tutors (and who is just wrapping up an M.Sc. looking at code quality in climate models). We got onto the topic of infrastructure for Version 4, which needs to be settled quickly.

a way to deliver content to students, including text and images, audio/video, exercises (with solutions), sample data sets, and useful software;
a way for students to feed questions back to the course organizers (asynchronously through email and bulletin boards and/or synchronously through VoIP and desktop sharing);
a way for instructors (who may or may not be contributors) to respond to students;
a way for lay contributors (who may also be students) to offer new content, from pointing out typos to providing exercises or whole new lectures; and
a way for core contributors to manage and edit contributions, create some of their own, et cetera.

This description implies some social infrastructure, including:

Models To Imitate

Thu, 01 Apr 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

My father once told me that a week of hard work can sometimes save you an hour of thought. In that spirit, I've been looking for asynchronous online courses to imitate. I previously mentioned MIT's Open Courseware, CMU's Open Learning Initiative, and (closer to my scale) Saleem Khan's Khan Academy. Google Code University's lessons on programming languages are also on my radar—I'll blog more about them once I finish the Python material—but another model that I'm looking at closely is Teaching Open Source, a collaborative effort to get more open source into college and university courses. I first encountered them through POSSE (Professors' Open Source Summer Experience), which they describe as:

Formats

Tue, 30 Mar 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As I said in last week's announcement, and mentioned again in a later post, one of the main goals of this rewrite is to make it possible for students to do the course when and where they want to. That means recording audio and video, but much of the material will probably still be textual: code samples (obviously), lecture notes (for those who prefer skimming to viewing, or who want to teach the material locally), and exercises will still be words on a virtual page. And even the AV material will (probably) be accompanied by scripts or transcripts, depending on what turns out to work best.

Instructional Design

Fri, 26 Mar 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As well as deciding on the format of the course, I have to re-shape its content. In contrast to e-learning, there seems to be a lot of solid material available on instructional design. The most useful guide I've found so far is Wiggins & McTighe's Understanding by Design. I was initially a bit put off by the micro-industry the authors have built around the book, but its step-by-step approach immediately felt right:

Online Delivery

Fri, 26 Mar 2010 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

As the announcement of Version 4 said, Software Carpentry is being redesigned so that it can be delivered in several ways. I want to support:

traditional classroom lectures, with someone at the front of the room talking over a series of slides and/or coding sessions to a captive audience;
students reading/viewing material on their own time, at their own pace, when and as they need it; and
hybrid models, in which students work through as much as they can on their own, then get help (face-to-face or over the web) when they hit roadblocks.

#1 isn't easy to do well, but the challenges involved are well understood. #2 and #3 are going to be a lot harder: it's new ground for me, and despite the fact that the Internet is older than many of my students, most of the educational establishment still thinks of it as "new" as well.

Presentation, Presentation, Presentation

Thu, 24 Sep 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

Right now, the Software Carpentry material is basically printed pages on the web. Each lecture is a linear HTML page: bullet point follows bullet point, interrupted only by code snippets, tables, and diagrams. If I'm going to update the content, I'd also like to update the presentation; the question is, "To what?" An audio recording of me talking over the slides would add some value, though I think that typing in what I would say would probably be more useful, since most people can read faster than I can speak, and audio still isn't googleable.

Next Steps

Sun, 02 Aug 2009 00:00:00 +0000

This post originally appeared on the Software Carpentry website.

It's clear from Friday's end-of-course review that the course needs shaking up. Before that starts, though, there's a higher-level question to answer: should the course notes be converted to a wiki to encourage contributions from others? It was always my hope that other people would contribute material, but in four years, only five ever have; perhaps wikification would change that.

Right now, the notes are stored as HTML pages in a Subversion repository and compiled by a little Python script to resolve cross-references, insert code samples, and so on. The advantages of this approach are: