One of ten papers you should read

Michael Fogus's blog posting "10 Technical Papers Every Programmer Should Read (At Least Twice)" includes my 1998 paper "Predicate Dispatch: A Unified Theory of Dispatch".

This was joint work with Craig Kaplan and Craig Chambers (who has two papers on the list!).

PhD Working Group at ESEC/FSE 2011

The PhD Working Group at ESEC/FSE 2011 had an admirable goal: students would learn about current trends in software engineering research and summarize the results to the rest of the attendees, and throughout the process students would interact with more senior researchers. Unfortunately, it was organized in such a way as to benefit neither students nor researchers. On the plus side, none of my students took part in this fruitless exercise.

I'll summarize what I saw of the PhD Working Group and offer some suggestions that might have produced better results.

Students were split into 7 working groups, each with its own topic such as “Agile Development”. Each group was assigned to learn about this topic from conference attendees. So far, so good.

The students presented a multiple-choice survey form and collected the answers — or just asked attendees to take the survey online, but the attendee invariably forgot to do so. So, the poor students would circle like hungry sharks, asking for survey participants and even interrupting conversations, and I saw some attendees trying to avoid anyone who looked like they might ask survey questions. This made the relationship between junior and senior researchers antagonistic, which prevented rather than encouraged conversations (not to mention the time wasted with the surveys, which could have been spent on meaningful communication instead).

A multiple-choice survey that is taken by both people expert and ignorant of the topic — and the students emphasized that they wanted both types of opinions — conveys nothing of value about current and future research directions. Popularity polls may be favored by the evening news when they do not want to do real reporting, but even there I don't see any value. I would rather understand the justification for a particular conclusion than just see that 27% of people agree with it. I hope none of the students came away thinking that a public poll is a valid methodology to learn about software engineering research. As was predictable, the student presentations in a plenary session were a waste of time.

Another serious problem was ambiguous and nonsensical questions on the survey form. I completed several surveys, but for one survey I gave up in the middle. It was full of questions with answers that were non sequiturs (they had nothing to do with the question), or that omitted choices that would be preferred by any expert, or that I couldn't interpret at all. For the most successful survey I took, the student interpreted the questions and I dictated my answers, rather than me working alone ticking off the multiple choice boxes. In fact, on several occasions the student changed my answers when I remarked that the question didn't make sense — the student said that my proposed alternative question is what they had meant to say. So much for the answers meaning anything. I conclude that the only redeeming result of the entire exercise is that a bright and thoughtful student might have learned something about how not to do questionnaire design; but proper questionnaire design should have been taught from the beginning.

As I mentioned, the sentiment behind the PhD Working Group is a noble one. Here is a different way it could have been run instead, which would have avoided some of the pitfalls that befell it this time around. The organizers could have given each group a list of 5 or so researchers at the conference who were expert in that area. The students would interview those people for 30 minutes or so — no one would get interviewed on more than one topic — and the group would evaluate and synthesize the responses, including adding their own opinions or justifications. With this design, the students have meaningful interactions with senior researchers, the students learn something, they provide a summary from which others might learn something, and everyone spends less time, and is interrupted less, than with the present model. There may be flaws in this approach, too — feel free to discuss them, and how to correct them, in the comments to this blog posting.

Document mark-up and correction with voice recognition

I spend a lot of my time commenting on document drafts. Traditionally, I do this with a red pen, and I hand the marked-up copy back to the author.

The traditional approach works well, for several reasons:

• Marking up with a pen gives great flexibility to draw pictures and to relate chunks of text with freehand arrows.
  
• It is easy and natural to flip among pages and to amend previous comments.
  
• There is no need to be connected to a computer, so it does not contribute to my hand and eye strain.

The traditional approach also has some disadvantages:

• When my collaborator doesn't work in my building, I have to send the comments by postal mail, or else scan them in color and email the scan, but the scanned version is invariably much harder to read than the hardcopy.
  
• Giving comments to multiple people on a collaborative project requires photocopies/scans, or else sharing a single hardcopy.
  
• My handwriting is sometimes hard to read.

I still frequently use the traditional approach, but I also sometimes give back comments electronically, using voice recognition.

I load a PDF onto my computer, point to some text of interest, and speak my comments about that text. My comments are transcribed into text annotations in the PDF, which I can email back to the author.

Because I use a tablet computer and a stylus, I can do all this while reclining on my couch, which saves me the eye and hand strain of sitting at the computer and typing. For long comments, this approach is considerably faster than typing, even accounting for correcting occasional speech recognition mistakes. For shorter comments, it's about the same speed, but the greater comfort, and the convenience of the electronic form, makes it well worth doing. It has improved an activity that I spend many hours on each week.

If you haven't used voice recognition recently, you owe it to yourself to give it another try. I was really impressed with the accuracy, especially compared to even a few years ago.

There are three key components to my setup:

1. A tablet computer with a stylus

I use a ThinkPad X61s, though this is an older model which has since been replaced by newer ones.

You want a real computer with a decent CPU, not a “slate computer” or “tablet” such as the iPad and its rivals. The reason is that voice recognition software is extremely CPU-intensive, and your computer will be going all-out to provide you accurate speech recognition.

My setup would work with any laptop/notebook computer, not just a tablet computer, but I love being able to get away from my desk and change my posture.

2. Dragon NaturallySpeaking

Dragon's products is so dominant — and so good! — that there isn't much competition in this product space. I tried to find a usable speech recognition program that would work under Linux, but there doesn't seem to be one.

Microsoft Windows 7 comes with built-in speech recognition that is pretty good. However, it is not quite as good as Dragon NaturallySpeaking. More importantly for me, the Windows 7 built-in speech recognition is not able to type into all text boxes in all applications, including pop-ups in Acrobat Professional.

Thanks to this new competition, NaturallySpeaking retails for only $200 ($100 for the “home” version, which I have not tried), though you can find it even cheaper and there is also a 50% academic discount. It's well worth the price.

Interestingly, NaturallySpeaking works better the faster you talk.  That is, it works best when you speak in full sentences, without pauses in the middle of a sentence.  The reason is that it uses context to determine what word you meant to say.  When marking up with a pen, I typically write one part of a sentence at a time and think about the best way to convey the rest of my thought.  This still works, but you may end up doing a bit more correction of the voice recognition than you would if you didn't use the pauses.

3. Adobe Acrobat Professional

When you select text, Acrobat lets you apply annotations/comments/markup to it, which highlights the text and associates a pop-up note with it. You can choose among different types of annotations: a comment, replacement text, crossing out, etc. Here are examples of how it looks:

After you have added your annotations, you can save the file and send it to your colleague. Your colleague can then click on each highlighted snippet of text to see your comments. This is a bit of a pain, because they have to be in front of a computer, have to click on each one individually, and can easily forget which ones they have already read. Some people I work with like this format, but most do not. Therefore, I always send my comments in two formats: both the original PDF, and also what Acrobat calls the “comments summary”. This is a format, suitable for printing, in which the original document is displayed along with a list of comments.

Acrobat 8 and 9 have a menu item “Comments > Summarize comments”, which offers 4 ways to summarize comments, including the one in the linked images (which is my favorite). Acrobat X has lesser functionality, and that functionality is harder to find: there is only one way to create the comments summary, and it appears as the “Summarize comments” button in the “File > Print” dialog box. Adobe has documentation about printing a comment summary.

I tried using free or cheaper products, such as those from Foxit, but their text annotation features are much more limited. They didn't have as many types of annotations, were less versatile, were visually uglier, and, most importantly, didn't seem to have functionality analogous to Acrobat's “summarize comments”.

The best exercise for mountain climbing

I used to teach mountain climbing. People would ask, "What exercise should I do to strengthen myself for climbing?" There was only one answer: climbing itself. Other exercises (including going to the climbing gym) can help some, but don't give anything like the same benefit.

The same advice applies to programming, research, or any other activity. Go do it, and thoughtfully learn from your successes and failures, and you will get better at it.