Here’s something I’d never thought of: doing a usability study on an API.
Two videos were posted to Channel 9 earlier in the week that show an intro & an actual usability study lab. Here’s the kicker: the lab participant isn’t allowed to use VS .NET, intellisense, help, or the workflow designer… they can only use notepad! Why? Because if the WinFX/WWF team built a killer designer/intellisense/help system, it could be because he API flat out sucked. What you want is a tight API and provide a designer and help system that will let you work with the API in a much more productive setting (IMHO at least).
From the C9 post:
This was an API usability lab where the lab participant was responsible for building a workflow application and providing feedback on the APIs. In the first part of this video, James Conard the evangelist for Windows Workflow Foundation, has a brief chat with several of the people involved in this usability study. In part 2 of this video, you will see the participant (Michael) actually writing the code to use WF and talk about his experience.