Monday, August 09, 2004

What makes for good research?

Claim 1: Any strong paper should possess at least some of the following characteristics:
  • Solve a significant/important problem (such as being of practical relevance, having a broad impact, solving a long-standing problem)
  • Come up with a significant conjecture (whose solution would be important)
  • Use a sophisticated/surprising solution method
  • Say something non-obvious/counter-intuitive
  • Correct a popular misconception
  • Shed new light/initiate a new line of research
  • Demonstrate significant improvement over state of the art
  • Make a valuable contribution to further research (e.g. present new metrics/models, provide new experimental data, present observations that clearly merit further study)
  • Present convincing results (e.g. by validating simplified analysis with realistic experiments)

Claim 2: the following features are must-haves in a good paper:

  • Demonstrate thoroughness and effort
  • Be technically rigorous
  • Present material in a clear, compelling manner
It would be nice to have some kind of a checklist like the above when writing a paper, to gauge its quality for oneself before submitting it for external review...

Tuesday, August 03, 2004

Building Knowledge, Together

"In the long history of humankind (and animalkind, too) those who learned to collaborate and improvise most effectively have prevailed." - Charles Darwin

Even a casual glance over my publication list will reveal that I am a great believer in both intra and inter-disciplinary research collaborations. In my first two years as an assistant professor, I have actively developed collaborations with a number of faculty members in several departments/institutes at USC: Electrical Engineering, Computer Science, Information Sciences Institute, Industrial and Systems Engineering. My collaborators span a wide range of disciplines and perspectives including experimental networking, signal processing, optimization, data management, distributed computing, and algorithms. My own research focus is on applying theoretical techniques (including modeling, performance analysis, and algorithms) for practical problems in sensor networks.

In nearly every case, I have found that the complementary nature of interests between my collaborators and me has yielded interesting new problems and helped to bring relevant techniques to bear on such problems. For instance working with more experimental colleagues I am able to understand practical considerations and determine what empirical observations need to be better understood through modeling and analysis. At other times I help translate practical problems into a formal setting so that I can work with colleagues who have expertise in particular relevant theoretical tools (e.g. estimation theory/randomized graphs/network flow optimization) to develop solutions for them.

Such collaborations are becoming increasingly fruitful because they bridge the gap between theory and practice, between independent yet related disciplines. Indeed, in recent years, the National Science Foundation (NSF) as well as other funding agencies have expressed a clear preference for funding collaborative research.

This may all be well and good, but I am now having second thoughts that my strong focus on collaborative research runs counter to a conventional "bean-counting" perspective to evaluating scholarship. Come tenure time, how can the individual contributions of a faculty member like myself be assessed "quantitatively" if most papers are written in collaboration with others? One base assumption in this perspective is that all papers have the same "value." Under this assumption, it follows that the individual credit for a genuinely significant collaborative paper with 3 authors (say with one student and two senior collaborating faculty members) is worth less than that for a paper with 2 authors (say just the student and his advisor).

The alternative perspective I would advocate strongly, but that I genuinely fear is not widely accepted in the academic world today, is expressed beautifully by Keith Dorwick in a thoughtful essay titled "The Ways We Build Knowledge." Although Dorwick's focus is on literary fields, the following comments from his essay apply equally well to engineering disciplines:

"Currently, there is only one dominant model for the creation of knowledge that we call research -- that model is a solitary one in which, generally speaking and especially among literary critics, scholarship is seen as a personal possession, one that is owned by the person who has done the work necessary to write the books and journals that bear her name and that, for those who find themselves in the professoriate, is not just a creative expression of one's talents, but the means necessary to keep one's job in a market that is only now possibly beginning to open up.

In this model, collaboration is seen as a problem, not an opportunity -- since both initial employment and tenure depend on the production of scholarly articles that are disseminated in peer-reviewed journals and books that are published by university presses that also depend on peer reviews, it is very necessary to know exactly who did what work. Collaboration becomes nothing less than an administrative problem -- how can tenure and promotion committees and deans, for instance, know whether or not to promote a local candidate who has spent much of her career in collaboration. The problem is simple, from this viewpoint: who owns the work, and who, therefore, should benefit from it?

Of course, the simple answer is this: "we did it together." A collaboration ought to be judged as the collective work of the individuals involved and tenure and promotion committees ought to see good work as something of which the department, college, and university can be proud. In fact, of course, as anyone who has worked in a good collaboration knows, the fact is that the work is often the stronger for being the product of two or more people... "