Putting the Science in Qualitative Methodology

Roberta Spalter-Roth

Research and Development Department, ASA Executive Office

The "science" in social science is often equated with predictive, quantitative models. But many sociologists do qualitative research. Can this research be scientific? How can it be strengthened and properly evaluated? Participants at a recent workshop held on the Scientific Foundations of Qualitative Research, funded by a National Science Foundation (NSF) grant, agreed that the quantitative and qualitative approaches were significantly different, but that qualitative research played an important role in the understanding of social structures and processes, mainly answering questions about how social processes work.

The purposes of the workshop were threefold. First, provide a primer for those submitting NSF grant proposals as to how to make qualitative projects competitive in NSF peer review. Second, provide guidance to reviewers concerning the characteristics of strong qualitative research. And third, provide a training manual. A report summarizing the workshop outcome was co-authored by Charles Ragin of Northwestern University; Joane Nagel of the University of Kansas and former program officer rotator in NSF's Sociology Program; and Patricia White, the Program Officer in NSF's Sociology Program. The full report can be accessed at www.nsf.gov/pubs/2004/nsf04219
/start.htm.

Developing Standards

According to Nagel, "the workshop grew out of the interest of NSF sociology program officers as well as NSF panelists who felt unclear as to how to review qualitative proposals. There was general agreement about the lack of uniform standards for evaluation."

Although, many sociologists avoid applying for NSF grants because they believe that only quantitative research is funded, in fact, about one-fourth of regular grants and one-third to one-half of dissertation grants in sociology fund qualitative research. Sociology program officers have noted that more and more of the dissertation grants use qualitative methods.

The workshop brought together scholars, who work in qualitative research, with the idea of contributing to building this aspect of sociological inquiry as a discipline. Although there was some disagreement about methods among the panelists that can be seen in the papers appended to the workshop summary, there also were areas of convergence. According to Nagel, there was strong agreement that because NSF funds theoretically driven, rigorous research, qualitative proposals should not be submitted at an early stage before anything is known about the topic, and, if possible, reliance on "grounded theory" should be avoided. The project should be located in a theoretical context. Investigators should describe how the research will contribute to theory, if successful, and, they should describe how the researcher's ideas or hunches could be disconfirmed, if not successful.

Among the guidelines to investigators in the Executive Summary are the following recommendations:

  • Write clearly and engagingly for a broad audience;
  • Situate the research in relation to existing theory;
  • Locate the research in the relevant literature;
  • Articulate the potential theoretical contribution of the research;
  • Outline clearly the research procedures;
  • Provide evidence of the project's feasibility;
  • Discuss the plan for data analysis;
  • Describe the strategy to refine the concepts and construct theory;
  • Include plans to look for and interpret disconfirming evidence;
  • Assess the possible impact of the researcher's presence and biography;
  • Provide information about research replicability; and
  • Describe plan to archive the data.

These criteria should not be limited to NSF applications, but rather to sociology as a discipline. The report also recommended ways to strengthen qualitative research including proposing qualitative methods training, holding additional workshops on qualitative research methods, and disseminating qualitative proposal review criteria.

The Sociology Program at NSF has taken some of its own advice. Along with the Political Science, Anthropology, Law and Social Science, and the Methodology and Statistics programs are co-funding a follow-up workshop to be held in mid-May, with sociologist Michelle Lamont as the Principal Investigator. In addition, the Sociology Program at NSF seeks rigorous qualitative research proposals to advance this area of sociological research. Full proposals are due by May 16, 2005. More information can be found on the NSF website at www.nsf.gov/sbe/ses/soc/sqrmii.jsp.

Summing up the purpose of this effort, White said, "The Sociology Program appreciates diversity in substantive focus and research methods. We hope that the workshop and follow-up activities on qualitative methods will communicate that we are interested in supporting strong qualitative research projects and want everyone to be on the same page as to what constitutes a high quality project for NSF."

 

 

From: S palter-Ross, Roberta. " Putting the Science in Qualitative Methodology". Footnotes. American Sociological Association. Washington, D.C. March 2005.