Sample Size and Qualitative Research

The Great Wave off Kanagawa (Photo credit: Wikipedia)

Someone had to do it.  Do what? Make a definitive answer to what the sample size should be in a qualitative research study published by a peer reviewed journal.

In this case the someone was Shari L. Dworkin, Associate Editor of the Archives of Sexual Behavior, the individual responsible for qualitative research submissions at this journal.  She takes her stand in an editorial (2012) 41:1319-1320.  Henceforth, the policy of her journal will be to recommend that:

25-30 participants is the minimum sample size required to reach saturation and redundancy in grounded theory studies that use in-depth interviews. 

A part of me admires her for putting a number to her answer.  I, too, have struggled many times when people ask me that question--"How many interviews does it take?" "What's enough data?"  "How do qualitative researchers figure out what is the right sample size?"  The answer that most qualitative researchers give:  "It depends," is not very satisfying for the novice or the researcher who is struggling to gain an understanding of this methodology. How easy it would be just to say "25-30; take it or leave it". 

In her editorial, she does a good job of talking through the rationale for her answer and discussing the literature upon which she drew for her answer (the primary grounded theory gurus).  She identifies this number as about right if you are seeking to find saturation...no new leads...not much new coming up. 

Just as I admire her...I also fear for her, because I worry that, in making her editorial decision, there is a lot of qualitative research territory that she hasn't taken into consideration.  So, let me unpack what some of the "it depends" concerns would be:

• Grounded theory studies are not the whole of qualitative research, and in relying so heavily on this one "flavor" in our field have important views been ignored.  (Now I personally believe that the whole qualitative research flavor discussion is over-rated...see earlier blog posts...but I think it may be relevant here.)
• Grounded theory studies are not always solely composed of interviews (as the editorial appears to suggest).  I would want to know how interviews are mixed with other forms of data before I could say that, for instance, 10 interviews, was insufficient.  [How many observations?  Of what kind and length?  Was there significant document analysis?  Was visual data obtained?  What about the use of journals from the participants?]  I would need to know how the interviews were contextualized in the sea of data that is possible to gather in a qualitative research study. 
• What do you mean by "in-depth interviews"?  From the editorial, I have the sense we are talking one interview per individual (maybe 1 hour?).  In my training with Buddy Peshkin...in-depth would have meant 4-5 interviews (yes 4-5 hours divided up over a period of time) with one individual.  Whose in-depth are we using? 
• What does 25-30 mean?  In the editorial it appears that this would be 25-30 interviews with separate individuals.  If you had 5 focus group interviews with 5 individuals in each (5 x 5 = 25)...would that count? I am working on a study that relied primarily on focus groups--I can report that we interviewed hundreds...but in groups. 
• What will a ruling like this do to the incredibly important smaller exploratory start-up study, and/or the dissertation study?  These are often of smaller size than a very well funded study by a senior researcher.  Will these valuable efforts become unpublishable?  Are we starting a new kind of ranking order for peer reviewed journals?  

When I shared my concerns about the policy described here with a friend who is a qualitative researcher, her response was--"...setting a number of interviews seems to me like giving up the expert's responsibility to vet the methodology.  Maybe the problem is that the methodology wasn't adequately described.  Shouldn't we review the number of interviews as part of reviewing the methodological description.  We still have that responsibility."  

I invite all comment on this issue.  This is the first time I have seen a journal take a stand of this sort.  I am glad that this has happened, because it opens up space for a well needed discussion. 

Jay Lang and Qualitative Research

Jay Lang elected to discuss the chapter on Grounded Theory from The Sage Handbook of Qualitative Research.  Here is his written and visual introduction to the chapter.

Lang's image is simple and startling--an upside down house suggesting the inductive approach to inquiry.  It made us stop, take note, and consider the notion of induction from a new vantage point. 

Grounded Theory Exposed

In the most recent issue of FQS, also known as Forum: Qualitative Social Research, there is a wonderful article by Edward Tolhurst titled "Grounded Theory Method: Sociology's Quest for Exclusive Items of Inquiry".  (volume 13, #3, Art 26, September 2012:  http://www.qualitative-research.net/) 

Tolhurst's piece speaks strongly to an issue that I wrote about earlier in this blog (but much less eloquently) re: what I referred to as the "ism's" of qualitative research. 

I think this article will become the "go to" reference for those of us who want to write our way out of the confines of the earlier generation of research strategies. 

The New Research Paradign is NOT Mixed Methods: It's Inclusion

Image via WikipediaIt's strange to wake up and think, "Oh, I've had a blog."  But that is what happened this morning.

I was giving thought to the wonderful team I am working with on the Sexting and Teens project...and that led me to thinking about how it was different in nature and quality from the structure and qualities of so many other research teams...like various kinds of research projects that try to 'work across the aisle' (as in mixed methods)...which led me to think about dissertation committees...and then that led me to thinking about  the work I have done over the past two years as an Associate at the UML Center for Women and Work...and now as a participant in their new Emergent Scholars program for undergraduates. 

What it brings me to is that I think in addition to all of the other revolutions our world is engaged in, we are also undergoing the throes of a paradigm shift in the ways we consider research methodology.  We are shifting from the old reality in which single disciplines dominated, hand-in-hand with single methodological approaches.  I find these studies to be, what I would call, descriptively based.  This is the world in which the separate divisions of qualitative research grew up--grounded theory, case study, ethnography, symbolic interactionism.

The new paradigm of research is highly interdisciplinary in subject matter, and highly integrative in methodology.   It is grounded in complexity and is what, I would call, as problem-based.  It requires far more community to find the answers--a community of researchers working in and with communities of users/experiencers.  It is driven by the factors that are driving so much of world change:  globalization, new technologies, new forms of visualization. 

Currently, we are passing through a hybrid stage that we refer to as "Mixed Methods".  It mixes the best and worst of both worlds. 

I think one dark side of the old paradigm is that it leaned toward the exclusionary--exclusion of other disciplines, exclusion of other methods, purism in methodological perspectives.  In this world, you were valued when you are pure.  You were valued when surrounded by others who are just like you.  When you leave those enclaves you needed to be wary...which leads to be prickly.  Micro and macro discrimination were prevelant around methodology. 

The light side of the new paradigm as I have experienced it is inclusion.  As a qualitative researcher, I feel included, honored, and valued.  I can be part of research discussions addressing a range of topics with a range of methodologies and I feel that my methodological approach is valued.  I can be expert in what I am expert in, and don't have to prove expertise in every other kind of methodology because to solve the problem, understand the phenomenon we all have to pitch in and bring what we have.  

This is not to say that single discipline, single method studies are suddenly useless or to be denigrated, but the ground is shifting under our feet and as research methodologists we will need to be able to enter these new environments.  I like the fact that I can be expert (a qualitative research) and be valued in an interdisciplinary group.  I like the fact that in these new environments the various kinds of blatent and less blatent discrimination that has been practiced against my kind is not considered acceptable. 


But this creates a dilemma...because now I also have to come up to the mark myself and practice inclusion, not exclusion.  This is a dirty secret that qualitative researchers would rather not say much about, but it is also there. 


Many, many thanks to the members of these research teams/groups/experiences that are inclusively moving forward. 

Memos and Qualitative Research: Kathy Charmaz

Image by darkmatter via Flickr

Since I started this blog I've written several entries about visual memos.  For some time I've meant to look at the discussion of memos/memoing in texts that I consider classics in this area to see how these descriptions might help me understand what I am trying to get at with the visual memo.  So here are some of my favorites on memoing:

Charmaz, K.(2006).   Constructing grounded theory: A practical guide through qualitative analysis.  Thousand Oaks, CA:  Sage Publications.

Emerson, R., Fretz,R., Shaw, L. (1995).  Writing ethnographic fieldnotes.  University of Chicago Press. 


Maxwell, J. (2005).  Qualitative research design:  An interactive approach (2nd ed.).  Thousand Oaks, CA: Sage Publications. 


Miles, M. & Huberman, A.M. (1994).  Qualitative data analysis: An expanded sourcebook.  Thousand Oaks, CA:  Sage Publications.


(Oops...just realized I left out one of my favorites by Anslem Strauss--I'll catch up with that one later.)

Today I am going to write about one of these:  Kathy Charmaz's discussion of memos. 


"Memo-writing" is the title of Chapter 4 in Charmaz's book.  It is an elegant description of memos in grounded theory.  It will really give you shivers. 

"Memos catch your thoughts, capture the comparisons and connections you make, and crystallize questions and directions you want to purse."  (p. 72)

Here are some of the functions memos fill for her:

• "conversing with yourself" (p. 72)
• "explicate and fill out categories" (p72) 
• "serve as the analytic core" for subsequent writing (p.76)

Her primary advice in creating memos is to "do what works for you" (p. 80) and "...engage a category, let your mind rove freely in, around, under, and from the category..." (p. 81)

Charmaz explores the ways the techniques that writers of many sorts employ to get themselves off the ground--clustering, freewriting, etc.--can all serve the qualitative researcher at the stage of memo writing.

She gives special attention to "Using memos to raise focused codes to conceptual categories" (91)  This seems to me key in thinking about the ways memos can assist in the refinement process, that is, the process of making a clump of data into something called evidence--that's like spinning flax into gold. 

One thing that cannot be ignored in talking about memos and qualitative research is that the discussion of memos is always right in the middle, squished between data collection and coding on one side...and products, representations, and presentations on the other side.  Memos, in the written textual mode, are always about refinement of raw data and leveraging preliminary interpretations.  As Charmaz says, the construction of the memo is the "pivotal intermediate step between data collection and writing drafts of papers" (72). 

For the novice researcher, I think that the notion of memos is hidden behind flashy things that catch their attention like coding or interviewing--things that stand out as if they have neon lights on them.  Memos, for the qualitative research connossieur, however, are like a fine wine.  They are swished around in the mouth and savored!  The tastes are subtle and learned.

I think I will stop here with Charmaz and take on another one of the classic memoists next.