Qualitative Research Methodology discussions in Non-Methodology Journals

Vraagteken (Photo credit: Wikipedia)

The semester is over and I am slowly picking up the pieces.  In re: the classes that are now behind me, one of the most important tasks is asking myself what worked, what did not work, and why.  What could I have done differently?  What assumptions did I bring to the assignment that were or were not fulfilled? 

One of these question marks was the last assignment for the doctoral qualitative research course.  I asked each class member (there were 12) to select an article describing a qualitative research study related to their research passion.  These passions ranged from lesson planning and the use of social media in community colleges to teaching logrhythims and universal design as a tool for working with troubled youth. 

I provided a chart of qualitative research issues to address in the analysis of the article.  All were representative of the issues raised in the textbook and readings--in other words, they represented the standard fare of methodologists.

Full APA Bibliographic Reference

Background/Problem:  What is the problem or dilemma faced here?

Question Article seeks to answer

Paradigmatic Stance/Research Strategy Used by Author

Ethical Processes and Issues discussed:  Subjectivity Concerns  Discussed

Data Collection:  What was collected/How/When/Where

Data Analysis:  What was the process of analysis described?

What were the key findings?

How is evidence marshalled to support findings?

How does the author account for validity or trustworthiness?

Quality of the analytic discussion? (How does the author probe findings? Question them? Offer challenges?)

Issues of representation

Significance:  What is the significance of this piece?  How does it connect to the discussion of the background or problem?

In the week's before the assignment came in, I had questions from several students asking about the articles they had chosen.  Several students complained that they couldn't find qualitative research studies in their area of interest.  These and others complained that the articles they found didn't talk about the methodological issues we had discussed in class.  Many said they were having to do a work around, that is, they were writing about the absence of a variety of methodological proofs.  I chalked up the questions to their own jitters and assumed things would be straightened out by the time I read the assignments.  

So, now that the grading is behind me:  What did I learn? 

Well, first I learned that I live in a methodological bubble.  I live, work, and breath in a world dominated by qualitative research methodology discussions.  I think that every researcher is with me under the same big tent.  However, this is not the case.  There are many researchers out there, very happily pursuing their various non-methodological interests.  These people make use of qualitative research to meet their ends, but they are not as worried about crossing their t's and dotting their i's.  If you know what I mean. 

This means that much of the qualitative research that is out there, reported upon, in various non-methodological journals, is doing its best to contribute to the understanding of a given topic, but it may not be representative of all the whistles, bells, and frills, for which the methodological purists are looking.

My students (all of whom were reporting on qualitative research studies published in peer reviewed journals) found that discussions of methodological tasks and steps were often subsumed under a simple label.  Grounded theory was the most popular label, although that title could cover a range of meanings.  Overall the mention of research strategies (as I would call the ways we refer to approaches like Grounded Theory, ethnography, etc.) got little space. 

I had asked them to look for information about the paradigm discussions we had read about in depth in the Handbook of Qualitative Inquiry.  No one found a word about that in the articles they had chosen.  So they wrote about the absence of paradigms and the implied paradigmatic stance of the author. 

The actual act of analysis was often collapsed into a one sentence description.  "Back and forth," meaning movement between an interview transcript and coding, was a frequent way to talk about analysis--so frequent that I got tired of writing:  "What does this 'back and forth' mean?!"   

Only one student had selected an article from one of the tried and true qualitative research methodology journals, and she alone could fill in the blanks for the assignment with close to the purity for which I had hoped.   For her, it was easy, because everything was laid out based on the template I had given her.  For the others, however, the task was not nearly as clear.  Clearly, she was not the only one who had done it right.  The issue was much larger. 

What should I do? 

Hating failure, my first thought was--I'll never give this assignment again.  But this was reactionary, and, in truth, they had actually done a very interesting job on the assignment. 

My second thought was:  I will assign all the articles in future and they will only have articles in their hands that represent the way qualitative research methodology SHOULD be described.  But that didn't seem to make great sense because from what I had learned from this assignment--that's not going to be what they enounter in the real world.  It's obvious that the template created by methodologists lives and breaths in our world, but is getting limited attention from those further from this limited group of gurus. 

My third thought was:  Let's do some more shared/guided reading of articles with strong and weak methodological descriptions and talk about the difference.  This will probably be where I end up. 

The challenge of working through the "Bringing it all Together" assignment, as I called it, is similar to the earlier blog mention I made of the journal that mandated a number of interviews as representing good methodological practice.  Lacking an adequate (read: informed) discussion of the qualitative research methodology in articles submitted to peer-reviewed journals, such mandates may seem like a reasonable response.  It seems that the other response--requiring good methodological description is much harder because writers, reviewers, and editors may lack the skills to provide this or shepherd these changes. 

So, should we stop permitting qualitative research results to be reported?  Is it too much trouble to get it right?  Or should we stop permitting their publication without a full methodological discussion?  What does a "full methodological discussion" mean?  Is "full" too long to be fit into the average peer-reviewed journal article?  What kinds of methodological short-hand is acceptable in reporting on qualitative research methodology? 

I think it is time to eat breakfast--there is no quick fix for this one. 

Action Research Journal Blog

Wassily Kandinsky, "On White II", 1923 (Photo credit: Wikipedia)

I am going to be teaching Practitioner Action Research, Spring 2013, and in preparation, I signed up for alerts from Sage's Action Research Journal.  Just discovered they have an interesting blog with extended content and commentary. 

Action Research Journal Blog

Take a look--you might find it interesting. 

Maxqda 11 Introduced

Simple emoticons of the five temperaments: Sanguine (top right), Choleric (bottom right), Melancholy (bottom left), and Phlegmatic (centre), with the new temperament Supine (top left) and Phlegmatic blends in between. (Photo credit: Wikipedia)

Exciting!  I just took a look at the new features of Maxqda 11.  It is definitely running neck and neck with what NVvivo is offering, BUT here are two new features that are very different:

1.  A cell phone app!!!  They are right--this is a first for the larger comprehensive qualitative data analysis software packages (There are apps, but not produced by these companies to interface with their products). 

2.  Emoticon coding!!!  I like this idea too.  Just as you can define a color for a coding stripe in some packages...now Maxqda lets you assign an emoticon as a quick visual reference for a code. 

Here is a link to the introductory video.  I am going to have to try this out. 


Introduction to Maxqda 11

Emerging Scholars at UMass Lowell

View of University of Massachusetts Lowell, Lowell, Massachusetts, USA. (Photo credit: Wikipedia)

For the past two years, UMass Lowell has been engaged in a wonderful undergraduate research project called "Emerging Scholars".  Conceived of by the Center for Women and Work, Emerging Scholars are a hand-picked group, selected from our top undergraduates.  Each one is assigned to work with a researcher who has submitted an application.  The match is made...and the results are great.

The Sexting project is in its second year.  Andy Harris from the Criminial Justice Program and I are sharing the joys of working with this year's Emerging Scholar for our project--Deborah Paul.  Follow this link to see the three of us at the mid-year get together, where the Emerging Scholars share their progress with the various researchers and other campus visitors. 


Emerging Scholar Presentations December 2012

It is always very exciting to hear your own Emerging Scholar describe the semester's work, but also very interesting to hear about what everyone else is doing.  Deborah has taken on independent analysis of a cache of data from the project...and she quickly became an NVivo whiz.  She is definitely going places.  We are pleased that she was chosen as our Emerging Scholar for 2012-2013. 

The Sage Handbook of Qualitative Research...riffs on articles

In my qualitative research course (07.704--Introduction to Qualitative Research Methods), students have just completed an assignment in The Sage Handbook of Qualitative Research (4th edition, eds. Norman Denzin & Yvonna Lincoln).  They read a self selected article in the area of strategies and paradigms and each created a one-page visual overview of that article. 

I was so taken with these and the unique ways that students depicted the information that I decided to publish their pieces here in my blog.  Each one provides a brief introduction, which I will post with their one-pager. 

Coming up....

Center for Women and Work at UMass Lowell: Another Great Holiday Fair

It was another great success--the annual holiday fair of the Center for Women and Work at UMass Lowell.  Thursday November 29 from 2-7 pm at the Inn and Conference Center--you had to be there or be square. 

For the second year, I was invited to serve as the official MC (translation: person who introduces musicians and urges everyone to shop more).  I know why I was asked--I have the right hat!  The Jester's hat!

And--here I am modeling the silent auction item I won--a crocheted piece by psychology professor Sarah Kuhn. 

This event has now become a holiday time feature on the campus--it celebrates the work of campus women, raises funds for the Center for Women and Work, and is a great time for all!  Don't miss next year's event. 

Sample Size and Qualitative Research

The Great Wave off Kanagawa (Photo credit: Wikipedia)

Someone had to do it.  Do what? Make a definitive answer to what the sample size should be in a qualitative research study published by a peer reviewed journal.

In this case the someone was Shari L. Dworkin, Associate Editor of the Archives of Sexual Behavior, the individual responsible for qualitative research submissions at this journal.  She takes her stand in an editorial (2012) 41:1319-1320.  Henceforth, the policy of her journal will be to recommend that:

25-30 participants is the minimum sample size required to reach saturation and redundancy in grounded theory studies that use in-depth interviews. 

A part of me admires her for putting a number to her answer.  I, too, have struggled many times when people ask me that question--"How many interviews does it take?" "What's enough data?"  "How do qualitative researchers figure out what is the right sample size?"  The answer that most qualitative researchers give:  "It depends," is not very satisfying for the novice or the researcher who is struggling to gain an understanding of this methodology. How easy it would be just to say "25-30; take it or leave it". 

In her editorial, she does a good job of talking through the rationale for her answer and discussing the literature upon which she drew for her answer (the primary grounded theory gurus).  She identifies this number as about right if you are seeking to find saturation...no new leads...not much new coming up. 

Just as I admire her...I also fear for her, because I worry that, in making her editorial decision, there is a lot of qualitative research territory that she hasn't taken into consideration.  So, let me unpack what some of the "it depends" concerns would be:

• Grounded theory studies are not the whole of qualitative research, and in relying so heavily on this one "flavor" in our field have important views been ignored.  (Now I personally believe that the whole qualitative research flavor discussion is over-rated...see earlier blog posts...but I think it may be relevant here.)
• Grounded theory studies are not always solely composed of interviews (as the editorial appears to suggest).  I would want to know how interviews are mixed with other forms of data before I could say that, for instance, 10 interviews, was insufficient.  [How many observations?  Of what kind and length?  Was there significant document analysis?  Was visual data obtained?  What about the use of journals from the participants?]  I would need to know how the interviews were contextualized in the sea of data that is possible to gather in a qualitative research study. 
• What do you mean by "in-depth interviews"?  From the editorial, I have the sense we are talking one interview per individual (maybe 1 hour?).  In my training with Buddy Peshkin...in-depth would have meant 4-5 interviews (yes 4-5 hours divided up over a period of time) with one individual.  Whose in-depth are we using? 
• What does 25-30 mean?  In the editorial it appears that this would be 25-30 interviews with separate individuals.  If you had 5 focus group interviews with 5 individuals in each (5 x 5 = 25)...would that count? I am working on a study that relied primarily on focus groups--I can report that we interviewed hundreds...but in groups. 
• What will a ruling like this do to the incredibly important smaller exploratory start-up study, and/or the dissertation study?  These are often of smaller size than a very well funded study by a senior researcher.  Will these valuable efforts become unpublishable?  Are we starting a new kind of ranking order for peer reviewed journals?  

When I shared my concerns about the policy described here with a friend who is a qualitative researcher, her response was--"...setting a number of interviews seems to me like giving up the expert's responsibility to vet the methodology.  Maybe the problem is that the methodology wasn't adequately described.  Shouldn't we review the number of interviews as part of reviewing the methodological description.  We still have that responsibility."  

I invite all comment on this issue.  This is the first time I have seen a journal take a stand of this sort.  I am glad that this has happened, because it opens up space for a well needed discussion. 

Urmitapa Dutta at UMass Lowell: Everyday Peace

UMass Lowell is pleased to welcome a new faculty member in the psychology department--Dr. Urmitapa Dutta.  She comes to us from the University of Illinois in Champaign, Illinois.  Dr. Dutta brings a wealth of experience about the day-to-day possibilities of creating peace in places where violence has become a normalized event. 

This fall I was lucky to have her as a speaker in my doctoral course on qualitative research methods.  She discussed her dissertation work in northern India where she worked with young adults using a blend of critical ethnography and participatory action research.

Spring 2013 she will be offering a very interesting class that will engage students in these same methods.

 

This is a not-to-be-missed experience!