Qualitative Research Methodology discussions in Non-Methodology Journals

Vraagteken (Photo credit: Wikipedia)

The semester is over and I am slowly picking up the pieces.  In re: the classes that are now behind me, one of the most important tasks is asking myself what worked, what did not work, and why.  What could I have done differently?  What assumptions did I bring to the assignment that were or were not fulfilled? 

One of these question marks was the last assignment for the doctoral qualitative research course.  I asked each class member (there were 12) to select an article describing a qualitative research study related to their research passion.  These passions ranged from lesson planning and the use of social media in community colleges to teaching logrhythims and universal design as a tool for working with troubled youth. 

I provided a chart of qualitative research issues to address in the analysis of the article.  All were representative of the issues raised in the textbook and readings--in other words, they represented the standard fare of methodologists.

Full APA Bibliographic Reference

Background/Problem:  What is the problem or dilemma faced here?

Question Article seeks to answer

Paradigmatic Stance/Research Strategy Used by Author

Ethical Processes and Issues discussed:  Subjectivity Concerns  Discussed

Data Collection:  What was collected/How/When/Where

Data Analysis:  What was the process of analysis described?

What were the key findings?

How is evidence marshalled to support findings?

How does the author account for validity or trustworthiness?

Quality of the analytic discussion? (How does the author probe findings? Question them? Offer challenges?)

Issues of representation

Significance:  What is the significance of this piece?  How does it connect to the discussion of the background or problem?

In the week's before the assignment came in, I had questions from several students asking about the articles they had chosen.  Several students complained that they couldn't find qualitative research studies in their area of interest.  These and others complained that the articles they found didn't talk about the methodological issues we had discussed in class.  Many said they were having to do a work around, that is, they were writing about the absence of a variety of methodological proofs.  I chalked up the questions to their own jitters and assumed things would be straightened out by the time I read the assignments.  

So, now that the grading is behind me:  What did I learn? 

Well, first I learned that I live in a methodological bubble.  I live, work, and breath in a world dominated by qualitative research methodology discussions.  I think that every researcher is with me under the same big tent.  However, this is not the case.  There are many researchers out there, very happily pursuing their various non-methodological interests.  These people make use of qualitative research to meet their ends, but they are not as worried about crossing their t's and dotting their i's.  If you know what I mean. 

This means that much of the qualitative research that is out there, reported upon, in various non-methodological journals, is doing its best to contribute to the understanding of a given topic, but it may not be representative of all the whistles, bells, and frills, for which the methodological purists are looking.

My students (all of whom were reporting on qualitative research studies published in peer reviewed journals) found that discussions of methodological tasks and steps were often subsumed under a simple label.  Grounded theory was the most popular label, although that title could cover a range of meanings.  Overall the mention of research strategies (as I would call the ways we refer to approaches like Grounded Theory, ethnography, etc.) got little space. 

I had asked them to look for information about the paradigm discussions we had read about in depth in the Handbook of Qualitative Inquiry.  No one found a word about that in the articles they had chosen.  So they wrote about the absence of paradigms and the implied paradigmatic stance of the author. 

The actual act of analysis was often collapsed into a one sentence description.  "Back and forth," meaning movement between an interview transcript and coding, was a frequent way to talk about analysis--so frequent that I got tired of writing:  "What does this 'back and forth' mean?!"   

Only one student had selected an article from one of the tried and true qualitative research methodology journals, and she alone could fill in the blanks for the assignment with close to the purity for which I had hoped.   For her, it was easy, because everything was laid out based on the template I had given her.  For the others, however, the task was not nearly as clear.  Clearly, she was not the only one who had done it right.  The issue was much larger. 

What should I do? 

Hating failure, my first thought was--I'll never give this assignment again.  But this was reactionary, and, in truth, they had actually done a very interesting job on the assignment. 

My second thought was:  I will assign all the articles in future and they will only have articles in their hands that represent the way qualitative research methodology SHOULD be described.  But that didn't seem to make great sense because from what I had learned from this assignment--that's not going to be what they enounter in the real world.  It's obvious that the template created by methodologists lives and breaths in our world, but is getting limited attention from those further from this limited group of gurus. 

My third thought was:  Let's do some more shared/guided reading of articles with strong and weak methodological descriptions and talk about the difference.  This will probably be where I end up. 

The challenge of working through the "Bringing it all Together" assignment, as I called it, is similar to the earlier blog mention I made of the journal that mandated a number of interviews as representing good methodological practice.  Lacking an adequate (read: informed) discussion of the qualitative research methodology in articles submitted to peer-reviewed journals, such mandates may seem like a reasonable response.  It seems that the other response--requiring good methodological description is much harder because writers, reviewers, and editors may lack the skills to provide this or shepherd these changes. 

So, should we stop permitting qualitative research results to be reported?  Is it too much trouble to get it right?  Or should we stop permitting their publication without a full methodological discussion?  What does a "full methodological discussion" mean?  Is "full" too long to be fit into the average peer-reviewed journal article?  What kinds of methodological short-hand is acceptable in reporting on qualitative research methodology? 

I think it is time to eat breakfast--there is no quick fix for this one.