Qualitative Research Methodology discussions in Non-Methodology Journals

Vraagteken (Photo credit: Wikipedia)

The semester is over and I am slowly picking up the pieces.  In re: the classes that are now behind me, one of the most important tasks is asking myself what worked, what did not work, and why.  What could I have done differently?  What assumptions did I bring to the assignment that were or were not fulfilled? 

One of these question marks was the last assignment for the doctoral qualitative research course.  I asked each class member (there were 12) to select an article describing a qualitative research study related to their research passion.  These passions ranged from lesson planning and the use of social media in community colleges to teaching logrhythims and universal design as a tool for working with troubled youth. 

I provided a chart of qualitative research issues to address in the analysis of the article.  All were representative of the issues raised in the textbook and readings--in other words, they represented the standard fare of methodologists.

Full APA Bibliographic Reference

Background/Problem:  What is the problem or dilemma faced here?

Question Article seeks to answer

Paradigmatic Stance/Research Strategy Used by Author

Ethical Processes and Issues discussed:  Subjectivity Concerns  Discussed

Data Collection:  What was collected/How/When/Where

Data Analysis:  What was the process of analysis described?

What were the key findings?

How is evidence marshalled to support findings?

How does the author account for validity or trustworthiness?

Quality of the analytic discussion? (How does the author probe findings? Question them? Offer challenges?)

Issues of representation

Significance:  What is the significance of this piece?  How does it connect to the discussion of the background or problem?

In the week's before the assignment came in, I had questions from several students asking about the articles they had chosen.  Several students complained that they couldn't find qualitative research studies in their area of interest.  These and others complained that the articles they found didn't talk about the methodological issues we had discussed in class.  Many said they were having to do a work around, that is, they were writing about the absence of a variety of methodological proofs.  I chalked up the questions to their own jitters and assumed things would be straightened out by the time I read the assignments.  

So, now that the grading is behind me:  What did I learn? 

Well, first I learned that I live in a methodological bubble.  I live, work, and breath in a world dominated by qualitative research methodology discussions.  I think that every researcher is with me under the same big tent.  However, this is not the case.  There are many researchers out there, very happily pursuing their various non-methodological interests.  These people make use of qualitative research to meet their ends, but they are not as worried about crossing their t's and dotting their i's.  If you know what I mean. 

This means that much of the qualitative research that is out there, reported upon, in various non-methodological journals, is doing its best to contribute to the understanding of a given topic, but it may not be representative of all the whistles, bells, and frills, for which the methodological purists are looking.

My students (all of whom were reporting on qualitative research studies published in peer reviewed journals) found that discussions of methodological tasks and steps were often subsumed under a simple label.  Grounded theory was the most popular label, although that title could cover a range of meanings.  Overall the mention of research strategies (as I would call the ways we refer to approaches like Grounded Theory, ethnography, etc.) got little space. 

I had asked them to look for information about the paradigm discussions we had read about in depth in the Handbook of Qualitative Inquiry.  No one found a word about that in the articles they had chosen.  So they wrote about the absence of paradigms and the implied paradigmatic stance of the author. 

The actual act of analysis was often collapsed into a one sentence description.  "Back and forth," meaning movement between an interview transcript and coding, was a frequent way to talk about analysis--so frequent that I got tired of writing:  "What does this 'back and forth' mean?!"   

Only one student had selected an article from one of the tried and true qualitative research methodology journals, and she alone could fill in the blanks for the assignment with close to the purity for which I had hoped.   For her, it was easy, because everything was laid out based on the template I had given her.  For the others, however, the task was not nearly as clear.  Clearly, she was not the only one who had done it right.  The issue was much larger. 

What should I do? 

Hating failure, my first thought was--I'll never give this assignment again.  But this was reactionary, and, in truth, they had actually done a very interesting job on the assignment. 

My second thought was:  I will assign all the articles in future and they will only have articles in their hands that represent the way qualitative research methodology SHOULD be described.  But that didn't seem to make great sense because from what I had learned from this assignment--that's not going to be what they enounter in the real world.  It's obvious that the template created by methodologists lives and breaths in our world, but is getting limited attention from those further from this limited group of gurus. 

My third thought was:  Let's do some more shared/guided reading of articles with strong and weak methodological descriptions and talk about the difference.  This will probably be where I end up. 

The challenge of working through the "Bringing it all Together" assignment, as I called it, is similar to the earlier blog mention I made of the journal that mandated a number of interviews as representing good methodological practice.  Lacking an adequate (read: informed) discussion of the qualitative research methodology in articles submitted to peer-reviewed journals, such mandates may seem like a reasonable response.  It seems that the other response--requiring good methodological description is much harder because writers, reviewers, and editors may lack the skills to provide this or shepherd these changes. 

So, should we stop permitting qualitative research results to be reported?  Is it too much trouble to get it right?  Or should we stop permitting their publication without a full methodological discussion?  What does a "full methodological discussion" mean?  Is "full" too long to be fit into the average peer-reviewed journal article?  What kinds of methodological short-hand is acceptable in reporting on qualitative research methodology? 

I think it is time to eat breakfast--there is no quick fix for this one. 

Action Research Journal Blog

Wassily Kandinsky, "On White II", 1923 (Photo credit: Wikipedia)

I am going to be teaching Practitioner Action Research, Spring 2013, and in preparation, I signed up for alerts from Sage's Action Research Journal.  Just discovered they have an interesting blog with extended content and commentary. 

Action Research Journal Blog

Take a look--you might find it interesting. 

Maxqda 11 Introduced

Simple emoticons of the five temperaments: Sanguine (top right), Choleric (bottom right), Melancholy (bottom left), and Phlegmatic (centre), with the new temperament Supine (top left) and Phlegmatic blends in between. (Photo credit: Wikipedia)

Exciting!  I just took a look at the new features of Maxqda 11.  It is definitely running neck and neck with what NVvivo is offering, BUT here are two new features that are very different:

1.  A cell phone app!!!  They are right--this is a first for the larger comprehensive qualitative data analysis software packages (There are apps, but not produced by these companies to interface with their products). 

2.  Emoticon coding!!!  I like this idea too.  Just as you can define a color for a coding stripe in some packages...now Maxqda lets you assign an emoticon as a quick visual reference for a code. 

Here is a link to the introductory video.  I am going to have to try this out. 


Introduction to Maxqda 11

Emerging Scholars at UMass Lowell

View of University of Massachusetts Lowell, Lowell, Massachusetts, USA. (Photo credit: Wikipedia)

For the past two years, UMass Lowell has been engaged in a wonderful undergraduate research project called "Emerging Scholars".  Conceived of by the Center for Women and Work, Emerging Scholars are a hand-picked group, selected from our top undergraduates.  Each one is assigned to work with a researcher who has submitted an application.  The match is made...and the results are great.

The Sexting project is in its second year.  Andy Harris from the Criminial Justice Program and I are sharing the joys of working with this year's Emerging Scholar for our project--Deborah Paul.  Follow this link to see the three of us at the mid-year get together, where the Emerging Scholars share their progress with the various researchers and other campus visitors. 


Emerging Scholar Presentations December 2012

It is always very exciting to hear your own Emerging Scholar describe the semester's work, but also very interesting to hear about what everyone else is doing.  Deborah has taken on independent analysis of a cache of data from the project...and she quickly became an NVivo whiz.  She is definitely going places.  We are pleased that she was chosen as our Emerging Scholar for 2012-2013. 

The Sage Handbook of Qualitative Research...riffs on articles

In my qualitative research course (07.704--Introduction to Qualitative Research Methods), students have just completed an assignment in The Sage Handbook of Qualitative Research (4th edition, eds. Norman Denzin & Yvonna Lincoln).  They read a self selected article in the area of strategies and paradigms and each created a one-page visual overview of that article. 

I was so taken with these and the unique ways that students depicted the information that I decided to publish their pieces here in my blog.  Each one provides a brief introduction, which I will post with their one-pager. 

Coming up....

Center for Women and Work at UMass Lowell: Another Great Holiday Fair

It was another great success--the annual holiday fair of the Center for Women and Work at UMass Lowell.  Thursday November 29 from 2-7 pm at the Inn and Conference Center--you had to be there or be square. 

For the second year, I was invited to serve as the official MC (translation: person who introduces musicians and urges everyone to shop more).  I know why I was asked--I have the right hat!  The Jester's hat!

And--here I am modeling the silent auction item I won--a crocheted piece by psychology professor Sarah Kuhn. 

This event has now become a holiday time feature on the campus--it celebrates the work of campus women, raises funds for the Center for Women and Work, and is a great time for all!  Don't miss next year's event. 

Sample Size and Qualitative Research

The Great Wave off Kanagawa (Photo credit: Wikipedia)

Someone had to do it.  Do what? Make a definitive answer to what the sample size should be in a qualitative research study published by a peer reviewed journal.

In this case the someone was Shari L. Dworkin, Associate Editor of the Archives of Sexual Behavior, the individual responsible for qualitative research submissions at this journal.  She takes her stand in an editorial (2012) 41:1319-1320.  Henceforth, the policy of her journal will be to recommend that:

25-30 participants is the minimum sample size required to reach saturation and redundancy in grounded theory studies that use in-depth interviews. 

A part of me admires her for putting a number to her answer.  I, too, have struggled many times when people ask me that question--"How many interviews does it take?" "What's enough data?"  "How do qualitative researchers figure out what is the right sample size?"  The answer that most qualitative researchers give:  "It depends," is not very satisfying for the novice or the researcher who is struggling to gain an understanding of this methodology. How easy it would be just to say "25-30; take it or leave it". 

In her editorial, she does a good job of talking through the rationale for her answer and discussing the literature upon which she drew for her answer (the primary grounded theory gurus).  She identifies this number as about right if you are seeking to find saturation...no new leads...not much new coming up. 

Just as I admire her...I also fear for her, because I worry that, in making her editorial decision, there is a lot of qualitative research territory that she hasn't taken into consideration.  So, let me unpack what some of the "it depends" concerns would be:

• Grounded theory studies are not the whole of qualitative research, and in relying so heavily on this one "flavor" in our field have important views been ignored.  (Now I personally believe that the whole qualitative research flavor discussion is over-rated...see earlier blog posts...but I think it may be relevant here.)
• Grounded theory studies are not always solely composed of interviews (as the editorial appears to suggest).  I would want to know how interviews are mixed with other forms of data before I could say that, for instance, 10 interviews, was insufficient.  [How many observations?  Of what kind and length?  Was there significant document analysis?  Was visual data obtained?  What about the use of journals from the participants?]  I would need to know how the interviews were contextualized in the sea of data that is possible to gather in a qualitative research study. 
• What do you mean by "in-depth interviews"?  From the editorial, I have the sense we are talking one interview per individual (maybe 1 hour?).  In my training with Buddy Peshkin...in-depth would have meant 4-5 interviews (yes 4-5 hours divided up over a period of time) with one individual.  Whose in-depth are we using? 
• What does 25-30 mean?  In the editorial it appears that this would be 25-30 interviews with separate individuals.  If you had 5 focus group interviews with 5 individuals in each (5 x 5 = 25)...would that count? I am working on a study that relied primarily on focus groups--I can report that we interviewed hundreds...but in groups. 
• What will a ruling like this do to the incredibly important smaller exploratory start-up study, and/or the dissertation study?  These are often of smaller size than a very well funded study by a senior researcher.  Will these valuable efforts become unpublishable?  Are we starting a new kind of ranking order for peer reviewed journals?  

When I shared my concerns about the policy described here with a friend who is a qualitative researcher, her response was--"...setting a number of interviews seems to me like giving up the expert's responsibility to vet the methodology.  Maybe the problem is that the methodology wasn't adequately described.  Shouldn't we review the number of interviews as part of reviewing the methodological description.  We still have that responsibility."  

I invite all comment on this issue.  This is the first time I have seen a journal take a stand of this sort.  I am glad that this has happened, because it opens up space for a well needed discussion. 

Urmitapa Dutta at UMass Lowell: Everyday Peace

UMass Lowell is pleased to welcome a new faculty member in the psychology department--Dr. Urmitapa Dutta.  She comes to us from the University of Illinois in Champaign, Illinois.  Dr. Dutta brings a wealth of experience about the day-to-day possibilities of creating peace in places where violence has become a normalized event. 

This fall I was lucky to have her as a speaker in my doctoral course on qualitative research methods.  She discussed her dissertation work in northern India where she worked with young adults using a blend of critical ethnography and participatory action research.

Spring 2013 she will be offering a very interesting class that will engage students in these same methods.

 

This is a not-to-be-missed experience!  

Historical Social Research publication

Istanbul (Photo credit: Greenwich Photography)

I am pleased to announce that Historical Social Research has republished an article I co-authored that first appeared in FQS this year. 

Cesar A. Cisneros Puebla and Judith Davidson

Qualitative Computing and Qualitative Research: Addressing the Challenges of Technology and Globalization

This piece appears in Historical Social Research (Historische Sozialforschung)Vol 37 (2012) 4, No. 142.  It provides an overview of the FQS special issue of selected papers from the 2011 "Qualitative Computing: Diverse Worlds and Research Practices Conference" that was held in Istanbul, Turkey. 

Take Note to Make Note

Graphic Notetaking (Photo credit: derekbruff)

I started blogging today with an entry about the Radcliffe Institute's "Take Note" conference that I had attended at the beginning of November. 

Between that entry and this one are eleven entries in which I share a note taking experiment in my doctoral course 07.704 Introduction to Qualitative Research methods.  Each student has created a 1 page visual introduction to a selected chapter from Denzin and Lincoln's "Sage Handbook of Qualitative Research". 

The eleven entries are diverse in their methods of presentation, ranging from simple webs or a single picture to a cartoon, a powerpoint slide, and visual/textual combinations that range across a page. 

As the instructor of this interesting crew, I can see the individual personality in each one from humor or confidence, to the sense of organization or wonder.  Personal expression flows to the surface of each one-page rendition. 

I also see the powerful draw of the disciplinary focus in which they are embedded, be it politics, science, math, school administration, or language learning.  Each discipline has shaped the individual's preferred way of visualizing in some respect. 

In creating this 1 page set of thoughts about an article, they are, of course, interacting with a specific author or authors.  Topics included critical pedagogy, feminism, mixed methods, case study, grounded theory and more.  As the individual learner from my class encountered this new material they also encountered the author who is distinctively represented within the article.  It was this individual who introduced my student to this topic.  What kind of a guide are they?  How well do these personalities mesh, regardless of the topic and its interest? 

There were certain visual elements in each piece that stopped us in our tracks:  Jay Lang's upside down house in the discussion of grounded theory is a good example or Aubrey Rochembeau's turned upside down article about Case Study method.  It is probably not strictly correct to call Nancy Nickerson's poem a visual--but it had a similar effect.   To write a poem in response to a poetry writing ethnographer appealed to our entire group. 

In describing the process of creating a visualization, students brought us more deeply into the process of their own reading experience and the making sense of the individual text with which they were grappling.  Dave Sciuto's twining vine reminds me of this...or Kim Gustenhoven's "cells" that surround the edge of her image frame. 

This assignment and its discussion took place in a class where I am constantly nudging students to think about the possibilities of the visual or visualization.  It's a fascination of mine that I seek to share with them.  Many would not have taken this route without the assignment.   Because each student read a different article, I cannot imagine what our discussion would have been like without a visual element to help us connect to the auditory experience of simply hearing a description of the article. 

I would say I have gone from Take Note to Make Note and quite a few things in between.

In attending the Take Note conference, I was particularly interested because as a qualitative researcher, my whole world is about taking notes--fieldnotes (observations, interviews, sketches, photos)--every way that I can possibly document the cultural experience of whoever I am studying.  As a teacher of qualitative research, I am teaching others to take notes in these forms, and, in so doing to consider what it means to "take note" as qualitative researchers do. 

In the exercise that I share here in my blog, my students are taking note of the ideas of other qualitative researchers and, through these notes, I hope becoming more attuned to a certain set of notions that are important to this field.  These were not private notes, but deliberately public notes.  Also these were not informal notes; they came with instructions about format and content.  So, are they really notes?  Is this a form of a paper?  As occurred at the Take Note Conference the notion of note has become deeply problematized (at least for me) through this inquiry. 

Melinda Willis and Qualitative Research

Melinda Willis is another student who returned to the circular, as I have dubbed it.  Pathways, circles, edges and borders--these seem to characterize a certain approach to this assignment of a 1 page visual.

Here is her written introduction to Kincheloe, McLaren & Steinberg's piece:

Critical Pedagogy and Qualitative Research

By J. Kincheloe, P. McLaren & S. Steinberg

A Brief Introduction

Power.  How have those in power shaped our understanding of and relationship with our culture and others within our greater culture? These are the questions at the heart of critical research, often performed as action or participatory research.

Talking with those not in power, the non-dominant culture, to learn their perspective, opens windows into an understanding of behaviors which may not be understood or misunderstood by someone outside of a micro-culture. In the past Eurocentric researchers observed other cultures and described them from a Eurocentric perspective. In critical research, researchers are working actively with the participants of their research, asking the participants to speak for themselves, enabling the researcher to get a glimpse inside the participants’ world from the participants’ perspective. Incorporating the methodology of bricolage, the researcher uses multi-disciplinary research techniques and intentionally notes their own biases, while working to gain an understanding of both the dominant culture and the non-dominate culture. Creating an open dialogue, the researcher works to increase cross-cultural understanding, which can be carried into the classroom, both with respect to understanding student behavior and academic discussions.

Reference

Kincheloe, J. L., McLaren, P. & Steinberg, S. R. (2011). Critical pedagogy and

qualitative research. In N. K. Denzin & L. S. Lincoln (Eds.), The Sage handbook of qualitative research (4^th ed.)(pp.163-177). Thousand Oaks, CA: SAGE.

Now, here is the visual that accompanied the description:

The image, as I am calling it, is actually constructed almost entirely of text organized in different visual clusters.  There are a few simple diagrams in strategic places.  The use of underlining, arrows, and circling are made use of to draw further attention to a component. 

Anne Sheehy and Qualitative Research

Anne Sheehy is a graduate student with a passion for the possibilities that technology can bring to learning.  She chose to review Teddlie and Tashakkori's article "Mixed Methods Research: Contemporary Issues in an Emerging Field" from the Sage Handbook of Qualitative Research.

Here is her one page analysis of that work:

As a technologist, Anne likes to make use of the organizational tools computers offer. 

Dave Sciuto and Qualitative Research

Dave Sciuto bit off a big piece when he took on Holstein and Gubrium's "The Constructionist Analytics of Interpretive Practice".  Here is his description of the guts of that article:

Intertwining Interpretive Constructionist Practices

In their article, “The Constructionist Analytics of Interpretive Practice,”  Holstein and Gubrium  (2011) discuss the development of a constructionist analytics of interpretive practice,  a derivative of  constructionist inquiry.  According to the authors, this concept resembles enough of the elements of constructionist inquiry to constitute its own research program.  They look briefly at the contributing constructionist studies that make up the program: social phenomenology, ethnomethology, ordinary language philosophy, and Foucauldian discourse analysis.

Traditionally, constructionism has focused on how social reality is constructed, managed, and sustained.  Ethnomethodologists  generally see the how questions (or how) analytics in qualitative inquiry, keenly attuned to the naturally occurring talk and social interactions.  Conversely, Michel Foucault, looks at what is being accomplished, under what conditions, and out of what resources noting that objects and subjects accent the constructive what questions (or whats) the discourse constitutes as much as the hows of discursive technology.  He is concerned with the physical location of the discourse:  the prison, hospital, and asylum, for examples. Although  Foucault is largely missing the hows, ethnomethodology is largely missing the whats, ethomethodology and conversation analysis (CA), and its similar variant of discourse analysis, discursive constructionism,  have recently begun to examine every day descriptions, claims, reports, assertions, and allegations as contributions to social order construction

The authors see this as an intertwining of the two forms of constructionist inquiries. Although what questions traditionally have played a lesser role than how question in a social construct, a related set of concerns now looks at the what of social reality.  They identify this interpretive practice as a way to turn us to both the hows and whats  of social reality, using postanalytic ethnomethodology narratives as a balance of the hows with the whats, using settings, cultural understand, and their everyday mediations reflexively with talk and social interaction.  To understand how these narratives operate in everyday life, we need to know the details and mediating conditions of narrative occasions.  These details can only be discerned from direct consideration of the mutually constitutive interplay between what we have traditionally called narrative work (hows) and narrative environments (whats).

 Now here is the image that goes with this description:

Sciuto, like his image, has a keen ability to simplify complexity when needed.  I love the contrast between the word dense description and the simple intertwining vine.  With our reading, how often can we get to the point like this?