Qualitative Research Methodology discussions in Non-Methodology Journals

Vraagteken (Photo credit: Wikipedia)

The semester is over and I am slowly picking up the pieces.  In re: the classes that are now behind me, one of the most important tasks is asking myself what worked, what did not work, and why.  What could I have done differently?  What assumptions did I bring to the assignment that were or were not fulfilled? 

One of these question marks was the last assignment for the doctoral qualitative research course.  I asked each class member (there were 12) to select an article describing a qualitative research study related to their research passion.  These passions ranged from lesson planning and the use of social media in community colleges to teaching logrhythims and universal design as a tool for working with troubled youth. 

I provided a chart of qualitative research issues to address in the analysis of the article.  All were representative of the issues raised in the textbook and readings--in other words, they represented the standard fare of methodologists.

Full APA Bibliographic Reference

Background/Problem:  What is the problem or dilemma faced here?

Question Article seeks to answer

Paradigmatic Stance/Research Strategy Used by Author

Ethical Processes and Issues discussed:  Subjectivity Concerns  Discussed

Data Collection:  What was collected/How/When/Where

Data Analysis:  What was the process of analysis described?

What were the key findings?

How is evidence marshalled to support findings?

How does the author account for validity or trustworthiness?

Quality of the analytic discussion? (How does the author probe findings? Question them? Offer challenges?)

Issues of representation

Significance:  What is the significance of this piece?  How does it connect to the discussion of the background or problem?

In the week's before the assignment came in, I had questions from several students asking about the articles they had chosen.  Several students complained that they couldn't find qualitative research studies in their area of interest.  These and others complained that the articles they found didn't talk about the methodological issues we had discussed in class.  Many said they were having to do a work around, that is, they were writing about the absence of a variety of methodological proofs.  I chalked up the questions to their own jitters and assumed things would be straightened out by the time I read the assignments.  

So, now that the grading is behind me:  What did I learn? 

Well, first I learned that I live in a methodological bubble.  I live, work, and breath in a world dominated by qualitative research methodology discussions.  I think that every researcher is with me under the same big tent.  However, this is not the case.  There are many researchers out there, very happily pursuing their various non-methodological interests.  These people make use of qualitative research to meet their ends, but they are not as worried about crossing their t's and dotting their i's.  If you know what I mean. 

This means that much of the qualitative research that is out there, reported upon, in various non-methodological journals, is doing its best to contribute to the understanding of a given topic, but it may not be representative of all the whistles, bells, and frills, for which the methodological purists are looking.

My students (all of whom were reporting on qualitative research studies published in peer reviewed journals) found that discussions of methodological tasks and steps were often subsumed under a simple label.  Grounded theory was the most popular label, although that title could cover a range of meanings.  Overall the mention of research strategies (as I would call the ways we refer to approaches like Grounded Theory, ethnography, etc.) got little space. 

I had asked them to look for information about the paradigm discussions we had read about in depth in the Handbook of Qualitative Inquiry.  No one found a word about that in the articles they had chosen.  So they wrote about the absence of paradigms and the implied paradigmatic stance of the author. 

The actual act of analysis was often collapsed into a one sentence description.  "Back and forth," meaning movement between an interview transcript and coding, was a frequent way to talk about analysis--so frequent that I got tired of writing:  "What does this 'back and forth' mean?!"   

Only one student had selected an article from one of the tried and true qualitative research methodology journals, and she alone could fill in the blanks for the assignment with close to the purity for which I had hoped.   For her, it was easy, because everything was laid out based on the template I had given her.  For the others, however, the task was not nearly as clear.  Clearly, she was not the only one who had done it right.  The issue was much larger. 

What should I do? 

Hating failure, my first thought was--I'll never give this assignment again.  But this was reactionary, and, in truth, they had actually done a very interesting job on the assignment. 

My second thought was:  I will assign all the articles in future and they will only have articles in their hands that represent the way qualitative research methodology SHOULD be described.  But that didn't seem to make great sense because from what I had learned from this assignment--that's not going to be what they enounter in the real world.  It's obvious that the template created by methodologists lives and breaths in our world, but is getting limited attention from those further from this limited group of gurus. 

My third thought was:  Let's do some more shared/guided reading of articles with strong and weak methodological descriptions and talk about the difference.  This will probably be where I end up. 

The challenge of working through the "Bringing it all Together" assignment, as I called it, is similar to the earlier blog mention I made of the journal that mandated a number of interviews as representing good methodological practice.  Lacking an adequate (read: informed) discussion of the qualitative research methodology in articles submitted to peer-reviewed journals, such mandates may seem like a reasonable response.  It seems that the other response--requiring good methodological description is much harder because writers, reviewers, and editors may lack the skills to provide this or shepherd these changes. 

So, should we stop permitting qualitative research results to be reported?  Is it too much trouble to get it right?  Or should we stop permitting their publication without a full methodological discussion?  What does a "full methodological discussion" mean?  Is "full" too long to be fit into the average peer-reviewed journal article?  What kinds of methodological short-hand is acceptable in reporting on qualitative research methodology? 

I think it is time to eat breakfast--there is no quick fix for this one. 

TQR: Bridging the Gaps Within Qualitative Research

This is an addition to the last post on what is happening in Marketing Research and Qualitative Research.  Absolutely not to be forgotten is the work that The Qualitative Report/ The Weekly Qualitative Report is doing to provide information on the full breadth of developments in qualitative research.  Marketing Research is not a forgotten country in their publication.

http://www.nova.edu/ssss/QR/WQR/wqr4_47.html

Their innovative online dissemination now has them reaching 5,000 subscribers!  They are one place where this new business approach to qualitative research is blended with work from other disciplines.  Thanks TQR! 

Qualitative Research Network at UMass-Lowell: Another Great Brown Bag

Image via WikipediaToday our qualitative research brown bag featured Pia Markkanen from the Department of Work Environment at the University of Massachusetts-Lowell.  Pia made a presentation on the mixed method study that became the Journal of Occupational Environmental Medicine (JOEM) article--"There's No Place Like Home: A Qualitative Study of the Working Conditions of Home Health Care Providers".  It was fascinating!

Pia described the qualitative to quantitative to qualitative interactions of the study like a dance.  Project SHARRP,as she called it, Safe Homecare and Risk Reduction for Providers, was funded by National Institute for Occupational Safety and Health of the Centers for Disease Control and Prevention.  Like it's name, Project Sharrp focused on accidents with sharp instruments in home health care and the issues related to blood borne pathagens. 

Her audience of qualitative researchers was interested in all the nitty-gritty detail, from how she selected participants and gained informed consent to the development of focus group questions and integration with quantitative data.  For more informationon this excellent study, contact Pia!  

In the course of this work, Pia has become a daily user of NVivo.  We didn't get a chance to look into her project, but she showed how coding was reflected in the article's charts, and she shared an NVivo model with us. 

Our last brown bag of the Fall 2011 semester will take place on Tuesday, December 13 from 12:30-1:30 pm.  Steve Tello and Yi Yang from Management will present.  We will be meeting in Southwick 240 on North Campus. 

Integrating Numerical and Non-numerical Data: Qualitative Quandries

I have been struggling with the issue of getting out of the quantitative/qualitative conundrum...figuring out how qualitative research fits in a world of big data, micro-applications and new forms of data collection, etc., etc.  This morning I came up with a diagram that begin to help me 'see' what I have been thinking about. 

In this diagram:  above the mid-line represents Numerical Data, and below the mid-line represents Non-Numerical Data.

Both numerical and non-numerical data are divided by forms of data created by the researcher and forms of data created by others (the background/context/literature of a study). 

There are four inter-linked quandrants:
Numerical Arena
1.  Upper left:  Numerical data created by the research dominates.  The furthest point out to the left is "Big Data"  where millions of data points are generated by cell phones, telephone bills, tweets, etc.  As you move closer toward the middle you have larger surveys...scaling down into smaller surveys...At the center point as you cross over you have descriptive statistics created by simple counting of the researcher.

2.  Upper right:  This area represents numerical data created by others.  This is numerical data that serves as background or context for the work of non-numerical researchers.  Increasingly we have access to a broad range of statistics that we employ to describe the context of our work.  

Non-Numerical Arena

3.  Lower left:  This is non-numerical data created by others:  This is "THE LITERATURE".  It's not on the chart, but probably pure theoretical work is someplace to the left of meta-studies...coming down to studies on specific projects. Numerical and non-numerical researchers turn to literature in their field as a kind of data.  Using qualitative data analysis software has made me more and more aware that this is a living, breathing form of data on a par with an interview or an observation. 

4.  Lower Right:  This is non-numerical data created by a specific researcher.  This is where qualitative researchers and their studies live and breath.  Close to the mid-point are non-numerical studies that have structure and variables much akin to numerical studies...and one moves out to the right from there to places that draw more and more on emic/insiders perspectives.  Observations...take you outside of the words you have chosen in the organization 'they' have created.  Non-numerical studies range from those that are short-term and semi-embedded in the environment of 'the other'....to those that are totally embedded and long term in duration.

One critical question that this diagram raises for me (actually re-raises for me because I have been coming back to it a lot in my head) and that is--What does non-numerical analysis really consist of?  How are we making those patterns or theories we think are at the heart of it?  As I work with coding in the Sexting project in which I am now involved, to be perfectly honest, a lot of my pattern making deals with...what do I have the most of...what do I have the least of...what category is heavy/rich/saturated?  what tags/codes should be there but I don't see it?  Now, when you are asking questions about more and less/absent or present--doesn't that imply a numerical perspective? 

OK--now some of you qualitative researchers may be saying--I don't do it that way.  I make my patterns in other ways...OK--how?  Can you describe what you are doing?  I am going to venture that more/less-present/absent is a starting point in which we are employing simplistic numerical concepts to launch us into meaningful places where we can find useful theoretical work.  This is why I had to draw each of the quadrants as a line that is shading/decreasing into the area of the nearby quadrants.

Disclosure:  I have been working this out in my head without paying attention to the growing literature on mixed methods.  Maybe this has been all hashed out already, maybe it is irrelevant to the emerging arguments about this issue.  Mixed methods people, tend to be closer to numerical discussions than I am.  From what I know of the lower right hand quadrant...and those who inhabit it...this has not be widely discussed. 

The discussion I am having here with myself draws a lot on what I have learned through working with qualitative data analysis software.  It is also related to the issue of inclusive and exclusive research approaches that I blogged about in September 2011.  Challenges to my secure vision of qualitative research are raising up from all around me.  

Edward Hackett and Emergent Technologies in the Human Sciences

Hackett, E. (2011). Possible dreams: Research technologies and the transformation of the human sciences.  In The Handbook of Emergent Technologies in Social Research, edited by Sharlene Nagy Hesse-Biber.  Chapter 1:  pgs 25-46.  Publisher:  Oxford Press. 

Hackett has made a follower--me!  This was a great chapter to lead off this volume--BIG PICTURE!  I note that he refers to "the human sciences", a term that takes me back in my mind to Dilthey, but he brings a new twist to the idea with the notion of "complementary research in the social and behavior sciences" (p. 27).

Another term that brought back memories to me was "ensembles" as in "ensembles of research technologies".  Years ago when I was working on the Hanau Model School Partnership I wrote about the ensembles of school technologies (I presented this at AERA, wrote the paper, but never finished the revisions--such is life).  Hackett points to environmental studies and the ways they have made strides through assembling new "ensembles of research technologies", urging the human sciences to consider this model.

Hackett brings experience working at NSF to bear on the discussion.  Clearly it provided him with the opportunity to look across emerging areas of science/social science and consider the lessons that were being learned. 

I like the way he has integrated history (a discussion of the development of the survey) and the future (the new models described above) to provide a context for thinking about the integration of research approaches.

He identifies four critical domains that research in the human sciences must contend with:  place, scale, time, and engagement.  Here's the quote that ties this all together for me:

In sum, ongoing changes in the importance and nature of place (from local to virtual), the scale of phenomena (from genetic or neural to global), the scope of time (from briefer than a blink to evolutionary), and the terms of engagement with research subjects and users are combining to form the ensembles of research technologies that will be developed by the social and behavioral sciences (p. 35).

  He offers 8 "categories of innovation" (p. 35) that address these issues--not to be missed.

What I take away in my quest to understand what this all means to qualitative research is:
-he doesn't mention qualitative research (nor quantitative research specifically)...he is talking on another plane
-he doesn't mention qualitative data analysis software...and yet is talking about the next generation of technologies that build on the CAQDAS base

-he IS talking about the kind of inclusive (multi-disciplinary/multi-methodological) research that I blogged about not to long ago.
-he IS talking about constant experimentation and purposing/repurposing of technologies to support the goals of research

A very good start to the reading of this book.  

Qualitative Research: What is a project? What are mixed methods?

Image via WikipediaI began to ponder these questions recently as I was comparing my two current projects in my head:
1.  The Journal Project:  A study of 18 months of my personal journals using qualitative data analysis software (QDAS) and arts-based approaches; and,
2.  Sexting and Teens:  A 3-state multi-disciplinary, mixed method study of teen...and adult view of teens views of sexting.

I was thinking about the characteristics of the two.  The Journal project is very internal--individual--inside of me. It's insular, no doubt about it.  No teamwork, no communal nothing.  It's very QR, meaning that it doesn't use cases and attributes in QDAS...it focuses on written texts and coding. 

The Sexting project is extroverted, communal and interactive.  A team project that uses mixed methods--meaning it uses cases and attributes in QDAS, AND written texts and coding. Quantitative data is organized in SPSS.  

Comparing the two got me thinking first about:  What is a project?  And this led me to think about:  What is mixed methods--and how is QDAS re-defining that term?

In re: project:  I asked myself--How do we live a project?  Where do we find projects?  How is a project like a family, relationship, book?  How near or far do we place ourselves? 

Then, the inevitable comparison to computer methods came up:  What is a project vs an e-project?  Are they different?  How does QDAS circumsribe the project?  Is a project a question?  a case? a place? a people? a culture?

One of the things that irritates me about working with people who know a little about qualitative research, enough to get themselves into trouble, is the inordinate amount of concern they expend on trying to define what kind of qualitative research a project is:  Is it case study?  Is it ethnography?   Is it grounded study?  These concerns never seem to make sense to me, and I realized that one problem is that I am more focused on "the project" or "the e-project" in my case.  My focus leads me to ask, how am I going to best organize within this tool?  What is the best granularity in terms of analysis--should this be broadly or finely coded?  And, most importantly--is this a study that will need cases and attributes or am I going to stick in the texts and codes area?

This morning I woke up and realized that, in part, what I am thinking about regarding the e-project is actually a discussion of the continuum of mixed methods as it is shaped by QDAS.   Discussions of mixed methods have to date been pretty much siloed, in that you can't actually connect the two because the quantitative piece is done with software and the qualitative piece is done with manual methods.  So the notion of the two "informing" each other can only be very loosely applied.  This is pseudo-integration. 

Only with full computerization for both quant and qual do you get mixed methods.  Without QDAS you cannot leverage variables against text(s), which is the real hallmark of what 'mix' could mean.  You do this in QDAS with cases, attributes, and searches.

Here is a simple continuum of the three methods (it runs top to bottom not left to right because of the funkyness of Blogger--sorry)

Quan/SPSS or other computer tool
-cases
-attributes
-numerical texts
-fixed information/
pre-assigned

QDAS/Mixed Methods
Cases
Attributes
written, visual, and audio texts
codes (pre-assigned and/or emergent)
memos

Qual/QDAS or Manual Methods
-written, visual, and audio texts
-codes
-memos
-preassigned codes....evaluation
-emergent codes...ethnography, phenomenology, etc. 

In looking at the continuum, it seems to me that the sorting approach is the critical piece and the form of text one uses.  With the quantitative perspective, you are using numerical texts and sorting by case and attributes; With the mixed methods approach you are using both numerical and non-numerical data and sorting by cases and attributes AND coding.  In a truly mixed method approach, you are importing the cases and attributes from the numerical data into the non-numerical data base and using those as a means of sorting the non-numerical data (or using a QDAS package in which numerical data can also be handled).  In a more strictly qualitative approach, you are sorting without reference to cases and attributes but focusing primarily on coding (which can be pre-assigned or emergent). 

My two studies represent two aspects of the continuum.  The Journal project is deep into the qualitative research end of things--code and retrieve, baby, forget cases and attributes.  The Sexting project, on the other hand, is shaping up to be an excellent example (if I do say so myself) of the truly mixed method approach in which computer technology is leveraged on both sides of the divide--quantitative and qualitative.  We are using SPSS and NVivo 9...and the SPSS numerical data is being imported in Excel to serve as the cases and attributes for the non-numerical data--connect it to the coding, and you have a data base that has real search capacity.  You can ask questions of many sorts.

This brings me to my final point, which is that, efficiency increases (and you significantly extend your reach) when you stop doing it all in your head and transfer some of the heavy lifting to a reliable tool.  This is a warning to both quantitative and qualitative researchers.  It's a well known fact that qualitative researchers have been ducking the use of QDAS, but it is less well understood that mixed methods researchers are doing the same.  In marketing and other forms of evaluation, quantitative data is being organized in Excel or SPSS, but qualitative data related to the study are processed entirely by manual methods.  This leaves no technical pathway by which to connect the two and do meaningful comparisons.  The comparisons can only be gross, intuitive, and guesswork.  What I am saying is that QDAS is the elephant on the table for both mixed methods and pure qualitative researches.

Hmm...I think this bears more consideration.  

Books from the ICQI conference

Image via Wikipedia

It's been a long hiatus from blogging, but that's what the end of the semester will do to you.

A week ago I was at the ICQI conference in Champaign, Illinois (what fun!) and as my annual treat I picked up an armload of new books.  I feel strange carting off this much paper in this age of online communications.  I am not sure how much longer this tradition will continue.  I may find myself 'beaming down' the latest book into my I-Phone or some other device.  But for now I have this nice pile of fresh pages.

Here are my choices:

Erotic Mentoring:  Women's Transformations in the University by Janice Hocker Rushing (Left Coast Press).  This is the one I've started to read.  I like her invitational style (which she attributes to encounters with Ellis and like-minded qualitative researchers).  I was particularly interested in this book because of the possible connections to my own journal study.  There is a strong Jungian perspective here, something I haven't run into for a while. 

Autoethnography as Method by Heewon Chang (Left Coast Press).  I was very curious to see how a qualitative research methodology text would be translated through the eyes of autoethnography.  I also met Heewon at the conference--she was sitting near me in the session on coding organized by Ray Maietta--and it was then that I realized she was one of the winners of the first QSR teaching grant.  Now I am curious to talk with her more about the ways she is using NVivo with autoethnographic content.  You can do it, of course, but for the technophobic, technology is often frowned upon with more humanistic content or approaches.  Heewon--I will be calling! 


Guyana Diaries: Women's lives across difference by Kimberly Nettles (Left Coast Press).  I thought this book would be an interesting opportunity to read a real ethnography--not another methodology book--not that I distain methodology books.  In keeping with my journal project, it seemed like this would be a place to encounter some good feminist theory.  Nettles describes her studies of The Red Thread Development Corporation, a woman-run activist organization. 

Poetry as Method:  Reporting Research Through Verse by Sandra Faulkner (Left Coast Press) couldn't help but catch my eye.  Faulkner not only talks about representing research in poetry but also explores the ways poets processes parallel/intersect with similar processes of researchers.  She also provides guidelines for evaluating research poetry. 

Finally, I've been searching for a book(s) that would be helpful for moving students into doing online research, and that's why I came home with Janet Salmons Online Interviews In Real Time (Sage).  I looked at several other books on related topics, many with a more comprehensive approach but Salmons seemed the most engaging.  It is very hard to buy books about new technologies because once the book appears, you know the technology has moved on.  It looked like it offered advice that could hold with a range of technological changes. 

It's summer, so back to blogging! 

Memos and Qualitative Research: Kathy Charmaz

Image by darkmatter via Flickr

Since I started this blog I've written several entries about visual memos.  For some time I've meant to look at the discussion of memos/memoing in texts that I consider classics in this area to see how these descriptions might help me understand what I am trying to get at with the visual memo.  So here are some of my favorites on memoing:

Charmaz, K.(2006).   Constructing grounded theory: A practical guide through qualitative analysis.  Thousand Oaks, CA:  Sage Publications.

Emerson, R., Fretz,R., Shaw, L. (1995).  Writing ethnographic fieldnotes.  University of Chicago Press. 


Maxwell, J. (2005).  Qualitative research design:  An interactive approach (2nd ed.).  Thousand Oaks, CA: Sage Publications. 


Miles, M. & Huberman, A.M. (1994).  Qualitative data analysis: An expanded sourcebook.  Thousand Oaks, CA:  Sage Publications.


(Oops...just realized I left out one of my favorites by Anslem Strauss--I'll catch up with that one later.)

Today I am going to write about one of these:  Kathy Charmaz's discussion of memos. 


"Memo-writing" is the title of Chapter 4 in Charmaz's book.  It is an elegant description of memos in grounded theory.  It will really give you shivers. 

"Memos catch your thoughts, capture the comparisons and connections you make, and crystallize questions and directions you want to purse."  (p. 72)

Here are some of the functions memos fill for her:

• "conversing with yourself" (p. 72)
• "explicate and fill out categories" (p72) 
• "serve as the analytic core" for subsequent writing (p.76)

Her primary advice in creating memos is to "do what works for you" (p. 80) and "...engage a category, let your mind rove freely in, around, under, and from the category..." (p. 81)

Charmaz explores the ways the techniques that writers of many sorts employ to get themselves off the ground--clustering, freewriting, etc.--can all serve the qualitative researcher at the stage of memo writing.

She gives special attention to "Using memos to raise focused codes to conceptual categories" (91)  This seems to me key in thinking about the ways memos can assist in the refinement process, that is, the process of making a clump of data into something called evidence--that's like spinning flax into gold. 

One thing that cannot be ignored in talking about memos and qualitative research is that the discussion of memos is always right in the middle, squished between data collection and coding on one side...and products, representations, and presentations on the other side.  Memos, in the written textual mode, are always about refinement of raw data and leveraging preliminary interpretations.  As Charmaz says, the construction of the memo is the "pivotal intermediate step between data collection and writing drafts of papers" (72). 

For the novice researcher, I think that the notion of memos is hidden behind flashy things that catch their attention like coding or interviewing--things that stand out as if they have neon lights on them.  Memos, for the qualitative research connossieur, however, are like a fine wine.  They are swished around in the mouth and savored!  The tastes are subtle and learned.

I think I will stop here with Charmaz and take on another one of the classic memoists next. 

Visual Memos in Qualitative Research: Part III

Image via Wikipedia

Began this morning with more thoughts about visual memos. 

Is a visual memo a cartoon?  Is it a story?
What is the form of the narrative of visual memos?
How do the elements of the narrative work?

Would qualitative researchers use visual memos for the same thing that they would use written memos for?  How does a researcher decide to insert the visual? 

This reminded me of single photos or drawings--artifacts from my data collection activities--that I came to refer to as memos becuase I found them so iconic and I used them repeatedly to talk about a concept/notion that had arisen in the interpretation.  An example is a photo of a 1st grade classroom that I have used many times to talk about multiple ways to read the photo and think about the cluster of things and practices.  One way I approach it is to talk about "reading the walls".  I can read the whole day on that one wall--from morning message through the day's schedule, to a host of other activities.  Another way to approach it is to look at the intersection of things (computers, record player, blackboard) and the things they imply...and the technological eras they represent. 

When photos take this relationship to the study, I no longer consider them an artifact, but a memo.  Do I have the right to do this?  Is this a legitimate way to refer to them?  Perhaps more important then if they are a memo or an artifact, is the peculiar transformation (in my mind) that has occurred to this piece of visual material.   Does a well used artifact become a memo when it has accreted experience for me...not just for them?  Meaning, do my experiences of explaining the photo from methodological perspectives facilitate the process of memo creation?

This reminded me of the kinds of problems that students in qualitative research class have with the notion of memo.  When first introduced, many struggle hard with it.  It seems very foreign to them.  They want to turn out business memos--something similar to ordering new office supplies.  The word memo seems like an impenetrable barrier.  I've tried changing the word, but it's like mixing medication in applesauce, they can smell out the duplicity.  They know there is something different here. 

I've examined many texts about memos in qualitative research--I suppose I should do something like Johnny Saldana's overview of coding--what are all the different ways qualitative researchers have of describing memos? 

This does not end here--there's more to consider.