Qualitative Research Methodology discussions in Non-Methodology Journals

Vraagteken (Photo credit: Wikipedia)

The semester is over and I am slowly picking up the pieces.  In re: the classes that are now behind me, one of the most important tasks is asking myself what worked, what did not work, and why.  What could I have done differently?  What assumptions did I bring to the assignment that were or were not fulfilled? 

One of these question marks was the last assignment for the doctoral qualitative research course.  I asked each class member (there were 12) to select an article describing a qualitative research study related to their research passion.  These passions ranged from lesson planning and the use of social media in community colleges to teaching logrhythims and universal design as a tool for working with troubled youth. 

I provided a chart of qualitative research issues to address in the analysis of the article.  All were representative of the issues raised in the textbook and readings--in other words, they represented the standard fare of methodologists.

Full APA Bibliographic Reference

Background/Problem:  What is the problem or dilemma faced here?

Question Article seeks to answer

Paradigmatic Stance/Research Strategy Used by Author

Ethical Processes and Issues discussed:  Subjectivity Concerns  Discussed

Data Collection:  What was collected/How/When/Where

Data Analysis:  What was the process of analysis described?

What were the key findings?

How is evidence marshalled to support findings?

How does the author account for validity or trustworthiness?

Quality of the analytic discussion? (How does the author probe findings? Question them? Offer challenges?)

Issues of representation

Significance:  What is the significance of this piece?  How does it connect to the discussion of the background or problem?

In the week's before the assignment came in, I had questions from several students asking about the articles they had chosen.  Several students complained that they couldn't find qualitative research studies in their area of interest.  These and others complained that the articles they found didn't talk about the methodological issues we had discussed in class.  Many said they were having to do a work around, that is, they were writing about the absence of a variety of methodological proofs.  I chalked up the questions to their own jitters and assumed things would be straightened out by the time I read the assignments.  

So, now that the grading is behind me:  What did I learn? 

Well, first I learned that I live in a methodological bubble.  I live, work, and breath in a world dominated by qualitative research methodology discussions.  I think that every researcher is with me under the same big tent.  However, this is not the case.  There are many researchers out there, very happily pursuing their various non-methodological interests.  These people make use of qualitative research to meet their ends, but they are not as worried about crossing their t's and dotting their i's.  If you know what I mean. 

This means that much of the qualitative research that is out there, reported upon, in various non-methodological journals, is doing its best to contribute to the understanding of a given topic, but it may not be representative of all the whistles, bells, and frills, for which the methodological purists are looking.

My students (all of whom were reporting on qualitative research studies published in peer reviewed journals) found that discussions of methodological tasks and steps were often subsumed under a simple label.  Grounded theory was the most popular label, although that title could cover a range of meanings.  Overall the mention of research strategies (as I would call the ways we refer to approaches like Grounded Theory, ethnography, etc.) got little space. 

I had asked them to look for information about the paradigm discussions we had read about in depth in the Handbook of Qualitative Inquiry.  No one found a word about that in the articles they had chosen.  So they wrote about the absence of paradigms and the implied paradigmatic stance of the author. 

The actual act of analysis was often collapsed into a one sentence description.  "Back and forth," meaning movement between an interview transcript and coding, was a frequent way to talk about analysis--so frequent that I got tired of writing:  "What does this 'back and forth' mean?!"   

Only one student had selected an article from one of the tried and true qualitative research methodology journals, and she alone could fill in the blanks for the assignment with close to the purity for which I had hoped.   For her, it was easy, because everything was laid out based on the template I had given her.  For the others, however, the task was not nearly as clear.  Clearly, she was not the only one who had done it right.  The issue was much larger. 

What should I do? 

Hating failure, my first thought was--I'll never give this assignment again.  But this was reactionary, and, in truth, they had actually done a very interesting job on the assignment. 

My second thought was:  I will assign all the articles in future and they will only have articles in their hands that represent the way qualitative research methodology SHOULD be described.  But that didn't seem to make great sense because from what I had learned from this assignment--that's not going to be what they enounter in the real world.  It's obvious that the template created by methodologists lives and breaths in our world, but is getting limited attention from those further from this limited group of gurus. 

My third thought was:  Let's do some more shared/guided reading of articles with strong and weak methodological descriptions and talk about the difference.  This will probably be where I end up. 

The challenge of working through the "Bringing it all Together" assignment, as I called it, is similar to the earlier blog mention I made of the journal that mandated a number of interviews as representing good methodological practice.  Lacking an adequate (read: informed) discussion of the qualitative research methodology in articles submitted to peer-reviewed journals, such mandates may seem like a reasonable response.  It seems that the other response--requiring good methodological description is much harder because writers, reviewers, and editors may lack the skills to provide this or shepherd these changes. 

So, should we stop permitting qualitative research results to be reported?  Is it too much trouble to get it right?  Or should we stop permitting their publication without a full methodological discussion?  What does a "full methodological discussion" mean?  Is "full" too long to be fit into the average peer-reviewed journal article?  What kinds of methodological short-hand is acceptable in reporting on qualitative research methodology? 

I think it is time to eat breakfast--there is no quick fix for this one. 

What is a Qualitative Research Project?

This image was selected as a picture of the week on the Farsi Wikipedia for the 13th week, 2011. (Photo credit: Wikipedia)I've been pondering this question this morning:  What is a Qualitative Research Project?

I think the question may have been provoked by a sub-conscious question that I have been holding in my head for some time now, which is, Is the Journal Project over?  Bringing this to the surface...brought me to:  What is a Qualitative Research Project?  and a host of other questions, such as...

What do Qualitative Research Projects contain?
How are they similar or different?
What are the visible, as well as the invisible, products of a project?
How is the data base counted as a product?
How do you conclude a project? 

As invariably happens when I start asking myself questions...I find myself sketching out my ideas in some visual form...and then searching for ways to compare one example to another. 

I ended up comparing three projects that I have been engage in over the last several years:
1.  The Journal Project
2.  Qualitative Research and Technology Project
3.  The Sexting Project

In comparing them I realized that #1 and #3 both required the development of a classical QR data base of some sort.  #2 was more focused on the research literature and the Internet--it was a study of methodology in transition. 

None of "the projects" were discrete in terms of their construction/contents/or relationship to each other.  They all bled across each other, and yet they were extremely distinct. 

Looking at the contents of the different projects got me thinking about the lack of attention/recognition/acknowledgement we give to the data bases of qualitative research.  They are recognized when peer reviewed publications emerge, but they are not shared nor made public.  As a result they often languish in the background.  I thought the emergence of qualitative research software would change this, but it hasn't.  For the most part, while the construction of the data base may take vast amounts of effort, we don't credit ourselves for that work--we only get credit for publications. 

Sketching things out and comparing products across projects, made me realize that different products emerge from different projects, and that many good products that come out of projects are not all that visible in the standard vita.  Vita are linear and organized by product, not project.  Yet, I find that I am organizing my life and my work by project.  Good readers of vita follow the ebb and flow of projects within the description of products fairly well.  However, given the ratings of different categories of products, they "see" some pages more clearly than others. 

Here's an example...Journal Project, outputs are
DataBase
Articles
Presentations
Blog Entries
Art Exhibitions
Integration of autoethnography and arts-based research in courses taught
Appointment as Associate at the Center for Women and Work

Here's the QR and Technologies Project
Co-Authored Book
Presentations
Articles
Webinar
Judge of an Award
Blog Entries
Integration of  QDAS in various forms of Coursework
Facilitation of Qualitative Research Network on Campus and Supervision of Campus-wide NVivo License
Chaired numerous dissertations that explored use of NVivo

Each project has completely different timeline and personnel configurations.  The Sexting Project is huge compared to the Journal Project, and it is externally funded.  For the Sexting Project there are many researchers, they combine in different ways to create products.  But in this project, like the Journal Project, the creation of an original data base was an essential component.  This is not to say that a methodology project like the QR and technology project doesn't have a data base--I actually don't know yet how to express that difference in clear terms. 

So my final conclusion is that as qualitative researchers we need to do two things:
1.  Start to acknowledge the creation of an original data base as a highly significant contribution on the order of writing many peer-reviewed journal articles (and recognize the responsibility this shift entails--sharing our data bases and making that available for review).
2.  Insist upon being viewed for our whole(s)--the projects--that center our professional work.  Let us find ways to make projects and their multiple products visible at the level of professional review (tenure/promotion/job selection). 

And so the day begins...

Wikis and Electronic Portfolios

Image via Wikipedia

The snow is falling gently in Northwestern Massachusetts as voters head to the polls to choose the successor to Ted Kennedy--the liberal lion.  The semester is almost upon me, so what better way to spend the day than working on my online classes, and, in particular, on my plan for Electronic Portfolios.

I have to thank Buddy (Alan) Peshkin for this idea. As a doctoral student in his course on qualitative research interviewing, he had us create a notebook of all of our work that was set-up like a textbook with a Table of Contents and Introduction.  I still have that final notebook--it is a very precious document.  I was so intrigued with the idea that every time I take a class, I am writing my own textbook.  I found much value in having to think back over the whole as I did when I wrote the introduction to my textbook of the class. 

Over the years, I have tried to implement Peshkin's innovative idea in a variety of ways.   In the core course for doctoral students, a kind of educational foundations course, that I taught for several years, in some of the classes I had them create a similar hard-copy notebook. Students regarded those notebooks with great pride. 

As I moved into on-line teaching, I discovered a way to create a similar electronic portfolio in Intralearn (the first Learning Management System with which I worked).  While I would like to say my goal was good pedagogy, a large part of the impetus was simply trying to find better ways to manage the online texts.  My class assignments usually possess multiple components and unfold over time with peer review, and revision.  I like to be able to see where students started and the process by which they got to the end product.  This means that each assignment is really equivalent to a folder of work.  Another feature that has evolved in my classes is that most assignments build on each other and at certain times in the semester, you need to return to earlier assignments and work with that material to create a later assignment.  Thus, having access to the full corpus of the work you develop over the semester is important.  Intralearn was not designed to do this, so I had to figure out a work around, which I did by creating individual teams (I know that sounds contradictory, but that was the work around).  It worked, but barely. 

It must have looked good to someone because the description of what I was doing with electronic portfolios won a mention from the Sloan Foundation as a notable practice.  As luck would have it, however, shortly thereafter our online program switched to Web CT, which then morphed into Blackboard.  Try as I might, I couldn't find a way to create anything like an electronic portfolio in this new system, so I gave up for a few semesters.

But then wiki's came on the scene, and they are a lovely, flexible tool that can make electronic portfolios with ease.  Moreover, there are no problems allowing students access to each others' work because they are built around principles of access and collaboration, unlike Learning Management Systems which seem to be built on the model of some old prison system.  I am in heaven! 

This semester each of my three classes will have a wiki connected to our Blackboard workspace.  The wiki will be the site for the development of the electronic portfolios.  I'm still trying to figure out the in's and out's of this particular wiki system (see my earlier blog on wikis),  the right kind of shell/template for each class, and how to provide support for individual use. 

Having each individual's electronic portfolio visible to the whole class does several things, in my opinion.  First, it is a great teaching tool.  If you have a question about what is required, or what it might look like, you can look into a range of electronic portfolios and see what is going up and how others approach the same issues.  Secondly, you've got access to everything you've done in the course, which is valuable because I've found that many students do not know how to organize their work to get the full benefit of review and reflection.   Third, I think it sparks creativity as the electronic portfolios develop and become more individualized.  Fourth, your progress is visible to all--including you.  The old fashioned assignment submission system leaves everyone in the dark about who is completing what, but this system makes it visible.  If you are missing a whole section of assignments that everyone else has posted, it's not just the instructor who is aware, but the whole class.    In my classes, students need access to each others' assignments because we review drafts, etc., so the wiki makes access easy, but it also makes it clear who is falling behind.  I would call this technique:  Useful Shame.  It's a kind of silent accountability. 

So, on to the wikis.  Long live electronic portfolios in some form or another! 


For more information on Alan Peshkin, you may want to look at Vol 6, Issue 2 of Qualitative Research Journal for a special memorial section of discussion of his work. 
Special Memorial Section on Alan Peshkin

The Journal Project: Containers for Thinking

Image via Wikipedia

Douglas Harrison has an interesting piece in the Fall 2008 NEA Journal--Thought and Action.  "Scholarly Voice and Professional Identity in the Internet Age"  He blogs at  http://averyfineline.com/  and this piece is about the issues related to being a blogging academic.  It certainly spoke to the excitement and fear that I have with this endeavor--how dare I do anything without anonymous peer review!!  How dare I communicate in new containers?

Actually, blogging exacerbates the problem of containers for thinking...but for me, it didn't start there.  It's been a long slow process.  You can think of me as a Crockpot on low.  The Blog was the moment when you lift off the top of a dish that has been simmering for 8 hours.  Before you could just smell it, but now you can taste it. 

The issue of what kind of container I will use for my thinking has emerged for me on several fronts.  It has been a crucial part of my exploration of Qualitative Data Analysis Software (QDAS or CAQDAS, which ever you prefer) and the notion of the E-Project, that little womblike electronic container that these software create for one to stow research materials in and to grow interpretation.  It is also an issue with my investigation into Arts-Based Research, which forces me to ask myself--what are other ways (containers) that I could use to express my thinking?  How are these different from ye old academic/social sicence genre?  What happens when I start using a different kind of container?

The Journal Project

These two ideas merge in the Journal Project.  Post-tenure I came down with a familiar academic malaise characterized by a sense of surprise at the positive outcome of the tenuredecision and a feeling of survivor guilt--why me?  As I cast around for answers, I wrote--I am a perpetual journal writer.  I write every morning and have done so for years. The writing has always been totally for me, a way to center myself, understand the world in which I move.  It is a place to explore the past and try to discern the future.  It's addictive.  I feel strange when I don't do it --like starting the day without a cup of tea, I can't seem to wake up properly without it.

I'm not sure now how it happened, but the use of QDAS and the interest in Arts-based research came together in the Journal Project.  I decided to enter two years of my journals--from the receiving of tenure in August 2006 to two years from that point--into my QDAS of choice--NVIVO.  I would analyze it there and see what I learned from studying myself from a distance with a tool devoted to the interpretation of qualitative research texts.  It was a mixture of autoethnography, self-study, document analysis, middle age angst, and several other approaches.  It goes slowly--like the proverbial Crock Pot--I devote about two hours a week specifically to this project.

In undertaking this study of self through documents one thing I have asked of myself is that I stretch the ways I interpret and represent what I am learning --within and beyond the confines of the alphabetic text.  As I type and code the entries (yes, I write my journals in longhand in composition books--what could be more retro), I seek out naturally occurring poems and stories that I separate out to review and work on at another time. 

No surprise to me, this being a  journal, the codes are highly emotional in nature.  It was a painful time during which I wrestled with hard personal issues.  I imagine, at some point, extracting the coding tree and working it into a piece of art.  Perhaps I should do some code weaving a la Johnny Saldana.  (Johnny Saldana in Methodspace)

But what has been most intriguing is trying to use methods of visual art to better understand the issues that are arising for me.  As a person who works with fiber and paper...in collage, mixed media, spun fiber, and felt, I have challenged myself to use the visual as I would use a memo in qualitative research....and to see what this does for me, and how it helps me to see the material.  I also ask myself to incorporate the ideas that are emerging from the journal project into different art pieces.  This has created pieces that are different from visual memos, but equally memorable in their own way. 

While I am enforcing this new method on myself, I have also been using it with students.  Each semester I try to find new ways to expand the possibilities available for data collection, interpretation, and representation using a range of arts-based methods.  

As time goes by I want to share my work in this area with the blogosphere...maybe tomorrow.