Artwork

Nội dung được cung cấp bởi AERA Qualitative Research SIG. Tất cả nội dung podcast bao gồm các tập, đồ họa và mô tả podcast đều được AERA Qualitative Research SIG hoặc đối tác nền tảng podcast của họ tải lên và cung cấp trực tiếp. Nếu bạn cho rằng ai đó đang sử dụng tác phẩm có bản quyền của bạn mà không có sự cho phép của bạn, bạn có thể làm theo quy trình được nêu ở đây https://vi.player.fm/legal.
Player FM - Ứng dụng Podcast
Chuyển sang chế độ ngoại tuyến với ứng dụng Player FM !

Episode 26: Episode 26: CAQDAS

49:57
 
Chia sẻ
 

Manage episode 294233895 series 1941203
Nội dung được cung cấp bởi AERA Qualitative Research SIG. Tất cả nội dung podcast bao gồm các tập, đồ họa và mô tả podcast đều được AERA Qualitative Research SIG hoặc đối tác nền tảng podcast của họ tải lên và cung cấp trực tiếp. Nếu bạn cho rằng ai đó đang sử dụng tác phẩm có bản quyền của bạn mà không có sự cho phép của bạn, bạn có thể làm theo quy trình được nêu ở đây https://vi.player.fm/legal.
In this episode, Amir Michalovich, PhD Candidate at University of British Columbia, interviews Dr. Christina Sliver on Computer Assisted Qualitative Data Analysis. They cover a wide range of issues and topics within CAQDAS, share numbers resources and recommendations, and talk at length about how graduate students might benefit from CAQDAS. The follow is the transcript of their conversation.
Amir 0:25
Hello, everyone, welcome to qualitative conversations, a podcast series hosted by the qualitative research special interest group of the American Educational Research Association. I'm Amir Michalovich, a member of the graduate students committee of the qualitative research special interest group, and a doctoral candidate in the department of language and literacy education at the University of British Columbia. As a guest podcast host. I'll be speaking today with Dr. Christina Silver on computer assisted qualitative data analysis, otherwise known as CAQDAS with a specific view of how graduate students might employ it, the kinds of challenges that they might face and some of the ways to address those challenges. Dr. Christina Silver manages the cognitive networking project based in the Department of Sociology in the University of Surrey in the UK, for which she leads the training and capacity building activities. She's the co founder and director of QDAS Qualitative Data Analysis Services, which provides customized consultancy services for individuals and groups engaged in qualitative analysis. She has many years of experience teaching CAQDAS, and has written extensively on the learning and adoption of CAQDAS. Christina is co author with an Lewin's of the book using software and qualitative analysis. And with Nick Wolf Of the five level QDA method. She has also published key articles and book chapters exploring the relationship between qualitative or mixed methodologies and technology, specifically, the use of dedicated contest packages. Alright, well, thank you, Christina, for joining us for this podcast episode. I am absolutely thrilled to speak with you today. I'd like to start with a basic question about CAQDAS. What is CAQDAS? And why should graduate students consider using CAQDAS software?
Christina 2:16
Okay, so Hi, thanks a lot for inviting me, it's great to have this chat. So CAQDAS is an acronym that stands for computer assisted qualitative data analysis. So it's used to refer to software and other applications digital tools that have been specifically designed to facilitate qualitative and mixed methods analysis. It's an acronym that was developed in around 1991, by Nigel fielding and reily, after they convened the first conference, looking at software to facilitate qualitative analysis. So now it's used as an umbrella term to relate to all of these digital tools of which there are now dozens available. The thing about CAQDAS packages in terms of their use by students, graduate students, also undergraduate students and other researchers is that they are one of the kind of tactics that we have available at our disposal to operationalize our analyses. There are some debates about their use. And that's something that we'll probably touch on later on in our discussion. For me, you know, it's really important at the outset to realize that, although there are many ways that CAQDAS packages can facilitate analysis, they can help us organize our data, they can help us access different aspects of our analysis process, it's still possible to do bad analysis using CAQDAS packages, just like it's possible to do good analysis without using them. So that's a really important starting point in thinking about whether and how to use these kinds of tools. But for me, you know, really, the main thing is the access that it gets to the process. So using a dedicated CAQDAS package gives us access to the materials that we're working with. And that will be the data that we're working with the qualitative data, but also, other supplementary materials that form the context of a given study also gives us access to the ideas we have about what's interesting and meaningful in our data, allowing us to get back to those earlier thoughts. And also really importantly, in terms of process, it gives us access to the history, the journey of our analysis process. And for students, that's really important because when we're doing a dissertation, it's often as much about how we went about doing it and the lessons that we learn from that as it is the findings that come out of our research. So it can help in various ways, but it's also a useful skill set to have. I think when you're learning about qualitative methods, generally, learning about the technologies that are designed to facilitate the process is really useful for anyone who wants to have a research career after their studies. Were there That's within academia or outside of academia, because many qualitative and mixed methods, researchers are using these tools for their work. Now,
Amir 5:08
that's fascinating. And I think that the notion of access is really important here in the sense of having that possibility to easily retrieve pieces of your data. And also, how do you conceptualize that data in different stages during the project. So we know that there's this growing need for graduate students to be familiarized with CAQDAS packages, but there is some confusion sometimes about how CAQDAS software is used, whether it's a method of analysis or something that kind of supports analysis. So why do you think people sometimes confuse CAQDAS software use with a method of analysis?
Christina 5:45
Yeah, a good point. And something I think that continues to be discussed, even after all of these many years since these tools have been available. For me, I think there's two interrelated reasons for the kind of confusion between the software and the method of analysis. First of all, a misunderstanding or an assumption that the software does the analysis for us. And secondly, unsubstantiated and outdated criticisms about the negative effects of using CAQDAS packages, in terms of the kind of craft work that qualitative analysis involves. So these kind of misunderstandings and criticism started in the earliest moments of the availability of CAQDAS packages. But despite a fairly large body of literature now that dispels the notion that CAQDAS is a method of analysis, these kinds of misunderstandings and criticisms are still being perpetuated informally by some teachers of qualitative methods who are not kept as users themselves. But also formally in the literature, we see these misunderstandings and criticisms promoted, and that's often also by non users of these tools. So, as a caveat before I, before I carry on and talk about that a little bit more, you know, I'm not saying that everybody should use them, or that it's wrong not to use them, it's just understanding the role of them is really important. I guess the assumption that CAQDAS does the analysis is the thing to think about first. So for me, there's kind of two differing reactions to the realization that the software doesn't do the analysis. First are those who are saddened that it doesn't do the analysis, you know, some people are looking for a shortcut to accomplishing their work, and they want the software to do the analysis. So they're disappointed when they realize that that's not the case. So technology is developing really fast at the moment. And there are now some CAQDAS packages that incorporate AI technologies, such as machine learning, and therefore provide a lot more assistance than than was the case a few years ago. But at the end of the day, it's always the user, the researcher, who decides what to do, who decides when to do it. And who decides what it all means. So interpretation, differentiating interpretation from analysis, I think something that it's really important. But on the other end, sometimes there are people who are kind of outraged at the idea that kept us would do the analysis. So that's the other the other end of that continuum. So those are the researchers who really price the human interpretive processes, which underlie many approaches to qualitative data analysis. And it tends to be the this group of researchers who criticize CAQDAS because they think that that the software is taking over in some way. And therefore, that's where the criticisms come from. So it's interesting to me that that idea that the software does the software do, the analysis is understood very differently, depending on our understandings and engagements with the tools. So I guess the other thing I just wanted to say is that some colleagues of mine, Christy Jackson, Trina Paulus, and Nick Wolf, they wrote a really excellent article on the perpetuation of unsubstantiated criticisms of CAQDAS that was published in 2018. And they look at four different criticisms and kind of debunk those, but also look at how the literature kind of perpetuates those ideas. And I think it's really interesting to, you know, reflect on those criticisms. And you know, if you become a user of a CAQDAS package, to understand the context of those debates, so that you can place yourself within that context, and sort of justify your use of the software.
Amir 9:36
Amazing. I think there's so much importance in understanding that context, and also and thinking very carefully about how we approach the role of CAQDAS in our work. What I found so interesting about your work with colleagues as well, is that you've really tried to unpack different ways in which we can think about that the role of CAQDAS in our work, how we can approach it and how we can operationalize it. And I've noticed that in your work in terms of First in terms of how we think about the kinds of skills that we need to attain when we work with when we want to work with CAQDAS, or the kinds of practices that we need to engage with. You mentioned three factors, among other factors that are particularly important in the learning and adoption of CAQDAS software. You mentioned in your work, methodological awareness, analytic adeptness, and technological proficiency. Could you unpack each of these a little bit and why they are important for graduate students conducting CAQDAS?
Christina 10:33
Yeah, absolutely. So those three factors come from some research that we did, looking at how new users learn about software, what the challenges are, and also how they adopt technologies in their practices. So methodological awareness, first of all, is to do with the familiarity with the variety in qualitative data analysis philosophies, and methodological approaches. And they underpin the choices that researchers make in undertaking analysis. So having a methodological awareness is the kind of overarching kind of starting point, I guess, in thinking about CAQDAS and adopting it, and it's important method, this method logical awareness, because qualitative research is so broad and so diverse. So the choices that we make about how to go about an analysis really need to be made carefully, and then need to be justifiable within the context of the broader qualitative research field. So I've often observed confusion around the terms methodology and methods, and with students often being very overwhelmed or confused about qualitative methodologies, and unclear about how methods relates to methodologies. So I spent quite a lot of time trying to talk about this with with my students. And the simplest way I found to explain it, I guess, is to say that methodology is a description or a roadmap of how a project will be undertaken, how the research questions will be addressed, I guess. So methodology describes how the entire project will be conducted. And it kind of provides the criteria for designing or selecting the methods. And methodologies are informed by two quite scary words for students ontology and epistemology. And they're used to think about our philosophies or our paradigms or our assumptions that we all bring to our research. So ontology, you know, in simple terms, is our view of the nature of reality. Whereas epistemology is to do with our perceived relationship with the knowledge that we're uncovering or discovering. So being aware of methodologies is really important, because it provides that description of how you're going to go about your study at a high level. And that informs the selection of your methods. So what are methods then methods are the data collection methods, so whether we're undertaking interviews or focus groups, or generating data from online interactions, or surveys, or asking participants to generate visual materials, etc. And then we also have analytic methods, which are how we're actually going to go about doing the analysis. And there are many different analytic methods, thematic analysis, interpretive phenomenological analysis, content analysis, discourse analysis, those are all examples of analytic methods. And so what one of the challenges is, is understanding how those methods, the data collection methods, or the analytic methods relate to the broader methodologies. And that kind of basic framing and understanding is really important, because it leads to how we use the software. So that's the first factor methodological awareness. The second one is what we call analytic adeptness. So this has to do with the experience in undertaking qualitative data analysis. So specifically, the skills in designing an analytic task in the context of that underlying methodological context. So it's to do with what we actually do when we analyze qualitative data, what we do, how we do it, the sequence in which we do it. And this all depends on the analytic methods that we're using. So those tasks would be different if I was doing a thematic analysis from a content analysis or a discourse analysis, for example. So some analytic methods are more prescriptive than others in terms of the sequencing and the actual tasks that are undertaken. So I kind of think about that in two ways. I guess, first of all, the kind of what I call the strategies level, which has to do with planning what we need to do and that's different from the The tactics level, which is how we actually go about it. So analytic tasks, deciding what to do is kind of at the strategies, the planning level, and how we accomplish it at the tactics level is where the software potentially comes into play. Because a CAQDAS package and the tools within it can be the tactics that we use to undertake those strategy level tasks. But of course, cactus packages are just one of the tactics that we have at our disposal. So you know, many people, before CAQDAS packages were around, we worked manually, right with pen paper, my first qualitative project, I cut up my transcripts with scissors and put bits of transcripts in different piles and had a big matrix on my wall. Other people are using general purpose software, Microsoft Word or Excel or other tools. So these are all the tactics that we have to undertake our analysis. And that's where analytical adeptness kind of leads into that third factor that you mentioned, which is the technical proficiency. So analytic adeptness really straddles from the methodological awareness to the kind of planning what we actually going to do. And then the next struggle to the technological proficiency is when we're using tools, digital tools, or any tools, how much competency we have in operating the software, and also the comfort that we have, with the idea of experimenting with the software, without fear of making mistakes. I think that's one of the key challenges that I see in new users of CAQDAS packages. In that often students are learning about qualitative methodologies, and software tools at the same time. And that can be really, really challenging. So many cactus packages, one of the best things about using a dedicated CAQDAS package is that it's incredibly flexible. And they offer dozens and dozens of features. But the challenge for the learner is that not all of those features are going to be useful for an individual analysis. So the learner has to navigate that all of the technological possibilities at the same time as thinking about which features are going to be useful for their analytic needs. And that's a really big challenge. So, you know, often, I see, in many of the workshops that I run, I see students being really worried that they're going to mess things up. So being scared that, oh, if I do this, then something might go wrong. And that's, that's often happening in the early stages of learning, learning a cactus package. So technological proficiency is about learning the architecture and the functionality of the software, and how to operate it. But doing so in a way that develops confidence to experiment and be creative about how we use the tools, because there's no one way of using any of these tools, just like there's no one way of undertaking a particular research project. So those three factors, I think, are really important in learning about and successfully adopting CAQDAS, methodological awareness and analytic adeptness are really closely related. And they're to do with the strategies that we put in place, whereas technological proficiency is at the level of operationalizing, those strategies using tools to accomplish our tasks. So despite the sense in which those factors are interrelated, and how they straddle one another a little bit, I do think that thinking about them separately, really helps students to understand the sense in which cactus packages are not methods, but tools. And hopefully, to see that learning about what to do is different from learning about how to do it. those are those are really important, but they are different. So differentiating between them, I think, is a useful framework.
Amir 18:52
And that is really important points. And I think they connect directly to your some of the work that I think you're kind of well known for, which is the work on the five level QDA method, where you really talk about the importance of thinking, what is the kind of strategic approach that I want to take to my data analysis? And how might I translate that using the particular features of the software that I'm thinking of working with, and perhaps combined different software features in creative ways or useful ways that are particular to the context in which I'm working? So you've really connected to that already. But I wonder if you want to say a few words about what is the five level QDA method and what what it might be useful? For?
Unknown Speaker 19:34
Sure. Yeah. So I think that leads on from from what I was talking about before, in terms of those three factors and our work with the five level QDA method built on our earlier work as well. The first thing I guess, to say about the five level QDA method is that it's a method of learning and teaching practice, not a method of analysis. Okay, so it's not an equivalent or replacement for thematic and analysis or grounded theory, or discourse analysis, or anything else is the pedagogy. Really, and we think about it as a mindset, a way of thinking about the role of CAQDAS packages in our work when we're undertaking any type of qualitative or mixed methods analysis and using any kind of digital tool. So our focus in the development was around three of the most well known practice packages, but it's, it's relevant to any digital tool, whether it's a dedicated CAQDAS package or not. And we developed it because we had observed over our many years of using and teaching these programs, and also talking with other colleagues that there were a number of key challenges with being able to kind of harness the software tool for the particular analysis need, if you like. So we did some more research into that. And basically, we unpacked what happens in the minds of those researchers who have become really proficient expert users of these CAQDAS packages, we've unpacked their unconscious processes, what they do without thinking, as they have become more proficient in the in the use of the software for their analysis, unpacking that, so that new users can develop the expertise that they need, without the many years of trial and error that myself and my co author on five level QDA Nick Wolf, you know, we both went through many years of trial and error with these programs before we developed the level of understanding and proficiency that we now have. So one of our key aims was to try to enable that learning to be shorter and more efficient, by really understanding those unconscious processes and unpacking them and making them clear. So it's really a framework for for thinking about the role of CAQDAS, and enabling new users to be able to harness in a powerful, but also a quick way, what they need to do in terms of students. They're doing dissertations, they have a short amount of time, they're learning multiple things. At the same time, as we've sort of touched on earlier in our discussion, Amir, and, you know, we wanted to enable them to adopt successfully their chosen CAQDAS package within the time constraints that they have. And I guess the key principle, which relates in part to what I was talking about earlier, is that our analytics strategies are different from our software tactics. Okay, so strategies are what we plan to do. And tactics are how we actually go about doing it. Okay. And related, again, I guess, to the idea that or the misconception that the CAQDAS packages are methods is that strategies are developed, outside of the context of the tools that we use, and really need to drive the way that we use the software, I have to choose the appropriate tactics for the task, the strategies, tasks that I have, in my mind right now. And that's the way that we like to think about how to operationalize our analysis using software. So in the context of qualitative research, our strategies are the objectives and the methodology and the analytic tasks that we develop in order to accomplish our objectives. And they are, you know, they are not driven by the software that we decide to use. They're accomplished through the use of our software. distinguishing between strategies and tactics is something that's not often or clearly done in the in the methodological literature. And often those two terms are conflated or used as if they're synonyms. So the basic underlying principle of the five level QDA method is recognizing that strategies and tactics aren't the same, and that strategies must drive our use of software.
AD 24:12
The qualitative research special interest group was established in 1987 to create a space within the American Educational Research Association. For the discussion of ethical, philosophical and methodological issues in qualitative research. We invite you to consider joining the qualitative research SIG today. for members of a era. The annual fee for joining qualitative research special interest group for regular non graduate student members is $10. And the annual fee for graduate students is $5. As members of the QR SIG, you will gain access to a network of fellow qualitative scholars, as well as our many activities ranging from mentoring opportunities to our podcast series. To update the news related to recent qualitative publications and jobs. Please visit the American Educational Research Association website at www dot era dotnet During the qualitative research sake today,
Amir 25:03
Right, WIWells I've engaged with that work quite at length. And I found it very useful in my training or work with graduate student peers. But one of the things that I've experienced is somewhat difficult at times, is to really try to find a concrete way to unpack how different methodological approaches might look like how they might differ from one another, as this first step, before we kind of translate them into software tactics or other kinds of tactics that may not even necessarily involve software. Sometimes grad students may not necessarily have the sufficient training in how these different approaches might even look. So that's an issue in itself. So I wonder if you could share with us one example, perhaps in which one methodological approach might significantly differ from another in the analytic strategy that it entails?
Christina 25:57
Yeah, so I guess when I talk about this, I mean, this, I think you're right, that this is something that's difficult to make tangible, in a car in a kind of concrete way. And I guess that comes, you know, for the user of a CAQDAS package that comes with experience, and with that experimentation, and that creativity that I mentioned earlier, but I guess it's also partly to do with breaking down the analytic strategies. So what we do, so if we think about analytic strategy of grounded theory, and then contrast that with classic quantitative content analysis, then, you know, I talk a lot about directionality of analysis. So, in broad terms, grounded theory approach is about generating theory from a body of data that we have in front of us. Okay, so generally speaking, we're working in a kind of bottom up way, in an inductive way, that's often described as I think in a grounded theory approach, often we do some more top down some more deductive work as well. But the general directionality of our work is starting with the data and generating an interpretation or theory from that body of data. Whereas their content analysis, content analysis is as tricky. A tricky term, actually, because it's understood quite differently in different disciplines and different national context I found as well. But content analysis classic quantitative content analysis, is about looking at instances of key words, phrases, and other structures or patterns in texts, which can be written texts, or can be visual texts, and counting those instances and potentially doing statistical work with those instances. So a content analysis can work either bottom up or top down, depending on whether we have a hypothesis that we're trying to test, some content analysis starts with a hypothesis. And our techniques are about testing that hypothesis. But we can also do a quantitative content analysis in a more inductive way, a more exploratory way, especially these days, when we have so much qualitative data at our disposal from the internet that we can harvest in all kinds of different ways. A classic example might be Twitter, data, right. And if I, if I gather 10s, of 1000s, potentially millions of tweets, then I don't know what's in that data. So I may do a content analysis in a much more exploratory way. So for me, it's, you know, it's not so much about saying, oh, in a discourse analysis, you do this, then this, then this, then this, whereas in a grounded theory, you do this, then this, then this, then this, it's more about the individual tasks that we undertake. And in the five level QDA method, that directionality is something that's kind of quite important to reflect upon. But specifically, we break it down to what we call an analytic task, which is the smallest unit of what we do the smallest unit of analytic activity, and they are specific to each individual project. But I believe that there are commonalities in analytic tasks across the different methodologies. So what we do is to try to, to think and teach how to harness CAQDAS packages at that level, the methodology and all of that background that we talked about earlier is really important, because that's what leads to specific analytic tasks, that at the moment that we accomplish a task in the software, it's just that task in and of itself, that's the important thing. So what is an analytic task? There's many examples. We've got many examples in our publications, but, you know, one task might be to read my interview transcripts to identify potential concepts for coding That's an example of an analytic task, a small unit of what we need to do. Now in any CAQDAS package, there are multiple tools that I could use to accomplish that task. Another task might be to compare theoretical and emerging concepts and explain their similarities and differences. And that may be a task that's relevant to all kinds of different methodologies. So I guess for me, the contrast, are the examples of methodological approaches that differ from one another in terms of the analytic strategy. Yes, I think that's an important thing to reflect upon. But I think it becomes concrete in terms of examples and learning, when we take it down a level and think about, okay, what is it that I actually need to do, and break that down to the smallest level, and then it's much more manageable to build up the activities more generally,
Amir 30:57
you've discussed so far, how important it is and your work, especially how important it is that notion of directionality in the sense of sort of planning ahead, what you're thinking of doing, and we've talked about how sometimes there are some gaps in terms of knowledge about what it is that we're supposed to do with certain approaches. But I wonder now, if in your experience, are there any interesting ways in which the work with the software is somehow influenced or somehow informed, analytic strategies? Or how we understand certain approaches, certain methodological approaches? Because I feel like it's such a fascinating area of inquiry?
Christina 31:36
I think it's a great question. And it's something that I've spent quite a lot of time thinking about, whilst we were developing the five level QDA method. And also since when I've been talking about it, and using it in my own work, and some of the conversations we had with our colleagues and other researchers using CAQDAS packages, really speak to this idea. And I kind of guess I've realized that there's a really important difference between what we say, which is that strategy should drive tactics. And that that's not the same as saying that tactics don't inform our strategies. So, you know, I've had some really interesting conversations with some colleagues who have said, Yeah, but there are things that are possible in CAQDAS packages that just aren't practically possible if you're not using a dedicated tool. And I definitely agree. So that is a way in which the availability of software tools, their affordances can change the way that we think about going about our analysis. But for me, that's an informing process, not a driving process. So let me give you an example that will hopefully bring what I mean to life a little bit more, I had a student come to one of my advanced software workshops A few years ago, who'd been using a CAQDAS package to do a literature review, because they've read and heard that it's really great, you can do that review, using these tools. And she'd come along to this advanced workshop because she was in a bit of a pickle. And she was like, she said, You know, I've done all of this work, and I can't see how it's helping me. So we looked at her project, and she had 1000s of codes in her software project. And upon looking at her project, and talking to her and asking her what she'd done, it became apparent to me that she hadn't read any of the literature. She had used tools like word frequency tools, to tell her what words were in the literature. And then she'd coded every sentence or every paragraph in which highly frequent words came up, imagining that that somehow was going to tell her what the literature was saying, imagining that she there that her use of the software would mean that she didn't need to read the literature that it would tell her what was going on in her materials, right, which speaks back to what we talked about right at the beginning about CAQDAS not being a method. Yeah. But in terms of this idea of driving and informing she was looking to the software to tell her what to do, and also to tell her what was there. And that's, you know, we're long way off software being able to do that. So, you know, I think there are certain ways in which the availability of tools can inform our thinking about what's possible and may change some of our processes, but the availability of tools should be used to inform our strategies not to drive our strategies. And I think that's, you know, that's a kind of semantic difference between those two terms. But I think that's a really important one.
Amir 34:50
What's very interesting to me and kind of how our conversation evolved here is that there seems to be seem to be two different areas in which we might see live or work on qualitative data analysis kind of moving forward into, it seems one direction is trying to really maybe based on the work of others, like Joe Maxwell, for example, who has written about the theory of qualitative data analysis, perhaps we could sort of advance this field and thinking about what might be some of the similarities across different methodological approaches, and what might be some of the important differences, that might be a useful way to kind of advance both our thinking and graduate students training in how you know, those approaches might be unpacked and translated to different kinds of tactics, including the use of software. But the other interesting point that you raised that was that interesting distinction between the sense in which software might drive our our analytic work, as opposed to informing our analytic work. And I think that's a really important area to consider. And thinking about qualitative data analysis with software
Christina 35:58
is to deal also with a set with serendipity isn't it, it's like, software, the use of software tools can tell us things unexpected that we weren't anticipating or looking for or, and that's, that's, that's a great thing, you know, and one of the things about familiarizing with our data and getting a real in depth understanding of what's going on, which is a key to many different approaches, software can really help with that. And you know, that balance, I think you actually say something that, that we should be reflecting on war.
Amir 36:31
And that sense of serendipity really connects to what you also write about quite a lot in your work about the iterative and emergent qualities of qualitative data analysis, right? So yeah, these things are coming to twined, in interesting, fascinating and yet to be explored ways. This leads me to the last two questions, I'll start with one question about the kinds of challenges that you might have seen, whether it be challenges that students experienced, or other kinds of researchers with translating analytic strategies and tasks into software tactics. So if we were to sort of go down that road as grad students, what kinds of challenges do you think we should expect?
Christina 37:11
Okay, so there are two key things, I guess, to say. And the first one, I think, has to do with what we were just talking about when you mentioned the sort of iterative and emergent nature of qualitative and mixed methods analysis, and that idea of thinking ahead and planning. So sometimes a reaction to the five level QDA method, and the emphasis that we put on planning is this idea that Yeah, but you know, quantitative research evolves, we don't know exactly what's going to happen, and you're trying to put a structure on it that isn't really there. And my and I think that's, you know, a fair point. But the way that we would talk about that is that, to varying degrees, all of our processes are iterative, in an emergent, some methodologies are more intuitive and emergent than others. But that doesn't mean that we don't plan what we do, it just means that our plans change, as we see new things and come across new things. So the planning aspect is something that I see students find real challenges with, because they haven't always thought that connect with that methodological awareness and analytic adeptness that we were talking about earlier. So they don't really know what to do, what should I do first? And it's kind of like, Okay, well, if you start to plan, what you might do, and then think about what that would allow you to do next, and then come up with another scenario, well, I could do it this way. And then what would I be able to do, and that helps you to see how important planning is. And I've really, you know, seen in with many of my own students, you know, the value of planning in terms of fair reflection back on it, later on. And so we have these analytic planning worksheets that we developed as part of our methods, which we developed as a means to facilitate the learning of software, but has actually unanticipated consequences of us developing those is that people are using them to document their process. And that speaks to transparency, and rigor, and all of those things that qualitative researchers historically haven't been very good at, you know, explaining in concrete terms, how I got from my pile of data to my interpretation is something that historically we're not very good at. But planning and doing that in a structured way, and allowing those plans to evolve and documenting that it becomes very useful. So I guess the other thing I just wanted to mention briefly is, is units so as I'm sure you're aware, we talk a lot about units in the five level QDA method. And this is something that quantitative researchers typically are pretty clear about units because they normally use units to determine their variables that they use for their statistical work right? But in my experience, qualitative researchers don't really think very clearly about units and are often not taught about units but units are fundamental to qualitative analysis. And we distinguish between units of analysis. So they're the kind of main entities that form the basis of our research. So that might be groups or individuals or organizations, for example, units of data. So the form that the data is actually in, so that might be a transcript, it might be a sentence, it might be a discussion, it might be an interaction, and then units of meaning, which are kind of related to units of data, but are to do with which bit of data do we need to capture in order to be able to access the ideas that we've had. So units is a difficult concept, I think, for students to come across. But in my experience, that the, you know, I've seen those kind of lightbulb moments amongst students most often when they get what units are, and why why they're important, generally, in their research design, but also in terms of how they translate their tasks into those software tactics.
Amir 40:58
Think I felt that light bulb moment when I've reflected on what is that selection that I'm making in software that I'm going to continue working with in terms of coding and connecting to memos, etc. So yeah, these are really important points. I also wanted to connect this to something else that you mentioned before, how sometimes graduate students may learn software tools and methodology at the same time as a challenge. But at the same time, you also, you've also written about in other places, I believe that there is sometimes a disconnect between the learning of software and the learning of methodology. I wonder if you could speak to that? Because those are, you know, on the one hand, the there's a problem with learning them at the same time. But what about that disconnect? Is that related somehow?
Christina 41:44
Yeah, this is, this is a topic that's, I'm thinking a lot about at the moment. In terms of, you know, there's different opinions about whether you should teach and learn methods, strategies, methods first, and then the software, or whether you can teach and learns the two together. And, you know, in my opinion, there are kind of three ways of thinking about about that. One is the kind of the sort of sidelining or ignoring of technology and just teaching the methods. Yeah. Which I think is does the next generation of researchers and students a great disservice. So you mentioned earlier that students expect to use digital tools these days, of course, they do, because their whole lives, for many of them are embodied by the use of digital tools. So they can't understand why you wouldn't think about using digital tools for qualitative analysis often. So those teachers who are not teaching them or even making them aware that there are digital tools, which I think happens far too often still, I think that's, I think that we're, you know, we're not being responsible as teachers, if we take that approach. And then then the, you know, then there are two ways of thinking about this in terms of teaching methods and technologies, and then how and when to do that. So there's the kind of methods first, so teach a range of qualitative methods or a specific method outside of technology, and then teach the technology and sometimes that's bolted on and isn't integrated. So it's like, here's the methods piece, okay, now go off and learn about the software. And there's not enough connect between the two, which is the disconnect, I think that you're you're referring to, another way of doing it is teach a little bit of methods, and then teach how that can be operationalized in software, and then teach another bit of methods and how that can be operationalized in the software. So that's what I would call methods into methods and technology interwoven. But then it's also possible and there are people who are teaching methods via software. And in the collaborative context, that's really useful. So tools that allow multiple students and researchers to work on the same transcript, for example, at the same time, so coding is a good example, okay? You have a piece of data, and then you ask students to code giving more or less direction for how or what they should be coding, and then you compare the coding and then you have discussions about Oh, why did you code it like that? I coded it like this. And then you're teaching what coding is, what the methods of coding is via the software. So there are different ways of interleaving or managing that potential disconnect. And I'm not sure that that methods first methods into work woven or methods via software, I'm not really sure that one or other is better. They're just different, but they're all possible. And they're all better than just not doing it at all. I think.
Amir 44:39
I think we can definitely agree on that one. All right. Well, we're moving toward the end of our podcast episode. And the last question I have for you is just if there are any resources that you would recommend to grad students who are looking to explore analysis or analytic strategies of different methodological approaches, one kind of deal Deep into some of the concrete ways in which they could think about their data analysis.
Christina 45:07
So there are lots of textbooks out there. My favorite textbooks, I guess, are the ones which which taught me back in the day. So Miles and Huberman then in the 1994 textbook remains, you know, really close on my shelf. And I draw on that still now. I also really like Harry Walcott's work. More recently, Patricia Bazeley. Her work her textbook on qualitative data analysis. I think it's excellent. So in terms of textbooks, those would be the three main authors that I would say, look at those first, there are loads of others really good authors around qualitative data analysis techniques as well. But you know, if you're asking for Okay, what's the starting point, then then I'd say Miles and Huberman, Walcott and Baseley. But there's also quite a lot of online resources now. So there's a website called Methods Space, which is pretty active and has lots of resources, which are curated around different topics. And this year, at the moment, they're going through the kind of whole research cycle, and each month focusing on a different topic. So I think at the moment, the topic is ethics. And previously, they did a whole topic on research questions, etc. So methods spaces are really great websites to tap into. And that's related to the Sage Publications, methods map. So there's lots of resources there, and really good links to online and offline publications, etc, around particular key methods. There's also a resource called Online QDA. So online qualitative data analysis, which was a project which finished quite a few years ago now, but was designed for graduate students to help them navigate qualitative data analysis. And what's really nice about that site, I think, is the methodologies area where there is a clear and short definition of different methodologies. And then there are links to further resources. So the key publications to read about each of those methodologies or methods. So that's really useful, I think, for students because, you know, if I need to learn about a particular methodology, grounded theory, say there's so much written about grounded theory, how do I know which of the things I should read and the Online QDA website has kind of done that job for students and said, Okay, this is what that methodology or this is what that method is, and these are the key things to read. So I think that's really, really useful. And the final thing I'd like to say is that learning about analysis and learning about cactus is really usefully done while still doing a literature review. So literature is a form of qualitative data writes text and embedded images, okay, reviewing is a form of analysis. Okay. And we can really usefully learn how to do qualitative analysis, whilst doing a literature review in our in our chosen CAQDAS package. So that's something that I think, is useful for all students who are doing dissertations who need to do a literature review. And there's a book by Wallace and Wray, Critical Reading and Writing for Postgraduates, which is a really, really excellent book, which talks about literature reviewing, not in the context of cactus packages, but in the context of understanding how to read critically how to discern whether what you're reading is of good quality or how it relates to your work, etc. and then how to write critically, which is something that I think students also struggle with in their dissertations.
Amir 49:00
All right, well, Christina, that was a fascinating conversation. And you've just given us so many useful resources on top of a really insightful and rich discussion. So again, I want to thank you for taking the time to participate in this podcast. And I hope you continue to do that important work that you've been doing in moving this field forward. So thank you for your time, and hope to chat again sometime soon.
Christina 49:28
Yeah, many thanks. I enjoyed it.
  continue reading

44 tập

Artwork
iconChia sẻ
 
Manage episode 294233895 series 1941203
Nội dung được cung cấp bởi AERA Qualitative Research SIG. Tất cả nội dung podcast bao gồm các tập, đồ họa và mô tả podcast đều được AERA Qualitative Research SIG hoặc đối tác nền tảng podcast của họ tải lên và cung cấp trực tiếp. Nếu bạn cho rằng ai đó đang sử dụng tác phẩm có bản quyền của bạn mà không có sự cho phép của bạn, bạn có thể làm theo quy trình được nêu ở đây https://vi.player.fm/legal.
In this episode, Amir Michalovich, PhD Candidate at University of British Columbia, interviews Dr. Christina Sliver on Computer Assisted Qualitative Data Analysis. They cover a wide range of issues and topics within CAQDAS, share numbers resources and recommendations, and talk at length about how graduate students might benefit from CAQDAS. The follow is the transcript of their conversation.
Amir 0:25
Hello, everyone, welcome to qualitative conversations, a podcast series hosted by the qualitative research special interest group of the American Educational Research Association. I'm Amir Michalovich, a member of the graduate students committee of the qualitative research special interest group, and a doctoral candidate in the department of language and literacy education at the University of British Columbia. As a guest podcast host. I'll be speaking today with Dr. Christina Silver on computer assisted qualitative data analysis, otherwise known as CAQDAS with a specific view of how graduate students might employ it, the kinds of challenges that they might face and some of the ways to address those challenges. Dr. Christina Silver manages the cognitive networking project based in the Department of Sociology in the University of Surrey in the UK, for which she leads the training and capacity building activities. She's the co founder and director of QDAS Qualitative Data Analysis Services, which provides customized consultancy services for individuals and groups engaged in qualitative analysis. She has many years of experience teaching CAQDAS, and has written extensively on the learning and adoption of CAQDAS. Christina is co author with an Lewin's of the book using software and qualitative analysis. And with Nick Wolf Of the five level QDA method. She has also published key articles and book chapters exploring the relationship between qualitative or mixed methodologies and technology, specifically, the use of dedicated contest packages. Alright, well, thank you, Christina, for joining us for this podcast episode. I am absolutely thrilled to speak with you today. I'd like to start with a basic question about CAQDAS. What is CAQDAS? And why should graduate students consider using CAQDAS software?
Christina 2:16
Okay, so Hi, thanks a lot for inviting me, it's great to have this chat. So CAQDAS is an acronym that stands for computer assisted qualitative data analysis. So it's used to refer to software and other applications digital tools that have been specifically designed to facilitate qualitative and mixed methods analysis. It's an acronym that was developed in around 1991, by Nigel fielding and reily, after they convened the first conference, looking at software to facilitate qualitative analysis. So now it's used as an umbrella term to relate to all of these digital tools of which there are now dozens available. The thing about CAQDAS packages in terms of their use by students, graduate students, also undergraduate students and other researchers is that they are one of the kind of tactics that we have available at our disposal to operationalize our analyses. There are some debates about their use. And that's something that we'll probably touch on later on in our discussion. For me, you know, it's really important at the outset to realize that, although there are many ways that CAQDAS packages can facilitate analysis, they can help us organize our data, they can help us access different aspects of our analysis process, it's still possible to do bad analysis using CAQDAS packages, just like it's possible to do good analysis without using them. So that's a really important starting point in thinking about whether and how to use these kinds of tools. But for me, you know, really, the main thing is the access that it gets to the process. So using a dedicated CAQDAS package gives us access to the materials that we're working with. And that will be the data that we're working with the qualitative data, but also, other supplementary materials that form the context of a given study also gives us access to the ideas we have about what's interesting and meaningful in our data, allowing us to get back to those earlier thoughts. And also really importantly, in terms of process, it gives us access to the history, the journey of our analysis process. And for students, that's really important because when we're doing a dissertation, it's often as much about how we went about doing it and the lessons that we learn from that as it is the findings that come out of our research. So it can help in various ways, but it's also a useful skill set to have. I think when you're learning about qualitative methods, generally, learning about the technologies that are designed to facilitate the process is really useful for anyone who wants to have a research career after their studies. Were there That's within academia or outside of academia, because many qualitative and mixed methods, researchers are using these tools for their work. Now,
Amir 5:08
that's fascinating. And I think that the notion of access is really important here in the sense of having that possibility to easily retrieve pieces of your data. And also, how do you conceptualize that data in different stages during the project. So we know that there's this growing need for graduate students to be familiarized with CAQDAS packages, but there is some confusion sometimes about how CAQDAS software is used, whether it's a method of analysis or something that kind of supports analysis. So why do you think people sometimes confuse CAQDAS software use with a method of analysis?
Christina 5:45
Yeah, a good point. And something I think that continues to be discussed, even after all of these many years since these tools have been available. For me, I think there's two interrelated reasons for the kind of confusion between the software and the method of analysis. First of all, a misunderstanding or an assumption that the software does the analysis for us. And secondly, unsubstantiated and outdated criticisms about the negative effects of using CAQDAS packages, in terms of the kind of craft work that qualitative analysis involves. So these kind of misunderstandings and criticism started in the earliest moments of the availability of CAQDAS packages. But despite a fairly large body of literature now that dispels the notion that CAQDAS is a method of analysis, these kinds of misunderstandings and criticisms are still being perpetuated informally by some teachers of qualitative methods who are not kept as users themselves. But also formally in the literature, we see these misunderstandings and criticisms promoted, and that's often also by non users of these tools. So, as a caveat before I, before I carry on and talk about that a little bit more, you know, I'm not saying that everybody should use them, or that it's wrong not to use them, it's just understanding the role of them is really important. I guess the assumption that CAQDAS does the analysis is the thing to think about first. So for me, there's kind of two differing reactions to the realization that the software doesn't do the analysis. First are those who are saddened that it doesn't do the analysis, you know, some people are looking for a shortcut to accomplishing their work, and they want the software to do the analysis. So they're disappointed when they realize that that's not the case. So technology is developing really fast at the moment. And there are now some CAQDAS packages that incorporate AI technologies, such as machine learning, and therefore provide a lot more assistance than than was the case a few years ago. But at the end of the day, it's always the user, the researcher, who decides what to do, who decides when to do it. And who decides what it all means. So interpretation, differentiating interpretation from analysis, I think something that it's really important. But on the other end, sometimes there are people who are kind of outraged at the idea that kept us would do the analysis. So that's the other the other end of that continuum. So those are the researchers who really price the human interpretive processes, which underlie many approaches to qualitative data analysis. And it tends to be the this group of researchers who criticize CAQDAS because they think that that the software is taking over in some way. And therefore, that's where the criticisms come from. So it's interesting to me that that idea that the software does the software do, the analysis is understood very differently, depending on our understandings and engagements with the tools. So I guess the other thing I just wanted to say is that some colleagues of mine, Christy Jackson, Trina Paulus, and Nick Wolf, they wrote a really excellent article on the perpetuation of unsubstantiated criticisms of CAQDAS that was published in 2018. And they look at four different criticisms and kind of debunk those, but also look at how the literature kind of perpetuates those ideas. And I think it's really interesting to, you know, reflect on those criticisms. And you know, if you become a user of a CAQDAS package, to understand the context of those debates, so that you can place yourself within that context, and sort of justify your use of the software.
Amir 9:36
Amazing. I think there's so much importance in understanding that context, and also and thinking very carefully about how we approach the role of CAQDAS in our work. What I found so interesting about your work with colleagues as well, is that you've really tried to unpack different ways in which we can think about that the role of CAQDAS in our work, how we can approach it and how we can operationalize it. And I've noticed that in your work in terms of First in terms of how we think about the kinds of skills that we need to attain when we work with when we want to work with CAQDAS, or the kinds of practices that we need to engage with. You mentioned three factors, among other factors that are particularly important in the learning and adoption of CAQDAS software. You mentioned in your work, methodological awareness, analytic adeptness, and technological proficiency. Could you unpack each of these a little bit and why they are important for graduate students conducting CAQDAS?
Christina 10:33
Yeah, absolutely. So those three factors come from some research that we did, looking at how new users learn about software, what the challenges are, and also how they adopt technologies in their practices. So methodological awareness, first of all, is to do with the familiarity with the variety in qualitative data analysis philosophies, and methodological approaches. And they underpin the choices that researchers make in undertaking analysis. So having a methodological awareness is the kind of overarching kind of starting point, I guess, in thinking about CAQDAS and adopting it, and it's important method, this method logical awareness, because qualitative research is so broad and so diverse. So the choices that we make about how to go about an analysis really need to be made carefully, and then need to be justifiable within the context of the broader qualitative research field. So I've often observed confusion around the terms methodology and methods, and with students often being very overwhelmed or confused about qualitative methodologies, and unclear about how methods relates to methodologies. So I spent quite a lot of time trying to talk about this with with my students. And the simplest way I found to explain it, I guess, is to say that methodology is a description or a roadmap of how a project will be undertaken, how the research questions will be addressed, I guess. So methodology describes how the entire project will be conducted. And it kind of provides the criteria for designing or selecting the methods. And methodologies are informed by two quite scary words for students ontology and epistemology. And they're used to think about our philosophies or our paradigms or our assumptions that we all bring to our research. So ontology, you know, in simple terms, is our view of the nature of reality. Whereas epistemology is to do with our perceived relationship with the knowledge that we're uncovering or discovering. So being aware of methodologies is really important, because it provides that description of how you're going to go about your study at a high level. And that informs the selection of your methods. So what are methods then methods are the data collection methods, so whether we're undertaking interviews or focus groups, or generating data from online interactions, or surveys, or asking participants to generate visual materials, etc. And then we also have analytic methods, which are how we're actually going to go about doing the analysis. And there are many different analytic methods, thematic analysis, interpretive phenomenological analysis, content analysis, discourse analysis, those are all examples of analytic methods. And so what one of the challenges is, is understanding how those methods, the data collection methods, or the analytic methods relate to the broader methodologies. And that kind of basic framing and understanding is really important, because it leads to how we use the software. So that's the first factor methodological awareness. The second one is what we call analytic adeptness. So this has to do with the experience in undertaking qualitative data analysis. So specifically, the skills in designing an analytic task in the context of that underlying methodological context. So it's to do with what we actually do when we analyze qualitative data, what we do, how we do it, the sequence in which we do it. And this all depends on the analytic methods that we're using. So those tasks would be different if I was doing a thematic analysis from a content analysis or a discourse analysis, for example. So some analytic methods are more prescriptive than others in terms of the sequencing and the actual tasks that are undertaken. So I kind of think about that in two ways. I guess, first of all, the kind of what I call the strategies level, which has to do with planning what we need to do and that's different from the The tactics level, which is how we actually go about it. So analytic tasks, deciding what to do is kind of at the strategies, the planning level, and how we accomplish it at the tactics level is where the software potentially comes into play. Because a CAQDAS package and the tools within it can be the tactics that we use to undertake those strategy level tasks. But of course, cactus packages are just one of the tactics that we have at our disposal. So you know, many people, before CAQDAS packages were around, we worked manually, right with pen paper, my first qualitative project, I cut up my transcripts with scissors and put bits of transcripts in different piles and had a big matrix on my wall. Other people are using general purpose software, Microsoft Word or Excel or other tools. So these are all the tactics that we have to undertake our analysis. And that's where analytical adeptness kind of leads into that third factor that you mentioned, which is the technical proficiency. So analytic adeptness really straddles from the methodological awareness to the kind of planning what we actually going to do. And then the next struggle to the technological proficiency is when we're using tools, digital tools, or any tools, how much competency we have in operating the software, and also the comfort that we have, with the idea of experimenting with the software, without fear of making mistakes. I think that's one of the key challenges that I see in new users of CAQDAS packages. In that often students are learning about qualitative methodologies, and software tools at the same time. And that can be really, really challenging. So many cactus packages, one of the best things about using a dedicated CAQDAS package is that it's incredibly flexible. And they offer dozens and dozens of features. But the challenge for the learner is that not all of those features are going to be useful for an individual analysis. So the learner has to navigate that all of the technological possibilities at the same time as thinking about which features are going to be useful for their analytic needs. And that's a really big challenge. So, you know, often, I see, in many of the workshops that I run, I see students being really worried that they're going to mess things up. So being scared that, oh, if I do this, then something might go wrong. And that's, that's often happening in the early stages of learning, learning a cactus package. So technological proficiency is about learning the architecture and the functionality of the software, and how to operate it. But doing so in a way that develops confidence to experiment and be creative about how we use the tools, because there's no one way of using any of these tools, just like there's no one way of undertaking a particular research project. So those three factors, I think, are really important in learning about and successfully adopting CAQDAS, methodological awareness and analytic adeptness are really closely related. And they're to do with the strategies that we put in place, whereas technological proficiency is at the level of operationalizing, those strategies using tools to accomplish our tasks. So despite the sense in which those factors are interrelated, and how they straddle one another a little bit, I do think that thinking about them separately, really helps students to understand the sense in which cactus packages are not methods, but tools. And hopefully, to see that learning about what to do is different from learning about how to do it. those are those are really important, but they are different. So differentiating between them, I think, is a useful framework.
Amir 18:52
And that is really important points. And I think they connect directly to your some of the work that I think you're kind of well known for, which is the work on the five level QDA method, where you really talk about the importance of thinking, what is the kind of strategic approach that I want to take to my data analysis? And how might I translate that using the particular features of the software that I'm thinking of working with, and perhaps combined different software features in creative ways or useful ways that are particular to the context in which I'm working? So you've really connected to that already. But I wonder if you want to say a few words about what is the five level QDA method and what what it might be useful? For?
Unknown Speaker 19:34
Sure. Yeah. So I think that leads on from from what I was talking about before, in terms of those three factors and our work with the five level QDA method built on our earlier work as well. The first thing I guess, to say about the five level QDA method is that it's a method of learning and teaching practice, not a method of analysis. Okay, so it's not an equivalent or replacement for thematic and analysis or grounded theory, or discourse analysis, or anything else is the pedagogy. Really, and we think about it as a mindset, a way of thinking about the role of CAQDAS packages in our work when we're undertaking any type of qualitative or mixed methods analysis and using any kind of digital tool. So our focus in the development was around three of the most well known practice packages, but it's, it's relevant to any digital tool, whether it's a dedicated CAQDAS package or not. And we developed it because we had observed over our many years of using and teaching these programs, and also talking with other colleagues that there were a number of key challenges with being able to kind of harness the software tool for the particular analysis need, if you like. So we did some more research into that. And basically, we unpacked what happens in the minds of those researchers who have become really proficient expert users of these CAQDAS packages, we've unpacked their unconscious processes, what they do without thinking, as they have become more proficient in the in the use of the software for their analysis, unpacking that, so that new users can develop the expertise that they need, without the many years of trial and error that myself and my co author on five level QDA Nick Wolf, you know, we both went through many years of trial and error with these programs before we developed the level of understanding and proficiency that we now have. So one of our key aims was to try to enable that learning to be shorter and more efficient, by really understanding those unconscious processes and unpacking them and making them clear. So it's really a framework for for thinking about the role of CAQDAS, and enabling new users to be able to harness in a powerful, but also a quick way, what they need to do in terms of students. They're doing dissertations, they have a short amount of time, they're learning multiple things. At the same time, as we've sort of touched on earlier in our discussion, Amir, and, you know, we wanted to enable them to adopt successfully their chosen CAQDAS package within the time constraints that they have. And I guess the key principle, which relates in part to what I was talking about earlier, is that our analytics strategies are different from our software tactics. Okay, so strategies are what we plan to do. And tactics are how we actually go about doing it. Okay. And related, again, I guess, to the idea that or the misconception that the CAQDAS packages are methods is that strategies are developed, outside of the context of the tools that we use, and really need to drive the way that we use the software, I have to choose the appropriate tactics for the task, the strategies, tasks that I have, in my mind right now. And that's the way that we like to think about how to operationalize our analysis using software. So in the context of qualitative research, our strategies are the objectives and the methodology and the analytic tasks that we develop in order to accomplish our objectives. And they are, you know, they are not driven by the software that we decide to use. They're accomplished through the use of our software. distinguishing between strategies and tactics is something that's not often or clearly done in the in the methodological literature. And often those two terms are conflated or used as if they're synonyms. So the basic underlying principle of the five level QDA method is recognizing that strategies and tactics aren't the same, and that strategies must drive our use of software.
AD 24:12
The qualitative research special interest group was established in 1987 to create a space within the American Educational Research Association. For the discussion of ethical, philosophical and methodological issues in qualitative research. We invite you to consider joining the qualitative research SIG today. for members of a era. The annual fee for joining qualitative research special interest group for regular non graduate student members is $10. And the annual fee for graduate students is $5. As members of the QR SIG, you will gain access to a network of fellow qualitative scholars, as well as our many activities ranging from mentoring opportunities to our podcast series. To update the news related to recent qualitative publications and jobs. Please visit the American Educational Research Association website at www dot era dotnet During the qualitative research sake today,
Amir 25:03
Right, WIWells I've engaged with that work quite at length. And I found it very useful in my training or work with graduate student peers. But one of the things that I've experienced is somewhat difficult at times, is to really try to find a concrete way to unpack how different methodological approaches might look like how they might differ from one another, as this first step, before we kind of translate them into software tactics or other kinds of tactics that may not even necessarily involve software. Sometimes grad students may not necessarily have the sufficient training in how these different approaches might even look. So that's an issue in itself. So I wonder if you could share with us one example, perhaps in which one methodological approach might significantly differ from another in the analytic strategy that it entails?
Christina 25:57
Yeah, so I guess when I talk about this, I mean, this, I think you're right, that this is something that's difficult to make tangible, in a car in a kind of concrete way. And I guess that comes, you know, for the user of a CAQDAS package that comes with experience, and with that experimentation, and that creativity that I mentioned earlier, but I guess it's also partly to do with breaking down the analytic strategies. So what we do, so if we think about analytic strategy of grounded theory, and then contrast that with classic quantitative content analysis, then, you know, I talk a lot about directionality of analysis. So, in broad terms, grounded theory approach is about generating theory from a body of data that we have in front of us. Okay, so generally speaking, we're working in a kind of bottom up way, in an inductive way, that's often described as I think in a grounded theory approach, often we do some more top down some more deductive work as well. But the general directionality of our work is starting with the data and generating an interpretation or theory from that body of data. Whereas their content analysis, content analysis is as tricky. A tricky term, actually, because it's understood quite differently in different disciplines and different national context I found as well. But content analysis classic quantitative content analysis, is about looking at instances of key words, phrases, and other structures or patterns in texts, which can be written texts, or can be visual texts, and counting those instances and potentially doing statistical work with those instances. So a content analysis can work either bottom up or top down, depending on whether we have a hypothesis that we're trying to test, some content analysis starts with a hypothesis. And our techniques are about testing that hypothesis. But we can also do a quantitative content analysis in a more inductive way, a more exploratory way, especially these days, when we have so much qualitative data at our disposal from the internet that we can harvest in all kinds of different ways. A classic example might be Twitter, data, right. And if I, if I gather 10s, of 1000s, potentially millions of tweets, then I don't know what's in that data. So I may do a content analysis in a much more exploratory way. So for me, it's, you know, it's not so much about saying, oh, in a discourse analysis, you do this, then this, then this, then this, whereas in a grounded theory, you do this, then this, then this, then this, it's more about the individual tasks that we undertake. And in the five level QDA method, that directionality is something that's kind of quite important to reflect upon. But specifically, we break it down to what we call an analytic task, which is the smallest unit of what we do the smallest unit of analytic activity, and they are specific to each individual project. But I believe that there are commonalities in analytic tasks across the different methodologies. So what we do is to try to, to think and teach how to harness CAQDAS packages at that level, the methodology and all of that background that we talked about earlier is really important, because that's what leads to specific analytic tasks, that at the moment that we accomplish a task in the software, it's just that task in and of itself, that's the important thing. So what is an analytic task? There's many examples. We've got many examples in our publications, but, you know, one task might be to read my interview transcripts to identify potential concepts for coding That's an example of an analytic task, a small unit of what we need to do. Now in any CAQDAS package, there are multiple tools that I could use to accomplish that task. Another task might be to compare theoretical and emerging concepts and explain their similarities and differences. And that may be a task that's relevant to all kinds of different methodologies. So I guess for me, the contrast, are the examples of methodological approaches that differ from one another in terms of the analytic strategy. Yes, I think that's an important thing to reflect upon. But I think it becomes concrete in terms of examples and learning, when we take it down a level and think about, okay, what is it that I actually need to do, and break that down to the smallest level, and then it's much more manageable to build up the activities more generally,
Amir 30:57
you've discussed so far, how important it is and your work, especially how important it is that notion of directionality in the sense of sort of planning ahead, what you're thinking of doing, and we've talked about how sometimes there are some gaps in terms of knowledge about what it is that we're supposed to do with certain approaches. But I wonder now, if in your experience, are there any interesting ways in which the work with the software is somehow influenced or somehow informed, analytic strategies? Or how we understand certain approaches, certain methodological approaches? Because I feel like it's such a fascinating area of inquiry?
Christina 31:36
I think it's a great question. And it's something that I've spent quite a lot of time thinking about, whilst we were developing the five level QDA method. And also since when I've been talking about it, and using it in my own work, and some of the conversations we had with our colleagues and other researchers using CAQDAS packages, really speak to this idea. And I kind of guess I've realized that there's a really important difference between what we say, which is that strategy should drive tactics. And that that's not the same as saying that tactics don't inform our strategies. So, you know, I've had some really interesting conversations with some colleagues who have said, Yeah, but there are things that are possible in CAQDAS packages that just aren't practically possible if you're not using a dedicated tool. And I definitely agree. So that is a way in which the availability of software tools, their affordances can change the way that we think about going about our analysis. But for me, that's an informing process, not a driving process. So let me give you an example that will hopefully bring what I mean to life a little bit more, I had a student come to one of my advanced software workshops A few years ago, who'd been using a CAQDAS package to do a literature review, because they've read and heard that it's really great, you can do that review, using these tools. And she'd come along to this advanced workshop because she was in a bit of a pickle. And she was like, she said, You know, I've done all of this work, and I can't see how it's helping me. So we looked at her project, and she had 1000s of codes in her software project. And upon looking at her project, and talking to her and asking her what she'd done, it became apparent to me that she hadn't read any of the literature. She had used tools like word frequency tools, to tell her what words were in the literature. And then she'd coded every sentence or every paragraph in which highly frequent words came up, imagining that that somehow was going to tell her what the literature was saying, imagining that she there that her use of the software would mean that she didn't need to read the literature that it would tell her what was going on in her materials, right, which speaks back to what we talked about right at the beginning about CAQDAS not being a method. Yeah. But in terms of this idea of driving and informing she was looking to the software to tell her what to do, and also to tell her what was there. And that's, you know, we're long way off software being able to do that. So, you know, I think there are certain ways in which the availability of tools can inform our thinking about what's possible and may change some of our processes, but the availability of tools should be used to inform our strategies not to drive our strategies. And I think that's, you know, that's a kind of semantic difference between those two terms. But I think that's a really important one.
Amir 34:50
What's very interesting to me and kind of how our conversation evolved here is that there seems to be seem to be two different areas in which we might see live or work on qualitative data analysis kind of moving forward into, it seems one direction is trying to really maybe based on the work of others, like Joe Maxwell, for example, who has written about the theory of qualitative data analysis, perhaps we could sort of advance this field and thinking about what might be some of the similarities across different methodological approaches, and what might be some of the important differences, that might be a useful way to kind of advance both our thinking and graduate students training in how you know, those approaches might be unpacked and translated to different kinds of tactics, including the use of software. But the other interesting point that you raised that was that interesting distinction between the sense in which software might drive our our analytic work, as opposed to informing our analytic work. And I think that's a really important area to consider. And thinking about qualitative data analysis with software
Christina 35:58
is to deal also with a set with serendipity isn't it, it's like, software, the use of software tools can tell us things unexpected that we weren't anticipating or looking for or, and that's, that's, that's a great thing, you know, and one of the things about familiarizing with our data and getting a real in depth understanding of what's going on, which is a key to many different approaches, software can really help with that. And you know, that balance, I think you actually say something that, that we should be reflecting on war.
Amir 36:31
And that sense of serendipity really connects to what you also write about quite a lot in your work about the iterative and emergent qualities of qualitative data analysis, right? So yeah, these things are coming to twined, in interesting, fascinating and yet to be explored ways. This leads me to the last two questions, I'll start with one question about the kinds of challenges that you might have seen, whether it be challenges that students experienced, or other kinds of researchers with translating analytic strategies and tasks into software tactics. So if we were to sort of go down that road as grad students, what kinds of challenges do you think we should expect?
Christina 37:11
Okay, so there are two key things, I guess, to say. And the first one, I think, has to do with what we were just talking about when you mentioned the sort of iterative and emergent nature of qualitative and mixed methods analysis, and that idea of thinking ahead and planning. So sometimes a reaction to the five level QDA method, and the emphasis that we put on planning is this idea that Yeah, but you know, quantitative research evolves, we don't know exactly what's going to happen, and you're trying to put a structure on it that isn't really there. And my and I think that's, you know, a fair point. But the way that we would talk about that is that, to varying degrees, all of our processes are iterative, in an emergent, some methodologies are more intuitive and emergent than others. But that doesn't mean that we don't plan what we do, it just means that our plans change, as we see new things and come across new things. So the planning aspect is something that I see students find real challenges with, because they haven't always thought that connect with that methodological awareness and analytic adeptness that we were talking about earlier. So they don't really know what to do, what should I do first? And it's kind of like, Okay, well, if you start to plan, what you might do, and then think about what that would allow you to do next, and then come up with another scenario, well, I could do it this way. And then what would I be able to do, and that helps you to see how important planning is. And I've really, you know, seen in with many of my own students, you know, the value of planning in terms of fair reflection back on it, later on. And so we have these analytic planning worksheets that we developed as part of our methods, which we developed as a means to facilitate the learning of software, but has actually unanticipated consequences of us developing those is that people are using them to document their process. And that speaks to transparency, and rigor, and all of those things that qualitative researchers historically haven't been very good at, you know, explaining in concrete terms, how I got from my pile of data to my interpretation is something that historically we're not very good at. But planning and doing that in a structured way, and allowing those plans to evolve and documenting that it becomes very useful. So I guess the other thing I just wanted to mention briefly is, is units so as I'm sure you're aware, we talk a lot about units in the five level QDA method. And this is something that quantitative researchers typically are pretty clear about units because they normally use units to determine their variables that they use for their statistical work right? But in my experience, qualitative researchers don't really think very clearly about units and are often not taught about units but units are fundamental to qualitative analysis. And we distinguish between units of analysis. So they're the kind of main entities that form the basis of our research. So that might be groups or individuals or organizations, for example, units of data. So the form that the data is actually in, so that might be a transcript, it might be a sentence, it might be a discussion, it might be an interaction, and then units of meaning, which are kind of related to units of data, but are to do with which bit of data do we need to capture in order to be able to access the ideas that we've had. So units is a difficult concept, I think, for students to come across. But in my experience, that the, you know, I've seen those kind of lightbulb moments amongst students most often when they get what units are, and why why they're important, generally, in their research design, but also in terms of how they translate their tasks into those software tactics.
Amir 40:58
Think I felt that light bulb moment when I've reflected on what is that selection that I'm making in software that I'm going to continue working with in terms of coding and connecting to memos, etc. So yeah, these are really important points. I also wanted to connect this to something else that you mentioned before, how sometimes graduate students may learn software tools and methodology at the same time as a challenge. But at the same time, you also, you've also written about in other places, I believe that there is sometimes a disconnect between the learning of software and the learning of methodology. I wonder if you could speak to that? Because those are, you know, on the one hand, the there's a problem with learning them at the same time. But what about that disconnect? Is that related somehow?
Christina 41:44
Yeah, this is, this is a topic that's, I'm thinking a lot about at the moment. In terms of, you know, there's different opinions about whether you should teach and learn methods, strategies, methods first, and then the software, or whether you can teach and learns the two together. And, you know, in my opinion, there are kind of three ways of thinking about about that. One is the kind of the sort of sidelining or ignoring of technology and just teaching the methods. Yeah. Which I think is does the next generation of researchers and students a great disservice. So you mentioned earlier that students expect to use digital tools these days, of course, they do, because their whole lives, for many of them are embodied by the use of digital tools. So they can't understand why you wouldn't think about using digital tools for qualitative analysis often. So those teachers who are not teaching them or even making them aware that there are digital tools, which I think happens far too often still, I think that's, I think that we're, you know, we're not being responsible as teachers, if we take that approach. And then then the, you know, then there are two ways of thinking about this in terms of teaching methods and technologies, and then how and when to do that. So there's the kind of methods first, so teach a range of qualitative methods or a specific method outside of technology, and then teach the technology and sometimes that's bolted on and isn't integrated. So it's like, here's the methods piece, okay, now go off and learn about the software. And there's not enough connect between the two, which is the disconnect, I think that you're you're referring to, another way of doing it is teach a little bit of methods, and then teach how that can be operationalized in software, and then teach another bit of methods and how that can be operationalized in the software. So that's what I would call methods into methods and technology interwoven. But then it's also possible and there are people who are teaching methods via software. And in the collaborative context, that's really useful. So tools that allow multiple students and researchers to work on the same transcript, for example, at the same time, so coding is a good example, okay? You have a piece of data, and then you ask students to code giving more or less direction for how or what they should be coding, and then you compare the coding and then you have discussions about Oh, why did you code it like that? I coded it like this. And then you're teaching what coding is, what the methods of coding is via the software. So there are different ways of interleaving or managing that potential disconnect. And I'm not sure that that methods first methods into work woven or methods via software, I'm not really sure that one or other is better. They're just different, but they're all possible. And they're all better than just not doing it at all. I think.
Amir 44:39
I think we can definitely agree on that one. All right. Well, we're moving toward the end of our podcast episode. And the last question I have for you is just if there are any resources that you would recommend to grad students who are looking to explore analysis or analytic strategies of different methodological approaches, one kind of deal Deep into some of the concrete ways in which they could think about their data analysis.
Christina 45:07
So there are lots of textbooks out there. My favorite textbooks, I guess, are the ones which which taught me back in the day. So Miles and Huberman then in the 1994 textbook remains, you know, really close on my shelf. And I draw on that still now. I also really like Harry Walcott's work. More recently, Patricia Bazeley. Her work her textbook on qualitative data analysis. I think it's excellent. So in terms of textbooks, those would be the three main authors that I would say, look at those first, there are loads of others really good authors around qualitative data analysis techniques as well. But you know, if you're asking for Okay, what's the starting point, then then I'd say Miles and Huberman, Walcott and Baseley. But there's also quite a lot of online resources now. So there's a website called Methods Space, which is pretty active and has lots of resources, which are curated around different topics. And this year, at the moment, they're going through the kind of whole research cycle, and each month focusing on a different topic. So I think at the moment, the topic is ethics. And previously, they did a whole topic on research questions, etc. So methods spaces are really great websites to tap into. And that's related to the Sage Publications, methods map. So there's lots of resources there, and really good links to online and offline publications, etc, around particular key methods. There's also a resource called Online QDA. So online qualitative data analysis, which was a project which finished quite a few years ago now, but was designed for graduate students to help them navigate qualitative data analysis. And what's really nice about that site, I think, is the methodologies area where there is a clear and short definition of different methodologies. And then there are links to further resources. So the key publications to read about each of those methodologies or methods. So that's really useful, I think, for students because, you know, if I need to learn about a particular methodology, grounded theory, say there's so much written about grounded theory, how do I know which of the things I should read and the Online QDA website has kind of done that job for students and said, Okay, this is what that methodology or this is what that method is, and these are the key things to read. So I think that's really, really useful. And the final thing I'd like to say is that learning about analysis and learning about cactus is really usefully done while still doing a literature review. So literature is a form of qualitative data writes text and embedded images, okay, reviewing is a form of analysis. Okay. And we can really usefully learn how to do qualitative analysis, whilst doing a literature review in our in our chosen CAQDAS package. So that's something that I think, is useful for all students who are doing dissertations who need to do a literature review. And there's a book by Wallace and Wray, Critical Reading and Writing for Postgraduates, which is a really, really excellent book, which talks about literature reviewing, not in the context of cactus packages, but in the context of understanding how to read critically how to discern whether what you're reading is of good quality or how it relates to your work, etc. and then how to write critically, which is something that I think students also struggle with in their dissertations.
Amir 49:00
All right, well, Christina, that was a fascinating conversation. And you've just given us so many useful resources on top of a really insightful and rich discussion. So again, I want to thank you for taking the time to participate in this podcast. And I hope you continue to do that important work that you've been doing in moving this field forward. So thank you for your time, and hope to chat again sometime soon.
Christina 49:28
Yeah, many thanks. I enjoyed it.
  continue reading

44 tập

Tất cả các tập

×
 
Loading …

Chào mừng bạn đến với Player FM!

Player FM đang quét trang web để tìm các podcast chất lượng cao cho bạn thưởng thức ngay bây giờ. Đây là ứng dụng podcast tốt nhất và hoạt động trên Android, iPhone và web. Đăng ký để đồng bộ các theo dõi trên tất cả thiết bị.

 

Hướng dẫn sử dụng nhanh