Emergent+Methodologies+in+a+Quantitative+World

This is a question I posed to the group at the start of the course. It generated quite a discussion! Below are some of my comments (highlighted blue) and responses from others that have helped shape my thinking.

Hi everyone, My question is for everyone. There seems to be a lot of emphasis amongst policymakers and those funding research for research to be quantifiable. This was even in the media recently. How does emergent research survive in this environment?


 * Elaine M**

//Interesting and perceptive question, Craig. Yes - one of the challenges for all qualitative research is getting the findings into "sound-bites" that make sense to politicians and can be shared in a 30 sec broadcast.// //This is the same problem as we have in assessment - how do you assess meaningful learning with true/false, closed answer (as in arithmetic) and multiple choice questions? To assess higher order thinking requires more than quantitative measures.// //So, we now have two challenges (a) how do we build theory and insight that is meaningful and holistic in our insider contexts and (b) how do we communicate our work in ways that can be shared through the media and have an impact on policy making (nationally and locally in our own contexts)?// //These are great questions to keep at the back of our minds as we plan and explore our own possible investigations.// //This is where using both quant and qual tools and thinking creatively is important for all of us. Also recognising that small shifts in thinking within a community can lead to policy change - within practitioner research of all kinds we are working with small shifts in thinking - and checking out their impact as we work within our communities.//


 * Craig**

Yes, I agree about the importance of being agents of change. On the other hand, sometimes we need to produce 'evidence' to influence decision-makers. I guess I am really asking how we can answer the claim that emergent research findings are 'anecdotal' and not generalisable to other situations? Or should the goal of change for an emergent researcher be limited just to that one particular context, or more broadly? I am hoping to focus my research in the area of online safety and privacy for adolescents. If I believed there needed to be change higher up the food chain with regards to either a school or national policy, wouldn't I need greater evidence than my own experience with my learners? In this respect, is it necessary to draw on the wider research (non-emergent) community/findings for back-up, and if so, aren't emergent researchers dependent on this for validity? And could this be considered an 'emergen'cy? (!)

//It seems that some of the 'emerging' methodologies need to be supported by data that is in essence quantitative to authenticate their perspectives. The person who has consented to be my supervisor next year has indicated that she will be interested to observe my research because it seems that there is a bit of a venn diagram where the various methodologies overlap and support each other!// //I wonder, if, as more deeper study into both qualitative and quantitative research and data is undertaken if the grey area between the two will serve to compliment education, decision making and utlimately the students that we want to see develop and succeed in their learning!// //Thanks for stimulating my thinking! //
 * Lynne M**


 * Janinka**

//I quite like evidence.// //So let's think for a bit about what such evidence might mean in the kinds of areas that are being talked about here.// //What do we want evidence of? Do we want evidence of what people think? or of how they are incrementally investigating the problems they face and working to resolve them? The kind of methodologically approaches we are discussing can provide that evidence - but we have to keep refining our design to make sure there's validity in representation.// //Or are we looking for evidence that the participants' opinions are 'right'? So what would 'right' mean in that context? and how useful would it be to any specific question to know? If it is useful then what next cycle of research should we undertake (and number crunching could well be an option for all of part of that cycle)? But sometimes it is the opinion itself that matters. For example, if we're looking at, say, embedded racism in a social group such as a school or a university, it's the attitudes, actions etc of the respondents that helps us understand what's going on, isn't it? and we could call that evidence, though the term may not be particularly helpful. One of the research approaches we look at in this course might be used by such a group to investigate and get rid of the embedded racism. Their success could be seen as evidence, but then there's no need for evidence, because the problem's gone. What might still be useful to others though is a well structured narrative of the process they took and the understandings they gained. Testimony rather than evidence, perhaps?//


 * Elaine**

//Perhaps there is a myth out their that ALL research is all about developing universal (generalisable) knowledge which is the kind of knowledge that I think Craig is talking about in relation to developing (proven) theory which can be influential in shaping policy. Perhaps political rhetoric suggests that the only research that influences policy is the kind where claims can be made that the finding are generalisable to other settings.// //What policy change has happened as a result of good documentation of emergent practice where social issues are addressed? Let's collect some examples ... (respond to this posting). I suspect that where local councils or administrations adopt fresh ways of approaching social issues their ideas are commonly built on stories of effective practices from elsewhere - but these stories have not been labelled as "research." (Think of the time our politicians spend going overseas to explore how other countries address social issues.)// //This course highlights how stories of effective practice can impact on policy making AND this course identifies this activity as research. In order to claim it as research it also requires the careful documentation around process and evidence and validity and critique of konwledge claims and so on - as we shall discuss. The aim (in social justice terms) is to enable practitioners to both value and share their experiential, tacit wisdom - so that they are enabled, and the people with whom they work are enabled.//


 * Craig**

Actually, here is the article which prompted the question - you may have seen this on the news a few months back, regarding the Families Commission, and criticisms from the government that their research was not robust enough to inform policy decisions. I like Helen's comment below that quantitative gives us information about differentials, but doesn't necessarily answer the question 'why'. Is this a case for mixed methods research? Many researchers will do a large survey based investigation and analyse the results, and follow it up with some more qualitative elements, such as interviews or focus groups, in order to get to the 'why'. Some people have mentioned the importance of narrative in informing policy. It is interesting that the same politicians who one minute are decrying 'anecdotal evidence', in the next minute are using these anecdotes in their speeches. I suppose also from a politicians point of view, whenever they make a decision, they are likely to receive criticism from somebody, and even if they think the stories make a good case, they probably feel safer being able to call upon 'robust quantitative research' to defend their decision. I would be interested to hear what others think of the TVNZ article above.


 * Elaine**

//Ah - this adds light to the discussion. The families commission is a specific group charged with specific responsibilities. This discussion is about the way in which the research they commission enlightens their decision making.// //This is quite a different question from the nature of high quality research.// //One of the aims of this course is to ensure that the emergent approaches to research that we are alerting you to are carried out in ways that are highly reputable by addressing the research issues that are being investigated in the most appropriate ways.// //It would be useful, at this point, to shift the focus of attention to questions about how to use the approaches we are discussing effectively. This means we need to be strengthening our understanding of these forms of research and learning to use them well. We could then return to this discussion from an informed basis and see how we would respond if we were taking part in the kind of debate that the Families Commission's research has raised.// //I point you to reading 1, Craig, and wonder what the ideas in that article add to your understanding of research. There are strong policial and social pressures and simplistic assumtions which push common understandings about research into particular, narrow, quantitative corners. The article you have pointed us to is an example - it pushes alternate, emergent approaches onto the defensive - let us not get too diverted before we have developed some deeper knowledge about alternative approaches and arguments.//


 * Helen H**

//Ka pai to patai. Good question. The positivist/quantitative paradigm may be useful for telling us where differentials occur its not so good at telling us why.// //Kia pai to ra.//


 * Janinka**

//As researchers we will have different aims- and each of us will have different aims for different purposes. It is again about One size does not fit all. Public policy makers do need to back up their decisions with evidence - otherwise the taxpayer will scream. And evidence as you have suggested comes in different forsm, accounts of learning stories and of effective practice and change implementations, results of interventions and experimentations, statistical anayysis of specific outcomes or behaviours, etc.// //But behind the policy decisions is a body of knowledge about learning and teaching that we are all adding to in our research. Much of it will initiallly not have been aimed at policy change as such - though hopefully it will inform it//