Organize your literature review notes in a matrix

I have wanted to make sure I got this link in my blog for future reference, so I have actually copied the link in my draft and nearly forgot about it. My bad!

It’s an awesome practical example on how to use the matrix to organize your notes shared by the famous Thesis Whisperer. I’m really honored to get a mention there. If you have missed her earlier sharing on this particular piece, here you can check it out below.

TW Backline Masters – Using matrix to organize your notes

I actually wanted to share something similar to show how I organized my literature review notes before I wrote my literature review but never got around to share a practical and good example. I’m glad Thesis Whisperer shares her example and I’m sure many have already benefited from it.

Happy writing, everyone! ūüôā

How literature review matrix works for me

When I decided to make the literature review (LR) or synthesis matrix during the LR process, I didn’t expect how this matrix would help me in writing the LR until the day I was working on my paper for submission during the writing workshop. My supervisor thought I needed more references in that paper and I needed to get the paper ready in less than an hour. I just opened my LR matrix on PLE that I made earlier, looked through it again on the different aspects that I’ve made notes on, then I easily added more references where I saw appropriate to my paper without the need to reread all the related articles. Within half an hour, I had the paper ready with extra related references.¬†That’s when I realized my hard work and time spent on my LR matrix has paid off.

I’ve done a few matrices on different research topics for my literature review but my LR matrix on PLE is the most comprehensive because that’s my major research topic that I focused on. I basically divided my matrix into a few aspects that I wanted to look into, so when I was reading each article about PLE, I’d take note of these aspects.

These aspects were the common issues about PLE that I wanted to talk about in my LR. All these notes really helped me in writing my LR on the overview of my understanding about the topic. The cite-able notes are the most valuable in my matrix because these notes were the ones that I cited in my LR as references using my own words. So, basically I’ve at least one cite-able note per article and sometimes I’ve more than one per article depending on what I was writing for LR.¬†In a glance, I’ve every article summarized nicely and neatly in table format. Whenever I’ve new article to read, I just simply add another column to the table and make notes on the same aspects like the others.

In conclusion, I’m speaking from my real life experience that LR matrix has helped me not only in organizing my literature review but in saving me time and effort writing my LR in the long run. Making the LR matrix is the best (early) effort ever in my studious life.

LR: Don’t forget your focus

I believe many are simply afraid of the mention of ‘literature review’. I used to be one, literally wanted to hide myself under a blanket and hoped the literature review (LR) would either write itself or just go away. In reality, none of these would ever happen, unfortunately. If I don’t start reading and trying to write, the LR will never get finished. Without LR, my research just won’t happen either. Simply, I would be stuck…forever and ever…

No, that’s not going to happen in this life. I have learned after some practices and many good advice that LR is not so scary after all. Today, I would only describe LR as tedious and full of hard work but not the most scary part in dissertation writing. All I need, I realize, is a good system to make ¬†things work for me.

One of the greatest problem in LR is the amount of articles to read in order to write. By collecting all related articles and seeing them unread in my Mendeley library are already good enough to scare me witless. Sometimes I just keep on reading articles, achieving nothing but only some new ideas from the authors. This just doesn’t work well for my progress in the final submission. So, I have learned to get my thoughts organized and set my focus in LR.¬†It’s always good to remember what I’m looking for while reading an article. I normally look at a few aspects in a topic while reading an article: definition, purposes, characteristics, approaches, rationales and challenges. For every article I read, I only focus on these aspects and try to see what the author has to say about them (if they even mention them). It’s important to note if the authors agree or disagree on certain aspects when compare to others. I collect all these in LR matrix for easy reference later. Whatever the authors’ point of views that related to my research, I want them in my LR to show as evidence or support alongside with my own arguments. It’s always good to show comparisons or similarities between authors’ point of views. That’s where the debate or argument can be further discussed with new insight.

I also learned to categorize my LR by topics as I don’t think all articles cover everything I want to say in my LR. So, I have different LR matrices for different topics, though they are interrelated somehow. By putting them in different matrices make it easy for me to discuss topic by topic and to show how they related from one topic to another when I’m writing my own synthesis. It’s important not trying to cramp everything in one too. Not only I will get lost in my own thoughts or arguments, I’m sure my readers will be lost too. Most importantly, always have focus while reading and always take note. Reading without taking note will only waste time. I normally read fast the first time trying to get the overview. My second read will be together with note-taking in Mendeley Desktop. I conclude my reading and note-taking by copying my notes into my LR matrix. I start to write about the topic when I have enough support from the articles I read and according to the aspects that I collected. During the writing process, it’s much easier referring to the LR matrix as I know which author is saying what and how I use their points as the base for my own argument. Citation is easier too especially if there are more than 1 author say the same thing.

All in all, I hope my opinion about LR here is good enough to help you if you feel stuck with LR too. All you need is just a system that works for you. Feel free to try my method or any other good methods out there. Just explore and try everything until you find one that works for you ūüôā

Literature review process

I spent my last Sunday working on the desk at home, facing my Sony Vaio for more than 8 hours. I was cleaning up my paper clutter when I saw this handouts from one of the workshops I attended earlier this year. It was a Doctoral Research Methodology Workshop conducted by Dr Edward Wong. It was from this workshop that Dr Edward had been telling us about literature review matrix. It was still at this page 5 when I found this handouts where there’s a diagram depicting the literature review process. I studied the diagram again and realized this is a good reference for all.

By looking at this diagram, it seems not so difficult to do the literature review. It’s more like an iterative process (and hard work!). I still remember being told to always start with my research questions and objectives. It’s even better if you can pin up the research questions and objectives where you can see them every day, so that you won’t deviate from your purpose. With that, we define what we want to search i.e. research keywords. With these keywords, we start our search and start to collect articles to be read and evaluate. Summarize the points after reading and put them in a literature review matrix (that’s what I do).

The only thing I haven’t really done right is the start drafting review. I have always thought to complete the search and reading before writing. However, after some times, I found it’s getting more difficult to define ‘complete search’. This stage has become an obstacle in writing. So now I have learnt to start writing even though I feel I haven’t read enough. It seems that while trying to write, I would found some other aspects that I want to talk about, hence I add new research keywords and search for more. Here’s when the process will continue to iterate until the final product of a written critical review of the literature.

I guess sometimes it really takes a lot of wrong turns to reach a destination when you are a novice. The more I err, the more I learn (I hope!). I believe I’m a much better learner now than I first started. Hope this diagram helps you (as much as it helps me) in the literature review process. ūüôā

Synthesis Matrix for literature review

When I first attempted to write on this Chapter 2, which is the literature review (LR) chapter, I was feeling a bit lost as if I was floating in the ocean. The articles I found were piling up and I didn’t know what I should really do about them other than to allocate time to read them. But that was not enough still. Just reading them, of course gave me great insight and new idea but I was still struggling to put them into proper writing. I failed miserably. Nothing was written for months even though I kept on reading those articles I found. I had serious problem in putting everything together to make a story.

When I was in a workshop, I heard this ¬†term of literature review matrix or some called it a synthesis matrix. According to the speaker, all PhD candidates (okay, I’m not! Thank God!) must produce a matrix to show their literature review work and every supervisors should demand to see it too. I was intrigued and went to search more about it.

LR is the most important part of the research (as if you still don’t know) and it can really help you moving forward or it can also act as a boulder that doesn’t let you pass. Sometimes I still feel this LR is the biggest barrier in my research life. Hehe! Why is it so hard? To me, it’s because I didn’t know the proper way to do it. I have learnt that LR is not supposed to just summarize each individual article I found. Yes, summarizing is a must skill in LR but it must be done in the right way. In LR, it requires analyzing skill while doing the summary before you attempt to do the synthesis and finally write the review.

I found some guides in creating a LR matrix and I went ahead to read the articles I found, particularly in the topic of Personal Learning Environment (PLE). Before that, I have already identified what I want to look for while reading the article. Specifically, I’m looking at the article for any mentioning of the definition of PLE, the pedagogy or principle behind PLE, the rationale for PLE, the skills required and the challenge in implementation. These subtopics or themes are the ones that I want to talk about in my Chapter 2 later.¬†When my matrix started to fill up, it quite amazed me. I’m rather glad that I start to see some patterns in those articles I read, particularly the definition. There are many, of course but mostly mentioned about learner control or autonomy and self-regulation in learning. Whenever I’m reading an article, these are what I’m looking at and sometimes some articles would mention some cool argument or idea. If I cannot fit them in any boxes, I would put them in Other category, hoping I can reference to it later if it fits. I also started to see gap in the matrix when the articles didn’t mention any of those that I’m looking for.

Here is my on-going effort on the matrix on PLE. I have started another matrix on Pedagogy, trying to analyze the different pedagogical approaches and technology used in higher education today.

This matrix is in MS Word and the references are generated by Mendeley add-in. I also saw some doing this matrix in MS Excel.

Sadly, I still haven’t started writing my review, though I have a clear outline and idea in Chapter 2. I promise I will start writing soon enough. *Cross my heart*

Problem Formulation

Let me put down what I’m trying to do based on the steps in doing literature review as mentioned earlier. There are five steps similar to doing primary research and here is the first step that I’m attempting to apply: Problem formulation.

In problem formulation, I need to figure out what’s the research questions that prompt me to do this literature review. So I’m digging back my Chapter 1 and here are my aim and objectives to start with:

Aim of research

I wish to investigate, design and develop (no choice but to develop a prototype/system as it’s part of the requirement in Computer Science) an integrated learning platform that support Personal learning environment (PLE) in higher education.

Objectives of research

I’m to further breakdown my aim into achievable objectives:

1. Understand the underlying learning principles/theories that influence the current learning environment and the emergence of new learning environment

2. Identify the factors/requirements for designing an effective learning environment in higher education that promotes learners’ autonomy/self-regulation and social participation

3. Propose a design of an integrated learning environment that supports PLE

4. Develop and evaluate prototype

Research questions

I learnt that research questions are generated from research objectives (i.e. breaking down the objectives further into a few questions that I want to get answers). I have quite a bit of research questions based on the objectives above but here are some general ones that I want to get answers from the literature review:

1. What are the associations of different learning theories to different learning environments?

2. What is PLE (definition, if any)? Why PLE (what motivates this new learning environment)?

3. What are the challenges in implementing PLE in higher education?

4. Can PLE be integrated with formal learning (i.e. LMS) in higher education? Comparison of LMS and PLE?

5. What processes and skills are essential to support PLE?

According to Cooper’s Taxonomy of Literature Review (as quoted by Justus), I have to first identify¬†the following six aspects for my literature review: focus, goal, perspective, coverage, organization and audience.

a) Focus: my literature review will be focusing on pedagogy theories and learning practices/application

b) Goal: my goal for review is to integrate/generalize the topic and identify central issues about PLE and teaching/learning in higher education

c) Perspective: I’m following the qualitative research methods which requires expousal of position (i.e. what’s my standing/hypotheses in this research) – at this moment, I strongly believe that the teaching/learning in higher education is in need for changes in trend with new development of Web 2.0 technology.

d) Coverage: I don’t think I can cover all related articles in exhautive review, so I’m opting for central/pivotal review (i.e. finding all the prominent articles in this topic and to convince you that I have done proper review ūüėõ )

e) Organization: I have planned to organize my review in conceptual format by explaining the different learning environments, the concept of PLE and why the trend.

f) Audience: as this is a dissertation, obviously my primary audience are my supervisor and examiners. However, I also hope this research is useful to similar field/interested scholars and practitioners/policymakers.

Next, I have also outlined the criteria for inclusion of articles. I have first searched the databases with keyword ‘personal learning environment’ and I believe I have found the prominent authors/researchers in this topic. From there, any articles I found will be included if they talk about PLE, self-regulation/personalization in higher education, Web 2.0 in higher education and¬†e-learning. My major resources are journal articles, conference proceedings and webpages (quantity in descending order).

Finally, I’m glad I have¬†devised a¬†rather good plan in this first step. I have already started the second step (data collection) and I will blog about it soon.

Pave the way for literature review

I recently found a good article: A guide to writing the dissertation literature review by Justus Randolph. It’s a long article but I recommend you to read it if you are trying to write literature review too. I agree with Justus that there are not many good journal article about writing literature review, ¬†therefore I really appreciate his article even more.
There’s no denying it that writing literature review is the hardest part of the dissertation and unfortunately, it’s the most important part of the dissertation too. I once heard a speaker said that the examiners always look at the literature review first before any other parts. If the literature review is good, the examiners automatically will presume the dissertation is good. Otherwise, they will presume you are not doing the dissertation right. Scary but it’s true and justified. If the literature review is not thorough to make the argument, what can others expect of the rest of your dissertation, right? That’s why I have been quite worried about doing my literature review. I wish there are clearer steps in doing it but all I found are quite ambiguous until I found Justus’s article.

Read Justus’s article here:
Randolph, J. J. (2009). A guide to writing the dissertation literature review. Practical Assessment, Research & Evaluation, 14 (13), 1-13. Available online: http://pareonline.net/getvn.asp?v=14&n=13.

In my opinion, Justus has clearly written about the different types of review, taxonomy of literature review and most importantly the steps in doing literature review. Surprisingly, the steps are similar to doing primary research and they are as followed:
1. Problem formulation
– What questions we are trying to answer in this literature review? What criteria for inclusion/exclusion of articles?
2. Data Collection
РJustus recommends that the process of collecting articles must be documented clearly and in detail. Apparently, searches through the databases can only produce 10% of articles we want (or is relevant). The other 90% requires us to check the references of the articles (we found in that 10%) and to find/read them. (Imagine the work!) The process is not completed until we reach saturation (which means no more new articles found during this cross-reference).
3. Data Evaluation
– Once we have collected ‘enough’ articles, we have to extract/evaluate information in the articles based on the inclusion criteria that we set earlier. Justus advises to use a ‘coding book’ to record all types of data to be extracted. You have to read his article to find out more because I’m not expert in this yet. However, I believe his idea of coding book is similar to literature matrix that my faculty mentioned.
4. Data Analysis and Interpretation
– With all the data extracted, it’s time to make sense of everything.
5. Public Presentation
– This part is where we start writing, I guess. But first we must find out what’s important to be put in writing and present to the world.
Simple and clear steps but I guess can take very long time to complete because it depends how big your literature review is. And for me, after reading this, I’m no way closer to completion of literature review any time soon. Sigh!