The Voice Magazine previously interviewed Dr. Kumar in April of 2014. That interview looked more at his background, while this time, Marie Well interviews him about some of this theories and accomplishments. You can read the first and second parts of this interview in our archives.
Marie: What is your particular philosophy on student evaluation?
Dr. Kumar: Ooh! I use the word evaluation even though I tend not to like that word too much. If we choose to use evaluation, it should be continuous. Imagine a student studying a course. Let’s say, at best, 10% of the time is spent evaluating the student’s understanding of the material. So, what we emphasize is the 10% of the time. Of course it is important to see formatively how the student has progressed. What we need to do is look at the big picture. 90% of the time, is it possible for us to observe, with the student’s permission, of course, how students study and translate that into part of my assessment? So, I would go for the 100% continual mechanism for assessment than the 10% assessment mechanism that we seem to be currently using, predominantly.
Marie: What pedagogical standpoint is most reflective of your way of teaching?
Dr. Kumar: What I preach to students about analytics, basically what I’m saying is “Students, this is what you are studying. Continually record what you study, reflect on it, regulate on it, so you can see where you are heading. The same thing applies to me as a teacher. What I need to do while teaching analytics. I need to continually record how I teach. Whatever I have said in the last five years to my students, I should have a recording of it, analyze it, find out what was effective, what was not effective, and make changes to my teaching habits. That is something I really strive to do?what I call teaching analytics. I think That’s what every teacher should do, given that information.
Marie: What technological shifts would you like to see online education take in the future?
Dr. Kumar: Well, my core research is about learning analytics. As a by-product, I expect to produce what is called a causal model. To give you an example, let’s say that we are able to observe every study-related activity that the student had performed for one full month in any given course. At the end of my thirty-day observation, first of all, is it possible for me to list all of the skills exhibited by students, or by this particular student with respect to the study processes? Is it possible for me to understand all the competencies that the student had gained in the last thirty days? I would confidently say yes. Now, we have the technology. In fact, we now have the ability to say, yes, it is possible.
Now, we are applying the same technique in at least eight different institutions in India because they all heard about this possibility and they all want to be part of it. India, Taiwan, China, and of course Athabasca University. So, learning analytics is going to take on the future of allowing students to see for themselves what happened to me when it comes to learning in the last any number of days. Just imagine the same information available to, say, teachers. This is my class of eight hundred students, twenty students, whatever, and I have information about what had happened in their study life in the last so many days. That’s a beautiful piece of information to have. As a committed teacher, I would be glad to have that information to make changes to how I would go about teaching the next month of my instruction.
So, technology is shifting more and more toward analytics, and we are pioneering that direction, I would say, at Athabasca University.
Marie: Can you tell us a little bit more about analytics, Doctor Kumar?
Dr. Kumar: Analytics is a way of collecting raw data about student’s study activities. Let’s say the student reads something today, assuming it is an etextbook. I can look at the speed with which the student read. I can look at the eye movement and the facial expressions to figure out what has been understood and what’s not comprehensible for my student. I can make reasonable guesswork on that. That is with reading. What about writing? What about conceptualizing? Critical thinking? Communicating, for example, solving math problems, or counting problems. I have a course on storytelling where students actually draw to create the story. I can capture that. Visualizing, creating things, with visual analysis, I can create a table that makes sense to others. Narrating, comparing, so, students are engaging a whole pile of activities.
We do have this technology now that allows students (please note the word “students”) to track their activities at the finest granular level. With their permission, they can share this raw data with researchers such as myself. We compile this raw data into meaningful pieces, such as “well, these type of learning outcomes you are supposed to have, and these are the outcomes you have targeted, and these are the levels of achievement in each of these outcomes, and this is how you can go about finishing the rest.” So, we can forward that kind of advice, not just during assessment, but also on a daily basis?even an hourly basis?to our students. So, That’s what analytics can do?continually observe what students undergo in their studies and translate those activities into measures such as competence or confidence and even match those activities and compile them into meaningful chunks for administrators, politicians, or, say, parents. My grandparents, if they were alive, would be glad to see what I’m doing today, right? Just imagine my grandmother now being able to see on a daily basis what my grandson achieved. What did he do? What made him tick? What made him happy today? Just imagine if that is available to them. That is now possible with analytics.
Marie: What is your view on social media in the online learning environment?
Dr. Kumar: I believe it is an essential part of the learning environment, mostly because it aids, going back to the mandate I mentioned before, in the knowledge creation process. Universities have knowledge creators. They ask the public, “Give us some money,” and social media helps us reach that goal. In that sense, I would like to have or explore the social media to push our students toward knowledge creation.
I did use it in one of my courses, COMPS 683, Learning and Knowledge Analytics because the course is designed with student contributions to media. What I want to do next is use social media in my research methods courses. This allows our students, for example, to connect with or explore research methods from around the world in other universities, so that is what I’m trying to see happen in advanced research methods courses. We do have a COMP 494, research methods course in computing for undergraduate students, and we also have a course 695 for graduate students, and I would like to bring media in there so that students are exposed to what else is happening around the world on research methods.
I have a caution here. This is just one more media, one more outlet, one more communication channel. I don’t want students to see this as there one only chance of exploration. Did you read the news today, this morning, that Future Shop is closing most of its stores in Canada? Do you know the reason they give?
Marie: Amazon?
Dr. Kumar: That’s competition, but they say people go to stores to have a firsthand touchy feely of the technology, so they can see for themselves and feel comfortable with it before they buy it, right? That’s what drove technology companies to have stores. But, now, they say that the same feel can be had by looking into the social media. So, I don’t really need to touch a machine to see how well it is going to work for me; instead, I go to a website and look at all the commentaries of people who had purchased that machine before and, based on their user experiences, would put some commentary in the media; that, I believe, is much more valuable. That is what is revolutionizing social media in the sense that once we have the necessary amount of feedback from the common public, we are in a position to powerfully challenge common notions.
So, the same thing can be said about traditional universities. Why would I say that traditional classrooms are good for me? [laughs] Anyway, I’ve taken it way off. The point is, social media is really becoming a powerful tool.
Marie: How do you hone your teaching practices?
Dr. Kumar: Teaching always confused me because, when you go to a conference about teaching, they always talk about, well, this is what happened in my classroom and this is how you can apply the same technique I used in your classroom. I am really kind of worried about this notion of generalizing teaching techniques. If someone says, This is what I did in my classroom, and these are the kind of students I had, and these are all the constraints they face, and this is what they are able to achieve because of these teaching practices, I think we need to use the case study as it is. I don’t want people to generalize it. That’s how I would like to see the world transfer its knowledge–exactly how it happened. When it comes to teaching, and I’m not talking about other domains, generalization is not helping much. We need precise information. That’s my feeling, and in that sense, That’s what I try to collect. I try to keep a really rich set of data about how the teaching worked or why it didn’t work.
Marie: If you could wave a magic wand and improve one thing about online education, what would it be?
Dr. Kumar: That challenge is this, which I talked about before: in learning analytics there is only so much we can do to estimate the mindsets of my students. Their capacity to learn, their study constraints, their levels of motivation, what are the triggers that would help them become better motivated students? What is their cultural background and how does that influence their study habits and capacities.
Is it possible for me, as a teacher, to get that holistic mindset, so that I can customize my teaching to suit the mindset of individual learners? If I had a magic wand, That’s what I’d wish for: the mindset of my students.
Marie: If you could give one piece of advice for online learners, what would it be?
Dr. Kumar: Again, I have to go back to my favourite research goal. I want students to continually analyze their learning experience. Continually, on a daily basis, on an hourly basis, they should analyze it for their own benefit. It’s almost like doing your exercises or eating your apple every day. This is something you should inculcate, build into, our study culture. They go back, take time out, and analyze “this is what I studied today, this is my experience. What did we learn from it? Where do we go from here?” That is what I would advise to my students.
Marie: So, how can I start analyzing my learning experiences?
Dr. Kumar: Do you have enough data, first of all, to analyze? Many students don’t. When I study, to be honest, what I do is record my video. I spend ten minutes reviewing a paper, and I record it with my video camera. I want to see how many times I lift my head out of the reading to think about it?how many times I take notes. Eventually, I want to see what I wrote as my comments as a review of this paper.
By doing this, it really helped me to understand my level of commitment. At times I see that I know this contributor from this journal, and it brings me down. I know the quality I would normally expect. The same level of commitment I would give to a top level journal article, I would do the same for anyone else. This kind of analysis helps me to keep my judgement keen, straightforward. I would really want my students to take this approach. You are not losing privacy, unless you do lose your data. Try to collect data about your study habits, thinking processes. Then, analyze it based on evidence that you have, not just based on some vague memory.
Marie: So, I could record audio, video, or maybe a statistical software package?
Dr. Kumar: Not only record, because recording gives you raw data. You have to translate that into your piece of information. Then move that information into, say, I’m good at this and I’m not good at that. So, you need to translate the information into competences. Say, these are the competences I don’t seem to have, but my goal, four years down the road, I want to be there. You should do the GAP analysis. This is something students should build as a day-to-day routine. By the time they finish their first year, they should have absolute confidence as to that.
These are things I already know because I’ve done analysis, not because of what I see in the transcript. We are not trying to sell the transcript here. We are trying to sell your own ability to analyze yourself. I think That’s what we should promote with our students. With online learning, this is very possible and very easy. Basically, we are asking students to be researchers of their own, right?
Marie: If you could confer one piece of wisdom on an upcoming student in computer science, what would it be?
Dr. Kumar: This is the easiest question. This I normally do on day one with my master’s students and grad students. I typically ask them, “What would be your one contribution to humanity five years from now?” Then, the follow-up is, “What do you think humanity will think of your single contribution five years from now?” We need to see things from an external perspective as well. Then, I want them to compare, match these two viewpoints, and then work toward the goal.