Meet you at The Sipping Point

In this post I would like to share some early reflections on The Sipping Point community which recently started up at DCU. Drawing loosely on the concept of Wenger’s community of practice, the idea of The Sipping Point is to provide an opportunity for staff with a shared interest in teaching to interact with each other in a non-formal setting. The basic premise is to try to foster a climate where staff across all disciplines can potentially learn from colleagues about aspects of teaching practice. The result so far is a gathering of members who meet up for one hour once a month to hear about and discuss various approaches that peers are adopting in their teaching context. 

Why this format?

Based on feedback and research into various academic professional development (APD) courses offered by our unit, (you can read more about those in Gormley, O’Keeffe & Ferguson (2017)), it is clear that not all staff can or wish to commit to the timeframes involved in our accredited offerings. Even for those who do complete these courses, there are limited opportunities for a cohort to continue their learning community and interact further once the module(s) have completed. Once-off workshops are also problematic since they don’t tend to engage lecturers deeply over a period of time. And I think it’s fair to say that we have also noticed a general lack of awareness of what other lecturing staff at DCU are doing that might be of value/interest across the academic community.

Hence The Sipping Point – a new avenue for like-minded souls interested in discussing teaching practice – came into being in April of this year, driven by an enthusiastic combination of positive change, collegiality and coffee. The general format is a one-hour session led by at least one lecturer presenting on a predetermined aspect of their practice for 10 minutes. After that, the topic goes open to the floor for general questioning and discussion. We meet at all three DCU campuses (Glasnevin, St Patrick’s and All Hallows) over the last week of the month.

So how has it gone down?

So far, the reaction to these sessions has been generally positive and the discussion has been lively although I should add the significant caveat that it is very early days yet. Numbers could not be described as enormous – the largest session attracted 18 participants at one campus – but we are pleased with the interest to date and, to be frank, do not expect stampede-like numbers. Initial signs are certainly promising and here’s a quick summary of what we’ve done to date:

  • April – We kicked things off with information-gathering sessions where we identified big-ticket items for future discussion, such as feedback challenges, assessment of large class groups, and lack of student engagement.
  • May –  This month we focused on sustainable methods of giving feedback – annotated documents, peer review, and audio feedback were all discussed.
  • June –  Staying with assessment, we discussed various approaches including group work with peer review feedback, learning portfolios, use of video for discussion assessment, and rubrics.
  • July – In a complete change of tack, and given the summery time of year, we took active participation to the next level, going on a campus walkabout with a learning spaces theme.

learning space walk 2

Any lessons learned?

While it’s certainly too early to reach any firm conclusions about this as a learning enterprise, a few things are beginning to stand out from an academic development perspective:

It’s important to meet people where they are at: I mean this in terms of participants’ geographical location (go to them rather than ask them to come to you) but also in relation to their teaching and learning goals. Not everyone may wish to make a dramatic overhaul of their assessment strategy, for example, but they may be interested in swapping small-scale ideas that can be immediately implemented and have worked well for others. Both types of interest should be facilitated and encouraged.

Don’t overdo the number of speakers: I did receive feedback to say that one of the sessions was more formal than originally anticipated. If we have too many opening speakers, or they speak more than 10 minutes, this will cut into the discussion time which is so valuable to many of those present.

Think about incentives: Anyone who has been approached to present at a session so far is giving up their time which is something that I think should be recognised. While I do hope the community will be self sustaining at some point, I probably need to bribe encourage members to present until we are well established. Tying in with the tea/coffee theme, I’ve approached a potter friend of mine at Brookwood Pottery to design a set of appropriately themed mugs as little gifts for those who take the trouble to present. Reactions to these have been great and I hope that these branded items might also serve as marketing tools in the future. See photo below.

IMG_20170704_105003_903

Of course further research will be required to find out what, if any, type of effect is happening as a result of these sessions. Tea-drinking is almost a national sport in Ireland but is conversation about teaching useful to all involved? What else might be happening and are there unintended negatives to consider? Are these events serving any valuable function at all, and if so, how? More to come on this in future, I hope.

References

  • Wenger, E., 2011. Communities of practice: A brief introduction.

 

 

 

What Martians (and others) can tell us about learning design workshops

OK, so the Martian headline is a little bit of a clickbait but bear with me, fellow earthlings, and its relevance should become clearer in due course. In today’s post, I’d like to talk about some recent experiences I’ve had facilitating workshops intended to help faculty embark on the design of online and blended programmes at DCU. It might be useful to explore what lessons were learned and consider how to potentially apply those lessons to the format of future workshops.

So what’s been done to date? Not surprisingly, time is always very limited so there are typically 2-3 workshops of no more than 2 hours in length. So far, in designing these initial sessions, I’ve deployed a number of approaches to help participants articulate a shared vision for their proposed programme. Drawing on techniques from Conole and Mor, we have included elements of:

  • Persona development: where participants are asked to come up with a credible profile of two or three potential students, highlighting motivators and potential obstacles
  • Course features cards exercise: where participants are asked to select 16 ‘ideal’ course features/elements from a set of pedagogy-oriented prompts
  • MOOC design patterns cards cross-check: participants review further potential course features (eg 6 minute videos, fishbowl approach) and select for their wish-list

These have all proved useful for getting creative juices flowing, but somehow I suspect that when we move the conversation to the learning outcomes something goes awry. Despite some discussions around the required programme and module learning outcomes, I’m not 100% convinced that these are being considered in as much depth as Biggs (or me!) would like. Quite possibly I’ve been making assumptions about how much people already know about writing learning outcomes that are not only clear and measurable, but are pedagogically-appropriate and constructively aligned. So what to do about it?

Well firstly, I believe there is a need to carefully step through the anatomy of a learning outcome, exploring both programme and module requirements, and discussing why learning outcomes matter. For some, this may be a revisiting/refresh exercise. But for many participants, this may be new territory which requires a ‘back to basics’ approach. Let’s be honest here, I think very few people enjoy dissecting learning outcomes but if they are going to serve as the foundations for course design in higher ed, then they do need to be discussed in some detail.  For the purposes of these workshops I won’t get into the politics but instead will focus more on how to write them in accordance with generally accepted standards.

But even the most elegantly written learning outcomes in the world are simply not enough without an assessment strategy that ensures the programme does what it says on the tin. About a year ago, I attended a very interesting EDIN workshop presented by Ivan Moore who introduced us to the principles of Orthogonal assessment, a technique which seems to hold promise as a way of unpacking the detail of potentially fuzzy learning outcomes. According to Moore, orthogonal assessment requires that you stipulate the core criteria by which you will make judgements about whether or not a learning outcome has been achieved.  This requires drilling down into the detail of each learning outcome at the design stage. He argues that this approach puts the focus on assessment of the learning outcome, instead of inadvertently focusing on the assessment components as so frequently occurs.  As the screenshot below illustrates, this approach can be visualised in the form of a table for each learning outcome with the criteria listed on the left and a description of various achievement standards (most importantly the minimum threshold standard) described in relevant columns to the right.

Screen Shot 2017-06-22 at 11.07.20
Above slide extracted from Ivan Moore presentation at EDIN workshop, April 2016

In previous workshops, participants have been asked to write such criteria after the sessions but there have been completion issues with this for several reasons. Writing these criteria collectively, as part of the workshop, may be a better way forward so next time (e.g to support a forthcoming online MSc in medical diagnostics and therapeutics), I plan to integrate this exercise into the workshop itself. To help attendees get their heads around it, we’ll start with a simple example where I ask participants to create the essential criteria needed to assess someone who needs to prove that they can make a cup of tea for the President of Ireland (a proud tea drinker, if I’m not mistaken). The ‘someone’ in this case would be a Martian who, presumably, knows nothing about the tea-making process. By way of illustration, here’s a highly simplistic draft example that lists the various criteria that need to be fulfilled, in my opinion, with some initial descriptors of various levels of achievement in tea-making expertise.

Screen Shot 2017-06-22 at 15.12.49

So to conclude, while I’ve mentioned the ideas above in previous workshops, my new plan is to step through the learning outcome criteria work during the workshop itself, allowing sufficient time for peer review of proposed outcomes and criteria.  Time will tell as to whether or not this approach works (Will participants be able to apply the same rationale to their courses? Is this approach feasible? Will they run screaming from the room?) but unless I’ve been abducted by aliens, I’ll be back to tell you how it goes.

References
Conole, G., 2014. The 7Cs of Learning Design—A new approach to rethinking design practice. In Proceedings of the 9th International Conference on Networked Learning (pp. 502-509).
Moore, I. (2016). Towards Best Practice in Assessment. Presentation for EDIN Conference.
Mor, Y., Warburton, S., Nørgård, R.T. and Ullmo, P.A., 2016, September. MOOC Design Workshop: Educational Innovation with Empathy and Intent.In European Conference on Technology Enhanced Learning (pp. 453-459). Springer International Publishing.

 

 

 

 

Keeping the collaboration love alive

A number of us at the DCU Teaching Enhancement Unit have been having conversations recently about the pros and cons of working with other people on educational research projects. As self-proclaimed blended professionals (as per Whitchurch, 2009), we sometimes become involved in collaborative research projects with lecturers and other staff. Amongst other things, the involvement of a learning technologist or academic developer on a research project can potentially:

  • Help source relevant literature from educational research
  • Advise on relevant learning theory
  • Provide input on methodology and ethics
  • Highlight potential journals/calls for papers
  • Review and provide feedback on abstracts and drafts
  • Advise on the structure and writing of paper
  • Act as a critical friend and sounding board for ideas

The advantages are mutual. For our part, working with lecturers on research projects gives us a deep insight into the challenges of teaching at university, raises awareness of pedagogical issues in particular disciplines, and helps us to stay abreast of emerging and evidence-based technologies and approaches. It also helps us to build up our network and portfolio of publications and who doesn’t want that?

But there’s a but…

While for the most part, these have been very positive experiences, leading to scholarly outputs that were significantly better for the multiple perspectives, there can be some risks involved, particularly with new working relationships. A recent article on co-authoring from Times Higher Education would seem to confirm the benefits of a pre-nup agreement of some kind to help negotiate the process. In particular, the following paragraph on the importance of clear division of labour stood out:

The transition from initial idea to published artefact usually involves a significant amount of time and effort pursuing a variety of tasks. These range from scanning the literature to gathering data, and from negotiating with editors to making the diagrams look presentable. For your co-authoring experience to feel collaborative it helps that these tasks are identified and shared among the members of your authoring team. Be clear on who is doing which bits.

My colleagues and I agreed that it might be a good idea to share the THE article upfront with those who might be new to the co-authoring process. Indeed one of my colleagues is working on a detailed set of guidelines for collaborative authorship, including the thorny questions of author order, what constitutes a ‘significant intellectual contribution’, and ownership of data.

Collaboration Checklist

Taking this one step further, I would also like to suggest that it might be helpful to review a checklist to confirm “who is doing which bits”. Once you’ve agreed that you’d like to work together, then it’s time to get down to some nitty gritty and ask questions such as:

  • What is the agreed order of author names?
  • Who is going to source potential publication opportunities?
  • Who is going to draft and submit the ethical clearance forms?
  • Who is going to write and submit the abstract?
  • Who is going to write the introduction, literature review, methodology, discussion, conclusion? (Or whatever format has been agreed.)
  • Who is going to review and provide constructive feedback on the first draft? How will that feedback be delivered?
  • How often will you meet?
  • Who is going to liaise with the publisher from beginning to end?

Obviously, this is not an exhaustive list but it might prompt some useful thinking about who is doing what and when. I would be of the view that the person listed as first author should typically do the lion’s share of the above – but it is entirely context-dependent and some writers may prefer a much more organic (and arguably more collaborative) approach to the writing process.Ultimately, it should be about working well together so you have to go with an approach that suits all involved. If you can establish that approach sooner rather than later, the chances of a blissful (or at least relatively harmonious!) collaboration are greatly improved.

Reference

Whitchurch, C. (2009). The rise of the blended professional in higher education: a comparison between the United Kingdom, Australia and the United States. Higher Education, 58(3), 407-418.

Peer Review – some lessons learned & some friendly advice

Inspired by events and discussions at the recent Y1 Feedback Symposium, I’ve been mulling over how I can improve the peer review process in an online staff development course that I teach. During his presentation, assessment researcher Professor David Nicol made the point that students seem to learn more in conducting a feedback review than actually being the receiving party. The Nicol, Thomson, and Breslin (2014) research on peer review makes convincing reading about the evaluative and cognitive benefits of the review process, and  I’ve decided to share that paper with my own students to explain why we’re using this approach.

This got me thinking about how I’ve used peer review (actually, I called it peer critique, which is probably not ideal) in the past. While not claiming to have vast experience with the approach, I have used it with a class of online students who were asked to give each other feedback on proposed strategies. Some unexpected issues came to light:

  1. Some peer feedback comments were interpreted as being undiplomatic and irrelevant
  2. Providing what I thought were helpful ‘prompts’ for questions/comments ended up being regurgitated directly, in a small number of cases
  3. While some students loved it (particularly those working in similar disciplines), some participants were unconvinced (as they felt they did not receive the type or quality of feedback they would expect)
  4. Because students were free to discuss using any communication mode of their choice, several offline conversations were not visible to other students or me
  5. Some students did not engage at all with the process and some left significant elements out (eg did not explain what aspects of feedback they planned to incorporate or omit)

There appears to be a fine line between providing appropriate scaffolding and micro-managing the process for students: you want to give them enough information to know how to get started with a peer review but not so much that it becomes a simplistic or somewhat pointless exercise. At the symposium, Prof. Nicol made the point that students should ideally generate their own criteria for quality when conducting peer review. While I might see this working in a small group face-to-face setting, I am not sure it would be as successful in an online context where silence (the equivalent of the blank page) is simply easier to ignore. So I suspect the online learner requires the provision of at least some guidelines as a starting point for the discussion and you can find helpful advice on peer feedback forms from the University of Hawaii, Manoa. Of course, to a large extent it depends on the course design but in the online context I think I’d prefer to play it safe by providing at least some suggested criteria or guidelines at the start.

So trying to tie all this neatly together, here are some ideas for the next time I use a peer review approach in online teaching:

  • Record a screencast or video that captures my thoughts ‘thinking out loud’ as I read and annotate a sample draft (this might help to model suggested feedback and tone, addressing points 1,2 and 3)
  • Set up a central location (such as a discussion forum) with a designated thread for each pair to respond to online (this might assist with points 3 and 4 by increasing visibility into the process and enabling greater access to other classmates’ reviews)
  • Create an infographic that provides a recommended pathway for the assignment, from beginning to end (this might clarify the multiple steps involved, addressing point 5)

As luck would have it, my 14-year-old daughter has been doing a peer review for her English class in recent times and has been subject to intense questioning about it by her mother. I should add that she is completely unimpressed by it as a learning device (sigh) and reckons that it is too hard to “be honest” with one’s friends. I know it is the job of a 14-year-old to be unimpressed by everything but she might have a point about that – could anonymity help or hinder the peer review process?

If anyone else has comments on proposed or previous experiences of online peer review, I’d love to hear your thoughts. Speak up, my friends.

References

Nicol, D., Thomson, A. & Breslin, C. 2014, “Rethinking feedback practices in higher education: a peer review perspective”, Assessment & Evaluation in Higher Education, vol. 39, no. 1, pp. 102-122.

 

The White-Knuckle Ride of the Virtual Classroom

motorbike on rollercoaster

 

 

 

 

 

 

 

Photo Credit: oiva_eskola via Compfight cc

Imagine if you had to check if your students could speak at the beginning of every class? Imagine if you were never 100% sure that they could hear you when you spoke? Imagine if you never knew for sure if the doors to your classroom were open (and would definitely stay open) for the duration of your class? Welcome, ladies and gentleman, to the rollercoaster ride of teaching in the virtual online classroom.

I’ve had the pleasure of teaching a professional development module on Online Teaching for the last semester at DCU. As you’d expect, we’ve used many different formats, approaches, and technologies – discussion forums, multimedia learning objects, Padlets, Soundcloud, Articulate, Google Docs and more. But without a shred of doubt, the virtual classroom experience has been the most challenging for me to get to grips with as a teacher.

Now part of me wonders if maybe it’s just me. But I consider myself moderately technically competent. I don’t think I’m stupid. And I’ve had the benefit of some practice and tinkering with systems such as Adobe Connect and Google Hangouts in the past. In fact, I’m part of a local (‘Meitheal’) community of practice where we regularly connect using web conferencing technology. So wouldn’t you think it would be a walk in the park? Well let me give you a flavour of how things went:

Webinar 1 –  over 20 participants attended and in retrospect, this was probably the most successful event technically. A written guide to Adobe Connect was distributed to attendees before hand, along with specific requests to complete connectivity diagnostic tests and audio configuration in advance. I benefited from great advice from Suzanne Stone about easing students into it with a fun, ‘no pressure’ orientation. At one point students were invited to use their microphones to tell us about their day. Despite some minor hiccups, most students were able to participate and use the mic. My colleague Pip Ferguson was also there to help moderate the discussion and keep track of the chat box activity while I led the session.  (Stress-o-meter rating: High. This was my first webinar of the course and I wasn’t sure how it would go.)

Webinar 2 – about 11 participants attended but it turned out that several others had tried and failed to access via Adobe Connect. This was due to an institution-wide ISS issue that I had no control over. That was rather disconcerting but at least I knew the session would be recorded. After being inspired by advice from Michael Hallissy, I was keen to use breakout rooms as part of the webinar so that people could split up into groups to discuss what they heard. Turned out my genius plan to split people into specific groups took longer than expected and there was noticeable delay while I moved people around into their respective groups.(Stress-o-meter rating: High. Dead air for a while – the minutes felt like hours.)

Webinar 3 – By now I was hoping that all of the teething problems would be sorted. Turns out that this time, several students had issues using their mics and there was a lot of not-so-scintillating “Can you hear me? No. I can hear you but you can’t hear me” conversation going on. This became particularly problematic during the breakout sessions which had to revert to chat box conversation mainly. This was particularly frustrating since I wanted participants to be able to talk about their online collaborative projects, by, well, collaborating online. (Stress-o-meter rating:  High. Growing despair about the way the technical issues were affecting how I wanted to run the course.)

Webinar 4 – about 14 participants attended. This time the technical problem was squarely at my end. Despite having reminded students about the importance of preparation, I was the one who couldn’t share my voice this time. Thankfully my colleague Muireann O’Keeffe was able to take the lead until I got sorted out. (Stress-o-meter rating:  Off the chart initially, when I couldn’t ‘speak’. Reduced by assistance from presenter who led the event at the start. )

So where am I going with with this saga, you may be wondering? And why does it matter to anyone else?  Well based on my own (limited) experience, and because forewarned is forearmed, I’d like to offer anyone considering hosting a virtual classroom session/webinar some words of advice. Here are five  critical conditions, as I see them, to give you a decent chance of success:

  1. Allow time for technology orientation – both for you as the host, and the students. This is particularly important if you want students to use the mic (see 4).
  2. Allow yourself flying time – expect a few wobbles as you gain confidence and skills. You will only improve over time but you won’t be perfect to begin with (if ever!)
  3. Have some technical backup – have a colleague and ideally some form of Tech Support on hand to help you moderate the discussion and/or step in as needed
  4. Do try to make the sessions interactive – it is probably much easier to broadcast rather than hear back from the class but then why bother with a webinar at all? If you’re doing all the talking, there’s a question to be asked. Polls, chat box questions, and most of all students’ voices help to democratize the learning experience.
  5. Run them again, and again, and again – the only way to get better at these, I suspect, is through frequency. Torturous as it may be, it’s probably best to practice as often as you can. They should get easier but I’ll get back to you on that!

I would also love to hear your experiences and thoughts too, so please do reply if you’ve got something to say about this. There are undoubtedly many many factors that determine if you will have a disaster or a dream live classroom event. And, of course, not all of them are within your control. But who ever learned by playing everything entirely safe?

 

 

 

 

LI501 Update: From Theme 1 to Theme 2

The title of this blog post will probably only make sense to participants of LI501, the Online Teaching module I am currently coordinating so I may revise it in future to something clearer. But for now, it will suffice.

As I write this, we are just over two weeks into the course, and we have completed the first Orientation & Reflection theme. The video below gives a general picture of how things went and also sets the scene for the forthcoming theme of Discussion:

What are the areas for improvement?

  1. To be blunt, I don’t like how things look on Moodle! I sensed from a few of the students’ discussion posts that they too were struggling to perhaps find things and make sense of it all. I need to figure out a way to show fewer resources. Although this approach is not unusual in a MOOC, it is possibly overwhelming, messy and confusing for those unfamiliar with that format. While I did manage to combine a number of resources within the Discussion Learning Object, I feel there is still a way to go with the interface. I may yet incorporate this blog in some way.
  2. I need to make it clearer that students can contribute resources/ideas via Diigo
  3. To keep the focus on active learning, I feel I need to move away from the learning object approach for the next theme at least – I may create one for the Creating/Reusing Resources theme but will confirm based on how things go over the next fortnight. I am still concerned that there may be playback issues despite hosting the learning object on Dropbox (instead of Drive).
  4. I need to use a visually appealing way of presenting the task for the Collaboration theme (which is the third one of the series). Above all, I wish to make the the instructions for the collaborative activity around course design crystal clear. It is a complicated enough activity without adding any additional layers of confusion.
  5. I need to ensure that Module Facilitation is my top priority and focus my time and energy on that rather than on creating new resources. For this theme I particularly want to ensure that students critically consider what they read and hear so my challenge will be to create responses that probe what is said.

As I  write though, I can see that the first post of the Discussion theme about the online learning animal has just come in. And indeed another participant has tweeted about it. Hopefully that’s a promising sign of things to come.

 

The Early Days of an Online Teaching Course

It’s day two of LI501 and all is rather eerily quiet so I would like to take some time to reflect on where I am as a tutor at this point in time. LI501 is the online teaching module that I’m currently co-ordinating at DCU. We got off to a flying start yesterday (with a three hour orientation session) and signs are very promising that this is a chatty bunch with plenty to say about their knowledge (or perceived lack of knowledge) in relation to technology. Whether this enthusiasm for talk will translate to the discussion boards remains to be seen.

We have a fantastically mixed group of schools (Business,  Health & Human Performance, Engineering, Science & Languages) were all represented. It was fascinating to hear the participants talk about why they have signed up for this course. Issues with large class sizes, differences in levels of understanding, and a desire to do ‘more’ with technology were all mentioned. But it was a feeling of lack of engagement that seemed to run through most of the comments and drove home to me how important that particular theme will be. While it is ‘officially’ due to start in several weeks, I see the activities that we engage in in the run up to that theme being highly relevant to the engagement objective too.

For my part my biggest challenge for now is to start building up that all-important sense of instructor presence and learning community. Through my meetings/conversations with the participants, the face to face session,  photos, videos, and even emails,  myself and Pip are doing our best to create a supportive and welcoming environment that invites participation. But will that be enough?

While I did emphasise the importance of discussion during my presentation, it remains to be seen if it actually occurs and I am nervous that participants just won’t engage. I should mention a ‘moment of doubt’ that occurred to me earlier. I became conscious that one of the students seemed confused about how the discussion would take place online (wondering how it would happen in reality). I think they were also wondering about when to do the discussion. As I write this I realise that I am in the fortunate position of having a video recording of my talk so I can go back and look at that and confirm what was said. Very useful indeed and since this was my first time using that camera, the benefits for both the students who had to miss aspects of the session  and for me are quickly becoming apparent. Getting over my discomfort at being recorded for this and the introductory videos will have to happen sooner rather than later  and I can see myself using more video technology for my own teaching into the future.

But for now, to go back to the question of discussion, I have made one tweak to the layout to make the discussion forums less ‘buried’ in the page. I felt I needed to make the discussion tasks very prominent so I’ve included some intro text and links to those tasks at the top of the theme to make sure participants have a clear idea of what to do when they login. That’s about all I can do for now, it’s a bit of a waiting game but for now, I’m happy to wait and see… .