First of all, our team wishes to thank the DML Competition for the opportunity to explore and play. We have already learned lots from the development phase. I expect to gain more insights and develop additional best practices as we move to the maintenance phase of the project.
I’ve pulled key points from our full report that may be of most interest to this group. The key points are:
What are the three essential questions the field needs to answer to move learning playlist design and implementation forward?
- How to create social engagement for continuous enrollment, self-paced learning? We know that education has a social component to it. The Community of Inquiry model has the Social Presence as one of its core components. How can a social component be designed for a group of learners who are completing their course independently?
- What is the characteristics/demographics who are particularly drawn to self-paced learning? How can we leverage those interests to increase participation? Alternatively, how can we shift our designs to appeal to learners not typically engaged?
- How can we increase the badge currency when the badge is issued by a unit that is unknown to the general public?
What did you learn through the design process? What would you do differently if you were to start over?
One instructional design that works for us is to create XPs that are included in all the playlists. For instance, we created “Showcase: Your work” and “Update: Your resume” for two reasons. One, we want to ensure the learners are explicitly guided to articulate how the artifacts they created and the experiences they gained that have real-life relevance. Two, this design help the instructional designer think about consistent user experience for all the Sky, Water, Earth playlists. In addition, we will create an XP that solicits feedback from the user on the usefulness of the playlist. This XP will also be present in every playlist and would provide valuable feedback to the instructional design team.
We feel learners would benefit from words of encouragement to stay on track and motivated. This would be especially true in a self-guided environment where there are no scheduled check-ins between the learners and the evaluators. For particularly complex playlists, we included words of encouragement in the XPs. For example, the first XP in the playlist on research design, we wrote “Hats off to you! You’ve made the decision to undertake this playlist towards building your foundation as a scientist or researcher! This playlist will hopefully challenge you in ways that will stimulate your growth as a professional, and you will rise to meet it.” In a later XP, we wrote “Great! You’re on the right track! You’ve built a solid foundation for your project!”.
What are the 3 most important things about designing your system or solution that you would share with another organization just getting started?
- Know your goal and who are your learners. Throughout the development process, we have repeatedly returned to our goal of developing career-oriented competencies for our target audience. This was helpful to help us narrow down the partners and the project scope.
- Know your asks. When reaching out to partners, it is key to first fully understand what you’re hoping both you and the other party can gain from a meeting or conversation. Even if it is simply to share your idea of the project and gain a third-party informal perspective, it’s important to know what you want to achieve. Otherwise, it’s easy to feel like you have taken a lot of steps but not made much progress.
- Know the resources needed to build the content and a community of learners (buy-in from senior management, common goals from all major project partners, personnel necessarily to review the submission and interact with the learners).
What is left to do? What is left unanswered? What might help you continue to succeed?
How can existing playlists be introduced to a different target audience, such as underserved youths who are not currently in the formal education system? Would the content need to be modified, and how?
Evaluation mechanisms are also left unanswered as LRNG’s single point of evaluation is not scalable. Having a peer evaluation functionality (similar to what is available on any of the MOOC platforms) would lessen the personnel needed to support the learners. Is there a feasible way to provide feedback to learners to further boost and encourage their learning or is simply completing each XP in a playlist sufficient? We also wish for learners to engage with one another via online communication for questions, encouragement, and feedback. For now, we are exploring creating a muut.com discussion community for each playlist. This allows the learners to share their artifacts and ask questions to like-minded learners.
At the crossroads of both evaluation and sustainability, are a few additional questions: how can expert evaluation proceed throughout the years? We have commitments from our partners to engage with learners, as we believe learners do need some positive reinforcement while completing their submissions. However, how will this look one, two, five years from now? Similar to this are questions of scalability. Our current design centers on partners within our local region of British Columbia. Is there value for learners outside of this area to participate in our LRNG playlists? Is this something that needs to be considered now?
What are the three essential questions the emerging field of connected learning playlist design needs to answer or make happen in order to move your work forward or scale it?
- What is the relationship between connected learning opportunities and more formalized institutions of learning?
- How can we build upon the resources of online learning and communities to improve feedback for learners, evaluation mechanisms and mentorship?
What have you done, or do you plan to do, to evaluate the efficacy of your learning playlists in your community/communities?
Obtaining feedback from relevant stakeholders has been a huge part of our design process thus far and will continue to be as we roll out the learning playlists to users. First, we plan to gauge the interests and motivations of learners when they start Sky, Water, Earth through a survey to understand what brought them to the platform and what they expect to gain from participating. We will also host additional student focus groups for learners to beta test our playlists, providing feedback of wording, activity feasibility and overall interest. Both of these steps will help us in throughout re-development stages to identify areas that align with student goals and those that may need to be edited.
Additionally, we have been working with UBC students, faculty, and staff to review our learning opportunities and feedback of perceived effectiveness, and make connections for use of the playlist in formal learning environments. Doing this has prompted more ideas of Finally, our partners at the H.R. MacMillan Space Centre and the Vancouver Aquarium have been essential in providing an employer/industry perspective of learning activities- how the playlists can truly strengthen learners’ skill sets while providing real-life opportunities and experience.
In our final Hackathon in March, we appealed for activity feedback from our participants. Participants were divided into three groups to discuss different elements of lessons as well as the content. Feedback was primarily aesthetic in nature, emphasizing the benefit to showcasing activities on the platform versus in written form. In terms of content, we received feedback that emphasized the need for activities to be more informal and accessible for all students– a lesson should not feel like an extension of school. This reflection was influential in shaping how we would introduce lessons and ‘hook’ students to each activity.
What are the 3 main challenges to widespread adoption or scale of these learning playlists for your organization?
Aside from the platform functionality limitations, here are 3 challenges:
- Because LRNG is non-FIPPA compliant (student information needs to be hosted on servers located in Canada) we can not mandate UBC students to use the platform.
- If the badges are issued by the University, the submissions need to meet a certain academic rigour. However, given the students are accessing the content for free, it would require an unsustainable amount of instructional personnel to provide the necessary feedback.
- Within the university, skepticism from some instructors to “release” their activities into the public domain. In addition, when we are introducing the playlist design to an educator, we needed to spend time discussion both the new design concept and an relatively unknown platform that has not proven itself to be sustainable. This makes it very difficult to convince educators to place content on the platform that may not be around in the long term.
What plans are in the works, or do you plan to put in the works in order to sustain your system?
We are soliciting peer evaluators through student associations and the Teacher Education program so students are Reviewers who provide feedback on the submitted XPs. The provides the students with instructional experience.
Furthermore, we will introduce the playlists to learners beyond UBC so other educational institutions or municipal offices would see the value of continuing the project and provide ongoing funding to hire dedicated instructional staff.
How are you getting institutional buy-in, or adoption from your learners or other stakeholders?
It can be difficult to convince educators to contribute to an online community that has not yet made a name for itself. Similarly, to raise awareness from the learners to invest their time and energy on an unknown platform. We needed to work on buy in from the senior administration level and have the call for participation come from that level rather than from the instructional support staff level. We also need to demonstrate to the instructors the LRNG community is expanding and there is a growing momentum of content contributors and users.
As the DML group had identified, increasing the badge currency is a major task. We began the conversation with admission committees at our faculty about accepting (or at least valuing) the playlist badges as part of the application package. We realize in the university academic setting, making that into a reality is a very long road. Currently, the Sky, Water, Earth badges do not have much extrinsic value to the learners. We follow the best practice of linking each badge to well-established competencies (in our case those defined by the National Research Council of Canada). This does give the learners and teachers who reviewed the playlists more confidence the earned badge will have more weight. We are also using a large portion of the grant to secure the real-life opportunities (tickets to networking events, honorarium for experts to attend the informational interview with the learners, etc). These opportunities do further motivate the learners to complete the playlists. However, we are aware this model is not sustainable long term without additional funding. We hope once we demonstrate the playlists are reaching and helping a sizeable portion of the population, we will be able to apply for additional funding from the municipal or provincial governments.
What outreach strategies will you employ to communicate and support your playlists?
We contacted:
- Undergrad science instructors, advisors, and program staff to share Sky, Water, Earth as an additional resource for their students
- Potential to target new UBC students or those in dedicated programs where more support is available (Academic on-ramp programs, Co-op programs, etc.)
- Career/college advisors at secondary schools
- Career centers at local universities and colleges sharing both the concept of connected learning playlists and how Sky, Water, Earth can complement their existing resources for students
- Educators and instructional support staff at various communities of practices, conferences, seminars. We are targeting ProD conferences for educators (e.g., the PSA Superconference).
It is also important to tap into career-oriented outreach programs offered through the community resources (libraries, science centres, community centres, etc). These existing programs can be integrated into the playlists as an XP, as the real-life opportunity offered to learners who unlock the badge, or as a stream of reviewers for the playlists. For example:
- The Vancouver Public Library’s #DreamJobs series explores new and upcoming careers. Participants hear panelists from key industries speak about what inspires and excites them about their chosen field. Contribution from Sky, Water, Earth to this existing program is to help live stream the event and show the VPL staff how they can stream future events on their own.
- http://www.skypeascientist.com/ matches scientists with classrooms for a Skype session where learners can ask questions related to the scientist’s expertise or what it’s like to be a scientist.
- https://www.scienceworld.ca/communityscientist Science World’s new Community Scientist Initiative (CSI) provides training to scientists, researchers and other science-based professionals with a keen interest in engaging with public audiences. The main goal of this project is for the scientists to develop programming for the Science World (not affiliated with Sky, Water, Earth). However, we are working on partnering with Science World so these trained scientists would also interact with the online learners on Sky, Water, Earth.
- We also work to become one of Faculty of Education’s Community Field Experience partners. In this program, teacher candidates complete a 6-week practicum in the community. This program helps teachers develop a broader, more holistic view of education that go beyond the traditional classroom settings. Nevertheless, the program has not considered online educational programs as a potential opportunity for practicum and there are a few logistically issues. We continue to explore this possibility because it has the potential to supply long term highly trained educators who can interact with the playlist learners.
Conversations with these community resources are very time intensive but the payoff can be long term partnerships that are sustainable and mutually beneficial.
Is there anything else you would like to share with us about your progress?
UBC is also considering using the LRNG platform and the playlist format for faculty and staff professional development on various educational technologies. For instance, how to create a WordPress blog, how to use Office software, etc. We would like to find out if the learning playlist design (compared with a webpage with a list of how-to videos on how to use a technology) is more useful and motivational to faculty and staff.