Pedagogical Delivery and Feedback for an Artificial Intelligence Literacy Programme for University Students with Diverse Academic Backgrounds: Flipped Classroom Learning Approach with Project-based Learning

Bulletin of the Technical Committee on Learning Technology (ISSN: 2306-0212) Volume 22, Number 1, 8-14 (2022) Received January 26, 2022 Accepted March 28, 2022 Published online May 25, 2022 This work is under Creative Commons CC-BY-NC 3.0 license. For more information, see Creative Commons License

Abstract:

Keywords: Artificial intelligence literacy, flipped classroom learning approach, pedagogy, project-based learning, university student.

I. INTRODUCTION

The digital transformation of our societies is affecting our lives in unprecedented ways. Artificial intelligence (AI) plays a vital role in this transformation. The challenge of creating decent human-centred work is on the verge of becoming much more difficult as AI remakes employment landscapes around the globe [1]. At the same time, more people and communities are recognising security, privacy and other ethical issues involved in the use of AI that need to be addressed. Given that AI is relevant to all of us [2], it is necessary to cultivate AI literacy among all citizens [3, 4].

To date, most studies of AI literacy have been solely aimed at computer science-related majors and have emphasised programming knowledge [5, 6, 7], seldom covering ethical considerations [8, 9]. There has been little consideration of effective means to foster AI literacy among participants of diverse academic backgrounds. Specifically, the areas of what to teach (curriculum) and how to teach
it (pedagogy) need to be addressed.

To fill this research gap, this study developed and implemented an AI literacy programme for university students of diverse academic backgrounds. This paper reports on the curriculum design and delivery of the AI literacy programme, incorporating feedback from the participants. It aims at addressing the following research question: is the flipped classroom learning approach with project-based learning adopted in this programme effective for fostering AI literacy among university students of diverse academic backgrounds?

II.    Curriculum Design

The curriculum of this literacy programme was designed based on the multi-dimensional conceptual framework of AI Literacy proposed by Kong and Zhang [4]. The cognitive dimension of this framework focuses on educating people on basic AI concepts and developing their competencies in using AI concepts to evaluate and understand the real world. The affective dimension emphasises the empowerment of participants, enabling them to react to the widespread use of AI in their daily lives and workplaces. The sociocultural dimension is aimed at encouraging the ethical use of AI.

Three content domains were the focus of this literacy programme: Machine Learning, Deep Learning and Developing AI Applications.

A. Course 1: Machine Learning

In Course 1, an introduction to the development and application of AI was provided, followed by content related to machine learning and the concepts of strong and weak AI. Participants were guided in a discussion of the potential and impact of AI in our society.

The participants then learned the ‘five steps of machine learning’: defining a problem, collecting data, pre-processing data, training the model and inference and prediction. After that, they were provided with hands-on experience using the five steps to conduct image recognition with Google Teachable Machine (https://teachablemachine.withgoogle.com/). Following this, the participants learned about two instances of supervised learning: ‘regression’ and ‘classification’. Related examples and hands-on experience were provided. Finally, the participants were taught the concept and working principles of unsupervised learning by applying k-means clustering in a series of case studies [3]. Overall, the course delivered major concepts concerning machine learning via analogies and real-life examples, aiming to foster participants’ understanding of the major concepts of machine learning and the principles underlying these concepts [3].

B. Course 2: Deep Learning

Course 2 focused on deep learning, building on the participants’ foundation in machine learning. It comprised data cleaning and augmentation, (convolution) neural networks, computer vision and deep learning. Real-life data samples were used to review the applications of the five steps of machine learning while introducing the ideas of data cleaning and augmentation. After that, neural networks were introduced through an explanation of core ideas like perception, layers and weights. Lab sessions on training convolution neural networks and follow-up discussions were arranged to deepen participants’ understanding. Computer vision was also discussed as a typical application of neural networks. Finally, the participants sampled additional machine learning tools, including Microsoft Azure Machine Learning Studio (Classic).

C. Course 3: Developing AI Applications

Course 3 was a capstone project in AI application development that aimed to foster participants’ ethical awareness based on their foundation in machine learning and deep learning. Prior to the course, the participants studied self-directed reading materials about ethical principles and dilemmas related to AI. Lectures were mainly structured as guided discussions on the usage and design of AI in relation to ethical principles. These real-life scenarios helped participants relate to and reflect on ethical principles when designing AI. Three additional examples of AI applications were used as case studies in the lectures to focus on AI-related ethical issues.

Following the introductory lectures, the participants formed groups and prepared their projects with preliminary feedback from the course tutors. Individual consultations were later organised to provide detailed project development instructions. Various AI project development platforms, such as Google Teachable Machine and Microsoft Azure Machine Learning Studio, were introduced in subsequent sessions. Each group presented their work for peer evaluation and discussion in the final session, which provided additional opportunities for the participants to reflect on and deepen their ethical awareness. Ethical awareness was a significant component of the assessment rubric.

III. PEDAGOGY AND DELIVERY

The programme was delivered using a flipped classroom approach, with self-directed learning materials provided before each course. Course 3 adopted a project-based learning (PBL) approach on top of the flipped classroom approach. Fig. 1 demonstrates the course structure.

Fig. 1. Flow chart showing course structure.

In the overall structure of the course, we adopted the flipped classroom learning approach. In this approach, the knowledge transmission and knowledge internalisation processes are shifted [10]. During the knowledge transmission process, students gain material knowledge and concepts, while during the knowledge internalisation process, they internalise this knowledge through active learning. In flipped classroom learning, the knowledge transmission portion of a typical face-to-face lecture is shifted out of class time and is replaced by active learning tasks during the lecture [11, 12]. The flipped classroom learning approach has benefits in terms of increasing student engagement and promoting active learning [13, 14].

We used a flipped classroom learning approach to foster students’ AI literacy. In addition to the commonly accepted benefits of this approach, there were two fundamental factors in our decision to use flipped classroom learning in the delivery of this course. First, it effectively addressed the issue of university students’ limited class time. Compared with K-12 students, class time is significantly shorter in the university context. When university students are introduced to content knowledge before the lesson, class time can be optimised to investigate topics in greater depth, create meaningful learning opportunities and conduct class activities such as using AI tools, hands-on practice, in-depth laboratory sessions, presentations, peer-review, project-based learning and skill development. The flipped classroom learning approach may also facilitate self-regulated learning, which is of great benefit to university students and may, in turn, make a significant contribution to better learning outcomes. When learning via the flipped classroom learning approach, students are required to take control of their own learning progress and take responsibility for their own learning needs, which establishes an environment that brings self-regulated learning to the forefront [15]. Self-regulated learning is important for university students and appropriate to the university context. It also accelerates learning while maintaining long-term retention rates [16], which contributes to better learning outcomes. Studies have shown that integrating self-regulated strategy into the flipped classroom approach enables university students to learn more effectively [17]. The importance for instructors who adopt the flipped classroom approach to prepare university students with self-regulated learning skills has also been discussed [18].

As has been argued by several scholars [19], learning via solving small pieces of concept-rich tasks, or flipped learning, is a better fit for our digital era society. For this reason, we introduced a flipped classroom approach to develop university students’ AI literacy.

In Course 3 of the AI application development, we adopted a PBL approach. Project-based learning is a student-centred pedagogical method that involves a dynamic classroom approach. It is believed that students who experience PBL acquire deeper knowledge through their active exploration of real-world challenges and problems. In this approach, students learn about a subject by working for an extended period of time to investigate and respond to a complex question, challenge, or problem. It is a style of active and inquiry-based learning. PBL contrasts with paper-based learning, rote memorisation, and teacher-led instruction that presents established facts or portrays a smooth path to knowledge because it instead presents students with questions, problems or scenarios [20]. The adoption of PBL in AI literacy is common [e.g. 21, 22], as the PBL learning methodology helps students search for real-world problems, develop complex solutions, and generate synergy among team members. In our study, the PBL was predicted to strengthen participants’ ethical awareness by having them go through the application development process and reflect on the ethical issues involved at each stage. It was also projected to deepen participants’ conceptual understanding through an application of the concepts in real-life scenarios.

IV. METHODOLOGY

The entire programme was delivered uniformly over a period of around 6 months, with around one month between each course to fit with the schedules of our participants due to their other learning commitments. The three courses took 7, 9 and 14 hours respectively, thus amounting to a total of 30 hours. Each of these courses was run over a period of 3 – 4 weeks to provide students with ample time to prepare for the self-directed learning and the project of Course 3 under our flipped classroom learning approach.

One hundred and twenty university students participated in Course 1. We publicised Course 2 among the 120 participants who had completed Course 1, and 82 participants volunteered to attend and completed Course 2. Course 3 was publicised among the participants who had finished Course 2, and 36 participants completed Course 3. For all three of the courses, approximately 75% of the participants were enrolled in bachelor’s degree programmes, including students in their first, second, third, and fourth years of study. The remaining participants were from postgraduate or higher diploma programmes. The participants came from a wide range of academic backgrounds: Mathematics, Information and Communication Technology, Health Education, Chinese Language Studies, Psychology, the Sciences (Natural Science and STEM Education), English Language Studies, General Studies, Music, History, Global and Environmental Studies and Global and Hong Kong Studies.

Three instruments were used to evaluate the participants’ feedback: 1) the Flipped Classroom Learning on AI Literacy survey; 2) students’ responses to focus group interview questions about flipped classroom learning in the AI literacy course and 3) students’ reflective statements on their understanding of AI and ethics.

TABLE I

Structure Of The Survey On Flipped Classroom Learning

The Flipped Classroom Learning on AI Literacy survey assessed the effectiveness of the flipped classroom learning approach in developing AI literacy among the participants. The survey contained four parts assessing the participants’ feedback on flipped classroom learning: learning preferences, the flipped classroom learning environment, self-regulated learning, and the overall flipped learning experience. Table I summarises the structure of this survey with a sample question from each of these 4 parts. The pre-/post-course survey was conducted both before and after the course. As parts 2 – 4 focus on the experience of the participants through the courses, it is more appropriate to collect participants’ feedback on these parts after Course 1, 2 and 3 instead of before Course 1. The survey for participants to complete before Course 1 thus focuses solely on the first part, i.e. the part on their learning preferences. This part contains 7 multiple-choice questions (with five choices per question). In the survey administered after Course 1, 2 and 3, the participants were asked to answer questions for all four parts, a total of 24 multiple-choice questions (with five choices per question).

Under this survey design we carried out two kinds of statistical analyses: 1) comparison of results from Part 1 (learning preferences) of the survey before Course 1, after Course 1, after Course 2 and after Course 3; and 2) comparison of results from the entire survey after Course 1, after Course 2 and after Course 3. When more than one set of survey results was compared, repeated measures ANOVA test was used with the survey results collected after different courses as the within-subjects factor. Findings of these two types of analyses were reported in the section “Results and Discussion”.

In the focus groups, the participants were interviewed about their views on the use of the flipped classroom learning approach after attending each course. They were asked about their preference for flipped classroom learning, the effectiveness of the approach, and suggestions for improvement.

The third instrument was the reflective statement that students wrote on their understanding of AI and ethics. The participants were asked to write 100 to 200 words in either English or Chinese about their understanding of AI and related ethical issues. They were asked to write this reflective essay before and after each course using the Moodle discussion forum.

V. Results and Discussion

This section reports on the results of the flipped classroom surveys, focus group interviews, and reflective statements. Overall speaking the three courses received positive evaluation results, as summarized in Table II, from all participants completing each of the three courses.

TABLE II

EVALUATION RESULTS BY PARTICIPANTS AFTER EACH OF COURSE 1, 2 AND 3

A. Feedback on Course 1

Most of the 120 participants in Course 1 enjoyed the flipped classroom learning approach. According to the results for all four parts of the survey after Course 1, the mean was 4.0 marks out of a maximum score of 5 (SD = 0.45).

Table III compares participants’ responses to the first part of the survey before and after Course 1. Statistically significant increase was observed, indicating participants’ higher preference over the flipped classroom learning approach after Course 1.

In line with the results of the surveys, the participants also demonstrated a preference for this learning approach through focus group discussions and reflective statements (see Table IV for quotes from some of the participants). Several participants mentioned that they enjoyed the flexibility in learning pace and the deeper learning they experienced in the flipped classroom. Some of them also reported that the flipped classroom learning approach facilitated self-directed learning, which is an important skill in the digital era. They also mentioned that, after the pre-lesson reading, the hands-on workshops have helped to deepen their understanding. Some participants even expressed their preference to have more workshops. All these demonstrated the importance of the workshops as an integral part of the learning process of the participants. Some participants, on the other hand, shared that they would prefer more time between lessons to study the self-learning materials better. This highlights the importance of both the pre-class learning and the synchronous classes for an effective implementation of the flipped classroom learning approach. More time has accordingly been allowed between classes in the further offering of our courses.

TABLE III

Results Of Part 1 (Learning Preference) In The Survey On The Flipped Classroom Learning Approach After Course 1

B. Feedback on Course 2

Table V summarises the change in responses to Part 1 of the survey by participants completing both Courses 1 and 2. Consistent with the above findings after Course 1, participants in this case showed a statistically significant increase in their responses to learning preference after Course 1 with the survey result staying at a similar level (around 4 marks out of 5) after Course 2.

Beyond the positive perception demonstrated in the survey results, the participants’ responses in the focus group interviews and reflective statements conducted after Course 2 confirmed the usefulness of the flipped classroom learning approach for their understanding of AI concepts. The participants pointed out that the novelty of topics concerning AI suited enquiry-based learning and that the flipped classroom learning approach was suitable for fostering AI literacy. Some of the participants also appreciated the learning materials provided. Participants also expressed that the hands-on experience during the workshops had supplemented their learning. Some participants, on the other hand, commented on the more difficult contents of Course 2. This indicated the importance of setting the right level of difficulty for the course contents in fostering AI literacy.

TABLE IV

Selected Quotes From Participant Interviews And Reflective Statements After Course 1 On The Usefulness Of The Flipped Classroom Learning Approach

TABLE V

Results Of Part 1 (Learning Preference) In The Survey After Courses 1 And 2

Table VI shows the results of the entire flipped classroom survey. The mean scores remained high throughout Course 2 (the mean scores were around 4 marks out of a maximum of 5). This shows that the participants appreciated learning about AI via the flipped classroom learning approach.

TABLE VI

Results Of The Survey On The Flipped Classroom Learning Approach After Courses 1 And 2

TABLE VII

Selected Quotes From Participant Focus Group Interviews And Reflective Statements After Course 2

C. Feedback on Course 3

Table VIII tabulates the responses to Part 1 (Learning Preference) of the survey from participants who had completed all three courses. In line with the results after Courses 1 and 2, a statistically significant increase in the participants’ learning preference over the flipped classroom learning approach was observed after Course 1. Such level of preference was maintained to the end of Course 3.

TABLE VIII

Results Of Part 1 (Learning Preference) In The Survey After All Courses

Considering responses for the whole survey, Table IX shows that the participants’ preference for the flipped classroom continued throughout the whole programme. The 36 participants who completed the programme showed an appreciation of the flipped classroom learning approach, with high marks throughout the entire programme. The mean scores remained above 4 out of a maximum of 5.

Table X shares some selected quotes from the focus group interviews and participants’ reflective statements after Course 3. Some participants commented on the appropriate workload on the pre-course reading. Some participants also pointed out the flipped classroom learning approach addresses the need of the course well: the theoretical part of the course could be covered in the pre-course learning, while the synchronous workshops with instructors and fellow classmates can be reserved for the practical part of the course. Other participants emphasised the importance on the workshops as well as the collaborative work and discussions during the workshops assisted them in learning. Some participants suggested more assistance be given on carrying out the project.

TABLE IX

Results Of The Survey After All Three Courses

D. Project-based Learning

Course 3 involved application development through a group project, and this project-based learning as a pedagogy was well-received by the participants. They perceived that the project-based learning had been useful in developing their understanding of concepts and ethical awareness (see Table XI).

Several students pointed out that comparatively speaking, the flipped classroom learning approach was more appropriate in Courses 1 and 2 and less appropriate in Course 3, in which more rapid support was needed. Al-Samarraie et al. [23] also point out that when the flipped classroom learning approach is applied in technology disciplines, it brings challenges in the form of a lack of immediate feedback, which may increase the frustration of weaker students.

TABLE X

Selected Quotes From Focus Group Interviews And Reflections After Course 3

VI. CONCLUSION AND FUTURE WORK

This paper presents the curriculum design and delivery of an AI literacy programme for university students in Hong Kong. The programme was 30 hours in duration. One hundred and twenty students attended Course 1, 82 attended Course 2 and 36 attended Course 3. The participants came from diverse academic backgrounds. We collected feedback from the participants through a survey, focus group interviews and reflective writing. The feedback showed that the students appreciated the flipped classroom learning approach. They also reported that project-based learning helped them to develop their understanding of concepts and their ethical awareness concerning AI.

These findings have several implications. First, the study examines and validates the use of flipped classroom learning approach with project-based learning in AI literacy education. It successfully fills the gap in pedagogy and delivery, which is a large step towards promoting AI literacy among citizens of diverse academic backgrounds.

TABLE XI

Selected Quotes From Reflections After Course 3

Second, the study suggests that setting an appropriate level of difficulty concerning course content is of great importance in literacy programmes. In this study, participants felt empowered after attending the programme. An important reason for this was that the programme was designed with an appropriate level of difficulty. To increase the accessibility of literacy courses for the public, we must consider the needs of the participants and their limited prior knowledge, adjusting the difficulty of the tasks to a moderate level. Only in this way will participants feel empowered to unleash their digital creativity.

The final implication is that cultivating students’ self-directed learning is important. Some of the participants in the focus group interview pointed out that applying the flipped classroom learning approach in Course 3 was not as effective as in Courses 1 and 2, as the participants failed to receive immediate assistance from the instructors. Several participants also reported that they were not self-motivated enough for this method, as learning through the traditional approach involves more guidance from instructors and makes their study easier. It is evident that too much reliance on instructors undermines the efficiency of the flipped classroom learning approach. The workplace is of growing importance as a model for learning, leading to an increasing need for self-directed learners. It is therefore vital to cultivate university students’ self-directed learning habits. In future offering of Course 3, we plan to adopt project-based learning as its major pedagogy since the project has served as a crucial part in driving participants’ learning in Course 3.

The significance of this study lies in its validation of a pathway for the development of AI literacy among citizens from diverse academic backgrounds. We encourage researchers and educators in the human-computer interface, AI and learning science communities to both engage in conversation around the curriculum and pedagogy in this study and use them as a guide for future empirical and design research on AI literacy. The programme should also be offered to members of the public, including senior secondary school students and citizens from all walks of life.

References

[1] International Commission on the Futures of Education, “Reimagining our futures together – A new social contract for education,” UNESCO, Paris, France, 2021.

[2] V. Van Roy, F. Rossetti, K. Perset, and L. Galindo-Romero, “AI watch, national strategies on artificial intelligence: A European perspective,” JRC Publications Office of the European Union, Luxembourg, JRC122684, 2021, DOI: 10.2760/069178, JRC122684.

[3] S.C. Kong, W. M-Y. Cheung, and G. Zhang. “Evaluation of an artificial intelligence literacy course for university students with diverse study backgrounds,” Computers and Education: Artificial Intelligence, vol. 2, pp. 1-12, 2021. DOI: 10.1016/j.caeai.2021.100026.

[4] S.C. Kong and G. Zhang. “A conceptual framework for designing artificial intelligence literacy programmes for educated citizens,” in Proc. of the GCCCE 2021. Hong Kong, China, 2021, pp.11-15.

[5] A.A. de Freitas and T.B. Weingart. “I’m going to learn what?!?: teaching artificial intelligence to freshmen in an introductory computer science course,” in Proc. of the 52nd ACM Technical Symposium on Computer Science Education. New York, United States, 2021, pp.198-204. DOI: 10.1145/3408877.3432530.

[6] M. Pouly, T. Koller, and R. Arnold. “A game-centric approach to teaching artificial intelligence,” in Proc. of CSEDU (1), Heraklion, Greece, Jan., 2019, pp.398-404. DOI: 10.5220/0007745203980404.

[7] N. Green. “An AI ethics course highlighting explicit ethical agent,” in Proc. of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, New York, United States, July, 2021, pp. 519-524. DOI: 10.1145/3461702.3462552.

[8] N. Garrett, N. Beard, and C. Fiesler. “More than ‘If time allows’: The role of ethics in AI education,” in Proc. of the AAAI/ACM Conference on AI, Ethics, and Society. New York, United States, Feb., 2020, pp. 272-278. DOI: 10.1145/3375627.3375868.

[9] J. Saltz et al. “Integrating ethics within machine learning courses,” ACM Trans. on Comput. & Educ., vol. 19, no. 4, 2019, pp. 1-26. DOI: 10.1145/3341164.

[10] A. Sams and J. Bergmann. “Flip your students’ learning,” Educational leadership, vol. 70, no.6, pp.16-20, Mar., 2013.

[11] L. Abeysekera and P. Dawson. “Motivation and cognitive load in the flipped classroom: definition, rationale and a call for research,” High. Educ. Res. Dev., vol. 34. No. 1, pp.1-14, Aug., 2015. DOI: 10.1080/07294360.2014.934336.

[12] S. C. Kong, “Developing information literacy and critical thinking skills through domain knowledge learning in digital classrooms: An experience of practicing flipped classroom strategy,” Comput. & Educ., vol. 78, pp.160–173, Sept., 2014. DOI: 10.1016/j.compedu.2014.05.009.

[13] O. Flores, I. Del-Arco, and P. Silva, “The flipped classroom model at the university: Analysis based on professors’ and students’ assessment in the educational field,” Int. J. Educ. Technol. High. Educ., vol. 13, pp.1-12, May, 2016. DOI: 10.1186/s41239-016-0022-1.

[14] N. Foldnes, “The flipped classroom and cooperative learning: Evidence from a randomized experiment,” Act. Learn. High. Educ., vol. 17, no.1, pp. 39-49, Mar., 2016. DOI: 10.1177/1469787415616726.

[15] B. J. Zimmerman, “Self-regulated learning and academic achievement: An overview,” Educ. Psychol., vol. 25, no. 1, pp. 3–17, Jun., 1990. DOI: 10.1207/s15326985ep2501_2.

[16] M. Boekaerts, “Self-regulated learning: Where we are today,” Int. J. Educ. Res., vol. 31, no. 6, pp. 445-457, 1999. DOI: 10.1016/S0883-0355(99)00014-2.

[17] C.-L. Lai and G.-J. Hwang, “A self-regulated flipped classroom approach to improving students’ learning performance in a mathematics course,” Comput. & Educ., vol. 100, pp.126–140, May, 2016. DOI: 10.1016/j.compedu.2016.05.006.

[18] M. Shih, J-C. Liang, and C-C. Tsai, “Exploring the role of university students’ online self-regulated learning in the flipped classroom: a structural equation model,” Interactive Learning Environments, vol. 27, no. 8, pp. 1192-1206, 2019. DOI: 10.1080/10494820.2018.1541909.

[19] V. Dagiene and G. Stupuriene, “Bebras–A sustainable community building model for the concept based learning of informatics and computational thinking,” Inform. Educ., vol. 15, no. 1, pp. 25-44, 2016. DOI: 10.15388/infedu.2016.02.

[20] J. S. Vogler, P. Thompson, D. W. Davis, B. E. Mayfield, P. M. Finley, and D. Yasseri, “The hard work of soft skills: augmenting the project-based learning experience with interdisciplinary teamwork,” Instr. Sci., vol. 46, pp. 457-488, Nov., 2018. DOI: 10.1007/s11251-017-9438-9.

[21] M. Dahmen, L. Quezada, M. Alfaro, G. Fuertes, C. Aballay, M. Vargas, “Teaching Artificial Intelligence using Project Based Learning”, EasyChair preprint, 2020, Art. no. 4789.

[22] M. Vargas et al., “A project based learning approach for teaching artificial intelligence to undergraduate students,” Int. J. Eng. Educ., vol. 36, no. 6, pp. 1773-1782, May, 2020.

[23] H. Al-Samarraie et al. “A flipped classroom model in higher education: A review of the evidence across disciplines,” ETR&D-Educ. Tech. Res. Dev., vol. 68, no. 3, pp. 1017-1051, Oct., 2020. DOI: 10.1007/s11423-019-09718-8.

The work described in this paper was substantially supported by a grant from the Research Grants Council, University Grants Committee of the Hong Kong Special Administrative Region, China (Project No. EdUHK CB302), and received funding support from the Li Ka Shing Foundation.

Authors