Earth Lab offers three courses that teach students the necessary earth data science skills to begin a career in the booming data science industry—Earth Analytics Bootcamp, Earth Analytics, and Earth Analytics Applications. These courses comprise our Earth Data Analytics professional graduate certificate, in addition to being offered to CU Boulder undergraduate and graduate students. They are designed to be feasible for students of all backgrounds and at all points in life, from working professionals with families to students residing outside of the U.S. in places where traditional data science courses are inaccessible. The Earth Lab Education team create and employ a variety of innovative technologies to make the class more inclusive and accessible including autograding infrastructure that allows the course to reach more students, a cloud-based coding platform that eliminates the need for computers, discussion boards to allow students to help one another when instructors are unavailable, and fully open online textbooks with all course materials. At the midpoint and end of each semester, the team administers a survey to collect student opinions on the use of technology. These surveys have provided valuable insights on student attitudes towards remote learning, open education resources, and other learning technologies. 

In-Person, Synchronous Online, and Asynchronous Online Lectures 

Earth Analytics courses are accessible to professional, non-traditional, and international students because each lecture can be accessed remotely in real time or as a recording. There are three options for viewing each lecture—attend in person at CU Boulder, attend remotely via Zoom, or watch the recorded version.

In fall 2018, ten students chose to attend lectures in person at CU Boulder, while five primarily or only used the other options. The fall 2019 class had a majority of students attending online—15 students usually either attended the lecture online or watched recorded lectures, while only seven regularly attended in person on campus. In Spring 2020, four students attended the lecture synchronously online, four attended in person, and four used Canvas and earthdatascience.org to interact with the course on their own schedule. 

Student feedback from all years indicated that students greatly appreciated the flexibility provided by the recorded or remote lectures, even when they rarely or never used them. 

One student related:  

“I was out of town [for] one of the sessions, so it was really great to have the option to attend online. When my internet connection kept bombing, it was great, when I finally gave up, to be able to...watch the recorded session.” 

This comment illustrates that flexible participation options make the course fit comfortably into the lives of students. When students cannot attend class in person, the remote access and recorded options offer convenience and flexibility. 

Another student noted:

“Participating in class via zoom was very easy and accessible. I greatly enjoyed the ability to communicate and contribute to class, as well as interact with my classmates via zoom breakout sessions.” 

Multiple students commented on the integration of technology in the classroom and indicated that attending online didn’t feel alienating. One student reported: 

“The online capabilities in this course were the best I've seen in my university career—well done!” 

For students that lived in different parts of the nation, attending class in person was not an option. Because there are not many courses in earth analytics offered across the country, many of the professional certificate students lived in different parts of the United States.

 Students that did attend in person commented that they preferred that method due to the opportunity to be able to speak with professors one-on-one before or after class, an increased attention span in classroom environments, and preferring in person interaction to online interaction. 

Those students that expressed preference for attending lectures online appreciated that they did not have to commute, saving carbon emissions and time, and noted that they felt able to ask questions and interact with both their in-person and remote classmates by using Zoom. 

Students that watched the recorded lectures appreciated that they were able to speed up, slow down, skip, and repeat portions of the lecture to suit their needs. They also were grateful for the scheduling flexibility to fit work and family obligations, or simply their mood and brainpower on a particular day. 

The majority of students that had attended class in person found it useful, and nearly every student in the fall 2019 Bootcamp cohort found the lecture recordings to be useful. In the fall 2018 Earth Analytics course, seven out of the ten students that used the recorded lectures found them useful, while in the spring 2020 course, nine of eleven students found the recorded lectures useful. The combination of in person and remote resources expanded the audience for this course and permitted many different types of students to be able to attend and thrive.

Insights And Actions

From the survey results regarding the in person and remote nature of the course, it is clear that students greatly appreciate the lecture flexibility. The Earth Analytics courses will continue to involve several options for lectures, allowing professional or remote students to take the course at their own pace or attend remotely. Starting in the fall of 2020, all instruction was moved to online with course meetings taking place entirely over Zoom due to COVID-19.

Many students mentioned that part of the reason they prefered attending in person was the value of using the instructor as a resource in class. Prior to COVID-19, Earth Analytics had in-person or remote one-on-one meetings with each student. These meetings had a positive overall reception, with all survey respondents indicating that they were “very useful.” 

Online Discussion Forum For Questions

Another online resource that is available for students in the Earth Analytics courses is an online discussion forum for asking and answering questions. This is a place for students to interact. They learn how to ask good coding questions, a critical skill thanks to the ubiquity of StackOverflow and similar sites, and are able to deepen their understanding of the material by answering colleagues' questions. When necessary, instructors answer questions and offer information. Instructors, however, are less available over the weekend when many working students do their homework, making it particularly helpful to have peers answering questions as well. The Earth Analytics courses have used Piazza (a forum for class questions and discussions) or Canvas (a Learning Management System used by CU Boulder and other universities) to host their discussion boards. 

The majority of students found the discussion forum useful for completing homework, asking questions, interacting with the instructor, and (to a lesser extent) answering classmates’ questions. Most students posted questions rather than answers on the discussion board. In the fall 2018 Earth Analytics course, students looked on the class discussion board and on Google with equal frequency if they encountered a problem. The students in the fall 2018 cohort used the discussion boards less frequently—only three students relied on the discussion board over Googling the answer or asking an instructor. All students in the 2019 summer section rated the discussion boards as either somewhat or very useful.  

Though students seemed to find the discussion boards helpful, their feedback revealed some of the board’s limitations. Students noted that it could be difficult to find threads with answers to their specific questions due to the boards’ lack of a good search feature and user interface. Several also mentioned that increased instructor oversight may have been nice, as threads would sometimes go in an incorrect or unhelpful direction. 

From the instructor’s perspective, managing the discussion board can easily become one of the most time consuming parts of teaching the class. Striking a balance between helping students when they’ve reached an impasse and are becoming frustrated and showing them that they can problem-solve through many of the issues on their own can be challenging.

One student summed up the complaints nicely, saying:

 “The Q+A strings sometimes got overly convoluted, especially in some of the homework strings, and it would be really nice to be able to collapse responses and only see top-level comments to be able to search through to see if a question I had was already asked.” 

While there were complaints, many students commented on how helpful the board was to completing their homework or getting a question answered. One student commented:

“I've never seen people use the discussion board for a course before. It was really helpful, I was grateful for the bravery that other students had to help answer questions, and [the instructor’s] quick responses were unexpected and awesome.” 

Insights And Actions

Student feedback indicates that the discussion boards are useful, though there is room for improvement. Nearly all students appreciated interacting with the instructor on the board, and some called for more instructor oversight. Instructors will continue to interact over the discussion forums in the future, but due to the time investment, this is also a case where a Teaching Assistant could be helpful. Regarding the layout and lack of search functionality of Piazza and Canvas, it may make sense to move to a different platform.  Moving to a platform with an advanced search feature and a system for tagging, pinning, and archiving posts could be the best solution.

A Free Online Course Textbook

Promoting open education and open reproducible science are fundamental to our courses and Earth Lab as a whole. The Earth Analytics courses use earthdatascience.org, which houses over 130 free, open source coding lessons, which comprise three earth data science  textbooks. All of the content taught in the course can be found in these textbooks.  

    In every year and course surveyed, student opinion of this resource was very positive. For the most part, students found this resource to be well organized, useful, and easy to navigate. 

A student in the fall 2019 cohort commented that: 

“I was impressed with how organized and easy it was to participate, follow along, and find answers to questions.”

While another student from the fall 2018 course said:

“I think the website is awesome. It provides a lot of information and looks great. No further comments for changing it.”

In the fall 2019 Earth Analytics Bootcamp course, all students rated the website as important for completing their homework and effective in explaining key concepts. Several students in both the fall 2019 Bootcamp and the fall 2018 Earth Analytics course felt neutral or badly about how easy it was to find information on the website, which could be an area for future improvement. 

Out of all the resources available to students, the spring 2020 cohort ranked earthdatascience.org as the most useful. When surveyed, all students responded that this resource was “very useful.” One student commented that the website was an:

 “extremely helpful resource—if I had a question about the material, it was so much simpler to thumb through this resource than try to find the spot in the lecture recording that reviewed the concept.”

This trajectory of opinions on the site between various class cohorts is positive overall, showing that the site is being improved. For the spring 2020 course, we changed the organization of the online material, transitioning from “courses”  to a textbook format. Students have reacted positively to this change, so this new system of organization is likely to persist. 

One student from the spring 2020 cohort commented:

“The textbooks were helpful (and often one of the top search responses when I had to troubleshoot which speaks a lot to the information they provide),” while a second student said, “I could find what I needed on earthdatascience.org.”

Despite the overall positive feedback, the students did have feedback on the organization of the textbooks, suggesting that they may still need some refinement.

The first student said:

“I think the formatting of the textbooks could be more consistent and [some material could be] easier to find,” while the second student noted, “The textbook felt a little scattered to me—not the individual lessons, but being able to look back and find things.  It would be great if each lesson could be a single webpage.  It would also be great if each chapter could be available as a complete PDF.  I would love to have a complete single ‘textbook’ for the course that is searchable, could be highlighted, notes in the margin, etc.  Honestly, while the content was good, the format drove me crazy.”

    This student feedback is valuable as we continue to develop our open source online resources to support the earth analytics professional certificate courses. 

Insights And Actions

    Student feedback on the earthdatascience.org course website was overwhelmingly positive, so this resource will continue to be available for students (and anyone else who would like to use it!) in the future. We are aware that the website can be difficult to  navigate and plan to launch a completely new, reorganized site in Fall 2020.

Overall, the use of technology, both in the form of open education resources and other materials, was received positively and improved the students’ experience in the courses. By offering in-person, remote, and recorded lectures, students of many different backgrounds and in varied places in their education were able to attend the course. Online discussion forums offered a place for all students to interact, giving remote or recorded lecture attendees the chance to ask and answer questions. The earthdatascience.org course website, an open education resource, was one of the most useful and highly rated parts of the Earth Analytics course experience. 

By collecting student feedback, the Earth Lab education team is able to ensure that the useful and inclusive parts of the student experience are maintained, while less well regarded parts of the experience are improved for upcoming years.

Collecting this information to improve and develop online courses has proven invaluable this spring (2021), with the outbreak of COVID-19 and subsequent move to entirely online classes. Because the Earth Analytics professional certificate courses already had a substantial and well-developed online option, transitioning to fully online was seamless and offered previously in-person students continuous support. The continued development of easy-to-access, flexible online courses is vital for the progression of education, particularly in these unpredictable times.