A Case Study on the Administrative Impacts of Different Engineering Capstone Teaching Modalities

Abstract

The capstone experience in engineering curriculums is a critical component focused on the unification of several years of student learning, but its unique nature can present challenges to engineering departments in faculty assignment and ensuring consistent, high-quality instruction. During a seven-year period, five instructional modalities, defining the interaction between students and faculty, were explored in capstone courses in the University of Maine Mechanical Engineering Department. By sharing the lessons learned from this case study, other engineering departments can make more informed decisions about how to operate capstone courses. We present the benefits and challenges of each modality and evaluate them for scalability, student satisfaction, project diversity, cost, and instructor workload. Annual data were collected on student, instructor, and project counts. Data from student evaluations and department budgets were used to evaluate student satisfaction and costs, respectively. Insights from the three authors, who were the primary capstone instructors during this study, are shared as part of the evaluations and lessons learned. Key results are that cost, student satisfaction, and project diversity did not depend strongly on the teaching modality. However, scalability and instructor workload were highly dependent on the teaching modality. The University of Maine Mechanical Engineering Department sees the most promise in a modality with multiple lead instructors who each oversee a portion of the teams, which provides scalability to add or remove instructors, and the ability for high-quality instruction through close coordination of a small instructor group.

1. Introduction

Capstone courses are a central component of engineering programs, serving as the culmination of a student’s academic journey and introducing a range of learning objectives that vary from one institution to the next. Additionally, they often represent a core element of the ABET accreditation process [1], ensuring that students are well-prepared to enter the workforce upon graduation. Capstone courses also provide an important bridge to industry, creating opportunities for students to be hired and facilitating meaningful collaborations between universities and companies [2,3]. As engineering becomes increasingly multidisciplinary, capstone courses are evolving to reflect this trend. However, this evolution is not without its challenges, as different disciplines have varying requirements for assessment and evaluation [4,5,6].

Capstone courses are designed to help engineering students develop a cohesive set of professional and technical skills. For the context of this study, we consider capstone courses to refer to team-project based courses executed near graduation, providing an open-ended problem which can incorporate most, if not all, of the IEA professional competencies of a Professional Engineer [7]. These courses typically involve teamwork, communication, and risk-based analysis, and require students to utilize material learned throughout the curriculum to develop an engineering design [2,8]. Along with these technical skills, capstone courses also provide students with an ethical perspective on the design task, and introduce them to industry-near practices and standards. In many cases, the student design is built and tested, which allows students to close the loop of the design–build–test sequence. The overarching goals of the capstone experience include integrating and synthesizing previous knowledge, preparing students for a successful transition to working life, encouraging lifelong learning, and providing a culminating experience that offers closure [9]. Students are challenged to think beyond straightforward analysis, to consider approaches suitable for the ever-increasing complexity of engineering challenges, and to root their designs with considerations for impacts to global society.

Often, capstone courses are unique within engineering curriculums in this goal set, presenting a rewarding but challenging experience for both students and instructors. Capstone courses display a great variety of instructional practices [10,11]; however, there is little literature on the effect of these practices [10,12] on faculty workload. Commonly, the workload in a course like capstone can be very high due to the frequent individual interactions needed with the teams, and increased engineering competencies, many of which are often primarily addressed in capstone courses [13]. While student satisfaction and learning outcomes are essential considerations for curriculum design, programs should consider faculty workload as well. Many engineering programs in both developed and developing countries are constrained regarding instructor resources [14]. Understaffed programs may be better served by a different capstone model than those with greater resources. Research has shown faculty workload to be a key factor in job satisfaction and retention [15].

The number and nature of the instructor–student interactions vary, but always rely on periodic meetings to supervise and mentor student team progress. During these meetings, feedback and advice are typically given on the technical aspects of their project, project management, and teamwork dynamics. It is critical that the instructor be able to support all elements of the interaction and be a good communicator, and often moderator, in the team process. As such, not only is technical expertise in the team-specific project topic needed, but exposure to industry and professional project management and teamwork processes is also highly beneficial.

Capstone courses can be taught in a variety of instructional models, each with its own benefits and drawbacks [10,12]. It is important for departments to consider multiple approaches and determine which would be most effective for their particular needs. The University of Maine (UMaine) Mechanical Engineering Department implemented five distinct instructional modes, defining how student teams interact with faculty, over the course of seven years. These modes were evaluated on metrics for success including scalability, project diversity, student satisfaction, cost, consistency of the assessment process, and faculty workload. By comparing the performance of each mode across these metrics, departments can make informed decisions about which model to implement. It is important to note that there is no one-size-fits-all approach, and what works for one department may not work for another.

2. Mechanical Engineering Capstone at the University of Maine

At UMaine, the Mechanical Engineering Capstone sequence consists of two semester-long courses, counting for a total of seven course credit hours, with credit hours defined per the US Code of Federal Regulations as approximately one hour of instructor contact and two hours of out-of-class work for fifteen weeks [16]. Teams of students with varying project assignments follow a structured process divided into four main phases: Conceptual Design, Detailed Design, Manufacturing, and Evaluation [17,18]. During the fall semester, each team begins with an open-ended problem statement and follows a conceptual design process to generate at least three alternative designs before downselecting. The teams then perform simulations and analysis at the system and component levels in order to produce a set of hardware drawings, materials lists, and performance predictions. Materials are provided to teams at the start of the spring semester to fabricate their designed device, which they then test and evaluate against performance predictions. The teams meet with instructors weekly for project-specific discussions in addition to twice-weekly lectures which serve to guide the structure of the design process and explain the deliverables. During the capstone sequence, students are trained by the instructors in many common tools and skills for engineering design—for example, Gantt charts, failure mode analysis, and guidelines for presenting. Students rely on knowledge from prior courses to complete the specific engineering science-based analysis necessary for their project. Each student completes individual deliverables to explore progress and challenges in a team setting. Team assignments comprise the majority of delivered work and report project progress, including all ABET-specific elements of the culminating design experience, and include reports for each phase, presentations, CAD packages, and continual team progress reporting with peer assessment. While most project ideas are generated by the instructor team and supported by departmental funding, some projects are externally funded, which, in addition to lowering the cost burden for the department, improve project diversity and often provide exceptionally relevant experience for students [2].

2.1. Evolution of Instructional Models at UMaine

Prior to the 2016–2017 academic year, UMaine Mechanical Engineering operated with a single instructor managing the entire capstone sequence, including all grading and the majority of team advising, save only teams with external clients. With continual enrollment growth over several years, deliverable numbers had been reduced to manage instructor workload. During the 2016–2017 year, the department adopted a new performance-indicator (PI)-based ABET assessment and evaluation system, which resulted in many PIs being assessed in the capstone sequence, requiring a revised number of deliverables. The structure of the course management process is covered in prior publications [17,18]. This significantly increased the workload despite a temporary reduction in capstone enrollment. New instruction modalities were considered, and the department settled on a configuration with two instructors jointly overseeing all teams, with the goal of reducing the individual workload, increasing project diversity, and ensuring consistent advising. During the following years, a variety of factors, including faculty sabbaticals and changing enrollment, caused the department to explore additional modalities of instructor assignment and student interaction. While these modalities were developed somewhat ad hoc, and they do not exhaustively cover all options, this case study presents advantages and disadvantages of five different modalities implemented over a seven-year period at UMaine. The modalities evaluated are summarized in

Table 1. Summary of capstone instructor assignment modes in the UMaine Mechanical Engineering Department.

Academic Year	Modality	Lead Instructors	Additional Faculty	Students	Teams
2016–2017 (and prior)	Single instructor (SI)	1	0	94	24
2017–2018	Multi-instructor with joint oversight of all teams (MI-J)	2	0	74	16
2018–2019	Multi-instructor with joint oversight and non-evaluating volunteer faculty (MI-VF)	2	2	86	18
2019–2020	** Multi-instructor with joint oversight and non-evaluating volunteer faculty (MI-VF)	2	3	81	14
2020–2021	Single instructor with evaluating support faculty (SI-SF)	1	4	73	14
2021–2022	Multi-instructor with split oversight (MI-S)	3	0	112	22
2022–2023	Multi-instructor with split oversight (MI-S)	3	0	83	17

** Only single lead instructor for spring semester.

, with further details on the motivations and implementation given below. The progression of instructor count is provided in Figure 1.

2.1.1. 2016–2017 and Prior: Single Instructor (SI) Model

The UMaine Mechanical Engineering capstone experience was overseen by a single faculty member prior to 2016. Over the preceding decade, the Department had seen strong student growth, from about 50 students to 94. Because of the limitations of a single instructor’s area of expertise and growing workload, project variety had been gradually reduced. In 2016–2017, 15 teams, comprising 60% of students, conducted the same project, autonomous land drones (Figure 2). This project was operated as a standardized internal competition of an autonomous all-terrain land vehicle. This project provides a strong framework for a culminating design experience, but the standardization left very few options for students to explore mechanical engineering topics missing in the land drone project. One advantage is the standardization of components, with all teams utilizing a gasoline engine as a power source with identical stocks of hydraulic pumps or electrical components used as drivetrain hardware by most teams.

Even with low project variety, the large number of students managed by a single instructor led to fewer deliverables and streamlined assessment process. Written reports, requiring significant time and technical knowledge to assess, were reduced or “farmed out” to supporting technical writing courses. Instead, simple forms or CAD drawings served as evaluation based on a straightforward rubric. The difficulty of scheduling progress meetings led to as-needed meetings which varied greatly in frequency between the teams. Few periodic assessments were included. This led to difficulty in evaluating the teams fairly, and little ability to evaluate individual students.

The lack of deliverables hindered the assessment of ABET student outcomes in this important curricular element. A total of 16 of the 34 PI were assigned to be assessed in the capstone sequence, requiring a complete reorganization of the deliverable structure of the course sequence, including the development of assessment procedures to address individual student performance in a team environment.

2.1.2. 2017–2018: Multi-Instructor with Joint Oversight of Teams (MI-J)

This mode was motivated by the need for additional instructor efforts to implement and grade the increased set of deliverables, and to allow for a wider variety of projects by including faculty with disparate topical knowledge. The instructors chose to jointly supervise all of the teams to allow each team to receive multiple perspectives and to ensure grading was carried out fairly. Project diversity was increased by presenting options from within the expertise of the two lead instructors and other department faculty members. Diversity was further increased by seeing external “clients” to create a more industrially relevant environment. Among other externally provided projects, a set of externally funded Uncrewed Aerial Vehicles (UAVs), ranging from remote sensing platforms (Figure 3) to lighter-than-air vehicles, were included to meet an increasing interest in aerospace engineering.

Student response to the change in modes was positive, with noted improvements in the effectiveness of assessment due to the increased number of deliverables. Project diversity increased and the students appreciated the exposure to multiple viewpoints. These multiple viewpoints also led to constructive, collaborative improvement of supporting materials and processes by the instructor team. This mode still presented challenges related to workload. Both instructors attended weekly meetings with all teams and read all reports. Thus, the contact hours and grading workload were similar to those of the single-instructor model. However, this modality provides excellent flexibility if an instructor is unavailable, as both are familiar with all teams and materials.

2.1.3. 2018–2019, 2019–2020: Multi-Instructor with Joint Oversight and Non-Evaluating Volunteer Faculty (MI-VF)

A concept to reduce the workload on the primary instructors was to enlist volunteer faculty to support the teams by providing technical know-how to select teams with projects related to that faculty’s areas of expertise. These volunteer faculty members did not evaluate deliverables or assign grades, and all lead instructors still attended all team meetings. The motivation was to reduce the primary instructor workload by delegating a portion of the technical advising, while retaining grading consistency by keeping all assessment with the lead instructors. The joint supervision of all teams by both lead instructors was retained due to the benefits of multiple viewpoints and evaluation consistency. This model was used in 2018–2019 (Figure 4), and was continued during 2019–2020.

The addition of faculty members with a more diverse knowledge base helped to increase the variety of projects, and, anecdotally, led to better project outcomes, but there was some confusion over the separation of technical advising and assessment. Some teams felt they had received conflicting information in cases where the collaborating faculty members were not sufficiently familiar with the capstone structure and assessment process. The workload for instructors remained high. Although there was a reduction in technical advising workload, the burden of training and coordinating with the collaborating faculty was similar or greater, especially in the case when volunteer faculty members understandably viewed capstone advising as a low priority.

A sabbatical of one instructor in spring 2020 led to that semester being continued with the same volunteer faculty support model but with a single lead instructor who would grade all deliverables and solely manage the administrative burden of the course. By eliminating other teaching obligations and relying on the now better-trained support faculty and a laboratory manager, the instructor workload initially appeared manageable. However, the spring 2020 semester was interrupted by the COVID-19 pandemic, which left teams unable to finish their hands-on build and test phases of the capstone sequence, with abbreviated deliverables substituted. Therefore, a fair evaluation of student satisfaction and instructor workload for this semester is not possible.

2.1.4. 2020–2021 Single Instructor with Evaluating Support Faculty (SI-SF)

The other usual capstone lead instructor was unavailable to teach for 2020–2021; therefore, a model was adopted to allow for a single lead instructor to manage the entire capstone cohort without reducing the project diversity or deliverable schedule. Support faculty members would again be leveraged. However, unlike prior years, support faculty members would be responsible for grading team deliverables for the teams they managed. The lead instructor conducted all lectures, assigned deliverables with clear rubrics, and graded individual assignments. The lead instructor managed nine teams, while the other five were managed by four support faculty. The lead instructor did not attend weekly meetings between the teams and the support faculty members.

With this model, the support faculty members were given a greater and more clearly defined responsibility, and the effort requested was prioritized by the Department by considering of capstone teams supported in each faculty’s overall teaching burden and overload payments.

The workload on the lead faculty of grading and managing teams was effectively reduced by this model, but the organization burden of training faculty and ensuring consistent and fair workloads and assessments between the teams resulted in a similar level of effort. In total, this model was found to consume a higher amount of department resources than other models due to the high amount of training and coordination effort. Instructor workload is high, as many faculty members are involved and efforts to keep the large team of faculty coordinated are a significant burden on the lead instructor(s). The students appreciated the technical knowledge of faculty advisors but expressed frustration with seemingly conflicting information received in lectures and from their support faculty advisor.

2.1.5. 2021–2022,2022–2023: Multi-Instructor with Split Oversight (MI-S)

For the 2021–2022 academic year, the Department sought to return to a multiple-lead-instructor model as it appeared to be a better balance of student satisfaction and faculty workload. However, with a cohort of 110 students, it would be infeasible for two instructors to co-supervise all teams while keeping team sizes relatively small. The model adopted was to use three lead instructors, each with a separate course section. A single combined lecture would be used to ensure consistency in schedule, deliverable expectations, and organizational guidance. All deliverables would use identical instructions and rubrics, with the lead instructor team closely coordinating on course administration and grading to ensure consistency. Each team would be supervised by one of the lead instructors, who would grade team deliverables. This modality continued in 2022–2023.

Each instructor was responsible for a weekly meeting and guidance for 7–8 of the 22 teams in 2021–2022, and 5–6 of the 17 teams in 2022–2023, typically those projects closest to the instructor’s areas of interest. The total effort, including lectures, grading, and team advising, was found to be equivalent to a typical lecture course. With three instructors covering a wide range of topics, very little technical advising support from additional faculty was needed, but a small amount of support from those interested faculty members or outside “clients” was still present. While teams only interacted with a single faculty member regarding technical project progress, the close coordination enabled by all team supervisors being lead instructors helped ensure equitable advising and consistent grading, including the instructor team comparing findings and making grade adjustments after grading major deliverables.

3. Discussion

3.1. Scalability

Highly scalable models would be those which can easily adapt to increasing or decreasing student enrollment, either gradual trends or sudden variations, without significant modifications to the modality or workload burden. Models that involve the responsibility of the instructor(s) to oversee all students equally (SI, MI-J) work well for student cohorts below a specific threshold, up to approximately 70 for the Mechanical Engineering capstone structure at UMaine. Above that number, the burden of meeting with all teams and evaluating deliverables becomes infeasible. Some flexibility may be possible by reducing other teaching duties of the lead instructors, but, in any case, these models will have a relatively hard upper limit to feasible cohort size. Models with support faculty (MI-VF, and SI-SF) can be scaled relatively well, as they allow for a more granular approach to increasing or decreasing the technical advising faculty effort. However, these models require a relatively long period of time to effectively scale, making them poorly suited for rapidly increasing enrollment. When several new support faculty members are included, it was found that the training and monitoring burden on the lead instructor(s) was high, providing an upper limit on the number of support faculty members which can be included. It is possible that, with several years of execution, enough faculty members would become familiar with the capstone structure and the burden on the lead faculty could be reduced. However, in three consecutive years of including support faculty members, this stage was not achieved at UMaine. It is expected that attempting to extend one of these methods to a year with a greatly increased student cohort would lead to an immense burden on the lead instructor(s), though slow, gradual growth may be somewhat feasible with these modalities. The MI-S model is well suited to scaling, provided that enough faculty members who can tightly coordinate are willing to carry out these duties. Though scaling is not as granular as MI-VF or SI-SF modalities, when an additional lead instructor is included in the team, they can effectively be trained and manage a significant workload during the first year. While there will be a burden on other lead instructors during this initial year, this training burden can essentially be eliminated by the second year, as was seen in 2022–2023 at UMaine.

3.2. Project Diversity

It is common for mechanical engineering students to develop specific interests in which to specialize. It is natural that increased student satisfaction and better learning outcomes will be achieved through a diverse set of projects [19,20]. Increasing project diversity requires appropriate technical expertise from team supervisors and can increase instructor workload compared to standardized projects. It is also possible that a diverse project set can also lead to inequality in student workload or grading.

Figure 5 shows the number of unique projects per capstone student for the seven years of the case study. In many cases, the same problem statement is given to multiple teams, with this being considered one unique project despite multiple teams. Larger values of unique projects per student indicate greater project diversity. Project diversity was extremely low in the initial year of the study. As noted above, this was intentional by the instructor. In subsequent years, the inclusion of multiple lead instructors increased the project diversity. The highest value of 0.151 unique projects per student was achieved in the first year of including volunteer support faculty. The inclusion of support faculty provides a more diverse technical expertise in team advising, allowing for a wider variety of offered projects. Often, the lead instructors of capstone have industry and teamwork experience and the support faculty members participate primarily due to their specific technical knowledge. In these models (MI-VF, SI-SF), the lead instructors will support the teams in teamwork aspects while delegating technical advising.

In the years following 2018–2019, the reduction in project diversity was mainly due to student choice. In each of these years, unselected projects were presented to students, while certain projects received enough interest to assign multiple teams the same project. The move to the MI-S model with three lead instructors was found to retain a sufficiently broad set of technical expertise to continue with similar levels of project diversity as with support faculty. The increase in diversity during 2022–2023 is attributed to the experienced instructor team developing an especially diverse set of project options. Other than the SI model, no other model severely limits the diversity of student projects.

3.3. Cost

The UMaine Mechanical Engineering Department provides the majority of the funding for capstone projects. Some projects, including the purchase of hardware, are supported by external clients. The projects are typically completed for approximately USD 1000, and problem statements are designed with this approximate figure in mind. However, there are several factors related to the teaching modality which can influence the total cost to the department.

Figure 6 shows the cost per project invested by the Department for each academic year. Data from the 2020–2021 year during the COVID-19 pandemic should not be used to draw conclusions as the build process that year was reduced in scope to deal with public health restrictions. The highest values of cost per team were achieved in years with volunteer faculty, a natural expectation when the instructor team expands and guidance is not as tightly coordinated. With fewer faculty coordinating more closely (MI-J, MI-S models), costs can be more tightly controlled, both through the definition of project scopes, and through careful hardware usage such as common components and standard parts, such as fasteners. The benefits of standardizing project options are evident in the 2016–2017 year (SI model), which achieved the second lowest cost per team behind the COVID-19-restricted year.

It should be noted that the costs within this case study, and likely in other executions of the changing capstone teaching modality, are greatly influenced by factors other than the teaching modality. For example, the 2021–2022 year saw a dramatic increase in capstone students to 112. With far more teams than in prior years, the instructor team would normally have offered projects with smaller, cheaper builds to contain costs. However, rollover money from the 2020–2021 year made that unnecessary. Other factors, such as external client funding, specific student project selection, and available department funding may have as much influence on the capstone costs to a department as the teaching modality.

3.4. Student Satisfaction

Capstone is a unique experience for most undergraduate engineering students, and can be frustrating, especially for students who are unexpectedly confronted by the need for modern engineering competencies such as managing engineering activities, communication, and lifelong learning. Key drivers to student satisfaction are interest level, driven largely by project topic, perceived grading fairness, manageable workload, consistency of information, and project success [2,9,20,21,22]. In particular, most capstone experiences do not require a successful project to achieve a high grade and student learning outcomes, but students can be frustrated if the projects are seen as complete failures. The choice of teaching modality can have a major impact on many of the above aspects. Student satisfaction in this case study is evaluated through university-administered student evaluations and through instructor observations. At UMaine, prior to 2019–2020, students completed a 29-question paper questionnaire to evaluate each course and instructor. Starting in 2019–2020, UMaine adopted a 19-question online questionnaire. Many of the questions were identical or near-identical. The results presented in this study focus on three questions which were all identical in the two questionaries. Several caveats must be provided regarding student evaluations. The first is that the data are limited to students who responded. It is possible that, in low-response-rate cases, only data from students with a strong positive or negative experience are recorded. The literature on the topic has noted either minimal impact of selection bias [23], or a slight overprediction of true rankings on average [24], although the impact of selection bias on a single course is very difficult to predict. Response rates ranged from 33% in 2021–2022 to 73% in 2017–2018. The second is that student evaluation ratings of courses and instructors are known to be influenced by grades given and many other factors separate from the teaching modality [25,26,27]. Hence, instructor observations are included in the following discussion. Finally, the ratings during the pandemic years of 2019–2020 and 2020–2021 may be significantly impacted by the changes that occurred to comply with public health restrictions, including meeting virtually, and reducing hands-on work.

Student evaluation data providing answers to three key questions are given in Figure 7, with the questions being provided in

Table 2. Questions from student evaluations of teaching presented in Figure 7.

Q1	What is your overall rating of this course?
Q2	Overall, how would you rate the instructor?
Q3	How fair were the grading procedures?

.

Generally, no strong correlations between teaching modality and student evaluations of course or instructor are evident. Both ratings have generally increased over time, which is attributed to the development of an improved course structure and revision of deliverables by a relatively consistent team of instructors, more than any other factor. In one example, the MI-VF model received the second lowest and the second highest ratings of instructors, even though the same instructors and model were being evaluated. The 2017–2018 year also saw the lowest overall course rating but an above-average instructor rating, showing that the course and instructor ratings are not strongly correlated.

A significant concern in modalities with multiple instructors is grading consistency. Especially for complex assignments, consistent grading across multiple evaluators, even with a detailed rubric, is a noted concern by students and instructors alike [28,29]. This is investigated by the student evaluation question: “How fair were the assessment procedures?”, with responses shown in Figure 7. Responses to this question have little variation between years, with the exception of 16–17, when the reduced number of relevant deliverables led students to believe that grades were being assigned relatively arbitrarily. Years with multiple lead instructors do not lead to a perceived unfairness in grading. In fact, the average response during the years 2021–2022 and 2022–2023, where each instructor only graded content from the portion of teams assigned to them, was higher than during the years 2017–2018 and 2018–2019, when both instructors co-graded content from all student teams. The most positive responses were in 2019–2020, when a single instructor was responsible for all grading, and 2020–2021, when four supporting faculty members were responsible for grading groups they advised, with close coordination from a single lead instructor. It is possible that this is evidence that strong guidance from a single lead instructor may lead to fairer grading. However, the small separation from year to year indicates that fair grading can be achieved with many modalities, provided the instructor team coordinates well.

The SI model had limited project diversity and few deliverables. Students were mostly focused on designing and building their projects, which led to poor student learning outcomes, but a moderate course rating when students could spend most of their time “tinkering”. However, the lack of deliverables left the students feeling as though grades were determined arbitrarily with little chance for feedback, a likely reason for low instructor rating. The students did note receiving consistent information in this model.

The ratings of models with volunteer faculty (MI-VF and SI-SF) had similar student feedback. Project diversity was high and technical advice was strong, leading to good project outcomes, but the consistency of information received and communication were issues, which is reflected in strong course ratings but relatively low instructor ratings, as the rating was that of the lead instructor, who was not necessarily the faculty advisor of the team. Although the lead instructor(s) extensively trained supporting faculty members on the expected level of engagement, the actual level varied between teams and was noticed by the students. This is not due to neglect, but because of different backgrounds and level of experience in mentoring teams. Inconsistency in technical expectations and stakeholder input is a reality of the industrial world and could be integrated into the learning experience. However, student frustration primarily centered on varying availability of team advisors and differing instructions for graded course deliverables. Models with support faculty require intensive training and coordination by the lead instructor to achieve high student satisfaction.

In the MI-J and MI-S models, the lower number of instructors but high attention received by each team led to strong student satisfaction. Students appreciated high project diversity and relatively consistent information. The low course rating in 2017–2018 can be attributed to the first time introduction of a large number of deliverables, which were refined in later years. With fewer faculty members included, messaging could be well coordinated. For example, more students in these models were able to understand that a project device with poor performance can still be a successful learning experience. More coordination is required in the MI-S model to ensure consistent grading, as not all deliverables are seen by all instructors. Coordination is also necessary to ensure equitable workload by all students, as not all instructors meet with each team regularly. Hence, the MI-S model does present some minor challenges to student satisfaction.

The MI-S model received strong ratings of both the course and instructors in 2021–2022. The following year, the highest ratings of both across the case study were achieved. This can be attributed not just to the model, but to the continued development of structure, project scope, deliverables, and grading rubrics in capstone over the prior seven years. While teaching modality can impact student satisfaction, continued refinements and knowledge transfer is extremely important. The authors argue that a strong focus on the quality of the capstone experience (evidenced by the compiling of data and completion of this study) is a major factor in the gradual improvement in student evaluations during this case study.

3.5. Difficulties with Student Evaluations

Obtaining actionable student evaluation data for capstone can be challenging for some instruction modalities. The unique nature of the course compared to most in the undergraduate curriculum, and the teamwork aspects mean that the scope of some evaluation questions may be unclear to students. The instructors may be acting in several roles (lecturer, teamwork mentor, technical advisor) and students may each be providing their ratings based on different interpretations of a question.

The prior UMaine student evaluation method, used until 2019–2020, asked students to evaluate multiple instructors with a single response. The new method allows for individual ratings, but leaves some ambiguity over what is necessary for students to complete. With supporting faculty members included, evaluating the quality of advising for each team is extremely difficult, even with custom developed questions. Sample sizes may only include a few students and without prior, similar experiences to compare to, student feedback is often not actionable. Reliably evaluating support faculty may require lead instructors attending additional meetings, further increasing workloads. Alternatives to traditional student evaluations are recommended to evaluate instructors for most modalities in this case study.

3.6. Faculty Workload

Exploration of teaching modes in this case study has been driven primarily by a balance between project diversity, student learning and assessment, and faculty workload. The course workload must account for both a suitable level for each individual faculty involved, and for the total effort put in by all faculty in a department.

We consider the number of students in project teams managed by an instructor to be a suitable proxy for faculty workload, as shown in Figure 8. Efforts that do not scale with number of students, including lecturing (1–2 h per week) and managing the course webpage, are relatively minor. The typical structure throughout the case study period is for an instructor to meet with each team of approximately five students weekly for 30 min. Approximately, an additional 3–6 h per semester are required for each group for grading deliverables, including reports and presentations. Despite many of the benefits described above, the MI-J and MI-VF modes of instruction do not significantly reduce instructor workload because each lead instructor is still responsible for many contact hours, attending weekly meetings with all teams, and for reviewing major deliverables from each team. The SI-SF model in 2020–2021 was introduced as a means of reducing lead instructor workload by delegating responsibility for weekly meetings and deliverable grading to support faculty. While these workload aspects were significantly reduced, this modality created a burden on the lead instructor to train and coordinate support faculty, especially considering the complex assessment requirements for ABET, which was equally time-consuming compared to advising teams directly. While it is likely that the training effort would decrease in subsequent years, it is also likely that the support faculty members will rotate in and out, minimizing the training reduction. During the three years with support faculty in some capacity, the overall burden on the lead instructor workload did not noticeably decrease. This modality was also quite burdensome on the Department. While only a single lead instructor was assigned to the course, the Department considered the effort of support faculty in course assignment, and paid an overload fee for the participating support faculty members.

The MI-S modality effectively reduced the workload for lead instructors without requiring significant investment across many faculty members. The coordination burden, including providing lectures and organization to the student cohort, was split between lead instructors, which was not the case with the support faculty members. This modality was considered the most manageable for instructors of those included in this case study.

4. Implications

The seven-year case study of the Mechanical Engineering capstone at UMaine can provide several insights into possible modes of instructor assignment and team interaction. Departments may consider new modalities due to changes in enrollment, faculty composition, or for improved student satisfaction. Departments should weigh factors such as scalability, student learning and satisfaction, cost, and faculty workload. This case study can provide lessons learned on the following aspects of a potential capstone teaching modality.

4.1. Lead Instructor Joint or Split Supervision

A benefit of including multiple instructors in any course is the exposure of students to multiple viewpoints. Capstone is an excellent choice for this, as students are developing their abilities to function in an engineering team. Models with joint supervision effectively provide this benefit of multiple viewpoints, and were found to be fairly high in student satisfaction with very consistent grading. However, joint supervision models are poorly suited for scaling and generally result in high workloads on instructors. Split supervision models have a major advantage in the scaling and workload of faculty members, especially in departments expecting significant year-to-year changes in cohort size, and, despite some challenges, are likely better models for resource limited departments. Steps to further manage resources include larger teams and grouping teams with similar projects for combined meetings. Close coordination between instructors can alleviate most potential issues, including grading fairness and messaging consistency, and at UMaine, it has been found that the effort for this type of coordination is much lower than the effort of jointly supervising all teams.

4.2. Supporting Faculty

Supporting faculty members should be primarily considered a means to increase project diversity and the quality of technical advising. In the UMaine Mechanical Engineering Department, the initial model using supporting faculty members placed them in the role of technical mentoring of one or two teams with needs matching their areas of interest. All lecturing and course organization was executed by the lead instructor(s) and the lead instructor(s) also attended team meetings. The subsequent model including assigning some teams directly to a support faculty member who would conduct meetings and grade the deliverables. A higher commitment was expected of the support faculty members and the workload of the lead instructor would be reduced. This effect was not seen, as the workload of training and coordinating the faculty members negated the reduction in team advising. In both models with supporting faculty members, the lead instructor’s workload was high, though a driver for that workload may be the high number of ABET PIs evaluated in this particular capstone sequence. There were challenges in both supporting faculty models to ensure equitable advising, student workload, and, in the case of the SI-SF model, grading. While the supporting faculty members did effectively increase project diversity and technical advising quality, they require a very high effort by the lead instructor to ensure the necessary consistency for student satisfaction. The SI-SF model is quite scalable. Initially, the SI-SF model was expected to have high levels of flexibility for the faculty members to move in and out of the roles. After one year of implementation, it was seen that the coordination strategy and supporting faculty training are both quite difficult to achieve, making it quite important to ensure consistency as regards the lead instructor and support faculty members for this model.

4.3. Faculty Teaming

The capstone experience at UMaine Mechanical Engineering, like that at many universities, is unlike the majority of courses that make up the curriculum. Therefore, students are especially unsure about the course structure, the expectations, and what they are to do. When multiple instructors are involved, inconsistency in either grading or instructions can be frustrating. While a single instructor completing all advising and grading is the most consistent, this method (SI) is the least scalable of all methods in this case study. Multiple faculty members increase scalability, project diversity, and flexibility, but with more faculty members, consistency is more difficult to ensure. A major factor in UMaine Mechanical Engineering moving forward with a MI-S model is the ease of coordinating between a limited set of 2–3 faculty members. The results from student evaluations show a trend of improvement that can be attributed to the consistency of the lead instructors over this seven-year period, with only three individuals serving as the lead instructors. A multi-instructor team should be composed of faculty members who have strong collaborative drive and consistency of this team will lead to better outcomes with any modality chosen. This team should also resolve to minimize inconsistencies in assessment and communication by incorporating detailed rubrics, grade normalization, joint review of selected assignments, cross-over observation, and joint communication through a combined lecture and common course website.

5. Conclusions and Future Modalities

An evaluation of five instructional modalities for capstone courses in the UMaine Mechanical Engineering Department over a period of seven years included the exploration of methods to distribute instructor workload across combinations of lead and support faculty members, and several ways for instructors to mentor student teams. Key findings were that the administrative burden on the lead faculty members when coordinating multiple support faculty members can be significant, and that modes reliant on single faculty interacting with all student teams face major challenges to scalability. Student satisfaction is not directly dependent on the instructional mode, but rather on related factors, including the level of coordination between instructors, which may be facilitated by the instructional mode. To best cope with an expectation of continual enrollment growth, the Department plans to continue with the MI-S model. As this model is refined with clearly defined team expectations and improvements to the guidance structure, it is expected that there will be an increased involvement of volunteer mentors for technical advising only. It will be evaluated whether a more well-defined overall set of course expectations and guidance documents presented to student teams can alleviate some of the challenges of including support faculty members while ensuring consistency of instruction across all teams.