Frequently Asked Questions - Online SIRS

What questions are one the survey?

The questions are identical to the ones on the paper survey. We have a non-functional example of the online survey , and we can also provide interested people with a fully functional demonstration. Please contact us at info@brokenemail.rutgers.edu please replace "brokenemail" with ctaar.rutgers.edu to arrange a demonstration.

Will this change the faculty rating? Will students who do not attend class but respond to the survey bias the outcome?

In our studies to date, changes in the average rating are not significant but we are continuing to collect data to determine if the online system affects the ratings. On average, individual ratings varied by ±0.48 points between Fall 2007 paper surveys and Fall 2008 online surveys. For comparison, individual ratings varied by an average of ±0.40 points between Fall 2006 and Fall 2007, both semesters using paper surveys. Comparisons were limited to instructors teaching the same course for more than one semester (i.e., we did not compare ratings for the same instructor teaching different courses, nor did we compare ratings for the same course taught by different instructors).

Will the number of students who respond drop?

For Fall 2008 and Spring 2009, the response rate (the number of students who filled out the survey divided by the enrollment) for the online surveys was 64% overall. This represents a drop for some departments, and an increase for others. We are closely tracking response rates as we implement our online ratings, and slowly expanding the use of online surveys so we can make adjustments. Based on evidence from other universities, we expect to see an initial drop in the overall number of students replying to the surveys followed by a gradual increase as students and instructors become more familiar with the system. More importantly, the evidence suggests that the change in response rate does not significantly affect the average ratings for individual instructors or departments as a whole.
Response rates to individual surveys may either decrease or rise depending on how the instructor communicates the details of the survey with the students, and may be affected by factors such as class size, attendance policies, and mode of instruction (lecture versus lab, etc.).

How do you enforce student responses?

The survey system notifies students by email of the availability of the survey. Students who do not reply to the survey will receive repeated reminders until they respond or until the survey ends. Additional methods of enforcement cannot be implemented until the university community has an opportunity to discuss the implications and practicality.

What can I do to get more students to take the survey?

Faculty and instructors are essential to ensuring that the students respond to the survey. Above all else, you must communicate with your students regarding the importance of the survey to improve your own teaching, as well as the importance to the university as a whole. Consider taking the following actions:

• Include a statement on your syllabus that students are expected to complete the SIRS survey
• Use informal, midcourse surveys throughout the term
• When the survey begins, take some class time to discuss the importance of the survey.
• Give the students personal examples of how you have used prior surveys to improve your teaching
• Inform the students that the surveys are used by the University in promotion, tenure, and reappointment decisions.
• Assure students that their comments and responses will always remain anonymous.
• Invite students to view survey data from previous semesters at http://sirs.rutgers.edu/

Why do students need to log in? Does this violate their anonymity?

Student log-in information is used only to determine which surveys a student can take and to prevent the students from responding more than once to the same survey. The survey software never reveals the students' identities, and all reports generated by CTAAR only include anonymous, aggregate data. See the privacy policy for more information. It is important that you communicate to your students that you will only see anonymous data, and only after final grades have been submitted.

How are the results used?

University-wide, the survey results are used as part of the faculty promotion and tenure review process. While the use of the survey data varies within individual academic units, it is often used as part of a review process for improving the curriculum, implementing changes to teaching strategies, reappointment review for part-time lecturers and teaching assistants. Many faculty and instructors use the survey data, in particular the comments, to assess and improve their own teaching methods.

Who gets to to see the results?

The anonymous summary statistic sheets for faculty and part-time lecturers are available to the entire university community at http://sirs.rutgers.edu/beginning with data from 2004. Older data is available on CD-ROM at the University Libraries. Data for Teaching Assistants is no longer published because of the requirements of the Family Educational Right to Privacy Act (FERPA). Student comments are not published. CTAAR distributes all the reports, including the data for teaching assistants and the student comments, directly to the academic departments and to the indivdiual instructors shortly after the grading period ends. By request, CTAAR also provides the raw, numerical student response data to departments that want to run their own statistical analysis.

Who gets to see the comments?

CTAAR sends the comments directly to the instructors and to the academic departments. The comments are completely anonymous and grouped by question. Comments are not published.