Adding Additional Questions to SIRS (evaluationkit.com pilot)
CTAAR is testing a new survey tool hosted at evaluationkit.com, with courses in SSW, SMLR, SC&I, and all courses taught in Canvas. The instructor of each course section designated to be surveyed by their department will have the opportunity to add questions approximately one week before their survey starts (see SIRS procedures and deadlines for more information). Questions must be added before the survey starts.
Adding questions to your survey is a two-step process. You must complete both steps to add additional questions to your survey:
- Create a new “Custom Question Survey” with your questions (this only needs to be done once, the survey can be reused in the future)
- Attach the new survey to your course (this must be repeated each semester).
To add additional questions to your SIRS:
Step One - create a new survey
This must be completed at least once, but you can reuse your existing surveys in the future. If you have already completed this step in the past, you can skip to step two.
- All questions must be before the survey start date. Questions cannot be changed after the survey begins.
- Go to the survey dashboard in evaluationKit - https://sirs.ctaar.rutgers.edu/pilot (link will open in a new tab or window) - and if necessary log in with your Rutgers NetID.
- Click in the “Custom Question Surveys” in the blue banner at the top of the page.
- Click “Create New Custom Question Survey” or if you want to edit existing question sets, you can click the pencil icon next to your previously saved surveys.
- If this is a new question survey, type a name for your own reference. This will allow you to reuse this set of new questions in the future. Click “Save New”
- Choose the question type and click “add”. We recommend “Single Selection” or “Open Ended” for consistency with SIRS. You will also see an “Item Bank Question” choice that allows you to choose from a set of common questions (preview the list of questions here).
- Type the text of your questions, and if need the response options that the students will see, in order. For consintency with SIRS, we recommend using “Strongly Disagree, Disagree, Neutral, Agree, Strongly Agree” and adding a “N/A” option, as shown in this image:
For online surveys, the “N/A” option is the functional equivalent of a pencil eraser and allows students to remove errant choices.
- Click “Save Item”, and repeat the process to add additional questions.
- When finished adding questions, click the evaluationKit “Home” link.
Step Two - Attach a survey to your courses
This must be completed each semester, for each course.
- In evaluationkit (starting on the “Home” screen), click the name of the semester in the “Custom Questions” box.
- Click “Add Custom Question Survey”.
- Click on the check mark to the far right of the set of questions that you want to use (you can also preview the questions if needed). Note that the checkmarks are static images - it's normal for all of them to remain ”checked“. The row you click will turn a subtle, pale yellow.
- Click “Select Courses”
- Click “Add Courses” (this is the second button)
- Place a checkmark next to all the courses to which you want to add the extra questions
- Click “Add Selected Courses”
Designing your own questions
The Student Instructional Ratings Survey is intended to assist faculty in collecting information from students about their teaching and their courses, and the statistical summary of responses are shared with the student population at the SIRS Results web site. Additional questions should be consistent with that purpose.
Remember, if you add many extra questions, students may not complete them, i.e. “survey fatigue”. You can currently add a maximum of 10 additional questions, however we recommend no more than 5.
When designing your own questions for surveys, you should decide what is the main issue to be addressed by the question. Make your questions specific. Each concept or idea to be addressed should be in a separate question.
- Keep the language simple and clear.
- Use words that do not need interpretation; instead of “often,” “sometimes,” “usually,” “recently,” “a lot,” etc., Use objective measures like “once a week.”
- Avoid leading questions, e.g. “Don’t you agree that our tests are fair?”
- Don’t combine concepts into one question, for example, “The textbook is easy to read and affordable.” Make that statement two questions.
- There are many formats for questions available that you can choose from. There are two main categories: “structured” with choices for responses and “unstructured” or open-ended questions.
When using structured questions, use appropriate scales with a “Neutral” or “N/A” or “Not Sure” choice. Failing to include such an option presents ambiguity to the students. Give them a clear set of choices
Be consistent with use of scales. The sirs form uses A five point scale. If you choose to use a different scale, e.g. seven point, then use that consistently for the questions that you design.
Open-ended questions should address one concept or idea at a time.
Don’t overuse the open-ended questions. This can lead to survey fatigue, unclear responses, or responses that are redundant or too brief to be useful.
Additional Resources on question design:
- The Pew Research Center provides guidelines for question development, structured and open-ended questions, and question wording.
- Qualtrics Survey Tool has 4 Common Sense Tips for Creating Surveys that Work.
- Survey Monkey has Tips for Writing Good Survey Questions.
Please contact CTAAR (848) 932-7466 firstname.lastname@example.org please replace brokenmail with docs.rutgers.edu with any questions or if you would like to discuss question design.