CrowdGrader
What is CrowdGrader
CrowdGrader is an online peer review tool that allows for peer assessment and feedback on submitted work.
What is the purpose of CrowdGrader
CrowdGrader supports instructors in conducting a peer review assignment by semi-automating the submission of student work and the allocation of classmates as peer reviewers.
How to use CrowdGrader
An instructor begins by setting up an account and creating an assignment - including settings for submitting student work and the creation of grading rubrics. After an assignment is created, the instructor can then specify the class roster or students can auto-enroll through a course-specific hyperlink. Note that CrowdGrader uses Google for authentication, and so all users - instructor and student - must use a previously created Google account login to access CrowdGrader.
Students submit their work to CrowdGrader and, upon a deadline, the review process begins. Students can see which reviews they need to do from their own profiles. Reviews are conducted anonymously. For each assignment, students need to complete the minimum number of reviews as specified by their instructor. CrowdGrader calculates each student’s mark by consensus-based grading that takes an average of the total scores granted through review. Instructors can track class completion of the review process through their account.
Beyond class sizes of 10 students or less, GrowdGrader works on a fee structure. Student-supported fees are discouraged at Western, particularly if the peer review component of the assignment is mandatory.
Where to find Support
This tool is not centrally supported
Help and Documentation
For help, please refer to the CrowdGrader documentation
Supported Tools to Consider
Tool Evaluation
Functionality
| Rating | Rationale |
|---|---|
 |
ScaleThe tool can be scaled to accommodate any size class. Note that CrowdGrader can be used for up to 10 students for free however a fee structure is in place for various class sizes: small (up to 35 students), medium (up to 100 students) and large (unlimited students). |
|
Ease of UseThe tool has a user-friendly interface that is easy to navigate and personalize |
 |
Tech Support/Help AvailabilityTechnical support and help documentation is available in the form of web pages or videos |
|
HypermedialityThe tool is restrictive in terms of the communication channels employed (audio, visual, textual) and presents information |
Accessibility
| Rating | Rationale |
|---|---|
 |
AODA FriendlyNo information on how the tool is compliant with standards for accessibility (e.g. AODA or W3C WCAG 2.0 Standards) has been made available for the tool |
|
User-focused participationThe tool has some limited capacity to address the needs of diverse users, their various literacies, and capabilities |
|
Required EquipmentProper use of the tool does not require equipment other than a computer and access to the internet |
Technical
| Rating | Rationale |
|---|---|
|
OWL IntegrationThe tool can only be offered in OWL through a link rather than a functional version of the tool itself |
|
Operating SystemsUsers can effectively utilize the tool with any standard, up-to-date operating system (i.e. Windows 10, Apple OSX, etc.) |
|
Web BrowserUsers can effectively utilize the tool with any standard, up-to-date web browser (i.e. Google Chrome, Safari, Firefox, Internet Explorer 10 etc.) |
|
Additional Tech RequirementsUsers can effectively utilize the tool without any browser extensions or downloaded software |
Mobile Design
| Rating | Rationale |
|---|---|
|
Mobile Operating SystemsThe tool is fully functional with a range of electronic mobile devices (laptops, tablets, touchscreens, mobile devices, etc.) |
|
Mobile FunctionalityBasic features of the main tool are functional on the mobile app but advanced features are limited |
|
Offline AccessThe platform cannot be used in any capacity offline |
Usage and Account Set Up
| Rating | Rationale |
|---|---|
|
Sign Up/Sign InAll users (instructors and students) must create an external account or additional login. Users require a Google account to access CrowdGrader. The privacy policy indicates personal information will not be shared with third parties. |
|
Cost of UseThe tool can be used for free (up to 10 students) however a fee structure is in place for larger class sizes: small (up to 35 students), medium (up to 100 students) and large (unlimited students) |
|
Archiving, Saving, and Exporting DataInstructors and TAs can download all assignment data however it requires the download of a GitHub code See How to Download all Assignment Data |
|
Data Privacy and OwnershipGrowdGrader’s Terms of Use specifies that “CrowdGrader does not claim ownership of any content posted to the Service. Users hereby grant CrowdGrader a non-exclusive, fully paid and royalty-free, transferable, worldwide license to use, copy, analyze, create derivative works of, distribute, and display the content they submit”. The Retention of Information section of the privacy policy indicates that while assignments can be made inactive and no longer accessible to other users GrowdGrader may choose to retain any information collected indefinitely and may, among other reasons, may analyze past assignments to “improve the algorithms and techniques used by the service”. |
Social Presence
| Rating | Rationale |
|---|---|
|
CollaborationBeyond supporting peer-to-peer review, CrowdGrader also allows for submission of group solutions |
|
User AccountabilityAccount logins are managed through Google Accounts. This means that Instructors cannot control student anonymity and user accountability may be challenging. |
|
DiffusionStudent familiarity with the tool is likely mixed, some will lack basic technical competence with its functions |
Instructor Presence
| Rating | Rationale |
|---|---|
|
FacilitationThe tool has easy-to-use features that would significantly improve an instructor’s ability to be present with students via active management, monitoring, and engagement |
|
CustomizationThe instructor can customize much of the assignment set up and review process. Customization of rubric design is somewhat limited. |
|
Learning AnalyticsInstructor can monitor students’ performance on limited measures. Grades are computed on a consensus-based grading algorithm. |
Cognitive Presence
| Rating | Rationale |
|---|---|
|
Enhancement of Cognitive Task(s)The tool enhances engagement in targeted cognitive task(s) that were once overly complex or inconceivable through other means |
|
Higher Order ThinkingUse of the tool easily facilitates students to exercise higher order thinking skills (given consideration to design, facilitation, and direction from instructor) |
|
Feedback on LearningThrough the tool, students can regularly receive formative feedback on learning (i.e. they can track their performance, monitor their improvement, test their knowledge, etc.) |
Security and Privacy
This tool has not been formally reviewed by Western’s Technology Risk Assessment Committee
Security and Privacy Flags
Accounts
- Users are required to create and use a non-Western account. It is highly recommended that users do not use the same username and password that is in use at Western.
Information Collection
- Application collects information that could identify the user specifically (email address, class schedule, etc..)
EULA
- Acceptance of the Application's Terms of Conditions or End User License Agreement may bind the institution to an agreement thereby requiring legal counsel to review the application contract.
Cost and Approval
- The application may require the instructor to enter credit card information
- The application may require students to enter credit card information
Integration
- This application is hosted outside of Canada
Last Updated: September 16, 2020
Rubric
The eLearning Toolkit Evaluations are made available under the terms of the CC BY-NC-SA 4.0 licence
Feedback
Leave feedback to comment or request that the tool be re-reviewed
Topic at a Glance
CrowdGrader supports peer review for peer-to-peer feedback on student work.
