Reviews

Four verdicts close a pending claim in Zealy: success and star both approve the claim, star marking a top contributor; fail and flag do not, and a flag also lands on the user's profile. Reviewers can batch-select pending claims, add bonus XP, leave a comment the contributor reads in their inbox, and undo any verdict afterwards.

Overview

When someone submits a quest claim, it appears in the reviews panel. If all tasks in a quest are set for automatic review, the system will mark the claim as either failed or successful. If there's at least one task requiring manual review and the automatic tasks pass, the claim status becomes 'pending' until a reviewer, admin, or editor evaluates it. Only successful claims earn XP or role rewards. It's the community's responsibility to distribute any other types of rewards.

Basics

Viewing reviews

Navigate through review items using the ↑ / ↓ keys on the list. Click on a review to see detailed information about the claim, user, and quest, and to perform any necessary actions.

Reviewing a pending review

Items awaiting automatic review are marked with a 'pending' symbol. Open a pending review to assess it via the form at the page's bottom. Use the lightning icon or CMD+K for a complete list of actions.

Choose from 'success', 'fail', 'star', or 'flag' status. 'Success' and 'star' indicate the claim is approved, with 'star' highlighting top contributors. A 'flag' means the claim is rejected and adds a flag to the user's profile.

Consider awarding bonus XP for exceptional work by selecting the lightning icon or CMD+K and searching for the bonus option. This adds to any existing XP reward.

Providing feedback through comments, especially for rejected claims, helps users improve. Utilize bookmarked comments with the # key for efficiency. This feedback will be visible to the user in their Inbox.

AI Review

On the Plus and Enterprise plans, you can turn on AI review for a quest so that submissions are assessed automatically. Switch it on per quest and write a prompt describing what a valid submission looks like; the AI applies that prompt to each claim.

The AI decides the claim. It does not leave a recommendation for a human to confirm — it marks the claim successful or failed, and the reason it gives is posted as the review comment the member reads in their Inbox. Reviews it submits are attributed to a Zealy AI reviewer, which appears alongside your team in the top-reviewers analytics — so exclude it before reporting how many claims your team handled. If you want a person to make the final call, leave AI review off for that quest.

Because the prompt is the only instruction the AI gets, it is worth being specific about what should fail as well as what should pass. Claims left pending on an AI-review quest are retried automatically.

Selecting multiple reviews

To select multiple reviews, hold shift while navigating with the arrow keys or when clicking a review item. You can also use the CMD+A shortcut to select all reviews or hold ALT while clicking to select toggle selection state for an individual item.

Note that it only works to batch review on pending claims.

Cancelling a review

You can cancel a review by clicking the 'Undo' button at the top of a reviewed claim's page. This resets the claim to 'pending', allowing for reevaluation.

Review Context

To make informed decisions, access comprehensive information about the user - including stars and flags - and the quest from the side panel. Open it by clicking the menu icon in the top right corner.

Video

FAQ