Evaluation Guidelines and Expectations

Under Construction

As you hopefully picked up from the big banner above this text block, this page is under construction! We’ll be regularly updating this page with additional information between now and January 1st, 2023. After that point, the page will still occasional updates as we continue to refine expectations and guidelines and provide more information to the community regarding general expectations. You can get a preview of what the various sections will look like under the L1 tab.

How to use this page

First off, this is not a page of requirements. You can find the requirements and processes for each level here. This is a list of general expectations and guidelines for each level (and eventually each advanced certification). They’re intended to provide a framework for evaluation – either of yourself or other judges – particularly when you’re not certain what is expected from a judge of a particular level. If you’re interested in leveling up, taking a look to see what the general expectations are and seeing if there’s something you feel you’re particularly good at, or an aspect that you believe you’d enjoy can help you get that extra boost of motivation. It can also help highlight areas where you could use some mentorship or feedback!

Similarly, if you’re a judge recommending a candidate, you can check to see if your standards are possibly too high! While it’s entirely okay to have your own set of standards and to focus on candidates who exceed the baseline, it’s important to not to lose sight of the bigger picture. An important part of all judge certifications is the willingness and drive to learn, and there’s always space for a candidate to learn more after advancing. You shouldn’t expect the same out of a new L2 that you would from someone who has held the role for years.

This document is intended to act as a set of guidelines for evaluating judges and candidates for various levels. This document goes into more detail than the concise guidelines available on the levels and advancement page. It is based on the same philosophy and a significant portion of definitions and explanations will overlap.

Before we delve into each of the categories below, let us define the terms being used as labels.

  • A candidate who is exceptional is above average even for experienced judges in that role.
  • A candidate who is very good is roughly average for a judge experienced in that role.
  • A candidate who is adequate meets the expectations of a judge who is new to the role.
  • A candidate who needs improvement indicates that the candidate is below average for a judge new to the role, and needs minor guidance.
  • A candidate who is deficient is very far below average and risks seriously underperforming and needs personal development and/or significant mentoring.

A single deficiency is not necessarily a reason on its own to not endorse a candidate, but at the very least, a concern that should be investigated. If a candidate is deficient and you are not comfortable endorsing them, you are not obligated to do anything further, however, mentoring is always encouraged.

This document is a guide – as always, use your best judgment.

The baseline expectation for a level one judge is to be able to effectively run an event at an LGS like an FNM or Prerelease. An L1 judge should have an understanding of the mechanics required to manage registration and launch an event, even if that process is frequently handled by the Tournament Organizer.

Additionally, their rules and policy knowledge should be enough for most of the players’ questions at the relevant events. They will also have knowledge of the resources available to make an informed ruling, in the event they are not wholly confident.

Finally, an L1 judge will be seen as a representative of the stores they work for and the judge community as a whole. They will be able to support and further the development of their local magic community and provide direction and resources to other individuals interested in becoming a judge themselves.

Leadership, Presence, and Charisma

All judges need to be a leader in some sense – at the very least, a judge needs to be able to issue instructions to the players in their event and have them followed. Having the presence or reputation as “The Rules Guru” is another important aspect of being a judge.

Exceptional: Can command the attention of 30 or more players easily, instructions are followed without pushback. Players approach the candidate regarding rules and policy questions even when the candidate is not acting in a role as a judge.

Very Good: Can command the attention of 15-30 players easily. Players call for the candidate when they have a rule or policy question when the candidate is the judge for the event, and sometimes even when the candidate is not acting in a role as a judge.

Adequate: Can get the attention of 15-30 players without significant difficulty and instructions are generally followed. Players call for the candidate when they have a rule or policy question when the candidate is the judge for the event.

Needs Improvement: Can either command a group of players or is the go-to resource for rules and policy questions, but not both.

Deficient: Can not get the attention of 15 players reliably. Instructions are frequently ignored. Players approach other individuals for rules and policy questions.

Development of Other Judges

At the rules expert at a local game store, a level one judge may be seen as a resource by Rules Advisors and other judge candidates. Being able to provide support and direction for judge candidates is desirable, even if it’s only helping interested parties find the Judge Academy website. However, it is not an expectation of level one judges to train or certify other judges.

Exceptional: Has participated in the training of other judge candidates, teaching rules and policy, as well as providing feedback in other areas, such as logistics or player interactions.

Very Good: Has a thorough understanding of the CR and the JAR, and can explain the basics to a candidate.

Adequate: Has a foundational understanding of the CR and the JAR. Knows where they have gaps in their understanding and doesn’t try to teach those subjects. Can direct a candidate towards educational resources.

Needs Improvement: Has difficulty explaining large portions of the CR and the JAR, but generally provides accurate information when they do teach.

Deficient: Provides inaccurate information regarding the CR or the JAR when talking with judge candidates. Can’t provide direction towards educational resources.

Logistics and Tournament Operations

A level one judge can be expected to run an 8-16 player FNM event with no assistance from the Tournament Organizer or store employees or run a 32-player pre-release with marginal administrative assistance. These events are the cornerstone of LGS play, and thus the baseline expectation for a level one judge.

Exceptional: Can run a 50+-player event with no assistance, including scorekeeping, registration, and answering calls.

Very Good: Can run multiple 8-16 player events simultaneously or a 32-player prerelease with no assistance.

Adequate: Can run a 16-player FNM with no assistance or a 32-player prerelease with minor assistance from the TO.

Needs Improvement: Struggles to run a 32-player prerelease without significant logistical assistance.

Deficient: Struggles to run an 8-16 player FNM without assistance or guidance.

Stress Management, Self-Evaluation and Maturity

While FNM and prereleases should generally be low-stress environments, judges should be able to manage stress levels so that it doesn’t impact the tournament. Knowing when to crack jokes and when to be serious are important elements of engaging with players and the community at large.

Being able to accurately evaluate yourself for weaknesses and areas of improvement allows for a mentor to more quickly narrow down what to focus on, and can consequently provide better support long term. Furthermore, listening to feedback from others is an important part of self-evaluation, as it allows the individual to calibrate their own perceptions and more accurately grade their own performance.

Exceptional: Proven comfort with high-stress situations, such as tournament software breaking during an event. Is very accurate with their evaluation of self. Has deep insight into why they behave the way they do. Responds positively to and seeks out critical feedback.

Very Good: Is able to manage stress frequently, but not always. Is generally accurate with their evaluation of themselves. Independently seeks feedback or performs self-evaluation regularly.

Adequate: Doesn’t get overwhelmed by common errors (misentered results, incorrect rulings). Is generally accurate with their evaluation of themselves. Usually reacts well to critical feedback.

Needs Improvement: Inconsistently responds to stressful situations. Moderately overestimates or severely underestimates their skill in a few areas. Reacts defensively to critical feedback.

Deficient: Is easily overwhelmed by logistical demands or is incapable of de-escalating minor conflicts. Severely overestimates their skill in multiple areas. Ignores critical feedback.

Teamwork, Diplomacy and Conflict Management

Judges should be able to get along with store employees and players. Being able to deliver difficult rulings and penalties diplomatically is key to creating a welcoming environment. While regular REL events should generally be low-conflict, judges should be able to mediate in-game disagreements or be able to involve the appropriate authorities in larger conflicts.

Exceptional: Tournament Organizers or other judges request or seek opportunities to work with the candidate. The candidate is seen as a sounding board for serious issues. Has extraordinarily rare lapses in diplomacy and maturity, if at all. Can de-escalate very upset players to the point where they can comfortably return to their match.

Very Good: Is sometimes sought out to work with. Recognizes their own diplomatic missteps quickly and seek education to correct them. Is able to manage significant disputes frequently, but not always.

Adequate: Gets along well with the majority of staff and players. Commits very infrequent diplomatic missteps only through ignorance, and seeks education to correct them. Is able to manage minor disputes consistently.

Needs Improvement: Has infrequent lapses of diplomacy or maturity, and seeks education to correct them, but does not always adjust behavior.  Adverse to conflict, and will avoid situations rather than address them.

Deficient: Gets into heated arguments with other staff members or players. Does not get along well with a significant portion of the community.

Penalty and Policy Philosophy

While the philosophy of the Judging at Regular document is more straightforward than the Infraction Procedure Guide, it is still important to understand the goals of policy, and what the intent is so that when the fix or penalty doesn’t line up with the intent, the judge is able to deviate in an effective manner that makes for a more positive experience for all involved individuals.

Exceptional: The candidate has a thorough understanding of the JAR, deviates from policy when circumstantially relevant, and is able to explain why they deviated.

Very Good: The candidate understands the philosophy of deviations and most of their deviations are reasonable decisions.

Adequate: Can execute the policy written in the JAR and understands the goal of education over penalization.

Needs Improvement: The candidate’s deviations are inconsistent with both the JAR and their own rulings but recognize their own mistakes.

Deficient: Any of the following are significant issues: Issues strict penalties or favors players in their rulings due to experience or who committed the error. Focuses on penalties and infractions, not fixes. Uses the philosophy of the IPG instead of the JAR to address problems. Handwaves or dismisses players’ concerns as not important.

Investigations

Investigations encompass a broad ability to understand and evaluate board states, including determining how to perform rewinds, managing card counts or life total discrepancies, and sometimes evaluating for cheating.

Being able to conclude such tasks efficiently and accurately is key to keeping the event running smoothly.

Exceptional: Can perform card counts and evaluate board states for rewinds while understanding the strategic implications of a backup. Understands and executes advanced investigation methodologies including seeking collateral truths and testing hypotheses in an efficient manner.

Very Good: Can comfortably execute backups and investigate for cheating. Can perform basic card counts.

Adequate: Is able to identify advantageous errors and cheating when they occur and can execute low-complexity backups without significant trouble.

Needs Improvement: Has some difficulty with either considering cheating or executing a backup.

Deficient: Can not perform a low-complexity backup. Only considers the possibility of cheating when someone else brings it up.

Rules and Policy Knowledge

Rules and Policy Knowledge are tested by the L1 Exam, as this is one of the more objective skills involved in judging. The areas of knowledge are: Turn Structure, Casting a Spell, Keyword Actions and Abilities, Layer Seven (Power & Toughness), Combat, Replacement Effects & Triggered abilities, State-Based Actions, and the JAR.

Exceptional: Has strong knowledge of L1 content and some knowledge of more advanced rules and policy, in addition to understanding underlying concepts.

Very Good: Has the knowledge taught in the L1 modules and tested on the L1 exam as well as some of the underlying concepts and principles that support those rules and policies.

Adequate: Has the knowledge taught in the L1 modules, and tested on the L1 exam, including the JAR and MTR basics, with infrequent errors.

Needs Improvement: Has most of the knowledge taught in the L1 modules with no more than two areas of rules or policy that present significant problems or frequent errors.

Deficient: Has several areas where they have significant problems and frequent errors.

Game and Format Knowledge

Understanding the format and various cards or interactions common to it helps judges better answer questions and provide a smoother experience for players. This is often reflected in rules knowledge and tournament logistics, but isn’t wholly encompassed by them.

Exceptional: Has a solid understanding of the meta in multiple constructed formats or a deep understanding of the gameplans in one or more constructed formats and understands the common strategies and decks built in the current limited format.

Very Good: Has a solid understanding of the meta in one or more constructed formats or regularly plays the current limited format.

Adequate: Recognizes common decks in common constructed formats. Knows a few key cards in current limited formats.

Needs Improvement: Is vaguely familiar with either constructed or limited, but not both.

Deficient: Is not familiar with any sort of sanctioned formats.

Advanced Skills

These are not skills that are expected or required of level one judges but are roles that advanced candidates or experienced L1s may have.

Side Events: Working on side events at larger events is a common role and is frequently the first role judges get at an event of that scale. The logistics of an event like that is very different from in-store play, where the judge is more likely to need to know how to operate tournament software and distribute product without any support.

Advanced Rules Knowledge: Some L1 candidates may have picked up advanced rules knowledge from play or study. This is to be encouraged unless it hampers the candidate’s understanding of the fundamentals of the Comprehensive Rules.

Competitive REL: Sometimes a judge candidate will have some knowledge of the IPG. While understanding more about Policy is good, the candidate should take particular care to not apply the fixes of the IPG to events at regular REL

This tab is currently empty, but content will be added in the forthcoming weeks.

Leadership, Presence, and Charisma


Development of Other Judges


Logistics and Tournament Operations


Stress Management, Self-Evaluation and Maturity


Teamwork, Diplomacy and Conflict Management


Penalty and Policy Philosophy


Investigations


Rules and Policy Knowledge


Game and Format Knowledge


The baseline expectation for a level three judge is someone who can head judge a 200-400 player competitive event without significant TO support or another high-level judge (but with appropriate staff), team lead at large competitive events of 800+ players, or floor judge the Pro Tour. An L3 judge should furthermore be able to evaluate candidates for L1, L2, and L3 (after some experience with the level), and is able to lead any team and provide logistical support or feedback to their head judge.

A level three judge’s rules and policy knowledge should be enough to handle every common scenario from memory, and non-corner case scenarios should require little to no effort to resolve. Beyond that, they should be able to explain the why behind both rules and policy to experienced judges who are looking for more knowledge and understanding.

On top of that, a level three judge is a positive influence in their community and at events, someone that judges turn to for aid not only with regard to the fundamentals of judging – rules and policy – but leadership, managing and responding to conflict and disagreement, evaluating oneself and others and more.

This page is not complete and additional details will be added as they are finalized.

Leadership, Presence, and Charisma

Level three judges are have earned the respect of players and judges as experts on the various tasks of judging and the optimal execution. Furthermore, they’re seen as exceptional leaders at events. They have the trust of the judges under their authority and the tournament organizers they work for to get the task done, and in a way that gives other judges the opportunities to grow and learn. They understand the difference between getting their team to complete the task and leading their team to complete the task. Judges enjoy working with them.

Exceptional: Is able to exert their influence on their community in a positive way. Is looked to as a guided for leadership and management, and a go-to resource for advice at large events, even when not in leadership roles.

Very Good: 

Adequate: 

Needs Improvement:

Deficient: Has trouble commanding the respect or attention of judges or players in their community, or on their team at events. Can confuse role-based authority with genuine leadership. Struggles to manage their team.

Development of Other Judges

Exceptional: Is able to consistently write quality reviews and provide detailed feedback. Their evaluation of judges is combined with practical mentorship and guidance. They are insightful and reveal understanding to their subjects that they had not realized about themselves. They are able to evaluate more experienced judges effectively as well as peers. Their influence as a mentor extends beyond a small group of judges.

Very Good:

Adequate:

Needs Improvement:

Deficient: Shows little to no mentorship at or outside events, or their mentorship is demonstrably ineffective. Reviews may only asses other judges superficially or lack depth, detail and accuracy, while not providing actionable feedback. Is unable to evaluate a broad range of pillars in other judges.

Logistics and Tournament Operations

Exceptional: The candidate can comfortably lead any team, and is a resource for best practices. They also develop new techniques, with a good success rate. They are able to identify essential tasks and provide support to other judges.

Very Good:

Adequate:

Needs Improvement:

Deficient: The candidate is effective in only some roles, and may lack the awareness of the greater event and problems that may crop up out of their sight.

Stress Management, Self-Evaluation and Maturity

The ability of assessing ones own strengths and weaknesses, and being honest about failures and finding ways to improve on their own is key to an L3 judge – who may not have peers at events to evaluate their performance.

Exceptional: Shows deep insights into their own judging practices and is able to identify strategies for improvement. Thrives under stress, where the pressure drives them to achieve more.

Very Good:

Adequate:

Needs Improvement:

Deficient: Self evaluation lack depth, detail and accuracy. The candidate is unable to assess their own performance in meaningful ways. While under pressure the candidate may become noticeably affected, and is unable to maintain composure or focus.

Teamwork, Diplomacy and Conflict Management

Exceptional: The candidate is known for their tactfulness, and is often sought to help mediate conflicts between other judges, and is an excellent ambassador for the game and the community. They are able to deescalate irate players with ease, and either is able to prevent conflicts from arising between judges or is able to resolve them quickly.

Very Good:

Adequate:

Needs Improvement:

Deficient: The candidate demonstrates difficulty following the leadership of others, or is notably less effective when not in leadership roles, often pushing back against other judges plans or ignoring them completely. They may have lapses in tact that cause discomfort or outrage in other judges – either in person or in online venues. They are not a good ambassador for the program or the game. The candidate may demonstrate difficulty in calming irate players and frequently finds themselves in conflict with other judges or is unable to resolve those conflicts.

Penalty and Policy Philosophy

Exceptional: The candidate has exceptional understanding of policy, and regularly provides feedback that improves written policy.

Very Good: The candidate is able to offer suggestions that could improve policy.

Adequate:

Needs Improvement:

Deficient: The candidate has extraordinarily poor policy knowledge or their philosophical views (the goals of policy) run contrary to the basic principles. They are unable to explain why policy is written the way it is, or explain policy to players or other judges.

Investigations

Exceptional: The candidate is effective at investigating a player’s intentions, or discovering past events of a game. They are able to do so in an effective and efficient manner. They also take into account information beyond testimony (include game state, risks, motivations).

Very Good:

Adequate:

Needs Improvement:

Deficient: The candidate does not dig deeper than the basic resolution of a situation presented in a judge call. They may take an excessive amount of time, or arrive at incorrect conclusions, whether for cheating investigations or card counts and life-total discrepancies.

Advanced Skills

These are not skills that are expected or required of level three judges but are areas where a level three judge can specialize a bit more.

Professional Event Head Judge: After working many professional-level events, a level three judge can aspire to professional event head judging. This is a highly demanding role, requiring the judge to not only have exceptional rules and policy knowledge, but be an exceptional leader, have exceptional logistics skills, and be able to balance the needs of a broadcast and player experience with tournament flow and integrity.

Evaluation Lead: After participating in several evaluations, an L3 can lead an evaluation interview. This requires a deep understanding of the pillars of a level three judge, and how to quickly understand a candidate, their strengths, and their weaknesses.

Rules and Policy Knowledge

Exceptional: Has a comprehensive knowledge of the rules and tournament policy. Is able to explain very complex concepts and rulings to players and judges of all experience levels.

Very Good:

Adequate:

Needs Improvement:

Deficient: The candidate has difficulty explaining or understanding uncommon (but relevant) rules and policy scenarios.

Game and Format Knowledge


The ability of a judge to evaluate themselves and their performance is fundamental to continued growth, particularly at higher levels. When you have less experience, there are more people who can easily provide actionable feedback. However, as you become more senior, fewer individuals have the knowledge or perspective required to be able to provide consistent actionable feedback.

As the Head Judge of a 200-400 player event, you might be the only head judge in the room and possibly the only person who has managed an event of that scale. While the feedback you receive from your team leads, floor judges, and the Tournament Organizer remains valid, you may be the only person to see the broader scope of the event, including the logistical details of how all the teams work together alongside the execution of the event.

It is important that, as a leader, you avoid complacency and regression, and to do that, be able to identify when you’re slipping and how you can address the issue.

As part of the level three process, you are required to perform a thorough self-evaluation on the pillars of a level three judge.

Requirements

Each of these self-reviews has several requirements:

  1. Evaluation: Using associated resources, compare yourself to the expectation for level three: major deficiency (0), minor deficiency (1), underdeveloped (2), adequate (3), above average (4), or exceptional (5). This benchmark will allow other judges to understand where you think you are in a very clear and direct manner.
  2. Strengths: It is essential for you to be able to identify what you are good at, so you know where to devote your energy.
  3. Weaknesses: Even more important is knowing where you aren’t up to par, and what needs to be improved.
  4. Plan for Improvement: These should be either practices you have already implemented (that have shown results) or strategies you plan to take and why you think they will work.
  5. Examples: The people reading your self-review in the later stages of your evaluation may not have worked with you, or if they have, not regularly. It’s not enough to state your conclusions, you must also provide evidence that supports your understanding.
  6. Critical Analysis: Furthermore, showing an understanding of cause and effect is important – explaining how your example demonstrates your strengths and weaknesses and how your plan will address those elements ties the whole thing together.

Evaluation of the Pillars

Understanding where you are in comparison to the baseline expectation for level three and where you are relative to experienced level three judges can be a difficult task. For explicit detail on all of the pillars and what the expectations are for level three, as well as explanations of the scale of ability from deficient to exceptional, check out the guidelines in the L3 tab on this page.

Even so, reading the documentation is no comparison to practical experience – the L3 process can take several years and most judges never begin, much less complete the journey. It is common for most people who pursue L3 to realize that their earlier estimations of their skill were higher than it should have been and their understanding of what is required and expected of L3s was lower than it should have been. However, recognizing that discrepancy and correcting your understanding is a hugely valuable growth opportunity.

The Purpose of the Self-Review

There is no expectation that an L3 candidate is able to analyze themselves perfectly, especially at the beginning of the process. Instead, the goal is to ensure that the candidate has critically evaluated themselves and considered the role of L3.

Furthermore, the candidate has the closest perspective of themselves and their skills – they’re the only one who can truly say why they behave in any given way or what they think will work for them for future development.

A self-review is not a solo evaluation. You are absolutely allowed and encouraged to collect feedback and perspectives from judges, tournament organizers, or even individuals outside the judge program to help you understand who you are and your performance. You should not simply regurgitate other people’s perspectives, but using their insights to develop a foundation or further your understanding is a great place to begin.

Who Sees the Self-Review

The self-review can be an intimidatingly open document – you’re talking about your personal failings and explicit weaknesses. Only a handful of people will see the self-review as part of your L3 process.

  • Validator: A Judge Academy employee will read and approve your self-reviews as you submit them. This person is not evaluating for accuracy – only confirming that the self-review meets the technical requirements. They may require additional comments from the candidate and may make suggestions to the candidate to help them on their journey of growth.
  • Interviewer: A Judge Academy employee will consolidate information gained from your practical evaluations, recommendations, and self-reviews to develop questions for your prep interview. This is likely the same individual as the validator.
  • Recommending Judges: If a judge recommends a candidate for a particular pillar, they should be granted access to the relevant self-reviews.
  • L3 Practicum Shadow: The L3 who acts as your shadow during your L3 practical will have access to all self-reviews, in order to evaluate the differences between your evaluation and theirs.
  • L3 Evaluators: Both the Evaluation Lead and Evaluation Observer will have access to your self-reviews, recommendations, interview, and any feedback from previous stages.

In case of significant differences in perception between evaluators, your self-review may be shared with those evaluators. However, if this occurs, you will be informed exactly who will be able to see your self-reviews.

Self-Review Accuracy

As mentioned above, your initial self-review is not expected to be perfectly accurate – but it should be as close to accurate as your understanding allows. It is not an uncommon occurrence that a candidate submits multiple evaluations during the L3 process, as their skills and their understanding of their skills grow.

Upon initial submission, the self-review is not evaluated for accuracy by Judge Academy. The first check for accuracy will be performed by the L3s who are writing your recommendations. In the event that there is a large difference between your evaluation and that of your recommender, you should have a conversation about why that difference in perception exists, and what can be done to remedy any deficient qualities.

The next check for accuracy is during the prep interview. The interviewer will have access to your evaluations from your recommendations, and any practicals you have completed. At this point in time, there will be a check to see if your evaluation of yourself matches your external perception of yourself. Any significant differences will be highlighted for the L3 Evaluation Interview to focus on, as potential areas of concern. If there are significant failings in self-evaluation, the process may be extended.

The final check for accuracy is during the L3 Evaluation Interview, where your observer from earlier in the weekend and the evaluation lead will use a wide array of resources to provide you insight into where your understanding of your skills differs from an external perspective, and what that means for you moving forward.

Approval and Rejection

When you submit your self-review, it will be reviewed by a Judge Academy staff member, and evaluated against the requirements that are both listed above and on each self-review assignment.

If your self-review meets the requirements, it will be approved, and you’ll be able to use the review as part of your L3 process. The self-review will be valid for one year from when it was approved.

If it does not meet the requirements, the validator will comment on your self-review and give feedback based on which requirements were not met. You will be asked to either submit an addendum in the form of a comment or re-submit the self-review.

In some cases, the validator will comment on your self-review, providing some guidance or tools that may help you on your L3 path, in addition to approving it.

Updating Your Self-Review

If you’re asked to update or add further information to a segment of your self-review, simply comment on the initial review, and the validator will look at your submission and either approve it or give further feedback.

If your self-review expires, you will be asked to submit a fresh review, which will include uploading an updated version, including examples from the past six to twelve months.

You are, of course, allowed and encouraged to update your self-review with new experiences if you ever feel you have grown or learned something relevant. You can do this by simply adding a comment to your approved self-review. You can also upload a new document, which will be validated independently of previous submissions.

Nobody comes into the L3 process ready to be L3. Heck, most people, when they pass their panel aren’t ready. Having practical experience and getting repetitions in that practical experience is what makes someone suitable. The L3 Leadership Practical requires the candidate to either Head Judge a 9+ round Comp REL event with at least 12 other judges on staff or Team Lead a 10+ round Comp REL event with at least 4 judges on their team, while leading a logistically intensive team (generally either End of Round or Deck Checks). The candidate will be shadowed and evaluated by an L3 judge who submits an evaluation to Judge Academy as part of phase two (Exam, Prep Interview and Practical) of the level three process. The evaluator must either be appeals judge (when head judging) or on the candidates team (when team leading). Requests for practicals should be submitted to advancement@judgeacademy.com at least one month before the event is scheduled.

The practical evaluations have multiple purposes.

  1. Is the candidate close to meeting that goal?
  2. What areas does the candidate need to improve in?
  3. Can the candidate demonstrate continued growth?

There are two parts to this evaluation, the Holistic Recommendation, and the Evaluation Rubric. The holistic recommendation looks at the candidate as a whole and has very loose guidelines. The evaluation rubric is more precise, and looks for more explicit details, regarding this particular event and the candidate’s performance.

Holistic Recommendation

The holistic recommendation aims to determine whether a candidate is prepared to move forward in the L3 process. A candidate should not be recommended if they demonstrate a major deficiency in any of the seven pillars of an L3:

  • Leadership, Presence, and Charisma
  • Development of Other Judges
  • Logistics and Tournament Operations
  • Stress Management, Self-Evaluation, and Maturity
  • Teamwork, Diplomacy, and Conflict Management
  • Penalty and Policy Philosophy
  • Investigations

A candidate does not need to demonstrate every quality in the practical evaluation, as some may not be tested due to the nature of the event. Any qualities that can not be evaluated should not be held against the candidate.

Do you believe that the candidate has no major deficiencies and is prepared to move forward in the L3 process? What pillars, if any, do you feel the candidate can significantly improve upon? What pillars, if any, were you unable to observe and evaluate?

Evaluation Rubric

Beyond the holistic recommendation, the candidate should be evaluated on several elements listed below. The purpose of the rubric is to provide guidance to the candidate and additional information for any future interviews or evaluations of the candidate.

The candidate’s score can range from 0 points to 35 points. There is no default passing score.

Head Judge - Evaluation Rubric

Task Prep __/5

  • Created staff schedule
  • Communication with staff
  • Has plan prepared for logistical tasks

Success with Task __/5

  • Event runs smoothly
  • In the event of logistical complications, manages to provide and execute solution
  • In the event of logistical complications, remains composed and focused

Delegation __/5

  • Tasks are appropriately assigned to team leads or point people
  • Let team leads guide their teams
  • Does not delegate tasks that they should personally handle

Event Overview and Feedback __/5

  • Understood how each team was proceeding with their task
  • Was aware of floor coverage
  • Was able to give feedback to at least their direct reports

Team Building and Mentoring __/5

  • Able to create an atmosphere of a unified team
  • Able to provide meaningful advice and guidance to members of the staff

Communication __/5

  • During the event was able to gather any necessary information
  • During the event was able to distribute any necessary information
  • Able to deescalate conflicts between players and judges

Rules, Policy, and Investigations __/5

  • Accurately upheld or overturned rulings on appeal.
  • Demonstrated understanding of policy philosophy in complex scenarios
  • Efficiently and effectively investigated in scenarios when appropriate
  • During contentious appeals and investigations is able to remain composed and keep the situation from escalating

Team Lead Evaluation Rubric

Task Prep __/5

  • Had necessary communications with HJ and other leads before the event
  • Has plan prepared for the team’s tasks
  • Communicated plan to the team before the event

Success with Task __/5

  • Event runs smoothly
  • In the event of logistical complications, manages to provide and execute solution
  • In the event of logistical complications, remains composed and focused

Delegation __/5

  • Tasks are appropriately assigned to members of the team
  • Let other members of the team provide guidance and mentorship as appropriate
  • Does not delegate tasks that they should personally handle
  • Was able to appropriately share members of their team with other teams that needed assistance

Event Overview and Feedback __/5

  • Understood how each member of the team was proceeding with their task
  • Was aware of where judges on their team were throughout the event
  • Was able to give some feedback to the judges on their team
  • Was able to give some feedback to the head judge

Team Building and Mentoring __/5

  • Able to create an atmosphere of a unified team
  • Able to provide meaningful advice and guidance to members of the team

At Event Communication __/5

  • During the event was able to gather any necessary information regarding the task
  • During the event was able to distribute any necessary information regarding the task
  • Was able to manage any conflicts that rose within the team
  • Was able to request additional resources, if required

Rules, Policy, and Investigations __/5

  • Was able to provide concise and correct answers to judges seeking advice in live calls
  • Was able to provide reasoned insight regarding potential rewinds, reversing decisions or HCE calls they were consulted on
  • Demonstrated understanding of policy philosophy in complex scenarios
  • Efficiently and effectively investigated, and escalated when appropriate

In addition to each of the levels, we’ll eventually be adding guidelines for all of the advanced roles and badges that can be earned, including both those associated with levels and those independent of levels.

  • (L1) Advanced Comprehensive Rules Badge
  • (L1) Competitive Policy Badge
  • (L2) Small Events Lead Badge
  • (L2) Medium Events Lead Badge
  • (L2) Policy Expert Badge
  • (L3) Panel Evaluation Lead Badge
  • (L3) PT Head Judge Badge
  • Commander Specialist
  • Digital Events Specialist

We’ll also have more specific documentation outlining what’s expected of recommendations and self-reviews, as well as a glossary covering various specific terms. Those may or may not end up on this page.