So, a bit over a year ago I started working with Mistress Inga, now my Laurel, on the Avacal Kingdom A&S Judging Forms. This consisted of a number of surveys of all of the Avacal artisans, and a more indepth one for the Laurels of Avacal. Based on those results we decided that a rubric would be the best way to go.
TL:DR scroll to the bottom for copies of the Avacal Judging Forms, the Judges & Entrants Handbook, and the Shire Judging Forms.
Making the Judging Forms
We then went through all of the judging forms of the Known World, yes all of them, and created the SPREADSHEET OF DOOM(tm). This listed all the kingdoms, a link to their judging form, notes on how they ran their championship, the categories they judged on, and the points given to each category (on page one). Page two was to determine the spread of points, looking at both the points for each category, but also assessing what percentage of the points were given for “documentation” related categories, and what percentage were given for “authenticity” related categories, and also what percentage those two were together. This was done because authenticity and documentation were considered the most important things in our survey of the Avacal Laurels. On average 20.5% of the points were given for documentation, and 23.5% were given for authenticity, which worked out to being 44% of the total score. Some kingdoms weighted documentation as low as 10% or as high as 30%, while some kingdoms weighted authenticity as low as 13% or as high as 40%. Together most kingdoms weighted them between 40-50% but one kingdom weighted them at 27% and one kingdom at 56%.
At this time we brought on a group of Laurels and the Kingdom A&S Minister to give us feedback as we went. That way we’d have people to bounce ideas off of and who could keep us focused or let us know when what we were doing was stupid.
Then we looked at what the categories from other kingdoms were, and it turns out there are only 18 categories used across the entire Known World:
- Historical Accuracy
- Technical Ability
Based on the surveys we knew that Avacal didn’t want to change the categories too much from what they’ve been in the past so we decided we could cover all 18 of those categories in our four:
- Technical Ability
Based on the surveys, again, we wanted to assign about 30% to Authenticity and Documentation, 25% to Technical Ability, and 15% to Display/Presentation. This would change later.
We decided 20 questions total would work well, as it would give us a good spread of questions with a total score of 100 points.
Then came the big part. Only three other Kingdoms used a rubric like what we were wanting to do, Artemesia, Trimaris, and An Tir. So we took every single one of their questions, it was only 38 of them, including the scale, and wrote it down. Which worked out to be about 11 pages. Then we assigned a category, based on Avacal’s four categories, to each question and gave each question one or two themes. Which gave us this chart:
|Authenticity||reasons for departure from period||1|
|Documentation||Documentation explains experiments||2|
|Documentation||Documentation explains project||3|
|Documentation||Documentation uses good sources||3|
|Documentation||Documentation explains research||3|
|Documentation||Documentation is well organized||1|
|Documentation||Documentation explains deviations from period||2|
|Technical Ability||Project shows mastery of skills/techniques||2|
|Technical Ability||Project displays high level of difficulty/complexity||2|
|Technical Ability||Project shows mastery of tools||2||Could also be authenticity|
|Technical Ability||Project shows mastery of methods||2||Could also be authenticity|
|Technical Ability||Project shows a high level of quality and has few visible flaws||1||could also be display|
|Technical Ability||Project shows Period style/design||2||Could also be authenticity|
|Technical Ability||Project shows Mastery of period materials||1||Could also be authenticity|
|Technical Ability||Techniques/processes/materials require high level of difficulty/complexity||1||Could be combined with general complexity|
|Technical Ability||Ammount of practice/training required to produce product||1|
|Technical Ability||Project succeeded||2|
|Display/Presentation||Display enhances project||2|
|Display/Presentation||Project impression – senses (sight, sound, taste, touch)||1|
|Display/Presentation||Presentation communicates well||1|
|Display/Presentation||Presenter can answer questions||1|
We used those 27 themes to write 24 questions. This started a lot of discussion with the advisory group as we tried to rewrite questions, or debated which ones should be merged together. In the end we had 28 questions. We whittled these down to 20:
- 5 authenticity
- 6 documentation
- 5 technical ability
- 4 display & presentation
Then we took those twenty and adapted some so they could be used for things that were not physical objects. That gave us two judging forms. Finally we took those twenty and created a third judging form for research papers. That took a while. In the end we had three judging forms with 20 questions. Then we started talking about what the bonus points could be. We decided they’d be “additional points” and gave a reasoning for each set of them. Then we looked at it all and realized there were some questions that were awarding points for virtually the same thing, so we merged them.
In the end we had our questions (each worth 5 points):
- 4 Display & Presentation (20 points + 2 additional)
- 5 Technical Ability (25 points +2 additional)
- 4 Authenticity (20 points + 2 additional)
- 5 Documentation (25 points +1 additional)
- 3 extra additional points
That gave us forms with 90 points maximum, but a theoretical maximum of 100/90.
It was a great deal of work, but we tested it out at KA&S in November, and again at Montengarde’s 12th Night. It seemed to work. And it was ready to be launched fully at KA&S 2018.
Making the Judges Handbook
One of the complaints that we heard frequently was that a lot of judges didn’t know how to judge, or how to use judging forms. So Mistress Inga and I set out to put together a handbook for both Judges and Entrants. We wanted to demystify the process. After a few months of work we had a finished product, which went to the advisory group to edit. After a few more versions we put it out to the populace about a month before KA&S 2018.
Shire Judging Forms
When we were testing the forms out in November we heard from a lot of people that the forms would be very useful at Shire level as there was a very big jump from Shire level to Kingdom level A&S. The problem was that most shire level entries wouldn’t score higher than a 3. And in fact that was by design. When we were designing the forms we wanted a 5 to be something that would win Kingdom, a 4 to be something that would win Baronial level and be expected at Kingdom, and a three to be something that would win Shire level and be expected at Baronial.
So after a bit of a break I went back to the drawing board and started cutting. A lot of the questions on the form deal with things that aren’t done as often at Shire level, so those were cut. Any score above 3 was cut. And most of the bonus points were cut. The end result was a maximum of 42 points, and a theoretical maximum of 45/42.
If you’re using the shire one, feel free to make some edits to it before you use it, that’s why I’ve included the excel file.
See it For Yourself
Here are links to the files: