While things are fresh in my mind, I wanted to just write down a few quick thoughts about how we ran the tournament from the adjudication core side, and what lessons for good or bad can be taken from it. CA-ing a large tournament is something that most people only do once, or at the most twice, so I thought it might be useful for people to have some ideas from someone who has done it. Some of the ideas here are specific to the US circuit, but I think a lot of them would be relevant to anyone hosting a competition, particularly a large-scale one.
1. Judges’ test
We opted for a judge test that was not a great debate and in fact one that was difficult to call. What we considered most important was to find judges who identified important arguments for each of the teams and problems than the result itself. Everyone can judge one debate badly, and particularly so with a lack of others to discuss things with. As such, I think the focus should be much more on reasoning than decisions, with other questions about debating serving to determine who had the requisite level of knowledge to judge important rooms.
2. Judge feedback
We had limited time at the competition and given a lack of personnel chose to prioritise ballot running over someone to specifically tabulate judge feedback. As a CA team we took a triage approach, picking out the best and worst forms and seeing if others correlated with them. Based on the limited number of outliers, our feeling is that we were fairly accurate with our initial allocation of judges. However, I think for other tournaments having someone tab feedback, which I know has been the norm elsewhere, would preferable if the personnel are available.
3. Judge allocation
We trusted the tab to do its job. There were only a couple of very rare instances where we changed people about, and that was only for clashes or where equal ranked judges were randomly selected to chair and we moved the more experienced to the chair of the panel. Too often CA teams allocate panels to teams that they’re friends with. Tournamon does a great job of ranking panels and allocating the strongest judges to where they’re needed (bubble rooms). Having this approach and altering judge rankings based on feedback rather than changing panels after the draw sped up the turnover of the rounds hugely. Along with the Herculean efforts of our tab team, this was one of the main reasons that we were able to turn rounds around so quickly and stayed tightly to schedule throughout.
4. Open adjudication
We opted to have open adjudication in all rounds. It is the norm at competitions on both coasts in the US.
There is a tension between “debating as competition” and “debating as education” that clearly exists on the US circuit. I belong strongly in the former camp. However, I see the merit to the arguments for open adjudication. A long time is often present between debates and announcement and I think that can lead to people to alter their views of debates and convince themselves more strongly that they have broken. Having them in room shortly afterwards means that people get immediate (and not forgetful/ drunken) reasons for why they did not break. If competitions are to have open adjudication throughout, then there needs to be a strong norm against ‘throwing’ rooms. Personally I am still in favour of one closed round at the end of a tournament as I think it the excitement of not knowing who has broken (both for teams that are in contention and for those who are on the same teams/ friends with them) outweighs those benefits.
5. Warm room
Based on our tab team’s advice we instituted a ‘warm room’ policy of posting results from previous rounds in the debaters’ rooms and asking speakers to check to ensure that their results matched what they were orally told by judges. I think this is a really good idea, and am thankful to the team for suggesting it. Combining it with the acetate system that has been used at Worlds would be even better.
However, based on the arguments of some of the people present, the results went up with speaker points present. This was decided in the moment. Although it adds some interest for people to see how they’ve done as they’ve gone along and for others to see who is speaking well, I wouldn’t recommend it to others. Round 6’s results weren’t posted until after the break announcement, so people still had some of the nice tension of not knowing who the last teams in the break were. However, I think leaving speaks over as a surprise until the end of the tournament is more fun for everyone involved.
6. Wing judge questions
We asked wing judges to ask two specific questions of chairs, and put them on their feedback forms. One was ‘have we allocated speaker points to the right speaker’ the other was to double-check the math on the ballot to ensure no low point wins were given. Judge errors happen, but judges do not want to think that they are the ones responsible. Based on the experiences of pilots, doctors and others, we decided that these two formal questions were useful. We had a lower level of ballot math errors than usual. In cases where it did occur, it is highly unlikely that people actually followed our request to double-check the scores. I would recommend that other tournaments follow this policy. If it became the norm it would hugely reduce the number of errors that can often lead to large time delays as an errant chair is tracked down and asked to re-do the scores.
7. Communication lines
Something will always go wrong. Competitions with over six hundred people will always have incidents. One of the most important things is letting people know what’s going on when that does happen. There were two moments like that during the tournament that had that heart-stopping effect. One was that the team allocation for one of the schools was entered with their original rather than new list, that required a delay to round one while we made sure all teams knew where they were going. The other was that due to a room number error, four of the teams were dropped from the tab in the last round.
Thankfully all four teams were from the same debate and all that was needed was to let them know positions and assign judges to that room. That took about a 15 minute delay, but we tried to make sure people waiting in all three rooms around campus knew what was happening. My experience, from Dublin Worlds forward, is that people are far more forgiving of delays and errors when they know the reason why, rather than seeing the CA team running around worrying but without that information.
The other major area of communications was responding to questions raised about various tournament policies such as the inclusion of international teams and the issuance of judge and debater guidance that went beyond the mere WUDC rules and also discussed accepted norms. In both instances, legitimate differences of opinion were present. Based on the experience of similar questions in the lead up to other major international competitions, I decided to respond quickly to such threads and to explain our position. I think in other cases things have got a bit out of hand because of an absence of prompt official response.
8. Written documents need to be orally explained
We gave out maps and phone numbers to everyone involved in the competition, and thought that would be sufficient to get people to rooms and to phone in case anything went wrong. We were wrong on that one. Things would have been eased if we had both given them maps and numbers, but also let them know where the buildings were and told them specifically in the briefings to refer to their maps. You cannot trust people to read what you give them!
9. Motion setting
We brain-stormed motions before the competition and finalised them over a nice lunch on the Friday that the competition started. We had some really interesting motions that were perhaps more novel that we didn’t run. We took a deliberate philosophy of erring towards debates that we knew ‘worked’ than taking a gamble on an entirely new, and perhaps very good debate. I’m not sure if that is better or not, but our view was that for a title championships we would favour motions that we knew from experience, or close approximation in other areas to be fair. On purely logistical sense, googledocs were wonderful and I would recommend them to every other CA team. You can all collaboratively edit motions and leave comments in real time. They made the process of creating and critiquing motions much easier before the DCAs arrived.
As a CA team, we were really happy that we managed to provide a fair and balanced set of motions for debate. The lowest percentage of any round was 1st prop for round 6 where only 8.6% of teams won. However, given that 23.9% of 2nd prop teams won, I think it was probably more of an issue of teams taking a while to figure out the motion than it being un-propable. Anyone interested can take a look at the statistics for every round here. I think it would be a good idea for as many tournament as possible to post this kind of data afterwards. Over time, they would help us to set fairer motions.
10. Thank You
Finally, and most important I would like to thank all those who made the tournament possible, be they judges, debaters, volunteers or colleagues.
As many of you know, I am shortly leaving Vermont after two very happy years to return to Europe. It has been wonderful to be part of the US BP circuit that is rapidly maturing into a force to be reckoned with. I owe a huge debt to all those on the US circuit and particularly those in the North East who have made me so welcome. Thank you.
No comments:
Post a Comment