motris (motris) wrote,
motris
motris

Open Letter to the World Puzzle Federation Regarding Sudoku Championships

As a solver at all four WSCs and as a past champion of the event, I am writing this letter to address recent problems with the World Sudoku Championships and offer some recommendations on changes that should be instituted into the Rules of Competition for all future hosts.


Puzzle Selection and Classic Sudoku:
Following WSC1 and 2, a letter by Simon Prett appeared in the 12th Newsletter of the WPF. Its point can be summarized as this: “By not including classic sudoku, the championships have failed to capitalize on the international craze”. While I found parts of the article insulting or at least ill-informed, such as the arguments that a top sudoku solver such as myself or Jana Tylova also being good puzzle solvers was “proof” of the failure or that the championships in Lucca did not include classics when they had several, the board should consider what are appropriate sudoku for these championships.

The response by organizers to the concerns stated by Prett and others was first, at Goa, to emphasize classic sudoku by running a separate classic tournament for just the regular puzzle and now, to run rounds like the “Guinness” round to show off the skills of the room. However, an analysis of the puzzles that have determined the winner of the Goa Classic Playoff (puzzle 4), or of the World Record Puzzle in Zilina amongst many others, show that these are not puzzles that can be solved by “human-identifiable” logical means aside from bifurcation (aka "guessing"), at least within the time limits placed upon them by competition. These puzzles would never be printed in any newspaper for their solvers to do, at least if they intended to keep their audience.

Therefore, even by emphasizing classic puzzles, organizers have not approached the process correctly. In part, this may result from the ease of computer-generation of tough classic puzzles. I for one look forward to the variants over classics specifically because I know many of them had to be hand-constructed, and therefore have some logical path to an answer, where the classics might be impossibly difficult without “guessing”. At the last two WSCs, there have been reasonable classics and unreasonable classics, and the number of unreasonable classics has been increasing. This trend leads me to wanting to “retire” from competition as having to guess on a classic sudoku at least once on a puzzle dozens of times in a competition, after having logically solved tens of thousands of sudoku without guessing in my lifetime, does not feel correct to me.

Online tools, such as Scanraid Sudoku (http://www.scanraid.com/sudoku.htm), provide an easy means to canonicalize the types of puzzles and types of required logical steps that should appear. This site includes approximately all the logical techniques known to lead to placements in a sudoku puzzle. They are listed in approximate order of difficulty. All solvers use techniques in the first 7 steps (singles, naked/hidden pairs/triples, pointing pairs, etc.). Tough strategies like X-Wing, Simple Colouring, Y-Wing, and Sword-Fish may be used, but typically by a Y-Wing or Swordfish the “fastest” solution of a puzzle is bifurcation. Techniques farther down on the list from Sword-Fish are far too difficult to spot within the time constraints of a round and should not be encouraged as the recommended steps in a puzzle. The Guinness Record puzzle, for example, requires a use of step 28-Finned Sword-Fish, to place the second number in the grid! Several more difficult steps were required to take that puzzle to its conclusion. There is no way any solver in a ten minute round used that Finned Sword-Fish to place the second digit, let alone solve the puzzle, but considering all the logical deductions known to the entire sudoku community, this is the simplest step to make any logical deduction.

The board should set standards for classic sudoku based on the set of known strategies and the easy ability to check the method a solver might encounter using a tool like Scanraid. It should helpfully develop standards for ranking puzzles that are appropriate for competition that use human-recognizable strategies within reasonable times, and allow for fast solvers, instead of lucky guessing, to lead to solutions. This is good for everyone from the competitors to the newspapers who would want to report on these championships as well. I cannot tell you how many people approached me during the Zilina championship asking if I knew how to "solve" a particular classic puzzle. These are accomplished solvers too, and the puzzle selection has completely failed to get consistently good competition-level classic sudoku. Solvers do not enjoy coming to a sudoku championship and being presented with "classics" that cannot be solved like the ones they have done in newspapers and books.

Variations:
On the topic of variations, these should occur at the World Sudoku Championship, but alongside reasonable classic sudoku. Many variations are actually relatively popular and known. The standards should be to have puzzles that are sudoku – to me this is a symbol-based cell-packing puzzle in which each cell is constrained in at least three ways – and not puzzles that are very far from the set of standard techniques used to solve classic sudoku. While I’ve written a book of Battleship Sudoku, that combines the puzzles Battleship and Sudoku, we can agree that this kind of puzzle – which rewards experience outside of sudoku – is not appropriate at a WSC. However, there are a whole range of variations that are appropriate. These include variations in grid size (6x6, 12x12, …), grid or cell shape (irregular/jigsaw, parquet, 3D/cube sudoku, …), additional constraints (diagonal, extra regions, …), properties of numbers (Consecutive, Kropki, Greater/Less Than, …), arithmetic (Killer sudoku, Arrow Sudoku, ...), and others I have missed. Having a balance across these variations (to not make all the puzzles "math sudoku" for example) is best. Innovations in design at the WSC, such as manipulative or mechanical sudoku, are the most memorable puzzles that occur and show off the talents of the puzzle designers. Variations should not be excluded from the WSC, as Simon Prett argued, but the proportion of variants should be reasonable. Recent championships, with ~50% variations, are at a reasonable precentage in my mind, particularly when many variations like "Diagonal" sudoku are not very far from the classic puzzles that serve as inspiration.

Rules of Competition:
While organizers may believe some “special” types of sudoku should be revealed during a competition or during a playoff, the WPF board should not allow an instruction booklet to be published without a formal presentation of the rules for the whole competition including the playoffs and finals.

To demonstrate what should have happened in Zilina, the rules should have included this kind of paragraph:
“Following the semifinal, a final will be held for the top 8 finishers from the semifinal. The final will consist of two puzzles, a classic 9x9 sudoku and a 'secret variant' that will be revealed beforehand. The fastest solver over both puzzles will become champion.”

Now, as a likely finalist, I would like to know what the “secret variant” in the above description is, which may be the surprise the organizers want to keep until the finals, but I certainly need and deserve to know the standards of competition as should all others. There should be no question how a champion will be determined. This problem is not unique to the recent event in Zilina but also occurred at WSC Goa where the means of winning including the number and type of puzzles, or the value of a time bonus, was not revealed until the day of the final, briefly before the actual competition. The best example may actually have been the early standard in Lucca where all 7 puzzles in the final, as well as the process of elimination, were clearly spelled out in the initial booklet. Keeping the rules "secret" from the competitors is unacceptable.

Instruction Booklet:
As the variations in sudoku should not take anywhere near the time to discuss as at an event like the WPC, the rules for these puzzles should be standardized and example puzzles released as soon as possible, certainly over a week and likely two weeks before the championship itself. The Czechs released a booklet with ~60 types of puzzles several weeks before the WSC in Prague to give solvers a sense of the puzzles that might appear at that championship. This is the standard that should be followed. Not all variations showed up, and a couple were added, but most of the “rules questions” were worked out well in advance of the competition. Having the rules presented online with a forum as on the worldpuzzle.org site is the right way to deal with “mistakes” in the booklet and add in corrections that can then be translated from English into the competitors’ languages before competition. Much as with the revealing of the rules of competition above, there should not be a benefit to a solver who is more fluent in English because no written version of a correct set of rules occurred.

Scoring:
Unlike many puzzle types seen at a WPC, sudoku have a clear means of determining how far a person is towards a solution – namely the number of correct digits placed in a cell - and WSC organizers should score with this in mind. A scoring system should reward correct solutions but also give partial credit (where deserving) to solutions that are far from blank. In a team round with a single large puzzle where almost no teams finished, the number of correct digits in the respective grids could easily have been used to assign a score to rank the teams in a logical fashion.

I personally propose the following example system as it makes sense to me: Puzzles are all graded out of 100. Correctly solving a puzzle, say a classic with 21 givens, gives you credit for all of the numbers you entered into the grid (60) as well as the givens (21) as well as 19 more points to get to 100 (60 + 21 + 19). Any solver that made a mistake in 1 cell gets a score equal to the number of correctly filled cells, 59. A blank grid is worth 0. If organizers want to score different puzzles differently, the score from this system should be treated as a “percent” of the scores and a correct solution to a 5 point puzzle is worth 5 points while a 59 cell solution with one mistake might be worth 3 (approximately 59% of 5) points. Whatever the scoring system, it should reward sudoku that are closer to solution, particularly since sudoku is almost always a puzzle type where it is easy to determine who is closer to a solution. It should certainly, in the playoffs, reward solvers who are closer to a 4th solution of a puzzle to finish a round than solvers who stopped after solving only 3 of the puzzles with a blank 4th grid.

Bonuses:
Bonuses on some rounds should be given to solvers that can complete the tasks early. The best bonuses should be based exclusively on the time to finish the round, with some amount of points per minute awarded to any solver that finishes the round with a correct paper. Rank bonuses are not as good as time bonuses because they cannot handle rounds that are poorly timed for some of the competitors. Jakob Ondrousek’s phenomenal performance in the Relay Round at Goa, India would not have been fairly served if his 12-minute-early finish was given simply 5 points for being 1st instead of the large time bonus it received instead. In Zilina, competitors weren’t even allowed to earn some bonuses as proctors would not throw out incorrect papers to restore the proper bonus for 1st. If 5 people hand in papers and only the fifth is correct, certainly that person should earn the highest bonus for the round. All solvers should be able to turn in their paper until all bonuses have been claimed by correct papers, not by people saying "done" who are not done.

So, if rank bonuses must be used, they should be adjusted to whoever actually is first. However, the best bonus system will be based on time of finish, and a time-bonus should be strongly encouraged to organizers over other systems.

Playoffs:
For many years the WPC occurred without a playoff and always crowned a person that the puzzle community would truly consider the champion. Since the introduction of playoffs, a measure of the excitement of sport has been added to the mix by allowing a reordering to occur on as few as one puzzle. Whether the WPF considers sudoku more like chess or more like football, it should agree on some standards that ensure the best sudoku solvers have a fair shot at winning. These standards should include the number of competitors who are admitted, and any bonuses that should be expected for order of solve such as a higher seed or better tie-break bonus.

Playoffs should never be run to encourage people to stop solving puzzles. At Zilina, in both round 2 (to qualify for the “Mini Playoff”) and in the Semifinal, the structure of the ranking encouraged solvers to stop solving the puzzles to post an earlier “time” for an incomplete round. 10-12 people said finished in Round 2 and 10-12 people said finished in the Semifinal and this gave the impression to any watching that someone had finished those rounds. In reality, no one finished correctly in either round. As mentioned above, in a situation where sudoku allows for partial scoring, the progress towards a solution should become the tiebreaker (if not that, then the ranking on the first day, which seems to be rewarded at a WPC but not at a WSC in the time I've competed at both) and solvers should not ever stop solving until time runs out. With 13 solvers in Zilina at 3 out of 4 puzzles solved in the semifinals, and with the solvers reaching that state at different times (I had 16 minutes left when I had just the 4th puzzle to go which unfortunately had that "human-solvable?" guessing property to it), different solvers adopted a different solve or quit strategy. The selection of places 4, 9, 11, 28, 29, 30, 33, 36 as the finalists was a result of both what three puzzles solvers had finished, and when solvers got their third puzzle solved and anyone who had 5 or fewer minutes at this point said finished and moved on to the detriment of the good solvers who were still trying to solve a fourth puzzle that they actually had the time to attempt.

Concluding Thoughts:
I’m sure that there are other particular issues that should be addressed, and that there are other problems from past World Sudoku Championships that should be mentioned that I have forgotten, but the steps of standardizing and checking the classic sudoku puzzles used in the championship to ensure human-solvability, scoring/ranking that is based specifically on sudoku’s ability for partial scoring, and fair revelation of the rules of competition to all competitors well before the finals themselves are to me not just suggestions but requirements before I will really feel like competing at a World Sudoku Championship again. While I do not know if I will compete or construct at the World Sudoku Championship in Philadelphia in 2010, I hope standards are adopted for the WSC to ensure a fair and fitting competition on sudoku for WSCs there and in the future. I am making this an open letter, and posting it on the World Puzzle forum as well as my blog, to encourage further discussion of the standards that should be applied to a sudoku competition from the community of solvers.
Tags: competition, sudoku, wsc
Subscribe

Recent Posts from This Journal

  • WPC Playoops

    So a tremendous WPC has ended. I simply wish my performance in the end lived up to my own expectations in the way that the quality of the puzzles and…

  • WPC Update

    I'll limit myself to about 250 words. First day is done, and team USA got off to a quick start in the first team round with the fastest solve (in 28…

  • USPC Preview

    After 9 straight years of competing (and 6 titles), this is my first year constructing for the USPC with 4 contributions from me. As such, I will…

  • Post a new comment

    Error

    default userpic

    Your reply will be screened

    When you submit the form an invisible reCAPTCHA check will be performed.
    You must follow the Privacy Policy and Google Terms of use.
  • 18 comments

Recent Posts from This Journal

  • WPC Playoops

    So a tremendous WPC has ended. I simply wish my performance in the end lived up to my own expectations in the way that the quality of the puzzles and…

  • WPC Update

    I'll limit myself to about 250 words. First day is done, and team USA got off to a quick start in the first team round with the fastest solve (in 28…

  • USPC Preview

    After 9 straight years of competing (and 6 titles), this is my first year constructing for the USPC with 4 contributions from me. As such, I will…