Log in

21 January 2013 @ 02:40 pm
Too Big to Solve?  
Not my tagline, but a good description for the Mystery Hunt that just happened. One line of dialogue after last year's Hunt that I led with in my wrap-up was a question of when is too soon for a Hunt to end. I said, in this era of a few competitive teams trying to grow to get over the winning hurdle, constructors aiming bigger was a mistake. The Hunt ending after 36 hours (Midnight Saturday) is fine if that makes the solving experience stretch over the weekend for everyone else. I won't comment generally on this year's effort but it seems a great example to point back to of too much ambition by too many people towards the further militarization of the size of Hunt so that by 2025 the team "The whole of new USA" can go after the coin against "USSReunited" for at least a month. The sense of "puzzle" versus "grindy work" is also a discussion I have every year and I don't choose to repeat myself. I've felt since 2008 that the Mystery Hunt is far from an event I'd regularly attend in person although I'm glad to have finally been onsite to play with Team Luck with whom I've been a "free agent" now for three years.

I had a good solving year as things go relatively, but it was mostly demoralizing personally. I soloed Palmer's Portals, for example, but spent many hours after basically solving 8/10ths with a need to tweak a very small and underconstrained set of things to get from that hard work state to a finished state. At some stage I told the team "I'm going to solve Portals and the Feynman meta and then go sleep" and I met this goal but in many times the expected time when I gave the statement. I led the solve of both Danny Ocean (with zebraboy stating the most necessary last bit to get my work over the cliff) and Richard Feynman (with Jasters). I obviously co-solved lots of the logic puzzles and other puzzles, and gave various finishing help to a range of things too. I think I did this best for "Kid Crossword" once when he had spent a lot of timing mastering the hard steps of a crossword/scrabble puzzle -- and could quite impressively fast rewrite out the set of steps I wanted him to do about the puzzle -- and the follow-up steps were not obvious but I led the killing of the beast. This was too often the feel for these puzzles, and my assassination rate was far lower than I wanted. My Sunday was spent earning 3 puzzle answers by actually going to an event, and then falsely believing the power to buy some answers would let me finish solving the Indiana Jones mini-metas -- where I had already mostly soloed Adventure 2's snakes with 5/8 answers, but then killed myself dead on #1/Ouroboros for the rest of the day for so long solving, as many solvers will say in hindsight, the puzzle that was meant to be in one of a dozen ways and not the puzzle it was. Let me state here as I did for hours with my team, the phrase "I'm not cut out for this" is horrible flavor. It implies both cut this out and, in a different way, also don't cut this out. This makes you want to cut it out, which takes a lot of time, but also to not invest too much time in cutting it out, so as to save the wasted time of doing a task you are being told not to do. Other wordings are far safer, and implied negatives within positives is one of the five worst flavor failure modes in my opinion. Puzzle editing and flavor text is an art and is certainly the biggest variable from year to year and constructing team to constructing team.

So yeah, Mystery Hunt happened. And there were the usual share of overwhelmingly incredible Aha moments. Endgame seemed very fun and I wish all teams could do just that for the weekend or at least a lot more things like that. More of that, and more sleep, would have both been some good choices this year. If only the puzzles solved on schedule.

ETA: And as I added far below around comment #300, as a solver who was both frustrated yet had fun in this Hunt, I do want to thank everyone on Sages for the incredible effort they put in. Making a Mystery Hunt is a gift for all solvers whether it matches expectations or not, and as a mostly thankless job I do want the constructors and editors and software engineers and graphic designers and cooks and phone center workers and everyone else to know I appreciated all you did over the last weekend to give us several days together for puzzling.

Further, as I was asked to write a larger piece elsewhere that has given me personally a lot more attention as the face of the criticism, and as I use the phrase "My team" a lot in general as solving forms this kind of bond, I want to be very clear: since Bombers broke up after 2009 I have been a free agent. I have solved recently with Team Luck but am not a core part of their leadership and these opinions I state are my own. I intend to form my own team next year to go after the coin again, and if you have a problem with what I have said anywhere on the internets, please hate me for it. I believe in my posts I have been offering constructive criticism, but even what I have said is without all the facts of what went on inside Sages so I could easily be speaking from ignorance a lot of the time.

EFTA: Thanks to tablesaw for pointing out this chronologic feature of posts. If you want to see all the additions to this post in time sorted order, go here http://motris.livejournal.com/181790.html?view=flat. We're on page 14 at the moment.
Cody B.: contemplationcodeman38 on January 23rd, 2013 03:08 pm (UTC)
Also, double-check after the fact that the answers that you identify as wrong are in fact wrong.

I'm not entirely sure how this incident played out on Sages' end, but there was one puzzle where Codex had managed to extract the answer "PLANAR" and called it in. We got the callback and were told it was wrong, so we spent hours trying to figure out what other ways we might be able to extract an answer. (In what seemed to be another common occurrence during this hunt, there was barely anything in the puzzle to confirm that our extraction process was the intended one, so this seemed like a perfectly reasonable idea at the time.)

As it turned out, those hours were completely wasted, because the answer was in fact "PLANAR". Which we only found out by calling in the same answer a second time.
David Glasserdavidglasser on January 23rd, 2013 05:49 pm (UTC)
In a much more minor note: early in Hunt, I called in a question about the crochet part of Uncharted Territories, and was told that many people had asked the same question but that the puzzle was correct as written. (The mistake was that it told you to do three of the four sides of the outside of the square and then connect immediately back to the start. Given that this was the round that was forming the red diamond, converting it into a weird triangle definitely made the puzzle harder to solve!)

Days later, after solving the puzzle, I discovered that the puzzle had been updated to be correct (asking you to crochet all four sides).

So there was a puzzle error, and people called it in, and it later got fixed. No big deal ... except that as far as I know, this was never announced as errata! Or maybe it was and it got lost in the email problem. Who knows.

To make this constructive criticism rather than whining: I highly suggest that Hunt organizers build a spot for errata into their software, so that it's easy to show all errata (and globally-available hints) on the website rather than relying on email.
motrismotris on January 23rd, 2013 05:53 pm (UTC)
This has certainly existed in the past, if memory serves me right. But that it did not exist this year, particularly given the fallibility of email, is an obvious weakness to address. Thanks for the constructive comment!
noahspuzzlelj on January 23rd, 2013 07:18 pm (UTC)
There were a number of usability features from the past few years which this year's website did not have. Easy access to the answer to puzzles that you already solved, access to wrong answers you'd called in, easy ways to navigate the site (e.g. a link to the main page), an errata page, and printable versions of the puzzles (page breaks and crosswords were particularly problematic).

Obviously I understand that Sages were a bit behind and so they didn't have time to implement all of these (and apparently some were implemented during hunt). But one of the tradeoffs of writing a very long hunt is that you don't have as much time for other things.
Jenny G: Crazy Puzzlehahathor on January 23rd, 2013 09:00 pm (UTC)
Yes - printing was so problematic that at Palindrome we tried to avoid printing, just recreating grids and lists of clues on spreadsheets. This had the advantage of making it easier for remote solvers to join in, but did take time and make things difficult when pieces of paper were meant to be manipulated (e.g., cut or folded).
Dr. C. Scott Ananiancananian on January 23rd, 2013 09:54 pm (UTC)
This makes me feel like all the time I spent with CSS stylesheets for the Borbonicus and Bodley hunt to make all of our puzzles printable (and paginated at proper places) was time well spent. I don't recall anyone mentioning anything about it at the time. But it certainly took a lot of work, and our team (for one) thought it was important to do. (If only I could show you how many internal filed bugs I had to deal with of the form "puzzle XYZ doesn't print correctly on [obscure browser PDQ]".)

I believe you'll find that all of our solutions (which were posted before wrapup, another recent "tradition" I was sad to find mislaid this year) are also printable, although that was one area we started cutting corners in the last few weeks before hunt.
THANK YOU - temvald on January 23rd, 2013 10:08 pm (UTC) (Expand)
(no subject) - hahathor on January 24th, 2013 02:08 am (UTC) (Expand)
(no subject) - jadelennox on January 24th, 2013 03:57 pm (UTC) (Expand)
devjoedevjoe on January 23rd, 2013 11:52 pm (UTC)
At the beginning of the hunt, the Status thing on the right was not only printing, but printing at the top of every page, sometimes covering parts of puzzles that ran for more than one page. This was fixed pretty quickly, to get styled so that Status did not print. We didn't have a whole lot of problems with printing after that point, but there were certain puzzles that had problems, like grids coded in HTML tables with shaded cells as cell backgrounds that did not print.
(no subject) - devjoe on January 24th, 2013 12:54 am (UTC) (Expand)
(no subject) - cananian on January 24th, 2013 04:00 am (UTC) (Expand)
(no subject) - dougo on January 24th, 2013 04:36 am (UTC) (Expand)
 Catherinecmouse on January 24th, 2013 01:14 am (UTC)
We had a lot of people who were not serious puzzle authors. Writing fewer puzzles had no effect on tech, those were separate issues.
Andrewbrokenwndw on January 24th, 2013 02:18 am (UTC)
Good on you-- one of our big issues was that novalis was both one of my more reliable editors and co-architect of the live hunt software, and while he was very good at both, there are only so many hours in the day. If I'd been less strapped for editorial power I would probably have kicked him off all of his puzzles so he could get more rest!
Doug Orleansdougo on January 24th, 2013 04:33 am (UTC)
The "Puzzle History" section (on the "Call In Answer" page) started working sometime on Saturday. This included all called in answers, including the correct answer. I would have liked timestamps too, but this was pretty minor.

There was a "Home" link at the top of the "Status" pop-down menu; I don't remember this not being there from the beginning, but maybe that was also added later?
Adrianywalme on January 23rd, 2013 08:38 pm (UTC)
I want to apologize about Uncharted Territories. I was almost certainly the one who took your call, and I had two people swear up and down to me that the puzzle was fine, only to wake up the next morning to find that an erratum had been emailed out (it may, in fact, have gotten delayed by the outage). I should have checked the puzzle personally, and I didn't. Mea maxima culpa.
(Anonymous) on January 23rd, 2013 11:49 pm (UTC)
same problem
I encountered the same problem with this puzzle, including calling it in and being told that many people had asked the same question, and that it wasn't an error. I even told them that I was 100% sure it didn't work with the pattern as written (there was no way to skip 3 stitches and attach to the original loop), so if it wasn't a clue, they should probably look into it. It took them a whole additional day to fix this. If you're fielding a bunch of calls from people about a pattern and it isn't related to the clue (as in, the clues weren't in the pattern text but in the result of the pattern)you should really look into whether or not there is an error in your pattern. Particularly because people who can successfully make it through 4 rounds of a granny square aren't going to get tripped up by the 5th round instructions. You should also make sure the teams that have been banging their head against the wall with the puzzle are told that you have updated the pattern to correct the error.
 Catherinecmouse on January 24th, 2013 12:57 am (UTC)
Things like that have to happen every hunt and I know a couple happened here. People are people and individuals in the call queue make mistakes.
motrismotris on January 24th, 2013 12:59 am (UTC)
But that the system -- which should by now be approaching an automated system even if a call out announcement -- logged in PLANAR and that was not flagged as correct is really inexcusable.

One model Microsoft and other Hunts has used is that the first two to three answers on a puzzle are auto-submits with auto-response, no calls either way. This would be fine. Multiple incorrect answers requires a request to HQ and call to unlock answer entry again. I'd like to see this happen sooner rather than later as an advance for MIT as the delay to know an answer is right got high at times, and mistakes should be eliminated, not accepted as a state of nature with any human system.

Edited at 2013-01-24 01:00 am (UTC)
 Catherinecmouse on January 24th, 2013 01:18 am (UTC)
It was flagged as "Correct". Software can't stop a sleepy person from picking up the phone, reading that, and telling you your answer is incorrect.
Dr. C. Scott Ananiancananian on January 24th, 2013 04:05 am (UTC)
But software can stop us from repeating to call in answers for a puzzle which has been marked correct.

Software can also display that the answer is correct on the call-in screen (all correct answers were just listed as "proposed answer").

I disagree with completely automating the system, I think the human contact is important (and fun). There are known ways of making this work well and insulating it from human error... those mechanisms were not implemented in 2013 -- probably the hunt-running team ran out of time?
 Catherinecmouse on January 24th, 2013 04:07 am (UTC)
The software folks certainly ran out of time. ;)
(no subject) - cananian on January 24th, 2013 04:52 am (UTC) (Expand)
(no subject) - brokenwndw on January 24th, 2013 07:52 am (UTC) (Expand)
Jessicajessicala on January 27th, 2013 03:29 am (UTC)
Speaking as someone who's hosted a number of Microsoft events using auto-response software: there's still plenty of human contact happening. During PuzzleHunt 11.0 I remember spending a lot of time in personal contact (both phone and email) with members of various teams.

Having the software pick up a lot of the auto-answering gave us (hosts I mean) more time for one-on-one interaction, to help when midrange teams needed a nudge or some guidance to help them keep having fun.
(Anonymous) on January 25th, 2013 01:38 am (UTC)
Can't stop it, but can reduce the odds. EG Text displayed on a giant green field when calling back a right answer; text displayed on a giant red field for a wrong answer.

But it seems like every year the Hunt running software is written at the last possible minute. In 2011, several of us were sitting around at 10 AM Friday with laptops "submitting" answers in order to simulate a rapidly sped up Hunt (Setec won in 20 minutes, IIRC).

David S.
Alisonlandofnowhere on January 25th, 2013 04:02 am (UTC)
We had green and white (though the colors were not that bold). I didn't actually get to respond to the answer queue, so I don't know how well it worked.

--Alison from Sages
Hooligan: neuron artjedusor on January 26th, 2013 09:54 am (UTC)
Setec won

Now I know you're making things up. ;)
(no subject) - ztbb on January 26th, 2013 04:53 pm (UTC) (Expand)
AJDdr_whom on January 24th, 2013 02:07 am (UTC)
So certainly for the Mario Hunt, and perhaps even as far back as SPIES, Plant was talking about phone-free auto-response for answer confirmation, and one of the reasons we decided not to implement that was because we thought people value the answer-confirmation calls as a point of human contact with the running team, so you don't feel quite so much like you're in a hermetically-sealed box for the duration of the Hunt. Do you think this is a valid concern?
motrismotris on January 24th, 2013 02:18 am (UTC)
Depends how some solvers use the thing. I never took the calls for the puzzles I answered. So I basically generated work by sending everything through the team line.

On the running side, as a smaller team I hated losing so much man power to the call lines and would prefer more frequent visits to every team HQ.

Opinions will be very broad on this point though and each Hunt team should encourage some amount of interaction with each team however that is achieved.
rlangmit on January 24th, 2013 03:18 am (UTC)
I think we at Up Late enjoy the calling mechanism as it is right now. It's nice to (a) have to think a bit about submitting something (i.e., not just play Funny Farm), (b) have some anticipation before finding out if it's correct, and (c) have some human interaction. For example, we got a lot of amusement responding "Oh, you don't say?" or something else pseudo-sarcastic when Sages would call back to confirm an answer they had just sold us. And for real solves, it's nice to hear that congratulations from an actual human voice.

I don't think there were any huge delays in the call queue--at least for us--so I don't think this is a huge problem. The devolution of the software was a problem.
Dr. C. Scott Ananiancananian on January 24th, 2013 04:06 am (UTC)
Incidentally: https://github.com/mysteryhunt contains all the software we wrote for the 2012 hunt. The goal was for this to be a resource for future hunts.