Log in

No account? Create an account
21 January 2013 @ 02:40 pm
Too Big to Solve?  
Not my tagline, but a good description for the Mystery Hunt that just happened. One line of dialogue after last year's Hunt that I led with in my wrap-up was a question of when is too soon for a Hunt to end. I said, in this era of a few competitive teams trying to grow to get over the winning hurdle, constructors aiming bigger was a mistake. The Hunt ending after 36 hours (Midnight Saturday) is fine if that makes the solving experience stretch over the weekend for everyone else. I won't comment generally on this year's effort but it seems a great example to point back to of too much ambition by too many people towards the further militarization of the size of Hunt so that by 2025 the team "The whole of new USA" can go after the coin against "USSReunited" for at least a month. The sense of "puzzle" versus "grindy work" is also a discussion I have every year and I don't choose to repeat myself. I've felt since 2008 that the Mystery Hunt is far from an event I'd regularly attend in person although I'm glad to have finally been onsite to play with Team Luck with whom I've been a "free agent" now for three years.

I had a good solving year as things go relatively, but it was mostly demoralizing personally. I soloed Palmer's Portals, for example, but spent many hours after basically solving 8/10ths with a need to tweak a very small and underconstrained set of things to get from that hard work state to a finished state. At some stage I told the team "I'm going to solve Portals and the Feynman meta and then go sleep" and I met this goal but in many times the expected time when I gave the statement. I led the solve of both Danny Ocean (with zebraboy stating the most necessary last bit to get my work over the cliff) and Richard Feynman (with Jasters). I obviously co-solved lots of the logic puzzles and other puzzles, and gave various finishing help to a range of things too. I think I did this best for "Kid Crossword" once when he had spent a lot of timing mastering the hard steps of a crossword/scrabble puzzle -- and could quite impressively fast rewrite out the set of steps I wanted him to do about the puzzle -- and the follow-up steps were not obvious but I led the killing of the beast. This was too often the feel for these puzzles, and my assassination rate was far lower than I wanted. My Sunday was spent earning 3 puzzle answers by actually going to an event, and then falsely believing the power to buy some answers would let me finish solving the Indiana Jones mini-metas -- where I had already mostly soloed Adventure 2's snakes with 5/8 answers, but then killed myself dead on #1/Ouroboros for the rest of the day for so long solving, as many solvers will say in hindsight, the puzzle that was meant to be in one of a dozen ways and not the puzzle it was. Let me state here as I did for hours with my team, the phrase "I'm not cut out for this" is horrible flavor. It implies both cut this out and, in a different way, also don't cut this out. This makes you want to cut it out, which takes a lot of time, but also to not invest too much time in cutting it out, so as to save the wasted time of doing a task you are being told not to do. Other wordings are far safer, and implied negatives within positives is one of the five worst flavor failure modes in my opinion. Puzzle editing and flavor text is an art and is certainly the biggest variable from year to year and constructing team to constructing team.

So yeah, Mystery Hunt happened. And there were the usual share of overwhelmingly incredible Aha moments. Endgame seemed very fun and I wish all teams could do just that for the weekend or at least a lot more things like that. More of that, and more sleep, would have both been some good choices this year. If only the puzzles solved on schedule.

ETA: And as I added far below around comment #300, as a solver who was both frustrated yet had fun in this Hunt, I do want to thank everyone on Sages for the incredible effort they put in. Making a Mystery Hunt is a gift for all solvers whether it matches expectations or not, and as a mostly thankless job I do want the constructors and editors and software engineers and graphic designers and cooks and phone center workers and everyone else to know I appreciated all you did over the last weekend to give us several days together for puzzling.

Further, as I was asked to write a larger piece elsewhere that has given me personally a lot more attention as the face of the criticism, and as I use the phrase "My team" a lot in general as solving forms this kind of bond, I want to be very clear: since Bombers broke up after 2009 I have been a free agent. I have solved recently with Team Luck but am not a core part of their leadership and these opinions I state are my own. I intend to form my own team next year to go after the coin again, and if you have a problem with what I have said anywhere on the internets, please hate me for it. I believe in my posts I have been offering constructive criticism, but even what I have said is without all the facts of what went on inside Sages so I could easily be speaking from ignorance a lot of the time.

EFTA: Thanks to tablesaw for pointing out this chronologic feature of posts. If you want to see all the additions to this post in time sorted order, go here http://motris.livejournal.com/181790.html?view=flat. We're on page 14 at the moment.
Andrewbrokenwndw on January 23rd, 2013 04:47 am (UTC)
In reply to myself: Noah Snyder makes a truly prescient comment in that post, the very first one.

I'd also like to say (especially since people from sages might be reading) that first time hunt writers *should* be writing short hunts. I think 107 puzzles was already ambitious for a first time team (though certainly pulled off brilliantly). There's a great tradition (Setec00, Plant06, Midnight07, Luck10, Codex12) of short clean "warmup" hunts before you really go nuts for your next hunt. It's the right way to run a first time hunt, and you really shouldn't feel bad *at all* about ending on the early side.
ze top blurberry: driftingztbb on January 23rd, 2013 06:38 am (UTC)
This. What makes me saddest about what we saw this weekend is that the mistakes Sages made were mostly well-known pitfalls that anyone competent giving them construction advice would have advised them to avoid. I don't know if they didn't ask for advice, or didn't listen, but....

I will say that I thought the length of the individual puzzles was a bigger problem than the number of puzzles. A puzzle has to be truly special for me to want to spend 6+ hours on it. The sheer volume of puzzles with tons of work to do -- 263 (!!) audio clips to listen to, 50 MIT location photos, etc, etc -- made this hunt unpleasant for me. If you're going to ask solvers to spend six hours doing work on a puzzle, that work had better be intellectually challenging in some way, because we're there to spend our weekend being intellectually challenged. (A well-constructed logic puzzle can obviously fill that role, and I'm looking forward to trying Portals at some point.)

Once a solver has their aha! moment on a puzzle, they should be able to count on eventually being rewarded with an answer, too many of the puzzles in this hunt did not do that for us -- either because the work after the aha! was too long and unrewarding for the solver to want to do, or the extraction didn't come out for us after hours of work, or hours of work and a successful extraction only led to a second layer of the puzzle rather than to an answer. At a certain point I stopped being interested in picking up new puzzles, because my trust in the constructor had been broken -- every puzzle seemed like a major time commitment that I could not expect would be worth the investment.

Only a very small number of teams come equipped with a hive mind that can tear through scutwork the way that Sages can.

A hunt with 170 beautiful puzzles that provided rewarding solve after rewarding solve and ran into Monday morning would have been just fine; this hunt was too much chaff and not enough wheat. I realize here that I'm being a bit harsh about something that a lot of good friends of mine (including some with whom I work very closely) spent the better part of a year working on, and I feel bad about that. But I think these things have to be said.
Andrewbrokenwndw on January 23rd, 2013 06:43 am (UTC)
I really don't want to get into recriminations. In short, yes, they had advice from us and even at least one member of Plant, and it's more or less exactly the advice you can see in that LJ thread. It is not necessarily the case that they ignored this advice; lots of things go wrong when writing a hunt, including things having to do with balancing what your team is like with what the hunt needs.
ze top blurberry: driftingztbb on January 23rd, 2013 06:59 am (UTC)
I am pretty sure I gave them some of the same advice, at least informally. Honestly I wouldn't have expected the advice even to be necessary. I've solved excellent puzzles written by many of the same people on previous occasions. A good number of them have written hunts before (including Derek, who was on Setec when we wrote Normalville). I know they know these principles. I can't begin to guess why or how they went out the window, and I agree with you that it's pointless to speculate here. I expected that this hunt would significantly surpass the community's expectations.
MellowMelonMellowMelon [wordpress.com] on January 23rd, 2013 08:43 pm (UTC)
The advice we received was not uniform.

I don't want to say much more than this since I feel like I'm already overstepping my bounds, but some internal discussions early in the hunt writing process suggested we were actually under a lot of external pressure to write a hunt that would run longer than 2011 and 2012 (with the implication that these hunts were not satisfactory). I have no clue myself what groups or individuals were doing this. But there is no doubt that "36 hours is not okay" was very clearly inflicted on a lot of our leadership.
motrismotris on January 23rd, 2013 08:51 pm (UTC)
That that kind of advice would unfortunately come from 100-member teams that want to Hunt till Sunday is the problem I wrote about last year. When some members of Luck complained last year the Hunt was short I chided them to say it is perfect for it to end for such a stacked super-solver team so that average teams can also get through most of the content before the end of the weekend. I think Saturday midnight should always be the target, and having 10 teams make runaround too. This may not be "tradition", but this is pareto optimized for the constructors, solvers, and mid-range team attitudes. Once the coin is found, hinting more readily is easy and solvers can Hunt till mid-day Sunday comfortably with wrap-up beating the flights back home.
(no subject) - noahspuzzlelj on January 23rd, 2013 09:00 pm (UTC) (Expand)
(Deleted comment)
(no subject) - zotmeister on January 23rd, 2013 09:34 pm (UTC) (Expand)
(no subject) - cmouse on January 24th, 2013 01:09 am (UTC) (Expand)
(no subject) - lunchboy on January 24th, 2013 01:59 am (UTC) (Expand)
noahspuzzlelj on January 23rd, 2013 08:54 pm (UTC)
I suspect a lot of that pressure was internal, as my understanding is that the only people we got that kind of feedback from were on Sages.
(no subject) - (Anonymous) on January 23rd, 2013 09:21 pm (UTC) (Expand)
(no subject) - motris on January 23rd, 2013 09:30 pm (UTC) (Expand)
(no subject) - (Anonymous) on January 23rd, 2013 10:14 pm (UTC) (Expand)
(no subject) - motris on January 23rd, 2013 10:20 pm (UTC) (Expand)
(no subject) - (Anonymous) on January 24th, 2013 04:20 am (UTC) (Expand)
(no subject) - motris on January 24th, 2013 04:34 am (UTC) (Expand)
(no subject) - (Anonymous) on January 24th, 2013 11:39 pm (UTC) (Expand)
Mayhem - (Anonymous) on January 24th, 2013 01:23 am (UTC) (Expand)
Re: Mayhem - motris on January 24th, 2013 01:26 am (UTC) (Expand)
(no subject) - canadianpuzzler on January 24th, 2013 02:14 am (UTC) (Expand)
Re: Mayhem - dr_whom on January 24th, 2013 02:11 am (UTC) (Expand)
(no subject) - (Anonymous) on January 25th, 2013 05:35 am (UTC) (Expand)
Dr. C. Scott Ananiancananian on January 23rd, 2013 10:04 pm (UTC)
Note that, for whatever reason, Sages lengthened the hunt in at least *three* different ways, all of which combined poorly:

a) more individual puzzles. (30% more?)
b) metas which were not backsolvable, and required close to 100% of their inputs to solve (some individual puzzles also seemed to deliberately require 100% identification, etc.)
c) significantly more aha steps per puzzle. (Really, puzzles should have 1 or 2 ahas. NO MORE. Put the other ahas in a new puzzle.)

Granted, Codex deliberately chose to make our metas backsolvable missing 2 or 3 answers (that's how we test solved them) in order to protect the meta structure from a few broken puzzles. We also pretty strictly limited the difficulty of individual puzzles -- I personally chided authors several times that, "you want to write a fun puzzle that people *solve* -- no one will think you're super-clever if they don't solve your puzzle, they'll just think you wrote a bad puzzle."

As a result we wrote a hunt which many teams solved, and many teams had fun solving. In retrospect, we probably could have either made our metas less backsolvable and/or written more puzzles (with a higher chance of having some be broken), and ended up with a slightly longer hunt. But clearly increasing all three axes of "hunt length" simultaneously was the wrong response to the 2012 hunt.
(no subject) - dr_whom on January 24th, 2013 12:29 am (UTC) (Expand)
(no subject) - coendou on January 24th, 2013 01:16 am (UTC) (Expand)
(Deleted comment)
(no subject) - davidglasser on January 24th, 2013 03:04 pm (UTC) (Expand)
(no subject) - foggyb on January 24th, 2013 03:30 pm (UTC) (Expand)
(no subject) - rlangmit on January 24th, 2013 06:30 pm (UTC) (Expand)
(no subject) - ztbb on January 24th, 2013 09:37 pm (UTC) (Expand)
(no subject) - noahspuzzlelj on January 25th, 2013 03:58 am (UTC) (Expand)
(no subject) - (Anonymous) on January 26th, 2013 09:31 pm (UTC) (Expand)
Meta backsolvability - pesto17 on January 28th, 2013 03:57 pm (UTC) (Expand)
(no subject) - lunchboy on January 23rd, 2013 10:20 pm (UTC) (Expand)
screwdestinyscrewdestiny on January 23rd, 2013 09:40 pm (UTC)
I definitely agree with the length of individual puzzles being too long. This is my first time doing Mystery Hunt (I was with Team Palindrome) and while the variety of puzzles was great, a lot of them involved what felt like "busy work." I'm the novice of novices, so I'm not sure if that's normal. Even reading the solution for Bottom of the Top (which we were on the right track for precisely because of the title hint, but didn't have time to do) makes me wonder why exactly Manic Sages didn't realize logistically their hunt was too long:

"The title of the puzzle is intended to help solvers narrow down the list of potential songs to listen to. All songs in this puzzle were #1 hits on the Billboard Hot 100
during the 50 years of that list's existence, narrowing down the search space to only about 1000 songs. :-) On Saturday night of the Hunt, before this puzzle had
been released to anyone, it became clear that another very long puzzle was... um... unnecessary, so I added a list of years that each of the songs could be found in,
which further narrowed the search. Another helpful feature of the puzzle is that the songs were given in alphabetical order by title"

It really took until Saturday night to realize we didn't need a puzzle where after you'd gotten the gist, you had to listen to songs where every 1 out of 33 was one that you needed? It's not a puzzle at that point anymore, it's me listening to songs on Rhapsody, failing to adequately distinguish the bassline, and entertaining the notion of downloading them and running them through Audacity before deciding my efforts are better spent elsewhere.

The traveling puzzles were also big time commitments. With Highlight Reel, at least all the doors were inside and on campus. But the Cambridge Waldo and Square Routes puzzles, which required you to either know Cambridge like the back of your hand or travel around from the locations that you did know and hoped you spotted the ones you didn't on the way (in the dark and/or cold, with a phone or laptop beside you) were logistical nightmares for our teams and while we made it a good faith effort to Google Map them (although I'm pretty sure the square dancing puzzle asked you to record things you couldn't see without actually being there), we'd pretty much decided it was just Too Much.

I expected to be super frustrated yet having a great time and while that was COMPLETELY TRUE (no jokes, I have so many wonderful memories), things like the above just stood out as "...really, though?"
lunchboylunchboy on January 23rd, 2013 07:03 am (UTC)
I still think the Producers hunt was pitched at the correct difficulty level overall. It may have ended too early from Sages' viewpoint, but they were five hours ahead of their nearest competition, and their status as an outlier had to do with their overwhelming numbers (which we didn't have control over). I think it's a mistake to write with the idea in mind of "let's write this so the fastest team finishes it at the exact moment we hope the Hunt will end" -- and it seems that's what Sages did this year, and, of course, then there was the problem that the team best suited to solve this Hunt in any sort of reasonable time frame was the only one not solving, because they were running the Hunt.
Andrewbrokenwndw on January 23rd, 2013 07:33 am (UTC)
Eh. I actually think they're a lot like Codex in terms of solving ability-- especially in terms of breadth vs. depth-- and thus not well positioned to handle a harder hunt. If anything they succeeded last time exactly because we gave them a lot of very thoroughly vetted, comparatively easy puzzles, letting them best apply their large amount of less Dankatzian labor.

Edit: And lest anyone think I mean this as a putdown, I definitely put myself in the non-Dankatzian camp. I'm just saying that I think the hard hunts are best for the small, concentrated teams.

Edited at 2013-01-23 07:36 am (UTC)
Dan KatzDan Katz on January 23rd, 2013 10:01 pm (UTC)
I am flattered to be an adjective.
Dan KatzDan Katz on January 23rd, 2013 10:00 pm (UTC)
We were the second team to finish Producers, which means it was shorter for us than it was for any other team except Sages, and I did not feel short-changed by the Hunt length. As Thomas said, I would rather have the best teams finish early and have more teams get to see the entire Hunt before the time-limit cutoff.
noahspuzzlelj on January 23rd, 2013 10:33 pm (UTC)
If enough people want hunt to go longer into Sunday, it'd be totally reasonable to experiment with a later announced time-limit cutoff. Personally, I think early Sunday afternoon is right (so that wrapup can be Sunday before dinner), but if a team wants to say in November (before plane tickets are bought) "We'll stay open until Midnight on Sunday and have wrap-up Monday morning" that'd be fine too. (Though there is a trade-off there because people who work on Monday will miss wrap-up and any other post hunt gatherings.)
Dr. C. Scott Ananiancananian on January 24th, 2013 04:14 am (UTC)
For what it's worth, codex got feedback (which we ignored) that the 2011 wrapup at 5pm was too late, because people had flights on Sunday they had to make. There were vigorous objections to the idea of a Monday wrapup.

You get a lot of contradictory feedback. Your job as hunt authors is to ignore the nonsense.