Log in

No account? Create an account
21 January 2013 @ 02:40 pm
Too Big to Solve?  
Not my tagline, but a good description for the Mystery Hunt that just happened. One line of dialogue after last year's Hunt that I led with in my wrap-up was a question of when is too soon for a Hunt to end. I said, in this era of a few competitive teams trying to grow to get over the winning hurdle, constructors aiming bigger was a mistake. The Hunt ending after 36 hours (Midnight Saturday) is fine if that makes the solving experience stretch over the weekend for everyone else. I won't comment generally on this year's effort but it seems a great example to point back to of too much ambition by too many people towards the further militarization of the size of Hunt so that by 2025 the team "The whole of new USA" can go after the coin against "USSReunited" for at least a month. The sense of "puzzle" versus "grindy work" is also a discussion I have every year and I don't choose to repeat myself. I've felt since 2008 that the Mystery Hunt is far from an event I'd regularly attend in person although I'm glad to have finally been onsite to play with Team Luck with whom I've been a "free agent" now for three years.

I had a good solving year as things go relatively, but it was mostly demoralizing personally. I soloed Palmer's Portals, for example, but spent many hours after basically solving 8/10ths with a need to tweak a very small and underconstrained set of things to get from that hard work state to a finished state. At some stage I told the team "I'm going to solve Portals and the Feynman meta and then go sleep" and I met this goal but in many times the expected time when I gave the statement. I led the solve of both Danny Ocean (with zebraboy stating the most necessary last bit to get my work over the cliff) and Richard Feynman (with Jasters). I obviously co-solved lots of the logic puzzles and other puzzles, and gave various finishing help to a range of things too. I think I did this best for "Kid Crossword" once when he had spent a lot of timing mastering the hard steps of a crossword/scrabble puzzle -- and could quite impressively fast rewrite out the set of steps I wanted him to do about the puzzle -- and the follow-up steps were not obvious but I led the killing of the beast. This was too often the feel for these puzzles, and my assassination rate was far lower than I wanted. My Sunday was spent earning 3 puzzle answers by actually going to an event, and then falsely believing the power to buy some answers would let me finish solving the Indiana Jones mini-metas -- where I had already mostly soloed Adventure 2's snakes with 5/8 answers, but then killed myself dead on #1/Ouroboros for the rest of the day for so long solving, as many solvers will say in hindsight, the puzzle that was meant to be in one of a dozen ways and not the puzzle it was. Let me state here as I did for hours with my team, the phrase "I'm not cut out for this" is horrible flavor. It implies both cut this out and, in a different way, also don't cut this out. This makes you want to cut it out, which takes a lot of time, but also to not invest too much time in cutting it out, so as to save the wasted time of doing a task you are being told not to do. Other wordings are far safer, and implied negatives within positives is one of the five worst flavor failure modes in my opinion. Puzzle editing and flavor text is an art and is certainly the biggest variable from year to year and constructing team to constructing team.

So yeah, Mystery Hunt happened. And there were the usual share of overwhelmingly incredible Aha moments. Endgame seemed very fun and I wish all teams could do just that for the weekend or at least a lot more things like that. More of that, and more sleep, would have both been some good choices this year. If only the puzzles solved on schedule.

ETA: And as I added far below around comment #300, as a solver who was both frustrated yet had fun in this Hunt, I do want to thank everyone on Sages for the incredible effort they put in. Making a Mystery Hunt is a gift for all solvers whether it matches expectations or not, and as a mostly thankless job I do want the constructors and editors and software engineers and graphic designers and cooks and phone center workers and everyone else to know I appreciated all you did over the last weekend to give us several days together for puzzling.

Further, as I was asked to write a larger piece elsewhere that has given me personally a lot more attention as the face of the criticism, and as I use the phrase "My team" a lot in general as solving forms this kind of bond, I want to be very clear: since Bombers broke up after 2009 I have been a free agent. I have solved recently with Team Luck but am not a core part of their leadership and these opinions I state are my own. I intend to form my own team next year to go after the coin again, and if you have a problem with what I have said anywhere on the internets, please hate me for it. I believe in my posts I have been offering constructive criticism, but even what I have said is without all the facts of what went on inside Sages so I could easily be speaking from ignorance a lot of the time.

EFTA: Thanks to tablesaw for pointing out this chronologic feature of posts. If you want to see all the additions to this post in time sorted order, go here http://motris.livejournal.com/181790.html?view=flat. We're on page 14 at the moment.
MellowMelonMellowMelon [wordpress.com] on January 23rd, 2013 08:43 pm (UTC)
The advice we received was not uniform.

I don't want to say much more than this since I feel like I'm already overstepping my bounds, but some internal discussions early in the hunt writing process suggested we were actually under a lot of external pressure to write a hunt that would run longer than 2011 and 2012 (with the implication that these hunts were not satisfactory). I have no clue myself what groups or individuals were doing this. But there is no doubt that "36 hours is not okay" was very clearly inflicted on a lot of our leadership.
motrismotris on January 23rd, 2013 08:51 pm (UTC)
That that kind of advice would unfortunately come from 100-member teams that want to Hunt till Sunday is the problem I wrote about last year. When some members of Luck complained last year the Hunt was short I chided them to say it is perfect for it to end for such a stacked super-solver team so that average teams can also get through most of the content before the end of the weekend. I think Saturday midnight should always be the target, and having 10 teams make runaround too. This may not be "tradition", but this is pareto optimized for the constructors, solvers, and mid-range team attitudes. Once the coin is found, hinting more readily is easy and solvers can Hunt till mid-day Sunday comfortably with wrap-up beating the flights back home.
noahspuzzlelj on January 23rd, 2013 09:00 pm (UTC)
Also people on the superfast teams can always go back and solve the puzzles that they didn't have time to look at. Heck, they can get recommendations from their teammates on which puzzles were the most fun and do those ones.
(Deleted comment)
Adam R. Wood: discozotmeister on January 23rd, 2013 09:34 pm (UTC)
Ditto'. [That's a fractal 'ditto', applying to the comment I'm directly replying to and the one it was replying to. It should be read as "ditto-prime", or "ditto-dashu" if you're an otaku.] - ZM
 Catherinecmouse on January 24th, 2013 01:09 am (UTC)
Hunt is running too short fix it was the most common by far piece of feedback we received. I think there's something more complex here - many people want to be able to solve all weekend and still be competitive.

From the feedback we got, many opinions are not represented in this tread and many people want to hunt competitively and hunt until Sunday. I would be heartbroken if the solution to this was for hunt to start ending on Saturday night and judging from the feedback we've consistently received so would a huge contingent of solvers.
lunchboylunchboy on January 24th, 2013 01:59 am (UTC)
Naturally the comments here are going to self-select for "people Thomas knows"; I don't know if your feedback was similarly self-selecting, but considering how many people I've heard opinions from, perhaps many opinions were not represented in your feedback?
(Deleted comment)
noahspuzzlelj on January 23rd, 2013 08:54 pm (UTC)
I suspect a lot of that pressure was internal, as my understanding is that the only people we got that kind of feedback from were on Sages.
(Anonymous) on January 23rd, 2013 09:21 pm (UTC)
I don't have access to your post-Hunt data, so you could well be right. However, judging from conversations I had around my HQ and the e-mails currently sitting in my inbox, people on my team were very happy that the Hunt went on longer than previous years. They mostly vary from (and I'm paraphrasing) "long hunts are awesome" to "I like long hunts, but this should ended Sunday night".

Now, thanks to some poor extrapolation on our part, our team was stupid large this year, so that might effect things but I'm pretty sure we were as sleep deprived as the rest of you. Also, I know there are been more than a little sadness the last few years when the coin was found earlier. Yes, HQ stays open longer these days, and we're thankful for that, but apparently it's just not the same for a decent chunk of our team.

It seems like a lot of teams here disagree with us, but I thought I'd add my two cents.

-Henry, Death from Mayhem
motrismotris on January 23rd, 2013 09:30 pm (UTC)
Well, not only did your team start large, but you merged to become larger during the Hunt. This is the negative reinforcement I choose to fight. I hope Death from Mayhem splits back to two somehow before next year.

Luck similarly got an offer to merge with another team during the Hunt (we turned it down, thinking among other things that it would be bad to either win or come close after doing such a thing).
(Anonymous) on January 23rd, 2013 10:14 pm (UTC)
I suspect you won't believe me but the intent was never to win the hunt. Honestly, we never thought we were higher than third, and that was after Sages came by to tell us that we were the only in contention team that didn't think they were in contention. It's not like it's ever been something we had to worry about before.

Also, why is everyone saying we merged during Hunt? We merged 2 weeks before hunt. At the time that the talks about merging started the Death e-mail list for the year was about 50, including several we suspected would not really be involved. Mayhem's reported number was 20-30. We were expecting maybe 100 which is large, but isn't that bad. By the Monday before Hunt we had 130 and realized we might have a problem, but it was too late by then. And by end of hunt we passed stupidly large.

I don't know why you care to fight this negative reinforcement or whatever it is. We had no particular desire to win, though I guess that may change. If everyone agrees that we had fun hunting together (and it seems to be leaning that way) we'll stay together and have fun. I'm sorry if that bothers you. Whether we win or not, I don't know. I suppose next year we'll have to talk before hand about whether we want the coin.

-Henry, Death

motrismotris on January 23rd, 2013 10:20 pm (UTC)
I was wrong on the timing and apologize. I thought it was mid-Hunt.

We were approached mid-Hunt, by another top three team, which is a real problem. I am very concerned when any team in the top three feels they should merge because they don't think they can finish Hunt. Teams that feel they are solving this bad come back next year with 10+ more solvers. This is the negative death spiral of Hunts of constructable size and Hunts of solvable size.

For example, in 2008, when we won Mystery Mystery, I never had any sense we were in front. But from the solving clock this was exactly the case. The lack of events, or signs of progress elsewhere, puts a team into a highly uncertain and uncomfortable state.

Or maybe, as others say, this is just some aura only my team has and every other team is all smiles with these puzzles.

Edited at 2013-01-23 10:25 pm (UTC)
(no subject) - (Anonymous) on January 24th, 2013 04:20 am (UTC) (Expand)
(no subject) - motris on January 24th, 2013 04:34 am (UTC) (Expand)
(no subject) - (Anonymous) on January 24th, 2013 11:39 pm (UTC) (Expand)
(Anonymous) on January 24th, 2013 01:23 am (UTC)
Mayhem basically had no choice but to merge -- we didn't have current MIT students any more.
Re: Mayhem - motris on January 24th, 2013 01:26 am (UTC) (Expand)
(no subject) - canadianpuzzler on January 24th, 2013 02:14 am (UTC) (Expand)
Re: Mayhem - dr_whom on January 24th, 2013 02:11 am (UTC) (Expand)
(Anonymous) on January 25th, 2013 05:35 am (UTC)
"Also, I know there are been more than a little sadness the last few years when the coin was found earlier. Yes, HQ stays open longer these days, and we're thankful for that, but apparently it's just not the same for a decent chunk of our team."

There's a simple solution: Don't announce when the Hunt is won. If anyone asks for reasons of trying to decide whether to continue or collapse into bed or run for their flights home, tell them, but otherwise, don't kill the thrill of the chase by announcing the end. Teams find out only at wrap-up or possibly when they finish.

--Dart, Luck
Dr. C. Scott Ananiancananian on January 23rd, 2013 10:04 pm (UTC)
Note that, for whatever reason, Sages lengthened the hunt in at least *three* different ways, all of which combined poorly:

a) more individual puzzles. (30% more?)
b) metas which were not backsolvable, and required close to 100% of their inputs to solve (some individual puzzles also seemed to deliberately require 100% identification, etc.)
c) significantly more aha steps per puzzle. (Really, puzzles should have 1 or 2 ahas. NO MORE. Put the other ahas in a new puzzle.)

Granted, Codex deliberately chose to make our metas backsolvable missing 2 or 3 answers (that's how we test solved them) in order to protect the meta structure from a few broken puzzles. We also pretty strictly limited the difficulty of individual puzzles -- I personally chided authors several times that, "you want to write a fun puzzle that people *solve* -- no one will think you're super-clever if they don't solve your puzzle, they'll just think you wrote a bad puzzle."

As a result we wrote a hunt which many teams solved, and many teams had fun solving. In retrospect, we probably could have either made our metas less backsolvable and/or written more puzzles (with a higher chance of having some be broken), and ended up with a slightly longer hunt. But clearly increasing all three axes of "hunt length" simultaneously was the wrong response to the 2012 hunt.
AJDdr_whom on January 24th, 2013 12:29 am (UTC)
One of the purposes of Meta testing! in the Mario hunt was to give teams that had never run the Hunt an idea of what meta testsolving looks like—including the fact that metas are tested missing two or three answers, and the reason for it.

I have no idea how Manic Sages tested their metas for this year, of course, and I didn't work much on solving any this weekend. But if they really were only solvable with close to 100% of the answers, that's kind of bad.
why's everything gotta be so intense with mecoendou on January 24th, 2013 01:16 am (UTC)
Yes, this.

In 2012, most of the top teams averaged 3 puzzles/hr over the course of the hunt (with sages blowing us all away at 4/hr). If teams had gone that rate this year, the hunt would have ended Sunday afternoon - but B and C prevented anyone from approaching that rate. We (Part I: Non-Contradiction Chapter I: The Theme) were only making about 2/hr as of Sunday night.

I will admit that I'm one that hates short hunts. I have solved on teams of every size and level of competitiveness over the past 15 years, and have always hated them. BUT I have only been on a huge, competitive team since the tradition of keeping HQ open til some minimum time on Sunday started in 2010. I still hate short hunts on a competitive team that will finish the hunt early even if we don't win, but I think I'd be fine with them on a non-competitive team like 80-90% of hunt teams.
(Deleted comment)
David Glasserdavidglasser on January 24th, 2013 03:04 pm (UTC)
Yeah, reading the answer to Megamix is frustrating. Not only did Rob know that many of the songs were so rare nobody would have heard of them (and lyrics were un-Googleable for, eg, Gravy and Mr. Capon-E) ... but if I am interpreting the answer extraction correctly, you literally needed every single one to be correct to do the calculation? Given the difficulty of identification, I don't think I would have ever guessed an extraction mechanism that needed that much precision (as opposed to individual resistors giving letter values).
Foggyfoggyb on January 24th, 2013 03:30 pm (UTC)
The pisser to Megamix is that Charles Steinhardt, one of the other members of Manic Sages, wrote a relatively straightforward song=resistance color as part of a practice warm-up. Had the "The ____ Album" connection been dropped, this puzzle could have been made much more solvable.
(Deleted comment)
rlangmit on January 24th, 2013 06:30 pm (UTC)
Agreed as someone who spent way too long on Megamix before buying it with our first options. We ended up getting all but three songs and even guessed (correctly, it turns out) the color of a missing one in order to produce a reasonably round resistor value.

I guess maybe we could have used the same logic to find the two missing songs in resistor 4? But we still weren't 100% sure what we were doing. There was silence at the end of that track, possibly indicating a missing song ("How about you add another electronic band"), and we could only really make out five songs anyway. We thought it would be great if all the resistors added up to 1 MOhm ("Megamix"), but we were already greater than 1 MOhm with the first 3, so...no.

Anyway, it just got to be too frustrating. The fun part was figuring out the colors and the resistor diagram. That should have been enough.
(no subject) - ztbb on January 24th, 2013 09:37 pm (UTC) (Expand)
(no subject) - noahspuzzlelj on January 25th, 2013 03:58 am (UTC) (Expand)
(Anonymous) on January 26th, 2013 09:31 pm (UTC)
[Atlas Shrugged] solved Megamix with two songs un-ID'd after noting that the two X||Y parallel pairs that we had all the songs for resolved to a number that could be encoded exactly in resistor notation (i.e. requiring no more than three decimal digits and an order of magnitude), which left only a single color possibility for the remaining un-ID'd songs. So, having guessed the extraction mechanism (which we got primarily from the page layout), it was possible to backfill at least one song per block of six.
pesto17 on January 28th, 2013 03:57 pm (UTC)
Meta backsolvability
Manic Sages didn't deliberately make the metas require almost all the answers or be unbacksolvable: our meta testsolves gave a random set of about 3/4 of the answers, and in the two metas I helped testsolve (Feynman and Rubik) I backsolved two answers each.

The metas were in fact too hard, but I don't think there's a reason for that other than those that apply to the whole hunt (editing, too many ahas, and fresh, stronger-than-we-thought testsolvers who like Sages-style puzzles).
lunchboylunchboy on January 23rd, 2013 10:20 pm (UTC)
Terrible advice should always be ignored, no matter how forcefully suggested! Hunts routinely run long even when a team isn't actively *trying* to write a Hunt that will last forever. People that want to solve for a longer amount of time should be on a team with fewer members -- period! Advocating to increase the length of the Hunt because the Hunt feels intangibly different before and after the coin is found seems like a weird position to advocate, and one that's not really in one's best interest. The longer the Hunt runs, the less likely HQ is to keep running the Hunt after the coin is found, meaning there is less opportunity for other teams to see the runaround. Why is that a good tradeoff?