NEWS

Fun with Surveys

  • 104 Replies
  • 19179 Views

Marcus

  • *
  • Prime Runner
  • *****
  • Posts: 2802
  • Success always demands a greater effort.
« Reply #90 on: <09-26-20/1441:25> »
3rd is a very different game from 4th. To me the lesson from 3rd that  always amused me was that 6 and 7 are the same number. Its odd and realization but it actually has meaning.
*Play-by-Post color guide*
Thinking
com
speaking

Shinobi Killfist

  • *
  • Prime Runner
  • *****
  • Posts: 2703
« Reply #91 on: <09-26-20/1450:59> »
I'm kind of surprised with the like of the early editions. I prefer them myself, but I figured they'd were old enough they'd have faded out.

Shinobi Killfist

  • *
  • Prime Runner
  • *****
  • Posts: 2703
« Reply #92 on: <09-26-20/1454:09> »
That shows SR6 waaaay behind everything else in the likes/dislikes ratio. Can't say that is surprising. I haven't played anything other than 5/6 though, but it makes me wish I had seen 3rd.

Other than the matrix rules changing whole-cloth each edition, 1e thru 3e were pretty similar.  Each one was basically an errata-included version of the previous.  4th was the first time things really got shook up... 1-3 to 4 was a bigger change than 5 to 6 was... but it's pretty close to how big a shakeup it was.

Yeah and I think people will have a better chance due to their similarities to like all 3, maybe have a preference, but maybe have things like preferring 3e skills, liking 1es vehicle/hardened armor rules(AKa when they juggernaut was a juggernaut) etc but overall prefer 2e.

Shinobi Killfist

  • *
  • Prime Runner
  • *****
  • Posts: 2703
« Reply #93 on: <09-26-20/1502:26> »
That shows SR6 waaaay behind everything else in the likes/dislikes ratio. Can't say that is surprising. I haven't played anything other than 5/6 though, but it makes me wish I had seen 3rd.

If I'm reading it right its the only edition that has more dislikes than likes. Its not a huge scientific study but its not irrelevant either, for a niche hobby the sample size isn't bad and it went across multiple sources.


0B

  • *
  • Chummer
  • **
  • Posts: 173
  • Be seeing you
« Reply #94 on: <09-26-20/1630:50> »
That shows SR6 waaaay behind everything else in the likes/dislikes ratio. Can't say that is surprising. I haven't played anything other than 5/6 though, but it makes me wish I had seen 3rd.

If I'm reading it right its the only edition that has more dislikes than likes. Its not a huge scientific study but its not irrelevant either, for a niche hobby the sample size isn't bad and it went across multiple sources.

Thanks! And you're right, when you compare it to how every other edition did, there is no overlap of confidence intervals. Therefore, every other edition is "liked" more in the total online population with 95% confidence.

And MC did bring up a good point about some of the sampling bias on SCN discord (Along with unnecessary comments about my "trustworthiness"): I included a lot of 5e LCs, but only one 6e LC. Well, there's only one 6e LC. MC also said that reddit could be biased, which is also possible.

I ended up adding a slide for RPG.net, which "should" be the most neutral for SR editions. Even if you've been "run off" other forums due to hostility about the edition, RPG.net shouldn't have any bias one way or the other about editions.



I was worried since RPG.net only had 80 respondents, but at least 30 responded to each category so we should be good with statistical significance. There's a brief run-down on skew and Clopper-Pearson in the speaker-comments for slide 6.

We get the same results: with 95% confidence, 6e is less "liked" than every other edition for RPG.net.

jim1701

  • *
  • Ace Runner
  • ****
  • Posts: 1070
« Reply #95 on: <09-28-20/2104:17> »
I'm somewhat surprised how well received 5th edition was.  That's the one that killed new editions of Shadowrun for me.  Of course if the quality level had been better I probably would have liked it more.  Well that and if the priority system were better balanced.   

0B

  • *
  • Chummer
  • **
  • Posts: 173
  • Be seeing you
« Reply #96 on: <09-29-20/1249:34> »
I'm somewhat surprised how well received 5th edition was.  That's the one that killed new editions of Shadowrun for me.  Of course if the quality level had been better I probably would have liked it more.  Well that and if the priority system were better balanced.

If you look at who I polled, most of the discords were 5E LC's. Granted, that's because most LC's are 5E, and there's maybe 6 regular players on the 6e LC. The RPG.net stats (Shown above) don't paint 5e in as bright a light. In fact, it's enough that we can say with p < 0.05 that the opinion of 5E on RPG.net versus all locations surveyed is significantly different. In other words, there is no overlap on the margin of error.

I don't know if this means that the places I surveyed overall like 5e more than "normal" for the online SR community, or if RPG.net likes it less than "normal" for the online SR community. It's also worth noting that the RPG.net results are at the bare minimum for us to make statistical judgements about.

I also would avoid conflating quality with how well liked an edition is.

jim1701

  • *
  • Ace Runner
  • ****
  • Posts: 1070
« Reply #97 on: <09-29-20/1340:29> »
I'm somewhat surprised how well received 5th edition was.  That's the one that killed new editions of Shadowrun for me.  Of course if the quality level had been better I probably would have liked it more.  Well that and if the priority system were better balanced.

If you look at who I polled, most of the discords were 5E LC's. Granted, that's because most LC's are 5E, and there's maybe 6 regular players on the 6e LC. The RPG.net stats (Shown above) don't paint 5e in as bright a light. In fact, it's enough that we can say with p < 0.05 that the opinion of 5E on RPG.net versus all locations surveyed is significantly different. In other words, there is no overlap on the margin of error.

I don't know if this means that the places I surveyed overall like 5e more than "normal" for the online SR community, or if RPG.net likes it less than "normal" for the online SR community. It's also worth noting that the RPG.net results are at the bare minimum for us to make statistical judgements about.

I also would avoid conflating quality with how well liked an edition is.

Agreed.  It was just that quality (or the lack thereof) in the editing and production values is what turned me personally off 5e and subsequent editions.  Didn't mean to imply anything beyond that.

0B

  • *
  • Chummer
  • **
  • Posts: 173
  • Be seeing you
« Reply #98 on: <09-29-20/1351:46> »
I'm somewhat surprised how well received 5th edition was.  That's the one that killed new editions of Shadowrun for me.  Of course if the quality level had been better I probably would have liked it more.  Well that and if the priority system were better balanced.

If you look at who I polled, most of the discords were 5E LC's. Granted, that's because most LC's are 5E, and there's maybe 6 regular players on the 6e LC. The RPG.net stats (Shown above) don't paint 5e in as bright a light. In fact, it's enough that we can say with p < 0.05 that the opinion of 5E on RPG.net versus all locations surveyed is significantly different. In other words, there is no overlap on the margin of error.

I don't know if this means that the places I surveyed overall like 5e more than "normal" for the online SR community, or if RPG.net likes it less than "normal" for the online SR community. It's also worth noting that the RPG.net results are at the bare minimum for us to make statistical judgements about.

I also would avoid conflating quality with how well liked an edition is.

Agreed.  It was just that quality (or the lack thereof) in the editing and production values is what turned me personally off 5e and subsequent editions.  Didn't mean to imply anything beyond that.

Ope, my bad! And you'll get no quarrel from me on the quality of 5e's editing and layout, nor on 6e's. I just think there are much better metrics on that then this poll.

Reaver

  • *
  • Prime Runner
  • *****
  • Posts: 6422
  • 60% alcohol 40% asshole...
« Reply #99 on: <09-30-20/1459:43> »
Some of those numbers from the 2nd poll make some sense to me.

1e was the original game. And when you look at it, it was tight, clean, and easy to run.. but lacking of the overall depth of world. (which is to be expected, as the game was fresh and new and hadn't nailed down all the lore yet.)

2e was mostly just a revision of 1e, with some changes to thresholds, target numbers, and modifiers. But you also started to see the Lore of SR get filled in and the "world" starting to flesh out more.

3e: was just an improvement on the mechanics of 1/2e. There was some changes for balance, and a reworking for the initiative system. But for the most part, it was the same game. World building was also at it's peak during the 3e run... and a lot of the lore and mystery of the SR universe was introduced in 3e.

4e: was a total reworking for Shadowrun... And changed everything... But the changes where also a bit of a mess that didn't work well under the hood.

4eA: fixed a lot of the "under the hood" issues that 4e broke, but still had lots of small issues (especially if you were transferring characters from 3e!)

5e: Well, it tried to "fix" some of the core issues of 4e, and it did that well enough, but introduced new issues.....

6e: is again a total rework of the edition rules...


However, there is a large underlining issue for the last 3 editions. Editing.
The editing int he last 3 editions has been poor. (Not even FanPro did a good job).... Which seems to be a lot of people's issues...

Which on one hand I get, and the other I don't. When i look back at all the RPGs I play/played.. none of them have been "great" in the editing department, but then again, I have been gaming for so long, I might have just gotten used to the poor editing and compensate for it after coming across it from so many companies over 3+ decades of gaming.. (anyone remember the old TSR books? How about White Wolf's original works? Palladium? Games Workshop's 2e books? Jackson Games GURPS original books? But then again, they got better... ) 
« Last Edit: <09-30-20/1502:05> by Reaver »
Where am I going? And why am I in a hand basket ???

Remember: You can't fix Stupid. But you can beat on it with a 2x4 until it smartens up! Or dies.

wraith

  • *
  • Chummer
  • **
  • Posts: 120
  • just another ghost in the machine
« Reply #100 on: <10-03-20/0300:08> »
You hit on the important part there. "They got better."

The TTRPG market is undergoing a renaissance right now, there are huge numbers of new games on the market, mostly put out by small teams with shoestring budgets.  If they consider good editing and layout to be make or break, and  can execute on it, why is worse acceptable from a well known studio with actual production budgets and which we know pays most of its writers as freelancers?

0B

  • *
  • Chummer
  • **
  • Posts: 173
  • Be seeing you
« Reply #101 on: <10-03-20/1358:35> »
A lot of the decisions surround 6e are confusing. The goal of it overall was to make the rules shorter (Since that was one of the complaints of 5e). Of course you'll run into issues with some rules like Multi-Attack being vague, but that isn't game-breaking.

But for a game that wanted to make rules shorter, it dragged out a lot of things. It doesn't talk about what a TTRPG is until the reader's gone through sections on lore and a short story. It's not the only edition that did this, but there are others that are better:

1e: p. 20, 15th page after Table of Contents (ToC).
2e: p. 10, 4th page after ToC, 3rd if you ignore a splash page of art.
3e: p. 8, 3rd page after ToC, arguably p. 6/1st page after ToC since that page goes over how to read the book.
4e (fanpro): p. 16, 3rd page after ToC, 2nd if you ignore a splash page of art.

4e (20A): p. 15, 10th page after ToC.
5e: p. 44, 37th page after ToC.
6e: p. 34, 28th page after ToC. 6e doesn't give a pass like 3e does for offering a roadmap on the first page since it insists that new players read through the fiction first before they get to the rules. This is arguably worse than not offering a roadmap at all, since a new player might conceivably skip through lore to get to rules (even unprompted), but now this tells them not to do that.

The editions in bold are the ones that did a good job of introducing what the reader is looking at early. You might say "well, a normal player would skip past the lore if they wanted to read the mechanics." Which seems reasonable, until you consider that the type of player who doesn't know what a TTRPG is would not know to do that, and therefore we cannot count on them to do that.

It's true that TTRPGs are mostly an oral tradition, but they don't always have to be. Sacrificing function for form by front-loading lore, world-building, and short stories is not a good thing. Every board game must be written as if it's the first board game someone's ever played. TTRPGs get a little bit of leniency on this, but only because just about every TTRPG out there commits this design sin. (Well, except the ones bolded above). I'm sure folks would also have an easier time introducing the game to parents, siblings, and friends if the rules were spelled out cleanly, and didn't require years of TTRPG experience to parse and make rulings on.

It's true that 6e isn't "as bad" as 5e with this, but the fact that you still need to go almost 30 pages to learn what a TTRPG is indicates to me that this wasn't a deliberate decision, it was incidental to trimming the fat in other places. The above should also explain why I think "better than 5e" is a very low bar when talking about different aspects of 6e. Sure it's "better than 5e," I'll bet that it's also better than the fantasy-heartbreaker game I made in middle school. 30% on a test is in fact better than 20% on a test. It's still bad.

This is just one aspect- I'm sure other folks can find places where it looks like Shadowrun "forgot" different lessons about game design. But the editions bolded happen to be the ones that were 1) After the very first edition of SR was published, and 2) When the bulk of SR's devs with "collective knowledge" were still working on the project. Even 4e 20A isn't as bad as the other editions, enough that it indicates to me that this was thought about, and that the decision to put fiction in before explaining TTRPGs was deliberate. You don't have to come to the same conclusions as me about when the right time is to explain what a TTRPG is, but you ought to understand why it's important.

At the same time: All the stuff with the pool embezzlement whatnot happened 10 years ago. There was less time than that in-between 1e and 2e, but FASA still started applying lessons-learned about RPG design. Why hasn't that happened here?

adzling

  • *
  • Prime Runner
  • *****
  • Posts: 3290
« Reply #102 on: <10-04-20/1153:50> »
A line dev that does not know what he is doing and a management that does not give two figs about shadowrun.

That's been my experience from working with/ interacting with the management of Catalyst.

wraith

  • *
  • Chummer
  • **
  • Posts: 120
  • just another ghost in the machine
« Reply #103 on: <11-13-20/2119:20> »
A line dev that does not know what he is doing and a management that does not give two figs about shadowrun.

That's been my experience from working with/ interacting with the management of Catalyst.

Yup.  The snake rots from the head, and there's nothing a bunch of freelancers can do when the guy in charge of the line doesn't give a fuck.

FastJack

  • *
  • Administrator
  • Prime Runner
  • *****
  • Posts: 6367
  • Kids these days...
« Reply #104 on: <11-13-20/2250:00> »
Thread's closed, the survey was done and we don't need any more input, especially when it's negative, insulting, and just plain non-helpful.