NEWS

An update on SR6e DriveThruRPG ratings

  • 188 Replies
  • 40781 Views

Michael Chandra

  • *
  • Catalyst Demo Team
  • Prime Runner
  • ***
  • Posts: 9922
  • Question-slicing ninja
« Reply #75 on: <09-15-19/0952:40> »
This is honestly crazy to me.  The number of people that assume all games will have to use houserules to even function.[...]

What gives SR6 (or SR5E) a pass?  What makes it OK for a game to REQUIRE houseruling to be able to even play?  This isn't 1991 anymore.
Not what I said AT ALL. What I meant, and you could have asked before jumping to conclusions and putting words in my mouth, is that while I can see them going either way in some areas once errata are out (while in plenty of cases is obvious but more explicit will help) I have a personal preference in a few cases strong enough to be willing to overrule the rules.

Again, there are plenty of reasons to want better from CGL. Misrepresenting other people's posts isn't needed to make a fair point.
« Last Edit: <09-15-19/0955:47> by Michael Chandra »
How am I not part of the forum?? O_O I am both active and angry!

penllawen

  • *
  • Omae
  • ***
  • Posts: 804
  • Let's go. In and out. Twenty minute milk run.
« Reply #76 on: <09-15-19/1033:03> »
I personally find that the aggression/hostility/dismissiveness is only actually coming from a few people on each side (just repeatedly). Do you find the same or feel different?
Actually, the same, yes. Didn’t always feel like that at times but on reflection I agree with you.

Quote
Beyond that point of curiosity, for what it is worth, I agree that your contribution to the discussion has only been constructive.
Thanks man. That means a lot.

FastJack

  • *
  • Administrator
  • Prime Runner
  • *****
  • Posts: 6374
  • Kids these days...
« Reply #77 on: <09-15-19/1036:56> »
Time to calm down. In regards to moderation:
  • Here's the Terms of Service. Please note that posters receive three warnings before any bans happen, so you may not SEE any moderation, it is occurring.
  • Due to the nature of these discussions, and the fact that many people get heated (myself included), moderation sometimes only occurs when people go too far. Per the ToS, these most commonly are personal attacks (on posters or people associated with the game/boards), flame wars (starting arguments), or being constructive. If we warned people every time a harsh word is said, instead of limiting it to egregious attacks, half the people on this thread (myself included, I'm sorry to say) would have received at least a warning or a banning.
  • As for respecting the posters' opinions, I think I can speak for AJ as well in regards to this. We have NEVER removed a post based on someone's opinions, unless they were extreme examples of profanity, not respecting others privacy, piracy, or going way beyond what is called for. Even in these cases, we see if we can edit some of the post to let the rest stand, instead of outright removal. This is why the calls of "the mods are censoring us!" are so strange, because all we've ever really done is ask the offending poster to turn down the atttitude (and in 95% of the cases they do).

Now, as myself a poster, I restrain myself a lot while posting, because I know my feelings reflect on my moderation. There's been a few times I've started to write up responses and posts, only to delete them because they are not constructive, attack people, or flame an argument. It's easy to do this, actually. All I have to do is type it up, walk away for five minutes, and if, when I come back, I read it and think that I would have to report the post for moderation, I delete it and move on.

stuh42l

  • *
  • Newb
  • *
  • Posts: 18
« Reply #78 on: <09-15-19/1920:01> »
Ok. Given the attitude and responses by some of the mods and team members this doesn't really seem a friendly place to discuss SR.

That's too bad. Wish you guys the best of luck.

FastJack

  • *
  • Administrator
  • Prime Runner
  • *****
  • Posts: 6374
  • Kids these days...
« Reply #79 on: <09-15-19/2005:02> »
Ok. Given the attitude and responses by some of the mods and team members this doesn't really seem a friendly place to discuss SR.

That's too bad. Wish you guys the best of luck.
Well, if that's the case, I'll just go back over here and be just a moderator and not try to engage in any of the discussions. Later all.

Michael Chandra

  • *
  • Catalyst Demo Team
  • Prime Runner
  • ***
  • Posts: 9922
  • Question-slicing ninja
« Reply #80 on: <09-16-19/0028:58> »
Asides from noting 'the game is playable, just not perfect by a long shot and needing significant errata to fix or clarify points of contention'* I think my previous posts already covered all I want to say:
Furthermore, I am not 'tied' to CGL. I am a volunteer demo agent, which is way less tied to CGL than being an errata team agent. I had 'I do not work for CGL' as my personal title for years. I have literally never written any CGL content. I have not even been able to attend any cons to GM at.


Something I should note: There are plenty of reasons to be mad at CGL. There are plenty of improvements to demand from them. And that is why I really do not understand the need to invent claims. To make incorrect claims about who said what, about the moderation involved, about the motives of those involved, about the acts taken during the design process of the current edition. Why poison the well, when you already have enough ammunition to request they both do better and prove sincerity about the ongoing errata-process? You have every right to want better, so why paint CGL in a worse light? What is the point to a contest about who can bash the most subtly or the most harsh?
How am I not part of the forum?? O_O I am both active and angry!

adzling

  • *
  • Guest
« Reply #81 on: <09-16-19/1028:05> »
Ok. Given the attitude and responses by some of the mods and team members this doesn't really seem a friendly place to discuss SR.

That's too bad. Wish you guys the best of luck.
Well, if that's the case, I'll just go back over here and be just a moderator and not try to engage in any of the discussions. Later all.

you're a very fair moderator FJ, don't let anyone tell you otherwise.

mcv

  • *
  • Chummer
  • **
  • Posts: 202
« Reply #82 on: <09-18-19/0502:45> »
I think there are a just a lot of people that hate on this game for no reason other than to hate on it and Catalyst. I'm wondering if some of these haters are fans or backers of independent games on the market. I love indie games, just bought Blades in the Dark, but I'm not going to jump on a forum or store and trash a game just because I prefer it to another. I hope that is not what is happening.
That's not what's happening. It's most likely people who love previous editions of Shadowrun and are disappointed with this new one, or possibly people love the idea of Shadowrun and are disappointed by all editions. But they are Shadowrun fans.

The thing is, when people are really passionate about something, that amplifies emotions, and can make them more militant about their disappointment about certain aspects of the thing they love. And yes, this can easily turn toxic. It often does turn toxic on many topics in many internet communities, and that's not healthy. Look at the reactions to recent Star Wars movies if you want to see fandom turn toxic. I even recognise it in myself: I have not read the new rules yet, but everything I hear about it sounds like it's not for me. And I want Shadowrun to be for me, because I love the setting. But I don't want to get toxic about it, so I hold back. I totally understand the passion of some of the other people who are disappointed with it, but at some point it's healthier to step back and accept that it may not be for you, but other people still like it.

Let them enjoy it. We'll survive on 5th edition. And we can still buy the setting and metaplot books and adapt them to our campaigns.

FastJack

  • *
  • Administrator
  • Prime Runner
  • *****
  • Posts: 6374
  • Kids these days...
« Reply #83 on: <09-18-19/0858:28> »
I think there are a just a lot of people that hate on this game for no reason other than to hate on it and Catalyst. I'm wondering if some of these haters are fans or backers of independent games on the market. I love indie games, just bought Blades in the Dark, but I'm not going to jump on a forum or store and trash a game just because I prefer it to another. I hope that is not what is happening.
That's not what's happening. It's most likely people who love previous editions of Shadowrun and are disappointed with this new one, or possibly people love the idea of Shadowrun and are disappointed by all editions. But they are Shadowrun fans.

The thing is, when people are really passionate about something, that amplifies emotions, and can make them more militant about their disappointment about certain aspects of the thing they love. And yes, this can easily turn toxic. It often does turn toxic on many topics in many internet communities, and that's not healthy. Look at the reactions to recent Star Wars movies if you want to see fandom turn toxic. I even recognise it in myself: I have not read the new rules yet, but everything I hear about it sounds like it's not for me. And I want Shadowrun to be for me, because I love the setting. But I don't want to get toxic about it, so I hold back. I totally understand the passion of some of the other people who are disappointed with it, but at some point it's healthier to step back and accept that it may not be for you, but other people still like it.

Let them enjoy it. We'll survive on 5th edition. And we can still buy the setting and metaplot books and adapt them to our campaigns.
Thank you mcv. This is what I've been trying to promote. Dislike/Like 6E, it doesn't matter. When you turn toxic, then it matters. Please, be productive, give us workarounds for what you don't like about 6E, tell us how you're running games in 5E, be positive about the game/setting you love.

But what is toxic is telling people that they shouldn't like 6E. Telling people they have blindness/stockholm syndrome because they disagree with you. Calling for the firing of developers/selling the IP. That is not positive, and that is toxic.

0B

  • *
  • Guest
« Reply #84 on: <09-19-19/2154:44> »
Also, by the logic you're presenting here, Shadowrun: Anarchy with it's 4.5/5 rating is a better version of Shadowrun.

This is true, tho

I wonder how well 6e would have done as 'Advanced Anarchy.'

I think it would've wrapped around to the same level of crunch... I would much rather prefer a book like Chicago Chaos, which can be used by other editions as well (So long as you restat the stat blocks).

Anyone mind if I derail this and rerail back to the original topic...?

I think what's more telling about how the book is received is the bimodal distribution. At the time of this poster's viewing, it's sitting with a mean of 2.9:

StarsVotes
5/55/30
4/511/30
3/51/30
2/53/30
1/510/30

From this, we can deduce:
  • Of those who liked it (5/5 and 4/5), most had some issues, enough to warrant a 4/5. However, those who liked it were in the majority
  • Almost nobody who rated this had neutral feelings (3/5). The only one who did left a long review, so there was (presumably) thought put into this rating, not just a "I can't decide."
  • Of those who disliked it (2/5 and 1/5), almost all chose the lowest possible rating, indicating that they disliked it completely/to the point where they found it unplayable.

This leads me to believe that the community is divided on 6E. (scientists discover that water is wet).

It's interesting that the same division isn't present in 5E reviews (Which are unimodal, with a bias), though the 5E stats are more reliable since there is a larger sample size. 5/5 has the most votes, followed by 4/5, then 3/5, etc.

Shadowrun Anarchy's distribution is actually closer to 6E's, even though it's mean is so high. The majority of it's reviews are at the 4/5 (20/49), followed by 5/5  (14/49) and then 1/5 (7/49). Common opinion about this is because Anarchy is vastly different than 5E, and did indeed have editing errors (And a few gameplay issues with late-stage progression). However, comma, I still like it the most because it needs the least work done by the GM to make it playable, unlike 5E. YMMV, but I don't think any of Anarchy's problems were worse than 5E's. You already know it's my favorite edition, so I have clear bias here, but I don't think it's the quality difference between 5E and Anarchy that caused this near-bimodal distribution.

Chicago Chaos is an even smaller pool. It's rated at 4.5 because one person gave it a 4/5 and I gave it a 5/5 (full transparency here). It's a campaign book for an edition that isn't played as much, so this sample size is very unreliable, even if you remove my score.

There are a few competing hypotheses here:
  • 6E is not as well-received as 5E because it is not as good.
  • Similar to Anarchy, 6E is not as well-received because it's very different from previous editions. This is for a multitude of reasons, namely the Edge system and combat being a bit more "spread out" between the archetypes. YMMV.
  • 6E is not as well-received as 5E because most of the current fan base is built around 5E, and people dislike change.
  • 6E is in fact as well-received as 5E was at release. (Follow-on explanation): 5E does better now because the DTRPG 5E book is after a few rounds of changes. Numerous sourcebooks exist for 5E that do not exist for 6E, and these sourcebooks solve a lot of core problems with 5E- namely, the Matrix and "magicrun" to a certain extent. Community support is good at directing people to which sourcebooks will enhance the experience, and multiple fan tools already exist. 5E as it is now is better than 6E, but that does not mean that 6E will not approach 5E's quality level in time.
  • DTRPG statistics are unreliable (NOTE: Unreliability does not prove/disprove one of the other hypotheses, it simply means they can't be used as evidence)
  • 0B hasn't taken a stats class since college and is pulling all of this out of her hoop

Most of these are subjective, the last one is almost certainly a factor. I think the truth is somewhere in-between a lot of these.

Just my thoughts on this!

wraith

  • *
  • Chummer
  • **
  • Posts: 120
  • just another ghost in the machine
« Reply #85 on: <09-20-19/0013:58> »
That's a lot of good statistical thought, but at the same time:

When the people being set up by CGL to promote the edition have to publicly tap out due to quality control issues, it is what we in the internet industry call a bad look.

FastJack

  • *
  • Administrator
  • Prime Runner
  • *****
  • Posts: 6374
  • Kids these days...
« Reply #86 on: <09-20-19/1010:02> »
That's a lot of good statistical thought, but at the same time:

When the people being set up by CGL to promote the edition have to publicly tap out due to quality control issues, it is what we in the internet industry call a bad look.
Query: who are the "people being set up by CGL"?

Iron Serpent Prince

  • *
  • Omae
  • ***
  • Posts: 384
« Reply #87 on: <09-20-19/1027:30> »
That's a lot of good statistical thought, but at the same time:

When the people being set up by CGL to promote the edition have to publicly tap out due to quality control issues, it is what we in the internet industry call a bad look.
Query: who are the "people being set up by CGL"?

I'd imagine it would be people like Roll4It that received copies of the rules early to highlight on their shows.
They were supplied, provided, or set up by CGL to promote the edition.

FastJack

  • *
  • Administrator
  • Prime Runner
  • *****
  • Posts: 6374
  • Kids these days...
« Reply #88 on: <09-20-19/1054:18> »
That's a lot of good statistical thought, but at the same time:

When the people being set up by CGL to promote the edition have to publicly tap out due to quality control issues, it is what we in the internet industry call a bad look.
Query: who are the "people being set up by CGL"?

I'd imagine it would be people like Roll4It that received copies of the rules early to highlight on their shows.
They were supplied, provided, or set up by CGL to promote the edition.
Thanks, I just wanted clarification since the language was a bit ambiguous.

ZeroSum

  • *
  • Omae
  • ***
  • Posts: 401
« Reply #89 on: <09-20-19/2019:46> »
This thread was... a read. Warning: beginning of off-topic opinion piece.

As an old geezer with far too much time on my hand who recently discovered that one of my favourite RPG settings had come out with a new edition, I can unequivocally say that the Drive-thru RPG review scores caused me, personally, to hold off on buying the 6th World core rulebook.

Not permanently, but the scores definitely merited some more research before investing time and money into a product. And so, I came here, looking for opinions from the fans. Spoiler alert: I had no idea what I was in for.

Some of the discussion in this thread has focused on the raw numbers, and then a whole lot of it was blatantly off-topic (pot, kettle, hello there!) with what, to a newcomer like me at least, seems like long-held grudges between a few individuals being aired in public.

While all of what follows, as well as my decision to not buy the book outright, is obviously anecdotal as far as the scientific method goes, I for one am here to learn more and hopefully play a game for fun, and not prove a scientific theory. I found it particularly interesting that several people jumped down the original posters' throat claiming that presenting numbers proved intent, instead of simply asking them what their intent was. Several of them were then, somewhat ironically, faced with the same treatment and responded about as well as one could expect when faced with what could easily be mistaken for personal attacks. Anyway, I digress...

Special mention to the few notable exceptions such as this post by 0b, and this post by dezmont, both of which were extremely on-topic and highly constructive to a discussion that drew my immediate interest (what do the numbers potentially mean!?).

Others... and there were unfortunately many such examples, not so much. I'm not going to go so far as to call out examples, but I think it's fair to say that most of the people in this thread could do with a bit of a break to re-evaluate what they spend their energy on, and how they come off to the new people who are just now joining the club.

The fact that people are declaring their intentions to leave the official forums, due to the way they've been treated, for a product I've loved since it first came out is something that I find... troubling. I had higher hopes when I first registered earlier today, I'll be honest. But, I'm not one to let first impressions deter me, and while this thread was a rough one I hope this is the exception rather than the rule,

I would like to thank AJCarrington in particular for this response; a breath of fresh air, it was, and some much appreciated calm in the eye of the storm.

All right, end of off-topic rant. Disclaimer: all of the above are my personal opinions, as someone who is new to the current iteration(s) of the game and as someone who has not spent a lot of time in online communities, this one least of all. Feel free to take from the above what you want.

On-topic, I do have some observations (also anecdotal, note that I'm not trying to prove a theorem, hypothesis, or theory here!) that I would like to add.

First of all, the review score numbers are just data. Anyone who has taken an introductory course in statistics knows that numbers without clear context can be used to represent pretty much whatever view you want. When presented with a rating system like the one used on DTRPG, I believe most people weight the total score as well as the number of reviews as an indication of user satisfaction. This is no different than when buying a product on Amazon; when presented with product options that have hundreds, if not thousands, of reviews and an average score of 4.5/5, I can safely say that I frequently just click "Buy" if the price is right on a product I'm looking for.

So, what do you do when presented with an option that seems less than satisfactory. Would you go to a restaurant with 3/5 stars on Yelp, or a garage with 2/5 stars on Google Maps, without first reading some of the reviews? I can't speak for you all, but I would want to know more before making a decision. And so, that is precisely what I did in this case.

As of the time of this writing, the Sixth World Core Rulebook has an average rating of 2.9 / 5 with 30 reviews. But only 13 of those reviews actually have comments. I find that it is much more common for people to rate something at either extremes of the scale if they don't have to leave comments, because these extremes are more often than not, at least in my personal experience, emotional responses.

If you only look at the scores of the 13 reviewers who also wrote comments, you're left with an average of 2.46 (5x 1*, 2x 2*, 1x 3*, and 4x 4*).

All right, that's not great, but these are still just numbers. Let's see if we can draw some common denominators from the various ratings. I pulled all 1181 words from the 1 and 2 star reviews into a word cloud generator, which created this: https://i.imgur.com/Aqe9I3O.png. Here are the top 10 words, with number of occurrences preceding it:
20   edge
13   game
13   get
13   new
11   just
11   one
10   point
10   sam
9   smoke
9   can

Edge is definitely a hot topic, and it's not hard to imagine that some people associate "new" with "bad"; a lot of people fear change, after all. The rest are harder to draw much meaning from, and combined they highlight the problem with attempting to infer meaningful insight from a limited sample size. This is a topic that was brought up several times in this thread, and which I wholeheartedly agree with.

Now let's look at the 3 and 4 star reviews. With a total word count of 830, we can generate this map and again draw the top 10 words: https://i.imgur.com/5M7hej6.png
12   like
11   book
10   game
10   can
8   edition
8   rules
7   Shadowrun
7   system
7   much
7   lot

Like is an obvious emotional statement, and while we cannot infer that all instances of a word are positive (one could state "do not like" for example), I think it's fair to deduce that a more positive review score (i.e. above the average) combined with an expression of emotion is more likely to be favourable.

What does this all boil down to? For me, not much, to be honest. Finding objective truth in subjective material such as review scores is incredibly difficult, as there are too many factors at play. Take the above restaurant reviews I mentioned, and pick a local restaurant near you with a high review count. I can pretty much guarantee that you will find some low ratings because the person had to wait for a table or for the food, or because the reviewers impression of the servers was unfavourable. What does this tell me about the quality of the food at the restaurant? Nothing, really, because those reviews as a whole is more about the subjective "overall dining experience" than the objective "quality" of the food.

In other words, trusting review scores to tell us much of anything about a product is a gamble at best, especially when the total review count is small. There are just too many unknowns at play, and for this product these could be that the reviewer has previous experience with Catalyst products (it seems grammatical consistency is a big issue with many, for example), or that the reviewer is emotionally projecting ("I do not like this product, therefore it is bad!"). These issues are not invalid, per se, because they detail the reviewers personal experience with the product, but on the whole they are not particularly helpful either.

TL;DR
Using subjective source data to find objective truths is hard.


I'd like to sign off by saying that I hope my rant hasn't come off as negative. I've been an avid fan of Shadowrun since 1st Edition, I played the game through most of 4th Edition, and then had to take a break for personal reasons. While I picked up many of the 5th Edition books for the fiction, I never did get around to playing it, so my familiarity with that system from a rules perspective is limited at best. My situation has improved, recently, and 6th Edition seemed like a good time to jump back in, but I think I will leave it alone for now and catch up on some novels while the errata team does their thing.

I look forward to engaging with you all in an earnest exchange of opinions that are, hopefully, free of some of the toxicity I've seen in this thread. I for one am a blank slate, and I hope that the release of 6th Edition brings many more like me back to the fold (or indeed, into the fold for perhaps the first time!). Ultimately, all I want for one of my favourite hobbies of all time is good things, and that is something I hope we can all agree on.