Shadowrun

Shadowrun General => General Discussion => Topic started by: 0B on <09-02-20/1537:21>

Title: Fun with Surveys
Post by: 0B on <09-02-20/1537:21>
New Survey

This one's (http://"https://docs.google.com/forms/d/e/1FAIpQLSdlJR3AshuRh7Q7lhXMjEK-cp3vW6H6sVuakgGuav8PYwvJUg/viewform?usp=sf_link") about your favorite editions

Old Survey

I don't want to veer too far off-topic in the State of 6e thread.

Basically, this is a form (https://docs.google.com/forms/d/e/1FAIpQLSc1ztvCa7MIBhH9aP7Q7XwRCbc7HyetZIB62epagcwwXNh2FA/viewform) asking different questions about what you like/dislike about 6e. Since it's been over a year since 6e was released, presumably a lot of the kneejerk reactions will have simmered down. Or maybe not!

I'll post the graphs in a week, but DM me if you want the raw google sheets data
Title: Re: Fun with Surveys
Post by: 0B on <09-08-20/1952:20>
Results!

It's here! The survey results that nobody asked for! DM me if you want the raw data.

Image Gallery (https://imgur.com/a/I7YeEpB)

A couple things to note:

* These are very small sample sizes! Even at 95% confidence level, you're going to have an interval of +/- 5 to 20 percent. It's also tricky to estimate population- reddit says it has 38,800 users, but only 200 or so are on at any given time. I'm sure some of those users never even saw the survey. I'm sure quite a few saw it, and ignored it! I overestimated total population to be registered users on each site, for a population of 60,000 total. (39,000 on reddit, 7,500 on official forms, and 13,500 on dumpshock).
* Not everyone answered every question. This is normal. However, some are more heavily impacted- IE, 52 people answered "what do you like about 6e", and 75 people answered "what don't you like about 6e."
* Where people came from (Dumpshock, Official Forums, Reddit) affects their responses: IE, a larger proportion of people on the Official Forums like 6e compared to Dumpshock. However, it's hard to say this for sure given the small sample sizes
* At minimum, understand this: 1.) This poll represents an estimate of the opinions of the three places I polled, it may not be representative of the entire SR playerbase. The phrase "Only 10% of people like 6e" is false. "Only 10% of those polled like 6e" is true. "Only about 10% of people on the official forums, dumpshock, and reddit like 6e" is close, but again there's large intervals, so this could be misleading. 2.) These results are significant. Sure, the actual amount of people that like 6e might not be 10%. It could be anywhere from 4 - 17%, with 95% confidence. However, doesn't that still tell us that less than half of the players on these three boards like SR 6e? 3.) Yes, some of these results are  going to be obvious. Of the things people like about 6e, it's the setting, the theme the dice system that's been used since 4e, skills, and the matrix. People dislike editing quality, edge, mechanics in general, character creation, and the lack of clear rules.

The questions for this survey are as follows:

Single-select Questions

For these, the confidence range is 95%. I used this site (https://www.surveysystem.com/sscalc.htm) to calculate the intervals. Use This one (http://www.raosoft.com/samplesize.html) if you run into trouble.

What sent you to this survey?

* 44/79 (55.7%) of respondents came from /r/Shadowrun
* 17/79 (21.5%) of respondents came from Dumpshock
* 17/79 (21.5%) of respondents came from Official Forums
* 1/79 (1.3%) put down "ZeeMastermind" in the "Other." *I'm assuming they came from reddit, since I use other usernames on the official forums and Dumpshock (Middle School me thinks "ZeeMastermind" is a better username than I do). This is not me, I did not fill out the survey. I simply made this a "Other" question when it really shouldn't have been*

Have you bought a Shadowrun CRB before the release of 6e?

* 73/79 (92.4%, +/- 5.8%) "Yes"
* 1/79 (1.3%, +/- 2.5%) "No"
* 5/79 (6.3%, +/- 5.4%) "I pirated a CRB (Only select if you HAVE NEVER bought a CRB)"

Have you played Shadowrun before the release of 6e?

* 78/79 (98.7%, +/- 2.5%) "Yes"
* 1/79 (1.3%, +/- 2.5%) "No"

Did you buy the 6e Core Rulebook (6e)?

* 36/79 (45.6%, +/- 11%) "Yes"
* 22/79 (27.8%, +/- 9.9%) "No"
* 21/79 (26.6%, +/- 9.7%) "I pirated it (Only select if you DID NOT buy 6e after pirating it)"

Do you like 6e?

* 8/77 (10.4%, +/- 6.8%) "Yes"
* 53/77 (68.8%, +/- 10.3%) "No"
* 16.77 (20.8%, +/- 9.1%) "It's complicated"

Here is the breakdown of the first few questions by source:

Dumpshock
* 100% have bought a Shadowrun CRB before the release of 6e
* 0% have pirated and never bought a Shadowrun CRB before the release of 6e
* 100% have played Shadowrun before the release of 6e
* 23.53% pirated and never bought the 6e CRB
* 17.65% bought the 6e CRB
* 5.88% liked 6e
* 0% said "it's complicated" about liking 6e
* 17 total respondents

Official Forums
* 100% have bought a Shadowrun CRB before the release of 6e
* 0% have pirated and never bought a CRB before the release of 6e
* 100% have played Shadowrun before the release of 6e
* 76.47% bought the 6e CRB
* 17.65% pirated and never bought the 6e CRB
* 17.65% liked 6e
* 35.29% said "it's complicated" about liking 6e
* 17 total respondents

/r/Shadowrun
* 86.36% have bought a Shadowrun CRB before the release of 6e
* 11.36% have pirated and never bought a CRB before the release of 6e
* 97.73% have played Shadowrun before the release of 6e
* 43.18% bought the 6e CRB
* 31.82% pirated and never bought the 6e CRB
* 9.09% liked 6e
* 22.73% said "it's complicated" about liking 6e
* 44 total respondents

bizarro-ZeeMastermind
* 100% have bought a Shadowrun CRB before the release of 6e
* 0% have pirated and never bought a CRB before the release of 6e
* 100% have played Shadowrun before the release of 6e
* 100% bought the 6e CRB
* 0% pirated and never bought the 6e CRB
* 0% liked 6e
* 0% said "it's complicated" about liking 6e
* 1 total respondent

Multi-select Questions

I do not provide confidence ranges for these questions. Feel free to calculate it.

What do you like about 6e/why do you play 6e?

For this question, the percentage based on the number of people who answered the question is on the left. The percentage based on the total number of respondents is on the right.

* 44/53 (83%) OR 44/79 (55.7%) "I like the setting"
* 8/53 (15.1%) OR 8/79 (10.1%) "I like the mechanics"
* 7/53 (13.2%) OR 7/79 (8.9%) "I like the mechanics (Magic)"
* 10/53 (18.9%) OR 10/79 (12.7%) "I like the mechanics (Combat)"
* 14/53 (26.4%) OR 14/79 (17.7%) "I like the mechanics (Matrix)"
* 16/53 (30.2%) OR 16/79 (20.3%) "I like the mechanics (Skills)"
* 8/53 (15.1%) OR 8/79 (10.1%) "I like the mechanics (Character Creation)"
* 17/53 (32.1%) OR 17/79 (21.5%) "I like the mechanics (Core dice pool mechanics)"
* 7/53 (13.2%) OR 7/79 (8.9%) "I like the mechanics (Edge)"
* 3/53 (5.7%) OR 3/79 (3.8%) "I like Catalyst Game Labs"
* 9/53 (17%) OR 9/79 (11.4%) "I like the CRB Fiction"
* 6/53 (11.3%) OR 6/79 (7.6%) "My group plays 6e"
* 0/53 (0%) OR 0/79 (0%) "Word-of-mouth"
* 1/53 (1.9%) OR 1/79 (1.3%) "Price"
* 1/53 (1.9%) OR 1/79 (1.3%) "Editing Quality (Spelling, grammar, active voice, etc.)"
* 5/53 (9.4%) OR 5/79 (6.3%) "Easy to Learn"
* 1/53 (1.9%) OR 1/79 (1.3%) "Rules are Clear"
* 1/53 (1.9%) OR 1/79 (1.3%) "Style/Syntax (Word choice, tone, sentence flow, etc.)"
* 23/53 (43.4%) OR 23/79 (29.1%) "Theme/genre"
* 1/53 (1.9%) OR 1/79 (1.3%) "Other: I don't play it"
* 1/53 (1.9%) OR 1/79 (1.3%) "Other: The dapper shark meme: https://cdn.discordapp.com/attachments/367180742595313664/618362220765642763/5qzthwxfh6k31.png (1), https://cdn.discordapp.com/attachments/365227581018079235/689905930682237004/9d198e5c57f6cc2853b70afe7a729849.png (2)"
* 1/53 (1.9%) OR 1/79 (1.3%) "Other:  The prospect of a less convoluted version of 5th Edition sounded nice..."
* 1/53 (1.9%) OR 1/79 (1.3%) "Other: Previous player, wanted to try current rule set"
* 1/53 (1.9%) OR 1/79 (1.3%) "Other: Treating drugs like toxins for unified rules! Yay!"
* 1/53 (1.9%) OR 1/79 (1.3%) "Other: to be clear, I *don't* like what has been done with the setting since 6e, but I am broadly fine with most aspects of it before 6e, with the notable exception that I think forcing black ops cyberware to be constantly broadcasting an obvious signal is incredibly stupid, and that if you're going to go the extra mile to support matrix PCs who absolutely refuse to act outside the matrix (not that I consider that a worthy goal), it should be done by providing them options in the environment if and where it makes sense, not by having the entire planet decide that wired reflexes which formerly worked just fine without a matrix connection spontaneously don't function without it."

What don't you like about 6e/why don't you play 6e?

For this question, the percentage based on the number of people who answered the question is on the left. The percentage based on the total number of respondents is on the right.

* 9/76(11.8%) OR 9/79 (11.4%) "I dislike the setting"
* 46/76 (60.5%) OR 46/79 (58.2%) "I dislike the mechanics"
* 29/76 (38.2%) OR 29/79 (36.7%) "I dislike the mechanics (Magic)"
* 34/76 (44.7%) OR 34/79 (43%) "I dislike the mechanics (Combat)"
* 27/76 (35.5%) OR 27/79 (34.2%) "I dislike the mechanics (Matrix)"
* 19/76 (25%) OR 19/79 (24.1%) "I dislike the mechanics (Skills)"
* 35/76 (46.1%) OR 35/79 (44.3%) "I dislike the mechanics (Character Creation)"
* 16/76 (21.1%) OR 16/79 (20.3%) "I dislike the mechanics (Core dice pool mechanics)"
* 48/76 (63.2%) OR 48/79 (60.8%) "I dislike the mechanics (Edge)"
* 40/76 (52.6%) OR 40/79 (50.6%) "I dislike Catalyst Game Labs"
* 16/76 (21.1%) OR 16/79 (20.3%) "I dislike the CRB Fiction"
* 6/76 (7.9%) OR 6/79 (7.6%) "I do not want to learn a new edition"
* 31/76 (40.8%) OR 31/79 (39.2%) "My group plays a different edition"
* 15/76 (19.7%) OR 15/79 (19%) "Word-of-mouth"
* 11/76 (14.5%) OR 11/79 (13.9%) "Price"
* 58/76 (76.3%) OR 58/79 (73.4%) "Editing Quality (Spelling, grammar, active voice, etc.)"
* 17/76 (22.4%) OR 17/79 (21.5%) "Not easy to Learn"
* 45/76 (59.2%) OR 45/79 (57%) "Rules are not Clear"
* 31/76 (40.8%) OR 31/79 (39.2%) "Style/Syntax (Word choice, tone, sentence flow, etc.)"
* 3/76 (3.9%) OR 3/79 (3.8%) "Theme/genre"
* 1/76 (1.3%) OR 1/79 (1.3%) "Other: The writers broke coherence between fiction and mechanics."
* 1/76 (1.3%) OR 1/79 (1.3%) "Other: The mechanics make no sense, they do not reflect real world gun fights nor combat in any form. They are useless at everything they try do combat-wise."
* 1/76 (1.3%) OR 1/79 (1.3%) "Other: Doubling Down on shit nobody liked in the previous Editions. Some games arnt ment to be simplified. Also lookin at you Todd Howard"
* 1/76 (1.3%) OR 1/79 (1.3%) "Other: Playing Pathfinder"
* 1/76 (1.3%) OR 1/79 (1.3%) "Other: Oversimplification"
* 1/76 (1.3%) OR 1/79 (1.3%) "Other: urning Seattle into Hong Kong because the HK people left is weaksauce. The whole Fiat-EMP plot is horrible and the missing SuperSoldiers from Hell are not better either. It's peak MagicRun."
* 1/76 (1.3%) OR 1/79 (1.3%) "Other: disliking matrix rules was a bit broad. I like the idea of technomancers. I don't like the current existing rules for them. I also don't like what has been done with the setting since 6e started in terms of plot."
* 1/76 (1.3%) OR 1/79 (1.3%) "Other: The nerfing of cyberware/bioware that started in 5th edtion.  I do not know why you had to make decking so stupidly expensive and making Street Sams all but unplayable."
* 1/76 (1.3%) OR 1/79 (1.3%) "Other: Low Production Quality"

Which of the following have you gone to for information on new games in the past year?

* 64/77 (83.1%) "reddit[.]com/r/shadowrun"
* 25/77 (32.5%) "reddit[.]com/r/rpg"
* 16/77 (20.8%) "Another reddit sub"
* 47/77 (61%) "Official Shadowrun Forums"
* 31/77 (40.3%) "Dumpshock Forums"
* 15/77 (19.5%) "RPG.net"
* 12/77 (15.6%) "Another forum"
* 39/77 (50.6%) "Drive-Thru RPG"
* 33/77 (42.9%) "Drive-Thru RPG Reviews"
* 13/77 (16.9%) "Another website"
* 17/77 (22.1%) "Local Game Store (FLGS)"
* 8/77 (10.4%) "SCN Discord (Shadowrun Community Network, previously Shadowcasters Network)"
* 22/77 (28.6%) "Another Discord Server"
* 2/77 (2.6%) "CGL Mailing List"
* 3/77 (3.9%) "ShadowRN Mailing List"
* 2/77 (2.6%) "Other Mailing List"
* 9/77 (11.7%) "Official CGL Website"
* 13/77 (16.9%) "Official Shadowrun Website"
* 1/77 (1.3%) "Other: srg"
* 1/77 (1.3%) "Other: 4chan"
* 1/77 (1.3%) "Other: Shadowrunners' Union Facebook group"
* 1/77 (1.3%) "Other: TheRPGsite, RPGpub"

Open-ended Question

What are your candid thoughts on 6e?

* Completely useless for quantitative analysis, but may help generate ideas that I hadn't thought of.
* Link (https://docs.google.com/spreadsheets/d/1oFH6qvigoCaOZeo1ODc5V9_k_doXOs3uzgFr7dCu8pQ/edit?usp=sharing)
Title: Re: Fun with Surveys
Post by: FastJack on <09-08-20/2140:56>
I meant to ask this before, but did you have a way of tracking/preventing someone from taking the survey more than once?
Title: Re: Fun with Surveys
Post by: 0B on <09-08-20/2229:50>
Nope! And if I asked folks to put in an email or something like that, or made them aware that I was tracking their IP, I would get under-reporting on piracy. (Similarly, it would be unethical to track users without telling them, especially when I'm asking them about something like piracy.) There's also no good way to make that perfect- changing IPs is trivial, and emails can be made for free or faked. Perhaps there's a google login feature, but that would certainly get me under-reporting on piracy if their answers are tied to an account.

So it's a debate between what's more likely:
* Someone doesn't want to associate their IP or email with piracy, and therefore gives me erroneous data
* Someone decides to take a survey multiple times in order to bias the data... in an unofficial survey that nobody at CGL is involved in or cares about. I don't see the payoff beyond trolling? Maybe?

The first one seemed like a definite occurrence, the second one seemed less likely. So, I didn't track survey-takers.

If we thought someone was borking with the data, the best way to look for it would be timestamps. I can't imagine anyone having the patience to wait more than 5 minutes between "tries" for something without a payoff. You can also try looking for similarities in writing style for the "candid thoughts" question, since 61 respondents answered that.

There are several responses within 5 minutes of each other on the first day that I posted this. However, the answers on each of them are different, and the "candid thoughts" appears to have different styles as well. If someone was answering the survey repeatedly, they were not doing so in a way to push an agenda. If someone went through the effort to send in a response every 30-60 minutes, using sentences crafted to appear as different people, then they've outsmarted me.

Either way, if you want to look at the raw data to be sure, it is here (https://docs.google.com/spreadsheets/d/10N4qFRjJFRU_mbd6Z9xeK-2i3bJ15IoyeQa2X27cnJ0/edit?usp=sharing).
Title: Re: Fun with Surveys
Post by: Lormyr on <09-09-20/0705:07>
My favorite take aways:

1. The top two things that are liked at SR6 are the setting and genre, or have nothing at all to do with the new mechanics.

2. 45.6% of people bought the book and only 10.4% like the edition, proving that people still throw their money at things before they even know if they will like it or not, also proving that sales figures of the crb do not necessarily prove success of the product.

3. 52.6% of the responders dislike catalyst itself.

Neat to see so raw statistics oB, thanks for taking the effort.
Title: Re: Fun with Surveys
Post by: 0B on <09-09-20/1532:54>
It's confirming a lot of other things we've seen: of the mechanics that people do like, it's the core d6 system, skills, and matrix. Matrix might actually be a bit higher, as one respondent clarified that they only dislike technomancers, not the rest of it, even though they selected that they disliked the Matrix. What you gonna do

This stuff's fun to do!
Title: Re: Fun with Surveys
Post by: Reaver on <09-09-20/1559:32>
The only thing that could be skewing your expectations is where you did your polling.
For example, I have accounts at all 3 sites. (here, dumpshock, Reddit), And I am sure you will find that is the case for most people... So you don't really have 3 separate communities with a total of 60,000 potential respondents. Its more likely your total pool size is closer to just 40K (Reddit), as the more dedicated fans will have accounts on all 3 sites, with this site being the smallest.


Not that it really matters as you are only tracking responses...


(And, for the record, I didn't take the survey as I have yet to purchase, or even fully read 6e, and the hour or so I spent with the book is not enough time to give an accurate accounting of the CRB in my opinion.)
Title: Re: Fun with Surveys
Post by: adzling on <09-09-20/1622:31>
....
1. The top two things that are liked at SR6 are the setting and genre, or have nothing at all to do with the new mechanics.

2. 45.6% of people bought the book and only 10.4% like the edition, proving that people still throw their money at things before they even know if they will like it or not, also proving that sales figures of the crb do not necessarily prove success of the product.

3. 52.6% of the responders dislike catalyst itself....

The first is a strength for Shadowrun I.P.

The second is pretty damning refutation of the "change is bad" meme that M.C. and other apologists have been pushing.

The third is an horrific indictment of Catalyst.
If 50% of your customers hate you then you're gonna have issues.

Overall, I couldn't think of a better argument for selling the I.P. to another holder than this.

Well, except the CEO of Catalyst laughing at his customers not getting product they paid for and embezzling enough money to force the writers to go hungry.
Title: Re: Fun with Surveys
Post by: Lormyr on <09-09-20/1640:34>
Yeah man, that last bit in particular. Like how much of a rolling shitbag do you have to be before people treat you according to your behavior? If I had been someone that suffered as a result of his actions there'd have been headlines.
Title: Re: Fun with Surveys
Post by: 0B on <09-09-20/1704:15>
The only thing that could be skewing your expectations is where you did your polling.
For example, I have accounts at all 3 sites. (here, dumpshock, Reddit), And I am sure you will find that is the case for most people... So you don't really have 3 separate communities with a total of 60,000 potential respondents. Its more likely your total pool size is closer to just 40K (Reddit), as the more dedicated fans will have accounts on all 3 sites, with this site being the smallest.

That doesn't create skew (https://www.youtube.com/watch?v=XSSRrVMOqlQ), population affects the size of the confidence intervals (https://www.youtube.com/watch?v=tFWsuO9f74o). I overestimated population to create larger confidence intervals: this means that my results are going to be less precise, but I will maintain accuracy. If I estimate a smaller population, my results will be more precise, but may not be as accurate.

To be honest, the "true population" is likely somewhere in the hundreds, maybe low thousands. /r/shadowrun has about 200 registered users online at any given time, and it's smaller for this forum and for dumpshock. If we were to only count users that are active in a community (IE, not ones that were last online 5 years ago), then the population size would be much smaller.

For sake of argument, let's say the wild mass guess of 40k is the true population. That would change the values of "Do you like 6e" to the following:

* 8/77 (10.39%, +/- 6.81%) "Yes"
* 53/77 (68.83%, +/- 10.34%) "No"
* 16/77 (20.78%, +/- 9.05%) "It's complicated"

Another way to guess the population is based on how many people clicked on the poll itself. Response rate (https://en.wikipedia.org/wiki/Response_rate_(survey)) might be anywhere from 5-50%. Understandably, not everyone who participates in these communities would notice the survey in the week it was up. Let's say only 5% of active community members partook in the survey. This would put the active population at 1,580. These are the values for this population:

* 8/77 (10.39%, +/- 6.65%) "Yes"
* 53/77 (68.83%, +/- 10.09%) "No"
* 16/77 (20.78%, +/- 8.84) "It's complicated"

So, the effects of the size of the total population are going to be minor upon the confidence intervals. Larger sample sizes would be a better way to mitigate this, but I can't force people to take a survey. In fact, I noticed an error running through this- I had been using a sample size of 79, not 77, in my calculations. These are the results for a sample size of 77 against a population of 60,000:

* 8/77 (10.39%, +/- 6.81%) "Yes"
* 53/77 (68.83%, +/- 10.34%) "No"
* 16/77 (20.78%, +/- 9.06%) "It's complicated"

....
1. The top two things that are liked at SR6 are the setting and genre, or have nothing at all to do with the new mechanics.

2. 45.6% of people bought the book and only 10.4% like the edition, proving that people still throw their money at things before they even know if they will like it or not, also proving that sales figures of the crb do not necessarily prove success of the product.

3. 52.6% of the responders dislike catalyst itself....

The first is a strength for Shadowrun I.P.

The second is pretty damning refutation of the "change is bad" meme that M.C. and other apologists have been pushing.

The third is an horrific indictment of Catalyst.
If 50% of your customers hate you then you're gonna have issues.

Overall, I couldn't think of a better argument for selling the I.P. to another holder than this.

Well, except the CEO of Catalyst laughing at his customers not getting product they paid for and embezzling enough money to force the writers to go hungry.

Be careful here- it's NOT 50% of customers. It's 50% of shadowrun fans on this site, reddit, and the official forums. We have no evidence to say one way or the other if this is a good representation of all CGL customers.
Title: Re: Fun with Surveys
Post by: adzling on <09-09-20/1848:00>
Be careful here- it's NOT 50% of customers. It's 50% of shadowrun fans on this site, reddit, and the official forums. We have no evidence to say one way or the other if this is a good representation of all CGL customers.

Good point, the folks on the sites are almost certainly more dedicated and knowledgeable than the ones who are not and so may not be a good cross section of opinion.
Title: Re: Fun with Surveys
Post by: penllawen on <09-10-20/0329:29>
To be honest, the "true population" is likely somewhere in the hundreds, maybe low thousands. /r/shadowrun has about 200 registered users online at any given time, and it's smaller for this forum and for dumpshock.
Specifically for /r/shadowrun, I suspect it has a small DAU but a large MAU, if you see what I mean. A lot of people check in once in a while but aren't regulars; the true regulars are a fairly small group.

I don't have much to go on other than my own observations of poster names, though.
Title: Re: Fun with Surveys
Post by: tenchi2a on <09-10-20/0707:48>
To be honest, the "true population" is likely somewhere in the hundreds, maybe low thousands. /r/shadowrun has about 200 registered users online at any given time, and it's smaller for this forum and for dumpshock.
Specifically for /r/shadowrun, I suspect it has a small DAU but a large MAU, if you see what I mean. A lot of people check in once in a while but aren't regulars; the true regulars are a fairly small group.

I don't have much to go on other than my own observations of poster names, though.
You also have to a lot of forum posters ( myself included) are shadowrun fans but for early editions. And come here for news not for 6th.
Title: Re: Fun with Surveys
Post by: topcat on <09-10-20/0951:12>
Would be interesting to see how the answers break down if you remove the people who didn't buy it or pirated it.  Neither of those are relevant for SR6 purposes, since they aren't customers.
Title: Re: Fun with Surveys
Post by: 0B on <09-10-20/1137:25>
I disagree to a certain extent- the opinions of people who bought a previous CRB, but not 6e, should also be significant. This is someone who is in the potential market for SR, but decided not to buy the most recent edition.

Look at it from a business perspective: if someone already bought the CRB, why should their opinion matter to CGL? Even if they dislike it, that just shows that they'll buy a book anyways, regardless of its quality. Food for thought.

You can use the data and COUNTIFS to look at other questions, but here is the breakdown for "Do you like 6e" based on if someone purchased a book:

(https://i.imgur.com/ECQysqL.png)

Under your criteria, of looking at people who bought the book:
* 7/36 (19%) like 6e
* 14/36 (39%) are "complicated" about 6e
* 15/36 (42%) dislike 6e

Looking at the criteria of people who bought a book in the past (May or may not have bought 6e):
* 7/71 (10%) like 6e
* 14/71 (20%) are "complicated" about 6e
* 50/71 (70%) dislike 6e
Title: Re: Fun with Surveys
Post by: Reaver on <09-10-20/1423:33>
I was refering to what I saw was your expectations of the community, not your data :P

The numbers are what they are, and reflect the answers by those who answered the poll.

Sadly, I will say thst I think the polling size is too small to get a feeling for anything other the the responders however.

We know the book (core book) has sold thousands of copies  a poll of less than a hundred responses isn't even 5% of the market.


On the flip side:
The fact that you only got less than 100 responses in a week after posting said poll to the '3 biggest platforms' in itself is a little telling...

Apathy towards the game (can't care enough to take a poll?)
Apathy rowards polls in general ("why bother? They are always wrong anyway!")
Title: Re: Fun with Surveys
Post by: Stainless Steel Devil Rat on <09-10-20/1439:18>
On the flip side:
The fact that you only got less than 100 responses in a week after posting said poll to the '3 biggest platforms' in itself is a little telling...

Apathy towards the game (can't care enough to take a poll?)
Apathy rowards polls in general ("why bother? They are always wrong anyway!")

Also it's an open question of how much of the Shadowrun fanbase is represented by the '3 biggest platforms'.  Shadowrunner's Union FB group, for example, has over 8 thousand members.  Even making the faulty assumption of presuming noone is a member of more than one of the three: I doubt dumpshock, r/Shadowrun, and this forum combined have even half that total.
Title: Re: Fun with Surveys
Post by: Marcus on <09-10-20/1450:28>
Well if the official site and Reddit are not the biggest platforms for SR, what are? I do think the IP had taken some fairly sizable hits particularly in the wake of the Xbox game. I still sometimes find people who think it died after that. Certainly the sample population is small. Less then 100 folks, but it accurately represents the opinions of folks in my area. I for one still hope someone will come to their senses fix AR/DR. Not betting on it mind you. But hoping, I mean miracles have happened.

Edit: thanks putting that together oB.
Title: Re: Fun with Surveys
Post by: 0B on <09-10-20/1451:45>
Yeah, here's the thing about sample sizes and statistical significance: They are highly dependent on the size of a population. That is why I also included the confidence intervals as the margin of error.

If I did a survey of 10 respondents, that seems small, but what if the total population is 15? That might give me statistically significant results. (Assuming random sampling, use of confidence intervals, etc)

The other part is that this is NOT a survey of all customers. This is a survey of three sites. So we aren't using the population of the customer base, since nothing about this survey necessarily pertains to the customer base.

It doesn't really matter what we feel the results mean, I'm just doing it based on what the math says. When we say that 10.39% +/- 6.81% of the population likes 6e, what I am actually saying is best put like this:

With 95% confidence, between 3.58% and 17.20% of visitors of /r/shadowrun, dumpshock, and the official forums like shadowrun 6e.

It is completely twisting the facts to say that only 10% of shadowrun fans like 6e. It's also twisting the facts to say that this data is useless because it only measures a few forums, or because it only had 79 respondents. A larger sample size would make the margin of error smaller, a smaller sample size would make the margin of error bigger. These forums are significant, although I bet there's dozens to hundreds of discord servers out there where people talk about SR.

Either way, if you don't like how minimum sample size is calculated or don't think the intervals are large enough, there's not much I can do. I can send you some academic papers about the mathematics surrounding sample size determination, the law of large numbers, and other foundations of statistics, but I don't really want to and I bet you don't want to read them either.

I don't know if you've done a lot of polls, but honestly even a response rate this high isn't bad, and is not an indicator of apathy. Consider that many visitors of one of those sites might not have even seen the post.
Title: Re: Fun with Surveys
Post by: 0B on <09-10-20/1504:34>
On the flip side:
The fact that you only got less than 100 responses in a week after posting said poll to the '3 biggest platforms' in itself is a little telling...

Apathy towards the game (can't care enough to take a poll?)
Apathy rowards polls in general ("why bother? They are always wrong anyway!")

Also it's an open question of how much of the Shadowrun fanbase is represented by the '3 biggest platforms'.  Shadowrunner's Union FB group, for example, has over 8 thousand members.  Even making the faulty assumption of presuming noone is a member of more than one of the three: I doubt dumpshock, r/Shadowrun, and this forum combined have even half that total.

That depends on how you define active members. Does the facebook group have 8 thousand members, or 8 thousand active members? If it's just 8 thousand total members, 2 out of 3 sites beat it out. r/shadowrun has 38,800 members. This site has 7,480 members. Dumpshock has 13,389 members.

If we're going off of active members, we can look at how the FB group's members react: measuring reactions is going to be better than counting comments, since of course people can comment twice on a single thing. The last 10 posts, according to my feed at this time, have these reactions:
* 39, 5 comments
* 19, 0 comments
* 0, 6 comments
* 5, 17 comments
* 19, 7 comments
* 10, 2 comments
* 39, 6 comments
* 1, 5 comments
* 12, 9 comments
* 14, 4 comments

The 5 announcements have these reactions:
* 113
* 72
* 218
* 249
* 124

Is that more or less active users than one of the other sites? It's hard to say. If you look for top posts on /r/Shadowrun from the past month, the top post has 439 upvotes (Which is possibly a higher number of interactions, since there could be some downvotes). If we look at the stickies, we see pen's post that has 129 upvotes and 115 comments. There are a few other 100+ posts in Hot as well.

Either way, I doubt all 8000 members are active on FB since I'm in the group and I only go on FB to talk to my mom
Title: Re: Fun with Surveys
Post by: adzling on <09-10-20/1524:46>
Looking at this poll (thanks OB, really well documented and it's clear you know your stuff) together with sales numbers for 6e core compared to 5e core (and other products) it does look like 6e:

1). Is not liked by the player base.
2). Is not being purchased at any where near the levels 5e was.

This fits my anecdotal evidence from talking to people I play with and on the forums I frequent.
Title: Re: Fun with Surveys
Post by: penllawen on <09-10-20/1531:36>
With 95% confidence, between 3.58% and 17.20% of visitors of /r/shadowrun, dumpshock, and the official forums like shadowrun 6e.
FWIW, I’m not a statistician but I sometimes have to impersonate one at my day job. 0B is completely correct here.
Title: Re: Fun with Surveys
Post by: Banshee on <09-10-20/1559:34>
Personally I find it interesting that roughly 70% of the responses that don't like 6E don't have the book. So how can they be giving it a fair assessment when they are obviously only able to judge on what others say rather than direct personal experience.

Title: Re: Fun with Surveys
Post by: Michael Chandra on <09-10-20/1614:37>
Personally I find it interesting that roughly 70% of the responses that don't like 6E don't have the book. So how can they be giving it a fair assessment when they are obviously only able to judge on what others say rather than direct personal experience.
And then there's the list of people who pirated the book and dislike it, and likely only pirated it because they already decided they hated it BEFORE it came out, pirated it the second it came out electronically (thanks to you-know-who), and we don't know how much of it they decided to actually read or give a fair shot.
Title: Re: Fun with Surveys
Post by: Lormyr on <09-10-20/1655:02>
Personally I find it interesting that roughly 70% of the responses that don't like 6E don't have the book. So how can they be giving it a fair assessment when they are obviously only able to judge on what others say rather than direct personal experience.

Well you have to bear in mind it is trivially easy to read over a book without buying it these days. Between pirating, reading a friend's copy, reading it in a store, ect. Also, 72% of those that replied listed either buying or pirating the book, so do have it. I think you inverted the chart.

pirated it the second it came out electronically (thanks to you-know-who)

Fucking LOL Mikey, your hyperbole is truly legendary! I am totally sure there was only one, single, solitary bro who made copies available. That is totally believable!

LOL.



Title: Re: Fun with Surveys
Post by: 0B on <09-10-20/1658:51>
Personally I find it interesting that roughly 70% of the responses that don't like 6E don't have the book. So how can they be giving it a fair assessment when they are obviously only able to judge on what others say rather than direct personal experience.

That's one possibility. Another is that they played with the quickstart rules, didn't like them, so they didn't get the book. Or that their GM bought the book, they didn't enjoy playing it when they tested it out, so the players didn't buy a copy. We have no evidence to say one way or another. This might be a good follow-up question for a future survey- "Why didn't you buy the book?" or "Have you played 6e?"

I did post up above the result of the COUNTIFS if you only look at who buys the book- 42% of people who purchased the book disliked it, and another 39% have "complicated" feelings about it.

You are welcome to look at the raw data (https://docs.google.com/spreadsheets/d/10N4qFRjJFRU_mbd6Z9xeK-2i3bJ15IoyeQa2X27cnJ0/edit?usp=sharing) to see the stats for yourself, and filter away based on responses to questions.

Of the 36 people who said they purchased, 34 answered the "what do you dislike" question. I've bolded the ones where it differs from all respondents by more than 5%.

27 disliked Editing Quality. (75.0% of purchasers, vs 73.4% of all respondents)
23 disliked that the rules were not clear. (63.9% of purchasers, vs 57% of all respondents)
21 dislike edge. (58.3% of purchasers, vs 60.8% of all respondents)
14 dislike CGL. (38.9% of purchasers, vs 50.6% of all respondents)
14 dislike Style/Syntax. (38.9% of purchasers, vs 39.2% of all respondents)
14 dislike combat mechanics. (38.9% of purchasers, vs 43% of all respondents)
14 dislike mechanics in general. (38.9% of purchasers, vs 58.2% of all respondents)
12 dislike character creation. (33.3% of purchasers, vs 44.3% of all respondents)
11 dislike magic. (30.6% of purchasers, vs 36.7% of all respondents)
11 state that their group plays a different edition. (30.6% of purchasers, vs 39.2% of all respondents)
9 dislike that it wasn't easy to learn. (25.0% of purchasers, vs 21.5% of all respondents)
9 dislike the Matrix. (25.0% of purchasers, vs 36.7% of all respondents)
8 dislike the CRB fiction. (22.2% of purchasers, vs 20.3% of all respondents)
6 dislike skills. (16.7% of purchasers, vs 24.1% of all respondents)
6 selected "word-of-mouth" under this question (16.7% of purchasers, vs 19% of all respondents)
5 dislike the core dice pool mechanics. (13.9% of purchasers, vs 20.3% of all respondents)
4 dislike the setting. (11.1% of purchasers, vs 11.4% of all respondents)
4 dislike the price. (11.1% of purchasers, vs 13.9% of all respondents)
3 dislike the theme/genre. (8.3% of purchasers, vs 3.8% of all respondents)
1 didn't want to learn a new edition. (2.8% of purchasers, vs 7.6% of all respondents)
1 said Other: The mechanics make no sense, they do not reflect real world gun fights nor combat in any form. They are useless at everything they try do combat-wise.
1 said Other: Turning Seattle into Hong Kong because the HK people left is weaksauce. The whole Fiat-EMP plot is horrible and the missing SuperSoldiers from Hell are not better either. It's peak MagicRun.
1 said Other: Doubling Down on shit nobody liked in the previous Editions.

Respondents who purchased the book are slightly more likely to dislike the lack of rules clarity.

They feel about the same on editing quality, edge, style/syntax, combat mechanics, learning difficulty, CRB fiction, "word-of-mouth", setting, price, theme/genre, and desire to learn a new edition.

They are slightly less likely to dislike magic, dislike skills, say that their group plays a different edition, or dislike core dice pool mechanics.

They are moderately less likely to dislike CGL, dislike mechanics in general, dislike character creation, or dislike the matrix.

Personally I find it interesting that roughly 70% of the responses that don't like 6E don't have the book. So how can they be giving it a fair assessment when they are obviously only able to judge on what others say rather than direct personal experience.
And then there's the list of people who pirated the book and dislike it, and likely only pirated it because they already decided they hated it BEFORE it came out, pirated it the second it came out electronically (thanks to you-know-who), and we don't know how much of it they decided to actually read or give a fair shot.

You have nothing to go off of to say this is true of any portion of the population.

In both cases... why would someone buy a book if they didn't think they would like it? IE, they read a bad review, played a game of it and didn't like it, etc.
Title: Re: Fun with Surveys
Post by: 0B on <09-10-20/1705:39>
Wasn't that leaked copy put up anonymously on a file sharing site? I know folks linked to it from reddit.

Or are you talking about the time one of the errata team talked about a rule, breaching NDA, but did not post the full text of the rule? (That's not piracy. Mechanics can't be copyrighted, only their text. US Copyright law is weird, but it does prevent Monopoly from having a monopoly on the use of six-sided dice)
Title: Re: Fun with Surveys
Post by: Lormyr on <09-10-20/1714:36>
Wasn't that leaked copy put up anonymously on a file sharing site? I know folks linked to it from reddit.

Or are you talking about the time one of the errata team talked about a rule, breaching NDA, but did not post the full text of the rule? (That's not piracy. Mechanics can't be copyrighted, only their text. US Copyright law is weird, but it does prevent Monopoly from having a monopoly on the use of six-sided dice)

oB, gotta say, I'm gaining some mad respect for you.
Title: Re: Fun with Surveys
Post by: 0B on <09-10-20/1733:45>
Wasn't that leaked copy put up anonymously on a file sharing site? I know folks linked to it from reddit.

Or are you talking about the time one of the errata team talked about a rule, breaching NDA, but did not post the full text of the rule? (That's not piracy. Mechanics can't be copyrighted, only their text. US Copyright law is weird, but it does prevent Monopoly from having a monopoly on the use of six-sided dice)

oB, gotta say, I'm gaining some mad respect for you.

TBH, we're working off of my memory since the reddit mods deleted the post (Piracy against the rules. Deletion was too quick for ceddit to grab the post). The file sharing site was also one where uploads only stay up for 24 hours, to evade DMCA, but I'm sure folks shared the file via discord or other private channels.

Also being fair here, LVN is trash and they did decide to hate it before it came out. That's one user, though, one who's been banned from SCN and various other SR networks for this kind of behavior, not really a representative of the community.

And don't get too mad at me about "well why do you know these things that only a pirate would know," since I've purchased every 6e book (At least the PDFs). Middle school me might not've bought the book, but middle school me also had a $5 weekly allowance that she either blew on snacks or saved for christmas presents. Her "internet browsing" habits did not affect the bottom line of WOTC one way or another (and getting into the game influenced parents to buy the books for her, eventually).
Title: Re: Fun with Surveys
Post by: adzling on <09-10-20/1757:05>
You don't actually need to buy the 6e book yourself to understand it's trash.
You only need to ask someone who's opinion you value.
Like say your GM who bought it, or another player in your group who bought it.
Or read the reviews on the interweb that highlight it's many deficiencies.

Just like you don't need to buy a Delorean to know that you're gonna have a lot of problems down the road...literally.

This jumped out at me OB:

"1 didn't want to learn a new edition. (2.8% of purchasers, vs 7.6% of all respondents)"

Does this put the final nail in the coffin of the "dedicated/long-time shadowrun players don't like new stuff" meme that M.C. (and others) continually push?
Title: Re: Fun with Surveys
Post by: Michael Chandra on <09-10-20/1828:46>
Or are you talking about the time one of the errata team talked about a rule, breaching NDA, but did not post the full text of the rule? (That's not piracy. [...])
If THAT is what you get out of what I said about someone allowing others to pirate the book the second it came out electronically, I honestly have no idea how to respond and it's probably better if I don't even try. Same with your claim that I might be talking about an anonymous source, when I made very clear the source of the incident was known.

And again people are disqualifying me as a member of the community because I played Shadowrun for a significant time and like SR6 more than SR5, yet somehow I am claiming all oldtimers hate SR6, despite the fact I've never made such a claim. >_> I think we're done here, if these are the angles y'all are pushing.
Title: Re: Fun with Surveys
Post by: 0B on <09-10-20/1841:44>
Or are you talking about the time one of the errata team talked about a rule, breaching NDA, but did not post the full text of the rule? (That's not piracy. [...])
If THAT is what you get out of what I said about someone allowing others to pirate the book the second it came out electronically, I honestly have no idea how to respond and it's probably better if I don't even try. Same with your claim that I might be talking about an anonymous source, when I made very clear the source of the incident was known.

And again people are disqualifying me as a member of the community because I played Shadowrun for a significant time and like SR6 more than SR5, yet somehow I am claiming all oldtimers hate SR6, despite the fact I've never made such a claim. >_> I think we're done here, if these are the angles y'all are pushing.

Did you actually read my post?

For crying out loud. I said LVN does not represent the community.

Also being fair here, LVN is trash and they did decide to hate it before it came out. That's one user, though, one who's been banned from SCN and various other SR networks for this kind of behavior, not really a representative of the community.

Am I being mean to LVN? Yeah, probably. But nothing in this says that LVN's not part of the community, and it has absolutely nothing to do with whether or not you are in the community. We are all painfully aware of your participation here.

Are you talking about piracy in general, then, not the pirated leak and not the NDA violation? Because I did also talk about the pirated leak. If you're talking about piracy in general, then you can look at the stats I posted earlier where it breaks it down by whether folks purchased or pirated it. I suspect that there is some underreporting, but it'd be irresponsible to predict the error there.

This jumped out at me OB:

"1 didn't want to learn a new edition. (2.8% of purchasers, vs 7.6% of all respondents)"

Does this put the final nail in the coffin of the "dedicated/long-time shadowrun players don't like new stuff" meme that M.C. (and others) continually push?

It's a strong signifier for these communities. The margin of error shrinks for rates that are close to 0% or 100%. For that one, we have 95% confidence that for the populations polled, the rate of folks not wanting to learn a new edition is between 1.7% and 13.6%. For purchasers only, it's between 0% and 8.3%.

It's possible that folks that purchase from FLGS and rely on physical copies may have a higher rate (No evidence one way or another), perhaps due to the higher start-up cost, being old, etc.
Title: Re: Fun with Surveys
Post by: Lormyr on <09-10-20/1904:10>
And again people are disqualifying me as a member of the community because I played Shadowrun for a significant time and like SR6 more than SR5, yet somehow I am claiming all oldtimers hate SR6, despite the fact I've never made such a claim. >_> I think we're done here, if these are the angles y'all are pushing.

Or maybe it's because we just don't fucking like your hyperbolic bullshit, and has absolutely nothing to do with your experience.
Title: Re: Fun with Surveys
Post by: adzling on <09-10-20/1922:57>
And again people are disqualifying me as a member of the community because I played Shadowrun for a significant time and like SR6 more than SR5, yet somehow I am claiming all oldtimers hate SR6, despite the fact I've never made such a claim. >_> I think we're done here, if these are the angles y'all are pushing.

I never said you weren't part of the community and I don't see where anyone else has.
So that seems like a strange thing to say.

I am pretty sure you have waved the "they're only complaining because they don't like change" meme around these parts more than a few times.

However I'm not gonna spend my time trawling through your old posts to find them so I'll just take you at you word if you say you didn't say those things and offer a mea culpa.

Regardless it's clear that the "they just don't like 6e because it's different" meme is total and utter crap.

Turns out they don't like 6e cause they think it's crap.

Which is what you would expect when served a crap sandwich.
Title: Re: Fun with Surveys
Post by: 0B on <09-10-20/2004:39>
In the interests of moving things back on track (And I am guilty of derailing as well), I've gotten comments both here and elsewhere about additional questions that should be asked- banshee brings up a good point that someone who has never seen or played 6e may not have a solid opinion on it.

These are some of the ones I have so far:

Let me know what you think
Title: Re: Fun with Surveys
Post by: Banshee on <09-10-20/2029:42>
For what it's worth my observation was only that a significant proportion of "haters" have chosen to be so without actually giving it a chance, and the survey results support this.

Are there issues with 6E? Yes. Is CGL ultimately responsible for them? Yes. But is 6E an "unplayable dumpster fire"? Absolutely not.

It really does play much better than it reads for those that are willing to give it a shot is all I'm saying. Even as one of the authors I've admitted to not liking portions of it and also wish I could have one more pass at tweaking the stuff I wrote once some fresh eyes were able to view it.

There are very few actually negative reviews (strictly concerning game play and excluding quality) that I give much credibility to simply because based on the evidence provided with those reviews they are based on just reading the book (usually without taking errata into account even) or playing it but obviously missing using the rules as presented.
Title: Re: Fun with Surveys
Post by: Shinobi Killfist on <09-10-20/2041:02>
For what it's worth my observation was only that a significant proportion of "haters" have chosen to be so without actually giving it a chance, and the survey results support this.

Are there issues with 6E? Yes. Is CGL ultimately responsible for them? Yes. But is 6E an "unplayable dumpster fire"? Absolutely not.

It really does play much better than it reads for those that are willing to give it a shot is all I'm saying. Even as one of the authors I've admitted to not liking portions of it and also wish I could have one more pass at tweaking the stuff I wrote once some fresh eyes were able to view it.

There are very few actually negative reviews (strictly concerning game play and excluding quality) that I give much credibility to simply because based on the evidence provided with those reviews they are based on just reading the book (usually without taking errata into account even) or playing it but obviously missing using the rules as presented.

Maybe, 27.8% didn't get the book, but their GM may have, they may have played at a demo of some kind etc.  As a quick example my group played Carbon 2185 recently. I made a character from online resources but did not buy the book as it was potentially a one off. My experience was fairly positive.  Its shallow has a pretty bizarre economy but quick and easy to play. We handled character interaction, dealing with security devices, combat. Its fairly cheap so i picked it up after the game. So I played, it could respond to a survey about it but did not have the book at the time.
Title: Re: Fun with Surveys
Post by: 0B on <09-10-20/2241:34>
For what it's worth my observation was only that a significant proportion of "haters" have chosen to be so without actually giving it a chance, and the survey results support this.

Are there issues with 6E? Yes. Is CGL ultimately responsible for them? Yes. But is 6E an "unplayable dumpster fire"? Absolutely not.

It really does play much better than it reads for those that are willing to give it a shot is all I'm saying. Even as one of the authors I've admitted to not liking portions of it and also wish I could have one more pass at tweaking the stuff I wrote once some fresh eyes were able to view it.

There are very few actually negative reviews (strictly concerning game play and excluding quality) that I give much credibility to simply because based on the evidence provided with those reviews they are based on just reading the book (usually without taking errata into account even) or playing it but obviously missing using the rules as presented.

I mean, if someone reads the book and gets a different interpretation than intended, isn't that more on the designer than on the player?

It's fair to say that rules with errata may be better. However, this is the first system I've played and run where I've felt the need to go to errata. I don't have much experience with game systems (Various editions of D&D, Eclipse Phase, Sprawl), but I don't think it's fair to count on the player having a lot of experience with game systems. I don't think all problems with the book require you to play it, either: missing rules at the beginning (Unarmed damage, essence value), typos, references to rules that don't exist- all of these are things that someone can identify without playing.

It's far too easy to fall into the trap of thinking that a player just "doesn't get it," "didn't go into it with the right mindset," "doesn't have the same game design expertise that I do," "was too harsh with criticism for it to matter," etc.

The advice I've always gotten for creating is: If something tells you something's not working for them, they're almost always right. Conversely, if they tell you how to fix it, the fix is almost always wrong.

Take multi-attack for example: we might identify that the rule is vague, that it can be OP using anticipation, or that it doesn't make sense when you take into account how weapons work IRL. There are a dozen different fixes we could come up with, but the designer is going to understand their system the best, and will be able to create the best fix for it.

"Ignoring the critics" is not a good solution. Someone doesn't need to prove to you that there is a problem. They might not have found the right answer to fixing the problem, or maybe they've misread it. The latter is on the designer, not the player. If the rules are not clear, there is a problem with the rules.

Quote
For what it's worth my observation was only that a significant proportion of "haters" have chosen to be so without actually giving it a chance, and the survey results support this.

It really doesn't.

All we see is that about a quarter of the folks on these three communities neither bought the book nor pirated it, give or take 10%. Nothing in the survey asks if they gave it a chance, if they played it, if they played the quickstart rules, if they did a mission at digital gencon, etc. Nothing in the survey shows whether this group is one of the "haters" who posts about "dumpster fires" all the time.

Edit: HOWEVER, comma: the popularity of a game is NOT the same as the game's quality. It wouldn't matter if I had a perfect random sampling of 2000-odd purchasers of the 6e CRB from across all platforms, mediums, whatever. It doesn't matter if they all hate it. The quality of a game is certainly a factor correlated with popularity, but it is NOT the same. I hope this hasn't been misconstrued.
Title: Re: Fun with Surveys
Post by: Marcus on <09-10-20/2303:12>
I've read the 6E CRB. I don't own it. I didn't pirate it.
I read a lot of TTRPG books, it's a hobby of mine.
The layout of 6e is professional and looks good to me. The content just doesn't match up.


Anyway we aren't making more progress

With 95% confidence, between 3.58% and 17.20% of visitors of /r/shadowrun, dumpshock, and the official forums like shadowrun 6e.

This makes perfect sense to me.

So was answer to my question we think various discords are now the most popular interface for SR fans?
Title: Re: Fun with Surveys
Post by: 0B on <09-10-20/2311:51>
I wouldn't go that far- it's more about getting a larger look at the "fanbase" as a whole. I don't have the means to send surveys out to everyone who bought the book, or everyone who considers themselves a SR fan. However, there may be folks on FB and discord servers that represent a different part of the online audience. Heck, there's still a few Shadowrun 3e MUDs going strong.
Title: Re: Fun with Surveys
Post by: jim1701 on <09-11-20/0211:58>
Well this hater read the reviews and saw too much of the 5e experience to want to spend yet more money on a new edition just to get kicked in the teeth again.  Is that fair?  Well, it's fair to me.  I paid a lot of money on books that never lived up to my (or my friends') expectation and despite repeated promises that all would be cleaned up it never actually happened and they moved on to a new edition instead. 

In strictly my own opinion if CGL were serious about cleaning up their reputation vis a vis Shadowrun there would be a new line dev running things for a new edition. I consider it entirely unfair to expect me, who was not satisfied with how 5e turned out and it still under the same management, to turn around and pay yet more money for a new edition. 

On the upside I found out about Rifts for Savage Worlds which is another setting I love and despite being made by a company even smaller than CGL manages to produce a high quality product.  So I suppose that's win/win.
Title: Re: Fun with Surveys
Post by: penllawen on <09-11-20/0417:35>
Personally I find it interesting that roughly 70% of the responses that don't like 6E don't have the book. So how can they be giving it a fair assessment when they are obviously only able to judge on what others say rather than direct personal experience.
That's one possibility. Another is that they played with the quickstart rules, didn't like them, so they didn't get the book. Or that their GM bought the book, they didn't enjoy playing it when they tested it out, so the players didn't buy a copy.
Another answer to this that hasn't been covered: the gnarlier bits of 6e have been discussed and picked apart, with cases made both for and against, over and over, in exhaustive detail, in every SR forum there is. You don't need to do a lot of reading of threads here (or on Reddit or FB etc etc) to have a very good working understanding of the 6e rules for, say, AR/DR/armour; or Anticipation; or Edge; or any other contentious subsystem. From there, I think people can reach a very reasonable and defensible conclusion that they don't like it without ever picking up the book.

Furthermore, I think that by doing that, they can easily get a better sense of the edition's shortcomings than they could from just from reading the book. I read 6e, noticed some issues, didn't notice others. Then I started reading threads on Reddit, thought "wait, what?", went back to the book, went "oh, right. That's bad. I didn't see that." This happened repeatedly.

We all make commercial judgements about whether media is worth our time and our money before we give it our time and our money - films, TV shows, books, comics, whatever. There's no reason RPGs should be immune to that.
Title: Re: Fun with Surveys
Post by: topcat on <09-11-20/1401:41>
I disagree to a certain extent- the opinions of people who bought a previous CRB, but not 6e, should also be significant. This is someone who is in the potential market for SR, but decided not to buy the most recent edition.

I've seen far more SR players who refuse to play, let alone buy, any version newer than the one they're invested in.  I'd be willing to bet these forums have more people playing dead versions than live.

When we look to people who are willing to buy new versions...

Quote
Under your criteria, of looking at people who bought the book:
* 7/36 (19%) like 6e
* 14/36 (39%) are "complicated" about 6e
* 15/36 (42%) dislike 6e

Looking at the criteria of people who bought a book in the past (May or may not have bought 6e):
* 7/71 (10%) like 6e
* 14/71 (20%) are "complicated" about 6e
* 50/71 (70%) dislike 6e

They're twice as likely to like SR6 and twice as likely to have an "it's complicated" view.  That's a massive shift from those who haven't purchased it.
Title: Re: Fun with Surveys
Post by: 0B on <09-11-20/1501:21>
They're twice as likely to like SR6 and twice as likely to have an "it's complicated" view.  That's a massive shift from those who haven't purchased it.

Why is it strange that people who bought the book are more likely to like it or have complicated feelings about it?

I bought a copy of 6e and played it with my players, way back before errata came out. They didn't like it, so they didn't buy their own copies of it. Why would they buy it if they preferred the 5e rules to the 6e rules? You could claim (Without any context whatsoever) that maybe I did something wrong as a GM, but I prefer 6e to 5e, personally. And maybe I didn't read the rules right, or maybe I should've waited months for the errata to come out before playing it. But I doubt I'm the only person who tried out 6e with their group before everyone had a copy of the CRB.

Or are you treating "complicated" the same as "liking"?

Even if we look at it that way, you're still left with 42% of the people who bought your product disliking it. A product isn't an election, you want as many people to enjoy it as possible. Certainly, you can't please everyone, but if almost half of your players don't like the game? There's a problem.

It's fair to say that these only apply to a few online communities.

I've seen far more SR players who refuse to play, let alone buy, any version newer than the one they're invested in.  I'd be willing to bet these forums have more people playing dead versions than live.

Yes, it's fair to say that more people are playing 5e or earlier than they are 6e. Again- if a game isn't enjoyable to someone, is that really on the player or is it on the designer? And ultimately, does it matter? Loss of sales is still loss of sales. Why should I play a game I don't like?
Title: Re: Fun with Surveys
Post by: topcat on <09-16-20/0926:05>
Why is it strange that people who bought the book are more likely to like it or have complicated feelings about it?

It's not strange at all.  The key point is that far fewer people who bought the book disliked it.  Most people would take "liked" to mean that they liked everything about it and "its complicated" to mean that there are things they liked and things they disliked.  Both are more positive than the "disliked" option.

Quote
I bought a copy of 6e and played it with my players, way back before errata came out. They didn't like it, so they didn't buy their own copies of it. Why would they buy it if they preferred the 5e rules to the 6e rules? You could claim (Without any context whatsoever) that maybe I did something wrong as a GM, but I prefer 6e to 5e, personally. And maybe I didn't read the rules right, or maybe I should've waited months for the errata to come out before playing it. But I doubt I'm the only person who tried out 6e with their group before everyone had a copy of the CRB.

Running the release version of a game you weren't ready for and poisoning your group isn't a great way to go about things, no.

Quote
Or are you treating "complicated" the same as "liking"?

See above.

Quote
Even if we look at it that way, you're still left with 42% of the people who bought your product disliking it. A product isn't an election, you want as many people to enjoy it as possible. Certainly, you can't please everyone, but if almost half of your players don't like the game? There's a problem.

Maybe, maybe not.  Depends on the audience you're going for.

Quote
Yes, it's fair to say that more people are playing 5e or earlier than they are 6e. Again- if a game isn't enjoyable to someone, is that really on the player or is it on the designer? And ultimately, does it matter? Loss of sales is still loss of sales. Why should I play a game I don't like?

I'd wager that more people are playing SR4A than SR6 - and wouldn't be suprised if more people are playing SR4A than SR5.  Some people are still playing earlier versions than SR4A, which have some of the worst rules ever within a popular RPG franchise.

The point is that those people weren't going to buy SR6 ever, regardless of what the rules were.  Literally anything they didn't love is an excuse not to buy.  Those people aren't customers anymore.  Their money was never available, so the product shouldn't be made to cater to them.
Title: Re: Fun with Surveys
Post by: adzling on <09-16-20/1139:07>
The point is that those people weren't going to buy SR6 ever, regardless of what the rules were.  Literally anything they didn't love is an excuse not to buy.  Those people aren't customers anymore.  Their money was never available, so the product shouldn't be made to cater to them.

The above does not ring true to me.

Far more people are playing 5e than 4 or any single prior version and all of those people were candidates to purchase 6e.

I know that was true at our table.

Until it became clear how bad 6e was.
Title: Re: Fun with Surveys
Post by: jim1701 on <09-16-20/1303:54>
Why is it strange that people who bought the book are more likely to like it or have complicated feelings about it?

It's not strange at all.  The key point is that far fewer people who bought the book disliked it.  Most people would take "liked" to mean that they liked everything about it and "its complicated" to mean that there are things they liked and things they disliked.  Both are more positive than the "disliked" option.

I would have to disagree with the assertion that far fewer people who bought the book disliked it.  Everything is relative as they say and a nine percent shift at the bottom of the percentile range is not something I'd see as very encouraging. A failing grade is still a failing grade even if it's a slightly better failing grade.  This is somewhat subjective of course but sitting at the bottom of the ladder you'd expect better performance when the audience is actually familiar with the work.  The higher up the ladder you move the harder is is to improve so a nine percent shift from say 81% to 90% would be much more significant. 

I bought a copy of 6e and played it with my players, way back before errata came out. They didn't like it, so they didn't buy their own copies of it. Why would they buy it if they preferred the 5e rules to the 6e rules? You could claim (Without any context whatsoever) that maybe I did something wrong as a GM, but I prefer 6e to 5e, personally. And maybe I didn't read the rules right, or maybe I should've waited months for the errata to come out before playing it. But I doubt I'm the only person who tried out 6e with their group before everyone had a copy of the CRB.

Running the release version of a game you weren't ready for and poisoning your group isn't a great way to go about things, no.
Yet nearly every company does this with their products so there must be something to it despite the criticism. If the product is seen as quality work that may need patching/errata based on customer feedback and customers have confidence that the company will follow thru then the model works.  If the product is not seen as quality work and/or customers do not have confidence the company will follow thru on patches/errata then it doesn't really matter what the reality is. 

In this regard I have personal experience though it's not in the gaming industry.  I used to work for a software company that put out a very popular product.  It was very robust and did a lot of stuff that its customers liked and despite not being a large company in many ways we were considered a industry leader. 

But the software had its share of bugs and over the years management prioritized adding new features over bug fixes.  This did eventually change but the damage had been done.  No matter how much effort we put into improving quality customer surveys kept telling us they no longer had confidence in the quality of our product.  This even impacted our new products which were built from the ground up with a zero bug policy.  In the end the company had to rebrand entirely.  The old software was retired and the new software was renamed along with the company. 

What I see here is a loss of confidence in the company.  That is certainly my position.  As Ob has already said you really need a comprehensive survey of Shadowrun customers to get a really good look at how customers overall view CGL but I still think this one tells us there is a need to improve before they reach a point of no return.  What steps they need to take is up to them but at some point that perception of failure will be out of their control regardless of what they do.  At that point Shadowrun will live only in previous editions, CGL will have to try and rebrand themselves or the IP gets transferred to a new company that isn't saddled with a bad reputation (deserved or not.)

Or are you treating "complicated" the same as "liking"?

See above.
I'm on the backside of the half century mark and I can't remember anyone ever in my entire life saying "It's complicated" in a good way.  In my experience "It's complicated" means "it" is bad but the person saying it's complicated doesn't want it to be bad or at least wishes it were something other than what it is.  This is also very subjective but I'd love to meet the person who says "it's complicated" and is also happy with whatever "it" is. 

Even if we look at it that way, you're still left with 42% of the people who bought your product disliking it. A product isn't an election, you want as many people to enjoy it as possible. Certainly, you can't please everyone, but if almost half of your players don't like the game? There's a problem.

Maybe, maybe not.  Depends on the audience you're going for.

I have to think that a successful business model for a rpg company has two accomplish two things to be successful, a) retain as much of the existing customer base as possible and b) expand the customer base by appealing to new players. 

If almost half don't like the new edition that is a problem for part A and I'd have to go look at the survey again to see if there was a question as to if the respondents were new to Shadowrun with 6e but if experienced GM's are having a hard time selling the merits of the game to new players then that's a problem with part B. 

Yes, it's fair to say that more people are playing 5e or earlier than they are 6e. Again- if a game isn't enjoyable to someone, is that really on the player or is it on the designer? And ultimately, does it matter? Loss of sales is still loss of sales. Why should I play a game I don't like?

I'd wager that more people are playing SR4A than SR6 - and wouldn't be surprised if more people are playing SR4A than SR5.  Some people are still playing earlier versions than SR4A, which have some of the worst rules ever within a popular RPG franchise.

The point is that those people weren't going to buy SR6 ever, regardless of what the rules were.  Literally anything they didn't love is an excuse not to buy.  Those people aren't customers anymore.  Their money was never available, so the product shouldn't be made to cater to them.

I'd have to say that while it is true that some players may have a preferred edition to any RPG (D&D stopped at 2nd Ed AD&D in my view) but that if you are executing a successful plan then most fans of the system will buy at least some of the new edition materials because they are fans.

If a company is losing more than half their existing customer base as you allude to when you say more players are staying with 4A rather than move on to 6E I can't see a way to spin this as a good thing.  I also don't agree that those customers are permanently lost at least not all of them.  What they need to do to right this ship (if anything) is something that CGL will have to decide for themselves.  All past, present and potential customers can do if they are not happy with the current state of affairs is wait and see. 
Title: Re: Fun with Surveys
Post by: Shinobi Killfist on <09-16-20/1436:23>

I'd wager that more people are playing SR4A than SR6 - and wouldn't be suprised if more people are playing SR4A than SR5.  Some people are still playing earlier versions than SR4A, which have some of the worst rules ever within a popular RPG franchise.

The point is that those people weren't going to buy SR6 ever, regardless of what the rules were.  Literally anything they didn't love is an excuse not to buy.  Those people aren't customers anymore.  Their money was never available, so the product shouldn't be made to cater to them.

SR 1e-3e still have better rules than 6e.
Title: Re: Fun with Surveys
Post by: SITZKRIEG on <09-16-20/1639:53>
Nope! And if I asked folks to put in an email or something like that, or made them aware that I was tracking their IP, I would get under-reporting on piracy. (Similarly, it would be unethical to track users without telling them, especially when I'm asking them about something like piracy.) There's also no good way to make that perfect- changing IPs is trivial, and emails can be made for free or faked. Perhaps there's a google login feature, but that would certainly get me under-reporting on piracy if their answers are tied to an account.

So it's a debate between what's more likely:
* Someone doesn't want to associate their IP or email with piracy, and therefore gives me erroneous data
* Someone decides to take a survey multiple times in order to bias the data... in an unofficial survey that nobody at CGL is involved in or cares about. I don't see the payoff beyond trolling? Maybe?

The first one seemed like a definite occurrence, the second one seemed less likely. So, I didn't track survey-takers.

Thanks for running the survey and analyzing/posting the results.  Sorry I missed the survey but I never go to reddit and only swing by dumpshock every week or two.  I'm here every day or two (sometimes three) but I rarely go further than the recent post header since I figured it would give me the two or three active threads for the day.  Did you post this over on the various Shadowrun facebook groups as well?

I'm normally pro-consumer/anti-tracking to extent that I can without professional knowledge but I'd be fine with IP tracking as long as there was a notice at the beginning of the survey that it would only be used to prevent spam results and a promise to delete the data after the survey was closed and those repeats thrown out.  As for asking about piracy, I wouldn't.  You can accomplish the same thing with a general question asking if they formed their opinion about the edition via alternative means like borrowing a friend's or a store copy meant for sharing, playing a demo game of the full rules (or the starter set), watching/reading detailed reviews or breakdowns, etc.  As long as it's a catch all phrase without recrimination, I don't think you'll scare anyone off.  It should work as long as you don't need a detailed breakdown of those alternative methods.
Title: Re: Fun with Surveys
Post by: 0B on <09-16-20/2046:24>
Why is it strange that people who bought the book are more likely to like it or have complicated feelings about it?

It's not strange at all.  The key point is that far fewer people who bought the book disliked it.  Most people would take "liked" to mean that they liked everything about it and "its complicated" to mean that there are things they liked and things they disliked.  Both are more positive than the "disliked" option.

I would have to disagree with the assertion that far fewer people who bought the book disliked it.  Everything is relative as they say and a nine percent shift at the bottom of the percentile range is not something I'd see as very encouraging. A failing grade is still a failing grade even if it's a slightly better failing grade.  This is somewhat subjective of course but sitting at the bottom of the ladder you'd expect better performance when the audience is actually familiar with the work.  The higher up the ladder you move the harder is is to improve so a nine percent shift from say 81% to 90% would be much more significant. 

It's a shift from total population of 69% disliking it to 42% disliking it. That sort of shift is significant.

However, a shift from total population of 11% liking it to 19% liking it may be close to double, but isn't far more. It's still only 7 people out of 36, compared to 8 out of 77.

When we look at it this way, however, we can see that 87.5% of the people who did like the book bought it. It says absolutely nothing about the character of people who didn't buy it- whether they "didn't give it a shot" or "are brainwashed by social media" etc etc.

The other thing is: nothing measures cause and effect here. topcat is trying to interpret this as "because they bought the book, they liked the edition," rather than "because they liked the edition, they bought the book." Chances are, both of these statements may be true for different people. It's also possible you'll get "because they bought the book, they disliked the edition," but less likely that you'll get "because they disliked the edition, they bought the book."

There's nothing to support whether any of the above statements is true for all buyers, most buyers, or even some buyers.

You cannot determine what "It's complicated" means- everyone who took the survey could have had a different meaning for what "It's complicated" means, and that's my fault as the survey designer. I should have done a 1-5 scale or just left it at yes/no. If someone selected "It's complicated," all that means is that they preferred that choice to "yes" or "no."

Quote
Running the release version of a game you weren't ready for and poisoning your group isn't a great way to go about things, no.

Ah yes- the assumption that I wasn't ready, based on nothing. If only I had called that... Still, it seems like you're saying that people shouldn't expect to play TTRPGs at release. (Or maybe it's just me? If so, please give me any feedback you have on my GMing style.) I disagree.

I had no issues running Dungeons & Dragons 4e at release. Sure, we ended up going back to 3.5e after a few months, just due to taste, but the rules were laid out clearly. We did not have to wait for errata. D&D 4e was significantly different from 3.5e, so it was a similar situation. I didn't have any issues running 5e at release, either, but that was within experience of a few other editions as well. We didn't have to wait for errata on that one, either.

It's not a big ask to want a game playable at release, and it's not a bad assumption to think that a game you purchased is playable.

Thanks for running the survey and analyzing/posting the results.  Sorry I missed the survey but I never go to reddit and only swing by dumpshock every week or two.  I'm here every day or two (sometimes three) but I rarely go further than the recent post header since I figured it would give me the two or three active threads for the day.  Did you post this over on the various Shadowrun facebook groups as well?

No worries! The next survey will be, I'm having a peer look at some of my questions before I send it out so that it's more clear next time.

Quote
I'm normally pro-consumer/anti-tracking to extent that I can without professional knowledge but I'd be fine with IP tracking as long as there was a notice at the beginning of the survey that it would only be used to prevent spam results and a promise to delete the data after the survey was closed and those repeats thrown out.  As for asking about piracy, I wouldn't.  You can accomplish the same thing with a general question asking if they formed their opinion about the edition via alternative means like borrowing a friend's or a store copy meant for sharing, playing a demo game of the full rules (or the starter set), watching/reading detailed reviews or breakdowns, etc.  As long as it's a catch all phrase without recrimination, I don't think you'll scare anyone off.  It should work as long as you don't need a detailed breakdown of those alternative methods.

I'll poke around in google forms to see if I can do IP tracking, but I might not be able to on a free account
Title: Re: Fun with Surveys
Post by: Reaver on <09-16-20/2210:17>
I would take "it's complicated" to mean there are things they both like and dislike.
However  we don't kniw what that is.

Also, keep in mind, this is only those that chose to answer the survey, and is a small selection at that. 77 respondants isn't even 5% of the book sales, let alone playerbase
Title: Re: Fun with Surveys
Post by: 0B on <09-16-20/2252:44>
The accuracy of a sample size is not based on what percent of the population is taken, the math for this is more complex. If you take a true random sampling of 1,000, out of a population of 100,000,000, the confidence level will vary based on the rate and what level of confidence you want.

Let's say you have a total population of 8 billion. You take a random sampling of 1,000 people- this is a fraction of a percent of the total population. From your sample, you end up with 50% of respondents being male, and 50% being female.

I use an online calculator to do this, but based on our sample of 1,000 people, we can have-

99% confidence that between 45.92% and  54.08% of the total population is male.
95% confidence that between 46.90% and 53.10% of the total population is male.
90% confidence that between 47.40% and 52.60% of the total population is male.
50% confidence that between 48.93% and 51.07% of the total population is male.


The confidence level means that if you were to take the survey many times (assuming true random sampling), X% of the confidence intervals would contain your true population. 95% is pretty standard, 99% usually has too wide an interval to be practical, and 90% or lower is just too unreliable.

It's true that this isn't a true random sampling. It's also true that this measures 3 specific online communities. It does NOT measure playerbase.

Now, let's say you have a total population of 10,000. You take a random sampling of 50 people- this is half a percent of the total population. Let's say you end up with 46% of respondents being male, and 54% being female.

Now, these are our levels:

99% confidence that between 27.89% and 64.11% of the population is male.
95% confidence that between 32.22% and 59.78% of the population is male.
90% confidence that between 34.43% and 57.57% of the population is male.
50% confidence that between 41.26% and 50.74% of the population is male.


So, now we can see that with a smaller sample size, the confidence levels are larger, even when the sample size is a larger percentage of the total population. However, the confidence levels are still mathematically accurate. They are wider because it is less precise with the smaller sample size, but a 95% confidence level with the sample of 50 is exactly as likely to contain the true proportion as the 95% confidence level with the sample of 1000.

So, if we find that 8 people out of 77 (Or, 10.39%) say they like something, and we assume a worst-case total population of 60,000, we can apply this to our population on /r/shadowrun, the official forums, and dumpshock:

99% confidence that between 1.44% and 19.34% of the population likes SR 6e.
95% confidence that between 3.58% and 17.20% of the population likes SR 6e.
90% confidence that between 4.67% and 16.11% of the population likes SR 6e.
50% confidence that between 8.05% and 12.73% of the population likes SR 6e.


Is this helpful at all? Is there a better way that I can explain why this works? OpenStax (https://openstax.org/books/introductory-statistics/pages/8-introduction) has an intro to stats book that explains confidence intervals more in-depth than I can.
Title: Re: Fun with Surveys
Post by: Marcus on <09-17-20/0015:04>


Quote
I bought a copy of 6e and played it with my players, way back before errata came out. They didn't like it, so they didn't buy their own copies of it. Why would they buy it if they preferred the 5e rules to the 6e rules? You could claim (Without any context whatsoever) that maybe I did something wrong as a GM, but I prefer 6e to 5e, personally. And maybe I didn't read the rules right, or maybe I should've waited months for the errata to come out before playing it. But I doubt I'm the only person who tried out 6e with their group before everyone had a copy of the CRB.

Running the release version of a game you weren't ready for and poisoning your group isn't a great way to go about things, no.


This statement is frankly offensive in my opinion. The 6e rules are what the 6e rules are. If exposure to the actual rules causes players not to like them, it's a problem with the rules, not the GM. Saying someone isn't ready to run something, is frankly a bazaar accusation. Especially considering the reality of the 6e rules. Topcat I think you owe oB and an apology.
Title: Re: Fun with Surveys
Post by: Reaver on <09-17-20/1235:37>
The accuracy of a sample size is not based on what percent of the population is taken, the math for this is more complex. If you take a true random sampling of 1,000, out of a population of 100,000,000, the confidence level will vary based on the rate and what level of confidence you want.

Let's say you have a total population of 8 billion. You take a random sampling of 1,000 people- this is a fraction of a percent of the total population. From your sample, you end up with 50% of respondents being male, and 50% being female.

I use an online calculator to do this, but based on our sample of 1,000 people, we can have-

99% confidence that between 45.92% and  54.08% of the total population is male.
95% confidence that between 46.90% and 53.10% of the total population is male.
90% confidence that between 47.40% and 52.60% of the total population is male.
50% confidence that between 48.93% and 51.07% of the total population is male.


The confidence level means that if you were to take the survey many times (assuming true random sampling), X% of the confidence intervals would contain your true population. 95% is pretty standard, 99% usually has too wide an interval to be practical, and 90% or lower is just too unreliable.

It's true that this isn't a true random sampling. It's also true that this measures 3 specific online communities. It does NOT measure playerbase.

Now, let's say you have a total population of 10,000. You take a random sampling of 50 people- this is half a percent of the total population. Let's say you end up with 46% of respondents being male, and 54% being female.

Now, these are our levels:

99% confidence that between 27.89% and 64.11% of the population is male.
95% confidence that between 32.22% and 59.78% of the population is male.
90% confidence that between 34.43% and 57.57% of the population is male.
50% confidence that between 41.26% and 50.74% of the population is male.


So, now we can see that with a smaller sample size, the confidence levels are larger, even when the sample size is a larger percentage of the total population. However, the confidence levels are still mathematically accurate. They are wider because it is less precise with the smaller sample size, but a 95% confidence level with the sample of 50 is exactly as likely to contain the true proportion as the 95% confidence level with the sample of 1000.

So, if we find that 8 people out of 77 (Or, 10.39%) say they like something, and we assume a worst-case total population of 60,000, we can apply this to our population on /r/shadowrun, the official forums, and dumpshock:

99% confidence that between 1.44% and 19.34% of the population likes SR 6e.
95% confidence that between 3.58% and 17.20% of the population likes SR 6e.
90% confidence that between 4.67% and 16.11% of the population likes SR 6e.
50% confidence that between 8.05% and 12.73% of the population likes SR 6e.


Is this helpful at all? Is there a better way that I can explain why this works? OpenStax (https://openstax.org/books/introductory-statistics/pages/8-introduction) has an intro to stats book that explains confidence intervals more in-depth than I can.

This is true when you have a truly random sample size to survey. But problems of bias come in during execution. And the biggest issue is usually who or where you are polling from.

For example:

If I went to catholic Churches and handed out a poll asking if they supported Abortion, the responses would be overwhelmingly "No." But that result wouldn't reflect the true opinion of a country as Abortion is against the tenets of Catholicism.
If I asked the same question to Twitter, I would get an overwhelming response of "yes". And again this would not reflect the actual opinion of the a country as Twitter is overwhelmingly used by those with a political left leaning, and only fractionally by those of a Right political leaning.

However, If I was to call 10,000 random households and ask the question of "do you support Abortion?", the results would fall much more closely into the national average,and could be a good barometer for a Country. (Again, depending on other factors. Only Conducting the survey on Sundays could again skew the results)

A big one that came up in my Province 20 years ago was Welfare reform. LOTS of polling was done... All of it said "everyone" wanted an increase of the welfare state... Right up until election day.. And the party that made it an election issue was wiped out (and has never recovered).
Turns out, the polling firms had introduced a Bias in their polling. They conducted their polling Monday to Friday from 10am to noon, right when the vast majority of people are at work, and the only people who could answer the surveys were those who were already on Social Assistance, and they, had a natural self interest in seeing the social systems expanded that was not reflected in the rest of the population.

In your case, Its the people that visit the sites.

If the average person bought 6e and encountered NO ISSUES whatsoever with the product, they would not be on these forums, or Reddit or the Facebook page(?).

Thus, they never saw your poll to respond.

You only have to look at the topics that are being posted to see if you are going to have issues with Bias. In the case of Reddit and here (Don't use facebook, so I can check to see.. and Dumpshock has a same posting patterns as here). The overwhelming topic that gets posted are "Help" topics... meaning people that come here, dumpshock, and reddit are looking for answers to issues.

And if they are looking for answers to issues, chances are they completely happy with the product...


To get a truly random sample for your survey, you would need a mailing list from say DTRPG, then email out the questionnaire and compile the results.

At best, you have taken the temperature of those that visit those locations, but possibly not the actual customer base.



Basically, 
you are asking people who have most likely come here seeking help for a problem, if they have a problem with the product. I wonder if that could skew your results any :P
Title: Re: Fun with Surveys
Post by: Lormyr on <09-17-20/1329:44>
If the average person bought 6e and encountered NO ISSUES whatsoever with the product, they would not be on these forums, or Reddit or the Facebook page(?).

Isn't this statement blatantly false though? For example, I joined the forums in like 2014, not because I had a problem, but just because I wanted to talk about shadowrun and my core group of nerds was not yet into it.
Title: Re: Fun with Surveys
Post by: Reaver on <09-17-20/1358:55>
If the average person bought 6e and encountered NO ISSUES whatsoever with the product, they would not be on these forums, or Reddit or the Facebook page(?).

Isn't this statement blatantly false though? For example, I joined the forums in like 2014, not because I had a problem, but just because I wanted to talk about shadowrun and my core group of nerds was not yet into it.

Is it?

The vast amount of threads are looking for rules clarification, reguardless of edition, followed by rules conflict resoulution. The same is true of dumpshock and reddit.

Fanfics and other works that just sing praises of SR (no matter the edition), are the minority.

Now this doesn't mean everyone here hates SR. On the contrary, it shows an investment in the material beyond the average buyer. But that investment doesn't mean its positive or negative....
For example, there are many people on these forums that are invested in SR, but have openly stated their non support, if not hatred and pulling of fiancial backing.

They are no longer "customers" but they are invested.
Mean while, "lil Johnny" buys every book, but never plays nor comes to the forums  or talks about the game.
He is a customer, but he's not invested in the game.
Ideally, you want both. But companies chase money.

Now Sure, you came here looking for "like minded" people, and I am sure that is the reason why many of the top posters are still here (Community is a wonderful thing). But for the vast amount of people, they show up, ask a question or 5, and never post again.
A rare few come here looking for ammo for use in a fight with their GM, throw a hissy fit when the answers don't match what they want to hear, and leave.
I think a quick look at post counts and members will tell you who is "invested" in the community, and who comes here for just answers. The number of people with over say, one hundred posts is fairly small...

Now, keep in mind I say that while not having purchased any 6e book yet...
Which is also the reason I didn't take the survey. I don't have a dog in the 6e race yet...
When 6e was announced, and released my table had just started a new campaign and voted to stick with 5e. That campaign continues.
The GM for the table I play at got the 6e CRB on release, and has decided it doesn't fit with our table, so we would not be changing editions  and so far, I have spent only an hour reading the book (his), so I don't have a full opinion yet.... so at best I am a potential customer...


It very well could be that 6e is Shadowrun's DnD4e.....
Or it could be the edition that makes SR go 'mainstream'...

The only thing we can say, is that it could have been received better. And ultimately, it will be consumers who determine what happens next for the franchise.
Title: Re: Fun with Surveys
Post by: adzling on <09-17-20/1700:16>
Or it could be the edition that makes SR go 'mainstream'...

I think we would have seen some significant movement in sales by now if this was going to happen as it's been a year already.

I don't see any sign of that in the partial data available and the continued negative reception it receives in most knowledgable communities.

I would guess the vast majority of sales are made in the first year as older players transition to the new version.

That's just not happening afaik, quite the opposite.

So how long do we wait to declare 6e dead?

Another year of anemic sales (two years after release)?
Title: Re: Fun with Surveys
Post by: FastJack on <09-17-20/1808:18>
Or it could be the edition that makes SR go 'mainstream'...

I think we would have seen some significant movement in sales by now if this was going to happen as it's been a year already.

I don't see any sign of that in the partial data available and the continued negative reception it receives in most knowledgable communities.

I would guess the vast majority of sales are made in the first year as older players transition to the new version.

That's just not happening afaik, quite the opposite.

So how long do we wait to declare 6e dead?

Another year of anemic sales (two years after release)?
You can declare it dead whenever you want. However, Catalyst is probably not going to agree with you.
Title: Re: Fun with Surveys
Post by: Michael Chandra on <09-17-20/1816:23>
You can declare it dead whenever you want. However, Catalyst is probably not going to agree with you.
I wish I could weigh in on this topic, but NDA so... :-X
Title: Re: Fun with Surveys
Post by: FastJack on <09-17-20/1818:39>
If the average person bought 6e and encountered NO ISSUES whatsoever with the product, they would not be on these forums, or Reddit or the Facebook page(?).

Isn't this statement blatantly false though? For example, I joined the forums in like 2014, not because I had a problem, but just because I wanted to talk about shadowrun and my core group of nerds was not yet into it.

Is it?

The vast amount of threads are looking for rules clarification, reguardless of edition, followed by rules conflict resoulution. The same is true of dumpshock and reddit.

Fanfics and other works that just sing praises of SR (no matter the edition), are the minority.

Now this doesn't mean everyone here hates SR. On the contrary, it shows an investment in the material beyond the average buyer. But that investment doesn't mean its positive or negative....
For example, there are many people on these forums that are invested in SR, but have openly stated their non support, if not hatred and pulling of fiancial backing.

They are no longer "customers" but they are invested.
Mean while, "lil Johnny" buys every book, but never plays nor comes to the forums  or talks about the game.
He is a customer, but he's not invested in the game.
Ideally, you want both. But companies chase money.

Now Sure, you came here looking for "like minded" people, and I am sure that is the reason why many of the top posters are still here (Community is a wonderful thing). But for the vast amount of people, they show up, ask a question or 5, and never post again.
A rare few come here looking for ammo for use in a fight with their GM, throw a hissy fit when the answers don't match what they want to hear, and leave.
I think a quick look at post counts and members will tell you who is "invested" in the community, and who comes here for just answers. The number of people with over say, one hundred posts is fairly small...

Now, keep in mind I say that while not having purchased any 6e book yet...
Which is also the reason I didn't take the survey. I don't have a dog in the 6e race yet...
When 6e was announced, and released my table had just started a new campaign and voted to stick with 5e. That campaign continues.
The GM for the table I play at got the 6e CRB on release, and has decided it doesn't fit with our table, so we would not be changing editions  and so far, I have spent only an hour reading the book (his), so I don't have a full opinion yet.... so at best I am a potential customer...


It very well could be that 6e is Shadowrun's DnD4e.....
Or it could be the edition that makes SR go 'mainstream'...

The only thing we can say, is that it could have been received better. And ultimately, it will be consumers who determine what happens next for the franchise.

Membership on the forums goes like this: Out of the 7498 registered user (current count as of right now [before I start cleaning spammers]).

Between 100-200 people have over 1,000 posts. but only 29 of those have posted in the last 30 days.
There's over 700 people with over 100 posts, but only 75+ have posted in the last 30 days.
Over 1700 members have never posted at all, but I know from looking a logs, they are still active, spending time just reading.
Title: Re: Fun with Surveys
Post by: 0B on <09-17-20/2013:07>
The accuracy of a sample size is not based on what percent of the population is taken, the math for this is more complex. If you take a true random sampling of 1,000, out of a population of 100,000,000, the confidence level will vary based on the rate and what level of confidence you want.

Let's say you have a total population of 8 billion. You take a random sampling of 1,000 people- this is a fraction of a percent of the total population. From your sample, you end up with 50% of respondents being male, and 50% being female.

I use an online calculator to do this, but based on our sample of 1,000 people, we can have-

99% confidence that between 45.92% and  54.08% of the total population is male.
95% confidence that between 46.90% and 53.10% of the total population is male.
90% confidence that between 47.40% and 52.60% of the total population is male.
50% confidence that between 48.93% and 51.07% of the total population is male.


The confidence level means that if you were to take the survey many times (assuming true random sampling), X% of the confidence intervals would contain your true population. 95% is pretty standard, 99% usually has too wide an interval to be practical, and 90% or lower is just too unreliable.

It's true that this isn't a true random sampling. It's also true that this measures 3 specific online communities. It does NOT measure playerbase.

Now, let's say you have a total population of 10,000. You take a random sampling of 50 people- this is half a percent of the total population. Let's say you end up with 46% of respondents being male, and 54% being female.

Now, these are our levels:

99% confidence that between 27.89% and 64.11% of the population is male.
95% confidence that between 32.22% and 59.78% of the population is male.
90% confidence that between 34.43% and 57.57% of the population is male.
50% confidence that between 41.26% and 50.74% of the population is male.


So, now we can see that with a smaller sample size, the confidence levels are larger, even when the sample size is a larger percentage of the total population. However, the confidence levels are still mathematically accurate. They are wider because it is less precise with the smaller sample size, but a 95% confidence level with the sample of 50 is exactly as likely to contain the true proportion as the 95% confidence level with the sample of 1000.

So, if we find that 8 people out of 77 (Or, 10.39%) say they like something, and we assume a worst-case total population of 60,000, we can apply this to our population on /r/shadowrun, the official forums, and dumpshock:

99% confidence that between 1.44% and 19.34% of the population likes SR 6e.
95% confidence that between 3.58% and 17.20% of the population likes SR 6e.
90% confidence that between 4.67% and 16.11% of the population likes SR 6e.
50% confidence that between 8.05% and 12.73% of the population likes SR 6e.


Is this helpful at all? Is there a better way that I can explain why this works? OpenStax (https://openstax.org/books/introductory-statistics/pages/8-introduction) has an intro to stats book that explains confidence intervals more in-depth than I can.

This is true when you have a truly random sample size to survey. But problems of bias come in during execution. And the biggest issue is usually who or where you are polling from.

For example:

If I went to catholic Churches and handed out a poll asking if they supported Abortion, the responses would be overwhelmingly "No." But that result wouldn't reflect the true opinion of a country as Abortion is against the tenets of Catholicism.
If I asked the same question to Twitter, I would get an overwhelming response of "yes". And again this would not reflect the actual opinion of the a country as Twitter is overwhelmingly used by those with a political left leaning, and only fractionally by those of a Right political leaning.

However, If I was to call 10,000 random households and ask the question of "do you support Abortion?", the results would fall much more closely into the national average,and could be a good barometer for a Country. (Again, depending on other factors. Only Conducting the survey on Sundays could again skew the results)

A big one that came up in my Province 20 years ago was Welfare reform. LOTS of polling was done... All of it said "everyone" wanted an increase of the welfare state... Right up until election day.. And the party that made it an election issue was wiped out (and has never recovered).
Turns out, the polling firms had introduced a Bias in their polling. They conducted their polling Monday to Friday from 10am to noon, right when the vast majority of people are at work, and the only people who could answer the surveys were those who were already on Social Assistance, and they, had a natural self interest in seeing the social systems expanded that was not reflected in the rest of the population.

In your case, Its the people that visit the sites.

If the average person bought 6e and encountered NO ISSUES whatsoever with the product, they would not be on these forums, or Reddit or the Facebook page(?).

Thus, they never saw your poll to respond.

You only have to look at the topics that are being posted to see if you are going to have issues with Bias. In the case of Reddit and here (Don't use facebook, so I can check to see.. and Dumpshock has a same posting patterns as here). The overwhelming topic that gets posted are "Help" topics... meaning people that come here, dumpshock, and reddit are looking for answers to issues.

And if they are looking for answers to issues, chances are they completely happy with the product...


To get a truly random sample for your survey, you would need a mailing list from say DTRPG, then email out the questionnaire and compile the results.

At best, you have taken the temperature of those that visit those locations, but possibly not the actual customer base.



Basically, 
you are asking people who have most likely come here seeking help for a problem, if they have a problem with the product. I wonder if that could skew your results any :P

This is why I have mentioned, ad nauseum, that this measures users on this site, Dumpshock, and reddit. So, this survey is a result of the opinions of people who use those sites. Mailing list from DTRPG won't cover the whole playerbase either, because DTRPG isn't the only place that sells SR.

Heck, that's even in the part you quoted "we can apply this to our population on /r/shadowrun, the official forums, and dumpshock."

I agree that it's not a measure of the customer base- I have also mentioned this ad nauseum.

This is an accurate measurement of the sampled populations. As with any survey (including your mailing list suggestion), we can expect some selection bias because responding to this survey was voluntary. So, this does not contain results from people on those three sites who do not like taking surveys. If there is a significant difference in opinion regarding 6e in the population who likes taking surveys versus the population who does not like taking surveys, this will affect the results. I left the survey running for a week so that I did not exclude people in different time zones or with different work schedules.

~~~
 
I think our difference of opinion then comes from whether or not this is useful. Well, useful to who?

A writer isn't going to care about the opinions of folks who haven't read the book- that makes sense. They've already been paid, so the total sales don't matter.

A publisher will care about people who were potential customers but did not buy the book- they need to identify the reasons why in order to make more sales next time. Whether they read the book or liked it is irrelevant.

There's also the question of how much of the total shadowrun community that these three sites take up. I'm sure folks will throw out "It's a lot!" and "It's not a lot!" with a whole lot of conjecture, so I'll just do that to get that over with.

You can declare it dead whenever you want. However, Catalyst is probably not going to agree with you.
I wish I could weigh in on this topic, but NDA so... :-X

You always gotta tease XD I think we can infer from the insta pic CGL dropped a bit ago that at least one more book is coming

Here's a better question you may be able to answer- do the people you work with at CGL enjoy working on Shadowrun?

Membership on the forums goes like this: Out of the 7498 registered user (current count as of right now [before I start cleaning spammers]).

Between 100-200 people have over 1,000 posts. but only 29 of those have posted in the last 30 days.
There's over 700 people with over 100 posts, but only 75+ have posted in the last 30 days.
Over 1700 members have never posted at all, but I know from looking a logs, they are still active, spending time just reading.

Thank you so much! So, spitballing, potentially 2000 people are "active," or a little over 25% of registered users.
Title: Re: Fun with Surveys
Post by: FastJack on <09-17-20/2131:19>
Thank you so much! So, spitballing, potentially 2000 people are "active," or a little over 25% of registered users.
You and I have very different definitions of "active".

61 forum members with more than 100 posts have been on the boards in the last week. If we look at people with 1 or more posts, it's somewhere between 100-200 (I can't get an exact count due to how the member list search works). There have been 20 new topics and 264 posts in the past week (I'm not sure if it counts spam or not against that, but I know I've deleted at least 50 spam posts in this week). It's pretty much those that are in this thread, plus a couple of outliers. Being generous, I'd put it at about 50 "active" (i.e., participating, not just reading) users. Of the approximately 7500 registered users, over 6700 have not been on the boards in more than six months. You're spitball is way off, and is actually closer to 0.67% of registered users.

Put that with your survey, means you got about 34% of the active users on the boards to answer your survey, or about 0.23% of registered users. Reaver's actually got it pretty right from what I see. Most people come on to ask questions, be they about rules or when the next book is announced, and a small number of people are on here as hardcore fans. I mean, of the top ten all time posters, 3 haven't been on the boards for more than a year, and one of them hasn't been seen on here in almost six years.
Title: Re: Fun with Surveys
Post by: 0B on <09-17-20/2252:59>
Thank you so much! So, spitballing, potentially 2000 people are "active," or a little over 25% of registered users.
You and I have very different definitions of "active".

61 forum members with more than 100 posts have been on the boards in the last week. If we look at people with 1 or more posts, it's somewhere between 100-200 (I can't get an exact count due to how the member list search works). There have been 20 new topics and 264 posts in the past week (I'm not sure if it counts spam or not against that, but I know I've deleted at least 50 spam posts in this week). It's pretty much those that are in this thread, plus a couple of outliers. Being generous, I'd put it at about 50 "active" (i.e., participating, not just reading) users. Of the approximately 7500 registered users, over 6700 have not been on the boards in more than six months. You're spitball is way off, and is actually closer to 0.67% of registered users.

Put that with your survey, means you got about 34% of the active users on the boards to answer your survey, or about 0.23% of registered users. Reaver's actually got it pretty right from what I see. Most people come on to ask questions, be they about rules or when the next book is announced, and a small number of people are on here as hardcore fans. I mean, of the top ten all time posters, 3 haven't been on the boards for more than a year, and one of them hasn't been seen on here in almost six years.

Yeah, that's fair to say. I aim high on population because that leads to larger intervals: larger intervals are more likely to contain the "true" value of the population.

When I say "active users," I mean anyone who might've seen the post and clicked on it. I don't ask about their posting habits in the survey, so it'd be wrong to assume that everyone who takes the survey is a regular poster.

If we say that only a third of users across all boards are sampled (60,000 -> 20,000), then the intervals become smaller. More precise, possibly less accurate if we're underestimating the population.

Given a total sampled population of 20,000, we have the following:

99% confidence that between 1.45% and 19.33% of the population likes SR 6e.
95% confidence that between 3.59% and 17.19% of the population likes SR 6e.
90% confidence that between 4.68% and 16.10% of the population likes SR 6e.
50% confidence that between 8.05% and 12.73% of the population likes SR 6e.


This means that there is a smaller margin of error in the survey, but only by about a hundreth of a percent or less.

Again, the proportion of the population that is sampled is not the determinant of how accurate it is. The OpenStax book explains this better than I can, I don't expect you to take me at my word. This isn't a debatable thing, it's how statistics work.

Whether this survey is an accurate sampling of the playerbase at large, or even of the online community, is far more debateable. I do not think it is a good measurement of the playerbase, and I'd say it's a low confidence measurement of the online community.
Title: Re: Fun with Surveys
Post by: penllawen on <09-18-20/0538:27>
Two things I want to highlight/call out/reinforce:

Again, the proportion of the population that is sampled is not the determinant of how accurate it is. The OpenStax book explains this better than I can, I don't expect you to take me at my word. This isn't a debatable thing, it's how statistics work.
This (like so much of statistics beyond the absolute basics) is counter-intuitive, but is completely correct.

And:

Quote
99% confidence that between 1.45% and 19.33% of the population likes SR 6e.
Even at these very tight confidence intervals and (IMO) pretty conservative assumptions about the underlying population, this still paints a pretty poor picture for 6e's reputation with Shadowrun players. This is about as forgiving to 6e as you can possibly be in interpreting these results. And it's still bad.
Title: Re: Fun with Surveys
Post by: FastJack on <09-18-20/0757:40>
You guys do remember I have a degree in Mathematics, right? Not Math-Education, but straight up Math...

I'm not disagreeing with the explanations to prove how accurate your survey is, but you're going on and on about the population of the survey when the true discussion on making a qualitative survey accurate is to make sure the sampling is as random as possible. The method you used is convenience sampling and it's the most biased of all types, because you're asking people to respond if they have the time. This leads to the only people responding as those that want to skew the survey to their "side".
Title: Re: Fun with Surveys
Post by: 0B on <09-18-20/0915:35>
Convenience sampling bias is a thing. I'd imagine that people who are really interested or really disinterested in the topic are more likely to respond. However... for what purpose? I'd also be curious how you would do a voluntary, true-random survey, while still targeting these specific populations.

Also, I got a new survey out- this one's (http://"https://docs.google.com/forms/d/e/1FAIpQLSdlJR3AshuRh7Q7lhXMjEK-cp3vW6H6sVuakgGuav8PYwvJUg/viewform") about your favorite editions
Title: Re: Fun with Surveys
Post by: Michael Chandra on <09-18-20/0944:19>
You guys do remember I have a degree in Mathematics, right? Not Math-Education, but straight up Math...
And there's a metal train that's a mile long
And at the very back end a lightning bolt struck her,
How long 'til it reaches and kills the driver,
Provided that he's a good conductor?
Title: Re: Fun with Surveys
Post by: DigitalZombie on <09-18-20/1037:44>
Convenience sampling bias is a thing. I'd imagine that people who are really interested or really disinterested in the topic are more likely to respond. However... for what purpose? I'd also be curious how you would do a voluntary, true-random survey, while still targeting these specific populations.

Also, I got a new survey out- this one's (http://"https://docs.google.com/forms/d/e/1FAIpQLSdlJR3AshuRh7Q7lhXMjEK-cp3vW6H6sVuakgGuav8PYwvJUg/viewform") about your favorite editions


Link isnt workin :/


I also did a poll on 6th ed quite  a while ago.
https://forums.shadowruntabletop.com/index.php?topic=30525.0

The set up is somewhat different. And also the purpose.
Title: Re: Fun with Surveys
Post by: FastJack on <09-18-20/1041:01>
Convenience sampling bias is a thing. I'd imagine that people who are really interested or really disinterested in the topic are more likely to respond. However... for what purpose? I'd also be curious how you would do a voluntary, true-random survey, while still targeting these specific populations.

Also, I got a new survey out- this one's (http://"https://docs.google.com/forms/d/e/1FAIpQLSdlJR3AshuRh7Q7lhXMjEK-cp3vW6H6sVuakgGuav8PYwvJUg/viewform") about your favorite editions
The only way to get the random sampling is to either go to GenCon or something similar and survey passerbys, or to take a random sample of your sample (but that requires a lot more responses).
Title: Re: Fun with Surveys
Post by: Lormyr on <09-18-20/1128:34>
Re: Reaver,

I get your point, and I certainly agree that it is a factor in terms of both posters in general, as well as posters who replied to the survey. All I am saying is that to sum it up as you did (paraphrased: only those with problems with the product would come to the forum) is patently false, myself being proof of that.

I wish I could weigh in on this topic, but NDA so... :-X

Didn't you ragequit up thread? Guess that didn't last!

0B, FastJack, Reaver and all you other guys talking hard statistical deep dive, the precision you are taking your analysis too is legit out of my depth, but I appreciate you doing it. Regardless of how representative of the entire fan base / consumer base / whatever the survey accurately represents, what demographic of those bases do you believe are the types that would take the survey? Like are you saying only the most embittered who want to see SR6 crash and burn would bother taking the survey, or only those more invested than a surface level would take the time, ect?

Title: Re: Fun with Surveys
Post by: FastJack on <09-18-20/1130:44>
Didn't you ragequit up thread? Guess that didn't last!
Please stop antagonizing MC or an official warning will come down.
Title: Re: Fun with Surveys
Post by: Lormyr on <09-18-20/1244:36>
Whatever man, do it. I don't see you "come down" on him when he is antagonizing the rest of us.
Title: Re: Fun with Surveys
Post by: FastJack on <09-18-20/1253:59>
Whatever man, do it. I don't see you "come down" on him when he is antagonizing the rest of us.
I'm not reading through every single thread. This one you did right in front of me. And if he comes down, there's this handy little "report to moderator" button in the forum post and I guarantee action will be taken as needed.

And, before you start the "favoritism" argument, I'd point out that MC has had four official warnings in the last 18 months while you've had one.
Title: Re: Fun with Surveys
Post by: Lormyr on <09-18-20/1255:44>
Sure, but I can handle it myself, I don't need to summon daddy to hide behind your skirt.
Title: Re: Fun with Surveys
Post by: FastJack on <09-18-20/1259:34>
Sure, but I can handle it myself, I don't need to summon daddy to hide behind your skirt.
Official warning #2 sent. Care to try again and go for a five day ban?
Title: Re: Fun with Surveys
Post by: Lormyr on <09-18-20/1305:50>
Do what you got to do man. I'll do the same.

Edit: Just saw your edit above. No, I don't think it has to do with favoritism, nor have a I said as much. I totally buy that your eyes aren't everywhere and you don't see it all, that stands to reason. The hiding behind your skirt comment was also not a dis on you. My point is when I talk shit I am prepared to take my licks. I don't need someone else to handle my battles for me.
Title: Re: Fun with Surveys
Post by: FastJack on <09-18-20/1310:32>
Do what you got to do man. I'll do the same.
See, THAT is not a post that receives any action. Why? Because you didn't insult anyone.

Let me be clear to everyone. "Handling it yourself" leads to only getting yourself warned/banned. By NOT using the tools given to you, you are not being "tough" or anything, you're just making it more difficult to moderate. If MC is such a burden to everyone, and you report his posts, and the reports have merit, he will be punished for those posts. If you think we are moderating only those that dislike 6E, you're very mistaken. We moderate based on actions and attitude.
Title: Re: Fun with Surveys
Post by: Lormyr on <09-18-20/1319:06>
My man, being tough or similar has nothing to do with it, nor is it applicable. No one is tough on the internet. It's about self respect. If that doesn't make sense to you, then I probably won't be able to explain it in a way that will with any more detail.

This has derailed 0B's good works enough though. If you want to continue you know my PM.
Title: Re: Fun with Surveys
Post by: 0B on <09-18-20/1932:59>
Remember folks, this is the "Fun" with surveys thread... I feel like every other thread, folks come in with their own baggage and history. I like hearing other folks' perspectives and Reaver and FastJack have been super helpful. A lot of the problems with my original survey were because I didn't set it up right to begin with (IE, the "It's complicated" answer being inherently complicated). I know I get frustrated if I don't think I'm getting my point across, "social stuff" isn't really my thing. I probably am too attached to the survey because I'm the one who made it, so I should try and listen to others more.

I think that there are issues with the dataset, but not to the extent that this is "unuseful."

However, there are definitely ways it could be improved. Honestly, the whole question about what you like/dislike about the edition isn't helpful because it's not specific enough: IE, one respondent said they disliked "matrix," but then clarified that they really just didn't like technomancers. It's possible others felt the same but didn't clarify.

Convenience sampling bias is a thing. I'd imagine that people who are really interested or really disinterested in the topic are more likely to respond. However... for what purpose? I'd also be curious how you would do a voluntary, true-random survey, while still targeting these specific populations.

Also, I got a new survey out- this one's (http://"https://docs.google.com/forms/d/e/1FAIpQLSdlJR3AshuRh7Q7lhXMjEK-cp3vW6H6sVuakgGuav8PYwvJUg/viewform") about your favorite editions


Link isnt workin :/


I also did a poll on 6th ed quite  a while ago.
https://forums.shadowruntabletop.com/index.php?topic=30525.0

The set up is somewhat different. And also the purpose.

Apologies! Didn't see this until after work. Link is: https://docs.google.com/forms/d/e/1FAIpQLSdlJR3AshuRh7Q7lhXMjEK-cp3vW6H6sVuakgGuav8PYwvJUg/viewform?usp=sf_link
Title: Re: Fun with Surveys
Post by: sagehoge on <09-19-20/0207:20>
shadowrun is fun :)
Title: Re: Fun with Surveys
Post by: 0B on <09-19-20/1418:29>
Convenience sampling bias is a thing. I'd imagine that people who are really interested or really disinterested in the topic are more likely to respond. However... for what purpose? I'd also be curious how you would do a voluntary, true-random survey, while still targeting these specific populations.

Also, I got a new survey out- this one's (http://"https://docs.google.com/forms/d/e/1FAIpQLSdlJR3AshuRh7Q7lhXMjEK-cp3vW6H6sVuakgGuav8PYwvJUg/viewform") about your favorite editions
The only way to get the random sampling is to either go to GenCon or something similar and survey passerbys, or to take a random sample of your sample (but that requires a lot more responses).

The issue there is that you'd be leaving out the European community. However, their books are different from ours so that might be an OK thing to do.

I think it's fair to say that not all of the US playerbase is represented by con-goers. You may miss out on the under-18 players, or on players whose jobs do not permit them a lot of freetime. Both of these groups may have different experiences with the game (How does a tight schedule affect whether someone wants to dedicate time to picking up a new edition, how does less money affect when a player decides to buy a new book, etc.)
Title: Re: Fun with Surveys
Post by: wraith on <09-21-20/0303:22>
Convenience sampling bias is a thing. I'd imagine that people who are really interested or really disinterested in the topic are more likely to respond. However... for what purpose? I'd also be curious how you would do a voluntary, true-random survey, while still targeting these specific populations.

Also, I got a new survey out- this one's (http://"https://docs.google.com/forms/d/e/1FAIpQLSdlJR3AshuRh7Q7lhXMjEK-cp3vW6H6sVuakgGuav8PYwvJUg/viewform") about your favorite editions
The only way to get the random sampling is to either go to GenCon or something similar and survey passerbys, or to take a random sample of your sample (but that requires a lot more responses).

The issue there is that you'd be leaving out the European community. However, their books are different from ours so that might be an OK thing to do.

I think it's fair to say that not all of the US playerbase is represented by con-goers. You may miss out on the under-18 players, or on players whose jobs do not permit them a lot of freetime. Both of these groups may have different experiences with the game (How does a tight schedule affect whether someone wants to dedicate time to picking up a new edition, how does less money affect when a player decides to buy a new book, etc.)

Yeah, given that Pegasus does separate German language specific content as well as significantly re-editing SR books to a much higher standard as part of their translation process, it would be very hard to make a 1:1 comparison.  They're effectively seperate but closely related games.
Title: Re: Fun with Surveys
Post by: 0B on <09-21-20/0804:12>
I might have to walk back my statement, though- recently got a PM that a lot of European gamers outside of Germany and France use the English versions of the books, since it's more common for them to have English as a second language than German.

Either way, I think this just shows the difficulty of getting a true sampling of the entire Shadowrun community, either random or via convenience.
Title: Re: Fun with Surveys
Post by: Michael Chandra on <09-21-20/0825:17>
Pegasus 'Since Orichalcum was made more expensive in SR5, let's take this 4m-per-dram reagent and ALSO make it way more expensive!' is not my primary booksource as European, and buying that book really made me regret bothering to give them money.
Title: Re: Fun with Surveys
Post by: 0B on <09-21-20/2159:39>
Oh, I'm sure folks have scar tissue over that edition, and probably over French versions as well. I haven't read either, I can't say.

The point is, different books may have different reactions. I think it would be a mistake to ignore those parts of the SR community, even though it does make gauging edition popularity more difficult. It's also possible that the quality is the same across the board but popularity is different, or that popularity is the same across the board but quality is different.

You'd be hard pressed to find an objective measure of how "good" a book is, but it's easier to gauge how many people like an edition. Except, of course, when different translations have different rules. Hopefully the decisions on when rules should be different aren't based on how easy it is for some rando to make a survey on it ;)
Title: Re: Fun with Surveys
Post by: 0B on <09-26-20/1156:03>
Survey results (https://docs.google.com/presentation/d/1l_gVMQxWGbiaV_pGIFp3Y6NgDoiXOyEjiTlcSIb2kg4/edit?usp=sharing)

Thanks to penllawen for editing!

TL;DR

Title: Re: Fun with Surveys
Post by: Lormyr on <09-26-20/1233:22>
That shows SR6 waaaay behind everything else in the likes/dislikes ratio. Can't say that is surprising. I haven't played anything other than 5/6 though, but it makes me wish I had seen 3rd.
Title: Re: Fun with Surveys
Post by: adzling on <09-26-20/1239:57>
I think that 6e is still in it's first printing (i.e. they still have stock of first printing) is also very telling.
Title: Re: Fun with Surveys
Post by: Stainless Steel Devil Rat on <09-26-20/1243:05>
That shows SR6 waaaay behind everything else in the likes/dislikes ratio. Can't say that is surprising. I haven't played anything other than 5/6 though, but it makes me wish I had seen 3rd.

Other than the matrix rules changing whole-cloth each edition, 1e thru 3e were pretty similar.  Each one was basically an errata-included version of the previous.  4th was the first time things really got shook up... 1-3 to 4 was a bigger change than 5 to 6 was... but it's pretty close to how big a shakeup it was.
Title: Re: Fun with Surveys
Post by: 0B on <09-26-20/1325:10>
I think that 6e is still in it's first printing (i.e. they still have stock of first printing) is also very telling.

Some of that can be attributed to printer slowdowns with Covid. On the other hand, 6e came out in August 2019 and the slowdowns didn't really start until late February 2020. You would have expected at least a second printing in the first 6 months
Title: Re: Fun with Surveys
Post by: Marcus on <09-26-20/1441:25>
3rd is a very different game from 4th. To me the lesson from 3rd that  always amused me was that 6 and 7 are the same number. Its odd and realization but it actually has meaning.
Title: Re: Fun with Surveys
Post by: Shinobi Killfist on <09-26-20/1450:59>
I'm kind of surprised with the like of the early editions. I prefer them myself, but I figured they'd were old enough they'd have faded out.
Title: Re: Fun with Surveys
Post by: Shinobi Killfist on <09-26-20/1454:09>
That shows SR6 waaaay behind everything else in the likes/dislikes ratio. Can't say that is surprising. I haven't played anything other than 5/6 though, but it makes me wish I had seen 3rd.

Other than the matrix rules changing whole-cloth each edition, 1e thru 3e were pretty similar.  Each one was basically an errata-included version of the previous.  4th was the first time things really got shook up... 1-3 to 4 was a bigger change than 5 to 6 was... but it's pretty close to how big a shakeup it was.

Yeah and I think people will have a better chance due to their similarities to like all 3, maybe have a preference, but maybe have things like preferring 3e skills, liking 1es vehicle/hardened armor rules(AKa when they juggernaut was a juggernaut) etc but overall prefer 2e.
Title: Re: Fun with Surveys
Post by: Shinobi Killfist on <09-26-20/1502:26>
That shows SR6 waaaay behind everything else in the likes/dislikes ratio. Can't say that is surprising. I haven't played anything other than 5/6 though, but it makes me wish I had seen 3rd.

If I'm reading it right its the only edition that has more dislikes than likes. Its not a huge scientific study but its not irrelevant either, for a niche hobby the sample size isn't bad and it went across multiple sources.

Title: Re: Fun with Surveys
Post by: 0B on <09-26-20/1630:50>
That shows SR6 waaaay behind everything else in the likes/dislikes ratio. Can't say that is surprising. I haven't played anything other than 5/6 though, but it makes me wish I had seen 3rd.

If I'm reading it right its the only edition that has more dislikes than likes. Its not a huge scientific study but its not irrelevant either, for a niche hobby the sample size isn't bad and it went across multiple sources.

Thanks! And you're right, when you compare it to how every other edition did, there is no overlap of confidence intervals. Therefore, every other edition is "liked" more in the total online population with 95% confidence.

And MC did bring up a good point about some of the sampling bias on SCN discord (Along with unnecessary comments about my "trustworthiness"): I included a lot of 5e LCs, but only one 6e LC. Well, there's only one 6e LC. MC also said that reddit could be biased, which is also possible.

I ended up adding a slide for RPG.net, which "should" be the most neutral for SR editions. Even if you've been "run off" other forums due to hostility about the edition, RPG.net shouldn't have any bias one way or the other about editions.

(https://i.imgur.com/lZVEauk.png)

I was worried since RPG.net only had 80 respondents, but at least 30 responded to each category so we should be good with statistical significance. There's a brief run-down on skew and Clopper-Pearson in the speaker-comments for slide 6.

We get the same results: with 95% confidence, 6e is less "liked" than every other edition for RPG.net.
Title: Re: Fun with Surveys
Post by: jim1701 on <09-28-20/2104:17>
I'm somewhat surprised how well received 5th edition was.  That's the one that killed new editions of Shadowrun for me.  Of course if the quality level had been better I probably would have liked it more.  Well that and if the priority system were better balanced.   
Title: Re: Fun with Surveys
Post by: 0B on <09-29-20/1249:34>
I'm somewhat surprised how well received 5th edition was.  That's the one that killed new editions of Shadowrun for me.  Of course if the quality level had been better I probably would have liked it more.  Well that and if the priority system were better balanced.

If you look at who I polled, most of the discords were 5E LC's. Granted, that's because most LC's are 5E, and there's maybe 6 regular players on the 6e LC. The RPG.net stats (Shown above) don't paint 5e in as bright a light. In fact, it's enough that we can say with p < 0.05 that the opinion of 5E on RPG.net versus all locations surveyed is significantly different. In other words, there is no overlap on the margin of error.

I don't know if this means that the places I surveyed overall like 5e more than "normal" for the online SR community, or if RPG.net likes it less than "normal" for the online SR community. It's also worth noting that the RPG.net results are at the bare minimum for us to make statistical judgements about.

I also would avoid conflating quality with how well liked an edition is.
Title: Re: Fun with Surveys
Post by: jim1701 on <09-29-20/1340:29>
I'm somewhat surprised how well received 5th edition was.  That's the one that killed new editions of Shadowrun for me.  Of course if the quality level had been better I probably would have liked it more.  Well that and if the priority system were better balanced.

If you look at who I polled, most of the discords were 5E LC's. Granted, that's because most LC's are 5E, and there's maybe 6 regular players on the 6e LC. The RPG.net stats (Shown above) don't paint 5e in as bright a light. In fact, it's enough that we can say with p < 0.05 that the opinion of 5E on RPG.net versus all locations surveyed is significantly different. In other words, there is no overlap on the margin of error.

I don't know if this means that the places I surveyed overall like 5e more than "normal" for the online SR community, or if RPG.net likes it less than "normal" for the online SR community. It's also worth noting that the RPG.net results are at the bare minimum for us to make statistical judgements about.

I also would avoid conflating quality with how well liked an edition is.

Agreed.  It was just that quality (or the lack thereof) in the editing and production values is what turned me personally off 5e and subsequent editions.  Didn't mean to imply anything beyond that.
Title: Re: Fun with Surveys
Post by: 0B on <09-29-20/1351:46>
I'm somewhat surprised how well received 5th edition was.  That's the one that killed new editions of Shadowrun for me.  Of course if the quality level had been better I probably would have liked it more.  Well that and if the priority system were better balanced.

If you look at who I polled, most of the discords were 5E LC's. Granted, that's because most LC's are 5E, and there's maybe 6 regular players on the 6e LC. The RPG.net stats (Shown above) don't paint 5e in as bright a light. In fact, it's enough that we can say with p < 0.05 that the opinion of 5E on RPG.net versus all locations surveyed is significantly different. In other words, there is no overlap on the margin of error.

I don't know if this means that the places I surveyed overall like 5e more than "normal" for the online SR community, or if RPG.net likes it less than "normal" for the online SR community. It's also worth noting that the RPG.net results are at the bare minimum for us to make statistical judgements about.

I also would avoid conflating quality with how well liked an edition is.

Agreed.  It was just that quality (or the lack thereof) in the editing and production values is what turned me personally off 5e and subsequent editions.  Didn't mean to imply anything beyond that.

Ope, my bad! And you'll get no quarrel from me on the quality of 5e's editing and layout, nor on 6e's. I just think there are much better metrics on that then this poll.
Title: Re: Fun with Surveys
Post by: Reaver on <09-30-20/1459:43>
Some of those numbers from the 2nd poll make some sense to me.

1e was the original game. And when you look at it, it was tight, clean, and easy to run.. but lacking of the overall depth of world. (which is to be expected, as the game was fresh and new and hadn't nailed down all the lore yet.)

2e was mostly just a revision of 1e, with some changes to thresholds, target numbers, and modifiers. But you also started to see the Lore of SR get filled in and the "world" starting to flesh out more.

3e: was just an improvement on the mechanics of 1/2e. There was some changes for balance, and a reworking for the initiative system. But for the most part, it was the same game. World building was also at it's peak during the 3e run... and a lot of the lore and mystery of the SR universe was introduced in 3e.

4e: was a total reworking for Shadowrun... And changed everything... But the changes where also a bit of a mess that didn't work well under the hood.

4eA: fixed a lot of the "under the hood" issues that 4e broke, but still had lots of small issues (especially if you were transferring characters from 3e!)

5e: Well, it tried to "fix" some of the core issues of 4e, and it did that well enough, but introduced new issues.....

6e: is again a total rework of the edition rules...


However, there is a large underlining issue for the last 3 editions. Editing.
The editing int he last 3 editions has been poor. (Not even FanPro did a good job).... Which seems to be a lot of people's issues...

Which on one hand I get, and the other I don't. When i look back at all the RPGs I play/played.. none of them have been "great" in the editing department, but then again, I have been gaming for so long, I might have just gotten used to the poor editing and compensate for it after coming across it from so many companies over 3+ decades of gaming.. (anyone remember the old TSR books? How about White Wolf's original works? Palladium? Games Workshop's 2e books? Jackson Games GURPS original books? But then again, they got better... ) 
Title: Re: Fun with Surveys
Post by: wraith on <10-03-20/0300:08>
You hit on the important part there. "They got better."

The TTRPG market is undergoing a renaissance right now, there are huge numbers of new games on the market, mostly put out by small teams with shoestring budgets.  If they consider good editing and layout to be make or break, and  can execute on it, why is worse acceptable from a well known studio with actual production budgets and which we know pays most of its writers as freelancers?
Title: Re: Fun with Surveys
Post by: 0B on <10-03-20/1358:35>
A lot of the decisions surround 6e are confusing. The goal of it overall was to make the rules shorter (Since that was one of the complaints of 5e). Of course you'll run into issues with some rules like Multi-Attack being vague, but that isn't game-breaking.

But for a game that wanted to make rules shorter, it dragged out a lot of things. It doesn't talk about what a TTRPG is until the reader's gone through sections on lore and a short story. It's not the only edition that did this, but there are others that are better:

1e: p. 20, 15th page after Table of Contents (ToC).
2e: p. 10, 4th page after ToC, 3rd if you ignore a splash page of art.
3e: p. 8, 3rd page after ToC, arguably p. 6/1st page after ToC since that page goes over how to read the book.
4e (fanpro): p. 16, 3rd page after ToC, 2nd if you ignore a splash page of art.

4e (20A): p. 15, 10th page after ToC.
5e: p. 44, 37th page after ToC.
6e: p. 34, 28th page after ToC. 6e doesn't give a pass like 3e does for offering a roadmap on the first page since it insists that new players read through the fiction first before they get to the rules. This is arguably worse than not offering a roadmap at all, since a new player might conceivably skip through lore to get to rules (even unprompted), but now this tells them not to do that.

The editions in bold are the ones that did a good job of introducing what the reader is looking at early. You might say "well, a normal player would skip past the lore if they wanted to read the mechanics." Which seems reasonable, until you consider that the type of player who doesn't know what a TTRPG is would not know to do that, and therefore we cannot count on them to do that.

It's true that TTRPGs are mostly an oral tradition, but they don't always have to be. Sacrificing function for form by front-loading lore, world-building, and short stories is not a good thing. Every board game must be written as if it's the first board game someone's ever played. TTRPGs get a little bit of leniency on this, but only because just about every TTRPG out there commits this design sin. (Well, except the ones bolded above). I'm sure folks would also have an easier time introducing the game to parents, siblings, and friends if the rules were spelled out cleanly, and didn't require years of TTRPG experience to parse and make rulings on.

It's true that 6e isn't "as bad" as 5e with this, but the fact that you still need to go almost 30 pages to learn what a TTRPG is indicates to me that this wasn't a deliberate decision, it was incidental to trimming the fat in other places. The above should also explain why I think "better than 5e" is a very low bar when talking about different aspects of 6e. Sure it's "better than 5e," I'll bet that it's also better than the fantasy-heartbreaker game I made in middle school. 30% on a test is in fact better than 20% on a test. It's still bad.

This is just one aspect- I'm sure other folks can find places where it looks like Shadowrun "forgot" different lessons about game design. But the editions bolded happen to be the ones that were 1) After the very first edition of SR was published, and 2) When the bulk of SR's devs with "collective knowledge" were still working on the project. Even 4e 20A isn't as bad as the other editions, enough that it indicates to me that this was thought about, and that the decision to put fiction in before explaining TTRPGs was deliberate. You don't have to come to the same conclusions as me about when the right time is to explain what a TTRPG is, but you ought to understand why it's important.

At the same time: All the stuff with the pool embezzlement whatnot happened 10 years ago. There was less time than that in-between 1e and 2e, but FASA still started applying lessons-learned about RPG design. Why hasn't that happened here?
Title: Re: Fun with Surveys
Post by: adzling on <10-04-20/1153:50>
A line dev that does not know what he is doing and a management that does not give two figs about shadowrun.

That's been my experience from working with/ interacting with the management of Catalyst.
Title: Re: Fun with Surveys
Post by: wraith on <11-13-20/2119:20>
A line dev that does not know what he is doing and a management that does not give two figs about shadowrun.

That's been my experience from working with/ interacting with the management of Catalyst.

Yup.  The snake rots from the head, and there's nothing a bunch of freelancers can do when the guy in charge of the line doesn't give a fuck.
Title: Re: Fun with Surveys
Post by: FastJack on <11-13-20/2250:00>
Thread's closed, the survey was done and we don't need any more input, especially when it's negative, insulting, and just plain non-helpful.