NEWS

An update on SR6e DriveThruRPG ratings

  • 188 Replies
  • 40256 Views

penllawen

  • *
  • Omae
  • ***
  • Posts: 804
  • Let's go. In and out. Twenty minute milk run.
« on: <09-12-19/0559:39> »
A few weeks ago, FastJack said:
Quote
Was my wording off, or were a lot of people surprised that sixth edition was more popular than they thought it would be? I suspect a lot of people are feeling shocked that Sixth World is the number one download on DriveThruRPG ever since it came out.

This made me curious so I compiled a list of the top 10 on DTRPG at the time, complete with the star rating that is displayed next to each item. This wasn't a good showing for SR6, with a rating at the time of 3.4/5 -- the next lowest in the top 10 was 4.8/5.

However, that was based on a scant four days of sales, so I made a note to revisit it when a few weeks had passed. I've now done this, and here it is. As you can see, SR6e has fallen from the #1 spot and is receiving a bit of a kicking from DTRPG reviewers.

   Position      Title      Score      Number of ratings      Release date   
   1      Cyberpunk Red Jumpstart Kit      4.8/5      42      August 01, 2019   
   2      Shadowrun, Sixth World Core Rulebook      2.8/5      19      August 26, 2019   
   3      Eclipse Phase Second Edition      5/5      13      August 09, 2019   
   4      Legend of the Five Rings: Courts of Stone      5/5      2      August 27, 2019   
   5      The Short Games Digest: Volume 2      5/5      5      September 09, 2019   
   6      Bayt al Azif #2: A magazine for Cthulhu Mythos roleplaying games      n/a      n/a      September 09, 2019   
   7      Stars Without Number: Revised Edition      4.9/5      154      December 29, 2017   
   8      Occult Philosophy      5/5      24      July 29, 2019   
   9      Rangers of Shadow Deep: Ghost Stone      4.3/5      3      August 02, 2019   
   10      Esper Genesis 5E Threats Database      5/5      6      August 15, 2019   
   11      Rangers of Shadow Deep: A Tabletop Adventure Game      4.8/5      62      November 02, 2018   
   12      Cyberpunk 2.0.2.0. The Second Edition, Version 2.01      4.5/5      79      March 09, 2014   
   13      Lords and Lands: a Witcher TRPG Expansion      3.3/5      12      August 01, 2019   
   14      Apocalypse World: Burned Over Hackbook      5/5      3      August 01, 2019   
   15      Star Trek Adventures: Alpha Quadrant Source Book      4.3/5      3      July 25, 2019   



FastJack

  • *
  • Administrator
  • Prime Runner
  • *****
  • Posts: 6367
  • Kids these days...
« Reply #1 on: <09-12-19/0820:53> »
Well, the reason the Cyberpunk Red retook the top spot is because they cut the price in half this past week.

As for reviews... The Streetpedia, which everyone here loved, has a 2.9/5 rating.

penllawen

  • *
  • Omae
  • ***
  • Posts: 804
  • Let's go. In and out. Twenty minute milk run.
« Reply #2 on: <09-12-19/0837:30> »
Well, the reason the Cyberpunk Red retook the top spot is because they cut the price in half this past week.
Incorrect. The PDF version was always $15, the $30 comparative price shown on DTRPG is for the physical release. Citation: many places, but for example, IGN: "The digital edition, available as of August 1, 2019, will retail for $15.00 USD and the physical edition (available exclusively at GenCon 2019 initially then at your local hobby shop) will cost $29.99."

Quote
As for reviews... The Streetpedia, which everyone here loved, has a 2.9/5 rating.
So there's nothing to see here, you think? Nothing worrying at all about those reviews?

Michael Chandra

  • *
  • Catalyst Demo Team
  • Prime Runner
  • ***
  • Posts: 9920
  • Question-slicing ninja
« Reply #3 on: <09-12-19/0856:30> »
Well, the reason the Cyberpunk Red retook the top spot is because they cut the price in half this past week.

As for reviews... The Streetpedia, which everyone here loved, has a 2.9/5 rating.
It's not as if the review-count is high enough to be significant. Plus a lot of people have 6w free through another store or official channels and cannot review it. So the review-score is just a number, not reliable at all. :-\
How am I not part of the forum?? O_O I am both active and angry!

penllawen

  • *
  • Omae
  • ***
  • Posts: 804
  • Let's go. In and out. Twenty minute milk run.
« Reply #4 on: <09-12-19/0859:42> »
It's not as if the review-count is high enough to be significant.
I disagree, I think the sample size is becoming large enough. But sure, we can give it a few more weeks and check back in.

So the review-score is just a number, not reliable at all. :-\
I imagine the review score is much more than "just a number" for potential purchasers browsing DTRPG. It's very prominent in the UI.

Lormyr

  • *
  • Catalyst Demo Team
  • Omae
  • ***
  • Posts: 820
« Reply #5 on: <09-12-19/0924:14> »
I'd love to see this revisited in another month.
"TL:DR 6e's reduction of meaningful choices is akin to forcing everyone to wear training wheels. Now it's just becomes a bunch of toddlers riding around on tricycles they can't fall off of." - Adzling

penllawen

  • *
  • Omae
  • ***
  • Posts: 804
  • Let's go. In and out. Twenty minute milk run.
« Reply #6 on: <09-12-19/0925:08> »
I'd love to see this revisited in another month.
Your wish is my command ;)

steelybran

  • *
  • Newb
  • *
  • Posts: 53
« Reply #7 on: <09-12-19/0936:11> »
Reviews by fans can be a mixed bag, especially if you're revising the system.  I'm not saying good or bad, I'm just saying you're likely to get a very galvanized response.

The Tekwych

  • *
  • Omae
  • ***
  • Posts: 400
« Reply #8 on: <09-12-19/0938:34> »
With several hundred purchases fro drivethru I don’t think I have ever once looked at a rating or a review. That said what you don’t have is sales numbers. A review sample that is 60% of sales shows that people care more about the product, good or bad, than a sample that is 2% of sales ( a more realistic number). SR6 CRB is still on the top sellers list so reviews are not stoping any purchases

penllawen

  • *
  • Omae
  • ***
  • Posts: 804
  • Let's go. In and out. Twenty minute milk run.
« Reply #9 on: <09-12-19/0948:17> »
That said what you don’t have is sales numbers.
We have relative sales numbers. We know the Cyberpunk Red quick-start PDF is outselling the SR6 CRB PDF, despite the former having been out almost a month longer.

Plan_B

  • *
  • Newb
  • *
  • Posts: 43
« Reply #10 on: <09-12-19/1102:01> »
My problem with this is that the distribution is not presented. At this moment, there are 22 reviews that give it a 3/5 star rating. Those reviews are 8 4-5 star ratings to 4 1-2 star ratings with 0 ratings of 3 stars. Of the ratings given, there is a 2:1 ratio with a positive opinion of the book. If we are going to talk about sales numbers and ratings scores, let's also look at what those ratings are and what their distribution is.
When the "milk run" goes sour, it's time for Plan B!

mcv

  • *
  • Chummer
  • **
  • Posts: 202
« Reply #11 on: <09-12-19/1112:57> »
   Position      Title      Score      Number of ratings      Release date   
   1      Cyberpunk Red Jumpstart Kit      4.8/5      42      August 01, 2019   
   2      Shadowrun, Sixth World Core Rulebook      2.8/5      19      August 26, 2019   
   3      Eclipse Phase Second Edition      5/5      13      August 09, 2019   
Just ignoring the review scores for a moment, just look at that top 3! Cyberpunk Red, Shadowrun and Eclipse Phase dominating the sales. Could this be a revival of the cyberpunk genre? Sure, new editions of course, but still.

penllawen

  • *
  • Omae
  • ***
  • Posts: 804
  • Let's go. In and out. Twenty minute milk run.
« Reply #12 on: <09-12-19/1140:20> »
My problem with this is that the distribution is not presented. At this moment, there are 22 reviews that give it a 3/5 star rating.
Interesting. That means since I posted this thread, it's received three more reviews, and the mean has moved from 2.8 to 3.0.

penllawen

  • *
  • Omae
  • ***
  • Posts: 804
  • Let's go. In and out. Twenty minute milk run.
« Reply #13 on: <09-12-19/1142:53> »
Plus a lot of people have 6w free through another store or official channels and cannot review it.
This would only matter if there's a cohort skew difference between people who got it from DTRPG and people who got it via other sources. Unless there's a reason for systemic bias I can't see, it's reasonable to assume the DTRPG purchasers are representative of purchasers on other platforms. Can you see a reason why DTRPG purchases would be more likely to leave a negative or positive review than purchasers through other channels?

Banshee

  • *
  • Catalyst Demo Team
  • Ace Runner
  • ***
  • Posts: 1095
« Reply #14 on: <09-12-19/1152:23> »
Plus a lot of people have 6w free through another store or official channels and cannot review it.
This would only matter if there's a cohort skew difference between people who got it from DTRPG and people who got it via other sources. Unless there's a reason for systemic bias I can't see, it's reasonable to assume the DTRPG purchasers are representative of purchasers on other platforms. Can you see a reason why DTRPG purchases would be more likely to leave a negative or positive review than purchasers through other channels?

Actually yes I can, people who only buy from the CGL store may be more positively biased, as well as a difference between people who only buy online when compared to people who only buy thru FLGS. There also very significant bias  when you don't know the reasons for the reviews (good or bad) as well as a major deviation factor due to ratio of responses to sales. Not saying it wrong, but also says it does not represent the actual market either ... just missing too much data. The only thing this particular rating shows is that it is selling well on DTRPG.
Robert "Banshee" Volbrecht
Freelancer & FAQ Committee member
Former RPG Lead Agent
Catalyst Demo Team