Pages: [1] 2 :: one page |
|
Author |
Thread Statistics | Show CCP posts - 0 post(s) |

Skihids
Bullet Cluster
1834
 |
Posted - 2013.08.08 17:46:00 -
[1] - Quote
I once killed a Madruger with a MD.
You instinctively realize that says nothing about the relative merits of the MD vs. the HAV. One anecdote with no accounting for any other factor imparts zero information. For all you and I know the driver had to AFK to attend to a real life emergency.
Yet these forums are replete with case after case of "proof" that one or another weapon is OP. The latest is where Xero attempts to prove that the MD is OP by showing us a single score of 32/3. A number of folks instinctively realized there were confounding factors such as running with a full proto-bear squad against randoms, but not being statisticians they could not fully articulate the the complete failure to provide evidence of his claim.
CCP has several statisticians on its staff, so you are going to have to do better than that if you want to convince them.
At the very least what you need to do is run multiple experiments with the two comparison weapons, the AR and MD in this case. You then measure the characteristics of the two populations of measurements and use a statistical test to determine if there is a significant difference between the two.
The two major characteristics of a population is the mean (average) and standard deviation (the measure of how spread out the data is). You can then use a simple T-test to determine if the two populations of measurements are different or not.
Now what usually happens is that you find that the variation in measurements (number of kills per match) obscures any difference. In that event you can attempt to remove the confounding factors (number and strength of players in your squad and team, length of match, presence of vehicles, strength and experience of the opposing force, type of match, etc.) to isolate the effect of the weapon. This is done with a randomized block design.
I'm not expecting everyone here to be able to design and run a proper experiment. I would be satisfied if more people were simply aware of the requirements for a proper statistical proof that could be used to balance with and refrain from polluting the forums with anecdotal accounts and calling them proof.. |

Aisha Ctarl
The Generals EoN.
972
 |
Posted - 2013.08.08 17:50:00 -
[2] - Quote
I think.....I think we all just got schooled with arithmetic guys. |

Buster Friently
Rosen Association
1497
 |
Posted - 2013.08.08 17:50:00 -
[3] - Quote
The closest we've come to real data so far:
https://forums.dust514.com/default.aspx?g=posts&t=92678
and:
https://forums.dust514.com/default.aspx?g=posts&t=94246
Obviously CCP has better data. |

Skihids
Bullet Cluster
1835
 |
Posted - 2013.08.08 17:55:00 -
[4] - Quote
Aisha Ctarl wrote:I think.....I think we all just got schooled with arithmetic guys.
It's simple statistics and data evaluation, and it is actually as interesting as it is useful. I minored in applied statistics in my undergrad and it comes in handy even though I'm not a statistician by trade.
Familiarity with just a few basic statistical concepts can help you evaluate claims such as one weapon being OP in relation to another, and helps you separate fact from opinion. |

Skyhound Solbrave
Rough Riders..
171
 |
Posted - 2013.08.08 18:23:00 -
[5] - Quote
The sad part is that when someone gives a bullet proof logical argument, the other side tends to shut down and not answer it. As a result, it is forgotten and the idiotic argument on Xero's post continues. |

Vickers S Grunt
Expert Intervention Caldari State
193
 |
Posted - 2013.08.08 18:29:00 -
[6] - Quote
http://www.quickmeme.com/meme/3vfuxx/ |

Mobius Wyvern
Guardian Solutions
2949
 |
Posted - 2013.08.08 18:31:00 -
[7] - Quote
Skihids wrote:I once killed a Madruger with a MD. You instinctively realize that says nothing about the relative merits of the MD vs. the HAV. One anecdote with no accounting for any other factor imparts zero information. For all you and I know the driver had to AFK to attend to a real life emergency. Yet these forums are replete with case after case of "proof" that one or another weapon is OP. The latest is where Xero attempts to prove that the MD is OP by showing us a single score of 32/3. A number of folks instinctively realized there were confounding factors such as running with a full proto-bear squad against randoms, but not being statisticians they could not fully articulate the the complete failure to provide evidence of his claim. CCP has several statisticians on its staff, so you are going to have to do better than that if you want to convince them. At the very least what you need to do is run multiple experiments with the two comparison weapons, the AR and MD in this case. You then measure the characteristics of the two populations of measurements and use a statistical test to determine if there is a significant difference between the two. The two major characteristics of a population is the mean (average) and standard deviation (the measure of how spread out the data is). You can then use a simple T-test to determine if the two populations of measurements are different or not. Now what usually happens is that you find that the variation in measurements (number of kills per match) obscures any difference. In that event you can attempt to remove the confounding factors (number and strength of players in your squad and team, length of match, presence of vehicles, strength and experience of the opposing force, type of match, etc.) to isolate the effect of the weapon. This is done with a randomized block design. I'm not expecting everyone here to be able to design and run a proper experiment. I would be satisfied if more people were simply aware of the requirements for a proper statistical proof that could be used to balance with and refrain from polluting the forums with anecdotal accounts and calling them proof.. Posting in an intelligent thread. |

Aero Yassavi
PIE Inc. Praetoria Imperialis Excubitoris
1123
 |
Posted - 2013.08.08 18:32:00 -
[8] - Quote
I once killed a Madruger with a scrambler pistol.
No joke, it was already damaged and I threw some packed AVs at it but it still had a sliver of armor let, fired my scrambler at it and it gave the pistol credit for the kill. |

Skihids
Bullet Cluster
1842
 |
Posted - 2013.08.08 18:33:00 -
[9] - Quote
Skyhound Solbrave wrote:The sad part is that when someone gives a bullet proof logical argument, the other side tends to shut down and not answer it. As a result, it is forgotten and the idiotic argument on Xero's post continues.
That's why I made this a separate topic. I didn't want it to be buried in the mud of bad arguments.
Interestingly you can see the variation I am speaking of in his first score. He went 13 and was in the middle of the pack of his team. So we have two data points, 13 and 32 kills. That's a spread of 19 points. |

calisk galern
BurgezzE.T.F
679
 |
Posted - 2013.08.08 18:35:00 -
[10] - Quote
I went 2/4 yesterday in one match, sniper rifles are UP buff them |
|

CoD isAIDS
Tronhadar Free Guard Minmatar Republic
141
 |
Posted - 2013.08.08 18:41:00 -
[11] - Quote
You probably killed it only because it had 100 armor left. |

Skihids
Bullet Cluster
1844
 |
Posted - 2013.08.08 21:06:00 -
[12] - Quote
The problem here is that certain people start with an opinion and then provide anecdotes as proof that they are correct in their opinion.
Science doesn't work that way.
You start with an idea, then design an experiment to determine if there is sufficient evidence to warrant that being an accurate model of the world or not.
The closest I've seen the argument come to science are the threads discussing damage per second, though it's tough to model every aspect. |

Buster Friently
Rosen Association
1516
 |
Posted - 2013.08.08 21:10:00 -
[13] - Quote
Skihids wrote:The problem here is that certain people start with an opinion and then provide anecdotes as proof that they are correct in their opinion.
Science doesn't work that way.
You start with an idea, then design an experiment to determine if there is sufficient evidence to warrant that being an accurate model of the world or not.
The closest I've seen the argument come to science are the threads discussing damage per second, though it's tough to model every aspect.
Just to be pedantic. Science, ideally, works slightly differently. You come up with a theory, and then test it for failure. The scientific method never proves anything. It can only disprove things. Over time, as more and more incorrect theories are weeded out, we are left with a functional, if not "true", model of whatever is being studied.
Unfortunately, arguments over DPS leave so much out of the equation when it comes to FPSes, that there's very little data to be separated from the opinions. That's at least a starting point based on facts though.
|

brekfest
Ahrendee Frontlinez Omega Commission
4
 |
Posted - 2013.08.08 21:31:00 -
[14] - Quote
"Whatcha got there? Numbers?"
+1 for science |

Skihids
Bullet Cluster
1847
 |
Posted - 2013.08.08 21:39:00 -
[15] - Quote
Buster Friently wrote:Skihids wrote:The problem here is that certain people start with an opinion and then provide anecdotes as proof that they are correct in their opinion.
Science doesn't work that way.
You start with an idea, then design an experiment to determine if there is sufficient evidence to warrant that being an accurate model of the world or not.
The closest I've seen the argument come to science are the threads discussing damage per second, though it's tough to model every aspect. Just to be pedantic. Science, ideally, works slightly differently. You come up with a theory, and then test it for failure. The scientific method never proves anything. It can only disprove things. Over time, as more and more incorrect theories are weeded out, we are left with a functional, if not "true", model of whatever is being studied. Unfortunately, arguments over DPS leave so much out of the equation when it comes to FPSes, that there's very little data to be separated from the opinions. That's at least a starting point based on facts though.
That's true, and real science is even messier than that, but the core notion s that you try to disprove your hypothesis, not cherry pick data that supports your idea. The deal is that individual anecdotes are worth nothing. You need hard data.
People instinctively understand the need for comparison when they tell complainers to use the supposed OP weapon for a while to see how it affects their KDR. It's not a statistically rigorous test but it does use repeated measurements to obtain a new average for the weapon. The idea is that with enough tests (matches) the user will encounter enough conditions that the results will reflect the average reality.
It's not perfect as KDR naturally fluctuates, but it's a good first test. |

Buster Friently
Rosen Association
1517
 |
Posted - 2013.08.08 21:43:00 -
[16] - Quote
Skihids wrote:Buster Friently wrote:Skihids wrote:The problem here is that certain people start with an opinion and then provide anecdotes as proof that they are correct in their opinion.
Science doesn't work that way.
You start with an idea, then design an experiment to determine if there is sufficient evidence to warrant that being an accurate model of the world or not.
The closest I've seen the argument come to science are the threads discussing damage per second, though it's tough to model every aspect. Just to be pedantic. Science, ideally, works slightly differently. You come up with a theory, and then test it for failure. The scientific method never proves anything. It can only disprove things. Over time, as more and more incorrect theories are weeded out, we are left with a functional, if not "true", model of whatever is being studied. Unfortunately, arguments over DPS leave so much out of the equation when it comes to FPSes, that there's very little data to be separated from the opinions. That's at least a starting point based on facts though. That's true, and real science is even messier than that, but the core notion s that you try to disprove your hypothesis, not cherry pick data that supports your idea. The deal is that individual anecdotes are worth nothing. You need hard data. People instinctively understand the need for comparison when they tell complainers to use the supposed OP weapon for a while to see how it affects their KDR. It's not a statistically rigorous test but it does use repeated measurements to obtain a new average for the weapon. The idea is that with enough tests (matches) the user will encounter enough conditions that the results will reflect the average reality. It's not perfect as KDR naturally fluctuates, but it's a good first test.
Well, this is why I, and others, collected the killfeed data. I believe it's the most unbiased data set that we have access too currently.
|

Zero Notion
Red Star Jr. EoN.
175
 |
Posted - 2013.08.08 21:44:00 -
[17] - Quote
Hah, as if 'Math' is real. Numbers are just control, 2+2=5, man!
http://www.youtube.com/watch?v=TbNymweHW4E
Good post. I hope it can provide some insight for other plays; our subjective experiences do not constitute a truth of a thing.
I'm not even sure if I am actually dying more to the MD, only that I'm noticing it being used more often (which has been true to some degree ever since Flaylocks took a hit). This does not mean its OP. Just more popular.
Explosive damage could be considered OP because of the nature of the public games (no FF), as there is no mechanic to help control the spamming of explosives (besides your own shields).
Of course, the more people who call for "Nerfs" and make threads regarding it, the more players who rely on these mechanics to have fun are going to be using it.
|

Skihids
Bullet Cluster
1849
 |
Posted - 2013.08.09 02:37:00 -
[18] - Quote
Interesting video.
Subjective experience is a great starting point, but we need rigorous investigation after that.
The lack of FF is definitely a factor, but I don't think it's a major one. The biggest factor is that it works differently than hit-scan weapons and there are some people who either can't or won't accept that. |

TheAmazing FlyingPig
Crux Special Tasks Group Gallente Federation
2453
 |
Posted - 2013.08.09 02:39:00 -
[19] - Quote
Posting in a thread that makes my head hurt, but is still good nonetheless. |

BLKDG
ROYAL SQUAD Shadow of the Apocalypse
43
 |
Posted - 2013.08.09 03:41:00 -
[20] - Quote
Thank you
I was starting to think the lunies ran the asylum
These forums... Man these forums. .. |
|

Soldier of Mawat
Militaires-Sans-Frontieres
140
 |
Posted - 2013.08.09 03:44:00 -
[21] - Quote
Aero Yassavi wrote:I once killed a Madruger with a scrambler pistol.
No joke, it was already damaged and I threw some packed AVs at it but it still had a sliver of armor let, fired my scrambler at it and it gave the pistol credit for the kill.
You're not the only one 
Video proof!
|

Skihids
Bullet Cluster
1851
 |
Posted - 2013.08.09 14:23:00 -
[22] - Quote
BLKDG wrote:Thank you
I was starting to think the lunies ran the asylum
These forums... Man these forums. ..
I put a lot of the blame on our educational system for neglecting basic reasoning and claim evaluation skills. People need a defense against the constant onslaught of Madison Avenue persuasion that as gotten quite scientific in its own right (creating price anchor points, giving false comparisons, etc.).
A very brief introduction to statistical concepts goes a huge way to arming you against bad arguments and misuse of data, and it's actually quite interesting. The math is extremely basic for the most part and everyone uses a computer to crunch the numbers anyway. |

Halador Osiris
Dead Six Initiative Lokun Listamenn
569
 |
Posted - 2013.08.09 14:24:00 -
[23] - Quote
I just saw this thread and thought of something somebody said on reddit last night: "The plural of anecdote is not statistics." It resonated well with me, and applies to this. |

Iron Wolf Saber
Den of Swords
7096
 |
Posted - 2013.08.09 14:25:00 -
[24] - Quote
Flux Grenades have more kills than Sleek AV grenades. 'Guardians' (spikes on support structures) have more kills than the Sleek AV grenades. Sleek AV grenades are at the bottom of the kills list according to CCP. |

The Robot Devil
Chatelain Rapid Response Gallente Federation
829
 |
Posted - 2013.08.09 14:59:00 -
[25] - Quote
brekfest wrote:"Whatcha got there? Numbers?"
+1 for science
Hey! I do the Futurama quotes around here. |

Halador Osiris
Dead Six Initiative Lokun Listamenn
569
 |
Posted - 2013.08.09 15:11:00 -
[26] - Quote
I find it funny that the guy who was collecting data wasn't fighting, which on its own technically screws up the data. It's like Heisenberg's uncertainty thing, you always mess something up by observing it. A better way to do it would have been to run 30 matches with a video capture card and then analyze the video afterwords. Or just pressure CCP for stats. |

Bittersteel the Bastard
WarRavens League of Infamy
91
 |
Posted - 2013.08.09 15:18:00 -
[27] - Quote
I like how Xero has chosen not to see this thread. |

Scheneighnay McBob
Bojo's School of the Trades
2848
 |
Posted - 2013.08.09 16:20:00 -
[28] - Quote
You must be new here.
Most forum-goers (and whoever is in charge of balancing) don't listen to logic. |

Scheneighnay McBob
Bojo's School of the Trades
2848
 |
Posted - 2013.08.09 16:21:00 -
[29] - Quote
Iron Wolf Saber wrote: Flux Grenades have more kills than Sleek AV grenades. 'Guardians' (spikes on support structures) have more kills than the Sleek AV grenades. Sleek AV grenades are at the bottom of the kills list according to CCP.
Even assault swarm launchers? |

Leovarian L Lavitz
Better Academy.
514
 |
Posted - 2013.08.11 16:27:00 -
[30] - Quote
Precisely |
|
|
|
|
Pages: [1] 2 :: one page |
First page | Previous page | Next page | Last page |