Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- One of the simplest ways to measure risk in stocks, bonds, crypto, mfs etc for ordinary people is Maximum Drawdown. Once you understand how it works, it takes the stress out of making and managing your own portfolio P — espexially your non retirement account. The question you have to ask your self is: "Given some portfolio with returns of X%, am I ok with this asset being down by Z% over T years — i.e the _computed/inferred_ drawdown?" If the answer on one end is no I cannot afford P to have any drawdown at all, then just put your money in a cash/MMF and call it a day. Generally people are ok with some risk on some percent of P and stash the rest in cash, and you _risk adjust_ for Z and T. The portfolio choices are surprisingly simple.
No other question matters. There are some risks but the key element is the understanding of the statistical variance because that allows me to say that "ok 50% of P can afford to be down for 3 years and I won't be homeless".
Most people don't know this but most money managers(managing money for ordinary Americans) that you hire compute this number _once_ and make tiny adjustments to your portfolio(I am talking once a year maybe) and take 1-2%.
I follow this guy called Dave Stein who has a B2C product called money for the rest of us that taught me this. I am not affiliated with them in any way.
by ghm2199 - I think the posted article is also the description of America's labor market. Many people don't use it on a day today basis and most people cannot engage their system 2 to assess risks over a few weeks out, let alone 5-10 years.
But I found This is a real practical everyday example of why statistical uncertainty is important to know about. It teaches you how to compare — risk adjust — two vastly different investments e.g. BTC vs S&P or indexing vs value strategies. Then you sleep at night.
It is also not intuitive and many people are very anxious and FOMO driven when it comes to money. So you need to internalize the idea for it to sink.
The drawdowns are a statistical measure and you could be unlucky to catch a big depression style thing once in 30 years. And T is typically longer than 5 years, typically 10 or more.
by ghm2199 - As a grad student at a public university in the USA, I worked as a teaching assistant for the sociology class "Introduction to Statistics." The course used SPSS software to enable the students to perform statistical analysis on data sets. Although we offered an excellent textbook (listed below), most of the undergraduate students struggled with the material, so my colleagues and I added a supplemental reading list to the syllabus for those students interested in studying the main concepts. The following books are the ones we recommended to the students. I have read all these texts, and they are pretty good at explaining statistics to anyone interested in learning the material at a conceptual level; anyone can read and understand these books, regardless of their educational background.
Ian Ayres (2007) Super Crunchers: Why Thinking by Numbers is the New Way to Be Smart
Joseph Healey (2020) Statistics: A Tool for Social Research [This is the textbook used in the class, which I highly recommend]
Catherine Marsh and Jane Elliott (2008) Exploring Data: An Introduction to Data Analysis for Social Scientists, second edition
John Allen Paulos (1988) Innumeracy: Mathematical Illiteracy and Its Consequences
Nate Silver (2012) The Signal and the Noise: Why So Many Predictions Fail, but Some Don't
Nassim Nicholas Taleb (2005) Fooled by Randomness, second edition;
_______ (2010) The Black Swan: The Impact of the Highly Improbable, second edition
by diogenes_atx - LOL, 62%? that seems low ...
I would say the overwhelming majority of even university faculty members lack a basic understanding of statistics.
In their heads, they’re doing something closer to informal Bayesian reasoning: they have prior beliefs based past observation, previous research, experience, etc., wich is then all baked into every choice and decision
But when it comes time to write the paper, they translate that into frequentist statistics: p-values. As if all should be compared to random chance: significance thresholds, confidence intervals.
They do this to give their conclusions formal credibility: oh look how compelling my result is, I found all this new surprising stuff.
Close to 100% of life sciences research works that way.
by DataDive - Even leaving aside the quantitative stuff (p-values, medians, whatever) and can't crunch the math, IMHO at least you should have seen how statistics can lead you to the exact opposite conclusion from reality, so that you at least know whether to think twice about a conclusion drawn from statistics thrown at you. Yet I was recently quite surprised to learn that even many folks in tech had never encountered Simpson's paradox before. All it takes to start explaining that is a scatterplot and a few lines, and yet it doesn't seem to be taught widely. It's rather terrifying, given that most people (myself often included) will be happy to believe "obvious" conclusions drawn from seeing one percentage greatly exceed another.by dataflow
- There is the old joke that 87 percent of all statistics is made up on the spot. I’ve told it many times but a fair amount of people seemed to believe it hook, line, and sinker.by zippyman55
- I love how they include statistics in the headline of a story about the lack of understanding of statistics. Epic troll.
100% of headlines of statistics-related articles must follow this rule.
by BLKNSLVR - What does it really mean to properly understand p-value? What I remember is that if the p-value is less than 0.05, the research result is considered statistically significant. That's about as far as my memory goes. I know that's actually a misunderstanding, but that's how most people understand it. I'm not sure how much I need to know to say I truly understand it.by jdw64
- Fair play to you for answering, but that’s not what a P value is! A p value is the chance of getting a result greater than the result you actually got, _assuming you are testing a hypothesis_.
I’m also pretty sure I would fall in the camp of saying “nope don’t understand P values” as I can’t remember anything else about them.
by maccard - Say you go out and you see a sports bar and the FIFA world cup is on. France vs Sweden is playing on the TVs in the bar and the place is packed with fans of both countries. You look at a few people (wearing the national football colours) and you think to yourself "the French fans seem shorter than the Swedish fans". You decide to run an experiment to test this hypothesis. Lucky for you at half-time exactly n fans of each side agree to let you measure their height. You do this and of your sample of n fans, the French are 2cms shorter on average than the Swedish fans.
Now: the p-value is the probability of you getting a result at least this extreme assuming the null hypothesis (that there is no difference in average height between the general population of French and Swedish people).
by seanhunter - It is mumbo jumbo (also people hearing “significant” treat this as “effect/change is large and important” which is in no relation to the actual amount of change).
Low p-value basically means how surprising your data would be if there were actually no effect (ie less than 5% of the time you’ll get this due to randomness if there is no change — which is rather impossible)
Sample size matters heavily. With more observations, estimates become more precise, so increasingly small differences can become statistically significant. With a large sample, you can therefore get a tiny, practically meaningless effect with a very small p-value.
Eg effect of $1 can be statistically significant (not random) which does not matter in practical terms if average is like $10000.
So the key point here is not only to look at the p-value but also at an actual change. If a drug gives you only 0.01% more hair, it doesn’t matter to you that it is guaranteed.
by piskov - Agree. I think it's more important to understand statistical fallacies (selection bias, regression to the mean, survivorship bias, etc). Those are extremely common trip wires but to recognize them you don't need to memorize formal definitions.by ngruhn
- P-values are almost always taught poorly, but it's not actually that difficult of a concept. I took statistics in high school, again in undergrad, and it wasn't until the third time in grad school that it actually made intuitive sense (thank you Julia Yang!). When you're testing a hypothesis in statistics, it's easier to formulate a "null hypothesis" which is the opposite of what you're testing, and then try to disprove that null hypothesis.
A p-value is the probability, assuming the null hypothesis is true, of obtaining a result at least as extreme as the one actually observed.
Put differently: if the null hypothesis were true, then for p=0.05 you'd see <things at least as far from the test statistic as what you just observed> at most 5% of the time.
Put differently again: If the null hypothesis you are testing is true, then for p=0.05 random sampling would not return an observation as far from the test statistic as you just observed, 95% of the time.
by stackghost - “Fortunately, concepts in statistics are grounded in intuition and rationality.“
Yeaaah… let’s talk about that.
Seems like a fair bit of stats were designed to intimidate —so as to get people to stop asking questions. Or at least that is the effect!
Stats designed for intuition are few. See Kill Math for how it might be done: https://worrydream.com/KillMath/
by dr_dshiv - I don't think it's a question of design. Some ideas in stats are just fundamentally unintuitive. Pedagogy can help mitigate this, but it would take a revolution to fit it into a neat and intuitive framework, like calculus.by breezybottom
- The article reads like work of a crackpot. There’s a reason why maths is done the way it is, and intimidating people is definitely not the motivation.by frontfor
- This is also a political problem. Most of the debates about racism for instance are between people who are illiterate in term of basic statistics, on both sides of the debate. They take extreme tails of distributions as benchmarks, think in term of stereotypes rather than distributions, infer rules from a handful of samples, etc. It makes the whole exercise sterile.
It is not just journalists and politicians that are guilty of that. I see that from many academics who should know better, particularly in humanities departments.
by cm2187 - Are you talking about actual academic work or the pop- version of it? Because most of the damage I see is in pop-psychology, pop-sociology. In my limited experience looking into the topic humanities researchers make specific balanced claims backed with some numbers and reasoning, and all the nuance gets lost when translated to news articles and pop-sci books or videosby dgellow
- This isn't new. "Lies. Damned lies and Statistics" as a phrase is already quite old.
Many people selectively shop around statistics that match their prejudice or their social circles social etiquettes.
by apexalpha - <https://en.wikipedia.org/wiki/Motivated_reasoning> affects us all. I'd like to believe education is an antidote, but really the only antidote is... not doing that.
- I'd say in reality it's way more than that. The first statistics course in uni was a very humbling experience. I realised that while I thought I understood a lot (and I was coming from a CS heavy background, olympiads and such) real statistics is way harder and a lot more counterintuitive than I thought. Granted, this talks about "basic" statistical understanding, but even that is way more complicated than most people assume.
- yeah, i've studied a lot of math and a lot of cs, and stats is tough. part of the problem is the terminology, and just giving a ton of complex machinery without telling you what it's actually doing. i've never learned that way, and it is very easy to feel like you're doing some sort of dark magic.
also, probability theory vs statistics is an important distinction: prob theory is a nice clean mathematical subject, while statistics is almost the philosophy of applying probability theory to the world.
by bbeonx - Yeah. Like most people, my only experience of statistics is studying as part of maths in secondary school. I was really strong in maths generally and most of it came relatively easily to me, but I struggled with statistics. Examples like the Monty Hall problem show just how unintuitive the basic concepts are.
Given how effective stories and anecdotes are in convincing people, it really seems like the human brain is not wired to grasp these concepts easily.
by NoboruWataya - I was having a conversation with my dad about how if a virus had a 10% death rate it would be terrifying.
His reply was that 10% was nothing, that means you've got 90% chance to survive. Then said you've probably got a 10% chance of being hit by a bus every time you cross the road. Just absolutely no idea what probabilities are at all or ability to extrapolate probabilities from real world lived experience.
by LandR - In my head he answered "in my day and age we had the black plague, with a 90% death rate". (It was 50%, but who checks these numbers anyway.)
Guess we are so used to made up numbers and exaggerations that it's hard to take anything at face value.
by OroPla - the short story The Pill plays out a world full of people who reason similarly, among many other things. https://escapepod.org/2022/11/03/escape-pod-861-the-pill-par...by boogieknite
- > His reply was that 10% was nothing
Ask him how many times he would roll that dice
by dgellow - >> found that 62% of respondents reported no or limited statistical knowledge
The other 38% didn't understand the question...(my extrapolation)
I did a couple semesters of statistics at uni. And I can confidently say that the number of people who can answer 3 simple questions on statistics (like say mean versus medians, confidence levels or margins of error) is, well, a rounding error from 0.
Indeed, statistically, no-one has a clue how statistics work.
I did however learn enough to know that statistics can tell you absolutely anything you want them to say. Assuming you don't just make them up, they're trivial to manipulate to generate the headline you want.
When used to evaluate risk, the comprehension goes down further (a fact willfully exploited by any decent marketing.)
Statistically, most statistics are meaningless.
by bruce511 - “I only believe in statistics that I doctored myself”
― Winston Churchill
by zombot - You mean, the other 48% didn't understand the question. :)by kazinator