Super Crunchers Ian Ayres Pdf Download
See a Problem?
Thanks for telling us about the problem.
Friend Reviews
Community Reviews
According to Ayres, supercrunching invo
Be prepared to encounter the words "supercrunch" (used as any part of speech) and "nano-" (used indiscriminately as a prefix) approximately one billion times in a mere 272 pages. Dr. Ayres wants to write the next Freakonomics , and makes his professional association with Steven Levitt known frequently. What comes out is a repetitive book on applied mathematics fleshed out with anecdotes and descriptions of research. It's okay, but nothing groundbreaking.According to Ayres, supercrunching involves running statistical analyses on large data sets. Supercrunching has better predictive power than experts do. Supercrunching is changing the way everyone and everything does anything. Supercrunching will pick you up if you're having car trouble. Supercrunching tops its pizza with ham and pineapple, just like you do. Supercrunching is attracted to you, respects that you're in a relationship, and wonders if you have a sister. Et cetera. Ayres, in his descriptions of supercrunching, vacillates from paranoid/creepy to smitten/exuberant.
Supercrunchers was fun at first. Then it got blah, seasoned with dabs of fun here and there. Luckily it is short enough that it didn't have time to grate. Of note, the book never mentions Grape-Nuts, not even once. Also of note, this review includes the word "supercrunch*" 999,999,889 fewer times than the book does. Supercrunch that, Ayres!
...moreI've been becoming interested in the overlapping fields of economics, human decision making, cognitive errors and statistics for quite some time now. One of the first books I read that started me off down this path was Freakonomics: A Rogue Economist Explores the Hidden Side of Everything. There were parts of that book that I found truly fascinating. I really like the idea of people looking for bizarre correlations between sets of data that tell us something new we did not know before. For example, Freakonomics showed that there was a relationship between abortion rates in one decade and dropping crime rates a couple of decades later. I'm a sucker for that sort of thing.
The writer of this one has done research with the guy that Freakonomics was written about. And this book fits neatly right into that school of thought. Let me try to sum it up for you. Humans aren't really all that great at working out the best options available to them. We think we learn from experience and that we get better at picking what will be the right outcome when it comes along by our hard won intuitions (see Blink, for example) but when push comes to shove even around things we believe require substantial human discernment (like picking potentially great sports people) are actually generally better understood and the answer better stated by applying a simple mathematical algorithm. The most interesting example given of this is deciding if a vintage of wine will be exceptionally good or not. It has been assumed until recently that this isn't something that can be decided until long after the grapes have been harvested, crushed, fermented and aged in oak. So, when someone said they could tell if a vintage was going to be good, not by tasting the wine, but by plugging in some values into the variables related to the climate while the grapes were growing in some formula – well, things were bound to get ugly.
Looking at it, this shouldn't have caused as much outrage as seems to have been the case (at least, as it is reported here). I say that because wine is a natural product. And as a natural product it would seem its quality is ultimately dependant on the quality of its inputs (that is first and foremost, the grapes). If the summer has not been hot enough it is reasonable to think that the grapes may not become ripe enough, the amount of rain will probably also have an effect on the grapes.
Given that all of the things a wine maker can control probably already is controlled, the 'acts of God' that cannot really be controlled are obviously going to be the main variables left to decide whether a vintage is to be exceptional or ordinary. Like I said, if it is true (as is reported in this book) that some wine critics were outraged that someone could say they knew years ahead of time if a vintage was going to be exceptional, it seems very silly of them to do so.
So, what this book is saying is that the best decision are often made not by 'experts' (like wine experts) but by objective mathematical algorithms in which you plug in your data and wait for the answer to pop out the other side. Clean, clear, untouched by human hands – we are talking of the truth; unalloyed and pure.
My problem with this is that too many of these examples seemed to depend entirely on there being a right answer to the question at hand – and, in many of them, I wasn't quite sure I understood what the 'right' answer was or how you could verify the answer the formula threw up was actually the right one.
I've just finished reading another book, for example, (Why We Make Mistakes: How We Look Without Seeing, Forget Things in Seconds, and Are All Pretty Sure We Are Way Above Average) in which people invariably preferred the bottle of wine labelled $90 over the one labelled as costing $15, even when they were the exact same wine. Having said that, although I can see the logic of knowing how ripe the grapes are when they are harvested and how much rain fell in the weeks leading up to harvest should make a difference to the quality of the wine once it is made– I also know that I've been fooled before by stories about subjects I know remarkably little about, particularly when those stories seem to make lots of perverse and twisted logical sense.
A case in point. Years ago I told my daughter (while she was still young enough to believe I was some kind of God) that it was likely that T-Rex didn't go stomping around ripping other dinosaurs apart, but rather crawled around on his belly and ate mostly carrion. An article I had read somewhere explained this in fascinating detail. You see, T-Rex was cold blooded and big – so he was going to be at a disadvantage in having the energy needed to do the whole Jurassic Park thing. Also, there were those pathetic little forearms (think Charles Atlas just before the sand-in-face moment) which don't seem to be of any use at all if he walked around upright, but might have made much more sense if he crawled about on all fours. Fi had a plastic T-Rex dinosaur and we used to have it crawling around on its belly as it dragged itself to dolls for a feed. As they say in Mythbusters, the guy who wrote the article had done the maths and the myth of the big, fast, angry T-Rex was busted.
Except, of course, that was before they worked out dinosaurs were less like lizards and more like birds. As a lizard, T-Rex wouldn't have had the energy to run and fight and kill, but as a bird none of that was a problem.
My point is that the presentation of this stuff as simply objective truth is a bit dangerous – not least because we humans (particularly we boy humans) just love numbers, especially either big numbers or numbers with decimal points. Tell a boy it happened 165 million years ago or it will cost 15 trillion dollars to fix or that you are 43.7 per cent sure of something and, in what is the closest sensation to post-coital bliss, he will become so excited he won't even know you made this stuff up on the spot. The most obvious reason why creationism is bound to fail is that any idea that stops men (well, boys) saying '9.8 billion years ago' and replaces it with 'around10,000 years ago and that's tops' really doesn't stand a sporting chance.
Sorry, off topic. There are very interesting discussions in this book on data mining, data warehousing and corporate manipulation of customers that, even though the author did not feel were all that bad, ought to be read so that you have an idea of what is being done to squeeze money out of you. The discussion on how casinos work out your 'pain point' so they can stop you from losing beyond that pain point and thus get you to come back again later is disgusting. The most useful message to come from this book is that we should all be afraid when a corporation is being nice to us – it almost invariably means we are paying too much for the service they are providing us.
The most disgusting lesson presented here is that if you are black or female in America and you need a new car then you really ought to think about becoming white and male. And don't be fooled if the dealer gets you someone to bargain with you of your own sex or race. Become white and become male – even if you have to pay someone both white and male a couple of hundred dollars to be you – it might save you thousands.
I worried while reading this book that a lot of the questions the author felt were settled by crunching numbers were still somewhat open for debate – I struggle with the idea that direct instruction (where teachers spend their days instructing kids literally by reading from a script with no discretion at all) can be the great boon to education that it is presented in this book to be. Figures were given that made direct instruction sound like the only sensible way to teach kids anything – however, even though I'm not a teacher, I still have strong doubts that this kind of rote learning (measurable as it is) will produce kids able to respond to the challenges of the modern world.
This book suffers from the same self-congratulation Freakonomics struggled with. I have a fairly natural aversion to people who pat themselves on the back quite so fulsomely. All the same, there were a number of things I learnt from this book and it was worth reading for its interestingly simply method of explaining regression and standard deviations.
...moreI loved the quiz on p. 113 that tested a person's ability to make unbiased estimates. It's shocking how inaccurate and overconfident I was with my own estimates.
Overall, I thought this was a well-written and researched book that really opens my eyes to the predictability of our world based on statistical data regression. It'
Fascinating look at the role of statistics and data sets today. Although the book was first published in 2007, the information still seems to be very relevant and up-to-date.I loved the quiz on p. 113 that tested a person's ability to make unbiased estimates. It's shocking how inaccurate and overconfident I was with my own estimates.
Overall, I thought this was a well-written and researched book that really opens my eyes to the predictability of our world based on statistical data regression. It's intriguing enough to make me consider pursuing this concept further.
interesting quotes:
"It's so cheap to test and retest alternatives that there's no good reason to blindly accept the wisdom of academic axioms." (p. 55)
"Dozens of studies dating back to 1989 found little support for many of the tests commonly included in a typical annual physical for symptom-less people. Routine pelvic, rectal, and testicular exams for those with no symptoms of illness haven't made any difference in overall survival rates. The annual physical exam is largely obsolete. Yet physicians insist on doing them, and in very large numbers." (p. 87)
"The openness of the Internet is even transforming the culture of medicine. The results of regressions and randomized trials are out and available not just for doctors but for anyone who has time to Google a few keywords. Doctors are feeling pressured to read not just because their (younger) peers are telling them to, but because increasingly they read to stay ahead of their patients." (p. 95)
"Osler would be turning in his grave at the thought of Google diagnoses and Google treatments because the Internet disrupts the dependence of young doctors on the teaching staff as the dominant source of wisdom. Young doctors don't need to defer to the sage experience of their superiors. They can use sources that won't take joy in harassing them." (p. 96)
"...Steve Levitt, looking at statistics, pointed out that 'on average, if you own a gun and have a swimming pool in the yard, the swimming pool is almost 100 times more likely to kill a child than the gun is.'" (p. 112)
"So humans not only are prone to make biased predictions, we're also damnably overconfident about our predictions and slow to change them in the face of new evidence." (p. 114)
"Power and discretion are definitely shifting from the periphery to the Super Crunching center. But that doesn't mean Super Crunchers are going to find they have an easier time dating." (p. 169)
"The slow erosion of the private sphere makes it harder to realize what is happening and rally resistance. Like a frog slowly boiling to death, we don't notice that our environment is changing." (p. 179)
'
in the Notes section: "It is time for the scientific community to stop giving alternative medicine a free ride. There cannot be two kinds of medicine - conventional and alternative. There is only medicine that has been adequately tested and medicine that has not, medicine that works and medicine that may or may not work." (p. 229) [by Marcia Angell and Jerome Kassirer, former editors-in-chief of the New England Journal of Medicine]
new words: heteroskedasticity, foreordain, quantiphobes, pellucid, polymath, curtilage, innumeracy
...moreWhile my inner merchant delights at the knowledge that huge data-masse
The helpful prompt from an online grocery-shopping site, "Do you really want to buy twelve lemons," was the phrase that left me feeling troubled; it seemed to encapsulate not just the fears about loss of privacy, but concerns about our perceptions of the "norm," the classification of humanity into categories, the paternalization of everyday decision-making, and, oh yes, a very personal dislike of being just like everyone else.While my inner merchant delights at the knowledge that huge data-masses can be massaged to offer up useful business-building information, it deepens the nagging sense that we are increasingly unable to identify the sources of much that impacts us.
It's easy to understand why teachers chafe against the notion of Direct Instruction, despite its proven successes; why doctors feel that diagnostic databases threaten them professionally, although these appear to be the road to better medicine; why writers resist statistical predictions about the probable success of plots and scripts. What is more difficult to quantify is what this all means. As human beings, we are physiologically little different from the cave-dwellers, yet our lives bear no resemblance to those times. It's not yet possible to tell how, for instance, being subjected to a daily recital of a thousand horrors from all over the world will affect a brain wired for the microcosm of the tribe or village. What will happen to the human if intangibles like job satisfaction, creativity, and trial-and-error are sacrificed on the altar of statistical advantage?
Although Ayres tries to make a case for a happy marriage of human intuition and data-flogged analyses, it's a hard sell. Given the propensity of the masses to follow the path most-traveled, it's difficult to imagine a future in which the intuitive few hold any sway over majority.
...moreHow wrong I was! More than a decade later, other than the big tech giants, not many other companies are ut
Wow... it's taken me more than three months to finish this book. In my defence, I started the introduction in January and for some reason, I had the impression that the book, written in 2007 would be outdated by now. I mean, we all know a data-analyst or two, right? And we see so much of it in businesses today that I wrongly assumed that whatever was written would have been old news by now.How wrong I was! More than a decade later, other than the big tech giants, not many other companies are utilising data! Not only that, schools aren't really emphasising statistics!
I like that the author had a really clear idea of the progression of the book. From what used to work to how it's going to work and even a very clear argument about why AI and big data aren't going to make humans redundant. The best part is actually the last chapter where the author argues that we all should learn about statistics and how to use that to aid us in making better decisions. We don't all need to be data analysts but we can use data to make better decisions.
I love it! So glad I didn't just shove it back into the bookshelf. I was soooo close to doing that 🙈
...moreAyres uses the words "Super Crunching" (over and over) to refer to the act of analyzing large data sets to make evidence-based conclusions about things that
I picked this book up because Lessig called it "the most important book I've read in as long as I can remember". That's some high praise. Indeed, the thesis of this book is an important one to take to heart when thinking about the world today and in the future, but to my mind the book falls short of being an excellent defense of that thesis.Ayres uses the words "Super Crunching" (over and over) to refer to the act of analyzing large data sets to make evidence-based conclusions about things that previously depended on the intuition of experts. He does a good job of indicating that this process is almost always more effective than human decision making — which he demonstrates to be true in virtually all fields of endeavor — but that there are positive and negative implications to that effectiveness.
So: granting Ayres his thesis, what do we do with it? Airlines have long engages in price discrimination based on factors like purchase date, frequent flier status, and the like; but what do we do when those practices are applied in every industry, and using data from all aspects of the customer's life?
One response, and the one that Ayres explores, is encouraging citizens to increase their numeracy and the attention they pay to purchasing decisions. Another, and he mentions this in passing, is the emergence of a "datamining free" certification, like "free range" or "organic", that promises prices or business practices set by humans.
Two problems are raised but inadequately addressed, for my taste. How do we preserve the humanity of commerce when algorithms determine how to not "leave money on the table", and when discretion is driven further up the chain of command? I don't want to get all Jaron LanierJaron Lanier on you, but these are real things to consider.
There are some amazing businesses that don't function very well from a pure economic analysis. Craigslist is famously used as a bad example in Harvard business classes, because there are so many unexploited opportunities for monetization. But isn't that lack of a sense of exploitation the real reason that it's a success, and a uniquely modern one at that?
In all, this book brings some important realities to the fore, and shouldn't be punished for that. It is accessible for a layperson, and really drives some points home. I wouldn't call it the most important book I've read in as long as I can remember, but I'd recommend it to somebody who wanted to learn more about the way decisions are made at the corporate and governmental levels.
...moreSuper Crunching is crucially about the impact of statistical analysis on real-world decisions. Two core techniques for Super Crunching are the regression and randomization.
1. Regression will make your predictions more acc
Great book on the importance of data-driven decision making. While I have always been someone that has let the data do the talking, I haven't found an easy way to explain why. Super Crunchers is that easy way! Below I have summarized some of the important points of the book....Super Crunching is crucially about the impact of statistical analysis on real-world decisions. Two core techniques for Super Crunching are the regression and randomization.
1. Regression will make your predictions more accurate (Historical approach):
It all starts with the use of regressions, and although this method is a basic statistical test of causal relationship it's still a very powerful tool that I need to re-introduce in my analytical life.
Regressions make predictions and tell you how precise the prediction is. It tries to hone in on the causal impact of a variable on a dependent. It can tell us the weights to place upon various factors and simultaneously tell us how precisely it was able to estimate these weights.
2. Randomization and large sample sizes (Present/Real-Time approach):
Reliance on historical data increases the difficulty in discerning causation. Large randomized tests work because the distribution amongst the sample are increasingly identical. Think A/B testing on steroids that allows you to quickly test different combinations! Boils down to the averages of the "treated and untreated" groups.
Government has embraced randomization as the best way to test what works. Statistical profiling led to smarter targeting of government support
With finite amounts of data, we can only estimate a finite number of causal effects
3. Neural network
Unlike the regression approach, which estimates the weights to apply to a single equation, the neural approach uses a system of equations represented by a series of interconnected switches.
Computers use historical data to train the equation switches to come up with optimal weights. But while the neural technique can yield powerful predictions, it does a poorer job of telling you why it is working or how much confidence it has in its prediction.
Super Crunching requires analysis of the results of repeated decisions. If you can't measure what you're trying to maximize, you're not going to be able to rely on data-driven decisions.
We humans just overestimate our ability to make good decisions and we're skeptical that a formula that necessarily ignores innumerable pieces of information could do a better job than we could.
...moreIt gets distracting. And annoying. It doesn't help that it sounds like a breakfast cereal. In the end, it becomes a little embarrassing.
(To be fair, the author notes that he "super crunched" the title -- he ran multiple titles through some regression testing to see w
Throughout this book, the author keeps trying to make the phrase "Super Cruncher/ing/ed" happen. I kept thinking back to the character in Mean Girls that said, "Gretchen, stop trying to make 'fetch' happen. It's not going to happen."It gets distracting. And annoying. It doesn't help that it sounds like a breakfast cereal. In the end, it becomes a little embarrassing.
(To be fair, the author notes that he "super crunched" the title -- he ran multiple titles through some regression testing to see which one would perform better.)
Aside from that, the book is okay. It's very Malcolm Gladwell-esque in that it's a series of anecdotes about how data analysis ("Big Data") is changing everything. A lot of the stories were admittedly interesting, but I struggle to figure out the audience for the book. This is not a detailed, instructional book, so was the author just trying to let people know that this stuff happens?
In the end, am I at all better for having read it? Probably not, but I've read this stuff before, so it wasn't new to me. If you've never heard of any of this, you might be fascinated. I recommend:
* "Big Data"
* "Automate This: How Algorithms Came to Rule Our World"
But, of course, both of those books are the same thing -- lots of anecdotes, presumably meant to prove a vague point.
The afterword of the book (from the paperback edition, I'm assuming) is a mess. The author starts talking about his weight loss and some bet he made, which doesn't seem to have anything to do with the rest of the book. Then he appears to start making a sales pitch for his own web startup.
...moreThe four interesting things I found in it:
1. The author went over a study done on Greyhound racing, experts vs. a computer model for predicting the winner. All the experts lost and the computer model make a 25% profit. My question: this is a freaking money machine! Why would you ever publish this study? Why not just capitalize on it? Something fishy was definitely going on.
2. All the way at the end of the book he goe
This book was trying to be another Freakanomics...it definitely missed its mark.The four interesting things I found in it:
1. The author went over a study done on Greyhound racing, experts vs. a computer model for predicting the winner. All the experts lost and the computer model make a 25% profit. My question: this is a freaking money machine! Why would you ever publish this study? Why not just capitalize on it? Something fishy was definitely going on.
2. All the way at the end of the book he goes over the need for the general public to know how standard deviation and combining probabilities work. I think he should've snuck this in a lot earlier and put more emphasis on it.
3. Several times the author stressed the need to know how statistics work and how to "SuperCrunch" for your own benefit but didn't go over any of the techniques.
4. The statistical technique of regression was discovered by Charles Darwin's cousin. It's called a regression because initially when he was using the technique he observed that the data "regressed towards the mean". So if you looked at really tall fathers, their sons would be shorter. The same with really smart parents, their children's intelligence was not greater than or equal to their parents even though it was a combination of two sets of "genius" genes. It also regressed towards the mean.
...moreThe first few chapters are what hooked me. They are filled with examples of real-world
I work in the Analytics field and am becoming more involved in prediction and predictive models so it was a genunine pleasure when I first picked up Supercrunchers. I actually read a preview of this book on iBooks on my iPhone at lunchtime which is what got me hooked and convinced me to buy the whole book. Looking back on it, I would still buy it however not with the same enthusiasm as iBooks led me to believe.The first few chapters are what hooked me. They are filled with examples of real-world predictive models being used for all sorts of problems that people face from predicting whether a wine will be good before it's even bottled through the well-known story of Moneyball and onto things such as healthcare, crime and financial markets. These chapters are packed end to end with case studies and examples and get you excited as to the possibilities.
The next few chapters go a bit more in-depth into different industries and longer case studies into areas such as evidence-based healthcare which was also a pleasure to read about, some of the history and how you need to fight for your beliefs even if there are naysayers.
Just after this point is where I found the cohesiveness of this book and its purpose started to fall apart. There is nearly a whole chapter dedicated to the author proving why he is right and someone else is wrong in terms of a predictive proof, multiple references to his "friend" who has a best selling book (author of Freakonomics in case you miss it the multiple times it is mentioned) and then a final chapter that is indicated is to give people a taste of some of the statistics so they can do their own research once the book is over. A teaser if you will into the world of standard deviations.
The only problem with the final chapters is they feel like they belong in another book. The book appears to start off at the end, the result of models and try to work back to the basics of how to do it, without quite getting there or being that enlightening about the technicalities of it. The entire section on why he and his "friend" are right and another statistician is wrong, really turned me off wanting to finish the book and left a bad taste in my mouth.
...moreHow come the NOAA have these massive super crunching computers and algorithms that are getting so good at predicting the weather but it seems like nobody can predict the next stock market crash and global reccession?
For instance the 2008 stock market crash or the current 2015 Chinese economic crash?
Well the book certainly inspired me to dig deeper and find out what numbers I can easily crunch to improve my own life and the lives of others. So I am on to other more techinical inclined books and online resources so that is why I give it 4 out of 5 in spite of it obviously "dated" feel in the fast moving world of predictive analytics and the Ted talks about how Shazam can predict the next global music hit... My next book on the subject to read will be "Data Smart" by John W. Foreman
...moreI have a near compulsion to look for trends in everything with the goal of forecasting future outcomes. I study patterns looking to predict what will happen next. I loved Mario Livio's book, The Golden Ratio: The Story of PHI, the World's Most Astonishing Number, am fascinated by the Fibonacci sequence and the study of probabilities. I guess I'm a manqué statistician at heart.
Although I enjoyed this book, I did have hi
I have a problem. My name is Mike and I'm a Super Cruncher (all: Hi, Mike!).I have a near compulsion to look for trends in everything with the goal of forecasting future outcomes. I study patterns looking to predict what will happen next. I loved Mario Livio's book, The Golden Ratio: The Story of PHI, the World's Most Astonishing Number, am fascinated by the Fibonacci sequence and the study of probabilities. I guess I'm a manqué statistician at heart.
Although I enjoyed this book, I did have higher hopes for it. Even though I knew many of the applications detailed, the case studies for crunching numbers clearly explicate the benefits of using regression. However, I was hoping the author would have taken the next step and described the mathematical process of regression in one of the case studies. I realize that the process is complicated but a basic "how to" would have been indispensable for me.
For most, this would be a great read just to understand how much companies know about you and how they use that information to predict your future behaviors, including your likelihood to get a divorce. Wild stuff.
...moreSeriously: This word appears on every single page, and once you notice it, you won't stop noticing it. Sometimes its there two or three times. At one point Ayres bragged that he used "Super Crunching" techniques to settle on the title for his book. An algorithm told him it would be more popular with potential readers than another option
I picked up this book on a whim at a used bookstore and hate to admit that yes, I'm a victim of Ayres favorite (and incredibly overused) term, "Super Crunching."Seriously: This word appears on every single page, and once you notice it, you won't stop noticing it. Sometimes its there two or three times. At one point Ayres bragged that he used "Super Crunching" techniques to settle on the title for his book. An algorithm told him it would be more popular with potential readers than another option he had chosen. So yes, I fell victim.
There are pros and cons to this book, and probably not in the way Ayres intended. Since this was published in 2007, some of the information is outdated, but some of Ayres predictions about how data-sharing would/does influence our lives is downright prescient. Comparing some of his predictions is entertaining.
For example, Ayres predicts many of our current-day fears about our news echo chambers: Ayres writes: "MSNBC.com has recently added its own "recommended stories" feature. It uses a cookie to keep track of the sixteen articles you've most recently read and uses automated text analysis to predict what news stories you'll want to tread…. Nicholas Negroponte, MIT professor and guru of media technology, sees in these "personalized news" features the emergence of the "Daily Me" -- news publications that expose citizens only to information that fits with their narrowly preconceived preferences. Of course, self-filtering of the news has been with us for a long time. Vice President Cheney only watches Fox News. Ralph Nader reads Mother Jones. The difference is that now technology is creating listener censorship that is diabolically more powerful."
And then I had to laugh at his choice of example for the dangers of facial recognition: "The law in the past didn't need to worry much about our walking around privacy, because out on the street we were usually effectively anonymous. Donald Trump may have had difficulty going for a walk incognito in New York, but most of us could happily walk across the length and breadth of Manhattan without being recognized."
But then I began to tire of Ayres praising of "Super Crunching" (tL;Dr: basically the use of collecting quantitative data on human behavior and using statistical analysis to predict the result. Think Google Ads, or, like above, curated news based on algorithms). Though he has some good points about how using evidence and statistical analysis makes sense, like in the application of evidence-based medicine (rather than relying on the 'intuition' of doctors, which happens more often than you might think), there are a lot of times where he neglects to play his own devil's advocate.
An example he presents is a company that claims to use an algorithm that can predict the qualities of a movie that consumers will most likely go see and end up enjoying: "The Super Crunching of art seems perverse, but it also represents an empowerment of the consumer. Epagogix's neural network is helping studios predict what qualities of movies consumers will actually like. It thus represents a shift of power from the artist/seller to the audience/consumer. Epagogix, from this perspective, is part and parcel of the larger tendency of Super Crunching to enhance consumer quality. Quality, like beauty, is in the eye of the beholder, and Super Crunching helps to match consumers with products and services that they'll find beautiful."
I also took umbrage with a weird little personal attack Ayres makes in the latter half of his book. Apparently Ayres has a critic, named John Lott, who attacked him online under the pseudonym "Mary Rosh." This was in response to Ayres' criticism of Lott's study that concluded concealed-carry laws decreased gun violence. I know I'm quoting at length, but I think it helps to see what I mean here when Ayres' presentation of the situation and his justification for including it are just downright bizarre:
"Donohue and I took Lott's data and ran thousands of regressions exploring the same issue. Our article refuted Lott's central claim. In fact, we found wice as many states that experienced a statistically significant increase in crime after passage of the law. Overall, however, we found that the changes were not substantial. And these concealed weapon laws might not impact crime one way or another.
That's when Mary Rosh weighted in on the web. Her comment isn't so remarkable for it's content -- that's part of the rough and tumble of academic disputes. The comment is remrkable because Mary Rosh is really John Lott…. Lott as Roche posted dozens upon dozens of comments tot he web praising his own merits and slamming the work of his opponents. Rosh, for example, identified herself as a former student of Lott's and extolled Lott's teaching. I have to say that he was the best professor I ever had," she wrote. "You wouldn't know that he was a 'right wing idealogue' from the class."
Lott is a complicated and tortured soul. He is often the smartest guy in the room. He comes to seminars and debates consummately prepared. I first met him at the University of Chicago when I was delivering a statistical paper on New Haven bail bondsmen. Lott had not only read my paper carefully, he'd looked up the phone numbers of New Haven bond dealers and called them on the phone. I was blown away."
Ayres then uses this interaction as a way to promote his own agenda:
"My contretemps with Lott suggests the usefulness of setting up a formalized system of empirical devil's advocacy akin to the role of an Advocatus Diaboli in the Roman Catholic Church. For over 500 years, the canonization process followed a formal procedure in which one person (a postulator) presents the case in favor and another (the promotor of the faith) presents the case against." and then : "Corporate boards could create devil's advocate positions whose job it is to poke holes in pet projects."
This whole part of the book just seemed odd to me.
While I enjoyed Ayres' obviously thorough research and knowledge of the subject, ultimately the book is rather outdated by this point and a little too overzealous for my taste. One last quote from Ayres:
"...a broader quest for a life untouched by Super Crunching is both infeasible and ill-advised. Instead of a Luddite rejection of this powerful new technology, it is better to become a knowledgeable participant in the revolution. Instead of sticking your head in the sands of innumeracy, I recommend filing your head with the basic tools of Super Crunching."
I rest my case.
...moreA wide variety of stories about how the computer is enabling huge changes in our schools, businesses, purchases What a book! Shockingly good! I heard about this book while listening to The World is Flat by Thomas Freidman. Immediately, I made a mental note to find and read this book about the impact of computing power on everyday lives. Algorithms, formulas, yikes! (I have a bit of a math phobia.) Thank goodness this book breaks down complex ideas into understandable and applicable explanations.
A wide variety of stories about how the computer is enabling huge changes in our schools, businesses, purchases, relationships and government. Inspirational and somewhat frightening, the book explains the high level of sophistication now practiced out 'there' in information gathering and "crunching" of the data. This new data is available to all levels of society, as long as they have access to the internet. In turn, this is leading us all towards new discoveries, more efficiency, greater understanding and of course, profits!
The book was so great, I listened to it three times!
...more
Since I am new to the practice of data crunching, this book provides a great introduction to what pitfalls and opportunities await. The author keeps it fairly light and general as much of the data world is bogged down in statistics, mathematics, theory, etc. I've never valued my college exposur
Great book! I stumbled upon this book at the library and was intrigued by the title. My interest was primed by some of the work I am doing with data analysis to inform my decision-making as a City Planner.Since I am new to the practice of data crunching, this book provides a great introduction to what pitfalls and opportunities await. The author keeps it fairly light and general as much of the data world is bogged down in statistics, mathematics, theory, etc. I've never valued my college exposure to those topics as much as I do now.
Perfect book for the time of my life.
...moreAfter reviewing tons of data Orley Ashenfelter determined that low levels of harvest rain and high average summer
I'm not a wine connoisseur by any means however I do enjoy a nice glass of wine on occasion. I never quite understood the rating system used to value wines. There is more than one wine rating and the results are totally subjective. I always thought that there has to be a better way to rate wine and it turns out that a "Super Cruncher" has come up with an objective way to rate wines.After reviewing tons of data Orley Ashenfelter determined that low levels of harvest rain and high average summer temperatures produce the greatest wines. Bordeaux wines are best when the grapes are ripe and when their juice is concentrated. When the temperature is high, grapes get ripe with lower acidity and when there is low rainfall the fruit gets concentrated. Ripe wine make supple (low acid) wines. Concentrated grapes make full bodied wines. Orley had the temerity to reduce his findings to a formula: Wine quality = 12.145 + 0.00117 x winter rainfall + 0.0614 x average growing season temperature – 0.00386 x harvest rainfall.
The great thing about Orley's formula is that it is objective; the vineyard will know the quality of their wine before it is even harvested and they don't need to pay a lot of "experts" to rate their wines. This is the beauty of crunching numbers. By using regression analysis it is possible to review the independent variables (temperature and rainfall) and determine the dependent variable (the wine rating). Of course the "experts" hate Orley's formula, however in reviewing the wines for the past thirty years the formula has been remarkably accurate. To this day, most wine "experts" do not accept Orley's formula. As Upton Sinclair once said, "it is difficult to get a man to understand something when his salary depends on not understanding it".
Regression analysis is great if you have a lot of historical data and are trying to predict the future (wine ratings, the best baseball players (remember Moneyball?), customers preferences etc.), but what if you don't have any historical data and you want to know what will happen in the future? For instance when a new drug is developed the manufacturer would like to know if it will be effective. Of course this is accomplished by doing trials. In the past it has always been a problem to determine if the drug was really as good (or bad) as the test indicated. Was there some unknown factor that was influencing the results; were both the control and placebo group statistically identical?
Trials are very expensive and because of this the population in most tests was usually small. Randomized testing has solved this problem. Number crunchers can now select a large population for the test and by the flip of a coin can assign the individual to the control group that gets the drug or the group that gets the placebo. Since you know that the both of the populations are homogeneous, due to crunching the numbers, you can be assured that both groups are statistically identical. Because you are now dealing in large numbers with both groups you can be highly confident (although) never positive that the drug helped, hurt or had no effect. To run tests like this requires massive number crunching, something that until the past 15 years wasn't possible. The new computers make this number crunching relatively easy and inexpensive.
In the past decisions were made by intuition and the experience of the decision maker. Those days are gone forever. Super Crunching will be the way decisions are made in the future. As everyone knows companies like google, Facebook, Amazon and even your corner grocery store are collecting data on everyone that will probably amaze us. We are just at the beginning of this process and it is only going to get bigger. There will be some good and some bad that will result. Let's hope for the most part that it is good.
This book gives a pretty good explanation as to how all of this happened and how it will impact us in the future. Intuition and experience are great but crunching the numbers will certainly improve the quality of our decisions in the future.
...moreQuotes:
how much each player is winning or losing. It combines these gambling data together with other infor mation such as the customer's age and the average income in the area where he or she lives, all
Many examples are a bit dated (book from 2007), but underlying principles about the good (and bad) that can be done with near limitless and cheap data still hold. Eg. A/B testing or RCTs, price discrimination, predictions and uncertainty, coupling data sets to get new information, privacy issues.Quotes:
how much each player is winning or losing. It combines these gambling data together with other infor mation such as the customer's age and the average income in the area where he or she lives, all in a data warehouse.
Harrah's uses this information to predict how much a particular gambler can lose and still enjoy the experience enough to come back for more. It calls this magic number the "pain point." And once again, the pain point is calculated by plugging customer attributes into a regression formula. Given that Shelly, who likes to play the slots, is a thirty-four-year-old white female from an upper-middle-class neigh borhood, the system might predict her pain point for an evening of gambling is a $900 loss. As she gambles, if the database senses that Shelly is approaching $900 in slot losses, a "luck ambassador" is dispatched to pull her away from the machine.
But I might at least want the option of having the government make predictions about various aspects of my life. Instead of thinking of the IRS as solely a taker, we might also think of it as an information provider.
Any policy that can be applied at random to some people and not others is susceptible to randomized tests.
Berwick thinks that medical care could learn a lot from aviation, where pilots and flight attendants have a lot less discretion than they used to have. He points to FAA safety warnings that have to be read word for word at the beginning of each flight. "The more I have stud ied it, the more I believe that less discretion for doctors would improve patient safety," he says. "Doctors will hate me for saying that."
When Britto started learning how to fly an airplane back in 1999, he was struck by how much easier it was for pilots to accept flight sup port software. "I asked my flight instructor what he thought ac counted for the difference," Britto said. "He told me, 'It is very simple, Joseph. Unlike pilots, doctors don't go down with their planes."
In contrast to these human failings, think about how well Super Crunching predictions are structured. First and foremost, Super Crunchers are better at making predictions because they do a better job at figuring out what weights should be put on individual factors in making a prediction. Indeed, regression equations are so much better than humans at figuring out appropriate weights that even very crude regressions with just a few variables have been found to outpredict hu mans. Cognitive psychologists Richard Nisbett and Lee Ross put it this "Human judges are not merely worse than optimal regression way, equations; they are worse than almost any regression equation."
Even the best studies need to be interpreted. Done well, Super Crunching is a boon to society. Done badly, database decision making can kill.
The rise of Super Crunching is a phenomenon that cannot be ig nored. On net, it has and will continue to improve our lives. Having more information about "what causes what" is usually good. But the purpose of this chapter 7 has been to point out exceptions to this general tendency.
This inability to speak to one another about dispersion hinders our ability to make decisions. If we can't communicate the probability of worst-case scenarios, it becomes a lot harder to take the right precautions.
...moreAlso I think the book is a little outdated in the year 2020 to that effect that Ayres prevalently only explains what "Super Crunching" is able to do. There are nice insi In my opinion, Ayres used the term "Super Crunching" way too inflationary. Yes, the book's title was derived by "super crunching" (simple A/B testing) itself, which is cool, we got it. However, instead of stressing a term that does not really have any technical meaning, Ayres could have written the book in a less repetitive way.
Also I think the book is a little outdated in the year 2020 to that effect that Ayres prevalently only explains what "Super Crunching" is able to do. There are nice insights but it would have been delighting to not only read about how companies can benefit from "Super Crunching" but also how customers can profit, apart form being saved from accidentally buying 12 lemons. I think it would have also been worth to discuss the (social) consequences of permanently crunching user's data.
All in all, it's an okay book (more from a statistics point of view rather than from a big data perspective) with nice anecdotes and insights but nothing groundbreaking.
...moreThis book was all about how statistics are taking over how we understand life and make business decisions, decisions about love, about farming, about crime, about health, and more, based on statistics rather than intuition. I got the book years ago for my girlfriend at the time, and only just got around to reading it.
The biggest weakness the book has now is that it's woefully out of date in 2020. So much has changed in the last 13 years, and I would be curious to
I listened to the audio version.This book was all about how statistics are taking over how we understand life and make business decisions, decisions about love, about farming, about crime, about health, and more, based on statistics rather than intuition. I got the book years ago for my girlfriend at the time, and only just got around to reading it.
The biggest weakness the book has now is that it's woefully out of date in 2020. So much has changed in the last 13 years, and I would be curious to see a new version that updates the content. It makes a lot of good points about how statistics can be used for all sorts of positive purposes to improve our lives--I just wonder if the author has made any changes to his own thoughts since writing this book.
Also I am curious to learn more about Direct Instruction!
...moreThe cheesy subtitle makes it sound like a bad self-help book, but it's really a good, broad overview of the different ways that people are starting to use statistics more heavily in their decision-making. Basically, he explains that people are able to collect and store loads more data than ever before, so the "Super Crunchers" are fitting very accurate equations to the data -- and many of their formulas are better at making accurate predictions than season
Lots to think about. Highly recommended.The cheesy subtitle makes it sound like a bad self-help book, but it's really a good, broad overview of the different ways that people are starting to use statistics more heavily in their decision-making. Basically, he explains that people are able to collect and store loads more data than ever before, so the "Super Crunchers" are fitting very accurate equations to the data -- and many of their formulas are better at making accurate predictions than seasoned experts with good intuition.
In a way, it's similar in style to Malcolm Gladwell's "Blink" but the opposite in its moral: Gladwell says to trust your intuition more than you think, and Ayres says to trust (experts') intuition less than we might think.
One thing that struck me was the idea of whether to trust the data and the resulting equation even when you don't understand the underlying mechanisms that the equation describes. For example, in the 1840s, a doctor named Semmelweis found that there was a huge drop in mortality rates at hospitals if doctors and nurses washed their hands. The germ theory of disease was not known yet, so Semmelweis couldn't explain *why* washing your hands would help keep your patients alive - he just knew it would. But many doctors ridiculed Semmelweis and his theory, complaining that hand-washing was a waste of time.
It's clear to us in retrospect that those other doctors should have listened to the man with the data instead of continuing to unwittingly kill their patients by refusing to wash their hands. But still today, too often, doctors tend rely on their intuition and the theories they learned in med school years ago, even if new studies have come out that show a different diagnosis or treatment would be better.
The same argument goes on in other fields. Ayres describes a scripted, structured, rote-learning method for elementary school instruction called Direct Instruction (DI). This seems entirely contrary to the creative, open, exploratory, problem-solving, student-centered philosophy that many people (including myself) subscribe to. But even though DI seems wrong and illogical, apparently DI students tend to perform significantly better on all kinds of tests - even ones that "required higher-order thinking," says Ayres. So should we stick with our instincts about what's a good way to teach our kids? Or should we go with the method that's been proven to give better results, even if it's anathema to our philosophy?
Of course, you can't have this argument if you're not able to measure your desired endpoint numerically. So you can argue that student performance on standardized tests is not how we should be evaluating our schools, and then you can ignore DI's successes that way. But then are we sure we're not just being like those 1840s doctors, saying "I expect that shouldn't work, so I'll ignore it," and possibly hampering millions of children's educational potential by ignoring something that does work?
Ayres clearly believes that expert opinions and theories are well and good when you can't crunch the data; but if good data is available and can be crunched, by all means trust it over the experts. Why argue about whether or not method A *should* be better than method B, when you can find out which one *does* perform better? It's a valid point and an important problem to think about.
Also, apparently some people have found ways to accurately predict the profits of a movie based solely on aspects of the script (ignoring the performance of actors, director, cinematography, etc). It's weird to think that the movie studios might begin rewriting scripts to maximize the expected profit based on an equation. Won't that take all the art out of it? But of course, Ayres points out, the studios already make many "artistic" decisions based purely on commercial (rather than artistic) merit. So all the equation does is improve the quality of those commercial decisions. If they're going to mess with your wonderful screenplay anyway, they may as well mess with with in the right way, instead of relying only on hunches about what works.
He has plenty of examples of cases where Super Crunching has been used for great good, as with MIT's Poverty Action Lab and similar groups that run experiments to find out what policies actually work in reducing poverty, and thus are able to convince governments to make the proven choice. On the other hand, there are also many scary examples of casinos crunching your personal data to figure out exactly how much money you can lose and still have a good time, or credit card companies learning exactly what interest rate to offer you to keep your business. Obviously businesses try to do this all the time - cutting costs and increasing revenue is what businesses are about, after all - but the degree to which they're now using your personal data (collated from many different sources, no less) is pretty intimidating.
Naturally, it sounds scary to think that our doctors, teachers, and everyone else should give up complete control over what they do to a bunch of formulas. But it's not all controlled by computers. At the top, somebody has to be in charge of the data, deciding what information to collect and what variables to put in the regressions and so on. So I wish Ayres talked more about the titular "Super Crunchers" themselves, not just their equations. Who becomes a super cruncher? What's it like to have a job where you create regression models, design experiments, and collect data? If this is the wave of the future, how should we train our students to be good at dealing with these huge data sets in a professional and responsible way?
...moreAyres teases at high level overview, then undermines himself with pedantry. This is a simple, short, and yet tiresome read.
My bias on the subject has me underwhelmed by this book.
This was published in 2007, and yet, despite Ayres trying valiantly to impress his term "Super Crunchers" on the reader, it doesn't seem to have made its way into the lexicon. Now, "big data" has, and that is essentially what this is about, with a little more on statistical analysis - regression and standard deviations.Ayres teases at high level overview, then undermines himself with pedantry. This is a simple, short, and yet tiresome read.
My bias on the subject has me underwhelmed by this book. I recommend Big Data by Timandra Harkness for a better, high level look at, well, ... big data.
...moreNews & Interviews
Welcome back. Just a moment while we sign you in to your Goodreads account.
Source: https://www.goodreads.com/book/show/1081413.Super_Crunchers
Posted by: reneasiecke0199259.blogspot.com
Post a Comment for "Super Crunchers Ian Ayres Pdf Download"