Identifying Quackery (and Other Mistakes)
Source: http://feedproxy.google.com/~r/typepad/WuQQ/~3/5D7R3cjtQi4/identifying-quackery.htmlPosted on Tuesday, October 6th, 2009 | In Investing Lessons, Market Commentary
div xmlns=”http://www.w3.org/1999/xhtml”PAt “A Dash” one of our principal missions is identifying the best sources of information on a given topic.nbsp; This is a major source of value, if done accurately.nbsp; In the investment blogosphere, however, the ratings mostly reflect the predispositions of the reading public./P
PstrongThe Path to Success in Blogging/strong/P
PIt is easy to figure out what works.nbsp; One needs only to look at the top-ranked blogs and those most followed on Seeking Alpha.nbsp; There is an aggressive cult following (a subject for another day) for specific perspectives on the economy, the role of government, the existence of conspiracies, and the stock market.nbsp; If one’s objective is high ratings, this is a blueprint for a strong business model./P
PstrongThe Path to Success in Investing/strong/P
POne of our missions is how to profit from this.nbsp; It means reading many perspectives and evaluating arguments and data.nbsp; As part of our work (eventually a book) we are analyzing various sources, with an emphasis on factual accuracy, freedom from bias, professionalism, and analytical skill./P
P style=”TEXT-ALIGN: center”emstrongDo not start with what you believe.nbsp; Start with data./strong/em/P
PstrongA Case in Point/strong/P
PWe offer the interpretation of various polls as a case in point.nbsp; This is partly based upon expertise in developing, conducting, and interpreting poll data.nbsp; We have done all of these, and most others have not.nbsp; Let us consider some experts in action./P
POur first expert is A href=”http://modeledbehavior.com/about/” target=_blankKarl Smith/A, of the University of North Carolina at Chapel Hill, now added to our list of featured blogs.nbsp; He has really stepped up his role in blogging, taking an expert public policy perspective on many issues.nbsp; He deserves to be widely read./P
PSmith takes a current poll and does a careful analysis.nbsp; The poll purports to show some A href=”http://modeledbehavior.com/2009/10/03/in-soviet-russia-poll-questions-ask-you-strategic-vision-controversy/” target=_blankterrible answers to some obvious questions./Anbsp; For example, What is the Supreme Law of the Land?nbsp; According to the poll, only 28% of Oklahoma students identified the Constitution and some offered up the Monroe Doctrine./P
PThis might be a true result, in a multiple choice test with attractive alternatives.nbsp; We once saw a similar survey of undergrads where most thought that the Electoral College was a small liberal arts school in Minnesota!/P
PBut Smith delves deeper into the data, which purported to permit free-form answers.nbsp; Look at his which is very strong, but here is an example:/P
blockquote
PThink about this from a student’s perspective. The first question uses the phrase “Supreme Law.” This means that answer has to be something important. The natural thing to do is to answer the most important thing you can think of. From free response I would not be surprised to get answers like “the bible”, “democracy” or “freedom”.nbsp; The student might also simply free associate with things that are like Supreme or like Law. Hence, I would not be surprised to get answers like “Federal government”, “Supreme Court”, “Congress” or even “Roe vs. Wade.”/P
PSince none of these answers were given, but Monroe doctrine was. I find this hard to believe unless the poll was multiple choice and the six answers were the only answers a student could give./p
/blockquote
PHe has accurately identified a false note in the poll./P
PFor our second expert we also take note of the fine work from A href=”http://www.fivethirtyeight.com/2008/05/no-im-not-chuck-todd.html” target=_blankNate Silver/A at fivethrityeight.com, now also added to our list of featured sources.nbsp; We have occasionally been accused of credentialism in identifying sources, mostly by those thinking that having a PhD in economics is worthless.nbsp; Silver is a fine contrary example.nbsp; He developed strong expertise in analyzing baseball statistics.nbsp; Anyone with statistical expertise can find many applications.nbsp; He has done so in election polling and political analysis, useful to anyone analyzing public policy.nbsp; We watch his work carefully./P
PHis take on the dubious polling data is summarized as follows:/P
blockquote
Pspan id=fullpostFor me, some of these results don’t pass the smell test. I agree that public schooling in the United States needs to be improved, particularly in the areas of government and citizenship. But only 23 percent of high school students in Oklahoma knew that George Washington was the first President? span style=”FONT-STYLE: italic”Really?/span I have difficulty accepting that claim at face value. In 2008, 68 percent of Oklahoma span style=”FONT-STYLE: italic”fifth/span graders passed the Oklahoma Core Curriculum Social Studies Test. You can read some of the questions on that test beginning on page 50 of this A href=”http://sde.state.ok.us/AcctAssess/pdf/Core/PSTG/Gr5PSTG.pdf”PDF/A; they’re generally quite a bit more difficult than the ones that Strategic Vision asks. (For instance, “Which was the most profitable export of the Jamestown settlement?” and “Which group would most likely agree with ideas presented in span style=”FONT-STYLE: italic”Common Sense/span?”). So either those smart fifth graders were span style=”FONT-STYLE: italic”really/span forgetful by the time they got to high school, or there’s something very wrong with this poll./span/p
/blockquote
Pstrongspan id=fullpostInvestment Implications/span/strong/P
Pspan id=fullpostThere is a special skill involved in interpreting data.nbsp; Surveys are the source of much of the information that investors and traders try to interpret.nbsp; Few are really expert in how the polls are conducted, the statistical significance of results (often not reported by the source), and the reasons and basis for revisions./span/P
Pspan id=fullpostIt is foolish to identify the source of the poll, and then dismiss the result if one does not like it./span/P
Pspan id=fullpostOur example does not have immediate investment implications, and that is intentional.nbsp; It is much easier to step away from the economic polling data, looking instead at an obvious error as a case in point.br/span/P
Pspan id=fullpostThen come back to the news of the day.nbsp; CNBC featured today a poll with a response rate of 25% of the CFO’s attending a convention.nbsp; Most of them thought we were “still in a recession.”nbsp; While we are interested in CFO opinion about current corporate plans, none of them are experts at recession dating.nbsp; Mostly they are saying that the economy is still well below trend, something we already know./span/P
Pspan id=fullpostThere is much more to analyzing polling data than looking at a snapshot of current opinion.nbsp; Longitudinal analysis is a starting point, something that most of us do when looking at survey series like the ISM data.br/span/P
P/P/div
pa href=”http://feedads.g.doubleclick.net/~a/N5UOHay2AQ6dMSC8KQrA84pf3iY/0/da”img src=”http://feedads.g.doubleclick.net/~a/N5UOHay2AQ6dMSC8KQrA84pf3iY/0/di” border=”0″ ismap=”true”/img/abr/
a href=”http://feedads.g.doubleclick.net/~a/N5UOHay2AQ6dMSC8KQrA84pf3iY/1/da”img src=”http://feedads.g.doubleclick.net/~a/N5UOHay2AQ6dMSC8KQrA84pf3iY/1/di” border=”0″ ismap=”true”/img/a/pimg src=”http://feeds.feedburner.com/~r/typepad/WuQQ/~4/5D7R3cjtQi4″ height=”1″ width=”1″/
Last 5 posts by Jeffrey Miller
- A Tough Nut to Crack - October 29th, 2009
- ETF Update: Looking to the Internet - October 25th, 2009
- Healthcare Reform Becoming Less Likely - October 21st, 2009
- ETF Update: Another Look at the Banks - October 18th, 2009
- Evaluating Predictions - October 5th, 2009
![]() About Jeffrey Miller (http://www.oldprof.typepad.com)
Jeffrey A. Miller, Ph.D. is a former college professor with a hands-on, real world attitude. His quantitative modeling helped inform state and local officials in Wisconsin for more than a decade. A Public Policy analyst, he taught advanced research methods at the University of Wisconsin, and analyzed many issues related to state tax policy. In 1987 Jeff began work for market makers at the Chicago Board Options Exchange. His approach included finding anomalies in the standard option pricing models and developing new forecasting techniques. Merging these quantitative techniques with specific company analysis, Jeff also generated trading ideas from sell-side analyst reports. Through his years of experience in trading options, futures and equities, Jeff has come to be regarded as an expert in interpreting the effect of news on the markets and individual stocks. Jeff has served as a forensic expert in several cases involving such issues. He has also written a series of papers on investment management, describing both quantitative methods and those related to behavioral economics. |



