![\"IPA-Media-Release-ABC-Fossil-Fuel\"](\"https:\/\/i0.wp.com\/nofibs.com.au\/wp-content\/uploads\/2014\/08\/IPA-Media-Release-ABC-Fossil-Fuel.jpg?resize=800%2C400\")
<\/p>\n<\/p>\n
Innumerate Ignorance or Calculated Deception<\/strong><\/p>\n It is unfortunate that the IPA, and in particular James Paterson, did not take note of the \u2018scholarship\u2019 referred to in their recent report<\/a> of the ABC\u2019s coverage of \u2018Australia\u2019s energy choices\u2019 and apply it to their own publication. It would undoubtedly have saved them the embarrassment of making public a deeply flawed piece of political rhetoric loosely dressed up as a quasi research effort that would never pass the most basic academic peer review.<\/p>\n A mere glance at the report\u2019s title is sufficient to sound warning bells that the proffered document may not tick all the ethical research boxes. Whilst \u2018Public Broadcaster or Green Activist?\u2019 smacks of a political journalistic headlining it is the follow up line \u2018How the ABC spins Australia\u2019s energy choices\u2019 that defines what is to come which is certainly not the objective approach real scientific research aspires to.<\/p>\n Does it matter? Well yes, it does. The IPA makes some serious allegations against the ABC in their report and supports the claims on the basis of their landmark scientific research.<\/p>\n \u201cThis report marks the first time a comprehensive, independent and scientific assessment of ABC bias has been undertaken.\u201d\u00a0<\/strong><\/p><\/blockquote>\n Furthermore, the IPA is an Approved Research Institute and hence qualifies as a deductible gift recipient whereby any donations to the IPA are tax deductible. The tax laws<\/a> pertaining to this status are clearly defined. The IPA may only use the tax-deductible donations for the purpose of scientific research.<\/p>\n The IPA states on its website<\/a> that it relies solely on the financial support of its members and donors. If the IPA has indeed funded this report from tax-deductible donations, to be legally compliant proper scientific research methodology and reporting does matter, a lot.<\/p>\n So, to the report itself. A legitimate basis for the IPA research goes like this. Every form of communication is \u2018framed\u2019 in some way so that a desired message is imparted. Framing may be implicit or explicit, objective or subjective. Media informs citizens and this facilitates decision-making. Consequently the media is a significant and powerful factor in the formation of public policy. Media stories that are consistently framed in a way that favors one particular view may therefore unduly influence the formation of public policy. Whilst this naturally pertains to the ABC as a media outlet the ABC is also specifically required to \u2018broadcast in the interests of all Australian citizens\u2019 as it is publically funded by the taxpayer. The IPA has reservations about the ABC\u2019s impartiality and consequently wished to conduct a scientific assessment of the ABC\u2019s broadcasts in order to determine whether there was a basis to their concerns.<\/p>\n Sounds good? Reasonable, considered, objective? Unfortunately this is not how the IPA conducted itself. I wrote this research proposal drawing upon loose threads scattered throughout the report and moderated emotive language.<\/p>\n What in fact the IPA proposed was:<\/p>\n \u201cThe ABC has a persistent and systematic left wing bias in its coverage of contemporary Australian political issues\u2026The ABC is required by law to provide an impartial news service\u2026 the Australia Network [failed] to broadcast Australian values\u2026[the energy mix has] been one of the most contentious political issues over the last decade\u2026the government should adopt an agnostic approach to energy sources, allowing the best technology to prevail without distortionary [sic] subsidies\u2026biases are of course inevitable in any communication of issue\u2026one reason that the ABC should be privatized\u2026[the ABC] is not a welcome home for conservatives or classical liberals\u2026\u201d\u00a0<\/strong><\/p><\/blockquote>\n The IPA\u2019s \u2018independent review\u2019 is characterized by pre-existing assumptions: the ABC is biased and has a political agenda, the only solution is privatization, and the renewable energy industry is unfairly advantaged over the coal and CSG industries. The agenda is set. Now to the gathering of the data.<\/p>\n The IPA commissioned media analysis firm, iSentia, \u2018to conduct a third party assessment of ABC bias\u2019 using a media measurement methodology called CARMA<\/a>. Media measurement evaluates the effectiveness companies, organizations or associations communicate their message to their target audience through various media channels.<\/p>\n \u00a0<\/em>\u201cTo develop the best possible measurement program for our clients CARMA begins each assignment with a consultation to study [its] goals and objectives\u2026[we work] with our clients to determine Key Performance Indicators (KPIs) that will gauge our clients\u2019 success against key objectives\u2026\u201d\u00a0<\/strong><\/p><\/blockquote>\n The IPA, not the ABC, is the client in this case. Consequently, the CARMA framework used in this study necessarily reflects the IPA\u2019s position and measures the ABC\u2019s reports against the IPA\u2019s key performance indicators. The IPA\u2019s viewpoint is that the ABC has a persistent left wing bias, and that economic growth and environmental protection are conflicting political policies. IPA\u2019s goals and objectives include privatizing the ABC and the removal of renewable energy subsidies. Economic benefits, jobs, royalties, tax contributions are \u2018beneficial\u2019, environment, public health, economic and social impacts are \u2018harmful\u2019.<\/p>\n The report does not reference any of the ABC stories used in the assessment, nor does it clarify what defines a story as being about a particular energy industry. Does a story<\/a> about the prosecution of a hoax emailer that resulted in a short-term drop in Whitehaven Coal\u2019s share price qualify as a coal industry story? If so, would it be deemed an unfavorable story because it reported a negative economic impact? James Paterson did not respond to my emailed request for a sample of the ABC reports iSentia selected.<\/p>\n For these reasons alone the value of the data collected in this study could be regarded as compromised. However, James Paterson, author of the IPA report, really destroys the data\u2019s creditability with this acknowledgement:<\/p>\n \u201cThe choice of interviewees reflects the fact that the coverage on coal mining was dominated in the sample period by the Abbot Point controversy.\u201d\u00a0<\/strong><\/p><\/blockquote>\n The sample of stories can hardly be considered representative of coal industry coverage when a single inflammatory issue with enormous environmental implications predominates. What this highlights is that six-months is too short a time period for a study of this type. Events such as elections, natural disasters, celebrity and controversial decisions attract intense nationwide media coverage. A much longer timeframe than the six months used in the IPA study would be necessary to ensure a truly representative cross section of stories can be sampled. What Paterson\u2019s comment also identifies is that he is ignorant of, or chooses to ignore, basic research sampling methodology.<\/p>\n So what did the IPA and Paterson do with the figures they arrived at, such as they were? Pages were covered with reams and reams of tables and bar charts, interspersed with the author\u2019s subjective commentary before concluding with a summary table and a repetition of the IPA viewpoint. A third of all articles referenced were the IPA\u2019s own publications and a table of relative energy costs was unreferenced. Statistical analysis was non-existent and large chunks of data were ignored. It would require a thesis length article to comment on all the \u2018findings\u2019 and it would be an exercise in the unnecessary.<\/p>\n At times Paterson demonstrates he clearly doesn\u2019t have the mathematical skills to understand his own figures or he is being deliberately misleading, or both. He repeatedly compares the categories \u2018favorable\u2019 to \u2018unfavorable\u2019 without defining what benchmark would, in the IPA\u2019s opinion, constitute impartial representation, a must given that that is the point of the research. Furthermore, the figures for \u2018neutral\u2019, which is a measurement of impartiality in itself, are largely ignored or given a negative connotation.<\/p>\n For example, of coverage specifically of the economic impact of the coal industry 57% of stories were identified as neutral and 43% framed, 59% of which were favorable and 41% unfavorable. Favorable representation and impartial framing are the winners here but the IPA does not appear to agree saying that:<\/p>\n \u201c\u2026a high proportion of stories focused upon the negative [economic] impact of the coal industry. Stories which recognized the positive benefit of the industry were significantly out numbered by those which treated the issue neutrally.\u201d\u00a0<\/strong><\/p><\/blockquote>\n The use of the term \u2018significantly\u2019 in a quantitative research study suggests statistical analysis has been used to support a conclusion. This clearly was not the case as such an analysis would have found that the proportion of stories that treated the economic benefits of the coal industry neutrally was not significantly different from those that treated it favorably (p > 0.1<\/a>).<\/p>\n As part of the CARMA analysis the tone of the language used in each story is evaluated as being of one of five types. \u2018Reason\u2019 and \u2018pragmatism\u2019 use rational argument and constructive terminology, \u2018fear\u2019 and \u2018hope\u2019 are characterized by negative or positive emotions and \u2018cynicism\u2019 by doubt. Pragmatism and reason would have to be considered admirable traits in any media story implying a common sense logical presentation of information. Paterson consistently ignores the majority proportion of stories that fall into these two categories and focuses solely upon the stories using emotive language. Once again he does not articulate the benchmark by which he judges impartiality, and once again he has misrepresented some outcomes to suit the IPA agenda.<\/p>\n The IPA contends that the ABC highly favors the renewable energy industry and this is reflected in the language used in the stories. Paterson does not elaborate as to why the \u2018overwhelming\u2019 number of reasoned and pragmatic stories is a criticism of the ABC reporting style but he certainly highlights his own bias in his representation of the emotive language indicators. The 93 stories of hope were deemed a \u2018significant\u2019 number but \u2018stories which deployed the language of fear and cynicism were extremely rare\u2019. From the graphical representation in the report it would appear 98 stories used the language of fear and cynicism.<\/p>\n The choice and mixture of interviewees a story quotes is regarded as \u2018one key mechanism by which frames are applied\u2019. Comments of the most frequently quoted spokespeople were collected and categorized as to whether they were favorable, neutral or unfavorable to the industry of interest. The IPA report found the \u2018analysis demonstrates that those who are interviewed is [sic] strongly correlated with the overall favorability rating of the story.\u2019 The IPA did not reference any test of correlation applied to the data to support this conclusion. Nor would a correlation have been found.<\/p>\n Greg Hunt, the Federal Environment Minister, was quoted in 29 stories about the coal industry and Green\u2019s Senator, Larissa Waters, in 25. The Greg Hunt quotes were evenly spread; 10 favorable, 9 neutral and 10 unfavorable, and Larissa Waters\u2019 were all unfavorable. Stories that quoted Greg Hunt were found to have an average favorability rating of 51.7 and those that quoted Larissa Waters had an average favorability rating of 46. Statistical analysis finds that these two ratings are not significantly different from each other or from 50, the neutral midpoint (p > 0.1<\/a>). Even though all Larissa Waters\u2019 quotes were unfavorable, on average the framing of the containing stories was neutral suggesting the ABC counterbalanced Larissa Waters\u2019 views with favorable comments about the coal industry.<\/p>\n The IPA report is fundamentally flawed on many levels. The subjective and inadequate sampling procedure, the lack of any analytical methodology and the inappropriateness, deliberate or otherwise, of some of the \u2018findings\u2019 leaves an enormous question mark over the entire report. It is in the conclusion, however, that the question mark is removed. The conclusion provides the definitive example of the IPA and Paterson\u2019s deliberate misrepresentation of figures to fit a preordained objective.<\/p>\n In summing up, Paterson presents the CARMA Favorability Rating, as produced by iSentia. The rating is an ultimate summary figure produced by the CARMA methodology to describe the ABC framing of stories about the coal, coal seam gas and renewable energy industries. It is:<\/p>\n \u201cA measure of quality that takes into account the tone, sources, messages and prominence of a story. As an overall metric, it provides the \u201caverage rating per story\u201d across all analyzed stories.\u201d\u00a0<\/strong><\/p><\/blockquote>\n And these are those figures as per the IPA report.<\/p>\n Favorability Rating \u00a0(where 50 is neutral)<\/strong><\/p>\n