Conclusions and Research Questions

1 Leave a comment on paragraph 1 0 This paper has explored what impact means, and the details of both tradmetrics and altmetrics as tools for understanding the impact of scholarly research. Tradmetrics are built upon decades of study. Altmetrics aim to take advantage of a wealth of newly available data that describe impact from an alternative viewpoint. This paper constructs an argument that the value of social learning communities (of practice) are not represented by either approach, and hypothesises that further investigation may show that this impedes the ability to provide a multidimensional understanding of impact.

2 Leave a comment on paragraph 2 4 Technical innovations, some of which are core to the availability of data that altmetric evaluations are based upon, may well be repurposed to explore the hypothesis. The ability to mash-up data from multiple sources, and a movement toward semantic nanopublishing (García-Castro et al. 2012; Groth & Gurney 2010; Hori et al. 2003; Corcho 2006), provide the technological underpinning to do this. Other approaches to publishing data openly and semantically (e.g. linked data, Database Wiki Project) may also be applied to publishing (TED 2009; Bizer et al. 2009; Buneman et al. 2011).

3 Leave a comment on paragraph 3 3 Although this paper is concerned with exploring a impact metrics using CoP theory, this exploration is part of a much larger picture and more complicated picture. The closed peer-review process utilised by the majority of academic journals arguably helps to prolong a preoccupation with citation-based impact factors (Curry 2012, cited in Galligan & Dyas-Correia 2013). A very recent example of how this can be problematic is the story of Reinhart, Rogoff and Herndon. The latter is a graduate student, who studied a notoriously cited paper by the two eminent economists (a paper oft-cited by finance ministers as an argument for austerity). Herndon noted some fundamental errors in the calculations used in the paper (Alexander 2013, Herndon & Pollin 2013), which had been peer-reviewed, and published in well ranked journal (SCG n.d.).

4 Leave a comment on paragraph 4 1 Although the sentiment is very hard to support empirically some suggest this kind of event could be the product of a highly politicised publishing community: “maybe if peer review was about methodology instead of politics this wouldn’t keep happening” (Ralph 2013). Assuming this is a politicised process, it maybe out of self-interest some scholars don’t shout louder about the issues. What is empirically supported (and referenced in this paper) are the structural flaws of how impact is calculated, and metrics interpreted. Because these practices are entrenched in long-established power and authority structures built atop the current publishing paradigms, altering behaviour is a multilateral issue.

5 Leave a comment on paragraph 5 2 Academic blogging and post-publication peer-reviews are good examples of the benefits of negating or augmenting the traditional peer-review/publication process (Galligan & Dyas-Correia 2013; Groth & Gurney 2010). Also evidence appears to conclude that open-access publications (according to tradmetrics) have wider influence than closed-access publications (Harnad & Brody 2004). Linking these studies to the main argument the paper presents, if CoP theories can be shown to have relevance to academic publishing communities – and to be a valid factor in a holistic impact metric – then the next logical line of enquiry would be to establish whether open review, open access publishing, and publishing outside of traditional venues can stimulate better CoP than traditional models.

6 Leave a comment on paragraph 6 0 Based upon the position established in the paper, the following research questions are proposed to inform further study:

  • 7 Leave a comment on paragraph 7 2
  • Is applying communities of practice theory to the social learning that takes place as a result of scholarly publishing a valid thing to do?
  • How can the value of academic communities of practice, arising from connections made via scholarly publishing, be measured?
  • Can the value of academic communities of practice become part of a holistic and multidimensional impact metric?
  • In what ways could describing emergent academic communities practice improve the process of literature searching and research?
page 8

Source: http://communitiesofimpact.joesart.org/conclusions-and-research-questions/?replytopara=2