Research and Advances
Artificial Intelligence and Machine Learning Virtual extension

Computing Journals and Their Emerging Roles in Knowledge Exchange

Posted
  1. Introduction
  2. Data Collection and Results
  3. Discussion
  4. Conclusion
  5. References
  6. Authors
  7. Footnotes
  8. Figures
  9. Tables

Scholarly journals are reliable means of communicating knowledge and findings in a scientific discipline. Researchers rely on articles published in the journals to support and extend their work while acknowledging them in the form of citations. A journal (citing) sends a citation to another journal (cited) when an article published in the former makes a reference to an article published in the latter. This creates a network in which journals are connected by citations made to each other. These journals play three unique but overlapping roles as sources, storers6 and synthesizers2 of knowledge communication. An understanding of these roles is useful for researchers, editors and institutions alike. Earlier studies have used various perception and citation based measures1,2,4,5,6 to assess the influence of journals. However their roles and influence as sources, storers and synthesizers remains unrecognized, in general. Also, a smaller sample of journals as used in the past studies has a likelihood to bias the results because of the inadequate representation and omission of relevant journals from the network.12

This study addresses the interconnectedness of journals from four perspectives. First, it expands the network to include 50 computing journals to enhance the reliability of results. Second, it simultaneously analyzes the importance of journals as synthesizers, sources and storers of knowledge communication in the network. Third, since the dynamism and “continuous evolution” of computing discipline warrants journal ranking studies on regular basis,9 it analyzes data from 2000 to 2004 and extends the time-period used in some recent studies. Finally, it draws on visual analysis of the inter-citation network along with dependence calculations to provide an insight about the domains and roles of computing journals.

A journal’s role as a knowledge source is based on the number of citations received. A journal tends to be cited more if its articles are perceived as important and are regularly utilized by researchers.6,8 The role as a storer can be judged by the extent of journal’s assimilation of knowledge from various sources. Articles in such journals mainly illustrate, test or apply existing knowledge to enrich and build cumulative tradition of the field. As a synthesizer, the journal plays the role of maintaining order and achieving cohesiveness in the network.2 Their interconnection is based on significant reciprocal citations and results in independent clusters of mutually dependent journals. Such clusters are linked by boundary-spanning journals that act as a bridge between various clusters. Without synthesizers, the clusters either get disconnected from one another or get fragmented into smaller entities. A network with low-level of cohesiveness reflects the lack of consensus whereas a highly cohesive network reflects the presence of harmony among its members.

Back to Top

Data Collection and Results

One of the main objectives of this study is to analyze data from a larger set of computing journals. For the purpose of this study, we define “computing journals” as the journals having focus on research related to the development, usage and benefits of computer and software systems for various purposes, the realm of computing discipline.10 Because a field is represented more by its own journals rather than by journals with a multi-disciplinary focus, some elite journals like Management Science etc. are therefore not included in this study.4 To this end, we examined relevant past studies and included all computing journals present in at least two of the studies. We also added journals that have recently gained importance and were present in one of the recent studies. Journals not indexed in ISI Web of Knowledge or those with insufficient data were also not included. Finally, 50 journals were used to create the network for analysis (Table 1). It is approximately twice the number of journals used in some recent studies. The ISI was used to obtain a list of all articles referenced between 2000 and 2004 for each of these journals. The frequency of citations sent by each journal to others in the network was computed and an asymmetric matrix was created where rows and columns represented the citing (sending) and cited (receiving) journals respectively.6,8 The value in each cell represents the number of citations sent from the citing to the cited journal and the diagonal entries represent self-citations.

LEM software11 was used to evaluate the log-linear and log-multiplicative models for the dimensions of association between sending and receiving journals in the network.6,8 Log-multiplicative model assesses a journal’s relative influence as a source and storer of knowledge by adjusting for selfcitations and variation in number of citations received or sent by a journal.6,8 Based on Bayesian Information Criteria, a log-multiplicative model with eight dimensions of association was determined as the best fit.6,8 The loglinear parameters were calculated by the software and used to estimate the journal’s relative influence as a source and storer of knowledge. The hierarchical clustering of journals was analyzed using Ward’s method. This identified clusters of journals using the scores of each journal along the dimensions of association between sending and receiving journals.8 Journals in these clusters were examined to identify the subject domains represented by the clusters and to determine the relative ranking of journals within their own domains and disciplines (Figure 1).

Entropy concepts from Information Theory were used to determine the extent to which each journal contributes towards achieving stability in the network.2 The cohesion between the journals was determined by examining the strength of interconnections based on their mutual citations. For this, we followed Cooper et al.2 which recommend an appropriate threshold to be the one at which the larger cluster in the network starts breaking into smaller clusters. While a lower threshold connects most of the members together, larger threshold fragment the network and disconnect most members from each other. For example, while all journals would get connected into a single cluster at 0% threshold, the network would break into 50 independent clusters, with each having a single journal at 100%. We found that the 5% threshold met the Cooper’s guidelines for our set of journals.

A Knowledge Communication graph was developed in which two journals were connected if at least 5% of one’s citations were sent to the other (excluding self-citations) and vice-versa (Figure 2). Such a network consists of one big system with smaller and independent clusters linked together by boundary-spanning journals. We also calculated the entropy of the network consisting of all journals (=133.59) and the entropy after excluding a journal from the network. The difference in the two values (entropy index) reflects the journal’s effect on network stability and its role as a synthesizer in the network. A positive score indicates the absence of journal lowers the stability and cohesiveness of the network, thereby indicating its importance in the network. A high negative score reflects the journal adds to the instability of the network, whereas a low negative score signifies the journal has little effect on the network’s cohesiveness.2

Table 1 shows the estimates of the journals’ parameter for receiving citations, sending citations and entropy index with their corresponding ranks as sources, storers and synthesizers in the network.

Back to Top

Discussion

An analysis of the expanded data in this study provides some interesting insights into the roles of computing journals. The results validate CACM as the premier source of knowledge for other journals in the network. This is followed by IEEE Computer, MISQ, IEEETSE, and AI in the 2nd to 5th positions respectively. The presence of two practitioner oriented journals among the top three positions reflects the importance of studies related to practical applications of computing. The high rankings received by technically oriented journals reflects their usefulness to researchers, which often goes unnoticed by the general audience.4

The dendogram derived from hierarchical clustering indicates the journals can be separated into two groups, “Technical” and “Socio-technical.” The technical group consists of journals related to Computer Science and Software-Engineering. For example, journals like JALG, JCMS and JACM in the “Problem Solving” domain focus on mathematical theories pertinent to the field of computing and algorithms. Journal in “Data Engineering” cluster (for example, IEEETKDE, ACMTDS, INFOSYS, ACMTIS) publish articles related to design and implementation of data models, data engineering methodologies, strategies and systems. The “Practitioner Oriented Journals” cluster includes journals like SPE, CACM, COMPJ and IBMSJ address the interests of computing professionals’ and interpret theoretical knowledge in terms of practical applications. Journals like IEEESW, JSSW, IEEEETSE, and ACMTOSEM in the “Software-Engineering” domain publish papers on topics such as design, development and management of software. Journals like BIT, IJHCI, HCI and IJHCS focus on topics related to human-computer interaction and constitute the cluster labeled “Computer Interactive Systems.” The cluster having journals like ESA, SMC AIMAG, AI, ML, and IEEESMC represents the fields of Intelligent Systems and Artificial Intelligence. We named this cluster “Computational Intelligence.” This connects to “Computer Interactive Systems” to form a cluster representing “Domain Specific Systems.” The combination of Software Engineering journals with Practitioner-oriented journals at the next level (forming “Practical Computing Applications”) reflects the relevance of business issues in Software Engineering. The presence of Software Engineering cluster as a constituent of Computer Science cluster reflects the close relationship between them.

The “Socio-technical group” contains journals related to IS. It is divided into two clusters, namely, “Positivist IS research” and “Interpretivist IS research.” While the positivist cluster consists of North-American journals that mainly conduct empirical research using quantitative methodologies (such as: lab/field experiments, surveys), the interpretivist cluster contains European journals that perform conceptual and non-empirical research using qualitative methodologies (such as: casestudy, action research).3,5 The positivist cluster is divided into two sub-clusters, namely, “Generic focus journals” and “Specific focus journals.” While the former consist of journals such as MISQ, ISR, and JMIS, that provide an integrated view of the IS discipline, the latter consists of journals having distinctive orientation. Specific focus cluster is further divided into three sub-clusters. The “Emerging Technologies” cluster consists of journals like IJEC, JOCEC that publish research related to the use of technology in e-commerce. Journals like DSS, ISF in the same cluster focus on the use of technology for supporting the decision process, and other promising technologies. The “IS Usage and Management” cluster (WIR, ISM, and IM) publishes research concerning IS usage and resource management, and focus on managing activities that collect, store and disseminate information. Journals such as INFSOCITY and IJIM in the “Social Informatics” cluster focus on the use of information technology in such a manner that considers its interaction with institutional and cultural contexts. The placement of “Human-Computer Interaction” journals in the “Technical group” shows the technology focus of this group rather than the social focus. It reflects that few IS researchers currently publish in these journals.

Figure 1 shows the relative ranking of journals as knowledge source at the domain and discipline levels along-with their overall rank in the entire network. For example, in the IS group, MISQ is the leading journal followed by ISR, JMIS, IM, and DSS. IJEC appears in the sixth position, indicating a relatively new journal achieving high rank within a short period of time. These results place them as leading journals in the IS field.

While the journals’ role as a source is recognized, their role as synthesizer is relatively unnoticed by the researchers. Their mutual dependence to uncover important phenomena by conducting research spanning multiple boundaries can bring “unity in diversity” of the discipline. Research has also hinted on the criticality of studying across domains in increasing the discipline’s knowledge base. Recognition of the importance of synthesizers can aid to increase the collaboration among journals beyond their respective domains and help in enhancing knowledge capital7 of the field. This will facilitate interaction in the network and helps it to operate more efficiently.7 An increase in appreciation of their role will augment the number of high quality research published in such journals. Because high synthesizers are involved in a denser citation traffic with diverse journals, the evolutionary process of an increase in a journal’s role as a synthesizer is likely to increase its influence as a source, which is also a measure of the performance of the journals.6

Table 1 provides the ranking of journals as a synthesizer in the network. It shows IEEECOMP to hold top rank, followed by IEEETC, JACM, AI, and CACM in 2nd to 5th positions respectively. As shown in Figure 2, IEEECOMP act as a bridge between CS and Software-Engineering journals and is indirectly connected to IS journals via CACM. Its removal will break the communication flow between these groups. Similarly, CACM facilitates the communication between IS and CS journals while IEEESMC act as a bridge between Computer Interactive Systems and Computational Intelligence.

While journal’s high influence as a source is important, its high influence as a synthesizer makes them together sufficient as a set for the journal to attain leadership position in the network.2 A matrix was developed to classify journals into 6 categories, namely Central Providers, Providing Facilitators, Pivots, Central Facilitators, Facilitators and Affiliates (Figure 3). This is based on the journal’s level as a source (above average or below average8) and as a synthesizer (contributing to disorder, contributing to order, or little effect2) in the network. Pivots are journals having high influence as both source and synthesizer and their cross-functional influence helps in channeling information from one part to another in the network. These journals both transmit and receive citations and occupy a position of leadership in the network. For example, CACM acts as a Pivot because of its influential role as a knowledge source and as a bridge between IS and CS journals. Providing Facilitators rank high as knowledge source and their modest role as a synthesizer helps to maintain order in the system. Their reciprocal ties with other journals develop and extend the sub-clusters in the network. For example, MISQ, ISR, JMIS are highly influential IS journals interconnected with one another. These are part of a big cluster but have few direct links with non-IS computing journals. Central Providers like ACMTIS and ACS rank high as knowledge sources but low as synthesizers. Their involvement in one-way flow of knowledge reflects lack of interdependence and collaboration among journals in the field.

Facilitators and Central Facilitators (for example, JALG, ACMTOSEM, JCSS, AIMAG, ACMTCS and BIT) do not rank high as a source but have significant influence as a synthesizer. These are part of cohesive cliques within the network and contribute towards maintaining and achieving order in the network. Their direct and indirect ties help to spread knowledge throughout the community. As these journals begin to emerge, they will receive more attention from researchers and authors, and may therefore be considered as rising stars of the field. Owing to paucity of acceptance in high source journals due to the field’s advancement and an increase in quality research seeking publication, facilitators should make good targets for publication. Further, their position in the network enables them to offer an opportunity to publish diverse articles and provide wide range of visibility to these articles. Finally, journals in the Affiliates category rank low as sources and synthesizers in the network. Their reciprocal links with fewer journals makes them part of detached small clusters in the network. For example, European interpretive journals like JIT and JSIS lack significant mutual interaction with other journals in the network. While positivist journals have a larger presence in the network compared to interpretive journals, both are required for the field’s continued prosperity. It is therefore imperative for them to understand each others’ role and get involved in diverse research outside their realm, thereby getting connected to one another.5

These findings are also relevant to the editors of journals. If the journal is high as a source, it can move towards the leadership position by pursuing eclectic research. Similarly, affiliates may broaden their focus and build new connections to enhance their reputation as both synthesizer and source. It suggests that an increase in prominence as a synthesizer is a viable road to achieve the leadership position (High Source-High Synthesizer) in the network. For this, the editors may plan to reorient or expand the journal’s focus to attract researchers from new areas, thereby enhancing its interaction with other journals. They may pursue joint publication strategies and publish special issues on cross-functional topics.12 An increase in frequency and number of articles can aid to publish wide range of research topics and increase their influence. For example, Pivots like CACM, IEEECOMP, and IEEETSE are found to publish a wide variety of articles every month.

Back to Top

Conclusion

This study presents the results of interconnection and relative standing of 50 computing journals. We analyzed the journals’ citation patterns to assess their influence as a source, storer and synthesizer in the knowledge network of computing journals. The study’s findings will motivate the researchers to appreciate the importance of journals as synthesizers, a role generally unnoticed by them. The results are also valuable to the editors as it will help them decide on the nature of research published in the journals. The findings also strengthen the identity of IS as a homogenous independent group consisting of tightly coupled journals forming core groups which are loosely coupled with other groups in IS.

The results presented here should be seen in light of certain limitations. Since the data for the study came from ISI, it may have prevented the inclusion of some journals (for example, CAIS and JAIS) that are not cataloged in ISI. However, this study was based on 50 journals over a period of five years and used a considerably larger data set than the ones in past studies. This research can be extended by assessing the change in different roles of journals using two or more time periods. Such an analysis will provide insights about the dynamism and progress of the field.

Back to Top

Back to Top

Back to Top

Back to Top

Figures

F1 Figure 1. Cohesiveness of Journals and Their Relative Influence Rankings as Knowledge Source

F2 Figure 2. Computing Journals Knowledge Network

F3 Figure 3. Source – Synthesizer Matrix

Back to Top

Tables

T1 Table 1. Role and Relative Influence of Journals in Knowledge Network

Back to top

    1. Barnes, S.J. Assessing the value of IS journals. Comm. ACM 48, 1, (Jan. 2005), 110–112.

    2. Cooper, R.B., Blair, D. and Pao, M. Communicating MIS research: A citation study of journal influence. Information Processing & Management 29, 1, (1993), 113–127.

    3. Evaristo, J.R. and Karahanna, E. Is North American IS research different from European IS research? ACM SIGMIS Database 28, 3, (1997), 32–43.

    4. Katerattanakul, P., Han, B. and Hong, S.G. Objective quality ranking of computing journals. Comm. ACM 46, 10, (Oct. 2003), 111–114.

    5. Lowry, P.B., Romans, D. and Curtis, A. Global journal prestige and supporting disciplines: A scientometric study of Information Systems journals. Journal of the Association for Information Systems 5, 2, (2004), 29–77.

    6. Nerur, S., Sikora, R., Mangalaraj, G. and Balijepally, V. Assessing the relative influence of journals in a citation network. Comm. ACM 48, 11, (Nov. 2005), 71–74.

    7. Oh, W., Choi, J.N. and Kim, K. Coauthorship dynamics and knowledge capital: The patterns of cross-disciplinary collaboration in Information Systems research. Journal of Management Information Systems 22, 3, (2006), 265–292.

    8. Pieters, R. and Baumgartner, H. Who talks to whom? Intra- and interdisciplinary communication of economics journals. Journal of Economic Literature 40, 2 (2002), 483–509.

    9. Rainer, R.K. and Miller, M.D. Examining differences across journal rankings. Comm. ACM 48, 2, (Feb. 2005), 91–94.

    10. Shackelford, R., Cross, J.H., Davies, G. and Impagliazzo, J. Computing Curricula 2005: The overview report, The Joint Task Force for Computing Curricula 2005, 2005.

    11. Vermunt, J. LEM: A general program for the analysis of categorical data, Tilburg University, Netherlands, 1997.

    12. Wade, M., Biehl, M. and Kim, H. Information Systems is not a reference discipline (and what we can do about It). Journal of the Association for Information Systems 7, 5, (2006), 247–269.

    DOI: http://doi.acm.org/10.1145/1592761.1592791

Join the Discussion (0)

Become a Member or Sign In to Post a Comment

The Latest from CACM

Shape the Future of Computing

ACM encourages its members to take a direct hand in shaping the future of the association. There are more ways than ever to get involved.

Get Involved

Communications of the ACM (CACM) is now a fully Open Access publication.

By opening CACM to the world, we hope to increase engagement among the broader computer science community and encourage non-members to discover the rich resources ACM has to offer.

Learn More