About me
Papers
Courses & Divulgation
Maps
aaa         a
Maps of the academic web in the European Higher Education Area: an exploration of visual web indicators

Jose Luis Ortega*, Isidro Aguillo*, Viv Cothey**, Andrea Scharnhorst***

*Internet Lab, CINDOC-CSIC, Joaquín Costa, 22. 28002 Madrid. Spain
{jortega; isidro}@cindoc.csic.es

** School of Computing and Information Technology, University of Wolverhampton, Lichfield Street, Wolverhampton, United Kingdom, WV1 1SB viv.cothey@wlv.ac.uk

*** Virtual Knowledge Studio, Royal Netherlands Academy of Arts and Sciences, Cruquiusweg 31, 1019 AT Amsterdam, The Netherlands, andrea.scharnhorst@vks.knaw.nl

Published in: Scientometrics,74(2): 295-308

    Abstract

     This paper shows maps of the web presence of the European Higher Education Area (EHEA) on the level of universities using hyperlinks and analyses the topology of the European academic network. Its purpose is to combine methods from Social Network Analysis (SNA) and cybermetric techniques in order to ask for tendencies of integration of the European universities visible in their web presence and the role of different universities in the process of the emergence of an European Research Area. We find as a main result that the European network is set up by the aggregation of well-defined national networks, whereby the German and British networks are dominant. The national networks are connected to each other through outstanding national universities in each country.

     Introduction

     Visualization of Information (VI) (TUFTE, 1997; CHEN, 2003) is a technique that it aims to show conceptual entities and their relationships through visual metaphors that allows us to interpret and extract conclusions about a certain complex phenomena. Inside of VI, Maps of Science (SMALL, 2003; SMALL & GRIFFITH, 1974; WHITE & GRIFFITH, 1981; McCAIN, 1990) are a model of the utility that show(s) the scientific relationships among authors or academic institutions through the citations, co-authorship, or co-word analysis. Although NOYONS (1999) defines Maps of Science as "landscapes of scientific research fields created by quantitative analysis of bibliographic data", recently web data have been used as additional source of information on scientific networks. LARSON (1996) was the first to map the out- and in-link relationships of several Earth Science web pages using co-link analysis and displaying in a Multidimensional Scaling (MDS) graph. POLANCO et al. (2001) also mapped and clustered 791 European universities web sites using co-link analysis. HEIMERIKS, HORLESBERGER, and VAN DEN BESSELAAR (2003) and HEIMERIKS (2005) more recently mapped 220 EU universities at the level of departments, universities and countries find cultural and linguistic pattern in their relationships. VAUGHAN and YOU (2005) and VAUGHAN (2006) introduced co-link maps as a technique to study the business relationship between companies and to know the presence of companies in concrete markets.
     The World Wide Web is a complex network that connects web pages and sites through hypertextual links creating a large and dense network of nodes (SCHARNHORST, 2003). Network Analysis is both a suitable way to present graphically the link relationship in the web and a technique to analyze and understand the web structure and topology. Recently, the web has been analyzed as a complex network from point of view of statistical physics (BARABASI, ALBERT & JEONG, 2000; ALBERT JEONG, & BARABASI, 1999; BRODER et al., 2000). ALBERT, JEONG and BARABASI (1999) estimated the diameter of the Web, i.e. number of links to cover whole web, to be 19 nodes. These same authors (BARABASI et al., 2000) discovered that the Web showed scale-free networks properties because just a few nodes attract a huge amount of links and the remaining majority only attracts a few of them. Meanwhile, the analysis of web graphs in terms of scale-free or small world networks has been incorporated into information science (BJÖRNEBORN, 2001, 2003; KATZ & COTHEY, 2006; THELWALL & WILKINSON, 2003). Web graphs based on hyperlinks are only one example of such studies. Different authors have developed thematic maps about several web objects (DODGE, 2004) with the intention of making the distribution of users, flow, servers, etc., along the world or in a certain visible region. In 1992 PATERSON and COX (1992) mapped the exponential growing of the internet traffic in the US detecting the main edges of information activity. In 1993 Brian Reid (DODGE & KITCHIN, 2001) mapped the flow of USENET network, and YOOK, JEONG and BARABASI (2001) made a map about the distribution of population against the number of routers connected to Internet

     Objectives

     In this paper we present first attempts to map the web presence of the European Higher Education Area (EHEA) on the level of countries and universities. The aim is to get insights how structured the European academic space in the web is. In particular, we ask which agglomerative aggregation of universities across European countries can be seen on the web. Do we get a random network? Which role plays geographic neighborhood? Or will we observe a superposition of national networks? We want to see the relationships among the universities in and between European countries in term of their hyperlink structures (including link as well as co-link structures). Through applying tools from Network Analysis to cybermetric data we intent to identify the main agents (universities or countries) and their role inside of the European academic web environment.

     Methods

    535 universities of the 14 European countries (EU except Luxembourg) in 2004 were selected from Webometrics Ranking of World Universities (www.webometrics.info ). This site ranks 3,000 universities according two main criteria: size (number of pages and rich files) and visibility (number of incoming links). This set of European universities were mapped according to the link relationships among them. Two different set of web data were used: search engine data and crawler data. The search engine was used to retrieve the link relationships between web sites and the crawler was used to extract the pages hosted in each web site. The combination of these different tools allows us to obtain suitable data in a fast and exhaustively way. For instance, the link extraction is a complex task to done with a crawler, whereas a search engine provide this information more easily. In any case, we think that in macro level studies a possible heterogeneity of the data might become less important, because fluctuations in the data (due to temporarily instability) and errors in the measurement will be leveled out on a high level of aggregation and are less important if the size of the system under study increases. The search engine data were obtained from Yahoo! Search with the query:

+site:{university domainA} +linkdomain:{university domainB}

     On the other hand, the crawler data were extracted with the software Blinker (COTHEY, 2004; COTHEY, 2005) to find the number of pages and domains of the 535 universities. Both set of data were obtained in August of 2005. These data were analyzed with the software Ucinet 6.109 and the application NetDraw 2.28 was used to built the network graphs.
     The resulting graphs were processed in two different ways. A graph was built through the link matrix retrieved from the search engine to illustrate the topology of the network and its connectivity degree. This graph was laid out with the Spring embedding algorithm through NetDraw (KAMADA & KAWAI, 1989). This layout shows the nodes and the arcs minimizing the cross points and the overlap of nodes to obtain an excellent network visualization and thus to detect the main characteristics of the network. Nevertheless, it is more appropriate to small and medium size networks because it is quite slow when it comes to configure the network (NOOY, MRVAR & BATAGELJ, 2005). Finally, multiple arcs with fewer than 50 links were removed to reveal a clearer graph of a network of 527 nodes.
     On the other hand, a co-link map (LEYDESDORFF & VAUGHAN, 2006) was constructed to detect the link pattern among the universities web sites and how these are grouped according to the co-link degree. The co-link degree between two web sites is the frequency which two web sites are linked by a third web site. It is a measure which points to a possible substantial relationship between the two co-linked websites. A asymmetrical matrix of links between university websites was built with the search engine data. Then it was converted to a symmetrical matrix applying the Salton's cosine measure (SALTON, WONG & YANG, 1975; SALTON, 1971). Next, distance coordinates were calculated from this symmetrical matrix through applying Multidimensional Scaling techniques (MDS) to locate the university web sites with regard of their co-link degree on a two-dimensional plane. Finally, the coordinates of universities according to the MDS of their co-link structure were plotted together with the network graph.
     Several social network measures were used to analyze the resulting graphs. Since the web is a graph of links that connect several web sites, the SNA techniques allow us to analyzed the structural and topological features of the European academic web network. Along this study we will explain the utility and the calculation of the indexes used.

   Results


Figure 1. European Universities Network of links (504 nodes; 8028 ties)

     Figure 1 shows the network graph of the 527 EU universities with an average distance among each other reachable of 1.52 nodes and a diameter of 3, hence this web graph is a dense and compact network. In Figure 1 the different countries are presented by different colors (or grey scale). Because of the density of the network we would like to point the reader additionally to the colored version of the graph on the web. As can be seen from the legend, in the different countries the university websites have been also classified into five categories according to their content. These five categories were created ad hoc to express the main academic subject area. Thus, Technology includes all technological schools and universities (fachhochschulen, universities of applied sciences, etc.), the Social Sciences group includes mainly Business Schools, Biomedicine set up the veterinarian and medical universities and the Humanities contains arts schools, human sciences universities and library schools. The rest of universities without a specific oriented activity were grouped under the General set. We will discuss the influence of the content of the website to its classification scheme at a later point in the paper. From a topological point of view, the graph shows the properties of a scale-free network (ALBERT et al., 1999), which means a few nodes attract a huge amount of links and the rest of nodes attract only a few of them.

University Domain nInDegree
University of Leeds leeds.ac.uk 0.839
University of Cambridge cam.ac.uk 0.808
University of Oxford ox.ac.uk 0.628
Free University of Berlin fu-berlin.de 0.575
University of Helsinki helsinki.fi 0.495
University of Edinburgh ed.ac.uk 0.479
University of Regensburg uni-regensburg.de 0.471
University of Karlsruhe uni-karlsruhe.de 0.466
University of Southampton soton.ac.uk 0.441
University College London ucl.ac.uk 0.416
Table 1. Ten universities with highest nInDegree

     Table 1 shows that universities with a high nInDegree. This index measures the normalized degree of in-coming links. Thus nInDegree is the percentage of in-coming links to a node compared with all in-coming links over the whole nodes in the network. This indicator allows to detect the universities that attract a great proportion of links. The most outstanding universities are the University of Leeds (0.839), the University of Cambridge (0.808) and the University of Oxford (0.628), where the first three are British universities and between the first ten there are six ones. This allows us to state that the British network receive more links than other countries, perhaps, due to linguistic reasons (THELWALL, TANG & PRICE, 2003). Another possible explanation is the relative large size of the British network which therefore offers a large number of target pages for link (KATZ & COTHEY, 2006).

University Domain nOutDegree
Humboldt University of Berlin hu-berlin.de 0.895
University of Helsinki helsinki.fi 0.638
University of Edinburgh ed.ac.uk 0.560
Linköping University liu.se 0.538
Technical University of Berlin tu-berlin.de 0.533
Rhine-Westphalia Technical University of Aachen rwth-aachen.de 0.502
Free University of Berlin fu-berlin.de 0.457
Jussieu Campus jussieu.fr 0.452
University of Alicante ua.es 0.450
Royal Institute of Technology kth.se 0.440
Table 2. Ten Universities with highest nOutDegree

     Table 2 shows the universities with a high nOutDegree. Like the index before, this one measures the normalized degree of out-coming links. The nOutDegree is the percentage of out-coming links from a node over compared with all out-links over the whole network. The most highlighted universities are the Humboldt University of Berlin (0.895), the University of Helsinki (0.638) and the University of Edinburgh (0.56). In this table the presence of German universities is higher than other countries, being four in the first ten ranks. Thus the German network is characterized by high proportion of out-going links, although many of these go to other German university web sites.
     One can also see that the universities are grouped by country. In particular, we can distinguish the German sub-network in red, the British in blue and the French in yellow. Surprisingly, France shows a barely connected network and with low volume of published pages. This might be caused by the existence of a lot of small size academic institutions (écoles) which it have not much presence in the web. Additionally, it is visible that there are small countries which universities do not form an homogeneous sub-network but are spread out over other large sub-networks such as in the case of Austria where Austrian universities are connected with universities in Germany or in the case of Ireland where Irish universities connect to British universities. It important to note that the Scandinavian countries constitute a compact and close sub-network. This Scandinavian network has also been detected in scientometric environments (BONITZ & SCHARNHORST, 2000; GLÄNZEL, 2001; WAGNER & LEYDESDORFF, 2005b).

Cluster Nodes InnerLinks OuterLinks p_in p_out
Portugal 11 37 57 0.672727 0.010200
Germany, Austria 117 1704 812 0.251105 0.017264
UK, Ireland, Netherlands, Belgium 136 1001 1071 0.109041 0.020561
Greece 11 29 31 0.527273 0.005548
Italy 47 240 201 0.222017 0.009061
Spain 51 432 166 0.338824 0.006955
Sweden, Denmark, Finland 63 381 493 0.195084 0.017161
France 81 265 274 0.081790 0.007723
Table 3. p-cliques of the EU Universities network

     Table 3 shows the p-cliques found in the European network of Universities web sites. A p-clique is a sub-graph with a high connectivity which the nodes have whole the possible links among them. This technique allows us to cluster nodes according to the connectivity degree. One can see the Scandinavian cluster set up by Sweden, Denmark and Finland. The largest cluster is shaped by UK, Ireland, Netherlands and Belgium. A possible explanation might be the use of English as dominant language (THELWALL, TANG & PRICE, 2003) but also collaboration structures (WAGNER & LEYDESDORFF, 2005a) and similar disciplinary profiles might be a possible explanation (BONITZ et al, 1993). Similar is the case of Germany and Austria due to in this case the use of the German language. This analysis confirms the visual appreciation which the EU academic network is made up of the aggregation of several regional and national networks.
     However, if the EU university network comprises national networks what is the main core of the network, the base on which the network rests? Through using the concept of the k-cores (SEIDMAN, 1983) we want to answer this question. A k-core is a maximal subnetwork in which each vertex has at least degree k within subnetwork (NOOY, MRVAR & BATAGELJ, 2005). The highest core found in our data is a 38-core which is composed by solely 50 German universities and one Austrian, so we can conclude that the vertex of the EU university network is rested on the German network.
     On the other hand, Figure 1 shows that the sites with a great number of pages are located in the center. Further, Figure 1 is based on the Kamada-Kawai algorithm which locates in the center of the map the nodes which are highest linked. The nodes of big size are also the nodes with highest centrality degree. In other words, the nodes centrally located attract more links than the rest ones. It has been shown earlier that a correlation exist between the number of pages on a web site and the in-links it attracts from other web sites (ADAMIC, 2002; THELWALL, 2004; KATZ & COTHEY, 2006). Thus, the size of a web site is key factor to achieve a high centrality degree in the web network.
     Because the EU university network is made up of the combination of national sub-networks, which we have identified by the p-cliques analysis, it is of interest to discover which university web sites act as hub or gatekeeper between the national networks and the European one. The Betweenness index measures the intermediation degree of a node to keep the network connected, that is to say, the capacity of one node to connect only those nodes that are not directly connected to each other. Thus, the Betweenness will allows to show the main hubs or gates that connect one network with other. The normalized Betweenness is the betweenness value of a node averaged over the whole nodes in the network.

Country University Domain Betweenness nBetweenness
UK University of Edinburgh ed.ac.uk 1,645,818 0.594
FI University of Helsinki helsinki.fi 1,313,489 0.474
AT University of Vienna univie.ac.at 1,295,428 0.467
NL University of Amsterdam uva.nl 1,231,140 0.444
SE Linköping University liu.se 1,126,263 0.406
BE Catholic University of Leuven kuleuven.ac.be 1,124,354 0.406
DE Free University of Berlin fu-berlin.de 1,093,416 0.394
IT University of Bologna unibo.it 962,142 0.347
ES University of Barcelona ub.es 739,074 0.267
GR Aristotle University of Thessaloniki auth.gr 644,534 0.233
IE University of Dublin, Trinity College tcd.ie 627,630 0.226
FR Jussieu Campus jussieu.fr 602,567 0.217
DK University of Copenhagen ku.dk 591,611 0.213
PT University of Coimbra uc.pt 540,819 0.195
Table 4. Betweenness scores of the main universities of each country

     Table 4 shows the highest Betweenness scores of only the top universities and normalized Betweenness scores of the main universities for each country, although there are countries with have two or three high scoring universities in Betweenness. For instance, in the UK the University of Edinburgh is followed by the University of Cambridge (0.554) and University of Oxford (0.536). Almost all the universities in Table 4 are know as outstanding academic institutions in its countries and as we see now they also act as key nodes for the academic web of its countries. An exception is the Linköping University in Sweden which has a better web presence in its country than prestigious universities such as University of Uppsala (0.278) or University of Stockholm (0.224). The low position of the Jussieu Campus on the ranking list in Table 4 is related to the relative extent of disconnection of the French network with the rest of the European networks (AGUILLO, ORTEGA & GRANADINO, 2006). The low link degree observed in the French network affect its international visibility, causing a low betweenness degree of the French universities.


Figure 2. European Universities Co-link map (f = 0,086)

     Figure 2 show the map of the EU universities according to the co-linkage degree among its web sites. The obtained stress in the MDS (f = 0,086) is quite low therefore the resulting model is acceptable for the analysis. Just like the Figure 1 the map shows defined and compact national clusters such as Germany in red, UK in blue and Spain in orange. One also can see how the small countries are connected with large countries such as Netherlands and Ireland with respect to the UK. However, one can see particular characteristics. Spain is located far away from the rest of the national clusters due to a low co-link degree of the Spanish universities with regard to the other country's universities and a high co-link degree between themselves. This is similar to Austria but to a lesser extent. On the contrary, the French network has less density because its universities have low a co-link degree. In general, the low link degree and the small size of the academic institutions in France causes a weakly connected network among the French institutions which is spread out widely in the whole network of European universities.
    Most universities are multidisciplinary therefore the thematic relationships according to the university's subject matter are not distinctive. However it is noticed that the technological universities tend to be related at the national level such as within Spain and Finland whereas less technological and more social science based universities such as the business schools tend to have more international connections.

National Visibility
Country University Domain Inlinks Outlinks
DE Free University of Berlin fu-berlin.de 27,249 19,743
FR University of Paris-Sorbonne, Paris IV paris4.sorbonne.fr 25,300 24,413
UK University of Cambridge cam.ac.uk 23,391 16,823
SE Royal Institute of Technology kth.se 15,999 17,923
FI University of Helsinki helsinki.fi 14,798 17,443
AT Innsbruck Medical University uibk.ac.at 11,372 2,161
ES Complutense University of Madrid ucm.es 9,781 9,061
NL Free University of Amsterdam vu.nl 9,482 9,805
IT University of Bologna unibo.it 8,532 5,792
DK Royal School of Library and Information Science db.dk 7,058 7,168
GR University of Macedonia uom.gr 6,970 5,819
BE University of Liège ulg.ac.be 4,616 1,927
PT Technical University of Lisboa utl.pt 3,352 2,655
IE Dublin City University dcu.ie 2,252 1,900
Table 5. National visibility of the main universities by country (total number of national in- and out-links).
 
International Visibility
Country University Domain Inlinks Outlinks
UK University of Leeds leeds.ac.uk 30,512 3,515
AT University of Vienna univie.ac.at 14,015 14,060
NL Utrecht University uu.nl 11,688 15,007
DK Technical University of Denmark dtu.dk 11,172 4,700
FI University of Helsinki helsinki.fi 10,847 15,513
DE University of Cologne uni-koeln.de 9,426 5,312
SE Uppsala University uu.se 8,452 11,443
BE Catholic University of Leuven kuleuven.ac.be 8,339 10,876
IT University of Bologna unibo.it 8,339 9,358
GR National Technical University of Athens ntua.gr 6,240 3,594
FR Jussieu Campus jussieu.fr 5,759 6,341
IE University of Dublin, Trinity College tcd.ie 5,410 4,477
ES Polytechnic University of Madrid upm.es 4,076 5,807
PT University of Coimbra uc.pt 3,315 4,105
Table 6. International visibility of the main universities by country (total number of international in- and out-links).

     Tables 5 and 6 show the visibility of each university according to the out- and in-links from or to the national universities or international ones. The tables show differences between the national and the international visibility. For instance the British university with the highest number of links inside UK is the University of Cambridge. But, the university with the highest number of links from abroad is the University of Leeds. However there are universities that have great visibility both inside and outside their country such as the University of Helsinki in Finland or the Catholic University of Leuven in Belgium. It is significant to notice the national and international visibility differ between the countries. The German, French and Spanish universities have high national visibility even though their international visibility is quite low.

     Discussion and Conclusions

     The general picture of the European academic network sector allows us to see that it is built up of multiple national networks. That is, there is not a single unique network but a network of national networks that are aggregated one with another. In this aggregation process the first component is the German network, followed by the addition of other national networks to build the complete EU network. But one has to have in mind that the web graph is a directed network. So the base position of Germany is not because it is a very linked country but because it is the country with most outlinks abroad. Thus, Germany, which universities have the highest outdegree, builds the EU network by means of outlinks that connect other European universities and then other national networks. At the same time British universities are the most linked. They also build the EU network by means of attracting links from other countries due to the amount of contents published in English. This can probably be explained due to amount of contents publish in English language. THELWALL, TANG & PRICE (2003) already showed the importance of the linguistic pattern to achieve links from outer countries in the European university web space. This dual role of UK and Germany explains the high cohesion that results from the small network diameter and distance between nodes. This allows us to argue that the EHEA is quite united in the web space, although these results need to be compared with other regions and the relationship of these regions with the European countries such as Spain with South America or UK with the Commonwealth countries.
     One can also conclude that the EU university network is made up by the aggregation of national networks. Thus a university is first linked to others within its country and then this national network is connected to other national networks. However, there are "pan-European" universities, detected through the Betweenness index, that link and are linked mainly to universities abroad. These universities are the hubs or gateways that connect a national network to the whole European network.
     The SNA techniques and measures make it possible to show the characteristics of the web presence of the EU academic network. The centrality degree measures have indicated those universities are more outstanding regarding to the links that they attract and make; we used k-cores to detect where the set of most interconnected web sites is and finally applied p-cliques and discovered that the EU university network is made up of national networks. In particular the betweeness index has been used to detected the intermediate universities between the national networks and the EU network. So one can to conclude that SNA techniques are a suitable tools to analyse the topology of the web and its relationships.
     The co-link map showed a different kind of relationship: the co-occurrence of incoming links. This allows to show the particular relationship between countries and universities. This shows that although the European academic network is highly connected there is particular countries with an open networks connected with other networks such as Netherlands, Belgium and the Scandinavian countries, and other countries such as Spain or Austria that are few co-linked internationally or France which network is barely united. Nevertheless, we have been not able to detect subject relationship because almost all the universities are multidisciplinary although some technical and social sciences universities have showed certain subject co-link behavior.
     For future research it would be interesting to look into some additional data in particular to understand the role of certain universities. As has been debated in the literature the meaning of hyperlinks can be very different reaching from administrative to content related reasons (THELWALL, 2006). Therefore, we avoid to discuss our results in terms of importance or collaboration or any other content based interpretation. However, further analysis we have done with the data set elsewhere (ORTEGA, 2007) show that there are high correlations between web data based on hyperlinks and other socio-economic or bibliometric data. That might be an indication that web graph based on units of analysis of a high level of aggregation as universities or countries are quite suitable as indicators for structures in the academic European system. At least, as our analysis show, for Europe we observe signs for cohesion and integration on the background of still dominant national science systems and an interesting interplay between "big players" in the field and small countries. Therefore, it seems reasonable to repeat such kind of exercise longitudinal to make trends visible.

aaaaaReference List

ADAMIC, L. A., HUBERMAN, B. A. (2002), Zipf's law and the Internet, Glottometrics, 3: 143-150 http://www.hpl.hp.com/research/idl/papers/ranking/adamicglottometrics.pdf
AGUILLO, I.F., ORTEGA, J.L., GRANADINO, B. (2006), Webometric evidence of an academic digital divide in France, The 9th International Conference on Science and Technology Indicators, University of Leuven, Leuven
ALBERT, R., JEONG, H., BARABASI, A.L. (1999), Internet - Diameter of the World-Wide Web, Nature, 401(6749): 130-131.

BARABASI, A.L., ALBERT, R., JEONG, H. (2000), Scale-Free Characteristics of Random Networks: the Topology of the World-Wide Web, Physica a, 281(1-4): 69-77.
BONITZ, M., BRUCKNER, E., SCHARNHORST, A. (1993), The Science Strategy Index, Scientometrics, 26(1): 33-50.
BONITZ, M., SCHARNHORST, A. (2000), National Science Systems and the Matthew Effect for Countries. In: OHLY, P., RAHMSTORF, G., SIGEL, A.(Eds.), Globalisierung und Wissensorganisation, Ergon Verlag, Würzburg, pp. 173-182.
BJÖRNEBORN, L. (2001), Small-world linkage and co-linkage, Proceedings of the 12th ACM Conference on Hypertext and Hypermedia, New York: ACM Press http://www.db.dk/lb/ht01_bjorneborn.pdf
BJÖRNEBORN, L. (2003), Small-World Link Structures across an Academic Web Space: A Library and Information Science Approach, Copenhagen: Royal School of Library and Information Science http://www.db.dk/lb/phd/phd-thesis.pdf
BRODER, A., KUMAR, R., MAGHOUL, F., RAGHAVAN, P., RAJAGOPALAN, S., STATA, R., TOMKINS, A., WIENER, J. (2000), Graph Structure in the Web, Computer Networks-the International Journal of Computer and Telecommunications Networking, 33(1-6): 309-320. http://www9.org/w9cdrom/160/160.html
CHEN, C. (2003), Mapping Scientific Frontiers: The Quest for Knowledge Visualization, Springer-Verlag, London.
COTHEY, V. (2004), Web-Crawling Reliability, Journal of the American Society for Information Science and Technology, 55(14): 1228-1238.
COTHEY, V. (2005), Some preliminary results from a link-crawl of the European Union Research Area Web. In: P. INGWERSEN, B. LARSEN (Eds), Proceeding of the 10th International Conference of the International Society for Scientometrics and Informetrics, Karolinska University Press, Stockholm.
DODGE, M., KITCHIN, R. (2001), Atlas of Cyberspace, Addison-Wesley, London.
DODGE, M. (2004), Cyber-Geography Research, University College London, London,
http://www.cybergeography.org/
GLÄNZEL, W., (2001), National Characteristics in International Scientific Co-authorship, Scientometrics, 51 (1): 69-115.
HEIMERIKS, G. (2005), Knowledge Production and Communication in the Information Society: Mapping communications in heterogeneous research networks, University of Amsterdam, Amsterdam.
HEIMERIKS, G., HORLESBERGER, M., VAN DEN BESSELAAR, P. (2003), Mapping Communication and Collaboration in Heterogeneous Research Networks, Scientometrics, 58(2): 391-413.
KAMADA, T., KAWAI, S. (1989), An algorithm for drawing general undirected graphs, Information Processing Letters, (31)
KATZ, J. S., COTHEY, V. (2006), Web indicators for complex innovation systems, Research Evaluation, 15(2): 85-95

LARSON, R. (1996), Bibliometrics of the World Wide Web: An exploratory analysis of the intellectual structure of Cyberspace, Proceedings of ASIS96, 71-78. http://sherlock.berkeley.edu/asis96/asis96.html
LEYDESDORFF, L., VAUGHAN, L.W. (2006), Co-occurrence Matrices and Their Applications in Information Science: Extending ACA to the Web Environment, Journal of the American Society for Information Science and Technology, 57(12): 1616-1628
McCAIN, K. (1990), Mapping Authors in Intellectual Space: A Technical Overview, Journal of the American Society for Information Science and Technology, 41(6): 433-443.
NOOY, W.d., MRVAR, A., BATAGELJ, V. (2005), Exploratory Social Network Analysis with Pajek, Cambridge University Press, Cambridge, UK.
NOYONS, E.C.M. (1999), Bibliometrics mapping as a science policy and research management tool, Leiden University, Leiden.
ORTEGA, J. L. (2007) Visualización de la web universitaria europea: Análisis cuantitativo de enlaces a través de técnicas cibermétricas [Visualization of the European academic web space: Quantitative link analysis through cybermetric techniques]. Madrid: Universidad Carlos III de Madrid,
POLANCO, X., BOUDOURIDES, M., BESAGNI, D., ROCHE, I. (2001), Clustering and Mapping European University Web Sites Sample for Displaying Associations and Visualizing Networks, NTTS&ETK 2001 Conference, Hersonissos.
SALTON, G. (1971), The Smart Retrieval System~Experiments. In: G. SALTON (Ed), Automatic Document Processing, Prentice-Hall, New York.
SALTON, G., WONG, A., YANG, C.S. (1975), A Vector Space Model for automatic indexing, Communications of the Association for Computing Machinery, 18(11): 613-620.
SCHARNHORST, A. (2003), Complex Networks and the Web: Insights from Nonlinear Physics, Journal of Computer-Mediated Communication, 8(4)
http://www.ascusc.org/jcmc/vol8/issue4/scharnhorst.html
SEIDMAN, S.B. (1983), Network structure and minimum degree, Social Networks, 5: 269-287.
SMALL, H. (2003), Paradigms, Citations, and Maps of Science: a Personal History, Journal of the American Society for Information Science and Technology, 54(5): 394-399.
SMALL, H., GRIFFITH, B.C. (1974), The structure of the scientific literatures Identifying and graphing specialties, Science Studies, 4(1): 17-40.
THELWALL, M., TANG, R., PRICE, L. (2003), Linguistic Patterns of Academic Web Use in Western Europe, Scientometrics, 56(3): 417-432.
THELWALL, M., WILKINSON, D. (2003), Graph structure in three national academic webs: power laws with anomalies, Journal of the American Society for Information Science and Technology, 54(8): 706-712

THELWALL, M. (2004), Link Analysis: An Information Science Approach, Elsevier Academic Press, Amsterdam et al.
THELWALL, M. (2006), Interpreting social science link analysis research: A theoretical framework, Journal of the American Society for Information Science and Technology, 57(1): 60-68

TUFTE, E. (1997), The Visual Display of Quantitative Information, Graphics Press, Chesire, Connecticut.
VAUGHAN, L. (2006), Visualizing linguistic and cultural differences using Web co-link data, Journal of the American Society for Information Science & Technology, 57(9): 1178-1193.
VAUGHAN, L.W., YOU, J. (2005), Mapping business competitive positions using Web co-link analysis. In: P. INGWERSEN, B. LARSEN (Eds), Proceeding of the 10th International Conference of the International Society for Scientometrics and Informetrics, Karolinska University Press, Stockholm.
WAGNER, C., LEYDESDORFF, L. (2005a), Network Structure, Self-Organization, and the Growth of International Collaboration in Science, Research Policy, 34(10): 1608-1618

WAGNER, C. & LEYDESDORFF, L. (2005b), Mapping Global Science using International Co-authorships: A Comparison of 1990 and 2000, International Journal of Technology and Globalization, 1(2): 185-208 .
WHITE, H.D., GRIFFITH, B.C. (1981), Author Cocitation: A literature measure of intellectual structure, Journal of the American Society for Information Science, 32: 163-171.