The potential in the era of big data comes not from drowning in a sea of data but navigating the most useful ways to derive insight and develop innovative strategies from that data.
Faced with complex problems, limited resources, and an increasingly small world many private and public diplomacy organisations are seeking to increase reach and influence through developing partnerships or unlocking their innovative potential through collaboration. At the same time the development of new technology has spawned new ideas, opportunities, and approaches to engaging with people around the world. Protesters demonstrated the ability to construct dispersed communication networks and coordinate action in the ‘battle in Seattle’, as recorded by John Sullivan. Similar network based approaches to public diplomacy have been identified in a recent article in Foreign Service Journal and at a conceptual level by Brian Hocking. As a result of these shifts, there is now potential to develop innovation in public diplomacy through “Big data”.
As a UN Global Pulse white paper noted big data is “an umbrella term for the explosion in the quantity and diversity of high frequency digital data. These data hold the potential—as yet largely untapped—to allow decision makers to track development progress, improve social protection, and understand where existing policies and programmes require adjustment”. In the context of public diplomacy big data allows organisations to look far beyond the daily tactical data, whether web metrics or shifting numbers of ”friends”, ”followers” and klout scores. Today public diplomacy strategists and practitioners are able to develop innovative strategy using insight from diverse sources of big data.
The potential to use big data for innovation comes with certain challenges. One challenge is to ensure that the people who receive insight from big data have the appropriate skills and authority to act on that data. The second is to recognise the technology has to be used for an appropriate purpose. These challenges were highlighted during field exercises run by the Pentagon through the Force XXI project, and following the sinking of the U.S. expeditionary fleet during of the Millennium Challenge exercise, conducted in the summer of 2002.
If the challenges can be addressed, a big data approach can uncover rich information about the communities with which public diplomats seek to engage and can reveal new perspectives about the world in which public diplomacy operates. Through the resulting insight, organisations can empower public diplomats and support the development of innovative strategies to bridge the ‘last three feet’.
This article discusses a big data approach to public diplomacy, first by highlighting the wider environment and sources of data. Subsequent sections discuss how big data can be applied within public diplomacy and finally how the resulting insight can support innovation.
Framing the big data environment
In 2008, I argued that, as the practice of public diplomacy develops and the barriers to entry become lower, there was an increasing need to consider new approaches, which could be added to the toolbox of existing methodologies. These new approaches include a shift towards an open-source approach to public diplomacy, through which to develop collective action with engaged communities.
2008 was the year Google announced it had identified 1 trillion (as in 1,000,000,000,000) unique URLs on the web at once. It was also the year Facebook closed the gap on MySpace, and metrics providers began naming it as the biggest social network site. Twitter traffic grew by 752 percent and users collectively sent an average 300,000 tweets per day.
Just three years ago these numbers sounded huge, but today they are dwarfed by the contemporary big data era. In 2011, tweets per day had reached 200 million while ITU, the U.N.’s International Telecommunications Union, estimated there were “over five billion mobile users and two billion subscribers to the Internet.” In addition, the rise of services such as Weibo further emphasise the growth of participative communities. Gone are the imagined herds of potential followers with hearts and minds only capable of being won over to your side or that of the enemy. Public diplomacy of the 21st century will rest on bridging the ‘last three feet’ between communities of “participants” and potential collaborators with whom to cooperate and co-create, rather than the hierarchical view of content producer and “target audience.”
In parallel with the growth in digital users, there has been a massive increase in the availability of systems that allow the individual user or small organisation to store data, connect different data sources and visualise the result. Free data storage systems include mongoDB, eXist, mysql, and the Apache Hadoop project which “develops open-source software for reliable, scalable, distributed computing.” These storage systems combined with data visualisation programs and low cost access to flexible server space, have lowered the barriers not just to communication but to the production of analysis and insight.
For example, when researchers analyzed the entire Facebook network of 721 million active users and 69 billion friendship links, the commercial value of the hardware they used was “only in the order of a few thousand dollars.” The size of the data storage in this example (72 GiB of memory and 1 TiB of disk space) is vastly bigger than most public diplomacy organisations will deal with regularly. However, access to the technical means to analyse data creates the potential for insight to support innovation in public diplomacy.
One of the most dynamic demonstrations of this evolution is “Cascade,” produced by the NYTLabs using MongoDB and the electronic sketchbook Processing. Cascade shows who shared a specific NYT story and when. According to the project page, this “allows for precise analysis of the structures which underly sharing activity on the web”.
Other more accessible services include Many Eyes, Yahoo Pipes, Google Fusion Tables, Impure, OpenDX, Processing, 3DEM and Blender amongst many others. These easy to access and often “software as service” options allow anyone with a quick enough internet connection to access data visualisation.
On the back of developing technology, the delivery of more complex data visualization is rapidly becoming a service in itself. For example, the data visualization and mapping team Development Seed is helping Internews and NAI expose patterns of violence against journalists on the ground in Afghanistan. In addition, the humanitarian technology network, Crisis Mappers;
“leverage mobile & web-based applications, participatory maps & crowdsourced event data, aerial & satellite imagery, geospatial platforms, advanced visualization, live simulation, and computational & statistical models to power effective early warning for rapid response to complex humanitarian emergencies”.
Used effectively, tools for making sense of big data have potential to further empower practitioners who operate in the complex architecture of human networks that straddle geographic borders.
A big data approach in public diplomacy
The ‘last three feet’ in the big data era
In complex human networks influence flows in multiple directions and coordinates around numerous hubs or focal points. In response, public diplomacy strategists increasingly identify the potential influence which can result from pursuing genuinely collaborative approaches with these interconnected communities.
Innovative strategies utilizing new technology have at times been framed in tension with the long established physical meeting of people. Such tensions regularly pivot on the often cited “last three feet”: As Edward Murrow put it, “The really crucial link in the international communication chain is the last three feet, which is bridged by personal contact — one person talking to another.” However, the effective use of big data rests on synergies between digital and physical strategies.
From a strategic perspective the bridge across the ‘last three feet’ is built on the type of interaction which participants find meaningful. Some individuals will prefer physical contact while others tend to interact via digital platforms, for example the use of Google Hangouts. Equally there are increasingly examples of digital and social media networks being used to identify participants for “physical world” public diplomacy events. Innovative strategy will frame public diplomacy as a combination of physical and digital interactions based on the preferences of participants and insights from online and offline research.
The success of bridging ‘the last three feet’ will continue to rest largely on the intercultural skills of those charged with the conduct of public diplomacy. However, organisations cannot rely solely on the ability or skills of the individual practitioner. Public diplomacy, at a strategic level, has the opportunity to capture the richness of complex societies and derive insight through which to empower the professionals working at operational and tactical levels. To put this into action, the first step for an organisation is to recognise the value of data and identify the instances in which a particular dataset is valuable.
What value to put on data?
All data are not of equal value. This is particularly the case when engaging with big data. Specific types of data are generated and are relevant at different stages of public diplomacy practice. Using the right data at the right point in the conduct of public diplomacy can allow innovation to flourish. Conversely, a dataset used at the wrong point will choke both existing and innovative practice.
One way to identify the most appropriate point in public diplomacy practice to use a specific big data source is to model activity based on interconnected planes, the “Action Plane” and the “Analysis Plane.” Within the action plane, activity exists at strategic, operational and tactical levels. Subsequently, each level on the activity plane has a counterpart on the analysis plane.
- Tactical actions tend to produce largely raw data – for example, who was invited to an event, who attended and their demographic characteristics. Equally, data from digital interactions can provide tactical data. This includes volume measures of those engaged online through the classic metrics tools delivered by services such as Google analytics or the numerous twitter analytics services.
- Operational level decisions provide a framework for tactical actions. The counterpart level on the analysis plane exists as a location for raw data to be aggregated, whether produced through the tactical actions of a public diplomacy organisation or data-streams available from other sources including social media and open data. At this analytical stage the data can be collated and filtered to identify the most useful data within the vast array that was available. This should inform operational decisions or feed into the process of delivering actionable insight to be used at the strategic level.
- Strategic level decisions provide the purpose and framework within which operational and tactical actions take place. These decisions require insight, whether derived from specifically conducted reactive research, unobtrusive research observing the behaviour of specific communities, open data or data produced during operational activity. It is likely innovative strategy will come from a dynamic synthesis of data from a diverse range of sources. As a result, innovative strategy exists in a symbiotic relationship with the data delivered from tactical and operational actions, and subsequent aggregation and filtering on the analysis plane.
Where can we get data?
As a McKinsey Global Institute report noted in 2011, an “organization that intends to derive value from big data has to adopt a flexible, multidisciplinary approach” which can engage with numerous sources of data. In public diplomacy terms, big data can come from combining a range of different approaches, including reactive and unobtrusive research, along with data from operational activity and open data sources.
- Reactive research, such as that commissioned by the US Broadcasting Board of Governors and the USIA Office of Research, can provide valuable information on reported perception, intention and action. This data can be gathered via phone or face-to-face interview. New technology also allows panels of mobile phone users to provide rapid answers to specific questions via text. Further innovation including smart phone applications such as “Show of Hands” expands the options for combining reported opinion into a big data approach.
- Unobtrusive research is probably less familiar within public diplomacy but with increasing use of social media this approach has growing potential to contribute significantly to the big data approach. In digital and social media many discussions and information sharing networks are open to unobtrusive research. In public diplomacy terms there is opportunity to genuinely listen to the way users express themselves about a specific event or issue.
Sentiment analysis may be a useful guide for easily identifiable concepts or brands, but has significant limitations around complexissues. An alternative combining semantic, discourse and network analysis creates the potential to identify the focal points for specific communities within a wider trending topic or complex issue. This allows the public diplomat to understand the nature of these conversations and engage with communities on their terms. For example, research into the use of twitter during the protests which followed the 2009 Iranian Election distinguish between a range of communities using the tag #IranElection. Other examples include evaluating the response in social media to a presidential visit, as shown in the InterMedia white paper on the impact of #ObamaInBrazil.
- Operational activity provides another opportunity to aggregate tactical data to support operational and strategic decisions. This can be as simple as recording attendance lists in an efficient manner. These are particularly likely to exist for events held in “secure” buildings, for example, embassies, consulates or High Commissions but should be good practice for almost all events. This type of data is also available for other types of public diplomacy. For example, data from exchange programmes can inform future activities at both a tactical level, by identifying individuals who could be engaged in further activity, and at the strategic level, by aggregating the involvement of individuals to identify wider trends or patterns in the levels and types of engagement. In the U.S., for example, much of this data is available through the Interagency Working Group (IAWG) on U.S. Government-Sponsored International Exchanges and Training.
- Open Data has become an increasingly important source which can provide context to bespoke research or specific analysis. These open data sources include that from international organisations such as the World Bank and national data including Kenya, U.S. and U.K. Further datasets are aggregated via initiatives such as datacatalogues.org and opensource.gov provides foreign open source intelligence. These sources of data have the potential to supplement research specifically commissioned to support public diplomacy. The open approach is a growing initiative, and the information available is a valuable source for practitioners and scholars alike.
Building innovative approaches
Big data exists. As Antti Halonen, of the Finnish Institute in London, argued, “We live in a data society … whether this is an inconvenient truth or a nerd’s dream … we must face the situation and make the best of it”. The key question for the practice of public diplomacy is how can we us it? This section identifies two potential areas where a big data approach can be used, first in analysing the operational activity of an organisation and second observing the greater networks of communication between communities around the world.
Dashboards and organisational data
The use of dashboards is a growing area within current public diplomacy practice as they offer easy access to often complex data. However, it is imperative to identify whether the data being represented is intended to be used at a tactical or strategic level, as inserting inappropriate data into the development of strategy will lead to inefficiency rather than innovation.
Dashboard: Collated data on InterMedia’s @Audiencescapes, top row shows volume measures, second row shows relationships between @Audiencescapes followers, identifying those potentially more influential within the follower network, and their stated geographic location.
Dashboards can provide access to numerous perspectives on tactical data from social media, but are only useful for the specific purpose for which they are built. For example, showing the trend in followers over time, or identifying individuals with the most followers, makes specific assumptions about what or who delivers influence in a community. Equally, if dashboards include metrics of “influence,” it must be clear to the user how these have been calculated. Otherwise there is a serious risk of chasing an increasing “influence score” rather than genuine impact.
In addition to constructing dashboards, data tools make it possible to easily identify the countries in which an organisation has a particular focus and compare this with those they engage through social media. For example, the locations of programmes highlighted in the recent EUNIC Yearbook for 2011 Europe’s Foreign Cultural Relations can be used to show the countries in which the organisation focused its activity. The aggregated data shows that many of the priority countries, in terms of number of activities, were in Europe despite the emphasis on “Foreign Cultural Relations.”
The connections between physical world public diplomacy events and social media are increasingly becoming part of public diplomacy operations. With the growing availability of open data, these findings could then be placed in a context, relevant to the specific issues of a public diplomacy organisation.
Observing the ”greater networks” of communication
While the potential for big data to provide insight into the projects of a public diplomacy organisation, their ”egosphere,” there is at least equal potential to analyse the wider “ecosphere” — the “greater network” of communication and influence. This allows public diplomacy practitioners to genuinely listen. It allows them to hear what others are saying and place public diplomacy activity in the context of other sources of influence which are experienced by communities around the world. This is particularly relevant given the growth of collaborative strategies in public diplomacy.
For example, a number of newspapers now make their archive available via an API (application programming interface) which makes the automated longitudinal analysis of the shifting themes of discussion easier to achieve.
25 years of science reporting in The New York Times
Key annual themes and individuals shown at five-year intervals
Examples of this type of analysis have been produced by Jer Thorp on his Blprnt blog using The New York Times API and David McCandless on his site Information is Beautiful. In addition, technological designs by Marcos Weskamp have added Newsmap and Flipboard to the design led approaches to filtering the flow of news content. Many of these tools make information filtering easy and the results clear and aesthetically pleasing. Further options, including Seesmic, allow people to create their own bespoke digital media management systems. Many of these are not social media publishing or tracking tools, but instead provide an important opportunity for scholars and practitioners to understand the nature of the information environment in which they work.
In addition, analysing the information environment in which a public diplomacy initiative will be conducted is a combination of the commonly conducted volume-based reach measures and developing an understanding of the interconnected landscape created by the media consumption behaviour of the community. For example, if conducting public diplomacy via radio, rather than asking about who is listening, there is potential to focus on which media outlets citizens are choosing to access. This is a subtle nuance, but the result is radically different.
When considering public diplomacy within the “greater network,” research can identify the information landscapes created by the media consumption behaviours across the community. This blends offline research with methods more familiar to digital media analysis and allows the position of a particular radio broadcaster, television channel or newspaper to be viewed relative to other providers in the media landscape.
This approach allows public diplomacy organisations to look beyond “reach” numbers to identify combinations of media consumption behaviours. For example, do international broadcasters, such as BBC World Service, tend to reach the same audience as other international broadcasters including and Deutsche Welle or CCTV? Or do their audiences come from distinct communities? Equally this approach allows for the identification of communities who choose similar combinations of media. For example, a community may read a particular newspaper and also watch a specific TV channel. At the strategic level this insight can be used to engage with a particular community through their preferred combinations of media sources.
The use of digital media also creates the opportunity for public diplomacy practitioners and scholars to identify communities which focus around a specific issue. An innovative strategy can be created in response to a dynamic situation such as the Arab Spring, if an organisation can identify the sources to which people turn for information during a crisis.
For example, this may start with the question, which media sources were shared during the Arab Spring? To answer this question, a public diplomacy organisation could aggregate the available data and filter the resulting dataset to focus on retweets, for example. This would allow analysts to identify specific news brands that were shared by different communities. In addition, analysts could also begin to identify users who fulfil different roles within the network (or networks) of information sharing. In the example of the Arab Spring, some journalists, primarily responsible for producing content and information, were much more active digitally, becoming “information brokers,’” who aggregated and filtered content. The result was that some journalists were important in driving traffic to particular news stories, blogs, videos and tweets. This type of insight can allow those conducting public diplomacy to contact specific journalists or information brokers, based on who regularly engages with the social media content they produce.
In future crises the identification of information brokers in a specific network could become an operational priority. At the strategic level, a public diplomacy organisation could make the decision to focus on developing information brokers, with the goal of generating more interest and attracting greater numbers of users to a news brand or organisation. Alternatively, there is potential to drive traffic toward particular stories and specific news organisations. These are some of the many possibilities for future innovation which can be supported by big data and which can allow a public diplomacy organisation to see the world from different perspectives.
Innovative strategies can be built on insights drawn from appropriate sources of big data. A ”deep dive” into the available big data contextualised by bespoke research can provide a valuable insight and foundation upon which innovative strategies can be developed.
Reflecting on the potential for innovation in the era of big data, navigating the most useful ways to derive insight from that data is only a start. While it is true as a McKinsey Global Institute noted that, “innovation, and growth simply couldn’t take place without data”, it is equally important to recognise that data is only one of the building blocks in developing innovation in public diplomacy.
Data can empower a public diplomat. Open source methodology can deliver greater opportunities for collaborative strategies. However, innovative strategies require a combination of insight from big data and implementation by those with strong intercultural communication skills who are able to build relationships and work collaboratively. Even in the era of big data, it is still up to the individual who bridges the last three feet to deliver results.
Ali Fisher is Associate Director of Digital Media Research for InterMedia. He joined InterMedia in December 2011 from Mappa Mundi Consulting, where, for the past four years he has gained notoriety for his work in identifying how information flows through both online and offline communities, and the influence of relationships on human behavior. Dr. Fisher is based in InterMedia’s U.K. office and leads the ongoing development of the company’s global digital research practice, providing clients with comprehensive and targeted digital media insight that drives decision-making. Prior to 2008, Ali directed the Counterpoint think tank at the British Council and was a lecturer in International Relations at the University of Exeter, U.K. He completed a master’s in U.S. Intelligence Services and his Ph.D. at the University of Birmingham. In addition to his commercial work, Ali regularly produces work for academic publication. Dr. Fisher is a widely published author on topics related to public diplomacy, digital information flows, and engagement and influence patterns. His book ”Collaborative Public Diplomacy” will be published in 2012.
 Keith Grint, “Problems, problems, problems: The social construction of ‘leadership,’”, Human Relations, Vol. 58, No. 11, (2005)
Herbert Blumer, “Social Problems as Collective Behavior”, Social Problems, vol. 18, no. 3, (1971)
 John Sullivan, “Gangs, Hooligans, and Anarchists —The vanguard of netwar in the streets,” Networks and Netwars: The Future of Terror, Crime, and Militancy , (eds.) John Arquilla, David Ronfeldt (RAND, 2002) p. 99 http://www.rand.org/pubs/monograph_reports/MR1382/MR1382.ch4.pdf
 Chris Bronk and Scott Smith, “Speaking Out; How Data Visualization can Change Diplomacy.” Foreign Service Journal, March 2012, p. 11 http://bakerinstitute.org/publications/BRONK_030212.pdf
Brian Hocking, ‘Multistakeholder Diplomacy: Forms Functions and Frustrations’, in Multistakeholder Diplomacy – Challenges and Opportunities. Ed by J. Kurbalija and V. Katrandjiev (2006)
Hocking, B. ‘Changing the terms of trade policy making: from the “club”
to the multistakeholder model’. World Trade Review, (2004) 3(1), 3-26
 Big Data for Development: Challenges & Opportunities, Global Pulse, May 2012.
Alterntive definitions and discussion of innovation can be found:
Big data: The next frontier for innovation, competition, and productivity, McKinsey Global Institute, 2011. http://www.mckinsey.com/mgi/publications/big_data/
An interview with Dr Rami Mukhtar, senior researcher at National ICT Australia, by Divina Paredes, ‘The big career shift: Big Data’ PC Advisor’, 05 April 2011, http://mobile.pcadvisor.co.uk/news/enterprise/3349560/big-career-shift-big-data/
 Ben Rooney, “Big Data’s Big Problem: Little Talent” The Wall Street Journal, 29 April 2012,
Julian Borger, “Wake-up Call”, The Guardian, 6 September 2002.
 We knew the web was big…, Google Official Blog, 25th July 2008 (http://googleblog.blogspot.com/2008/07/we-knew-web-was-big.html)
 200 million Tweets per day, Thursday, Twitter blog, June 30th, 2011, http://blog.twitter.com/2011/06/200-million-tweets-per-day.html
Number of Internet users worldwide reaches two billion: UN, The Independent, 26 January 2011
x Hadoop Project, Apache Software Foundation, What is Hadoop? http://hadoop.apache.org/#What+Is+Apache+Hadoop%3F
MongoDB – http://www.mongodb.org/
MySQL – http://www.mysql.com/
Cascade page of NYTLabs: http://nytlabs.com/projects/cascade.html
Processing – http://processing.org/
Google Fusion Tables – http://www.google.com/fusiontables/Home/
Impure – http://www.impure.com/
OpenDX – http://www.opendx.org/
Blender – http://www.blender.org
 Nancy Snow, Information war: American propaganda, free speech and opinion control since 9/11, Seven Stories; 1st edition (August 5, 2003)
 An example of Google Hangouts in use, see: http://www.guardian.co.uk/news/datablog/video/2012/may/21/economics-google-debate
 The importance of recognising the distinction between strategic and tactical levels was made forcefully by Maj Ric Cole in his presentation: The Role of Information Operations in the ‘Battle of the Narratives’ and the use of ‘the Media as a Proxy’ at, Information Ops Europe: Impact of Extremism 2012
 Big data: The next frontier for innovation, competition, and productivity, McKinsey Global Institute, 2011. http://www.mckinsey.com/mgi/publications/big_data/
 Show of Hands – http://itunes.apple.com/us/app/show-of-hands/id336442597?mt=8
 Description of the distinction between reactive and unobtrusive research see;
Raymond Lee, Unobtrusive Methods in Social Research, Open University Press; 1 edition (August 11, 2000)
Eugene J. Webb, Donald T. Campbell, Richard D. Schwartz, Lee Sechrest, Unobtrusive Measures: Nonreactive Research in the Social Sciences, Rand Mcnally; Underlining edition (June 1966)
 Media, Power, and Politics in the Digital Age: The 2009 Presidential Election Uprising in Iran, (ed.) Yahya Kamalipour (Rowman & Littlefield, 16 Sep 2010)
 Ali Fisher and David Montez, Evaluating Online Public Diplomacy using Digital
Media Research Methods: A Case Study of #ObamainBrazil, InterMedia White Paper, 2011
For discussion of #ObamaGhana see Ali Fisher, “Mapping the Great Beyond; Identifying Meaningful Networks in Public Diplomacy” CPD Perspectives in Public Diplomacy, Paper 2, 2010 pp. 61-63
 Ali Fisher, “Mapping the Great Beyond; Identifying Meaningful Networks in Public Diplomacy” CPD Perspectives in Public Diplomacy, Paper 2, 2010 pp. 38-41
And via http://data.gov
A more limited dataset is available about U.K. Chevening scholarships: http://www.fco.gov.uk/en/about-us/what-we-do/scholarships/chevening/facts-figures/
 Antti Halonen, Being Open About Data, Analysis of the UK open data policies and applicability of open data, Finnish Institute in London (2012)
Information is Beautiful – http://www.informationisbeautiful.net
 Susan Gigli and Ali Fisher, “Networked Audiences”, The Channel, Issue 2, 2011, pp. 34-35.
 Big data: The next frontier for innovation, competition, and productivity, McKinsey Global Institute, 2011. http://www.mckinsey.com/mgi/publications/big_data/