#publisher alternate (BUTTON) Close Skip to main content sign in * Saved for later * Comment activity * Edit profile * Email preferences * Change password * Sign out subscribe search dating more from the guardian: * dating * jobs change edition: * switch to the UK edition switch to the US edition switch to the AU edition International * switch to the UK edition * switch to the US edition * switch to the Australia edition The Guardian * home * home * UK * world * sport * football * opinion * culture * business * lifestyle * fashion * environment * tech * travel browse all sections close Public Leaders Network Big data and open data: what's what and why does it matter? Both types of data can transform the world, but when government turns big data into open data it's especially powerful • What government can do with big data – livechat ESO's Visible and Infrared Survey Telescope image showing Helix Nebula Not all big, open data comes from government, such as scientists' research in astronomy. Photograph: ESO/VISTA/J. EMERSON / HANDOUT/EPA Joel Gurin, New York University Tuesday 15 April 2014 10.49 BST Last modified on Tuesday 3 June 2014 10.30 BST * Share on Facebook * Share on Twitter * Share via Email * Share on Pinterest * Share on LinkedIn * Share on Google+ * Share on WhatsApp Big data and the new phenomenon open data are closely related but they're not the same. Open data brings a perspective that can make big data more useful, more democratic, and less threatening. While big data is defined by size, open data is defined by its use. Big data is the term used to describe very large, complex, rapidly-changing datasets. But those judgments are subjective and dependent on technology: today's big data may not seem so big in a few years when data analysis and computing technology improve. Open data is accessible public data that people, companies, and organisations can use to launch new ventures, analyse patterns and trends, make data-driven decisions, and solve complex problems. All definitions of open data include two basic features: the data must be publicly available for anyone to use, and it must be licensed in a way that allows for its reuse. Open data should also be relatively easy to use, although there are gradations of "openness". And there's general agreement that open data should be available free of charge or at minimal cost. The relationship between big data and open data Venn diagram showing the relationship between big data and open data Source: Joel Gurin This Venn diagram maps the relationship between big data and open data, and how they relate to the broad concept of open government. There are a few important points to note. Big data that's not open is not democratic: Section one of the diagram includes all kinds of big data that is kept from the public – like the data that large retailers hold on their customers, or national security data like that collected by the NSA. This kind of big data gives an advantage to the people who control it but may disempower the rest of us. It's this kind of big data that has become most controversial. Open data doesn't have to be big data to matter: Modest amounts of data, as shown in section four, can have a big impact when it is made public. Data from local governments, for example, can help citizens participate in local budgeting, choose healthcare, analyse the quality of local services, or build apps that help people navigate public transport. Big, open data doesn't have to come from government: This is shown in section three. More and more scientists are sharing their research in astronomy, genomics, and other areas in a new, collaborative research model. Other researchers are using big data collected from social media – most of which is open to the public – to analyse public opinion and market trends. But, when the government turns big data into open data, it's especially powerful: Government agencies have the capacity and funds to gather very large amounts of data (such as the US examples in section six), and opening up those datasets can have major economic benefits. I now direct the Open Data 500 study at the GovLab at New York University. We've found 500 examples of US-based companies that are building their businesses on open government data, and much of that data is big data as well. We're now planning to work with Open Data Institute to replicate our study in the UK, and I expect to find the same pattern. Applying open data principles to big data can help solve some of the difficult issues that big data has raised. The biggest threat to public wellbeing is the risk that private, personal data can be collected and used as big data in ways the subjects of the data – namely, all of us – may not want or approve of. Paradoxically, opening up this sensitive data, in a specific and controlled way, may actually make it more secure. The problem now is not only that government agencies and some businesses are collecting personal data about all of us; it's also that we as individuals don't know what's being collected and don't have access to the information about ourselves. If we knew more, we could control more. The UK government's midata initiative, which has encouraged businesses to share customer records with customers themselves, is part of the solution. Similar US programmes, such as Blue Button for health records and Green Button for energy usage data, are also having a positive effect. Both big data and open data can transform business, government, and society – and a combination of the two is especially potent. Big data gives us unprecedented power to understand, analyse, and ultimately change the world we live in. Open data ensures that power will be shared – and that the world we change will, with luck, become a fairer and more democratic one. Joel Gurin is senior advisor at GovLab at New York University, where he directs the Open Data 500 study. He is author of Open Data Now. • What government can do with big data – livechat • Want your say? Email us at public.leaders@theguardian.com. Join the Public Leaders Network for more comment, analysis and job opportunities, direct to your inbox. Follow us on twitter via @Guardianpublic. __________________________________________________________________ More blogposts Topics * Transparency * Big data __________________________________________________________________ * Share on Facebook * Share on Twitter * Share via Email * Share on Pinterest * Share on LinkedIn * Share on Google+ * Share on WhatsApp * Reuse this content View all comments > comments Sign in or create your Guardian account to join the discussion. This discussion is closed for comments. We’re doing some maintenance right now. You can still read comments, but please come back later to add your own. Commenting has been disabled for this account (why?) (BUTTON) Order by * (BUTTON) newest * (BUTTON) oldest * (BUTTON) recommendations (BUTTON) Show 25 * (BUTTON) 25 * (BUTTON) 50 * (BUTTON) 100 * (BUTTON) All (BUTTON) Threads * (BUTTON) collapsed * (BUTTON) expanded * (BUTTON) unthreaded Loading comments… Trouble loading? (BUTTON) View more comments popular The Guardian back to top * home * UK * world * sport * football * opinion * culture * business * lifestyle * fashion * environment * tech * travel all sections close * home * UK + education + media + society + law + scotland + wales + northern ireland * world + europe + US + americas + asia + australia + africa + middle east + cities + development * sport + football + cricket + rugby union + F1 + tennis + golf + cycling + boxing + racing + rugby league * football + live scores + tables + competitions + results + fixtures + clubs * opinion + columnists * culture + film + tv & radio + music + games + books + art & design + stage + classical * business + economics + banking + retail + markets + eurozone * lifestyle + food + health & fitness + love & sex + family + women + home & garden * fashion * environment + climate change + wildlife + energy + pollution * tech * travel + UK + europe + US + skiing * money + property + savings + pensions + borrowing + careers * science * professional networks * the observer * today's paper + editorials & letters + obituaries + g2 + weekend + the guide + saturday review * sunday's paper + comment + the new review + observer magazine * membership * crosswords + blog + editor + quick + cryptic + prize + quiptic + genius + speedy + everyman + azed * video * Public Leaders Network * › Transparency IFRAME: /email/form/footer/37 * Facebook * Twitter * Facebook * Twitter * all topics * all contributors * solve technical issue * complaints & corrections * terms & conditions * privacy policy * cookie policy * securedrop © 2016 Guardian News and Media Limited or its affiliated companies. All rights reserved.