post comment most recent posts ‘age of the customer’ requires real-time data and kpis american express charges into the world of bigdata automation is the new reality for bigdata initiatives tips for building your next-generation data platform * search __________________ submit * * recent bigdata ebooks - bigdata analytics predictions for 2016 - the definitive guide to retail analytics - the database decision - real-world analytics in the cloud - take a smart approach to bigdata analytics - analytics, cloud, and the insight economy - improve access to data across your company - the guide to real-time hadoop - predictive analytics: opportunities, challenges and use cases click here to view our full library of ebooks. * * categories - advanced analytics - analytics and bigdata glossary - analytics technology - bigdata management - bigdata analytics use cases - bigdata skills - cloud computing: hybrid and private cloud - customer analytics - data visualization - internet of things (iot) - latest news on bigdata analytics - operational analytics * join the conversation [659de830-867b-439a-809e-872d132cc6ec.png] [ae05c418-e4a9-484a-898f-7ec0af3fdc05.png] [8f45ef68-49df-4455-8a88-5222115317da.png] [738a0fcd-e7aa-498b-9d96-98f31426118c.png] [c23e3344-f4fa-4f06-9822-b8933756f120.png] * * * about us * contact us * advertising * advisers * privacy policy * site map * terms and conditions 20 carematrix drive, dedham, ma 02026, usa | sales and customer service: 1.781.751.8799 | email: customer@wispubs.com copyright © 2016 wellesley information services. all rights reserved. skip to main content search form _______________ search (search) search menu home times higher education (the) * professional + news + comment + reader comments + books + features + awards + summits + digital editions * jobs + find a job + jobs by email + search recruiters + careers advice * rankings + world university rankings + by subject + news + about * student + find a university + news + advice * login * register × the please log in e-mail or username * ____________________________________________________________ password * ____________________________________________________________ (log in) log in not a member yet? join us for free forgotten password? ×close the join us to get full access e-mail * ____________________________________________________________ a valid e-mail address. all e-mails from the system will be sent to this address. the e-mail address is not made public and will only be used if you wish to receive a new password or wish to receive certain news or notifications by e-mail. password * _________________________ first name * ____________________________________________________________ last name * ____________________________________________________________ [ ] i agree to the cookie policy * to comply with privacy regulations in the eu, we require your consent for timeshighereducation.co.uk to place and use cookies. please read our cookie policy. [ ] i agree to the terms and conditions * by clicking on the 'join us' button you agree to abide by the the terms and conditions. newsletters [ ] higher education updates from the the editorial team [ ] world university rankings news [ ] new offers, products and services from the/tsl choose your profile * [- select a value -______] (join us) join us already have a the account? log in now the green paper needs bigdata students cannot be put at the heart of the system without a statistics upgrade, argues paul clark november 18, 2015 * facebook share * twitter * linkedin * by paul clark data the green paper on higher education in england placed something else alongside students at the heart of the new system – data and information. as we move further towards a marketised sector in england, the importance of data becomes ever more pronounced. good data allow students to make informed choices, allow policymakers and regulators to make better decisions, promote public trust and confidence in the system, enable institutions to be competitive and provide a lever to incentivise or penalise behaviour in the absence of public funding. good data also lead to efficiency – both through exposing opportunities where efficiencies can be made, and through providing competitive data for benchmarking purposes. all of these themes are present in the green paper, and are also prominent in the thinking of the other three countries of the uk. the higher education statistics agency is just one month away from closing its data futures consultation into the transformation of our data collection systems and we are about to embark on a series of workshops around the uk, at which we hope to have a dialogue with the sector about how we make sure that providers will have the information about their students that they need to stay efficient and competitive. just as the regulatory and funding architecture of the he sector needs to be upgraded to take account of a new environment, so does the sector’s data infrastructure, which is struggling to reflect the diversity and dynamism of today’s system. this infrastructure is fragmented, and in some places inefficient and out of date. with data becoming the fuel of the new public policy, rectifying these problems is urgent. while changes undoubtedly need to be made, some things should not be lost in transition. these include the high degree of trust that can be placed in the uk’s higher education data owing to the independence and impartiality of its acquisition and processing mechanisms, the uk-wide service that the infrastructure provides and the aspiration to make as much of the core data as open and as accessible as possible. in essence, this is a set of proposals to move to frequent in-year collections of data (beginning with student data), while at the same time reducing the overall number of separate collections made by different agencies. through transforming the technology platform, redesigning the data specification and overhauling the sector’s data governance structures, the principle of “collect once, use multiple times” is achievable. further developments can build out from this – providing enhanced analytical tools for users and providers, opening up larger stores of data for analysis and innovation, linking datasets across government departments and policy areas to improve decision-making and reducing the transactional costs associated with data flows around the sector. the potential benefits for all parties are considerable. for policymakers and regulators, it will deliver accurate, high-quality data more frequently throughout the cycle, enhancing decision-making and providing early warning of emerging problems. for higher education providers it will better support their own strategic planning and decision-making in a more competitive environment. it will reduce costs and generate efficiencies that could lead to redeploying resource internally, and will ensure that statutory responsibilities can be met in the most cost-effective way. most importantly, for students and the wider public, it will ensure that there is a source of trustworthy and reliable data available when they need it, to make informed decisions about the future. paul clark is chief executive of the higher education statistics agency. you've reached your article limit register to continue registration is free and only takes a moment. once registered you can read a selection of articles each month plus: * sign up for the editor's highlights * receive world university rankings news first * get job alerts, shortlist jobs and save job searches * participate in reader discussions and post comments free registration already registered? sign in now if you are already registered or a current subscriber. or subscribe for unrestricted access to our digital editions and ipad and iphone app. sign in nowforgotten password? related content have your say log in or register to post comments featured jobs university secretary birmingham city university chief financial officer birmingham city university tenure-track assistant professor in mixed-signal integrated circuit design nanyang technological university tenure-track assistant professor position in the field of power engineering nanyang technological university assistant director (operations) university of derby register to continue you've enjoyed reading five the articles this month. register now to get five more, or subscribe for unrestricted access. you might also like terrorism, radicalisation, extremists of all hues 'regard students as low-hanging fruit' january 14, 2016 overworked man slumped on desk universities should ask whether their academics work too much january 14, 2016 india on globe indian student numbers continue to fall at uk universities january 14, 2016 globalisation world’s most international universities 2016 january 14, 2016 tweets by @timeshighered most commented globalisation world’s most international universities 2016 times higher education world university rankings data reveal the top 200 most outward-looking institutions rhodes oxford plaque oxford college agrees to remove cecil rhodes plaque but the fate of rhodes statue remains uncertain as anti-racism campaigners call for it be taken down people in maze dressed as santa claus ref: static ranking raises questions about management policies performance targets to boost league table position are questioned as analysis shows little change in russell group ref order busker playing guitar upside-down cambridge music job advert strikes discordant note in academia academics are divided over whether it was right to use unusually specific criteria when seeking applicants for new post silhouette of people using desktop pc moocs: international credit transfer system edges closer six universities in talks on global credit transfer system for online courses * contact us * client services * write for the the * terms & conditions * cookie policy times higher education subscriptions subscribe if you like what you're reading online, why not take advantage of our subscription and unlock all of times higher education's content? you'll get full access to our website, print and digital editions, and the times higher education app for apple ipad and iphone. subscribe #rss feed a social media today community [network-drop-down.png] * [brand-bar-wheel.png] about social media today ______________________________________________________________ * [logo.png] the world's best thinkers on energy & climate ______________________________________________________________ * [logo.png] the world's best thinkers on the urban future ______________________________________________________________ * [logo.png] the world's best thinkers on business intelligence & data analytics ______________________________________________________________ * [logo.png] the world's best thinkers on healthcare ______________________________________________________________ * [logo_0.png] the world's best thinkers on social media sign up | login with → smartdata collective the world's best thinkers on data * home + about us & contact + work with us * post here * analytics + social media analytics + text analytics + sentiment analytics + predictive analytics + modeling + web analytics + workforce analytics * bigdata + social data + data warehousing + visualization + workforce data + data mining + data quality * business intel + business rules + decision management + knowledge management + market research + crm * data management + risk management + transparency + culture & leadership + policy & governance + best practices + privacy * it + mobility + security + hardware + location + cloud computing * software + hadoop + mapreduce + sql + open source + marketing automation + r programming language * reports * webcasts * help + faq posted by: [screen%20shot%202013-06-06%20at%2015.18.37.png] bernard marr posts profile comments follow please login or register to follow this user. bigdata in big oil: the amazing ways shell uses analytics to drive business success like it? 92 comments 0 posted november 17, 2015 keywords: the bigdata guru image the oil and gas industries are facing major challenges - the costs of extraction are rising and the turbulent state of international politics adds to the difficulties of exploration and drilling for new reserves. in the face of big problems, its key players are turning to bigdata in the hope of finding innovative solutions to these pressing issues. bigdata is the name used to describe the theory and practice of applying advanced computer analysis to the ever-growing amount of digital information that we can collect and store from the world around us. over the last few years businesses in every industry have enthusiastically developed data-led strategies for overcoming problems and solving challenges, and the oil and gas industries are no different. royal dutch shell is one of the largest oil and gas companies – one of the “supermajors” which also include bp, chevron, total and exxonmobil – and the world’s fourth largest company by revenue. for some time now it has been developing the idea of the “data-driven oilfield” in an attempt to bring down the cost of drilling for oil – the industry’s major expense. a recent survey by accenture and microsoft of oil companies and those involved in the support industries found that 86% to 90% of respondents said that increasing their analytical, mobile and internet of things capabilities would increase the value of their business. the search for new hydrocarbon deposits demands a huge amount of materials, manpower and logistics. with drilling a deep water oil well often costing over $100 million, no one wants to be looking in the wrong place. surveying of potential sites involves monitoring the low frequency seismic waves that move through the earth below us due to tectonic activity. probes are put into the earth at the spot being surveyed, which will register if the pattern of the waves is distorted as they pass through oil or gas. in the past this would involve taking a few thousand readings during the typical survey of a potential drilling site. but in the past few years technology has advanced to the level where it could involve over a million – vastly increasing the amount of data gathered during exploration. shell uses fibre optic cables, created in a special partnership with hewlett-packard, for these sensors, and data is transferred to its private servers, maintained by amazon web services. this gives a far more accurate image of what lies beneath. data from any prospective oil field can then be compared alongside that from thousands of others around the world, to enable geologists to make more accurate recommendations about where to drill. production forecasting is one of the first jobs – determining the likely output of the reservoir is key to determining what resources should be spent on collecting it. when this decision is data-led, operators can have more confidence that this will be done efficiently. shell also uses bigdata to ensure its machines are working properly and spending as little time as possible offline due to breakdowns and failure. machinery used in drilling has to operate in harsh conditions for prolonged periods of time so is prone to wear and damage. to counteract this, the machinery is fitted with sensors collecting data about its performance and comparing it with aggregated data, meaning parts can be replaced in an efficient manner and downtime minimized, further reducing overheads. as well as exploration, bigdata is being put to use to streamline the transport, refinement and distribution (retail) of oil and gas. shell is vertically integrated, and therefore involved in every aspect of the process through to packaging and selling it to the consumer as fuel for their car for heating their home. refineries have limited capacity and fuel needs to be produced as close as possible to its point of end use, to minimize transportation costs. complex algorithms take into account the cost of producing the fuel as well as diverse data such as economic indicators and weather patterns to determine demand, allocate resources and set prices at the pumps. of course, we’ve long been conscious of the fact that we could eventually use up all of the non-renewable oil and gas buried under the earth – perhaps sooner than we think. while this is an environmental concern to us all, it’s a financial one to companies like shell. dwindling reserves mean the cost of getting at what is available goes up, as they are forced to look deeper underground in ever more remote locations. one alternative is offered by the growing hope that “unconventional resources”, such as shale gas and tight oil will fill the gap. these resources, trapped in shale and sandstone, now supply 20% of the gas used in the usa and their use is expanding rapidly around the globe. however they demand new extraction methods such as hydraulic fracturing (“fracking”). because this is relatively new far less data is available so exploration can be more hit-and-miss. however new techniques are being developed to use the data that is being collected from probing and drilling these sites to help pinpoint other locations where reserves could be hiding. all of these elements form the “big picture” – every part of which must be synchronized if a company is going to prosper in the costly, dangerous and highly competitive field of oil and gas production. thanks to bigdata analytics, the interconnection between these elements of the business can be examined and monitored in detail. this means models can be built and simulations created by analysts, to explore how minor tweaks to a certain area of operations could have big impacts on the productivity or efficiency of another. the vast amount of data collected from all areas of the company’s activity means the result of the simulations will hopefully be as close as possible to the way things will play out in the real world. ultimately this leads to decision-makers being better equipped to make the decisions that affect the company’s fortunes. as always, thank you very much for reading my posts. you might also be interested in my new books, which are available to pre-order now: key business analytics: the 60+ business analysis tools every manager needs to know and bigdata in practice. * login or register to post comments * [screen%20shot%202013-06-06%20at%2015.18.37.png] connect: twitter linkedin facebook website authored by: bernard marr bernard marr is a globally regognized bigdata and analytics expert. he is a best-selling business author, keynote speaker and consultant in strategy, performance management, analytics, kpis and bigdata. he helps companies to better manage, measure, report and analyse performance. his leading-edge work with major companies, organisations and governments across the globe makes him a globally ... see complete profile would you like to contribute to this site? get started » other posts by bernard marr 3 ways the internet of things will change every business - november 10, 2015 bigdata in motorsports: how f1 and nascar compete on analytics - october 13, 2015 4 ways bigdata will change every business - october 6, 2015 of bigdata, the iot, and trust: tales from google's nest - september 29, 2015 the amazing ways bigdata drives success at rolls royce - september 15, 2015 related posts there are no posts that fit this criteria. » already a member? login now to comment! » not a member? register to comment! like it? 11 january 7, 2016 webb rowan says: regardless of the nature of any business, finance is the key factor that helps the firm progress forward toward success. when operating costs increase, their profits will in turn reduce. hence, to address this ongoing issue, business owners need to always find smart alternatives that have all along been tactfully disregarded due to its subtle nature like analytics. when studied on an in-depth basis, it can help a business regain the losses slowly, but surely. share this comment: twitter facebook linkedin * login or register to post comments please note: your first comment as a registered user will be held for moderation up to 24 hours (usually less). for more information about comments on our site, please read our faq and terms of use. image the moderated business community for business intelligence, predictive analytics, and data professionals. image * twitter * iframe: http://platform.twitter.com/widgets/follow_button.html?screen_name= smartdataco * rss * rss as xml addtomyyahoo4 subscribe with bloglines add to google * email * get our best posts in your inbox weekly first name ____________________ last name ____________________ email address ____________________ sign up! how do you innovate effectively and maintain a competive edge? image learn how in our exlcusive ebook, "bad data need not apply: designing the modern data warehouse environment." download here. sdc special columns big-data-guru riksy business big-data-mops events no events listed add your own! recommended to follow ahamed parpia follow a t follow jan henderyckx follow brent leary follow orlando doehring follow daniel kraft follow comments * featured * popular * recent “the final aim of mcc is to allow implementation of rich smartphone web apps on an excess of cell phone devices, with a gorgeous user experience.” january 7th, 2016 by jamesmartin057 “by including more factors to score a person’s credit worthiness would make it even tougher for borrowers to obtain a loan approval. this in turn causes the finance industry to face a fluctuating business trend. there are definitely pros and cons to this major change in a loan application process but one thing for sure is to ensure borrowers shall only be allowed to take up a loan which is ...” january 7th, 2016 by webbrowan “thanks for the insight connie!” january 4th, 2016 by daniel matthews “there are clearly a number of changes coming for small businesses as they await ...” january 6th, 2016 by nguyennga “very useful tips, especially from the point of view of customer service.” january 4th, 2016 by isabella1977 “this is such a scary article. no doubt there is a lot of things that you have ...” january 6th, 2016 by webbrowan “the final aim of mcc is to allow implementation of rich smartphone web apps on ...” january 7th, 2016 by jamesmartin057 “regardless of the nature of any business, finance is the key factor that helps ...” january 7th, 2016 by webbrowan “such comparison tools are great for borrowers to perform an estimated ...” january 7th, 2016 by webbrowan “when it concerns money, you cannot afford to commit even a single mistake and ...” january 6th, 2016 by webbrowan “due to modernization we have found good technology advanced in the society; ...” january 7th, 2016 by edwardmiller “social collaboration seems like a funny way to describe the process of speaking ...” january 6th, 2016 by webbrowan show more “the final aim of mcc is to allow implementation of rich smartphone web apps on an excess of cell phone devices, with a gorgeous user experience.” january 7th, 2016 by jamesmartin057 “by including more factors to score a person’s credit worthiness would make it ...” january 7th, 2016 by webbrowan “due to modernization we have found good technology advanced in the society; ...” january 7th, 2016 by edwardmiller “regardless of the nature of any business, finance is the key factor that helps ...” january 7th, 2016 by webbrowan “such comparison tools are great for borrowers to perform an estimated ...” january 7th, 2016 by webbrowan “there are clearly a number of changes coming for small businesses as they await ...” january 6th, 2016 by nguyennga “when it concerns money, you cannot afford to commit even a single mistake and ...” january 6th, 2016 by webbrowan “social collaboration seems like a funny way to describe the process of speaking ...” january 6th, 2016 by webbrowan “this is such a scary article. no doubt there is a lot of things that you have ...” january 6th, 2016 by webbrowan “thanks for the insight connie!” january 4th, 2016 by daniel matthews show more popular * read * commented * authors h z - points john staunton - points jan rubio - points sarma ranga - points dawn bronkema - points in memoriam: robin fray carey (1) 1/6/2016 by sdc staff 72% of people aren’t familiar with hosted voip (0) 1/4/2016 by josh rose how to recycle old it equipment and keep your data safe (0) 1/4/2016 by ryan kidman 5 predictions for trends in data, analytics and machine learning in 2016 (0) 1/4/2016 by bruce robbins will 2016 be the year you clean up your dirty data? (0) 1/5/2016 by martin doyle 3 ingenious use cases for business intelligence tools 1/11/2016 by eran levy 3 perks of an office 365 migration that you didn’t know about 1/12/2016 by kyle cebull nosql databases: 4 game-changing use cases 1/13/2016 by jim scott flash vulnerabilities & exploits: an information security primer 1/12/2016 by thu pham nosql and the internet of things 1/14/2016 by jim scott advisory panel about the panel aaron aders more » bruno aziza more » stephen baker more » paul barsch more » gary cokins more » jill dyché more » themos kalafatis more » bernard marr more » james taylor more » smartdata collective * audio archive: the cutting edge of bigdata monetization * audio archive: delivering value through business analytics * audio archive: trendspotting for growth * audio archive: finding value in unstructured social data * audio archive: risks and rewards of bigdata in the cloud * audio archive: putting customer value to work categories » analytics » best practices » bigdata » book review » business intelligence » business rules » cloud computing » collaborative data » commentary » crm » culture/leadership » data management » data mining » data quality » data visualization » data warehousing » decision management » ebook » hadoop » hardware » inside companies » it » jobs » knowledge management » location » mapreduce » market research » marketing » marketing automation » mobility » modeling » new products » news » open source » podcasts » policy and governance » predictive analytics » privacy » r programming language » risk management » security » sentiment analytics » smartdata collective exclusive » social data » social media analytics » software » sql » statistics » text analytics » transparency » unstructured data » web analytics » webcasts » workforce analytics » workforce data show more featured contributors * you * melanie aizer * paul barsch * meta s. brown * jason burke * gary cokins * ted cuzzillo * barry devlin * chris dixon * erica driver * michael fauscette * bill franks * david hecht * julie hunt * lachlan james * doug lautzenheiser * alex olesker * david smith * james taylor * daniel tunkelang * bigdata smart data cloud social data unstructured data data in the cloud business intelligence analytics social media today llc © 2012 terms of use | community guidelines | privacy policy | get a badge | help | about & contact quantcast [awlogo.png] * * subscribe * aw editors * products * factory * process * engineering * it * operations * [moxasearch.png] magazine subscribe advertise contact current issue about magazine digital edition conferences automation conference & expo all events playbooks batch process automation continous process automation factory & machine automation tactical briefs collaborative manufacturing control panel optimization embedded systems & trends embedded vision in manufacturing view all channels totally integrated automation device integration strategies 360 product review supplier news sps magazine webinars predictive asset management: a success story changes to your automation programs happen. are you managing them? digitizing complex discrete manufacturing processes to accelerate your competitiveness view all leadership view 2015 leaders in automation looking to receive automation world magazine and newsletters? [subscribeitems.png] subscribe now search form _______________ (go) go [awlogo.png] [magnifyingglass.png] search form _______________ (go) go subscribe about magazine advertise close factory & machine automation playbook this one-of-a-kind factory automation playbook is packed with best practices, practical tips and pitfalls to avoid on a wide range of topics, from defining project objectives to selecting components to implementing technologies that can make your automated systems smarter and more productive. download bigdata startup unveils end-to-end it management opsdatastore’s new product builds relationships between systems, applications and machines across the enterprise to provide a clear view of interactions and anomalies in the work environment. by stephanie neil , senior editor, on november 17, 2015 share article: filed in: all, data acquisition http://www.automationworld.com/big%20data%20startup%20unveils%20end-to- end%20it%20management%20 news bigdata startup unveils end-to-end it management opsdatastore’s new product builds relationships between systems, applications and machines across the enterprise to provide a clear view of interactions and anomalies in the work environment. by stephanie neil , senior editor, on november 17, 2015 share article: atlanta-based startup opsdatastore, which launched its business just last month with $3 million in series a funding, will announce the availability of its first product today. opsdatastore 1.0 is a bigdata back end that provides real-time visibility and transparency across operations and application performance so that information technology (it) professionals can proactively prevent performance and availability issues, thereby avoiding downtime and its associated revenue loss. the company’s new technology can blend and manage different data sources, providing a unified view of all it systems, applications and machine relationships across the enterprise—and the factory floor, company officials said. the technology solves the problem of disjointed legacy management frameworks that have been set up as closed software suites focused on only a slice of enterprise data, be it storage, networking, servers or operating systems. it organizations, however, need an end-to-end monitoring and troubleshooting system, which is what opsdatastore delivers. the key to the company’s ability to deliver a holistic management system is its partner ecosystem, also announced today. opsdatastore has aligned itself with management tool companies appdynamics, dynatrace and extrahop, as well as visual analytics vendors qlik and tableau. platform partners include microsoft and vmware. in addition, a data collector software development kit (sdk) allows anyone to write a plug-in to integrate a new source of data. “we are the first vendor to accept data from all vendors and end up with relationships of data, not just dumb data,” said opsdatastore ceo bernd harzog. “and we have a business model that allows us to partner with vendors instead of compete.” the product is so new that the company has no customers yet, however, harzog sees a huge potential for this technology in manufacturing organizations. “the purpose is to use the relationships and the analytics built into the product to tell you where things are going wrong,” he said. and, while there is a heavy it focus, having insight into these systems is an important part of keeping an integrated plant up and running because the technology shows anomalies in things that are deterministically related to each other. “if you see things going wrong with real time instrumentation on the factory floor, the first thing you want to do is figure out what could be causing it.,” harzog said. “and the first step is figuring out what that machine is connected to, or what it’s dependent upon, because the machine is not an island.” opsdatastore is built on apache cassandra, an open source distributed database system designed for storing and managing large amounts of information across commodity servers. it is the same backend that linkedin servers use to process millions of transactions to make recommendations of people you may know, for example. this makes it a powerful data consumption and analytics tool. “we are an it technology designed to keep up with the pace of operations technology (ot),” harzog said. key features include: * open data collection architecture to consume data from any platform and any management software product. * a dynamic object model which relates items to each other at ingest time and maintains a continuously updated topology map of the entire environment over time. * real-time scalable bigdata back end built on open source, low latency and bigdata technologies. * open query architecture supporting both rest api and odbc as an api for data access. * support for leading business intelligence tools with dashboards implemented in qlikview and tableau. pricing for the on-premise software is $100 per year for each instance of windows or linux from which data is collected. so, the cost to a manufacturing environment with 1,000 machines supported by 1,000 servers in it would be $100,000 per year. there is no charge for the machines or end user devices generating the data. companies in this article microsoft vmware it delivers on automation’s promise it delivers on automation’s promise sign up to receive timely updates from the editors at automation world and download this free special report on the transformative power of data in manufacturing. by integrating information and automation technologies, manufacturers are finally achieving major gains in productivity from their automated systems. email * _______________ (get my free download ») get my free download » don't miss the latest insight: get the factory automation newsletter [newsletterwidget.png] ____________________ sign up read more on: related articles how the internet of things is changing industry investment advantech finalizes b+b smartworx acquisition 2015 leadership in automation: first team honorees newsletters don't miss intelligence crucial to your job and business! click on any newsletter to view a sample. enter your email address below to sign up! news insights news & analysis [ ] product insights latest automation products [ ] best of blogs industrial automation columnists [ ] automation focus sponsored white papers, videos and products [ ] continuous processing oil & gas, chemical and more [ ] automation skills improve industry skills [ ] industrial internet of things network application of ie [ ] on the edge blog workforce development [ ] factory automation technology for discrete manufacturing [ ] food bev & pharma automation applications & trends [ ] opconnect newsletter [ ] profinews na [ ] totally integrated automation [ ] device integration strategies [ ] automation strategies [ ] fully integrated components [ ] each newsletter ranges in frequency from once per month to a few times per month at most. enter your email add (sign up) sign up control systems integrators report download aw's exclusive new survey results on system integrators, and gain access to the industry's premier integrator directory: [csiaheroshot_v3.png] email * ____________________________________________________________ country * [- select -__________________________________] which best describes your company? * (*) i work for a system integrator ( ) we're using or considering system integrators (get your free report) get your free report leadership logo data acquisition mouse over to browse these companies * [ls_icon2.png] acromag * [ls_icon2.png] allied electronics * [ls_icon2.png] antaira technologies, llc * [ls_icon2.png] automationdirect * [ls_icon2.png] banner engineering * [ls_icon2.png] beckhoff automation * [ls_icon2.png] emerson process managemen... * [ls_icon2.png] ge automation & controls * [ls_icon2.png] ge digital * [ls_icon2.png] idec corporation * [ls_icon2.png] inductive automation * [ls_icon2.png] kepware technologies * [ls_icon2.png] mitsubishi electric autom... * [ls_icon2.png] moxa americas, inc. * [ls_icon2.png] national instruments * [ls_icon2.png] omega engineering * [ls_icon2.png] omron industrial automati... * [ls_icon2.png] opto 22 * [ls_icon2.png] red lion controls * [ls_icon2.png] rockwell automation * [ls_icon2.png] schweitzer engineering la... * [ls_icon2.png] sick inc. * [ls_icon2.png] siemens industry * [ls_icon2.png] softing * [ls_icon2.png] wago corporation * [ls_icon2.png] wonderware by schneider e... don't miss the latest insight: get the factory automation newsletter [newsletterwidget.png] ____________________ sign up [facebookgrey.png] [twittergrey.png] [linkedingrey.png] [rssgrey.png] subscribe advertise contact privacy policy terms of use payment terms © 2016 pmmi media group quantcast #rss feed a social media today community [network-drop-down.png] * [brand-bar-wheel.png] about social media today ______________________________________________________________ * [logo.png] the world's best thinkers on energy & climate ______________________________________________________________ * [logo.png] the world's best thinkers on the urban future ______________________________________________________________ * [logo.png] the world's best thinkers on business intelligence & data analytics ______________________________________________________________ * [logo.png] the world's best thinkers on healthcare ______________________________________________________________ * [logo_0.png] the world's best thinkers on social media sign up | login with → smartdata collective the world's best thinkers on data * home + about us & contact + work with us * post here * analytics + social media analytics + text analytics + sentiment analytics + predictive analytics + modeling + web analytics + workforce analytics * bigdata + social data + data warehousing + visualization + workforce data + data mining + data quality * business intel + business rules + decision management + knowledge management + market research + crm * data management + risk management + transparency + culture & leadership + policy & governance + best practices + privacy * it + mobility + security + hardware + location + cloud computing * software + hadoop + mapreduce + sql + open source + marketing automation + r programming language * reports * webcasts * help + faq posted by: [screen%20shot%202015-07-01%20at%209.37.44%20am.png] xander schofield posts profile comments follow please login or register to follow this user. how the healthcare industry can save big with bigdata like it? 108 comments 0 posted november 17, 2015 keywords: workforce data, bigdata, business intelligence when it comes to bigdata, just about every industry can benefit in some way or another. the healthcare sector is just starting to realize the potential value of bigdata, and the next several years will illustrate just how powerful and useful it can be to have detailed records of all kinds of patients over time. the healthcare industry has been known to periodically incorporate trending technical best practices to cut costs and save lives. the creation of websites with a strong emphasis on userability has been a recent push, and by extension the application of stronger network performance monitoring software, free or paid. hospitals and individual clinics are leveraging an overall digital approach to record-keeping and storage. these technical optimizations made within the healthcare industry have primed the way for greater advancement within the new field of bigdata. the idea of collecting and analyzing as much customer information as possible has now become a major goal in the most cutting-edge marketing departments. it lets the company understand exactly how consumers behave and what attract them to the brand, reducing the costs of acquiring and retaining customers. however, the healthcare industry is new to the concept of bigdata. it is only recently that care providers have been legally obligated to start switching over to electronic medical records. that has opened the door to storing all of that data for useful analysis and potential cost-saving measures. one of the most difficult aspects of health care is that each patient responds to treatment in different ways. as a result, it can be a long, expensive, and dangerous process to find exactly the right treatment for any combination of patient and condition. skipping some or all of this trial and error period to find a useful intervention right from the start would save a lot of time and money, while also reducing the risk to the patient. that is what some researchers are doing right now—developing tools that clinicians can use to build personalized treatments. it might sound like it would be more expensive to tailor treatments to fit each patient, but in reality, the opposite is true. for example, one team of researchers built a tool that would let doctors put in the characteristics of a patient with pulmonary nodules, and then the tool would search the world's largest database of cases of those nodules and come up with some matching cases for comparison. this lets the doctor see what worked and what didn't work in those similar cases. the potential savings from tapping into personal data in this way are quite large. a report from mckinsey had a ballpark estimate of twelve to fifteen percent cuts in the cost of administering care. considering the amount of money that america spends on healthcare each year, that is huge. it is also important to note that the coming wave of retiring boomers will lead to a higher level of health care expenditures. that means it is even more important to start finding ways to cut costs now, before it gets out of control. the potential savings do not only come from personalized medicine. there is also the area of genetic testing, which is a different kind of bigdata. researchers are already aware of some genetic markers that are associated with an increased risk of developing particular forms of cancer or other diseases. right now, the information does not have a great deal of predictive power, but as more people get their genomes sequenced and as healthcare providers track the outcomes of those people, we might see a better picture emerge, where genes can predict a lot about the most important health risks each person faces. while it does raise privacy concerns, proper information security and network monitor investment will go a long way towards protecting patient data from anyone who wants to steal it. the cost savings that the industry could experience are so large that it is hard to imagine care providers not trying to tap into bigdata in some way. * login or register to post comments * [screen%20shot%202015-07-01%20at%209.37.44%20am.png] authored by: xander schofield after a couple of failed startups and one that stuck, xander schofield has turned his attention to independant consulting. schofield now spends his time consulting startups and small businesses on enterprise-level solution implimentation on a smaller scale. when he's not on the road, he can be found at home with his wife and and two french bulldogs. see complete profile would you like to contribute to this site? get started » other posts by xander schofield how iot applications can help businesses grow internally - january 7, 2016 how the cia is reinventing a case for bigdata - december 16, 2015 not seeing the results of bigdata? maybe you have a lot of data, not big data - october 13, 2015 the usa is building the world's fastest supercomputer - september 30, 2015 amazon's cloud computing giant is getting closer to full takeover - august 7, 2015 related posts what’s in store for bigdata analytics in 2016 january 14, 2016 by stevesarsfield nosql and the internet of things january 14, 2016 by kingmesal 3 ingenious use cases for business intelligence tools january 11, 2016 by eran levy how iot applications can help businesses grow internally january 7, 2016 by xanderscho business intelligence: how to make your workplace perform smarter january 6, 2016 by ryank image the moderated business community for business intelligence, predictive analytics, and data professionals. image * twitter * iframe: http://platform.twitter.com/widgets/follow_button.html?screen_name= smartdataco * rss * rss as xml addtomyyahoo4 subscribe with bloglines add to google * email * get our best posts in your inbox weekly first name ____________________ last name ____________________ email address ____________________ sign up! how do you innovate effectively and maintain a competive edge? image learn how in our exlcusive ebook, "bad data need not apply: designing the modern data warehouse environment." download here. sdc special columns big-data-guru riksy business big-data-mops events no events listed add your own! recommended to follow ahamed parpia follow a t follow jan henderyckx follow brent leary follow orlando doehring follow daniel kraft follow comments * featured * popular * recent “the final aim of mcc is to allow implementation of rich smartphone web apps on an excess of cell phone devices, with a gorgeous user experience.” january 7th, 2016 by jamesmartin057 “by including more factors to score a person’s credit worthiness would make it even tougher for borrowers to obtain a loan approval. this in turn causes the finance industry to face a fluctuating business trend. there are definitely pros and cons to this major change in a loan application process but one thing for sure is to ensure borrowers shall only be allowed to take up a loan which is ...” january 7th, 2016 by webbrowan “thanks for the insight connie!” january 4th, 2016 by daniel matthews “there are clearly a number of changes coming for small businesses as they await ...” january 6th, 2016 by nguyennga “very useful tips, especially from the point of view of customer service.” january 4th, 2016 by isabella1977 “this is such a scary article. no doubt there is a lot of things that you have ...” january 6th, 2016 by webbrowan “the final aim of mcc is to allow implementation of rich smartphone web apps on ...” january 7th, 2016 by jamesmartin057 “regardless of the nature of any business, finance is the key factor that helps ...” january 7th, 2016 by webbrowan “such comparison tools are great for borrowers to perform an estimated ...” january 7th, 2016 by webbrowan “when it concerns money, you cannot afford to commit even a single mistake and ...” january 6th, 2016 by webbrowan “due to modernization we have found good technology advanced in the society; ...” january 7th, 2016 by edwardmiller “social collaboration seems like a funny way to describe the process of speaking ...” january 6th, 2016 by webbrowan show more “the final aim of mcc is to allow implementation of rich smartphone web apps on an excess of cell phone devices, with a gorgeous user experience.” january 7th, 2016 by jamesmartin057 “by including more factors to score a person’s credit worthiness would make it ...” january 7th, 2016 by webbrowan “due to modernization we have found good technology advanced in the society; ...” january 7th, 2016 by edwardmiller “regardless of the nature of any business, finance is the key factor that helps ...” january 7th, 2016 by webbrowan “such comparison tools are great for borrowers to perform an estimated ...” january 7th, 2016 by webbrowan “there are clearly a number of changes coming for small businesses as they await ...” january 6th, 2016 by nguyennga “when it concerns money, you cannot afford to commit even a single mistake and ...” january 6th, 2016 by webbrowan “social collaboration seems like a funny way to describe the process of speaking ...” january 6th, 2016 by webbrowan “this is such a scary article. no doubt there is a lot of things that you have ...” january 6th, 2016 by webbrowan “thanks for the insight connie!” january 4th, 2016 by daniel matthews show more popular * read * commented * authors h z - points john staunton - points jan rubio - points sarma ranga - points dawn bronkema - points in memoriam: robin fray carey (1) 1/6/2016 by sdc staff 72% of people aren’t familiar with hosted voip (0) 1/4/2016 by josh rose how to recycle old it equipment and keep your data safe (0) 1/4/2016 by ryan kidman 5 predictions for trends in data, analytics and machine learning in 2016 (0) 1/4/2016 by bruce robbins will 2016 be the year you clean up your dirty data? (0) 1/5/2016 by martin doyle 3 ingenious use cases for business intelligence tools 1/11/2016 by eran levy 3 perks of an office 365 migration that you didn’t know about 1/12/2016 by kyle cebull nosql databases: 4 game-changing use cases 1/13/2016 by jim scott flash vulnerabilities & exploits: an information security primer 1/12/2016 by thu pham nosql and the internet of things 1/14/2016 by jim scott advisory panel about the panel aaron aders more » bruno aziza more » stephen baker more » paul barsch more » gary cokins more » jill dyché more » themos kalafatis more » bernard marr more » james taylor more » smartdata collective * audio archive: the cutting edge of bigdata monetization * audio archive: delivering value through business analytics * audio archive: trendspotting for growth * audio archive: finding value in unstructured social data * audio archive: risks and rewards of bigdata in the cloud * audio archive: putting customer value to work categories » analytics » best practices » bigdata » book review » business intelligence » business rules » cloud computing » collaborative data » commentary » crm » culture/leadership » data management » data mining » data quality » data visualization » data warehousing » decision management » ebook » hadoop » hardware » inside companies » it » jobs » knowledge management » location » mapreduce » market research » marketing » marketing automation » mobility » modeling » new products » news » open source » podcasts » policy and governance » predictive analytics » privacy » r programming language » risk management » security » sentiment analytics » smartdata collective exclusive » social data » social media analytics » software » sql » statistics » text analytics » transparency » unstructured data » web analytics » webcasts » workforce analytics » workforce data show more featured contributors * you * melanie aizer * paul barsch * meta s. brown * jason burke * gary cokins * ted cuzzillo * barry devlin * chris dixon * erica driver * michael fauscette * bill franks * david hecht * julie hunt * lachlan james * doug lautzenheiser * alex olesker * david smith * james taylor * daniel tunkelang * bigdata smart data cloud social data unstructured data data in the cloud business intelligence analytics social media today llc © 2012 terms of use | community guidelines | privacy policy | get a badge | help | about & contact quantcast #tech times rss feeds tech times * personal tech * biz tech * future tech * science * life * t-lounge * tech tv facebook tweeter google rss ____________________ close * personal tech * biz tech * future tech * science * life * t-lounge * tech tv * topics * smartphone * tablets * computers * wearable tech * video games * software&apps * internet * gadgets * reviews * what's hot * ces 2016 * apple * virtual reality * samsung * microsoft * topics * deals * money * legal * start ups * business * what's hot * ces 2016 * apple * virtual reality * samsung * microsoft * topics * auto * space/aviation * military * robots * energy * smart home * what's hot * ces 2016 * apple * virtual reality * samsung * microsoft * topics * earth * space * environment * energy * what's hot * ces 2016 * apple * virtual reality * samsung * microsoft * topics * health * society * what's hot * ces 2016 * apple * virtual reality * samsung * microsoft tag ibm, xilinx, open power, bigdata, machine learning ibm and xilinx boost power chips to further expand into bigdata and machine learning by vincent lanaria, tech times | november 17, 12:02 am like follow share tweet reddit subscribe ibm xilinx partnership ibm and xilinx announced a partnership, and the two companies are working on integrating xilinx fpgas with power systems. the pair intends to accelerate workloads such as bigdata analytics and machine learning, to name a few. (photo : xilinx) ibm and chip maker xilinx team up to bring ibm's power processors to more servers, looking to garner a bigger portion of the data center market than intel. last year, the big blue initiated its open power program in an effort to boost power by licensing the design for use by other companies. now, with this partnership, ibm will be able to take things up a notch. the pair will work with everything, including infrastructure, middleware and software. they'll be focusing on integrating xilinx field-programmable gate arrays (fpga) with power-based systems to allow faster processing rates to accelerate a wide variety of workloads, such as machine learning, bigdata analytics, genomics, network-functions virtualization and high-performance computing. "the combination of ibm and xilinx provides our clients not only with a new level of accelerated computing made possible by the tight integration between ibm power processors and xilinx fpgas, but also gives them the ability to benefit directly from the constant stream of innovation being delivered by the rapidly expanding openpower ecosystem," ken king, general manager of open power, says in a press release. on top of that, ibm intends to both develop and qualify xilinx accelerator boards into ibm power systems servers. on the other hand, xilinx plans to roll out power variants of its leading software named sdaccel development environment, including libraries for the community of open power developers. in a bid to provide a faster computing performance, the two companies will persist in working on ibm's coherent accelerator processor inferface (capi), which delivers accelerate compute, accelerate storage and accelerate networking. seeing that xilinx chips will be compatible with capi, they will be able to access memory caches on power processors, which will considerably increase performance. ibm used to be the sole maker of power servers, mainly using only ibm technologies. now, big blue has opened up, allowing third parties to build power products, which is probably because the unix market is in a downturn. ibm has collaborated with nvidia before to integrate gpus with the watson supercomputer, which is a power system. also, both e4 computer engineering and penguin computing are making their own high-performance computers with power processors. like follow share tweet reddit subscribe tech times subscribeclose [popup-subscribe-img.jpg] enjoyed reading this story? for more awesome news, reviews, features and analyses, subscribe to tech times newsletter. we value your privacy and we will never sell or distribute your email or personal data to third party advertisers. ____________________ [button input] (not implemented)_________ related articles ibm's watson and the boston children's hospital are teaming up to cure rare diseases ibm and apple deepen partnership with watson-based apple watch app, free or discounted apple watches for ibm employees ibm buys the weather company’s digital assets to have watson host forecast data in 'internet of things' expansion ibm nears acquisition of weather co. digital assets in deal that could go over $2b ibm's q3 net profit falls 14 percent: 14th quarter in a row of declining revenue © 2016 tech times, all rights reserved. do not reproduce without permission. please enable javascript to view the comments powered by disqus. don't miss business tsmc pilot line ready in june for apple a10 chips production: report business jony ive's promotion as apple's chief design officer triggers speculations business tivo reportedly planning to launch cox on demand services trending * facebook might allow businesses to chat with consumers via whatsapp * jony ive's promotion as apple's chief design officer triggers speculations * samsung announces merger of cheil and c&t prior to transition of power in founding family * amazon prime now service offers one-hour delivery from local stores recommend close iframe: http://www.techtimes.com/home/plugin/banner/videos/video_player.htm?cat egory=an0n2 [bn-ces2016.jpg] most popular most commented must read 15 january '16 it's been a while but buick becomes cool again with avista sports coupe concept 12 january '16 samsung galaxy s6, s6 edge getting android 6.0.1 marshmallow beta as galaxy note 4 finally gets android 5.1.1 lollipop 12 january '16 how to install and set up remix os to run android on your pc 14 january '16 spider starburst crops up on mars: nasa explains what these strange markings are 14 january '16 these underrated drones could give the dji phantom a run for its money * * * * * * email newsletter * ____________________ [button input] (not implemented)______ 14 january '16 huawei honor 5x is coming to the us: here’s what early reviewers have to say 14 january '16 green pea galaxy gives scientists clues about the early evolution of the universe 15 january '16 microsoft cuts support for windows 8: what this means for you 13 january '16 google forms virtual reality division and this is what it plans to do 11 january '16 mysterious snail-like object spotted on high-res photo of pluto: what could it be? 11 january '16 mountain lion discovered with a full set of teeth and whiskers on top of its head 12 january '16 samsung galaxy s6, s6 edge getting android 6.0.1 marshmallow beta as galaxy note 4 finally gets android 5.1.1 lollipop 11 january '16 people obviously aren't doing the math on this facebook meme about the $1.3 billion powerball jackpot 10 january '16 the ehang 184 is a human-sized autonomous drone that can fly you around without a pilot, but don’t get excited yet 11 january '16 mysterious snail-like object spotted on high-res photo of pluto: what could it be? * * * * * * email newsletter * ____________________ [button input] (not implemented)______ 11 january '16 peach is the cool new messaging app everyone’s talking about: here’s how to use it 10 january '16 mark zuckerberg shares vaccination photo of daughter max on facebook: top reasons to get babies vaccinated 10 january '16 dimming quasar: this is what happens when a black hole has eaten its fill 9 january '16 kylo ren is part of the resistance and rey is obi-wan's granddaughter: the 'star wars' fan theories keep coming! 9 january '16 this is the perfect way to slice pizza, according to mathematicians 9 january '16 galactic archaeology: scientists create age map of the milky way 10 january '16 rosetta mission update: esa racing time to wake up dead comet lander philae 8 january '16 iphone 6c prototype leaks with 4-inch display and iphone 6s metal case 12 january '16 how to install and set up remix os to run android on your pc 8 january '16 new geological epoch anthropocene marked by impact of human activity 12 january '16 6 true crime documentaries to stream on netflix after watching 'making a murderer' 13 january '16 google forms virtual reality division and this is what it plans to do 14 january '16 world's largest canyon hides beneath antarctic ice sheet 9 january '16 vietnam officials seize mysterious metal balls that fell from space 8 january '16 ‘destiny’ players get strange coins, motes of light and exclusive emblem as new year’s gift from bungie 10 january '16 mark zuckerberg shares vaccination photo of daughter max on facebook: top reasons to get babies vaccinated 11 january '16 mountain lion discovered with a full set of teeth and whiskers on top of its head 9 january '16 the ehang 184 is a human-sized autonomous drone that can fly you around without a pilot, but don’t get excited yet 12 january '16 marijuana effect on the brain: pot may cause schizophrenia-like changes in teenagers 11 january '16 oculus ceo on playstation vr: 'not as high-end as ours' * * * * * * email newsletter * ____________________ [button input] (not implemented)______ 23 november '15 americans do not think global warming is real: why? 4 september '15 bret hart rips hulk hogan, says he's glad that ex-wrestler got caught in racial tirade 16 november '15 microsoft explains why ‘grand theft auto,’ ‘red dead redemption’ and ‘skyrim’ are not backward compatible… yet 11 january '16 mark zuckerberg sparks debate over vaccination photo of baby max: why the vaccination issue persists 12 january '16 nintendo's virtual boy failure was a 'real shame' that hurt the vr industry, says oculus' palmer luckey 12 january '16 samsung galaxy s6, s6 edge getting android 6.0.1 marshmallow beta as galaxy note 4 finally gets android 5.1.1 lollipop 9 january '16 this is the perfect way to slice pizza, according to mathematicians 10 january '16 dimming quasar: this is what happens when a black hole has eaten its fill 14 january '16 ge relocating to boston, expects technology-fluent workforce and generous incentive package 10 january '16 nasa unveils new asteroid detection program: what happens if an asteroid hits earth? 11 january '16 mark zuckerberg's vaccination post triggers debate between anti-vaxxers and vaccination supporters 13 january '16 microsoft teases highly requested xbox one party feature in new video: here’s what it’s all about 5 may '15 more lg g3, g2 android 5.0 lollipop problems reported by users: battery drain, slow charging, wi-fi issues continue 13 january '16 google forms virtual reality division and this is what it plans to do 11 january '16 recent powerball lottery meme : t-lounge : tech times t-lounge latest trailer for 'the division' shows off new rpg gameplay 6 true crime documentaries to stream on netflix after watching 'making a murderer' playstation essentials sale includes 'mass effect' trilogy, 'dragon age: inquisition' and 'persona 4 golden' alan rickman is the absolute best part of 'galaxy quest' people obviously aren't doing the math on this facebook meme about the $1.3 billion powerball jackpot new 'deadpool' emoji billboard ad is every bit as juvenile as you would expect correction: oculus ceo on playstation vr: 'isn't quite as high-end as ours' does this spider-man hoodie reveal spidey's 'captain america: civil war' look? kylo ren is part of the resistance and rey is obi-wan's granddaughter: the 'star wars' fan theories keep coming! these category codes will unlock hidden genres in netflix read more tech times * about us | terms & conditions | privacy policy | contact us | media kit * copyright ⓒ 2016 techtimes.com all rights reserved * subscribe to tech times * ____________________ [button input] (not implemented)_______ go backgo homefull site real time analytics clicky counter quantcast [p?c1=2&c2=19747886&cv=2.0&cj=1] * * * * ____________________ submitsearch cbr - computer business review register for free updates * verticals + finance + public sector + b2c + tmt + small business + other verticals inc web/manu/bio/trans + cio agenda + the boardroom * internet of things + consumer + wearables + m2m + smart technology * bigdata + analytics + software + hardware + platforms * mobility + apps + devices + networks + security * cloud + aas + hybrid + private + public * new style of business + converged + software defined + hybrid delivered * data centre + web scale + enterprise + commercial + infrastructure * enterprise it + server + storage + it network + it services + software * telecoms + carrier + connectivity + network + uc * cybersecurity + data + physical + business * social media + infrastructure + management * c-level resources + cbr digital magazines + cbr newsletters + cbr special reports + whitepapers + blogs + cbrtv + suppliers + companies + cbr dining club news blog sections * verticals * internet of things * bigdata * mobility * cloud * data centre * enterprise it * telecoms * cybersecurity * social media * new style of business bigdata/analytics bigdata in pole position on f1 starting grid analytics joao lima 10:39, november 18 2015 share on facebook more sharing services image analysis: formula one veteran mark gallagher and grand prix winner david coulthard share their views. winning a formula one race is also no longer about the best driver and the fastest car, it is about making the best use of data. at liveworx europe 2015, in stuttgart, germany, mark gallagher, md of cms motor sport and 30 years formula one veteran, said: "you want really robust reliable technology to deliver a safe product. and then of course we want to have great performance." for the last 20 years, formula one has been working on technology and trying to utilise new technological innovations to get the most out of driver and car. "there is so much about bigdata, iot, and other things that we can do with in this new world. it is pretty extraordinary what you can do when using this technology." by using data, f1 has been able to build safer vehicles and reduce the risk posed to drivers. this led to no deaths for a period of ten years between 2004 and 2014, with gallagher saying: "data improves safety. we now produce much safer products and that is helping to reduce fatalities." as formula one businesses integrate data environments to reach outcomes, real time data is today the big discussion within the industry. however, gallagher has found a problem in the way work is carried out. he said: "there is a problem, they [formula one] have spent millions of dollars building these systems out, but then they put it on human hands." to further discuss the role of data in this sport, gallagher was joined on stage by formula one legend david coulthard, winner of 13 grand prix. the scottish driver said that he did not see himself as a problem, but agreed that it is all about the data. he said: "formula one creates a wonderful piece of technology and then it delivers it to me to drive it as fast as i can. i do not think i am part of the problem. "the data does not lie. it is part of being a human to cover up, data does not do that, it tells us the fact of performance. "you have to work as a team. the team is developing a low volume production, might just be five cars a year. the ultimate goal in formula one is to win in the construction field." he said that different teams share data between themselves and that modern teams do a lot to make sure they are completely transparent with what they do with the car. "you need to be completely open. briefings [at the circuit] happen in front of competitors; about 15 engineers share information between themselves." with the use of data - and the exchange of data - racing teams have managed to improve circuit assistance performance. gallagher said: "continuous improvement post 2010 led to increased efficiency even more." he exemplified how much data can impact a driver's performance in a race using ferrari. the company, through the use of data, has reduced the standard six seconds spent around the car during a race to only three seconds. red bull racing has also utilised data, and following ferrari's steps, has reduced assistance time to less than a second and a half. "the car comes in, they are in position and change the tyre without the tyre even stop spinning." just like the car revolution that is about to hit roads worldwide, racing will also see automated cars drive in into the sport. when? "it is getting closer," said gallagher. get a cbr bigdata weekly update ____________________ ____________________ ____________________ ____________________ [industry_____________________] [united kingdom________________] [button input] (not implemented)________ registered successfully terms & conditions & privacy policy. * share submit to reddit twitter facebook linked in * share: * delicious * digg * submit to reddit * slashdot * buzz * facebook * twitter * linked in * viadeoshare latest news * post-3 how barclays is cashing in on bigdata & hadoop to stay ahead in fintech * post-3 are google x moon shots aiming disruptive technology at healthcare? * post-3 compliance with data regulation: how bigdata analytics vendors are tackling data protection * post-3 7 considerations for data consolidation ahead of your bigdata programmes comments post a comment comments may be moderated for spam, obscenities or defamation. please enable javascript to view the comment. get cbr's daily morning update ____________________ ____________________ ____________________ ____________________ [united kingdom________________] [select industry______________] [button input] (not implemented)________ registered successfully terms & conditions & privacy policy. trending on cbr * most read * most commented * from “slightly crazy” to key saas software provider, salesforce appexchange celebrates 10th anniversary * how barclays is cashing in on bigdata & hadoop to stay ahead in fintech * barclays and tesco join innovate uk in future retail contest * google claims cloud price victory in war with amazon cloud * devops – the need for speed * good ceo on blackberry: bringing together two sides of the same coin cbrtv * latest videos * most watched * [49921910001_4684679871001_th-568a6f6ce4b093fd8913033b-767904719001 .jpg?pubid=49921910001] how to make buying a professional services saas solution work for you * [49921910001_4656604882001_4656595613001-th.jpg?pubid=49921910001] from where are the new cybersecurity threats coming and how can they be prevented? * [49921910001_4642122116001_4642019355001-th.jpg?pubid=49921910001] beware the consequences of failing to embrace digital transformation * [49921910001_4441193750001_4441181301001-th.jpg?pubid=49921910001] enabling workforce productivity * [49921910001_4380616245001_4380288145001-th.jpg?pubid=49921910001] practical approaches to software defined infrastructure * [49921910001_4121169312001_4121096819001-th.jpg?pubid=49921910001] where is software defined network taking the industry? * [49921910001_4148749566001_3836445112001.jpg?pubid=49921910001] the shifting cyber security landscape * [49921910001_2617678242001_thumbnail-for-video-2592062834001.jpg?pu bid=49921910001] cbr meets recommind's co-founder * [49921910001_2968180118001_th-52b95d74e4b05e49470da69e-672293882001 .jpg?pubid=49921910001] is data a strategic asset? * [49921910001_2726070945001_th-5252ae00e4b046f6b50c97db-672293882001 .jpg?pubid=49921910001] from mainframe to mobile with bmc whitepapers * extreme savings: cutting costs with wide-area data services organizations of all sizes strive to be more productive and run low cost operations. particularly in difficult economic times, many organizations... * getting value from a single customer view? regulators are asking for it; will the industry exploit it? suppliers directory * spotlight * by technology * a-z * bcs - enabling the information society established in 1957, bcs, the chartered institute for it, promotes wider social and economic progress through the advancement of information... * dataflux - data management technology dataflux enables business agility and it efficiency by providing innovative data management technology and services that transform data into a... * windward – reporting, document generation and business intelligence windward offers advanced data software solutions for business intelligence, enterprise reporting, document generation, mail merge and web-based... verticals finance public sector b2c tmt small business other verticals inc web/manu/bio/trans cio agenda the boardroom internet of things consumer wearables m2m smart technology bigdata analytics software hardware platforms mobility apps devices networks security cloud aas hybrid private public data centre web scale enterprise commercial infrastructure enterprise it server storage it network it services software telecoms carrier connectivity network uc cybersecurity data physical business social media infrastructure management # abcdefghijklmnopqrstuvwxyz * computer business review * about cbr * about us * accessibility * advertise with us * contact us * help * privacy * rss feeds * site map * resources * verticals * internet of things * bigdata * mobility * cloud * data centre * enterprise it * telecoms * cybersecurity * social media * cbr dining club * cbr tv * companies * special reports * white papers * extra content * blogs * * join our network * suppliers * digital and print magazine * business review sites * automotive * banking * clean technology * drinks * energy * food * insurance * government computing * logistics * medical devices * packaging * pharmaceutical * retail * webinars * industry reports * © cbr 2016 | part of progressive digital media group plc. * home * privacy * contact us * site map dcsimg #rdf metadata [p?c1=2&c2=3005435&cv=2.0&cj=1] executive insights & innovation security & risk strategy team building & staffing it strategy digital business project management programming languages dr. dobb's enterprise applications operating systems productivity/collaboration apps careers & people threat intelligence iot attacks & breaches application security cloud security endpoint security mobile security perimeter security risk management operations security analytics vulnerabilities & threats security & risk strategy infrastructure as a service platform as a service software as a service cloud storage data centers mobile applications mobile devices mobile business enterprise mobility management bigdata analytics hardware/architectures software platforms networking storage wireless infrastructure data center unified communications cloud infrastructure applications careers pcs and servers net security government healthcare wall street & technology bank systems & technology insurance & technology advertise with us about us contact us digital subscription [iwk_mobile_user_nav.png] [spacer.gif] [iwk_mobile_search_icon.png] [spacer.gif] [iwk_mobile_sm.png] [iwk-sections-nav.png] [mobile_close.jpg] * home * news & commentary * authors * slideshows * video * reports * white papers * events * university * interop * strategic cio * iot * devops * software * security * cloud * mobile * bigdata * infrastructure * industries * it life [mobile_close.jpg] * about us * contact us * digital subscription * advertise with us [mobile_close.jpg] ____________________ [spacer.gif] [mobile_close.jpg] * facebook * twitter * linkedin * google+ * rss informationweek: connecting the business technology community ____________________ [spacer.gif] follow iw: [spacer.gif] [spacer.gif] [spacer.gif] [spacer.gif] [spacer.gif] [spacer.gif] [spacer.gif] [spacer.gif] [spacer.gif] home news & commentary authors video reports white papers events university elite100 interop strategic cio iot devops software security cloud mobile bigdata infrastructure industries it life bigdata // bigdata analytics news 11/15/2015 10:06 am jessica davis jessica davis news connect directly twitter rss e-mail [comment.png] 0 comments comment now login [thumbs-up.png] 50% [thumbs-down.png] 50% tweet ibm datapalooza, oracle updates bi platform: bigdata roundup ibm continued with its datapalooza roadshow, oracle released a new version of its bi software with enhanced visualization capabilities, and informatica rolled out its bigdata management platform. all that and more in our bigdata roundup for the week of nov. 15. 6 ways bigdata is driving personalized medicine revolution 6 ways bigdata is driving personalized medicine revolution (click image for larger view and slideshow.) bigdata is expected to make a big impact in healthcare and personalized medicine. but where are the real projects in this field and what progress is being made? this week, informationweek took a deeper look at bigdata projects for personalized medicine. plus, we have news on google open sourcing its machine learning library, tensorflow. we've also got news from informatica, ibm, oracle, and more. let's start with our collection of bigdata personalized medicine projects. this week, informationweek pulled together some of the top personalized medicine projects utilizing bigdata. it and bigdata are helping drug trials search for better treatments for conditions from arthritis to cancer. take a look at these ground-breaking projects here. (image: ponywang/istockphoto) (image: ponywang/istockphoto) google this week open sourced its machine-learning library, tensorflow under the apache 2.0 license. the company said that this software can turn machine-learning algorithms that have been written as graphs of symbolic impressions into efficient low-level code. the software is not tied to google infrastructure and can be run on other systems, too. see the full story here. informatica used a virtual event this week to roll out its bigdata management platform, designed to help users easily connect all front-end analytics software to all types of back-end data infrastructure. the company said the platform combines integration, governance, and security in a single package to make it easy for organizations to utilize the data they have. take a look at the full story here. meanwhile, ibm continued on a mission it declared in june to create a million new data scientists. the company added about 230 during its datapalooza educational event this week in san francisco, where prospective data scientists got to work with ibm's newly launched data science workbench and bluemix cloud services to build their first analytics apps. ibm will take its show on the road next year to a dozen cities around the world, including berlin, prague, and tokyo. read the full story here. in addition to personalized medicine, the topic of the internet of things (iot) comes to mind when bigdata is discussed. it research firm gartner this week released a forecast of the iot market that said 6.4 billion connected objects, including consumer devices, will be in use by 2016 -- a 30% rise from this year. where will that figure be by 2020? gartner said we will hit 20.8 billion by then. overall, this iot market will support total services spending of $235 billion in 2016, up 22% from this year. get the full story and forecast here. back in enterprise it land, many cios are grappling with the big issue of how to make their backend systems and customer-facing systems talk to each other with zero latency, and how to deploy them in an agile way. well, american express global business travel had the funding and gumption to say, "to heck with the update, let's rip it all out and start with cloud-based systems!" here's the story of how the company did it, what it chose, and how long it took. everyone knows that visualization is one of the keys to making business intelligence meaningful to users. with that in mind, this week oracle announced general availability of its oracle business intelligence 12c, which includes a new visual analysis component for data. oracle said that the update allows people across an entire organization to leverage a single integrated platform for self-service and visual data discovery. more on oracle's new release is here. [find out what intel is doing with internet of things. read intel primes the internet of things pump.] finally, idc this week released a new bigdata forecast. the company is forecasting a compound annual growth rate for the market of 23% over the 2014 to 2019 forecast period, with annual spend reaching $48.6 billion in 2019. the market analyst firm said it expects all three bigdata submarkets -- infrastructure, software, and services -- to grow over the next five years. infrastructure will grow at a 21.75% cagr (compound annual growth rate). software, including analytics, information management, and applications software, will grow at a cagr of 26.2%. services, including professional support and support services for infrastructure and software will grow at a cagr of 22.7%. as bigdata matures, idc expects its share of the larger business analytics market to increase. however, the year-over-year growth within the bigdata market is forecast to gradually slow, idc said, caused by increased price pressures for infrastructure and higher rates of commercialization of open source software. the availability and skill level of bigdata it and analytics talent will also have a direct impact on the market, idc said. here's a full version of the idc statement about the market forecast. jessica davis has spent a career covering the intersection of business and technology at titles including idg's infoworld, ziff davis enterprise's eweek and channel insider, and penton technology's mspmentor. she's passionate about the practical use of business intelligence, ... view full bio comment | email this | print | rss more insights webcasts is your business prepared? get the business view - a dark reading virtual event best practices for resilient inline security deployments more webcasts white papers how netflix is taking capacity & tco to new heights [case study] how nestle used video for internal communications more white papers reports [informationweek & dark reading report] 2015 strategic security survey results 10 recommendations for outsourcing security more reports [spacer.gif] comments newest first | oldest first | threaded view [close this box] [spacer.gif] be the first to post a comment regarding this story. [spacer.gif] reports [spacer.gif] [spacer.gif] infographics [spacer.gif] 6 tools to protect bigdata 6 tools to protect bigdata most it teams have their conventional databases covered in terms of security and business continuity. but as we enter the era of bigdata, hadoop, and nosql, protection schemes need to evolve. in fact, bigdata could drive the next big security strategy shift. download now! [spacer.gif] more reports bigdata brings big security problems bigdata brings big security problems why should bigdata be more difficult to secure? in a word, variety. but the business wont wait to use it to predict customer behavior, find correlations across disparate data sources, predict fraud or financial risk, and more. [comment.png] 1 comments | read | post a comment [spacer.gif] 2014 state of database [comment.png] 0 comments [spacer.gif] in a fever for bigdata [comment.png] 0 comments [spacer.gif] more infographics register for informationweek newsletters subscribe to newsletters [spacer.gif] live events [spacer.gif] [spacer.gif] webinars [spacer.gif] [ubm-tech.png] more ubm tech live events learn newest developments in video at enterprise connect see adam swidler of google for work at enterprise connect interop las vegas applications track is your business prepared? get the business view - a dark reading virtual event transforming customer business apps and vdi desktops with flash-enabled vsan ready nodes how technology is transforming manufacturing [spacer.gif] webinar archives white papers how netflix is taking capacity & tco to new heights [case study] how nestle used video for internal communications [checklist] moving to the cloud combating the insider threat [cyber security] how well does your threat intelligence feed rank? [spacer.gif] more white papers current issue [screenshot2015-12-07at122615pm.jpg] how to knock down barriers to effective risk management risk management today is a hodgepodge of systems, siloed approaches, and poor data collection practices. that isn't how it should be. download this issue! subscribe now! [spacer.gif] back issues | must reads video [video-arrow_left_off.gif] [1568176135_4489522385001_thumbnail-for-video-4234403102001.jpg?pubid=1 568176135] informationweek presents 'get to the point' [comment.png] 1 comments [1568176135_4232614152001_4232562810001-th.jpg?pubid=1568176135] heartland ceo: dealing honestly with data ... [comment.png] 7 comments [1568176135_4234072761001_4234047388001-th.jpg?pubid=1568176135] where data science meets it [comment.png] 4 comments [1568176135_4234440318001_4234365405001-th.jpg?pubid=1568176135] modern data centers: how we'll get there [comment.png] 1 comments [1568176135_4232444203001_4232432578001-th.jpg?pubid=1568176135] enterprise it: act like a software startup [comment.png] 1 comments [1568176135_4234091599001_4234071225001-th.jpg?pubid=1568176135] how astrazeneca's cio embraced insourcing [comment.png] 0 comments [1568176135_4386836911001_thumbnail-for-video-4386951387001.jpg?pubid=1 568176135] windows 10: inside nyc launch day [comment.png] 10 comments [1568176135_4232897123001_4232839543001-th.jpg?pubid=1568176135] conocophillips drills down into analytics [comment.png] 1 comments [1568176135_4232971722001_4232949108001-th.jpg?pubid=1568176135] royal caribbean keeps customers connected ... [comment.png] 1 comments [1568176135_4232964260001_4232945668001-th.jpg?pubid=1568176135] the age of biotechnology has arrived [comment.png] 1 comments [1568176135_4232958763001_4232945664001-th.jpg?pubid=1568176135] chris anderson: making drones and ... [comment.png] 2 comments [1568176135_4313612547001_thumbnail-for-video-4313097826001.jpg?pubid=1 568176135] nba cio focuses on analytics, data and fans [comment.png] 10 comments [video-arrow_right_on.gif] [spacer.gif] all videos [camera-24x16.png] slideshows iphone 7 rumors: 7 that have people talking iphone 7 rumors: 7 that have people talking [comment.png] 0 comments | read | post a comment [spacer.gif] cios detail hiring plans for first half of 2016 [comment.png] 0 comments [spacer.gif] 10 medical-device wearables to improve patients' lives [comment.png] 5 [spacer.gif] more slideshows twitter feed tweets about "from:informationweek or #informationweek or @informationweek" informationweek radio archived informationweek radio informationweek live for the week of december 13, 2015 join us for a roundup of the top stories on informationweek.com for the week of december 13, 2015. [spacer.gif] upcoming! friday, january 15, 3pm est informationweek live for the week of january 10, 2016 [spacer.gif] full schedule | archived shows [spacer.gif] sponsored live streaming video everything you've been told about mobility is wrong attend this video symposium with sean wisdom, global director of mobility solutions, and learn about how you can harness powerful new products to mobilize your business potential. [comment.png] 465 comments | read | post a comment [spacer.gif] full schedule | archived shows information week about us contact us customer support reprints editorial calendar twitter facebook linkedin google+ rss ubm tech * ubm tech brands * black hat * cloud connect * dark reading * enterprise connect * fusion * gdc * gtec * gamasutra * hdi * informationweek * interop * network computing * no jitter * tower & small cell summit * communities served * enterprise it * enterprise communications * game development * information security * it services & support * working with us * advertising contacts * event calendar * tech marketing * solutions * contact us * licensing * terms of service * privacy statement * copyright © 2016 ubm tech, all rights reserved [client_pathlog.asp?p=%2finformationweek%2fsection%2f277&f=%2finformati onweek%2fsection%2f277%2f1323150&rndserial=19366] to rate this item, click on a rating below. [lightreading_rating_dot_10x7.gif] [lightreading_rating_dot_10x7.gif] [lightreading_rating_dot_10x7.gif] [lightreading_rating_dot_10x7.gif] [lightreading_rating_dot_10x7.gif] [lightreading_rating_dot_10x7.gif] [lightreading_rating_dot_10x7.gif] [lightreading_rating_dot_10x7.gif] [lightreading_rating_dot_10x7.gif] [lightreading_rating_dot_10x7.gif] [lightreading_rating_dot_10x7.gif] [lightreading_rating_dot_10x7.gif] [lightreading_rating_dot_10x7.gif] [lightreading_rating_dot_10x7.gif] [lightreading_rating_dot_10x7.gif] [close this box] * * to save this item to your list of favorite informationweek content so you can find it later in your profile page, click the "save it" button next to the item. * * if you found this interesting or useful, please use the links to the services below to share it with other readers. you will need a free account with each service to share an item via that service. * [twitter_intevol_18x18.gif] tweet this * [close this box] [spacer.gif] #gizmodo - all stories ____________________ log in / sign up * follow gizmodofollowing gizmodo * related blogs * [javascript] sploid * [javascript] paleofuture * [javascript] toyland * [javascript] gizmodo en español * [javascript] io9 * blogs you may like * [javascript] deadspin * [javascript] gawker * [javascript] gizmodo * [javascript] jalopnik * [javascript] jezebel * [javascript] kotaku * [javascript] lifehacker ____________________ followfollowing you can skip this ad in 5 seconds. the promise—and perils—of predictive policing based on bigdata h v jagadish 11/17/15 3:30am filed to: policing * crime * data * bigdata * forecasting * modelling * ai 333 * edit * invite manually * promote * dismissundismiss * block for gizmodo * hide * share to kinja * go to permalink the promise—and perils—of predictive policing based on bigdata police departments, like everyone else, would like to be more effective while spending less. given the tremendous attention to bigdata in recent years, and the value it has provided in fields ranging from astronomy to medicine, it should be no surprise that police departments are using data analysis to inform deployment of scarce resources. enter the era of what is called “predictive policing.” some form of predictive policing is likely now in force in a city near you. memphis was an early adopter. cities from minneapolis to miami have embraced predictive policing. time magazine named predictive policing (with particular reference to the city of santa cruz) one of the 50 best inventions of 2011. new york city police commissioner william bratton recently said that predictive policing is “the wave of the future.” the term “predictive policing” suggests that the police can anticipate a crime and be there to stop it before it happens and/or apprehend the culprits right away. as the los angeles times points out, it depends on “sophisticated computer analysis of information about previous crimes, to predict where and when crimes will occur.” advertisement at a very basic level, it’s easy for anyone to read a crime map and identify neighborhoods with higher crime rates. it’s also easy to recognize that burglars tend to target businesses at night, when they are unoccupied, and to target homes during the day, when residents are away at work. the challenge is to take a combination of dozens of such factors to determine where crimes are more likely to happen and who is more likely to commit them. predictive policing algorithms are getting increasingly good at such analysis. indeed, such was the premise of the movie minority report, in which the police can arrest and convict murderers before they commit their crime. predicting a crime with certainty is something that science fiction can have a field day with. but as a data scientist, i can assure you that in reality we can come nowhere close to certainty, even with advanced technology. to begin with, predictions can be only as good as the input data, and quite often these input data have errors. sponsored but even with perfect, error-free input data and unbiased processing, ultimately what the algorithms are determining are correlations. even if we have perfect knowledge of your troubled childhood, your socializing with gang members, your lack of steady employment, your wacko posts on social media and your recent gun purchases, all that the best algorithm can do is to say it is likely, but not certain, that you will commit a violent crime. after all, to believe such predictions as guaranteed is to deny free will. feed in data, get out probabilities what data can do is give us probabilities, rather than certainty. good data coupled with good analysis can give us very good estimates of probability. if you sum probabilities over many instances, you can usually get a robust estimate of the total. for example, data analysis can provide a probability that a particular house will be broken into on a particular day based on historical records for similar houses in that neighborhood on similar days. an insurance company may add this up over all days in a year to decide how much to charge for insuring that house. a police department may add up these probabilities across all houses in a neighborhood to estimate how likely it is that there will be a burglary in that neighborhood. they can then place more officers in neighborhoods with higher probabilities for crime with the idea that police presence may deter crime. this seems like a win all around: less crime and targeted use of police resources. indeed the statistics, in terms of reduced crime rates, support our intuitive expectations. likely doesn’t mean definitely similar arguments can be used in multiple arenas where we’re faced with limited resources. realistically, customs agents cannot thoroughly search every passenger and every bag. tax authorities cannot audit every tax return. so they target the “most likely” culprits. but likelihood is very far from certainty: all the authorities know is that the odds are higher. undoubtedly many innocent individuals are labeled “likely.” if you’re innocent but get targeted, it can be a big hassle, or worse. incorrectly targeted individuals may be inconvenienced by a customs search, but predictive policing can do real harm. consider the case of tyrone brown, recently reported in the new york times. he was specifically targeted for attention by the kansas city police because he was friends with known gang members. in other words, the algorithm picked him out as having a higher likelihood of committing a crime based on the company he kept. they told him he was being watched and would be dealt with severely if he slipped up. the algorithm didn’t “make a mistake” in picking out someone like tyrone brown. it may have correctly determined that tyrone was more likely to commit a murder than you or i. but that is very different from saying that he did (or will) kill someone. suppose there’s a one-in-a-million chance that a typical citizen will commit a murder, but there is a one-in-a-thousand chance that tyrone will. that makes him a thousand times as likely to commit a murder as a typical citizen. so it makes sense statistically for the police to focus their attention on him. but don’t forget that there is only a one-in-a-thousand chance that he commits a murder. for a thousand such “suspect” tyrones, there is only one who is a murderer and 999 who are innocent. how much are we willing to inconvenience or harm the 999 to stop the one? kansas city is far from being alone in this sort of preemptive contact with citizens identified as “likely to commit crimes.”last year, there was considerable controversy over a similar program in chicago. balancing crime reduction with civil rights such tactics, even if effective in reducing crime, raise civil liberty concerns. suppose you fit the profile of a bad driver and have accumulated points on your driving record. consider how you would feel if you had a patrol car follow you every time you got behind the wheel. even worse, it’s likely, even if you’re doing your best, that you will make an occasional mistake. for most of us, rolling through a stop sign or driving five miles above the speed limit is usually of little consequence. but since you have a cop following you, you get a ticket for every small offense. in consequence, you end up with an even worse driving record. yes, data can help make predictions, and these predictions can help police expend their resources smarter. but we must remember that a probabilistic prediction is not certainty, and explicitly consider the harm to innocent people when we take actions based on probabilities. more broadly speaking, data science can bring us many benefits, but care is required to make sure that it does so in a fair manner. the promise—and perils—of predictive policing based on bigdata h v jagadish is a bernard a galler collegiate professor of electrical engineering and computer science at the university of michigan. this article was originally published on the conversation. read the original article. image by jaromir chalabala/shutterstock iframe: https://counter.theconversation.edu.au/content/48366/count.gif reply33 replies leave a reply you may also like gawker · rich juzwiak steven averyʼs ex-fiancée says heʼs a guilty, abusive "monster" yesterday 5:33pm gawker · ashley feinberg donald trump has no idea how the government works yesterday 10:19pm the slot · madeleine davies this donald trump theme song is the most upsetting thing youʼll ever hear yesterday 3:40pm popular from gizmodo * 43 * 349 * 196.2k gizmodo · maddie stone a mysterious mammoth carcass could change human history yesterday 2:00pm * 9 * 140 * 29.2k io9 · germain lussier watch the mysterious new trailer that makes you think itʼs cloverfield 2 (updated) yesterday 11:20pm * 109 * 809 * 126.1k io9 · charlie jane anders 10 things that star trek got right (that have never been copied) yesterday 2:03pm 333 __________________________________________________________________ __________________________________________________________________ * * about blog * need help? * content guide * permissions * privacy * terms of use * advertising * jobs * rss [p?c1=2&c2=6770184&cv=2.0&cj=1] kinja is in read-only mode. we are working to restore service. [dfp_audience_pixel;dc_seg=22540930;blog=gizmodo;ord=1?] * * * * ____________________ submitsearch cbr - computer business review register for free updates * verticals + finance + public sector + b2c + tmt + small business + other verticals inc web/manu/bio/trans + cio agenda + the boardroom * internet of things + consumer + wearables + m2m + smart technology * bigdata + analytics + software + hardware + platforms * mobility + apps + devices + networks + security * cloud + aas + hybrid + private + public * new style of business + converged + software defined + hybrid delivered * data centre + web scale + enterprise + commercial + infrastructure * enterprise it + server + storage + it network + it services + software * telecoms + carrier + connectivity + network + uc * cybersecurity + data + physical + business * social media + infrastructure + management * c-level resources + cbr digital magazines + cbr newsletters + cbr special reports + whitepapers + blogs + cbrtv + suppliers + companies + cbr dining club news blog sections * verticals * internet of things * bigdata * mobility * cloud * data centre * enterprise it * telecoms * cybersecurity * social media * new style of business bigdata/software bigdata boom sparks record revenue for talend software eleanor burns 15:34, november 17 2015 share on facebook more sharing services news: ceo mike tuchen declares talend the 'juggernaut' of bigdata which 'no other company can match'. the exponential growth of data has certainly benefitted talend, with the company reporting record company revenues for q3. with little else reported from the company's q3, bar the 136% year-to-date rise in talend bigdata subscription bookings, the company ceo was keen to assert the company's bigdata dominance. "talend's bigdata business is a juggernaut that is experiencing rapid growth and is now driving 40 percent of our overall sales," said mike tuchen, ceo, talend. "our lead over the competition continues to widen as a result of our ability to execute and deliver significant customer value. no other company can match talend's real world bigdata experience or provide the performance, scale and real-time support that we offer today." the company did add that q3 also saw international expansion, with direct sales capabilities added in canada and australia. the company plans to broaden its presence in europe, initiating direct sales in the nordics and south east asia later this year and into early 2016. get a cbr bigdata weekly update ____________________ ____________________ ____________________ ____________________ [industry_____________________] [united kingdom________________] [button input] (not implemented)________ registered successfully terms & conditions & privacy policy. * share submit to reddit twitter facebook linked in * share: * delicious * digg * submit to reddit * slashdot * buzz * facebook * twitter * linked in * viadeoshare latest news * post-3 selling the cloud: oracle sales jobs hit emea * post-3 why hybrid it is turning fujitsu from systems integrator to saas provider * post-3 apple steps away from iad, gives ad control to publishers * post-3 from “slightly crazy” to key saas software provider, salesforce appexchange celebrates 10th anniversary comments post a comment comments may be moderated for spam, obscenities or defamation. please enable javascript to view the comment. get cbr's daily morning update ____________________ ____________________ ____________________ ____________________ [united kingdom________________] [select industry______________] [button input] (not implemented)________ registered successfully terms & conditions & privacy policy. trending on cbr * most read * most commented * from “slightly crazy” to key saas software provider, salesforce appexchange celebrates 10th anniversary * how barclays is cashing in on bigdata & hadoop to stay ahead in fintech * barclays and tesco join innovate uk in future retail contest * google claims cloud price victory in war with amazon cloud * devops – the need for speed * good ceo on blackberry: bringing together two sides of the same coin cbrtv * latest videos * most watched * [49921910001_4684679871001_th-568a6f6ce4b093fd8913033b-767904719001 .jpg?pubid=49921910001] how to make buying a professional services saas solution work for you * [49921910001_4656604882001_4656595613001-th.jpg?pubid=49921910001] from where are the new cybersecurity threats coming and how can they be prevented? * [49921910001_4642122116001_4642019355001-th.jpg?pubid=49921910001] beware the consequences of failing to embrace digital transformation * [49921910001_4441193750001_4441181301001-th.jpg?pubid=49921910001] enabling workforce productivity * [49921910001_4380616245001_4380288145001-th.jpg?pubid=49921910001] practical approaches to software defined infrastructure * [49921910001_4121169312001_4121096819001-th.jpg?pubid=49921910001] where is software defined network taking the industry? * [49921910001_4148749566001_3836445112001.jpg?pubid=49921910001] the shifting cyber security landscape * [49921910001_2617678242001_thumbnail-for-video-2592062834001.jpg?pu bid=49921910001] cbr meets recommind's co-founder * [49921910001_2968180118001_th-52b95d74e4b05e49470da69e-672293882001 .jpg?pubid=49921910001] is data a strategic asset? * [49921910001_2726070945001_th-5252ae00e4b046f6b50c97db-672293882001 .jpg?pubid=49921910001] from mainframe to mobile with bmc whitepapers * the human factor in laptop encryption: uk study encryption is one of the most important security tools in the defense of information assets. ponemon institute has conducted numerous studies on... * getting value from a single customer view? regulators are asking for it; will the industry exploit it? * sdl tridion r5 technical white paper this white paper describes how r5 architecture and technological choices help organisations achieve their business goals. this document is for... * addressing the public - delivering value with effective address & data management 'addressing the public' is the exciting new white paper from data management guru, graham rhind. the white paper explores effective data management... suppliers directory * spotlight * by technology * a-z * teneo - infrastructure management, performance and virtualization services teneo is an infrastructure optimization company, providing solutions that help customers with the management, performance and virtualization of... * capscan capscan is a leading supplier of international address management solutions and data integrity services. capscan has more than 1800 customers... * sdl tridion - web content management solutions sdl tridion is a global leader in web content management (wcm) solutions. verticals finance public sector b2c tmt small business other verticals inc web/manu/bio/trans cio agenda the boardroom internet of things consumer wearables m2m smart technology bigdata analytics software hardware platforms mobility apps devices networks security cloud aas hybrid private public data centre web scale enterprise commercial infrastructure enterprise it server storage it network it services software telecoms carrier connectivity network uc cybersecurity data physical business social media infrastructure management # abcdefghijklmnopqrstuvwxyz * computer business review * about cbr * about us * accessibility * advertise with us * contact us * help * privacy * rss feeds * site map * resources * verticals * internet of things * bigdata * mobility * cloud * data centre * enterprise it * telecoms * cybersecurity * social media * cbr dining club * cbr tv * companies * special reports * white papers * extra content * blogs * * join our network * suppliers * digital and print magazine * business review sites * automotive * banking * clean technology * drinks * energy * food * insurance * government computing * logistics * medical devices * packaging * pharmaceutical * retail * webinars * industry reports * © cbr 2016 | part of progressive digital media group plc. * home * privacy * contact us * site map dcsimg * * * * ____________________ submitsearch cbr - computer business review register for free updates * verticals + finance + public sector + b2c + tmt + small business + other verticals inc web/manu/bio/trans + cio agenda + the boardroom * internet of things + consumer + wearables + m2m + smart technology * bigdata + analytics + software + hardware + platforms * mobility + apps + devices + networks + security * cloud + aas + hybrid + private + public * new style of business + converged + software defined + hybrid delivered * data centre + web scale + enterprise + commercial + infrastructure * enterprise it + server + storage + it network + it services + software * telecoms + carrier + connectivity + network + uc * cybersecurity + data + physical + business * social media + infrastructure + management * c-level resources + cbr digital magazines + cbr newsletters + cbr special reports + whitepapers + blogs + cbrtv + suppliers + companies + cbr dining club news blog sections * verticals * internet of things * bigdata * mobility * cloud * data centre * enterprise it * telecoms * cybersecurity * social media * new style of business bigdata/analytics bigdata analytics fights uk superbugs analytics james nunns 10:37, november 18 2015 share on facebook more sharing services news: exasol research finds link between deprived areas and higher levels of antibiotic prescriptions. bigdata analytics are being put to good use in the fight against superbugs. research undertaken in partnership between exasol and antibiotic research uk sought to discover insights into five years of prescribing data in gp surgeries in england, which amounts to 83gb of uncompressed data or 602 million rows. using exasol's analytic database it was found that antibiotic prescriptions are coming down, however, gp practices in the most deprived parts of the country are prescribing 20% more antibiotic prescriptions. in clacton-on-sea, the uk's most deprived area; antibiotic prescribing rates are almost twice the national average. other findings saw london doctors prescribe 21% fewer antibiotics than the north and a gap in prescribing between the least and most deprived areas of england is 20%. prescriptions per head peaked in 2012 with 3.8 million, a number which has dropped by 5.6%. doctors prescribe 59% more in december than they do in august, despite illnesses treated by antibiotics not being seasonal. the data was visualised by combining the prescribing dataset with geographic data, it was then plotted on a map to create a choropleth heatmap over 32,000 geographic "lower level super output areas." sean jackson, cmo, exasol said: "the heatmap is so detailed that it is possible to single out individual surgeries overprescribing." get a cbr bigdata weekly update ____________________ ____________________ ____________________ ____________________ [industry_____________________] [united kingdom________________] [button input] (not implemented)________ registered successfully terms & conditions & privacy policy. * share submit to reddit twitter facebook linked in * share: * delicious * digg * submit to reddit * slashdot * buzz * facebook * twitter * linked in * viadeoshare latest news * post-3 how barclays is cashing in on bigdata & hadoop to stay ahead in fintech * post-3 are google x moon shots aiming disruptive technology at healthcare? * post-3 compliance with data regulation: how bigdata analytics vendors are tackling data protection * post-3 7 considerations for data consolidation ahead of your bigdata programmes comments post a comment comments may be moderated for spam, obscenities or defamation. please enable javascript to view the comment. get cbr's daily morning update ____________________ ____________________ ____________________ ____________________ [united kingdom________________] [select industry______________] [button input] (not implemented)________ registered successfully terms & conditions & privacy policy. trending on cbr * most read * most commented * from “slightly crazy” to key saas software provider, salesforce appexchange celebrates 10th anniversary * how barclays is cashing in on bigdata & hadoop to stay ahead in fintech * barclays and tesco join innovate uk in future retail contest * google claims cloud price victory in war with amazon cloud * devops – the need for speed * good ceo on blackberry: bringing together two sides of the same coin cbrtv * latest videos * most watched * [49921910001_4684679871001_th-568a6f6ce4b093fd8913033b-767904719001 .jpg?pubid=49921910001] how to make buying a professional services saas solution work for you * [49921910001_4656604882001_4656595613001-th.jpg?pubid=49921910001] from where are the new cybersecurity threats coming and how can they be prevented? * [49921910001_4642122116001_4642019355001-th.jpg?pubid=49921910001] beware the consequences of failing to embrace digital transformation * [49921910001_4441193750001_4441181301001-th.jpg?pubid=49921910001] enabling workforce productivity * [49921910001_4380616245001_4380288145001-th.jpg?pubid=49921910001] practical approaches to software defined infrastructure * [49921910001_4121169312001_4121096819001-th.jpg?pubid=49921910001] where is software defined network taking the industry? * [49921910001_4148749566001_3836445112001.jpg?pubid=49921910001] the shifting cyber security landscape * [49921910001_2617678242001_thumbnail-for-video-2592062834001.jpg?pu bid=49921910001] cbr meets recommind's co-founder * [49921910001_2968180118001_th-52b95d74e4b05e49470da69e-672293882001 .jpg?pubid=49921910001] is data a strategic asset? * [49921910001_2726070945001_th-5252ae00e4b046f6b50c97db-672293882001 .jpg?pubid=49921910001] from mainframe to mobile with bmc whitepapers * how to smash the reporting bottleneck top 7 reasons why microsoft office is the ideal report design, document generation and mail merge tool. suppliers directory * spotlight * by technology * a-z * teneo - infrastructure management, performance and virtualization services teneo is an infrastructure optimization company, providing solutions that help customers with the management, performance and virtualization of... verticals finance public sector b2c tmt small business other verticals inc web/manu/bio/trans cio agenda the boardroom internet of things consumer wearables m2m smart technology bigdata analytics software hardware platforms mobility apps devices networks security cloud aas hybrid private public data centre web scale enterprise commercial infrastructure enterprise it server storage it network it services software telecoms carrier connectivity network uc cybersecurity data physical business social media infrastructure management # abcdefghijklmnopqrstuvwxyz * computer business review * about cbr * about us * accessibility * advertise with us * contact us * help * privacy * rss feeds * site map * resources * verticals * internet of things * bigdata * mobility * cloud * data centre * enterprise it * telecoms * cybersecurity * social media * cbr dining club * cbr tv * companies * special reports * white papers * extra content * blogs * * join our network * suppliers * digital and print magazine * business review sites * automotive * banking * clean technology * drinks * energy * food * insurance * government computing * logistics * medical devices * packaging * pharmaceutical * retail * webinars * industry reports * © cbr 2016 | part of progressive digital media group plc. * home * privacy * contact us * site map dcsimg #text analytics gurus debunk four bigdata myths - rss false #icrunchdata news » feed icrunchdata news » comments feed icrunchdata news » can banking regulations benefit from bigdata and analytics? comments feed alternate alternate * about * channels + analytics edge + artificial intelligence + breaking bigdata + career insights + data science + exclusive interviews + hadoop * magazines * events * jobs * advertise * ____________________ search icrunchdata news the personality of data analytics become a contributor. * * * * * can banking regulations benefit from bigdata and analytics? november 17, 2015 by deena zaidi leave a comment can banking regulations benefit from bigdata and analytics? can banking regulations benefit from bigdata and analytics the adoption of bigdata in the banking industry has become more of a necessity than a matter of choice. the banking industry is slowly replacing the traditional ways of banking with more technology-driven methods. along with increased efficiency amongst banks, bigdata is helping banks in making more informed decisions and managing risk in real time. by combining the transactional, behavioral and social data of customers, banks are simplifying their daily working procedures and maximizing the value of unstructured datasets within a small time frame. post 2008 crisis, the changes in the regulatory framework for banks have been extensive and complicated. the new rules have increased pressure on banks that struggle to meet the regulations as quickly and as promptly when asked. in march 2012, bigdata research and development initiative was introduced under the obama administration with the aim to improve federal government’s ability to make meaning out of the large amount of data. however, missing from the list of participants was any agency accountable for regulations of the financial services industry. extensive regulations with introduction of many regulations like the dodd-frank wall street reform and consumer protection act in usa, solvency ii in europe and basel iii for global central banks, the role of bigdata becomes significantly relevant and constructive to regulatory compliance procedures. the dodd-frank act, adopted for regulations of banks extensively runs in 14,000 pages and with 71.2% finalized, it becomes important for banks to adapt to technologies that will help them comply with the new reforms on time. risk management and internal control with fraudulent crimes like anti-money laundering, trading abuse and other banking scandals, the dodd frank act and financial industry regulatory authority (finra) regulations require banks and other financial institutions to monitor banking activities closely. real-time fraud detection can be done through bigdata usage that will also help to identify anti-banking activities. digital reasoning’s cognitive computing platform can not only sieve through structured and unstructured data, but also successfully locate any traces of insider trading or other suspicious activity. ibm’s bigdata and analytics platform enables banks to manage credit risk and avoid situations of default. stress-testing besides detection of banking crimes, data analytics can help in building new compliance reports and performing stress tests. the annual stress tests by regulators require banks to aggregate data that is scattered across applications, databases, lines of business and separate legal entities. hence, updating data and sourcing the adequate data are crucial to the stress-testing process. as banks use processes that feed data into a variety of models, they should use internal and external data to run these models from regulators’ perspective. time-management according to deloitte, the requested information in many banks is not defined properly, and since information is managed in silos, it becomes a time-consuming task to produce it. this usually leads to a lot of manual shifts and extensive rounds of corrections. automation and reliance on bigdata analytics for regulatory framework across banking industry can assist banks in meeting the regulatory requirements with the least error on time. banks can also be alerted beforehand in case of any discrepancy in the actual and the expected requirement and take the essential action to correct it before the regulatory deadline. cost-cutting the high cost of meeting the regulatory requirement is also burdening many banks. besides, violations of the requirements are expensive and sometimes unaffordable for smaller banks that generate lesser capital than bigger banks. extensive regulations are causing many small banks to shut down and in many cases leading to a growing number of mergers. so far, dodd-frank has come with a cost of $21.8 billion and 60.7 million paperwork burden hours, according to american action forum. the cost is likely to increase as more regulations in the act get finalized. data analytics will prove more cost beneficial in the long run in comparison to billions of dollars in fees and fines over small errors or oversights. the effectiveness of dodd-frank also depends on regulators’ ability to use information that grows exponentially within a day. banks should be proactive in adopting the new technological trend so that they are well-prepared for any future and sudden change in regulations. having said that, organizations providing bigdata analytics should look at ways to provide the technological revolution to smaller banks at a reasonable cost. without a time lag, both small and big banks will find it easier to comply with the revised version of different regulations by adopting data analytics in the working system. article written by deena zaidi for icrunchdata news seattle, washington usa filed under: analytics edge, breaking bigdata, trending tagged with: banking, bigdata, financial services, regulation, risk management trending news * incoming! how to handle the influx of unstructured data incoming! how to handle the influx of unstructured data bigdata is everywhere — and it’s growing. by 2020, companies ... * differentiating business intelligence and analytics differentiating business intelligence and analytics i have been in forums and discussions where people interchange the ... * ftc releases bigdata report for businesses ftc releases bigdata report for businesses last week the federal trade commission released a new report, big ... * bridging the skills-gap in data science bridging the skills gap in data science furthering its commitment as a leader in data science, analytics and ... * stefan groschupf ceo datameer datameer ceo stefan groschupf talks analytics and disruptors stefan groschupf is chief executive officer and chairman of san ... [deena-zaidi-v2.jpg] about deena zaidi deena is a financial author for various online financial websites and academic journals. her focus articles are analytical and include digital trends in the banking and financial industry. she holds a degree in mba-finance and msc. international banking from university of durham. leave a reply cancel reply your email address will not be published. required fields are marked * comment _____________________________________________ _____________________________________________ _____________________________________________ _____________________________________________ _____________________________________________ _____________________________________________ _____________________________________________ _____________________________________________ name * ______________________________ email * ______________________________ website ______________________________ post comment twitter tweets by @icrunchdata trending news * incoming! how to handle the influx of unstructured data incoming! how to handle the influx of unstructured data * differentiating business intelligence and analytics differentiating business intelligence and analytics * ftc releases bigdata report for businesses ftc releases bigdata report for businesses * bridging the skills-gap in data science bridging the skills gap in data science * stefan groschupf ceo datameer datameer ceo stefan groschupf talks analytics and disrupt… hello this is icrunchdata news. we’re the leading brand for news and perspective in data science, analytics and everything bigdata. our editorial team is passionate about connecting the human side of data. we bring you high quality, original content on breaking news, trends and happenings plus exclusive interviews with industry experts and thought leaders. have a question or comment? contact us. digital magazines read stories that matter to you in our thoughtfully curated, beautifully formatted digital magazines on flipboard. scorching content on the daily! icrunchdata news analytics edge artificial intelligence now breaking bigdata sports analytics now social * * * * * explore advertiser solutions employer solutions events jobs news © 2015 · icrunchdata · the power to grow #andrei macsin's posts - bigdata news comments - bigdata: the amazing numbers in 2015 - bigdata news bdn daily press releases - bigdata news ____________________ search * sign up * sign in bigdata news * data science central * my page * bigdata jobs * data science book * dsc digest * top links * webinars * search * contact us subscribe to our newsletter * all blog posts * my blog * add bigdata: the amazing numbers in 2015 * posted by andrei macsin on september 29, 2015 at 2:37pm * view blog guest blog post by bernard marr bigdata is growing — in fact, the sector is growing so fast and we are producing data so voraciously, that no one can afford to ignore it as a “fad” any more. and, it’s going to affect all companies, large and small, across all segments of the market — from healthcare to public safety, and retail to wholesale. [capture.png] bigdata is changing the world as we know it. if you don’t believe me, check out some of these (frankly staggering) statistics: * 60% one study suggests that retailers who fully leverage the power of bigdata could see an increase in their operating margins by as much as 60 percent! (source) * 4.4 million that’s how many it jobs there will be internationally in the big data field by the end of 2015, and it’s estimated that 1.9 million of these jobs will be in the u.s. alone. it also suggests that there will be a large gap between the number of jobs available and the number of skilled workers to fill those jobs. (source) * $3.4 billion the advanced and predictive analytics (apa) software market is expected to grow to $3.4 billion worldwide by 2018. (source) * $8 million in one study, the average business expected to spend $8 million this year on bigdata and related projects. (source) * $50.1 billion bigdata was a $28.5b market in 2014, growing to $50.1 billion overall in 2017. (source) * 5 times more likely companies that use bigdata analytics are five times more likely to make decisions “much faster” than their competition. (source) * $65.7 million as little as a 10 percent increase in the accessibility of data can mean an additional $65.7 million net income for a typical fortune 1000 company. (source) * $200 million the obama administration is investing more than $200 million in big data initiatives. (source) * 90 percent ninety percent of the data in existence was created in the last two years. (source) * $3.57 billion customer experience enhancement is expected to be the largest big data business category, and the one with the most growth, with forecasts saying this sector will grow from $0.75 billion in 2015 to $3.57 billion in 2020. * 26 hours macy’s has reportedly been able to save 26 hours every time it optimizes pricing for its 73 million products through use of big data, allowing them to change pricing more frequently to follow retail trends. (source) * $500 billion one study suggests that the u.s. government could save $500 billion annuallyby implementing bigdata projects. (source) studies also show that companies (especially small and medium sized companies) put cost and personnel problems as the top reasons they haven’t implemented bigdata projects yet, but these projects need to become priority fast, or those companies risk being left behind. [capture2.png] other articles by bernard marr do you have questions or comments about bigdata? i’d love to hear them in the comments below. dsc resources * career: training | books | cheat sheet | apprenticeship | certification | salary surveys | jobs * knowledge: research | competitions | webinars | our book | members only | search dsc * buzz: business news | announcements | events | rss feeds * misc: top links | code snippets | external resources | best blogs | subscribe | for bloggers additional reading * data scientist reveals his growth hacking techniques * 10 modern statistical concepts discovered by data scientists * top data science keywords on dsc * 4 easy steps to becoming a data scientist * 13 new trends in bigdata and data science * 22 tips for better data science * data science compared to 16 analytic disciplines * how to detect spurious correlations, and how to find the real ones * 17 short tutorials all data scientists should read (and practice) * 10 types of data scientists * 66 job interview questions for data scientists * high versus low-level data science follow us on twitter: @datasciencectrl | @analyticbridge views: 1185 tags: like 1 member likes this share tweet * < previous post * next post > comment by emma r on december 17, 2015 at 10:21pm bigdata help you to understand the information contained within the data and help to identify the data which is most important for the business future business decisions. the bigdata market report aims in estimating the current size and the future growth potential of this market across the different solutions, applications, verticals, and regions. ( download pdf ) e.g. europe bigdata market is expected to grow from $3.90 billion in 2013 to $13.70 billion in 2018 at an estimated of cagr of 28.5% by 2018. comment you need to be a member of bigdata news to add comments! join bigdata news rss welcome to bigdata news sign up or sign in follow us @datasciencectrl | rss feeds on data science central bdn daily press releases why we should all be rooting for women in technology looker secures $48 million series c funding from kleiner perkins enterprises face unnecessary risks to data loss and downtime, says report by forbes insights and bmc predictive analytics firm rapidminer raised $16mm in equity financing e-commerce giant showroomprive.com uses predictive analytics to anticipate and reduce churn wordstat 7.1: geospatial intelligence meets text analytics analytics engines and almac group awarded innovate uk funding to develop bigdata analytics platform for bioinformatics comtrade facilitates global access to cern research the expo line won't reduce l.a. traffic – but that doesn't mean you shouldn't take it consumerizing it with tibco simplr and tibco activematrix bpm 4.0 © 2016 bigdatanews.com is a subsidiary of datasciencecentral llc and not affiliated with systap powered by ning badges | report an issue | terms of service hello, you need to enable javascript to use bigdata news. please check your browser settings or contact your system administrator. [p?c1=2&c2=6770185&cv=2.0&cj=1] #publisher bigdata is on call to deliver more food for a growing population home main menu * insights * events * videos secondary menu * sustainability * energy * buildings * cities * water * transportation * supply chain * design greenbiz on social media twitter facebook google+ linkedin this site (rss) search ____________________ search search toggle navigation bigdata is on call to deliver more food for a growing population barbara grady monday, november 16, 2015 - 2:00am awhere bringing bigdata solutions to farming could solve some major challenges. amid the looming sustainability challenges of our time is how the world will feed 2.5 billion more people in a few decades — even as rising sea levels and warmer temperatures take away some arable land. one answer is bringing bigdata to the rescue. farm productivity will need to increase and food waste decrease if 9 billion people are to have enough food to survive. and those productivity gains will be most needed in developing nations in africa, southern asia and latin america where population gains are expected to be the fastest, according to the united nations. yet in developing nations, the food supply comes mostly from the tilling of smallholder farms — farmers with a few acres who generally rely on rainwater as irrigation. as tech innovation took up agriculture this year — attracting a record $2.06 billion in venture capital investment in 228 ag tech deals during the first half of this year — a plethora of new agtech products emerged. but very few new agtech tools are for smallholder farmers, especially those without computers, sensors, ipad-equipped tractor — or even electricity, according to agfunder, which tracks agricultural and tech investment. most, according to agfunder, target data from soil using sensors that sophisticated and well-heeled farmers can use to monitor conditions. "many of the options out there today target the very large operations in the u.s., which will likely be much more complex, with more equipment and teams than smaller operations. so how does a technology address the needs of both big and small operations?" wrote agfunder author, agriculture professor lauren manning. now, however, a b corp. that specializes in data intelligence for agriculture, awhere, is pairing with software development platform company apigee to offer applications smallholder farmers can get via text messages on their cell phones. it turns out that while smallholder farmers in the developing world often don't have computers or smartphones, they do have cell phones. the two companies also are opening up the agriculture and weather data platform to application developers to create other tools for farmers and food companies, targeting smallholder farmers. "there are 580 million farmers in the world and most of those are smallholder farmers," said stewart collis, founder and cto of awhere. actionaid.org smallholder farmer in rwanda. “we are asking smallholder farmers to increase production while also having to increase weather adaptation. they can no longer trust grandfather’s advice on when to begin planting," collis said, because warming climates have changed conditions. and yet smallholder farmers rely on rain as irrigation. his belief is that if smallholder farmers get timely information on weather forecasts, weather patterns and regional market conditions, they will make better decisions, which in turn will boost their harvest productivity. so awhere, his 16-year-old data intelligence firm that collects and analyzes billions of pieces of data around the globe each day, is joining with apigee, an api software platform company to offer a platform for application development for apps that help smallholders and is releasing its own apps. "farmers need to make decisions in the next few days" when something happens such as an unexpected rain — or a prolonged lack of rain, collis said. they need to know if planting now or two weeks from now is the wiser choice, he said, and the difference can have dramatic consequences on their harvest. so with a product that "bridges the gap between short-term weather forecast and long-term climate models," farmers will have tools to make decisions "we’ve developed a platform that is two parts," collis said. "one is a global bigdata technology on weather all over the world, pulled from multiple sources, such as the national doppler weather sites" and also regional satellite radars in areas where the doppler system doesn't reach. "we pull all that data and produce weather forecasts down to the field level" and extending out 15 days. it also includes historical weather trends and mathematical predictions out to 20 years. that app is called weather awhere, for field level weather and agronomic information. all sorts of better decisions can be made by bridging that gap between short term weather forecast and long term climate models. the second product is an open source community development platform which gives software developers access to awhere's apis and apigee's platform for creating their own applications. the two companies don't intend to market to smallholder farms directly but rather to the agricultural service companies and regional farm agencies that they work with. "emerging markets are a challenge because a lot of farmers don’t have access to websites and smartphones. but they do have cell phones," collis said. "so, one way to send farmers information is to send out sms texts to small farmers. our customer may be the agricultural services companies that provide ag services to farmers in ghana. they push out text information to their farmers and they also have a call center, so farmers can call into find out information." its weather awhere and community development platform also will be marketed to commercial agriculture companies in the u.s. and europe as well as consumer packaged food companies. "anyone who has to do with food industry. we can provide them a global view of what is happening at field level," he said. but having grown up on a farm and as a former aid worker in kenya where he provided farmers with data to help them make decisions, collis has a personal passion to help smallholder farmers. so the agricultural service company is one target market in emerging nations. "the other is telecommunications companies," he said. in ghana, for example, vodafone uses awhere services to provide a wireless text service of agricultural information. he said awhere and apigee are in the final stages of negotiating another big telecommunications contract with another wireless company for its new products. the apigee intelligent api platform is sold to large businesses to use apis to securely share data and services across devices to serve customers in real time, according to the company. in a statement, an apigee vice president said the agriculture world is ready for this. "the innovation awhere is bringing to global agriculture illustrates the power of digital connectivity powered by apis — at scale," said ed anuff, apigee vice president of strategy. "agriculture is being revolutionized by new information about farming and mobile applications, and awhere's targeted, real-time agronomic and weather data is helping take agtech to the next level." apigee's platform today includes apigee edge, its flagship api management product; apigee insights, an advanced predictive analytics product; and apigee link, an api-first product for the internet of things. most recently, apigee introduced apigee sense, an intelligent api security product. topics: * food & agriculture * data tags: * precision agriculture * bigdata share this article * twitter * facebook * linkedin barbara grady avatar barbara grady senior writer greenbiz group related content from aerofarms to syngenta, ag tech sowed new solutions in 2015 featured image from aerofarms to syngenta, ag tech sowed new solutions in 2015 bybarbara grady can an ex-googler use bigdata to help farmers grow more food? featured image can an ex-googler use bigdata to help farmers grow more food? byheather clancy 13 companies sowing solutions for food resilience featured image 13 companies sowing solutions for food resilience byheather clancy will silicon valley ag tech really grow with farmers? featured image will silicon valley ag tech really grow with farmers? bybarbara grady how the agricultural movement tackles challenges of a warming world featured image how the agricultural movement tackles challenges of a warming world bylisa palmer greenbuzz e-news subscribe and get in-depth sustainable business news delivered daily. trending if i was right, why was i fired? featured image if i was right, why was i fired? bydavid crane david crane's farewell letter to nrg employees featured image david crane's farewell letter to nrg employees bydavid crane my ngo hall of fame: what i learned from the best of the nonprofits featured image my ngo hall of fame: what i learned from the best of the nonprofits bybob langert investors to companies: it's time to come clean on climate risk featured image investors to companies: it's time to come clean on climate risk bykeith larsen fuel cells: a $2 billion secret of the clean economy featured image fuel cells: a $2 billion secret of the clean economy byrp siegel early-bird ends friday featured videos [thumbnailcady.png?itok=pcznbpof] nasa's cady coleman on the necessity of circular economy thinking [tom_and_van.png?itok=u5vt9o0w] van jones and tom steyer on the business opportunity of including all more from video featured whitepapers [macktechnologiescover.png?itok=xxu3x-pf] mack technologies achieves unprecedented wireless control [lighting_and_the_efficient_data_center.jpg?itok=xac1r6jb] lighting and the efficient data center learn more state of green business report 2015 get your copy sign-up for the greenbuzz newsletter footer menu 1 * about us * our team * media kit * contact us footer menu 2 * executive network * reports * white papers * webcasts footer menu 3 * career resources * consulting svcs * jobs twitter facebook google+ linkedin this site (rss) secondary footer menu * privacy policy © 2016 greenbiz group - greenbiz.com® is a registered trademark of greenbiz group inc. #publisher ian henderson software mass appeal * free newsletters * [greynav_div.gif] * it directory * [greynav_div.gif] * news alerts * [greynav_div.gif] * rss * [greynav_div.gif] * news tip-offs * [greynav_div.gif] * add to favourites ____________________ [search.gif]-submit [netxactics0116_109x18-an.gif] [itweb_logo_sml.png] storage * home * opinion * in depth * surveys * jobs * services * events * publications * video * about * samsung [- by sector_____________________] * business * channel * computing * enterprise * financial * hardware * internet * networking * security * software * telecoms * columnists * booth's bites * double take * tech forum * industry insight * features * industry solutions * white papers * reviews * survey archive + open source + unified communications + byoa + energy management + contact centres 2013 + storage survey 2013 * it salary survey 2015 * cio survey 2015 * post your cv * post your vacancies * it jobs * engineering jobs * financial jobs * sales jobs * virtual press offices * itdirectory * executive forums * advertising * mobile apps * itweb events * governance risk and compliance 2016 * ca it management symposium 2016 * business intelligence summit 2016 * security summit 2016 * gartner events * itweb events * itweb africa * brainstorm * cio directory * the margin * careerweb * defenceweb * trainingweb * hr pulse * ifashion * mobile apps * it directory * about us * contact us * advertising on itweb * privacy policy * competition policy * bee [virtual press offices™�]company zoneses] advertise on itweb fri, 15 jan, 12:22:39 pm advertisement take advantage of the latest technology, and top skills in data with eoh mc solutions. advertisement what’s the real cost of cybercrime? get the facts, understand the threat, develop your plan. advertisement micollab – collaboration, mobility & messaging in a single solution. click here now learn more deal to solve ska's bigdata conundrum by admire moyo, itweb's senior news journalist. johannesburg, 17 nov 2015 * read in this story * hidden secrets * understanding the universe the move signals the unlocking of the hidden secrets in the immense amount of data generated by the ska. the move signals the unlocking of the hidden secrets in the immense amount of data generated by the ska. south africa and the netherlands are set to sign a memorandum of understanding (mou) which seeks to address the bigdata conundrum at the square kilometre array (ska). president jacob zuma and dutch prime minister mark rutte will meet during rutte's visit from 17 to 18 november "to elevate and reaffirm the longstanding and warm bilateral relations" between the countries. zuma invited rutte to sa when the two met at davos at the world economic forum in january. rutte's visit will include a pivotal south african-dutch data science partnership between key institutions from both countries to understand the volume of data generated by the ska. hidden secrets ska sa says this signals the unlocking of the hidden secrets in the immense amount of data generated by the ska – the world's biggest radio telescope. the agreement is part of the visit to sa by the prime minister of the netherlands and his trade delegation of 75 companies. see also * german institute boosts meerkat project * universities unite to boost ska * * ska * mark rutte ska sa and the university of cape town, through the newly established inter-university institute for data intensive astronomy (idia), will sign a mou with fellow research institutions in the netherlands, ibm and astron, to collaborate in a ground-breaking research project entitled precursor regional science data centres for the ska. the mou will be celebrated as part of ‘house of the future' – a programme of workshops, seminars, presentations and round tables with south african and dutch stakeholders, taking place from 16 to 20 november in turbine hall, johannesburg. the sa-dutch agreement on data science aims to establish national and regional data centres – to tackle one of the most significant challenges presented by the ska: how to manage, process, and make accessible the immense amount of data the telescope will generate. the data centres will provide astronomers around the world with access to the large-scale data infrastructures and associated high performance computing needed to make sense of the data. "we assume there will be at least two astronomy-focused sites, one each in sa and netherlands," says professor russ taylor, idia founding director and joint university of cape town/university of the western cape ska research chair. "the initial focus of the centres will be to service the current and future data archiving, distribution and science exploration needs of the meerkat and lofar radio telescopes in sa and the netherlands, respectively. the activity, combining both operational and research components, is an important step on the path towards being able to efficiently extract major science value from the massive astronomical datasets which will be collected by the ska," says dr jasper horrell, general manager of science computing and innovation at ska sa. the techniques developed can, in turn, be applied in other fields such as bigdata analytics, high performance computing, green computing, and visualisation analytics. understanding the universe the ska will be the world's largest radio telescope – a hundred times bigger than any current radio telescope; it will revolutionise the world's understanding of the universe. it will be built in two phases – ska1 and ska2 – starting in 2018. ska 1 will include two instruments – ska1 mid (to be built in sa) and ska1 low (to be built in australia); they will observe the universe at different frequencies. with sa's meerkat and the netherlands' apertif telescopes both expected to come online in 2016, the scale of such data collection is poised to increase significantly. the large scale of the datasets and the requirements of the astronomers to perform complex scientific analyses, which are often compute-intensive, demand innovative approaches. data at these scales present unique challenges not only for managing the collection, but also for how researchers extract their science. ibm and astron have been working together since 2012 in a five-year collaboration totalling €32.9 million to research exascale computer systems that will be needed by the ska. subscribe to our newsletter subscribe to brainstorm download our mobile apps [subscribe_now_envelope.gif] subscribe now! essential storage news filtered for you our comments policy does not allow anonymous postings. read the policy here [jurumani_logo.gif] __________________________________________________________________ __________________________________________________________________ best read storage * telkom suffers hardware failure * building scalable architectures with veeam and cisco ucs c3000 storage servers * networks unlimited partners with mellanox; brings huge performance gains to ethernet in southern africa * tradebridge enhances storage, virtualisation efficiencies with datacentrix related articles * embracing the bigdata world * 2015 visionary cio speaks to itweb * top maths, science matrics get ska bursaries * top maths, science matrics get ska bursaries __________________________________________________________________ company news business * cloudbox: all-in-one cloud solution for smes * wd partners with milestone enterprise * opentext a leader in gartner's magic quadrant * changing e-learning ecosystem and its future channel * regus, business continuity institute partner security * tyco security products enhances visitor management white paper [building_scalable_inside.jpg] * building scalable architectures with veeam and cisco ucs c3000 storage servers recommended for you sick of razors? use a laser [dot_fold.gif] bigdata take advantage of the latest technology, and top skills in data with eoh mc solutions. [dot_fold.gif] data what’s the real cost of cybercrime? get the facts, understand the threat, develop your plan. [dot_fold.gif] collaborate micollab – collaboration, mobility & messaging in a single solution. click here now learn more sponsors message co-sponsor [infinidat_logo.gif] infinidat changes the paradigm of enterprise-class storage, at a disruptive price point, with multi-petabyte capacity in a single rack,incredible performance, mainframe-class reliability and an unprecedented 99.99999% availability. * infinibox unified storage: nas capabilities * storage designed to support an oracle database * customer storage priorities __________________________________________________________________ africa * kenya: authorities planning tech strategy for 2017 elections * econet takes regulator to court over tariff * orange adds airtel subsidiaries to acquisition tally * nigerian govt can't freeze mtn's bank accounts * mtn rewards sa's top learners __________________________________________________________________ __________________________________________________________________ [bi2016-textbox-2016.gif] itweb business intelligence 2016 1-3 march 2016 / the forum, bryanston martha bennett - principal analyst serving cios, forrester research keynote speaker, martha bennett from forrester research, local experts and industry specialists will tackle the pertinent issues that keep you awake at night, including mobile bi, analytics, bi in the cloud, big data, self-service bi, and more! register your seat now and ensure you hear about emerging technologies first diamond sponsor platinum sponsor [bi2016_qv_tb.gif] [bi2016_sas_tb.gif] [footer-1.gif] __________________________________________________________________ publications http://www.brainstormmag.co.za/ gagged why the fpb wants to censor the internet click here to get your 3 free trial issues! andy robb networking falling short * contact us * advertise * newsletters * terms of use * privacy policy * itnews@itweb.co.za * twitter * facebook * google+ * linkedin * youtube * playstore * google play * itunes * rss copyright (c) 1996 - 2016 itweb limited. all rights reserved. would you like to see your news here? contact us for more details at itnews@itweb.co.za * * * * [opa.jpg] connect with us: techrepublic search ____________________ search * cxo * innovation * cloud * security * bigdata * more + startups + data centers + networking + software + tech & work + all topics + sections: + photos + videos + all writers + o newsletters o forums o resource library o tech pro free trial + editions: us o australia o united kingdom o united states o japan * + newsletters + forums + resource library + tech pro free trial * membership membership gravatar my profile + people + subscriptions + my stuff + preferences + send a message + log out * techrepublic * + search ____________________ search * + topics: + cxo + innovation + cloud + security + bigdata + startups + data centers + networking + software + tech & work + all topics * sections: * photos * videos * all writers * + newsletters + forums + resource library + tech pro free trial * editions: us + australia + united kingdom + united states + japan membership membership * gravatar my profile + people + subscriptions + my stuff + preferences + send a message + log out open source sap's hana will lose the bigdata war without open source, as proven by 21 new security flaws while the world is moving to open data infrastructure, sap's hana won't survive with a proprietary dream. matt asay explains why the bigdata platform is in trouble. by matt asay | november 11, 2015, 11:44 am pst * rss * comments 0 * facebook * linkedin * twitter * more + email + print + reddit + delicious + digg + pinterest + stumbleupon + google plus image: istockphoto/undrey sap has been boasting about its "revolutionary" bigdata platform, sap hana, for years. while its claims have always been a bit suspect, recent revelations that hana is riddled with critical security flaws only reinforce the mantra that, when it comes to bigdata infrastructure, open source is best. most other companies get this, even hitherto proprietary giants like ibm. will sap get the memo in time to rejigger its approach to big data? probably not, which is why sap customers should probably check out what open source has to offer. (almost) everything has open source inside even the most proprietary software generally has open source inside. that's why gartner analyst martin kihn can declare with utmost assurance that everything—everything—is open source now, to some degree: i am willing to guess you—yes, you—would be shocked if you really understood to what extent that whizzy piece of expensive cloud software you're using actually (deep, deep in its soul) was running on absolutely free, not-developed-here, open source technology that you—yes, you—could probably bang into something almost as useful if you only knew how to do it. it's also why you really, really shouldn't be futzing with sap hana, or any other proprietary data infrastructure that tries to go it alone without the aid of the open source community. cloudera chief strategy officer mike olson perhaps said it best: there's been a stunning and irreversible trend in enterprise infrastructure. no dominant platform-level software infrastructure has emerged in the last 10 years in closed-source, proprietary form. which brings us to...hana. bigdata, big problems hana, sap's bigdata darling, has been the subject of controversy for some time. for years wall street analysts like peter goldmacher (formerly of cowen & co.) have criticized sap's financial treatment of hana, arguing that the legacy software vendor had been misrepresenting hana revenue to project an "inflated growth rate." in short, he and others argued hana's zero-to-$1 billion rapid growth story was "highly, highly unlikely." but wait! it gets worse. that's because security firm onapsis just uncovered 21 significant security flaws in hana, eight of which it deemed "critical." how critical? unless companies act to change system configurations, "unauthenticated attackers could take full control of vulnerable sap hana systems, including stealing, deleting or changing business information, as well as taking the platform offline to disrupt key business processes." not a cheery thought. and, it's why host analytics (and data infrastructure expert) dave kellogg advises hana customers to switch to "standard infrastructure," in part because it's "more proven." peaceful coexistence...for now like, for example, apache spark. of course, spark-sponsor databricks will be quick to say that sap hana and spark are complementary, that the one is great for analyzing legacy enterprise data stuck in a crm or erp system while the latter handles...pretty much everything else. this is true, but maybe not relevant. at least, not for long. after all, as kamlesh barvalia, business intelligence and analytics leader at ge, argued, there is "a great deal of overlap" between the two in terms of features and use cases, and many (like he) will "bet on spark for the long haul." why? because spark is open source (so "you do not run the risk of getting yourself trapped in proprietary development platforms" like hana), cheaper, and "there is a great deal of momentum behind spark and it appears that the feature overlap as well as breadth and depth of offering will only increase as the time goes by." stated in pithier fashion, "what dave kellogg said." spark isn't the only open source challenge to hana's alleged momentum (barely ahead of informix in terms of overall popularity). given the pace at which the open source community keeps leapfrogging itself with better and better data infrastructure, hatched and released by companies like google, facebook, and linkedin, who manage scale and speed that even sap can hardly fathom, this is the open source community's market to lose. but it won't, for all the reasons mike olson called out in his post. ultimately, all data infrastructure will be open, or it will be irrelevant. click here to automatically subscribe to our linux and open source newsletter. subscribe also see * manage complex bigdata pipeline challenges with these approaches (techrepublic) * microsoft, oracle, aws are the top database leaders, according to gartner (techrepublic) * three reasons you need to run spark in the cloud (techrepublic) * apache spark is doomed (techrepublic) * comments 0 * facebook * linkedin * twitter * more + email + print + reddit + delicious + digg + pinterest + stumbleupon + google plus about matt asay matt asay is a veteran technology columnist who has written for cnet, readwrite, and other tech media. asay has also held a variety of executive roles with leading mobile and bigdata software companies. * full bio * contact * see all of matt's content * google+ * mjasay × full bio matt asay is a veteran technology columnist who has written for cnet, readwrite, and other tech media. asay has also held a variety of executive roles with leading mobile and bigdata software companies. powered by livefyre add your comment editor's picks we-commerce: the sharing economy's uncertain path we-commerce: the sharing economy's uncertain path 'paypal mafia' redefined success in silicon valley 'paypal mafia' redefined success in silicon valley how aaron levie and his childhood friends built box how aaron levie and his childhood friends built box conflict minerals funded a war that killed millions conflict minerals funded a war that killed millions latest on tech pro research quality assurance checklist for outsourced projects quality assurance checklist for outsourced projects vendor reference worksheet: business software purchases vendor reference worksheet: business software purchases research: byod, wearables and iot research: byod, wearables and iot virtualization policy virtualization policy explore * blogs * downloads * q&a * discussions * meet the team * tech pro research * resource library * photos * videos services * about us * membership * newsletters * rss feeds * site map * site help & feedback * faq * advertise * reprint policy © 2016 cbs interactive. all rights reserved. privacy policy | cookies | ad choice | terms of use | mobile user agreement a zdnet site | visit other cbs interactive sites: [select site______] skip to main content wrkf logo site menu donate * menu * news + wrkf news + npr news * programs + schedule + podcasts + capitol access + louisiana cultural vistas magazine + louisiana's lab + out to lunch + bayou garden + all programs a-z * support + why we give + donate now + become a sustaining member + join the circle of friends + leave a legacy + get your gift matched + donate your car * sponsorship + why sponsor wrkf + wrkf's reach + testimonials + sponsors + message guidelines + media kit * about + staff + board of directors + community advisory board + opportunities + contact * search * menu * news + wrkf news + npr news * programs + schedule + podcasts + capitol access + louisiana cultural vistas magazine + louisiana's lab + out to lunch + bayou garden + all programs a-z * support + why we give + donate now + become a sustaining member + join the circle of friends + leave a legacy + get your gift matched + donate your car * sponsorship + why sponsor wrkf + wrkf's reach + testimonials + sponsors + message guidelines + media kit * about + staff + board of directors + community advisory board + opportunities + contact * search using bigdata to make political campaigns smarter by wallis watkins • nov 16, 2015 sharetwitter facebook google+ email for many watchers of political races, it’s a numbers game. dr. david sathiaraj, assistant professor for research in geography and anthropology at lsu, thinks there’s a better way to crack those numbers. listen listening... / 2:00 traditional polling is good, says sathiaraj," but in some cases, traditional polling can tend to be particularly small sample sizes, and so they may not be representative of the electorate." [wrkf_horz_transparent_0.png] credit dr. david sathiaraj traditional polling is typically done by selecting a group of people, asking them questions - like who they plan on voting for - and using that as an indicator of what the entire electorate may do. sathiaraj wanted to dig deeper. his goal was to capture individual’s feelings of a candidate or an issue. he developed an algorithm that would pull information like voting history, voter registration and polling results. "the algorithm basically provides scores for people on how they perceive a certain candidate or an issue. you’re able to come up with a more realistic scenario as compared to a broad, statewide poll," he says. sathiaraj believes campaigns big or small can use his algorithm to better pinpoint who they should reach out to. by enabling campaigns to see if they're weak in areas like a parish or precinct, says sathiaraj, "they can start targeting, canvassing, getting the message out in those areas." that individualized approach leads to a smaller margin of error. sathiaraj used the algorithm on last year’s senate runoff election in louisiana. his model was nearly spot-on, just .2 percent off the actual results. the algorithm correctly predicted how fifty-seven of the sixty-four parishes would vote. on average, most election polls have a margin of error of three to four percent. sathiaraj thinks that granular approach helps. "polls are much broader, done at a statewide level, and the sample can be randomly selected. you may miss out on certain parishes that may be swaying one way or another. because it’s so broad-based, a sample size of eight hundred is typically too small for an electorate that’s about three million," he explains. the next step for sathiaraj and his team is to scale the new technology to test on larger voting populations. related program: louisiana's lab tags: wrkf news polls related content * picking the winner: how polls affect elections 2 months ago * secretary of state: voter turnout science not easy 2 months ago view the discussion thread. * wrkf on twitter * wrkf on facebook * © 2016 wrkf * privacy policy * contest rules * contact #bernard marr's posts - data science central comments - how apple uses bigdata to drive success - data science central announcements - data science central ____________________ search * sign up * sign in data science central * analytics * bigdata * hadoop * data plumbing * dataviz * jobs * webinars * digest + previous digests * search * contact subscribe to dsc newsletter * all blog posts * my blog * add how apple uses bigdata to drive success * posted by bernard marr on may 22, 2015 at 1:30pm * view blog apple’s old slogan was “think different” – and while it is now retired, and the ethos may not be as apparent in the company’s products as it once was, it is true for their approach to bigdata. [bor55.png] in some ways, despite being the most profitable tech company in the world, apple found itself having to play catch-up with bigdata. while apple traditionally employed teams of highly paid experts in aesthetics and design to produce systems that they thought people would want to use, competitors like google examined user data to see how people actually were using them. this gave those competitors an edge with the everyday apps that made smart phones so popular – maps, navigation, voice recognition and other aspects of computing that we want to do on the move. but while they may have been slow off the starting block, they have now entered the race with a strong stride. their powerful presence in the mobile market has put their devices in the hands of millions and they have been keen to encourage development of apps that are based on monitoring and sharing of user data. a notable example is their recently announced partnership with ibm to facilitate the development of health-related mobile apps. it has also provided a range of applications targeted at other industries, including air travel, banking and insurance, also developed in partnership with ibm and aimed at bringing analytical capabilities to users of its mobile devices in those fields. the launch of the apple watch could potentially accelerate this process in a dramatic fashion – if, as many commentators are saying is possible, it turns out to be the device which finally brings wearables into the mainstream. designed to be worn all day long, and to collect a wider variety of data thanks to additional sensors, even more personal data is available for analysis. as well as positioning itself as an “enabler” of bigdata in other people’s lives, it has also been put to use in its own internal systems. apple has often been secretive about the processes behind its traditionally greatest strength – product design. however it is known that bigdata also plays a part here. data is collected about how, when and where its products – smart phones, tablets, computers and now watches – are used, to determine what new features should be added, or how the way they are operated can be tweaked to provide the most comfortable and logical user experience. the siri voice recognition features of idevices have proved popular with users too, and this is also powered by bigdata. voice data captured by the machine is uploaded to its cloud analytics platforms, which compare them alongside millions of other user-entered commands to help it become better at recognizing speech patterns (machine learning) and more accurately match users to the data they are seeking. apple keeps this data for two years – disassociated from your real identity and assigned with a unique anonymous indicator, as a concession to ensuring privacy. like its biggest competitors, apple also offers cloud-based storage, computing and productivity solutions, for both consumer and business use. last month it was reported that it had purchased foundationdb, a popular proprietary database architecture widely used for bigdata applications. it is thought that this could be used to bring increased analytical prowess across its suite of online services such as icloud, apple productivity works (formerly iwork) and its upcoming streaming music service. aiming to capture a share of the market dominated by pandora, spotify and google music, this service will be built on the technology acquired by their purchase last year of beats music. beats developed algorithms designed to match users with music they are likely to enjoy listening to, in a similar way to recommendation engines used by amazon and netflix. sales through apple’s itunes service have declined as the popularity of streaming services has usurped downloading as the favorite method of accessing music online. the new service, expected in june, is apple’s attempt to get a slice of this action. apple may have been slower in its uptake of bigdata and analytics than many of its rivals, but it has clearly seen that it has to play a big part in its future if it wants to stay ahead of the pack. it seems likely that it will try and use it to move away from relying on hugely expensive, episodic product releases to drive its growth as a business, and towards the more organic, constantly-regenerating model of growth favoured by its competitors in the software and services markets. if apple can meld its trademark excellence in design and user-friendliness with innovative uses of bigdata analytics, it could continue to surprise us with products and services which become culturally ingrained in everyday life, just like the imac, ipod and iphone – ensuring it remains the world’s most valuable brand for some time to come. i hope you found this post interesting. i am always keen to hear your views on the topic and invite you to comment with any thoughts you might have. about : bernard marr is a globally recognized expert in analytics and bigdata. he helps companies manage, measure, analyze and improve performance using data. his new book is: bigdata: using smart bigdata, analytics and metrics to make better decisions and improve performance you can read a free sample chapter here. dsc resources * career: training | books | cheat sheet | apprenticeship | certification | salary surveys | jobs * knowledge: research | competitions | webinars | our book | members only | search dsc * buzz: business news | announcements | events | rss feeds * misc: top links | code snippets | external resources | best blogs | subscribe | for bloggers additional reading * data scientist reveals his growth hacking techniques * 10 modern statistical concepts discovered by data scientists * top data science keywords on dsc * 4 easy steps to becoming a data scientist * 13 new trends in bigdata and data science * 22 tips for better data science * data science compared to 16 analytic disciplines * how to detect spurious correlations, and how to find the real ones * 17 short tutorials all data scientists should read (and practice) * 10 types of data scientists * 66 job interview questions for data scientists * high versus low-level data science follow us on twitter: @datasciencectrl | @analyticbridge views: 5503 tags: like 2 members like this share tweet * < previous post * next post > comment you need to be a member of data science central to add comments! join data science central comment by tarun sharma on may 26, 2015 at 10:00pm hey bernard. thanks for sharing the article. i think we are always interested in knowing how such big companies are making use of that data and how they are able to manage it. that's really good information. keep up the good work . cheers :) rss welcome to data science central sign up or sign in or sign in with: * * * top categories machine learning r programming python for data science visualization, dashboards nosql and newsql bigdata cheat sheets internet of things excel follow us @datasciencectrl | rss feeds resources videos * the past present and future of data science a live roundtable the past present and future of data science a live roundtable added by tim matteson 0 comments 0 likes * how flextronics uses data visualization and analytics to improve customer satisfaction how flextronics uses data visualization and analytics to improve customer satisfaction added by tim matteson 0 comments 1 like * add videos * view all * facebook announcements creating a strong business advantage with multi-genre advanced analytics for iot kickstart your data initiatives in 2016 enroll now: wharton marketing online program begins march 14 storytelling with data in 8 minutes online master of science in predictive analytics pivotal webinar: your open source playbook for 2016 learn how to understand decisions driven by bigdata with nyu stern 6 must attend bigdata conferences : best price ends soon the past, present and future of data science – a live roundtable - dsc webinar 2016 bigdata predictions from mike stonebraker and others © 2016 data science central powered by ning badges | report an issue | terms of service hello, you need to enable javascript to use data science central. please check your browser settings or contact your system administrator. [p?c1=2&c2=6770185&cv=2.0&cj=1] #rss feed a social media today community [network-drop-down.png] * [brand-bar-wheel.png] about social media today ______________________________________________________________ * [logo.png] the world's best thinkers on energy & climate ______________________________________________________________ * [logo.png] the world's best thinkers on the urban future ______________________________________________________________ * [logo.png] the world's best thinkers on business intelligence & data analytics ______________________________________________________________ * [logo.png] the world's best thinkers on healthcare ______________________________________________________________ * [logo_0.png] the world's best thinkers on social media sign up | login with → smartdata collective the world's best thinkers on data * home + about us & contact + work with us * post here * analytics + social media analytics + text analytics + sentiment analytics + predictive analytics + modeling + web analytics + workforce analytics * bigdata + social data + data warehousing + visualization + workforce data + data mining + data quality * business intel + business rules + decision management + knowledge management + market research + crm * data management + risk management + transparency + culture & leadership + policy & governance + best practices + privacy * it + mobility + security + hardware + location + cloud computing * software + hadoop + mapreduce + sql + open source + marketing automation + r programming language * reports * webcasts * help + faq posted by: [untitled-1_2.jpg] brigg patten posts profile comments follow please login or register to follow this user. how bigdata is shaping the future of the learning and development industry like it? 249 comments 0 posted july 22, 2015 keywords: bigdata these days, nearly all of our daily actions both at work and in our personal lives are done using some form of technology and internet: online shopping, sending work emails, connecting with friends on social media, watching television or movies, etc. each time we utilize the web to accomplish any of these daily tasks, data is collected that helps these applications understand our particular tastes and preferences, allowing them to suggest things to us and predict what future choices we will make online. data mining to obtain bigdata, this process of extracting unique patterns in data from users and consumers, is quickly becoming the standard by which businesses operate. large organizations such as wal-mart, target, and google use bigdata frequently, but even small local businesses are beginning to implement some form of data mining and/or collection-based software. businesses are able to accurately depict products and services that customers would find useful rather than using a general spam-like approach to customer interaction. though bigdata is clearly a great tool for the retail and sales industry, it is also proving to be a valuable asset for the learning and development field—particularly with corporate training companies. bigdata has been shown to be very beneficial for employee training. whether a company is seeking to improve its training for new hires or for current employees, bigdata can have a significant effect on the whole process. employers can use bigdata to track how well the employees have understood the courses and which information is being retained and applied, which then allows employers to see what parts of the training need to be improved. by doing this and moving away from the traditional, non-customized style of employee training, companies will see a huge improvement in their employees’ performances. as tim martin of rustici software explained, “companies are starting to recognize how employees actually learn and allowing them to do it the way they wish to, rather than forcing them into a draconian system.” bigdata can also help employers effectively motivate and inspire their employees. entrepreneur.com says, “by tracking, analyzing and sharing employee performance-related data, employers and hr not only gain more insight on employees, but boost individual motivation and overall engagement.” when companies have a good handle on their employees and really understand them as people, they can thus find ways to give them more validation and job satisfaction. happy, validated, satisfied employees produce better work, so using bigdata to help you understand your employees and the way they tick will really benefit your employees and your company as a whole. most people are familiar with the notion that bigdata is a great way for companies to target consumers and advertise based on a user’s shopping and viewing history, but bigdata has grown into an important learning and development tool for businesses looking to improve the performance of their employees. as bigdata helps to uncover employees’ potential and abilities, employers are able to carefully adapt not only their training methods but also their employee engagement levels and incentives, optimizing both the company’s overall performance as well as employee morale. * login or register to post comments * [untitled-1_2.jpg] authored by: brigg patten brigg patten writes in the business and tech spaces. he's a fan of podcasts, bokeh and smooth jazz. his time is mostly spent learning the piano and watching his golden retriever julian chase a stick. see complete profile would you like to contribute to this site? get started » other posts by brigg patten what ibm’s recent $3m investment in iot means for you and me - december 4, 2015 how predictive analytics will affect bigdata’s future - november 9, 2015 the differences between how the government and the private sector use big data - october 7, 2015 the difference between business intelligence and real data science - september 8, 2015 big learner data and corporate training - august 5, 2015 related posts what’s in store for bigdata analytics in 2016 january 14, 2016 by stevesarsfield nosql and the internet of things january 14, 2016 by kingmesal how iot applications can help businesses grow internally january 7, 2016 by xanderscho 5 predictions for trends in data, analytics and machine learning in 2016 january 4, 2016 by bruce robbins how to recycle old it equipment and keep your data safe january 4, 2016 by ryank image the moderated business community for business intelligence, predictive analytics, and data professionals. image * twitter * iframe: http://platform.twitter.com/widgets/follow_button.html?screen_name= smartdataco * rss * rss as xml addtomyyahoo4 subscribe with bloglines add to google * email * get our best posts in your inbox weekly first name ____________________ last name ____________________ email address ____________________ sign up! how do you innovate effectively and maintain a competive edge? image learn how in our exlcusive ebook, "bad data need not apply: designing the modern data warehouse environment." download here. sdc special columns big-data-guru riksy business big-data-mops events no events listed add your own! recommended to follow ahamed parpia follow a t follow jan henderyckx follow brent leary follow orlando doehring follow daniel kraft follow comments * featured * popular * recent “the final aim of mcc is to allow implementation of rich smartphone web apps on an excess of cell phone devices, with a gorgeous user experience.” january 7th, 2016 by jamesmartin057 “by including more factors to score a person’s credit worthiness would make it even tougher for borrowers to obtain a loan approval. this in turn causes the finance industry to face a fluctuating business trend. there are definitely pros and cons to this major change in a loan application process but one thing for sure is to ensure borrowers shall only be allowed to take up a loan which is ...” january 7th, 2016 by webbrowan “thanks for the insight connie!” january 4th, 2016 by daniel matthews “there are clearly a number of changes coming for small businesses as they await ...” january 6th, 2016 by nguyennga “very useful tips, especially from the point of view of customer service.” january 4th, 2016 by isabella1977 “this is such a scary article. no doubt there is a lot of things that you have ...” january 6th, 2016 by webbrowan “the final aim of mcc is to allow implementation of rich smartphone web apps on ...” january 7th, 2016 by jamesmartin057 “regardless of the nature of any business, finance is the key factor that helps ...” january 7th, 2016 by webbrowan “such comparison tools are great for borrowers to perform an estimated ...” january 7th, 2016 by webbrowan “when it concerns money, you cannot afford to commit even a single mistake and ...” january 6th, 2016 by webbrowan “due to modernization we have found good technology advanced in the society; ...” january 7th, 2016 by edwardmiller “social collaboration seems like a funny way to describe the process of speaking ...” january 6th, 2016 by webbrowan show more “the final aim of mcc is to allow implementation of rich smartphone web apps on an excess of cell phone devices, with a gorgeous user experience.” january 7th, 2016 by jamesmartin057 “by including more factors to score a person’s credit worthiness would make it ...” january 7th, 2016 by webbrowan “due to modernization we have found good technology advanced in the society; ...” january 7th, 2016 by edwardmiller “regardless of the nature of any business, finance is the key factor that helps ...” january 7th, 2016 by webbrowan “such comparison tools are great for borrowers to perform an estimated ...” january 7th, 2016 by webbrowan “there are clearly a number of changes coming for small businesses as they await ...” january 6th, 2016 by nguyennga “when it concerns money, you cannot afford to commit even a single mistake and ...” january 6th, 2016 by webbrowan “social collaboration seems like a funny way to describe the process of speaking ...” january 6th, 2016 by webbrowan “this is such a scary article. no doubt there is a lot of things that you have ...” january 6th, 2016 by webbrowan “thanks for the insight connie!” january 4th, 2016 by daniel matthews show more popular * read * commented * authors h z - points john staunton - points jan rubio - points sarma ranga - points dawn bronkema - points in memoriam: robin fray carey (1) 1/6/2016 by sdc staff 72% of people aren’t familiar with hosted voip (0) 1/4/2016 by josh rose how to recycle old it equipment and keep your data safe (0) 1/4/2016 by ryan kidman 5 predictions for trends in data, analytics and machine learning in 2016 (0) 1/4/2016 by bruce robbins will 2016 be the year you clean up your dirty data? (0) 1/5/2016 by martin doyle 3 ingenious use cases for business intelligence tools 1/11/2016 by eran levy 3 perks of an office 365 migration that you didn’t know about 1/12/2016 by kyle cebull nosql databases: 4 game-changing use cases 1/13/2016 by jim scott flash vulnerabilities & exploits: an information security primer 1/12/2016 by thu pham nosql and the internet of things 1/14/2016 by jim scott advisory panel about the panel aaron aders more » bruno aziza more » stephen baker more » paul barsch more » gary cokins more » jill dyché more » themos kalafatis more » bernard marr more » james taylor more » smartdata collective * audio archive: the cutting edge of bigdata monetization * audio archive: delivering value through business analytics * audio archive: trendspotting for growth * audio archive: finding value in unstructured social data * audio archive: risks and rewards of bigdata in the cloud * audio archive: putting customer value to work categories » analytics » best practices » bigdata » book review » business intelligence » business rules » cloud computing » collaborative data » commentary » crm » culture/leadership » data management » data mining » data quality » data visualization » data warehousing » decision management » ebook » hadoop » hardware » inside companies » it » jobs » knowledge management » location » mapreduce » market research » marketing » marketing automation » mobility » modeling » new products » news » open source » podcasts » policy and governance » predictive analytics » privacy » r programming language » risk management » security » sentiment analytics » smartdata collective exclusive » social data » social media analytics » software » sql » statistics » text analytics » transparency » unstructured data » web analytics » webcasts » workforce analytics » workforce data show more featured contributors * you * melanie aizer * paul barsch * meta s. brown * jason burke * gary cokins * ted cuzzillo * barry devlin * chris dixon * erica driver * michael fauscette * bill franks * david hecht * julie hunt * lachlan james * doug lautzenheiser * alex olesker * david smith * james taylor * daniel tunkelang * bigdata smart data cloud social data unstructured data data in the cloud business intelligence analytics social media today llc © 2012 terms of use | community guidelines | privacy policy | get a badge | help | about & contact quantcast #rss feed a social media today community [network-drop-down.png] * [brand-bar-wheel.png] about social media today ______________________________________________________________ * [logo.png] the world's best thinkers on energy & climate ______________________________________________________________ * [logo.png] the world's best thinkers on the urban future ______________________________________________________________ * [logo.png] the world's best thinkers on business intelligence & data analytics ______________________________________________________________ * [logo.png] the world's best thinkers on healthcare ______________________________________________________________ * [logo_0.png] the world's best thinkers on social media sign up | login with → smartdata collective the world's best thinkers on data * home + about us & contact + work with us * post here * analytics + social media analytics + text analytics + sentiment analytics + predictive analytics + modeling + web analytics + workforce analytics * bigdata + social data + data warehousing + visualization + workforce data + data mining + data quality * business intel + business rules + decision management + knowledge management + market research + crm * data management + risk management + transparency + culture & leadership + policy & governance + best practices + privacy * it + mobility + security + hardware + location + cloud computing * software + hadoop + mapreduce + sql + open source + marketing automation + r programming language * reports * webcasts * help + faq posted by: [screen%20shot%202013-06-06%20at%2015.18.37.png] bernard marr posts profile comments follow please login or register to follow this user. bigdata: the amazing numbers in 2015 like it? 294 comments 0 posted june 23, 2015 keywords: the bigdata guru image bigdata is growing — in fact, the sector is growing so fast and we are producing data so voraciously, that no one can afford to ignore it as a “fad” any more. and, it’s going to affect all companies, large and small, across all segments of the market — from healthcare to public safety, and retail to wholesale. bigdata is changing the world as we know it. if you don’t believe me, check out some of these (frankly staggering) statistics: * 60% one study suggests that retailers who fully leverage the power of bigdata could see an increase in their operating margins by as much as 60 percent! (source) * 4.4 million that’s how many it jobs there will be internationally in the big data field by the end of 2015, and it’s estimated that 1.9 million of these jobs will be in the u.s. alone. it also suggests that there will be a large gap between the number of jobs available and the number of skilled workers to fill those jobs. (source) * $3.4 billion the advanced and predictive analytics (apa) software market is expected to grow to $3.4 billion worldwide by 2018. (source) * $8 million in one study, the average business expected to spend $8 million this year on bigdata and related projects. (source) * $50.1 billion bigdata was a $28.5b market in 2014, growing to $50.1 billion overall in 2015. (source) * 5 times more likely companies that use bigdata analytics are five times more likely to make decisions “much faster” than their competition. (source) * $65.7 million as little as a 10 percent increase in the accessibility of data can mean an additional $65.7 million net income for a typical fortune 1000 company. (source) * $200 million the obama administration is investing more than $200 million in big data initiatives. (source) * 90 percent ninety percent of the data in existence was created in the last two years. (source) * $3.57 billion customer experience enhancement is expected to be the largest big data business category, and the one with the most growth, with forecasts saying this sector will grow from $0.75 billion in 2015 to $3.57 billion in 2020. * 26 hours macy’s has reportedly been able to save 26 hours every time it optimizes pricing for its 73 million products through use of big data, allowing them to change pricing more frequently to follow retail trends. (source) * $500 billion one study suggests that the u.s. government could save $500 billion annually by implementing bigdata projects. (source) studies also show that companies (especially small and medium sized companies) put cost and personnel problems as the top reasons they haven’t implemented bigdata projects yet, but these projects need to become priority fast, or those companies risk being left behind. as always, thank you very much for reading my posts. you might also be interested in my new book: bigdata: using smart bigdata, analytics and metrics to make better decisions and improve performance you can read a free sample chapter here. * login or register to post comments * [screen%20shot%202013-06-06%20at%2015.18.37.png] connect: twitter linkedin facebook website authored by: bernard marr bernard marr is a globally regognized bigdata and analytics expert. he is a best-selling business author, keynote speaker and consultant in strategy, performance management, analytics, kpis and bigdata. he helps companies to better manage, measure, report and analyse performance. his leading-edge work with major companies, organisations and governments across the globe makes him a globally ... see complete profile would you like to contribute to this site? get started » other posts by bernard marr bigdata in big oil: the amazing ways shell uses analytics to drive business success - november 17, 2015 3 ways the internet of things will change every business - november 10, 2015 bigdata in motorsports: how f1 and nascar compete on analytics - october 13, 2015 4 ways bigdata will change every business - october 6, 2015 of bigdata, the iot, and trust: tales from google's nest - september 29, 2015 related posts there are no posts that fit this criteria. image the moderated business community for business intelligence, predictive analytics, and data professionals. image * twitter * iframe: http://platform.twitter.com/widgets/follow_button.html?screen_name= smartdataco * rss * rss as xml addtomyyahoo4 subscribe with bloglines add to google * email * get our best posts in your inbox weekly first name ____________________ last name ____________________ email address ____________________ sign up! how do you innovate effectively and maintain a competive edge? image learn how in our exlcusive ebook, "bad data need not apply: designing the modern data warehouse environment." download here. sdc special columns big-data-guru riksy business big-data-mops events no events listed add your own! recommended to follow ahamed parpia follow a t follow jan henderyckx follow brent leary follow orlando doehring follow daniel kraft follow comments * featured * popular * recent “the final aim of mcc is to allow implementation of rich smartphone web apps on an excess of cell phone devices, with a gorgeous user experience.” january 7th, 2016 by jamesmartin057 “by including more factors to score a person’s credit worthiness would make it even tougher for borrowers to obtain a loan approval. this in turn causes the finance industry to face a fluctuating business trend. there are definitely pros and cons to this major change in a loan application process but one thing for sure is to ensure borrowers shall only be allowed to take up a loan which is ...” january 7th, 2016 by webbrowan “thanks for the insight connie!” january 4th, 2016 by daniel matthews “there are clearly a number of changes coming for small businesses as they await ...” january 6th, 2016 by nguyennga “very useful tips, especially from the point of view of customer service.” january 4th, 2016 by isabella1977 “this is such a scary article. no doubt there is a lot of things that you have ...” january 6th, 2016 by webbrowan “the final aim of mcc is to allow implementation of rich smartphone web apps on ...” january 7th, 2016 by jamesmartin057 “regardless of the nature of any business, finance is the key factor that helps ...” january 7th, 2016 by webbrowan “such comparison tools are great for borrowers to perform an estimated ...” january 7th, 2016 by webbrowan “when it concerns money, you cannot afford to commit even a single mistake and ...” january 6th, 2016 by webbrowan “due to modernization we have found good technology advanced in the society; ...” january 7th, 2016 by edwardmiller “social collaboration seems like a funny way to describe the process of speaking ...” january 6th, 2016 by webbrowan show more “the final aim of mcc is to allow implementation of rich smartphone web apps on an excess of cell phone devices, with a gorgeous user experience.” january 7th, 2016 by jamesmartin057 “by including more factors to score a person’s credit worthiness would make it ...” january 7th, 2016 by webbrowan “due to modernization we have found good technology advanced in the society; ...” january 7th, 2016 by edwardmiller “regardless of the nature of any business, finance is the key factor that helps ...” january 7th, 2016 by webbrowan “such comparison tools are great for borrowers to perform an estimated ...” january 7th, 2016 by webbrowan “there are clearly a number of changes coming for small businesses as they await ...” january 6th, 2016 by nguyennga “when it concerns money, you cannot afford to commit even a single mistake and ...” january 6th, 2016 by webbrowan “social collaboration seems like a funny way to describe the process of speaking ...” january 6th, 2016 by webbrowan “this is such a scary article. no doubt there is a lot of things that you have ...” january 6th, 2016 by webbrowan “thanks for the insight connie!” january 4th, 2016 by daniel matthews show more popular * read * commented * authors h z - points john staunton - points jan rubio - points sarma ranga - points dawn bronkema - points in memoriam: robin fray carey (1) 1/6/2016 by sdc staff 72% of people aren’t familiar with hosted voip (0) 1/4/2016 by josh rose how to recycle old it equipment and keep your data safe (0) 1/4/2016 by ryan kidman 5 predictions for trends in data, analytics and machine learning in 2016 (0) 1/4/2016 by bruce robbins will 2016 be the year you clean up your dirty data? (0) 1/5/2016 by martin doyle 3 ingenious use cases for business intelligence tools 1/11/2016 by eran levy 3 perks of an office 365 migration that you didn’t know about 1/12/2016 by kyle cebull nosql databases: 4 game-changing use cases 1/13/2016 by jim scott flash vulnerabilities & exploits: an information security primer 1/12/2016 by thu pham nosql and the internet of things 1/14/2016 by jim scott advisory panel about the panel aaron aders more » bruno aziza more » stephen baker more » paul barsch more » gary cokins more » jill dyché more » themos kalafatis more » bernard marr more » james taylor more » smartdata collective * audio archive: the cutting edge of bigdata monetization * audio archive: delivering value through business analytics * audio archive: trendspotting for growth * audio archive: finding value in unstructured social data * audio archive: risks and rewards of bigdata in the cloud * audio archive: putting customer value to work categories » analytics » best practices » bigdata » book review » business intelligence » business rules » cloud computing » collaborative data » commentary » crm » culture/leadership » data management » data mining » data quality » data visualization » data warehousing » decision management » ebook » hadoop » hardware » inside companies » it » jobs » knowledge management » location » mapreduce » market research » marketing » marketing automation » mobility » modeling » new products » news » open source » podcasts » policy and governance » predictive analytics » privacy » r programming language » risk management » security » sentiment analytics » smartdata collective exclusive » social data » social media analytics » software » sql » statistics » text analytics » transparency » unstructured data » web analytics » webcasts » workforce analytics » workforce data show more featured contributors * you * melanie aizer * paul barsch * meta s. brown * jason burke * gary cokins * ted cuzzillo * barry devlin * chris dixon * erica driver * michael fauscette * bill franks * david hecht * julie hunt * lachlan james * doug lautzenheiser * alex olesker * david smith * james taylor * daniel tunkelang * bigdata smart data cloud social data unstructured data data in the cloud business intelligence analytics social media today llc © 2012 terms of use | community guidelines | privacy policy | get a badge | help | about & contact quantcast #rss feed a social media today community [network-drop-down.png] * [brand-bar-wheel.png] about social media today ______________________________________________________________ * [logo.png] the world's best thinkers on energy & climate ______________________________________________________________ * [logo.png] the world's best thinkers on the urban future ______________________________________________________________ * [logo.png] the world's best thinkers on business intelligence & data analytics ______________________________________________________________ * [logo.png] the world's best thinkers on healthcare ______________________________________________________________ * [logo_0.png] the world's best thinkers on social media sign up | login with → smartdata collective the world's best thinkers on data * home + about us & contact + work with us * post here * analytics + social media analytics + text analytics + sentiment analytics + predictive analytics + modeling + web analytics + workforce analytics * bigdata + social data + data warehousing + visualization + workforce data + data mining + data quality * business intel + business rules + decision management + knowledge management + market research + crm * data management + risk management + transparency + culture & leadership + policy & governance + best practices + privacy * it + mobility + security + hardware + location + cloud computing * software + hadoop + mapreduce + sql + open source + marketing automation + r programming language * reports * webcasts * help + faq posted by: [screen%20shot%202013-06-06%20at%2015.18.37.png] bernard marr posts profile comments follow please login or register to follow this user. amazon: using bigdata analytics to read your mind like it? 953 comments 2 posted february 5, 2014 keywords: analytics, the bigdata guru, data mining, predictive analytics, modeling, bigdata, business intelligence, inside companies, smartdata collective exclusive image amazon.com, the seattle-based ecommerce giant, has always leveraged data. in one of their latest business moves, the company has obtained a patent to ship us goods before we have even made a decision to buy it, purely based on their predictive bigdata analytics. i don’t think that back in 1995 when jeff bezos started the company in a garage, he could have imagined that it would one day grow into a fortune 500 global retail empire. i believe that the key building blocks of amazon’s success are their ability to use data and an eye for the right innovations and patents. in the early days, when amazon was primarily a book retailer, the company was the first to extensively use algorithms so that it could provide recommendations for customers: “customers who bought this item, also bought this one…”. today, it uses item-to-item collaborative filtering on many data points such as what users have bought before, what they have in their virtual shopping card or wish list, the items they have rated and reviewed, as well as what other similar users have bought, to heavily customize the customer browsing experience. another big coup for amazon was when it obtained the patent for it’s ‘one click buy’ feature. this was pure genius and who would have thought a company could ever get a patent for that. what amazon has just done is combine the two (strengths in data analytics and it’s instinct for patenting key features) to obtain a patent for what it calls: anticipatory shipping. what amazon has patented here is the process of shipping an item to a customer in anticipation that this customer will order that product. this means that amazon believes the bigdata analytics insights will become so accurate that it can predict who will order what and when. the reason for this is that amazon wants to be able to deliver products faster. this is also why it negotiated sunday deliveries and why amazon started to experiment with unmanned drones that might deliver our parcels in the future and. other, more traditional retailers have long used predictive analytics to ensure the right items are in stock, based on past buying patterns as well as social media analytics and weather predictions. what is new here is that amazon is taking it to a personal level, predicting the items you might buy. this is different to a local supermarket stocking items that the people in that community might want to buy. one problem with anticipatory shipping is that amazon has to get it right. if their bigdata algorithms get it wrong, then it could potentially lose the company a lot of money because the logistics costs for shipping the product out and then returning it would be lost. the way amazon proposes to deal with cheaper unwanted items is to either heavily discount them or give them away as a free gift to build customer ‘good will’. another problem with anticipatory shipping is the question about how much a company should be allowed to act on the insights gained from analysing our personal behaviours. for example, my wife bought a pregnancy swimsuit from amazon as a present for one of her friends who was expecting a baby. the problem that followed was that for the following 9 months or so she had to look at pregnancy related recommendations or watch pregnancy related ads. just imagine if she had to return all those diapers, baby blankets or baby wipes that a predictive anticipatory shipping algorithm might send in the future! as a bigdata guy, i am fascinated by the increasingly accurate predictions commercial companies can make about our behaviours. as a consumer i am excited about the prospect that the stuff i order will be with me quicker, because it will already be on its way before i place my order. but as a private individual, i am getting a little concerned about the power predictive analytics puts into the hands of commercial companies. ----- check out the other posts in the bigdata guru column and feel free to connect via twitter, linkedin, facebook and the advanced performance institute. * login or register to post comments * [screen%20shot%202013-06-06%20at%2015.18.37.png] connect: twitter linkedin facebook website authored by: bernard marr bernard marr is a globally regognized bigdata and analytics expert. he is a best-selling business author, keynote speaker and consultant in strategy, performance management, analytics, kpis and bigdata. he helps companies to better manage, measure, report and analyse performance. his leading-edge work with major companies, organisations and governments across the globe makes him a globally ... see complete profile would you like to contribute to this site? get started » other posts by bernard marr bigdata in big oil: the amazing ways shell uses analytics to drive business success - november 17, 2015 3 ways the internet of things will change every business - november 10, 2015 bigdata in motorsports: how f1 and nascar compete on analytics - october 13, 2015 4 ways bigdata will change every business - october 6, 2015 of bigdata, the iot, and trust: tales from google's nest - september 29, 2015 related posts what’s in store for bigdata analytics in 2016 january 14, 2016 by stevesarsfield how iot applications can help businesses grow internally january 7, 2016 by xanderscho 5 predictions for trends in data, analytics and machine learning in 2016 january 4, 2016 by bruce robbins 72% of people aren’t familiar with hosted voip january 4, 2016 by josh rose the unlikely marriage of cloud computing and multi-level marketing december 30, 2015 by philcohen4 » already a member? login now to comment! » not a member? register to comment! like it? 795 february 25, 2014 ali mahmoud says: interesting read bernard. at first glance i was incredibly skeptical and couldn't see amazon implementing this anytime in the near future. it sounded like a placeholder patent (with the bonus of some press coverage.) the more i think about it i can picture amazon rolling this out on the heaviest of users and the most predictable buying habits. i agree that their analytics are not perfect (i keep telling my wife not to use my account as it messes up all the recommendations) but only amazon knows their own customers purchase patterns. i am a light to medium volume customer but i have to imagine that the top 1% of prime customers are generating loads of purchase data and buying habits. for example there could be customers that purchase novels as soon as they come out and amazon predictively ships it as soon as its available. i don't think this is a case of someone purchasing one maternity bathing suit for a friend and amazon losing millions shipping maternity clothes. we all know that they are much smarter than that. i think this is a clever strategy to maintain a fierce grip on their top 1% of customers. those early adopters and loudest champions. it could be a brilliant marketing ploy. if anyone can do it, it is definitely amazon. share this comment: twitter facebook linkedin * login or register to post comments like it? 926 february 7, 2014 sergei veinberg says: spot on analysis bernard; and this might work in us. but i would like to add an australian perspective. from what anecdotal evidence that i have from my friends using amazon, their marketing, is much too pervasive. people are also spooked by ads, like the pregnancy-related one you mention. how people will respond if they find an actual 'automatically pre-ordered' item rather than just an ad. also, amazon tracks people by ip address... so the whole household has the same buying habits as far as they are concerned. big no-no. i get the feel and please correct me if i am wrong, that what amazon gets in number crunching power, they lack in 'human retail' experience. and that will come to bite them one day. share this comment: twitter facebook linkedin * login or register to post comments please note: your first comment as a registered user will be held for moderation up to 24 hours (usually less). for more information about comments on our site, please read our faq and terms of use. image the moderated business community for business intelligence, predictive analytics, and data professionals. image * twitter * iframe: http://platform.twitter.com/widgets/follow_button.html?screen_name= smartdataco * rss * rss as xml addtomyyahoo4 subscribe with bloglines add to google * email * get our best posts in your inbox weekly first name ____________________ last name ____________________ email address ____________________ sign up! how do you innovate effectively and maintain a competive edge? image learn how in our exlcusive ebook, "bad data need not apply: designing the modern data warehouse environment." download here. sdc special columns big-data-guru riksy business big-data-mops events no events listed add your own! recommended to follow ahamed parpia follow a t follow jan henderyckx follow brent leary follow orlando doehring follow daniel kraft follow comments * featured * popular * recent “the final aim of mcc is to allow implementation of rich smartphone web apps on an excess of cell phone devices, with a gorgeous user experience.” january 7th, 2016 by jamesmartin057 “by including more factors to score a person’s credit worthiness would make it even tougher for borrowers to obtain a loan approval. this in turn causes the finance industry to face a fluctuating business trend. there are definitely pros and cons to this major change in a loan application process but one thing for sure is to ensure borrowers shall only be allowed to take up a loan which is ...” january 7th, 2016 by webbrowan “thanks for the insight connie!” january 4th, 2016 by daniel matthews “there are clearly a number of changes coming for small businesses as they await ...” january 6th, 2016 by nguyennga “very useful tips, especially from the point of view of customer service.” january 4th, 2016 by isabella1977 “this is such a scary article. no doubt there is a lot of things that you have ...” january 6th, 2016 by webbrowan “the final aim of mcc is to allow implementation of rich smartphone web apps on ...” january 7th, 2016 by jamesmartin057 “regardless of the nature of any business, finance is the key factor that helps ...” january 7th, 2016 by webbrowan “such comparison tools are great for borrowers to perform an estimated ...” january 7th, 2016 by webbrowan “when it concerns money, you cannot afford to commit even a single mistake and ...” january 6th, 2016 by webbrowan “due to modernization we have found good technology advanced in the society; ...” january 7th, 2016 by edwardmiller “social collaboration seems like a funny way to describe the process of speaking ...” january 6th, 2016 by webbrowan show more “the final aim of mcc is to allow implementation of rich smartphone web apps on an excess of cell phone devices, with a gorgeous user experience.” january 7th, 2016 by jamesmartin057 “by including more factors to score a person’s credit worthiness would make it ...” january 7th, 2016 by webbrowan “due to modernization we have found good technology advanced in the society; ...” january 7th, 2016 by edwardmiller “regardless of the nature of any business, finance is the key factor that helps ...” january 7th, 2016 by webbrowan “such comparison tools are great for borrowers to perform an estimated ...” january 7th, 2016 by webbrowan “there are clearly a number of changes coming for small businesses as they await ...” january 6th, 2016 by nguyennga “when it concerns money, you cannot afford to commit even a single mistake and ...” january 6th, 2016 by webbrowan “social collaboration seems like a funny way to describe the process of speaking ...” january 6th, 2016 by webbrowan “this is such a scary article. no doubt there is a lot of things that you have ...” january 6th, 2016 by webbrowan “thanks for the insight connie!” january 4th, 2016 by daniel matthews show more popular * read * commented * authors h z - points john staunton - points jan rubio - points sarma ranga - points dawn bronkema - points in memoriam: robin fray carey (1) 1/6/2016 by sdc staff 72% of people aren’t familiar with hosted voip (0) 1/4/2016 by josh rose how to recycle old it equipment and keep your data safe (0) 1/4/2016 by ryan kidman 5 predictions for trends in data, analytics and machine learning in 2016 (0) 1/4/2016 by bruce robbins will 2016 be the year you clean up your dirty data? (0) 1/5/2016 by martin doyle 3 ingenious use cases for business intelligence tools 1/11/2016 by eran levy 3 perks of an office 365 migration that you didn’t know about 1/12/2016 by kyle cebull nosql databases: 4 game-changing use cases 1/13/2016 by jim scott flash vulnerabilities & exploits: an information security primer 1/12/2016 by thu pham nosql and the internet of things 1/14/2016 by jim scott advisory panel about the panel aaron aders more » bruno aziza more » stephen baker more » paul barsch more » gary cokins more » jill dyché more » themos kalafatis more » bernard marr more » james taylor more » smartdata collective * audio archive: the cutting edge of bigdata monetization * audio archive: delivering value through business analytics * audio archive: trendspotting for growth * audio archive: finding value in unstructured social data * audio archive: risks and rewards of bigdata in the cloud * audio archive: putting customer value to work categories » analytics » best practices » bigdata » book review » business intelligence » business rules » cloud computing » collaborative data » commentary » crm » culture/leadership » data management » data mining » data quality » data visualization » data warehousing » decision management » ebook » hadoop » hardware » inside companies » it » jobs » knowledge management » location » mapreduce » market research » marketing » marketing automation » mobility » modeling » new products » news » open source » podcasts » policy and governance » predictive analytics » privacy » r programming language » risk management » security » sentiment analytics » smartdata collective exclusive » social data » social media analytics » software » sql » statistics » text analytics » transparency » unstructured data » web analytics » webcasts » workforce analytics » workforce data show more featured contributors * you * melanie aizer * paul barsch * meta s. brown * jason burke * gary cokins * ted cuzzillo * barry devlin * chris dixon * erica driver * michael fauscette * bill franks * david hecht * julie hunt * lachlan james * doug lautzenheiser * alex olesker * david smith * james taylor * daniel tunkelang * bigdata smart data cloud social data unstructured data data in the cloud business intelligence analytics social media today llc © 2012 terms of use | community guidelines | privacy policy | get a badge | help | about & contact quantcast #» feed » comments feed » argentina: future regional leader in big data? comments feed bigdata for the social sciences russian government supporting international cooperation in the bigdata field alternate alternate search wordpress.com menu * home * about * links search for: ____________________ search argentina: future regional leader in bigdata? written by dieter cavalleri, embassy of switzerland in argentina data-driven strategies in the market – the bigdata and analytics market is growing fast in argentina. after an education phase in 2013, bigdata is gaining increasing attention from the industry and the government, which consider it to be an important opportunity for business and research. buenos aires 1 as revealed by a survey of 1,139 it decision-makers in 2015, “interest in bigdata continues its steady rise. more than half (53%) of the respondents are currently implementing or plan to implement data-driven projects within their organizations in the next year — an increase of 6 percentage points from 2014 — and a further 8% are considering doing so. organizations’ approach to data-driven projects varies, sometimes significantly, depending on size and industry.” furthermore, nearly one in three respondents report that their organization’s data-driven initiatives have improved their decision-making and/or their planning and forecasting. however, companies are still scrambling to manage ongoing data growth even as they pursue projects designed to generate more value from the data they already have. for many, even keeping pace remains a challenge.[1] another limiting challenge is the growing “skill gap” as more and more employees with data analysis and data management skills are needed. [2] in general, there is a need for further trainings on “bigdata” in order to generate a critical mass of professionals in argentina.[3] despite these limitations, “bigdata” continues expanding to all the industries; from agriculture to banking, but also telecommunications with companies like telefónica de argentina which implemented ibm solutions (puredata system for analytics) to manage close to 20 million customers (fixes and mobile phones) and link them to data coming from the internet and social networks. for the city of buenos aires, the implementation of sap (systems, applications, and products for data processing) was a solution to pass from paper-management to automation. implemented four months ago, the platform hana helps manage city resources which exceed 700,000 assets, including street-lamps, parks, bus stops, drainage systems, buildings and bridges. thanks to hana, the city government can process in real time claims in social networks (through its twitter account) and link sensors information (located in the led street-lamps– with antennas transmitting the information to hana), as well as in drainage systems which measure rainfalls and flow rates) with cleaning plans.[4] “bigdata” national strategy work on the definition of a national strategy around “bigdata” began in 2013 with the first edition of the “jornadas de definición estratégica en bigdata” (strategic definition in bigdata days) organized by the national ministry of science, technology and productive innovation and the sadosky foundation of it investigation and development. the event intended to provide inputs for the development of a roadmap for the implementation of policies aimed at addressing national challenges in this area. it brought together a group of experts comprised of representatives from the government and from academic and business fields.[5] this event allowed confirming the existence in argentina, of a group of relevant stakeholders that are currently working on this theme; the availability of inputs and resources such as data and communication infrastructure; and the need for enhanced efforts in the formation of new professionals. the working group also identified the need for incentives and projects from the national state which could drive innovation, and therefore boost emerging companies and generate new ones, while also allowing for synergies between fields to take place. as the development of “bigdata” is considered a question of national and regional sovereignty, the argentinian national strategy currently aims at transforming the country into a relevant actor in this field within the next five years. despite its growing importance in the world, bigdata remains in early development stages, which makes it easier for argentina to position itself as a regional leader in the big data revolution. the result of this encounter led to the adoption of a “strategic roadmap for data science in argentina 2013-2018”, which includes all relevant stakeholders. this strategic roadmap involves building a data platform comprised of local data and capacities in order to develop a school of thought in relation to bigdata. the construction of this platform will strengthen the advances in science and technologies, enable the development of services for the economy and the society, and lead to growth and creation of new knowledge companies and of international market competitiveness. this platform should necessarily bring together stakeholders from the academic, private and public sectors, since bigdata is naturally multidisciplinary. the originality and unicity of this platform lies in the use of its own data and in its specific objectives and applications stemming directly from the argentine reality. an important part of this platform will be comprised of geolocated data coming from strategic areas for the country (natural resources, economic activity, health, education, security, defense, innovation…) to which biological, bioinformatics data and e-government data will be added, while preserving in all circumstances, their privacy and security. this will be combined with the promotion of institutional and infrastructure development of training centers in order to effectively reduce the “skills gap” currently present in the “bigdata” market. other drafted initiatives consist in creating a national bioinformatics platform, a national centre for data science, a national cloud, a cluster of “bigdata” companies, data science laboratories, as well as a postgraduate training in data science (as currently the university of buenos aires only offers a master degree in data mining and knowledge discovery). from the projects which have already been implemented, it is worth mentioning the creation of the “palenque platform” by the sadosky foundation; a geolocated agro-data space in argentina – the single one based on a public platform. it is meant to provide technological solutions to agricultural producers based on bigdata, and to help them increase their productivity through precision farming (sensors, smart planting and irrigation etc.) and weather forecasts.[6] the sadosky foundation is also pursuing other initiatives with companies from different sectors, such as the banco hipotecario (mortgage bank). they recently signed a convention for collaboration in bigdata research, which will have two lines of action: provide banking services to more sectors of the population and establish optimized decision models for credits cards customers. currently it has close to one million customers handling more than 10 million transactions per month, from which many data can be extracted. open data the experience of open government was first implemented in the city of buenos aires in early 2012 through its web portal “buenos aires data” – where it publishes all its digital content under a creative common license which allows it to be used for free with proper attribution. the open data catalogue allows users to visualize the data, examine apps that have been created using the data and even includes a design lab for posting app ideas. the government publishes substantial data sets on diverse topics such as health, education, transit, and culture among others. as of january 2015, the portal contained 111 datasets and at least three application programming interfaces – including one, which measures traffic loads on the motorway in real time. [7] the website also embodies the principles of openness in its design as it was built implementing ckan, an open source data management system, and its code is available for reuse via github.[8] buenos aires 2 figure 1: buenos aires open data catalogue since the implementation of this program, various strategies have been carried out for the co-creation and development of tools to support civic, democratic and economic engagement. to promote the use of the data, the open government team has organized hackathons (bahackaton) and mobile apps competitions (ba apps ba).[9] the events drew many developers, but also participants from media and civil society, creating therefore a real community. the government of argentina has also indicated its willingness to apply the principles of open government. in order to achieve this objective, it created a working group on open government which formalized the creation of a national portal of public data in 2013.[10] the working group is currently working on a national action plan 2015-2017.[11] _________ [1] http://www.cxo2cio.com/2015/07/2015-big-data-analytics-survey-insights. html [2] ibid. [3] http://triton.exp.dc.uba.ar/datamining/files/charlas_y_paneles/p1_efeue rstein.pdf [4] el cronista. “it business, el poder de la economía de datos”, 18 august 2015. [5] http://www.fundacionsadosky.org.ar/grandes-datos-grandes-oportunidades/ [6] http://triton.exp.dc.uba.ar/datamining/files/charlas_y_paneles/p1_efeue rstein.pdf [7] http://www.opendataresearch.org/dl/symposium2015/odrs2015-paper28.pdf [8]http://www.gobiernolocal.gob.ar/sites/default/files/jolias_lucas_ope n_data_a_nivel_local_en_argentina_transparencia_o_innovacion_democratic a_panel_023.pdf [9]http://www.gobiernolocal.gob.ar/sites/default/files/jolias_lucas_ope n_data_a_nivel_local_en_argentina_transparencia_o_innovacion_democratic a_panel_023.pdf [10] http://blogs.lanacion.com.ar/data/datasets/se-presento-el-portal-nacion al-de-datos-publicos/ [11] http://www.gobiernoabierto.gob.ar/novedades/participa-del-ii-plan-de-ac cion-2015-2017-de-la-republica-argentina_n249 share this: * twitter * facebook * google * like this: like loading... related posted in bigdata, internet of things, open data, smart data and tagged data platform, hana, national bioinformatics platform, national centre for data science on august 26, 2015 by statementglobal. leave a comment post navigation ← bigdata for the social sciences russian government supporting international cooperation in the big data field → leave a reply cancel reply enter your comment here... ____________________________________________________________ ____________________________________________________________ ____________________________________________________________ ____________________________________________________________ fill in your details below or click an icon to log in: * * * * * iframe: googleplus-sign-in gravatar email (required) (address never made public) ____________________ name (required) ____________________ website ____________________ wordpress.com logo you are commenting using your wordpress.com account. ( log out / change ) twitter picture you are commenting using your twitter account. ( log out / change ) facebook photo you are commenting using your facebook account. ( log out / change ) google+ photo you are commenting using your google+ account. ( log out / change ) cancel connecting to %s [ ] notify me of new comments via email. [ ] notify me of new posts via email. post comment search search for: ____________________ search calendar caption: august 2015 m t w t f s s « jul sep » 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 recent articles * a bigdata primer november 17, 2015 * “telecom paristech’s “bigdata” post-master’s is the first program of this kind to have been created in france.” november 4, 2015 follow blog join 41 other followers ____________________ follow us contact state secretariat for education, research and innovation globalstatement(at)sbfi.admin.ch blog at wordpress.com. the suits theme. follow follow “” get every new post delivered to your inbox. join 41 other followers ____________________ sign me up build a website with wordpress.com iframe: likes-master %d bloggers like this: 403 forbidden __________________________________________________________________ nginx #publisher alternate close skip to main content sign in * saved for later * comment activity * edit profile * email preferences * change password * sign out subscribe search dating more from the guardian: * dating * jobs change edition: * switch to the uk edition switch to the us edition switch to the au edition international * switch to the uk edition * switch to the us edition * switch to the australia edition the guardian * home * home * uk * world * sport * football * opinion * culture * business * lifestyle * fashion * environment * tech * travel browse all sections close higher education network bigdata in universities how bigdata has transformed research we speak to the academics behind four inspiring projects to see how big data is being used to develop and enhance research claire shaw interviewed by claire shaw @clurshaw email friday 10 july 2015 07.45 bst last modified on friday 10 july 2015 10.26 bst * share on facebook * share on twitter * share via email * share on pinterest * share on linkedin * share on google+ * share on whatsapp illustris simulation facebook twitter pinterest the picture shows dark matter density overlaid with the gas velocity field. photograph: illustris simulation shy genel, hubble fellow at the astronomy department of columbia university, us describe your research project: illustris is a computer simulation of the evolution of the universe, through which we study how galaxies and their constituent stars and black holes form and evolve over cosmic time. iframe: https://www.youtube.com/embed/gpynqagzfr0 illustris simulation of the universe. how did you involve bigdata? for the project we had to simulate as large a portion of the universe as possible and resolve each individual galaxy in this simulated universe in as much detail as possible. the calculation was performed on a supercomputer where 8,192 processors shared the memory and the computations between them, constantly communicating and reporting to each other on their respective up-to-date results. the calculation also produced a big amount of data, over 300,000 gigabytes, to be stored for long term, which requires hundreds of hard drives to be joined together into a single huge virtual storage space. what challenges did you face? performing the simulation on thousands of processors in parallel was difficult. it requires sophisticated calculations as well as a strong and stable supercomputer, of which there are only a few in the world. what do you think the future holds for bigdata and research? bigdata is allowing us to build more realistic models of various systems in nature. as bigdata capabilities improve, we will be able to connect those different scales in a more self-consistent way – these advances are guaranteed to provide us with new insights into the workings of nature. science lab facebook twitter pinterest researchers at the institute for computational health sciences are looking at the difference between cancer and normal to help provide changes in clinical treatment given today to cancer patients. photograph: stefan wermuth/reuters idit kosti, postdoctoral fellow at the institute for computational health sciences, university of california, san francisco describe your research project: i’m working on improving clinical treatment and basic biological knowledge in various types of cancer tumors using bigdata. how did you involve bigdata? bigdata, in my case, is gene and protein expression levels coming from cancer tumors and normal tissues from patients and healthy volunteers. using this information i’m able to study the difference between cancer and normal, verify my hypothesis, create models and suggest changes in clinical treatment given today to cancer patients. the remarkable part is that dna and rna sequencing (the technology used for gene expression) has been around since the 70s, but the method we use today, next generation sequencing, was fully developed only around 2004 and became commercially available around 2006-2007. the development of this technology is the only reason we have access to the large amount of data. what challenges did you face? first challenge is finding data sources – patients and healthy volunteers. for that we need either patient or family consent, depending on data type. second challenge is collecting the data. this process is an expensive and time-consuming one, and requires not only the data collection, but also reducing noise levels coming from both technology limitations and natural human variation. what do you think the future holds for bigdata and research? a future with medical and biological bigdata holds a great promise for human health and research. i imagine a world where our genetic profiles, combined with physical measurements, is a key element in medical decisions: from simple ones, such as everyday dosage of headache pills using genetic profile, to complex ones such as designing cancer treatment by using the gene and protein expression taken from a tumour biopsy. yarn facebook twitter pinterest examples of online stories created by the community. simon popple, deputy head, school of media and communication, university of leeds describe your research project: yarn is a new community designed digital storytelling platform. it was produced through the ahrc-funded pararchive project and allows communities and organisations to create online stories and do research by sharing archive material, as well as their own interesting possessions. it does this by making institutional bigdata, such as museum collections, available for anyone to access. how did you involve bigdata? by partnering with the science museum group and bbc archives, we wanted to create connections between large collections of data held in traditionally closed archives and allow communities to explore, experience and be creative with these resources. organisations can directly make recourses available through the platform and allow people to incorporate them directly in their stories, providing illustrations, evidence and creative resources. what do you think the future holds for bigdata and research? bigdata is something of a catch-all concept and an obvious buzzword at the moment. it is a term often and simply defined in terms of scale rather than content, accessibility and intelligibility. for it to be really useful to communities it needs to be carefully translated and made relevant. data needs humanising and stories help to do that. mobileminer the picture shows how mobileminer can track a train journey from london to cardiff by using open source information on cell tower locations. tobias blanke, senior lecturer, department of digital humanities, king’s college london, uk describe your research project: the growing use of mobile and digital devices has led to a massive increase in the amount of data each of us generates, but until now, access to mobile data has been restricted to just a handful of companies and government agencies. and for this reason, we have little knowledge about the kinds of data that our mobile phones collect and transmit. the project – our data, ourselves (odo) – aims to reverse this. it makes the data and how it’s transmitted visible in order to develop new modes of citizen engagement, as well as new avenues for research and creativity. how did you involve bigdata? in order to trace and make visible the digital breadcrumbs we leave behind through our mobile devices, we have developed a mobile app called mobileminer, which tracks incoming and outgoing communications in mobile devices. for example, we can understand how mobile phones track the movement of people using cell tower location information or how gaming apps leak data to marketers around the world without our knowledge. if bigdata is increasingly collected by companies through everyday sensors and mobile phones, we want to show the flows of this data and reclaim ownership of our data. what challenges did you face? collaboration with those who produce the data was key to our research but also our biggest challenge; as using and understanding what happens to data requires both data and computer knowledge. we worked together with young rewired state, an organisation that teaches young people how to code through collaborative events. together we developed a toolkit to help others understand what happens to their data and how to take ownership of it. what do you think the future holds for bigdata and research? there is a lot of potential for understanding history and society through the bigdata we leave behind. an often-quoted example is the george w. bush presidential center holdings of 200 million white house emails of the second bush administration, which are only just becoming available for access under the us freedom of information act. it is here that bigdata can be useful to researchers in the humanities and social science, who can trace how emails appear around events and understand from their content the mindset of decision makers. the big data that business companies and state agencies collect from real-life sensors and mobile devices could also become an important object for research. the question remains, however, how to ensure access to this data, particularly when the institutions that collect it have little incentive to make it available for research. join the higher education network for more comment, analysis and job opportunities, direct to your inbox. follow us on twitter @gdnhighered. __________________________________________________________________ more features topics * research (higher education network) * academics * higher education * science (education) * bigdata * research (education) __________________________________________________________________ * share on facebook * share on twitter * share via email * share on pinterest * share on linkedin * share on google+ * share on whatsapp * reuse this content view all comments > comments sign in or create your guardian account to join the discussion. this discussion is closed for comments. we’re doing some maintenance right now. you can still read comments, but please come back later to add your own. commenting has been disabled for this account (why?) order by * newest * oldest * recommendations show 25 * 25 * 50 * 100 * all threads * collapsed * expanded * unthreaded loading comments… trouble loading? view more comments popular the guardian back to top * home * uk * world * sport * football * opinion * culture * business * lifestyle * fashion * environment * tech * travel all sections close * home * uk + education + media + society + law + scotland + wales + northern ireland * world + europe + us + americas + asia + australia + africa + middle east + cities + development * sport + football + cricket + rugby union + f1 + tennis + golf + cycling + boxing + racing + rugby league * football + live scores + tables + competitions + results + fixtures + clubs * opinion + columnists * culture + film + tv & radio + music + games + books + art & design + stage + classical * business + economics + banking + retail + markets + eurozone * lifestyle + food + health & fitness + love & sex + family + women + home & garden * fashion * environment + climate change + wildlife + energy + pollution * tech * travel + uk + europe + us + skiing * money + property + savings + pensions + borrowing + careers * science * professional networks * the observer * today's paper + editorials & letters + obituaries + g2 + weekend + the guide + saturday review * sunday's paper + comment + the new review + observer magazine * membership * crosswords + blog + editor + quick + cryptic + prize + quiptic + genius + speedy + everyman + azed * video * higher education network * › research iframe: /email/form/footer/37 * facebook * twitter * facebook * twitter * all topics * all contributors * solve technical issue * complaints & corrections * terms & conditions * privacy policy * cookie policy * securedrop © 2016 guardian news and media limited or its affiliated companies. all rights reserved. #publisher alternate close skip to main content sign in * saved for later * comment activity * edit profile * email preferences * change password * sign out subscribe search dating more from the guardian: * dating * jobs change edition: * switch to the uk edition switch to the us edition switch to the au edition international * switch to the uk edition * switch to the us edition * switch to the australia edition the guardian * home * home * uk * world * sport * football * opinion * culture * business * lifestyle * fashion * environment * tech * travel browse all sections close public leaders network bigdata and open data: what's what and why does it matter? both types of data can transform the world, but when government turns bigdata into open data it's especially powerful • what government can do with bigdata – livechat eso's visible and infrared survey telescope image showing helix nebula not all big, open data comes from government, such as scientists' research in astronomy. photograph: eso/vista/j. emerson / handout/epa joel gurin, new york university tuesday 15 april 2014 10.49 bst last modified on tuesday 3 june 2014 10.30 bst * share on facebook * share on twitter * share via email * share on pinterest * share on linkedin * share on google+ * share on whatsapp bigdata and the new phenomenon open data are closely related but they're not the same. open data brings a perspective that can make big data more useful, more democratic, and less threatening. while bigdata is defined by size, open data is defined by its use. big data is the term used to describe very large, complex, rapidly-changing datasets. but those judgments are subjective and dependent on technology: today's bigdata may not seem so big in a few years when data analysis and computing technology improve. open data is accessible public data that people, companies, and organisations can use to launch new ventures, analyse patterns and trends, make data-driven decisions, and solve complex problems. all definitions of open data include two basic features: the data must be publicly available for anyone to use, and it must be licensed in a way that allows for its reuse. open data should also be relatively easy to use, although there are gradations of "openness". and there's general agreement that open data should be available free of charge or at minimal cost. the relationship between bigdata and open data venn diagram showing the relationship between bigdata and open data source: joel gurin this venn diagram maps the relationship between bigdata and open data, and how they relate to the broad concept of open government. there are a few important points to note. bigdata that's not open is not democratic: section one of the diagram includes all kinds of bigdata that is kept from the public – like the data that large retailers hold on their customers, or national security data like that collected by the nsa. this kind of bigdata gives an advantage to the people who control it but may disempower the rest of us. it's this kind of bigdata that has become most controversial. open data doesn't have to be bigdata to matter: modest amounts of data, as shown in section four, can have a big impact when it is made public. data from local governments, for example, can help citizens participate in local budgeting, choose healthcare, analyse the quality of local services, or build apps that help people navigate public transport. big, open data doesn't have to come from government: this is shown in section three. more and more scientists are sharing their research in astronomy, genomics, and other areas in a new, collaborative research model. other researchers are using bigdata collected from social media – most of which is open to the public – to analyse public opinion and market trends. but, when the government turns bigdata into open data, it's especially powerful: government agencies have the capacity and funds to gather very large amounts of data (such as the us examples in section six), and opening up those datasets can have major economic benefits. i now direct the open data 500 study at the govlab at new york university. we've found 500 examples of us-based companies that are building their businesses on open government data, and much of that data is bigdata as well. we're now planning to work with open data institute to replicate our study in the uk, and i expect to find the same pattern. applying open data principles to bigdata can help solve some of the difficult issues that bigdata has raised. the biggest threat to public wellbeing is the risk that private, personal data can be collected and used as bigdata in ways the subjects of the data – namely, all of us – may not want or approve of. paradoxically, opening up this sensitive data, in a specific and controlled way, may actually make it more secure. the problem now is not only that government agencies and some businesses are collecting personal data about all of us; it's also that we as individuals don't know what's being collected and don't have access to the information about ourselves. if we knew more, we could control more. the uk government's midata initiative, which has encouraged businesses to share customer records with customers themselves, is part of the solution. similar us programmes, such as blue button for health records and green button for energy usage data, are also having a positive effect. both bigdata and open data can transform business, government, and society – and a combination of the two is especially potent. bigdata gives us unprecedented power to understand, analyse, and ultimately change the world we live in. open data ensures that power will be shared – and that the world we change will, with luck, become a fairer and more democratic one. joel gurin is senior advisor at govlab at new york university, where he directs the open data 500 study. he is author of open data now. • what government can do with bigdata – livechat • want your say? email us at public.leaders@theguardian.com. join the public leaders network for more comment, analysis and job opportunities, direct to your inbox. follow us on twitter via @guardianpublic. __________________________________________________________________ more blogposts topics * transparency * bigdata __________________________________________________________________ * share on facebook * share on twitter * share via email * share on pinterest * share on linkedin * share on google+ * share on whatsapp * reuse this content view all comments > comments sign in or create your guardian account to join the discussion. this discussion is closed for comments. we’re doing some maintenance right now. you can still read comments, but please come back later to add your own. commenting has been disabled for this account (why?) order by * newest * oldest * recommendations show 25 * 25 * 50 * 100 * all threads * collapsed * expanded * unthreaded loading comments… trouble loading? view more comments popular the guardian back to top * home * uk * world * sport * football * opinion * culture * business * lifestyle * fashion * environment * tech * travel all sections close * home * uk + education + media + society + law + scotland + wales + northern ireland * world + europe + us + americas + asia + australia + africa + middle east + cities + development * sport + football + cricket + rugby union + f1 + tennis + golf + cycling + boxing + racing + rugby league * football + live scores + tables + competitions + results + fixtures + clubs * opinion + columnists * culture + film + tv & radio + music + games + books + art & design + stage + classical * business + economics + banking + retail + markets + eurozone * lifestyle + food + health & fitness + love & sex + family + women + home & garden * fashion * environment + climate change + wildlife + energy + pollution * tech * travel + uk + europe + us + skiing * money + property + savings + pensions + borrowing + careers * science * professional networks * the observer * today's paper + editorials & letters + obituaries + g2 + weekend + the guide + saturday review * sunday's paper + comment + the new review + observer magazine * membership * crosswords + blog + editor + quick + cryptic + prize + quiptic + genius + speedy + everyman + azed * video * public leaders network * › transparency iframe: /email/form/footer/37 * facebook * twitter * facebook * twitter * all topics * all contributors * solve technical issue * complaints & corrections * terms & conditions * privacy policy * cookie policy * securedrop © 2016 guardian news and media limited or its affiliated companies. all rights reserved. #publisher alternate close skip to main content sign in * saved for later * comment activity * edit profile * email preferences * change password * sign out subscribe search dating more from the guardian: * dating * jobs change edition: * switch to the uk edition switch to the us edition switch to the au edition international * switch to the uk edition * switch to the us edition * switch to the australia edition the guardian * home * home * uk * world * sport * football * opinion * culture * business * lifestyle * fashion * environment * tech * travel browse all sections close free our data datablog bigdata and the end of theory? does bigdata have the answers? maybe some, but not all, says mark graham computer data. bigdata is everywhere - but does it have the solution? photograph: john wilkes/getty images mark graham friday 9 march 2012 14.39 gmt last modified on tuesday 29 december 2015 16.52 gmt * share on facebook * share on twitter * share via email * share on pinterest * share on linkedin * share on google+ * share on whatsapp in 2008, chris anderson, then editor of wired, wrote a provocative piece titled the end of theory. anderson was referring to the ways that computers, algorithms, and bigdata can potentially generate more insightful, useful, accurate, or true results than specialists or domain experts who traditionally craft carefully targeted hypotheses and research strategies. this revolutionary notion has now entered not just the popular imagination, but also the research practices of corporations, states, journalists and academics. the idea being that the data shadows and information trails of people, machines, commodities and even nature can reveal secrets to us that we now have the power and prowess to uncover. in other words, we no longer need to speculate and hypothesise; we simply need to let machines lead us to the patterns, trends, and relationships in social, economic, political, and environmental relationships. it is quite likely that you yourself have been the unwitting subject of a bigdata experiment carried out by google, facebook and many other large web platforms. google, for instance, has been able to collect extraordinary insights into what specific colours, layouts, rankings, and designs make people more efficient searchers. they do this by slightly tweaking their results and website for a few million searches at a time and then examining the often subtle ways in which people react. most large retailers similarly analyse enormous quantities of data from their databases of sales (which are linked to you by credit card numbers and loyalty cards) in order to make uncanny predictions about your future behaviours. in a now famous case, the american retailer, target, upset a minneapolis man by knowing more about his teenage daughter's sex life than he did. target was able to predict his daughter's pregnancy by monitoring her shopping patterns and comparing that information to an enormous database detailing billions of dollars of sales. this ultimately allows the company to make uncanny predictions about its shoppers. more significantly, national intelligence agencies are mining vast quantities of non-public internet data to look for weak signals that might indicate planned threats or attacks. there can by no denying the significant power and potentials of big data. and the huge resources being invested in both the public and private sectors to study it are a testament to this. however, crucially important caveats are needed when using such datasets: caveats that, worryingly, seem to be frequently overlooked. the raw informational material for bigdata projects is often derived from large user-generated or social media platforms (e.g. twitter or wikipedia). yet, in all such cases we are necessarily only relying on information generated by an incredibly biased or skewed user-base. gender, geography, race, income, and a range of other social and economic factors all play a role in how information is produced and reproduced. people from different places and different backgrounds tend to produce different sorts of information. and so we risk ignoring a lot of important nuance if relying on bigdata as a social/economic/political mirror. we can of course account for such bias by segmenting our data. take the case of using twitter to gain insights into last summer's london riots. about a third of all uk internet users have a twitter profile; a subset of that group are the active tweeters who produce the bulk of content; and then a tiny subset of that group (about 1%) geocode their tweets (essential information if you want to know about where your information is coming from). despite the fact that we have a database of tens of millions of data points, we are necessarily working with subsets of subsets of subsets. bigdata no longer seems so big. such data thus serves to amplify the information produced by a small minority (a point repeatedly made by ucl's muki haklay), and skew, or even render invisible, ideas, trends, people, and patterns that aren't mirrored or represented in the datasets that we work with. bigdata is undoubtedly useful for addressing and overcoming many important issues face by society. but we need to ensure that we aren't seduced by the promises of bigdata to render theory unnecessary. we may one day get to the point where sufficient quantities of bigdata can be harvested to answer all of the social questions that most concern us. i doubt it though. there will always be digital divides; always be uneven data shadows; and always be biases in how information and technology are used and produced. and so we shouldn't forget the important role of specialists to contextualise and offer insights into what our data do, and maybe more importantly, don't tell us. mark graham is a research fellow at the oxford internet institute and is one of the creators of the floating sheep blog more open data • data journalism and data visualisations from the guardian • who's who on the datablog world government data • search the world's government data with our gateway development and aid data • search the world's global development data with our gateway can you do something with this data? • flickr please post your visualisations and mash-ups on our flickr group • contact us at data@guardian.co.uk • get the a-z of data • more at the datastore directory • follow us on twitter • like us on facebook __________________________________________________________________ more blogposts topics * free our data * internet * twitter * bigdata __________________________________________________________________ * share on facebook * share on twitter * share via email * share on pinterest * share on linkedin * share on google+ * share on whatsapp * reuse this content view all comments > comments sign in or create your guardian account to join the discussion. this discussion is closed for comments. we’re doing some maintenance right now. you can still read comments, but please come back later to add your own. commenting has been disabled for this account (why?) order by * newest * oldest * recommendations show 25 * 25 * 50 * 100 * all threads * collapsed * expanded * unthreaded loading comments… trouble loading? view more comments popular the guardian back to top * home * uk * world * sport * football * opinion * culture * business * lifestyle * fashion * environment * tech * travel all sections close * home * uk + education + media + society + law + scotland + wales + northern ireland * world + europe + us + americas + asia + australia + africa + middle east + cities + development * sport + football + cricket + rugby union + f1 + tennis + golf + cycling + boxing + racing + rugby league * football + live scores + tables + competitions + results + fixtures + clubs * opinion + columnists * culture + film + tv & radio + music + games + books + art & design + stage + classical * business + economics + banking + retail + markets + eurozone * lifestyle + food + health & fitness + love & sex + family + women + home & garden * fashion * environment + climate change + wildlife + energy + pollution * tech * travel + uk + europe + us + skiing * money + property + savings + pensions + borrowing + careers * science * professional networks * the observer * today's paper + editorials & letters + obituaries + g2 + weekend + the guide + saturday review * sunday's paper + comment + the new review + observer magazine * membership * crosswords + blog + editor + quick + cryptic + prize + quiptic + genius + speedy + everyman + azed * video * news * › free our data iframe: /email/form/footer/37 * facebook * twitter * facebook * twitter * all topics * all contributors * solve technical issue * complaints & corrections * terms & conditions * privacy policy * cookie policy * securedrop © 2016 guardian news and media limited or its affiliated companies. all rights reserved. #publisher alternate close skip to main content sign in * saved for later * comment activity * edit profile * email preferences * change password * sign out subscribe search dating more from the guardian: * dating * jobs change edition: * switch to the uk edition switch to the us edition switch to the au edition international * switch to the uk edition * switch to the us edition * switch to the australia edition the guardian * home * › tech * home * uk * world * sport * football * opinion * culture * business * lifestyle * fashion * environment * tech selected * travel browse all sections close data protection the shrinking of the bigdata promise cory doctorow markets do not have much patience for a commitment to techniques that don’t deliver. unfortunately, spy agencies aren’t subject to this discipline using a computer it is not sorcery to predict that a woman who buys folic acid is pregnant photograph: johner images / alamy/alamy tuesday 30 june 2015 11.47 bst last modified on tuesday 30 june 2015 11.51 bst * share on facebook * share on twitter * share via email * share on pinterest * share on linkedin * share on google+ * share on whatsapp “regression to the mean” is one of the subtlest concepts in statistical literacy – and yet it’s terribly simple. in plain english, “regression to the mean” is the idea that normally, things are pretty normal. that is, if you observe something abnormal – a high fever, a high share price, a long, unseasonable stretch of sunny or rainy days – then chances are that all will soon fall back within normal range. if you’re sick, you’ll probably get better (this is why so many quack cold remedies “work” – they take your mind off the passage of time while you wait for regression to the mean to assert itself). regression to the mean applies to commerce. an advertising technique that works very well today will probably work less well over time, as repetition and competition eat away at its novelty. look at those “ghost ads” on the sides of faded victorian buildings: apparently, it was once profitable to advertise soap with a slogan like “it makes you clean.” remember when xynga’s “social games” like farmville seemed to colonise the limbic systems of everyone you knew, stealing away their hours with a fiendishly addictive game-mechanic? in short order, most of xynga’s players grew inured to the game’s temptations, leaving behind a rump of especially susceptible players who were not enough to sustain the game, nor its makers’ high-flying share price. likewise, the “surveillance business model” of building up detailed electronic dossiers on internet users in order to predict what they want to buy and how to sell it to them produced some genuinely impressive results in its early years. the serendipity of seeing an ad for something you had been thinking about proved very powerful in the early days of facebook and the first generation of “retargeting” services. but a look at facebook’s ad-card rates shows that the novelty of this technique wears off fast. facebook was founded on the premise that it could use its mounting dossier on your behavior to figure out how to sell you things faster than your natural defenses would repel its pitches. if that ideology was borne out, you’d expect to see the company’s cost-per-thousand ad rates climbing into the stratosphere. instead, they’re damned close to the rate you’ll pay for regular, minimally targeted display advertising elsewhere. the quarterly facebook investor calls tell the story: facebook’s growth area for revenue aren’t from predictive, targeted ads. instead, the company is making good money from venture-backed games firms that pay “per acquisition” – from users actually installing their games. as facebook doesn’t have to pay itself to advertise on its own service, it can simply wash its service in ads for games until the acquisitions take place. the cost per acquisition on facebook is substantially in excess of what any game company could hope to earn from an average player, suggesting that this line of business is due for a crash. the other profitable line for facebook is sneakier, and possibly longer-lived. the company can easily see which of the commercial/brand/business pages on its service are growing fastest. these correspond to the businesses that are exerting the most energy to get their customers to follow them on facebook and making facebook most integral to their daily business. when facebook’s algorithms predict that a business is well and truly reliant upon facebook to reach its customers, it simply switches off the business’s ability to reach those customers, so that new updates only go to a small fraction of the company’s followers. thereafter, a facebook salesperson gives the business a call and offer to turn the tap back on – for a price. that’s not the surveillance business-model. it’s a much older one: the drug-dealer business-model, where the first taste is free. the bigdata success stories for predicting human behavior over long terms don’t bear scrutiny. it’s not a triumph of bigdata to predict that someone searching for “used cars” might respond to an ad for used cars. neither is it sorcery to predict that a woman who buys folic acid is pregnant. it’s not bigdata to get paid when someone clicks on a loan application or installs a game. markets don’t solve all our problems, but they do not have much patience for an irrational, ideological commitment to techniques that don’t deliver. facebook doubtless has internal fiefdoms that will be threatened by the company backing off its surveillance commitment, but it is also growing non-surveillance-oriented tendrils as fast as it can. unfortunately, spy agencies aren’t subject to this kind of discipline. the fact that the billions spent spying on everyone, always, has spectacularly failed to catch any terrorists is taken as proof that they’re not doing enough surveillance – not that untargeted, mass surveillance without particularised suspicion is a waste of money. of course, it helps that there are so many contractors and suppliers who lobby for spies to buy more of their gear and services. every technology is overhyped at its birth. the gartner hype cycle has bigdata sliding into the long, deep “trough of disillusionment.” as the cycle astute observes, overpromising doesn’t mean there’s no there there. as bigdata techniques stabilise into a few applications where it works well and long, more of the surveillance business model will blow away. __________________________________________________________________ more comment topics * data protection * surveillance * internet __________________________________________________________________ * share on facebook * share on twitter * share via email * share on pinterest * share on linkedin * share on google+ * share on whatsapp * reuse this content view all comments > comments sign in or create your guardian account to join the discussion. this discussion is closed for comments. we’re doing some maintenance right now. you can still read comments, but please come back later to add your own. commenting has been disabled for this account (why?) order by * newest * oldest * recommendations show 25 * 25 * 50 * 100 * all threads * collapsed * expanded * unthreaded loading comments… trouble loading? view more comments popular the guardian back to top * home * uk * world * sport * football * opinion * culture * business * lifestyle * fashion * environment * tech selected * travel all sections close * home * uk + education + media + society + law + scotland + wales + northern ireland * world + europe + us + americas + asia + australia + africa + middle east + cities + development * sport + football + cricket + rugby union + f1 + tennis + golf + cycling + boxing + racing + rugby league * football + live scores + tables + competitions + results + fixtures + clubs * opinion + columnists * culture + film + tv & radio + music + games + books + art & design + stage + classical * business + economics + banking + retail + markets + eurozone * lifestyle + food + health & fitness + love & sex + family + women + home & garden * fashion * environment + climate change + wildlife + energy + pollution * tech selected * travel + uk + europe + us + skiing * money + property + savings + pensions + borrowing + careers * science * professional networks * the observer * today's paper + editorials & letters + obituaries + g2 + weekend + the guide + saturday review * sunday's paper + comment + the new review + observer magazine * membership * crosswords + blog + editor + quick + cryptic + prize + quiptic + genius + speedy + everyman + azed * video * technology * › data protection iframe: /email/form/footer/37 * facebook * twitter * facebook * twitter * all topics * all contributors * solve technical issue * complaints & corrections * terms & conditions * privacy policy * cookie policy * securedrop © 2016 guardian news and media limited or its affiliated companies. all rights reserved. #rdf metadata [p?c1=2&c2=3005435&cv=2.0&cj=1] executive insights & innovation security & risk strategy team building & staffing it strategy digital business project management programming languages dr. dobb's enterprise applications operating systems productivity/collaboration apps careers & people threat intelligence iot attacks & breaches application security cloud security endpoint security mobile security perimeter security risk management operations security analytics vulnerabilities & threats security & risk strategy infrastructure as a service platform as a service software as a service cloud storage data centers mobile applications mobile devices mobile business enterprise mobility management bigdata analytics hardware/architectures software platforms networking storage wireless infrastructure data center unified communications cloud infrastructure applications careers pcs and servers net security government healthcare wall street & technology bank systems & technology insurance & technology advertise with us about us contact us digital subscription [iwk_mobile_user_nav.png] [spacer.gif] [iwk_mobile_search_icon.png] [spacer.gif] [iwk_mobile_sm.png] [iwk-sections-nav.png] [mobile_close.jpg] * home * news & commentary * authors * slideshows * video * reports * white papers * events * university * interop * strategic cio * iot * devops * software * security * cloud * mobile * bigdata * infrastructure * industries * it life [mobile_close.jpg] * about us * contact us * digital subscription * advertise with us [mobile_close.jpg] ____________________ [spacer.gif] [mobile_close.jpg] * facebook * twitter * linkedin * google+ * rss informationweek: connecting the business technology community ____________________ [spacer.gif] follow iw: [spacer.gif] [spacer.gif] [spacer.gif] [spacer.gif] [spacer.gif] [spacer.gif] [spacer.gif] [spacer.gif] [spacer.gif] home news & commentary authors video reports white papers events university elite100 interop strategic cio iot devops software security cloud mobile bigdata infrastructure industries it life bigdata // bigdata analytics news 11/16/2015 09:06 am jessica davis jessica davis news connect directly twitter rss e-mail [comment.png] 2 comments comment now login [thumbs-up.png] 50% [thumbs-down.png] 50% tweet intel chief data scientist shares secrets to successful projects before joining intel, the company's first-ever chief data scientist created models to predict the stock market, improve glaucoma diagnostics, and include unstructured data in healthcare applications. here's what bob rogers is doing now at intel and his advice for fellow and future data scientists. 6 ways bigdata is driving personalized medicine revolution 6 ways bigdata is driving personalized medicine revolution (click image for larger view and slideshow.) what's a dream job for a data scientist? it just may be serving as the chief data scientist at a big technology vendor that doesn't care so much about selling analytics or bigdata solutions. instead, it cares about empowering the analytics ecosystem and helping organizations apply bigdata analytics to any number of problems and challenges of business, science, and humanity. that's where intel's chief data scientist bob rogers is right now. he joined the chip giant in january and spends his time working both internally on intel's own data science projects and externally as the "bigdata evangelist" for intel, spreading the word about bigdata and helping organizations become successful with their analytics and data science projects. bob rogers, chief data scientist, intel (image: via linkedin) bob rogers, chief data scientist, intel (image: via linkedin) "intel sells chips. we don't sell services. we don't sell software. our overall strategy is to empower the analytics ecosystem," he said. "i get to go around and help customers be successful without needing to sign on the dotted line or anything else. i help customers understand analytics and data problems and move the ball forward." that's what he does externally. among the internal projects rogers works on with intel are efforts to improve the it help desk process by using semantic engines on text, adding unstructured data to improve the engine that provides successful best practices to intel resellers, and serving as part of the data science center of excellence, a weekly and voluntary internal forum at intel designed to enable data scientists within the company to help each other tackle big problems in their respective groups. rogers brings an eclectic range of training and experience to the role. he was trained as a physicist, and his phd thesis was based on a computer simulation of what would happen to objects sucked into black holes. over the course of his career, he worked on problems including the creation of simulations to predict the stock market for hedge funds, improving glaucoma diagnostics, and fixing doctor access to all relevant patient information. it was in that last role as cofounder of apixio that he started working with intel. apixio applies bigdata to problems with electronic health records (ehr). "because each ehr is a silo. what we discovered in our studies is that 65% of the information doctors should know about you is not in structured data." in the example of apparent heart failure, about 30% of cases are not actually heart failure. there are false negatives and false positives, and much of that information is in email, not ehrs. that means it's unstructured data. a different approach now as chief data scientist at intel, rogers has a vantage point to see a wide array of challenges and possibilities enabled by bigdata across many different types of organizations. what is the biggest mistake that organizations make when implementing bigdata projects? "one of the biggest problems i see is that enterprises want to build a bigdata stack, shove all their data in, and hope that insights bubble to the surface," he said. "but at the end of the day that's a good way to end up with an expensive project that doesn't seem to show any value." instead of this, he advocates more of an agile approach. "start with a specific challenge and then build the minimum infrastructure needed," he said. intel itself has built up its own bigdata infrastructure internally alongside traditional business intelligence infrastructure, and those two systems talk to each other, according to rogers. in an initial project, intel built a recommendation engine, by tapping into both structured and unstructured data, in order to help intel offer resellers insights about how to be more successful, rogers said. the company built up this bigdata stack on hadoop, using cloudera. the important component of this was adding the unstructured data. "the structured data is the same data you've been looking at for years," he said. "but if you add even a small amount of the unstructured data, you get a huge step forward in performing and creating value. that's one of the big areas that i see data science advancing in very rapidly." one of the other keys to success is crafting the right question. rogers recently sat on a panel with a handful of other high-level data scientists at new york university, addressing the school's data science graduate students. they wanted to know about skills that are important to be successful in the field. "there are technical skills -- math and statistics and modeling and computer science," he said. "then there is understanding the business needs." realistically, there aren't any people who embody all the skills of the perfect data scientist, he said. that's why it's important to be able to work collaboratively and draw on the skills of a team. "we are looking for people with a mix of skills. what's really important is the ability to handle ambiguity. … [y]ou may not have an exact answer that is analytically measurable, and that can put people outside their comfort zone. "another aspect is creativity," rogers said. that means being able to look at a problem from multiple directions. for instance, instead of lumping all car buyers into one group, if you break them apart according to demographics and think about them separately, you will learn much more. [looking for more on intel's bigdata initiatives? read intel's tap big data platform gains healthcare, cloud partners.] "those attributes are at least as important as the technical skills," rogers said. "data is messy." in terms of tools and programming languages, the best tools for fledgling data scientists to start with and learn are r and python, according to rogers. the most exciting developments in terms of languages that are evolving for bigdata are functional languages like scala that let you write complex analytics that immediately scale across huge clusters. looking forward looking ahead to the future of bigdata and analytics, rogers said he believes two of the most exciting areas offering big potential yet to be realized are the internet of things (iot) and unstructured data. in terms of iot, rogers said he believes there will be a great impact from having intelligence at the edge, for instance, in terms of wearables for healthcare applications and sensors coupled with analytics for smart cities and transportation. while we have gotten good at analyzing text in terms of unstructured data, there is still much work to be done to understand images, video, and audio, rogers said. "intel is working very hard in this area," rogers said. "that's an exciting area of growth and advancement." jessica davis has spent a career covering the intersection of business and technology at titles including idg's infoworld, ziff davis enterprise's eweek and channel insider, and penton technology's mspmentor. she's passionate about the practical use of business intelligence, ... view full bio comment | email this | print | rss more insights webcasts is your business prepared? get the business view - a dark reading virtual event best practices for resilient inline security deployments more webcasts white papers how netflix is taking capacity & tco to new heights [case study] how nestle used video for internal communications more white papers reports [informationweek & dark reading report] 2015 strategic security survey results 10 recommendations for outsourcing security more reports [spacer.gif] comments newest first | oldest first | threaded view [close this box] jagibbons login [thumbs-up.png] 50% [thumbs-down.png] 50% jagibbons, user rank: ninja 11/18/2015 | 12:25:42 pm re: data they're growing that market, but i also like to see that they are contributing to the industry as a whole without it being part of a software product or services sale. reply | post message | messages list | start a board danielcawrey login [thumbs-up.png] 50% [thumbs-down.png] 50% danielcawrey, user rank: ninja 11/17/2015 | 12:40:50 pm data interesting look at how intel as a chipmaker looks at data and software. the company has a market cap of around $151 billion, most of that coming from chips. but they also create software for their products, and they bought mcafee and turned it into intel security. the company is primarily hardware, but software and the data generated around it is growing too. reply | post message | messages list | start a board [spacer.gif] reports [spacer.gif] [spacer.gif] infographics [spacer.gif] 6 tools to protect bigdata 6 tools to protect bigdata most it teams have their conventional databases covered in terms of security and business continuity. but as we enter the era of bigdata, hadoop, and nosql, protection schemes need to evolve. in fact, bigdata could drive the next big security strategy shift. download now! [spacer.gif] more reports bigdata brings big security problems bigdata brings big security problems why should bigdata be more difficult to secure? in a word, variety. but the business wont wait to use it to predict customer behavior, find correlations across disparate data sources, predict fraud or financial risk, and more. [comment.png] 1 comments | read | post a comment [spacer.gif] 2014 state of database [comment.png] 0 comments [spacer.gif] in a fever for bigdata [comment.png] 0 comments [spacer.gif] more infographics register for informationweek newsletters subscribe to newsletters [spacer.gif] live events [spacer.gif] [spacer.gif] webinars [spacer.gif] [ubm-tech.png] more ubm tech live events learn newest developments in video at enterprise connect see adam swidler of google for work at enterprise connect interop las vegas applications track is your business prepared? get the business view - a dark reading virtual event transforming customer business apps and vdi desktops with flash-enabled vsan ready nodes how technology is transforming manufacturing [spacer.gif] webinar archives white papers how netflix is taking capacity & tco to new heights [case study] how nestle used video for internal communications [checklist] moving to the cloud combating the insider threat [cyber security] how well does your threat intelligence feed rank? [spacer.gif] more white papers current issue [screenshot2015-12-07at122615pm.jpg] how to knock down barriers to effective risk management risk management today is a hodgepodge of systems, siloed approaches, and poor data collection practices. that isn't how it should be. download this issue! subscribe now! [spacer.gif] back issues | must reads video [video-arrow_left_off.gif] [1568176135_4489522385001_thumbnail-for-video-4234403102001.jpg?pubid=1 568176135] informationweek presents 'get to the point' [comment.png] 1 comments [1568176135_4232614152001_4232562810001-th.jpg?pubid=1568176135] heartland ceo: dealing honestly with data ... [comment.png] 7 comments [1568176135_4234072761001_4234047388001-th.jpg?pubid=1568176135] where data science meets it [comment.png] 4 comments [1568176135_4234440318001_4234365405001-th.jpg?pubid=1568176135] modern data centers: how we'll get there [comment.png] 1 comments [1568176135_4232444203001_4232432578001-th.jpg?pubid=1568176135] enterprise it: act like a software startup [comment.png] 1 comments [1568176135_4234091599001_4234071225001-th.jpg?pubid=1568176135] how astrazeneca's cio embraced insourcing [comment.png] 0 comments [1568176135_4386836911001_thumbnail-for-video-4386951387001.jpg?pubid=1 568176135] windows 10: inside nyc launch day [comment.png] 10 comments [1568176135_4232897123001_4232839543001-th.jpg?pubid=1568176135] conocophillips drills down into analytics [comment.png] 1 comments [1568176135_4232971722001_4232949108001-th.jpg?pubid=1568176135] royal caribbean keeps customers connected ... [comment.png] 1 comments [1568176135_4232964260001_4232945668001-th.jpg?pubid=1568176135] the age of biotechnology has arrived [comment.png] 1 comments [1568176135_4232958763001_4232945664001-th.jpg?pubid=1568176135] chris anderson: making drones and ... [comment.png] 2 comments [1568176135_4313612547001_thumbnail-for-video-4313097826001.jpg?pubid=1 568176135] nba cio focuses on analytics, data and fans [comment.png] 10 comments [video-arrow_right_on.gif] [spacer.gif] all videos [camera-24x16.png] slideshows iphone 7 rumors: 7 that have people talking iphone 7 rumors: 7 that have people talking [comment.png] 0 comments | read | post a comment [spacer.gif] cios detail hiring plans for first half of 2016 [comment.png] 0 comments [spacer.gif] 10 medical-device wearables to improve patients' lives [comment.png] 5 [spacer.gif] more slideshows twitter feed tweets about "from:informationweek or #informationweek or @informationweek" informationweek radio archived informationweek radio informationweek live for the week of december 13, 2015 join us for a roundup of the top stories on informationweek.com for the week of december 13, 2015. [spacer.gif] upcoming! friday, january 15, 3pm est informationweek live for the week of january 10, 2016 [spacer.gif] full schedule | archived shows [spacer.gif] sponsored live streaming video everything you've been told about mobility is wrong attend this video symposium with sean wisdom, global director of mobility solutions, and learn about how you can harness powerful new products to mobilize your business potential. [comment.png] 465 comments | read | post a comment [spacer.gif] full schedule | archived shows information week about us contact us customer support reprints editorial calendar twitter facebook linkedin google+ rss ubm tech * ubm tech brands * black hat * cloud connect * dark reading * enterprise connect * fusion * gdc * gtec * gamasutra * hdi * informationweek * interop * network computing * no jitter * tower & small cell summit * communities served * enterprise it * enterprise communications * game development * information security * it services & support * working with us * advertising contacts * event calendar * tech marketing * solutions * contact us * licensing * terms of service * privacy statement * copyright © 2016 ubm tech, all rights reserved [client_pathlog.asp?p=%2finformationweek%2fsection%2f277&f=%2finformati onweek%2fsection%2f277%2f1323145&rndserial=19421] to rate this item, click on a rating below. [lightreading_rating_dot_10x7.gif] [lightreading_rating_dot_10x7.gif] [lightreading_rating_dot_10x7.gif] [lightreading_rating_dot_10x7.gif] [lightreading_rating_dot_10x7.gif] [lightreading_rating_dot_10x7.gif] [lightreading_rating_dot_10x7.gif] [lightreading_rating_dot_10x7.gif] [lightreading_rating_dot_10x7.gif] [lightreading_rating_dot_10x7.gif] [lightreading_rating_dot_10x7.gif] [lightreading_rating_dot_10x7.gif] [lightreading_rating_dot_10x7.gif] [lightreading_rating_dot_10x7.gif] [lightreading_rating_dot_10x7.gif] [close this box] * * to save this item to your list of favorite informationweek content so you can find it later in your profile page, click the "save it" button next to the item. * * if you found this interesting or useful, please use the links to the services below to share it with other readers. you will need a free account with each service to share an item via that service. * [twitter_intevol_18x18.gif] tweet this * [close this box] [spacer.gif] #bits » sizing up bigdata, broadening beyond the internet comments feed from campaign war room to big-data broom daily report: deepening ties between n.s.a. and silicon valley rss 2.0 alternate sections home search skip to content the new york times bits | sizing up bigdata, broadening beyond the internet advertisement search subscribe now log in 0 settings close search search sponsored by site search navigation search nytimes.com ____________________ clear this text input go 1. loading... see next articles see previous articles site navigation site mobile navigation supported by bits - business, innovation, technology, society ____________________ search bigdata 2013 sizing up bigdata, broadening beyond the internet by steve lohr june 19, 2013 11:09 pm june 19, 2013 11:09 pm photo how we feel a visual representation of recent sentiment, as expressed on the internet. good feelings are brighter; negative ones are darker. how we feel a visual representation of recent sentiment, as expressed on the internet. good feelings are brighter; negative ones are darker.credit in his young career, jeffrey hammerbacher has been a scout on the frontiers of the data economy. bigdata 2013 a special section on the business and culture of bigdata. in 2005, mr. hammerbacher, then a freshly minted harvard graduate, did what many math and computing whizzes did. he went to wall street as a “quant,” building math models for complex financial products. looking for a better use for his skills, mr. hammerbacher departed to silicon valley less than a year later and joined facebook. he started a team that began to mine the vast amounts of social network data facebook was collecting for insights on how to tweak the service and target ads. he called himself and his co-workers “data scientists,” a term that has since become the hottest of job categories. facebook was a fabulous petri dish for data science. yet after two and a half years, mr. hammerbacher decided it was time to move on, beyond social networks and internet advertising. he became a founder of cloudera, a start-up that makes software tools for data scientists. then, starting last summer, mr. hammerbacher, who is now 30, embarked on a very different professional path. he joined the mount sinai school of medicine in manhattan as an assistant professor, exploring genetic and other medical data in search of breakthroughs in disease modeling and treatment. the goal, mr. hammerbacher said, is “to turn medicine into the land of the quants.” graphic bigdata will get bigger the story is the same in one field after another, in science, politics, crime prevention, public health, sports and industries as varied as energy and advertising. all are being transformed by data-driven discovery and decision-making. the pioneering consumer internet companies, like google, facebook and amazon, were just the start, experts say. today, data tools and techniques are used for tasks as varied as predicting neighborhood blocks where crimes are most likely to occur and injecting intelligence into hulking industrial machines, like electrical power generators. bigdata is the shorthand label for the phenomenon, which embraces technology, decision-making and public policy. supplying the technology is a fast-growing market, increasing at more than 30 percent a year and likely to reach $24 billion by 2016, according to a forecast by idc, a research firm. all the major technology companies, and a host of start-ups, are aggressively pursuing the business. demand is brisk for people with data skills. the mckinsey global institute, the research arm of the consulting firm, projects that the united states needs 140,000 to 190,000 more workers with “deep analytical” expertise and 1.5 million more data-literate managers, whether retrained or hired, by 2020. yet the surveillance potential of bigdata, with every click stream, physical movement and commercial transaction monitored and analyzed, would strain the imagination of george orwell. so what will be society’s ground rules for the collection and use of data? how do we weigh the trade-offs involving privacy, commerce and security? those issues are just beginning to be addressed. the debate surrounding the recent disclosure that the national security agency has been secretly stockpiling telephone call logs of americans and poring through e-mail and other data from major internet companies is merely an early round. bigdata is a vague term, used loosely, if often, these days. but put simply, the catchall phrase means three things. first, it is a bundle of technologies. second, it is a potential revolution in measurement. and third, it is a point of view, or philosophy, about how decisions will be — and perhaps should be — made in the future. the bundle of technologies is partly all the old and new sources of data — web pages, browsing habits, sensor signals, social media, gps location data from smartphones, genomic information and surveillance videos. the data surge just keeps rising, doubling in volume every two years. just two days of the current global data production, from all sources — five quintillion bytes (a letter of text equals one byte) — is about equal to the amount of information created by all the world’s conversations, ever, according to research at the university of california, berkeley. yet the importance of the sheer volume of data — and its exponential growth path — can be overstated. there’s a lot of water in the ocean, too, but you can’t drink it. beyond advances in computer processing and storage, the other essential technology is the clever software to make sense of all that data. these are largely tools taken from the steadily evolving world of artificial intelligence, like machine learning. the increasing volume and variety of data, combined with smart software, may well open the door to what some people call a revolution in measurement. this technology, they say, is the digital equivalent of the telescope or the microscope. both of those made it possible to see and measure things as never before — with the telescope, it was the heavens and new galaxies; with the microscope, it was the mysteries of life down to the cellular level. data-driven insights, experts say, will fuel a shift in the center of gravity in decision-making. decisions of all kinds, they say, will increasingly be made on the basis of data and analysis rather than experience and intuition — more science and less gut feel. data, for example, is an antidote to the human tendency to rely too much on a single piece of information or what is familiar — what psychologists call “anchoring bias.” bigdata, its proponents insist, will be the next big trend in management. erik brynjolfsson, director of the mit center for digital business, cites the familiar business truism, “you can’t manage what you can’t measure.” and as it opens new horizons in measurement, the modern data era, mr. brynjolfsson said, will transform the practice of management. bigdata, he said, will “replace ideas, paradigms, organizations and ways of thinking about the world.” but caveats are in order. bigdata is a descendant of frederick winslow taylor’s “scientific management” of a century ago. his instruments of measurement and recording were the stopwatch, clipboard and his eyes. taylor and his acolytes used these time-and-motion studies to redesign work for maximum efficiency. yet eventually, the excesses of that approach became apparent and even satirical grist for the movie “modern times” by charlie chaplin. and the enthusiasm for quantitative methods has waxed and waned ever since. discrimination by statistical inference is a real risk in the bigdata world, as some personal data trails suggest a correlation that may be wrong. david c. vladeck, a former senior federal trade commission official and a professor of law at georgetown university, offers this example: imagine spending a few hours looking online for information on deep fat fryers. you could be looking for a gift for a friend or researching a report for cooking school. but to a data miner, tracking your online viewing, this hunt could be read as a telltale sign of an unhealthy habit — a data-based prediction that could make its way to a health insurer or potential employer. and, again, the surveillance potential of bigdata technology, if it runs amok, is scary. but all technologies involve trade-offs and risks. in ancient times, fire could cook your food and keep you warm, but, out of control, could burn down your hut. cars pollute the air and cause traffic deaths, but they have also increased personal mobility and freedom, and stimulated the development of regional and national markets for goods. bigdata technology is not fundamentally different. its advance is probably inevitable, and the risks seem manageable and the potential benefits enormous. one glimpse of the potential payoff can be seen at the mount sinai medical center, in the work being pursued by the group mr. hammerbacher has joined. photo data man jeffrey hammerbacher studies genetic and other medical data. data man jeffrey hammerbacher studies genetic and other medical data.credit fred r. conrad/the new york times the 100-member team at the icahn institute for genomics and multiscale biology is headed by eric e. schadt, a leading researcher in genomics and biomathematics. dr. schadt joined mount sinai less than two years ago, lured by ample financing and the promise that his group’s work would not be research in isolation but part of the medical center in treating patients. the genomics revolution is on the cusp of realizing its promise, according to dr. schadt, thanks to the advancing technology of genetic sequencing and analysis. the government-financed human genome project, completed in 2003, cost $2.7 billion. today, whole human genome sequencing, identifying all three billion chemical units in the human genetic instruction set, can be done for $3,000. in three years, dr. schadt predicts, the cost will be less than $1,000, and in 5 to 10 years, less than $100, almost like a blood test today. the technology makes it possible not only to observe life at the molecular level as never before, but also to explore how the minute ingredients of biology and the environment influence each other in individual humans — and personalize treatment. people with similar genetic traits, dr. schadt notes, often have very different health outcomes. chronic ailments like cancer, heart disease and alzheimer’s are not caused by single genes, he said, but are “complex, networked disorders.” the mount sinai researchers, dr. schadt said, intend to combine genetic information with the medical histories — weight, age, gender, vital signs, tobacco use, toxic exposure and other data — to build more sophisticated models of biology and health outcomes. “we’re trying to move medicine in the direction of climatology and physics; disciplines that are far more advanced and mature quantitatively,” he said. dr. schadt recruited mr. hammerbacher, an overture that coincided with mr. hammerbacher’s research into where next to best apply his skills. he describes his career as a matter of “following the smartest people to find the best problem.” health care, in his view, is “the best problem by far,” where his talents could do the most good. at mount sinai, mr. hammerbacher said he hoped to learn a lot and assemble a small group of computing and data experts to help accelerate the genomic and medical research there. mr. hammerbacher remains the chief scientist of cloudera and splits his time between san francisco and manhattan. but he is spending more time in new york these days and just bought an apartment on the lower east side for himself and his wife. mr. hammerbacher has qualms about the bigdata realm he has helped create, including the surveillance potential of the technology. “what does it mean,” mr. hammerbacher pondered at one point, “to live in an era where things and people are infinitely observed?” and he appreciates that there is a lot of truth beyond data. “just because you can’t measure it easily doesn’t mean it’s not important,” he observed. while he is perhaps a qualified enthusiast, mr. hammerbacher is a data believer. he calls data the “intermediate representation of science.” the genome, he said, is “the quantification of the core of what we are.” he says he thinks that medicine, and nearly every other field, will increasingly fall under the sway of what he calls “the numerical imagination,” which can be distilled in a question: “what is the story the data tells us?” a version of this article appears in print on 06/20/2013, on page f1 of the newyork edition with the headline: sizing up bigdata. bigdata 2013, bigdata2013, computers and the internet, research, science and technology, social networking (internet) what's next loading... * previous post from campaign war room to big-data broom * next post daily report: deepening ties between n.s.a. and silicon valley a special section on the business and culture of bigdata. visit the technology section for complete coverage of the industry. » most viewed 1. 1. when a unicorn start-up stumbles, its employees get hurt 2. 2. uber’s no-holds-barred expansion strategy fizzles in germany 3. 3. at c.d.c., a debate behind recommendations on cellphone risk 4. 4. yahoo’s brain drain shows a loss of faith inside the company 5. 5. fans demand details after death of a 13-year-old youtube star latest from bits * amazon subsidiary gains approval to ship ocean freight * daily report: tech earnings season begins with a warning * surveymonkey names new chief, its second in six months * the zappos exodus continues after a radical management experiment * daily report: the netflix-amazon connection follow us on @nytimesbits on twitter * twitter follow advertisement go to home page » site index the new york times site index navigation news * world * u.s. * politics * n.y. * business * tech * science * health * sports * education * obituaries * today's paper * corrections opinion * today's opinion * op-ed columnists * editorials * contributing writers * op-ed contributors * opinionator * letters * sunday review * taking note * room for debate * public editor * video: opinion arts * today's arts * art & design * artsbeat * books * dance * movies * music * n.y.c. events guide * television * theater * video games * video: arts living * automobiles * crossword * food * education * fashion & style * health * jobs * magazine * n.y.c. events guide * real estate * t magazine * travel * weddings & celebrations listings & more * classifieds * tools & services * times topics * public editor * n.y.c. events guide * blogs * multimedia * photography * video * nyt store * times journeys * subscribe * manage my account subscribe * subscribe * times insider * home delivery * digital subscriptions * nyt opinion * crossword * email newsletters * alerts * gift subscriptions * corporate subscriptions * education rate * mobile applications * replica edition * international new york times * © 2016 the new york times company * contact us * work with us * advertise * your ad choices * privacy * terms of service * terms of sale * site map * help * site feedback * subscriptions #bits » handicapping the half-life of ‘bigdata’ comments feed ways to make your online tracks harder to follow in hot pursuit of numbers to ward off crime rss 2.0 alternate sections home search skip to content the new york times bits | handicapping the half-life of ‘bigdata’ advertisement search subscribe now log in 0 settings close search search sponsored by site search navigation search nytimes.com ____________________ clear this text input go 1. loading... see next articles see previous articles site navigation site mobile navigation supported by bits - business, innovation, technology, society ____________________ search bigdata 2013 handicapping the half-life of ‘bigdata’ by steve lohr june 19, 2013 10:40 pm june 19, 2013 10:40 pm the technologies that now fly under the banner of bigdata are undeniably advancing across the economy, well beyond their early stronghold in consumer internet companies like google, amazon and facebook. exploring that evolution is the main theme of the articles in a special section of the new york times on thursday. but what about the term bigdata itself? what is its likely life span? bigdata 2013 a special section on the business and culture of bigdata. i’ve written about the origins of the term before. the first person to use bigdata in its current meaning, it seems, was john mashey, chief scientist of silicon graphics in the 1990s. but that is a probability, not a certainty — appropriately enough, since bigdata is all about probabilities and correlations. as to its longevity, bigdata, i’m betting, will cycle out of general use over time. not because it is a catchall marketing term, which it is, among other things. the best marketing and sales language is distilled communication. photo credit john hersey my bet against bigdata, i suppose, boils down to a linguistic bias. it is too straightforward. it lacks the whiff of poetry, the tension that tightens the bond between words in a phrase. by way of contrast, look at “artificial intelligence.” its inspiration was a sales pitch of sorts. in 1955, john mccarthy, a mathematician and computer scientist, was seeking funds from the rockefeller foundation for a conference the following year that would explore the growing excitement that computers might become more than big number-crunching calculators — that they might, in their way, actually be able to mimic human thought. in an interview 45 years later, mr. mccarthy explained that he “just cooked up the phrase” when drafting the grant proposal. as mr. mccarthy recalled, “my idea was to nail the flag to the mast, as it were.” mr. mccarthy, who died two years ago, certainly did that. artificial intelligence was a deft turn of phrase in the 1950s, and one that took on greater meaning over the years, evoking both the inspiring ambition of science and unnerving qualms about machine intelligence. but the history of artificial intelligence is instructive in another way. the enthusiasm for the technologies of artificial intelligence, or a.i., has gone through up and down cycles, and at times they were very down. there were two long stretches when investment and interest in artificial intelligence fell sharply — roughly 1974-80 and 1987-93. those years were known as “a.i. winters.” researchers scrambled to find other names for projects, anything but artificial intelligence. a similar pattern seems likely for the technologies of bigdata. there may be up and down cycles, the term bigdata may fall from favor, but the technology itself will keep progressing. indeed, the tools of artificial intelligence, like machine learning, are behind the promise of bigdata, which is to find useful insights in an ever-growing universe of digital data. bigdata 2013, internet, artificial intelligence, bigdata2013, data-mining and database marketing, google inc, mccarthy, john what's next loading... * previous post ways to make your online tracks harder to follow * next post in hot pursuit of numbers to ward off crime a special section on the business and culture of bigdata. visit the technology section for complete coverage of the industry. » most viewed 1. 1. when a unicorn start-up stumbles, its employees get hurt 2. 2. uber’s no-holds-barred expansion strategy fizzles in germany 3. 3. at c.d.c., a debate behind recommendations on cellphone risk 4. 4. yahoo’s brain drain shows a loss of faith inside the company 5. 5. fans demand details after death of a 13-year-old youtube star latest from bits * amazon subsidiary gains approval to ship ocean freight * daily report: tech earnings season begins with a warning * surveymonkey names new chief, its second in six months * the zappos exodus continues after a radical management experiment * daily report: the netflix-amazon connection follow us on @nytimesbits on twitter * twitter follow advertisement go to home page » site index the new york times site index navigation news * world * u.s. * politics * n.y. * business * tech * science * health * sports * education * obituaries * today's paper * corrections opinion * today's opinion * op-ed columnists * editorials * contributing writers * op-ed contributors * opinionator * letters * sunday review * taking note * room for debate * public editor * video: opinion arts * today's arts * art & design * artsbeat * books * dance * movies * music * n.y.c. events guide * television * theater * video games * video: arts living * automobiles * crossword * food * education * fashion & style * health * jobs * magazine * n.y.c. events guide * real estate * t magazine * travel * weddings & celebrations listings & more * classifieds * tools & services * times topics * public editor * n.y.c. events guide * blogs * multimedia * photography * video * nyt store * times journeys * subscribe * manage my account subscribe * subscribe * times insider * home delivery * digital subscriptions * nyt opinion * crossword * email newsletters * alerts * gift subscriptions * corporate subscriptions * education rate * mobile applications * replica edition * international new york times * © 2016 the new york times company * contact us * work with us * advertise * your ad choices * privacy * terms of service * terms of sale * site map * help * site feedback * subscriptions #bits » bigdata gets its own photo album comments feed the iphone 5: pogue’s first impressions social networks can affect voter turnout, study says rss 2.0 alternate sections home search skip to content the new york times bits | bigdata gets its own photo album advertisement search subscribe now log in 0 settings close search search sponsored by site search navigation search nytimes.com ____________________ clear this text input go 1. loading... see next articles see previous articles site navigation site mobile navigation supported by bits - business, innovation, technology, society ____________________ search bigdata gets its own photo album by steve lohr september 13, 2012 6:00 am september 13, 2012 6:00 am rick smolan, the photographer and impresario of media projects, has tackled all sorts of big subjects over the years, from countries (“a day in the life of australia” in 1981) to drinking water (“blue planet run” in 2007). he typically recruits about 100 photographers for each, and their work is crafted into classy coffee-table books of striking photographs and short essays. but mr. smolan concedes that his current venture has been “by far the most challenging project we’ve done.” small wonder, given his target: bigdata. john guttag, left, and collin stultz developed software that sifts discarded data from heart-monitoring machines looking for signs that patients are at high risk for a second heart attack. jason grow/the human face of bigdatajohn guttag, left, and collin stultz developed software that sifts discarded data from heart-monitoring machines looking for signs that patients are at high risk for a second heart attack. massive rivers of digital information are a snooze, visually. yet that is the narrow, literal-minded view. mr. smolan’s new project, “the human face of bigdata,” which is being formally announced on thursday, focuses on how data, smart software, sensors and computing are opening the door to all sorts of new uses in science, business, health, energy and water conservation. and the pictures are mostly of the people doing that work or those being affected. in these digital times, the book is only one part of the bigdata project. later this month, on sept. 25, a software application for iphones and android phones will be released. the idea is to get as many people from around the world as possible to use the application. the program will be able to collect data on travel and movement (through the smartphone’s gps and accelerometer), food (take a picture and shortly after the program identifies the food, including estimates of calories and fat content) and attitudes (the user answers questions posed by the app). the data will be fed into a “measure our world” database, and people can see how their habits and attitudes compare with others by, say, where a person lives, gender and age. later, on nov. 8, a bigdata-related program for students, in collaboration with tedyouth, will get under way. when the book is released on nov. 20, some 10,000 copies will be delivered by federal express to influential people around the world, mr. smolan said. it’s an eclectic group, including president obama; carmelita jeter, the olympic sprinter; anna wintour, editor of vogue; sonia gandhi, president of the indian national congress; and jiang jiemin, chairman of petrochina. for the less influential who are not getting free copies, the price will be $50, with a portion of the proceeds going to charity. an ipad application, based on the book but including interactive features, will come soon after. and a documentary film, scheduled for next year, is planned. mr. smolan said he got interested in doing a bigdata project because friends in silicon valley were talking about the growing abundance of internet-era data and its potential. at first, he said, such conversations struck him as “hot air and buzzwords.” but the more he listened, the more he became convinced something significant was happening. “it reminded me of the early days of the internet,” mr. smolan said. “cyberspace was the buzzword then. but soon you started to see that things were taking off.” bigdata technology, mr. smolan said, makes it possible to measure things as never before in real time. the result, he said, could someday be a “planetary nervous system.” and guesswork and projections will give way to knowledge, and better decisions and policy-making. low-cost sensors, real-time data collection, high-speed processing and data-visualization tools, he said, might mean “people could not deny global warming because you could see it happening right in front of you on the screen.” shwetak patel developed technology that measures energy and water use in homes. peter menzel/the human face of bigdatashwetak patel developed technology that measures energy and water use in homes. mr. smolan visited recently to offer a glimpse of what will be in “the human face of bigdata” and the imaginative photo composition involved in bringing technical subjects to life. one photograph shows shwetak patel, an assistant professor at the university of washington, who has developed technology that measures energy and water use in homes; with wireless sensors and clever software to determine what appliances and gadgets in a home use the most electricity and water, the software suggests ways to conserve — information delivered graphically on an ipad. the photo shows young mr. patel in the backyard of his cousin’s house in hayward, calif., with his cousin’s family, surrounded by what looks to be every single appliance, digital device, faucet and toilet in the household. another photo illustrates software technology that captures previously discarded data from heart-monitoring electrocardiogram machines. the software program sifts the data, looking for subtle heart abnormalities that identify patients that are at high risk of suffering a second heart attack within a year. the photo shows two m.i.t. scientists, john guttag and collin stultz, who developed the technology, standing in a small mountain of paper, which is 10 hours of printout data from an e.k.g. machine. michael cogliantry/the human face of bigdataa. j. jacobs enlisted all kinds of sensors in pursuit of a healthier lifestyle. still another photo accompanies an essay by a.j. jacobs, an author and journalist, who enlisted all kinds of sensors, including fitbit for movement and zeo sleep manager, in pursuit of a healthier lifestyle. “i crave maximum data about myself,” mr. jacobs writes. “i am a quantification fiend.” mr. jacobs has already written a book on his quest, “drop dead healthy,” published earlier this year. his picture, which also appears on the inside cover his book, shows mr. jacobs sporting his devices and other visual props, holding a barbell in one hand and a head of broccoli in the other. emc, the data storage giant, is the main sponsor of “the human face of bigdata.” jeremy burton, emc’s chief marketing officer, first met mr. smolan more than a year ago. the introduction came from an emc board member, and mr. burton agreed to meet mr. smolan for coffee for a half-hour in burlingame, calif. mr. burton figured the obligatory meeting would soon be over, but it lasted two and a half hours — “the longest cup of coffee i’ve ever had,” he said. emc is in the business of bigdata. and mr. smolan’s approach seemed original and appealing to mr. burton. “let’s face it, most of the discussion of bigdata is a geekfest — people talking about hadoop and other technical tools,” mr. burton said. “rick smolan wanted to focus on how bigdata is affecting humanity.” typically, mr. smolan tries to line up a handful of sponsors for his projects. but mr. burton decided that emc would be almost the sole backer of the project (fedex is contributing the delivery of the 10,000 books on the publication day). emc, mr. burton said, had no editorial control and the cost was “a big chunk of change,” a multimillion-dollar expenditure. “it is a risk, but the elements of the project from the book to the ipad application seem really cool,” he said. “and we’ll get the halo effect.” a version of this article appears in print on 09/17/2012, on page b4 of the newyork edition with the headline: putting a face on the world of bigdata. bigdata, bigdata, books, emc, federal express, rick smolan what's next loading... * previous post the iphone 5: pogue’s first impressions * next post social networks can affect voter turnout, study says visit the technology section for complete coverage of the industry. » most viewed 1. 1. when a unicorn start-up stumbles, its employees get hurt 2. 2. uber’s no-holds-barred expansion strategy fizzles in germany 3. 3. at c.d.c., a debate behind recommendations on cellphone risk 4. 4. yahoo’s brain drain shows a loss of faith inside the company 5. 5. fans demand details after death of a 13-year-old youtube star latest from bits * amazon subsidiary gains approval to ship ocean freight * daily report: tech earnings season begins with a warning * surveymonkey names new chief, its second in six months * the zappos exodus continues after a radical management experiment * daily report: the netflix-amazon connection follow us on @nytimesbits on twitter * twitter follow advertisement go to home page » site index the new york times site index navigation news * world * u.s. * politics * n.y. * business * tech * science * health * sports * education * obituaries * today's paper * corrections opinion * today's opinion * op-ed columnists * editorials * contributing writers * op-ed contributors * opinionator * letters * sunday review * taking note * room for debate * public editor * video: opinion arts * today's arts * art & design * artsbeat * books * dance * movies * music * n.y.c. events guide * television * theater * video games * video: arts living * automobiles * crossword * food * education * fashion & style * health * jobs * magazine * n.y.c. events guide * real estate * t magazine * travel * weddings & celebrations listings & more * classifieds * tools & services * times topics * public editor * n.y.c. events guide * blogs * multimedia * photography * video * nyt store * times journeys * subscribe * manage my account subscribe * subscribe * times insider * home delivery * digital subscriptions * nyt opinion * crossword * email newsletters * alerts * gift subscriptions * corporate subscriptions * education rate * mobile applications * replica edition * international new york times * © 2016 the new york times company * contact us * work with us * advertise * your ad choices * privacy * terms of service * terms of sale * site map * help * site feedback * subscriptions #bits » bigdata shrinks to grow comments feed daily report: judge upholds searches of travelers’ devices at border why hire a p.r. firm when there’s taskrabbit? rss 2.0 alternate sections home search skip to content the new york times bits | bigdata shrinks to grow advertisement search subscribe now log in 0 settings close search search sponsored by site search navigation search nytimes.com ____________________ clear this text input go 1. loading... see next articles see previous articles site navigation site mobile navigation supported by bits - business, innovation, technology, society ____________________ search bigdata shrinks to grow by quentin hardy january 1, 2014 10:00 am january 1, 2014 10:00 am photo anthony goldbloom, kaggle's founder and chief executive. anthony goldbloom, kaggle's founder and chief executive.credit what’s next for bigdata? maybe a little thinning down of the industry and a little more focus. in fact, it may be underway. google trends shows searches of the term “bigdata” peaked in october, ending a nearly ceaseless climb that began three years earlier. google doesn’t offer absolute numbers, but in that time there was a hundredfold increase in interest in the term. that seems a reasonable match to the number of start-ups, public relations pitches and gurus using the term over this period. all high-tech hype cycles must end, and usually when they do there’s a crash. some pundits gloat at bad ideas that got funded and fell apart. the true believers set to building something durable from the remnants of the boom. in some cases, like the ’90s internet hype, the outcome more than fulfills the promise, but few see it coming. in the case of bigdata, this probably means less focus on back-end technologies like new types of storage or database frameworks, and a rethinking about how best to integrate human knowledge, algorithms and diverse sets of data. in december, the company kaggle changed its bigdata business model in important ways. kaggle had been a darling of bigdata followers for its contests that sought the world’s best statistical analysts. now kaggle has decided to focus on a few specific industries, starting with oil and gas. “we liked to say ‘it’s all about the data,’ but the reality is that you have to understand enough about the domain in order to make a business,” said anthony goldbloom, kaggle’s founder and chief executive. “what a pharmaceutical company thinks a prediction about a chemical’s toxicity is worth is very different from what clorox thinks shelf space is worth. there is a lot to learn in each area.” oil and gas, which for kaggle means mostly fracking wells in the united states, have well-defined data sets and a clear need to find working wells. while the data used in traditional oil drilling is understood, fracking is a somewhat different process. variables like how long deep rocks have been cooked in the earth may matter. so does which teams are working the fields, meaning early-stage proprietary knowledge is also in play. that makes it a good field to go into and standardize. kaggle may also work in the drug industry again, as well as insurance, mr. goldbloom said. “we will still do competitions, they are a great engine for finding talent,” he said, “but our returns as a business will be higher if we focus.” there is reason to think so. kaggle’s sharper focus follows the odyssey of the climate corporation, which in its original incarnation as weatherbill tried to sell predictive data about weather to farmers, house painters and golf courses. after it focused on agriculture, the climate corporation was bought in october by monsanto for $930 million. another company that has looked at bigdata problems in an industry-specific way is palantir. in december, palantir, which initially focused on government security work and has moved into areas like finance, disaster relief and pharmaceuticals, raised $100 million on a valuation of $9 billion. in september, palantir had raised money at a valuation of $6 billion. not everyone believes bigdata has to narrow its focus. “predictive modeling is still going to change the world in every area,” said jeremy howard, a close collaborator with mr. goldbloom who left kaggle over the change in direction. he said he was focusing on building new kinds of software that could better learn about the data it was crunching and offer its human owners insights on any subject. “a lone wolf data scientist can still apply his knowledge to any industry,” he said. “i’m spending time in areas where i have no industrial knowledge and finding things. i’m going to have to build a company, but first i have to spend time as a lone wolf.” a version of this article appears in print on 01/06/2014, on page b5 of the newyork edition with the headline: bigdata shrinks to grow. agriculture and farming, computers and the internet, data-mining and database marketing, kaggle inc, oil (petroleum) and gasoline what's next loading... * previous post daily report: judge upholds searches of travelers’ devices at border * next post why hire a p.r. firm when there’s taskrabbit? visit the technology section for complete coverage of the industry. » most viewed 1. 1. when a unicorn start-up stumbles, its employees get hurt 2. 2. uber’s no-holds-barred expansion strategy fizzles in germany 3. 3. at c.d.c., a debate behind recommendations on cellphone risk 4. 4. yahoo’s brain drain shows a loss of faith inside the company 5. 5. fans demand details after death of a 13-year-old youtube star latest from bits * amazon subsidiary gains approval to ship ocean freight * daily report: tech earnings season begins with a warning * surveymonkey names new chief, its second in six months * the zappos exodus continues after a radical management experiment * daily report: the netflix-amazon connection follow us on @nytimesbits on twitter * twitter follow advertisement go to home page » site index the new york times site index navigation news * world * u.s. * politics * n.y. * business * tech * science * health * sports * education * obituaries * today's paper * corrections opinion * today's opinion * op-ed columnists * editorials * contributing writers * op-ed contributors * opinionator * letters * sunday review * taking note * room for debate * public editor * video: opinion arts * today's arts * art & design * artsbeat * books * dance * movies * music * n.y.c. events guide * television * theater * video games * video: arts living * automobiles * crossword * food * education * fashion & style * health * jobs * magazine * n.y.c. events guide * real estate * t magazine * travel * weddings & celebrations listings & more * classifieds * tools & services * times topics * public editor * n.y.c. events guide * blogs * multimedia * photography * video * nyt store * times journeys * subscribe * manage my account subscribe * subscribe * times insider * home delivery * digital subscriptions * nyt opinion * crossword * email newsletters * alerts * gift subscriptions * corporate subscriptions * education rate * mobile applications * replica edition * international new york times * © 2016 the new york times company * contact us * work with us * advertise * your ad choices * privacy * terms of service * terms of sale * site map * help * site feedback * subscriptions #andrei macsin's posts - bigdata news comments - the 7 most unusual applications of bigdata you’ve ever seen! - bigdata news bdn daily press releases - bigdata news ____________________ search * sign up * sign in bigdata news * data science central * my page * bigdata jobs * data science book * dsc digest * top links * webinars * search * contact us subscribe to our newsletter * all blog posts * my blog * add the 7 most unusual applications of bigdata you’ve ever seen! * posted by andrei macsin on november 3, 2015 at 5:52pm * view blog guest blog post by bernard marr it’s all well and good to talk about customer experience and managing inventory flow, but what has bigdata done for me lately? i’ve rounded up seven of the most interesting — and unique — applications for bigdata i’ve seen recently and how they may be impacting your life. [bor55.png] bigdata billboards outdoor marketing company route is using bigdata to define and justify its pricing model for advertising space on billboards, benches and the sides of busses. traditionally, outdoor media pricing was priced “per impression” based on an estimate of how many eyes would see the ad in a given day. no more! now they’re using sophisticated gps, eye-tracking software, and analysis of traffic patterns to have a much more realistic idea of which advertisements will be seen the most — and therefore be the most effective. iphone’s researchkit apple’s new health app, called researchkit, has effectively just turned your phone into a biomedical research device. researchers can now create studies through which they collect data and input from users phones to compile data for health studies. your phone might track how many steps you take in a day, or prompt you to answer questions about how you feel after your chemo, or how your parkinson’s disease is progressing. it’s hoped that making the process easier and more automatic will dramatically increase the number of participants a study can attract as well as the fidelity of the data. bigdata and foraging the website fallingfruit.org combined public information from the u.s. department of agriculture, municipal tree inventories, foraging maps and street tree databases to provide an interactive map to tell you where the apple and cherry trees in your neighborhood might be dropping fruit. the website’s stated goal is to remind urbanites that agriculture and natural foods do exist in the city — you might just have to access a website to find it. bigdata on the slopes ski resorts are even getting into the data game. rfid tags inserted into lift tickets can cut back on fraud and wait times at the lifts, as well as help ski resorts understand traffic patterns, which lifts and runs are most popular at which times of day, and even help track the movements of an individual skier if he were to become lost. they’ve also taken the data to the people, providing websites and apps that will display your day’s stats, from how many runs you slalomed to how many vertical feet you traversed, which you can then share on social media or use to compete with family and friends. bigdata weather forecasting applications have long used data from phones to populate traffic maps, but an app called weathersignal taps into sensors already built into android phones to crowdsource real time weather data as well. the phones contain a barometer, hygrometer (humidity), ambient thermometer and lightmeter, all of which can collect data relevant to weather forecasting and be fed into predictive models. yelp hipster watch whether you want to hang with the hipsters or avoid them, yelp has you covered. with a nifty little search trick they call the word map, you can search major cities by words used in reviews — like hipster. the map then plots the locations for the reviews in red. the darker the red, the higher the concentration of that word used in reviews — and when it comes to hipsters, ironic tee shirts and handlebar mustaches. even bigdata bras? website true&co. is using bigdata to help women find better fitting bras. statistics show that most women wear the wrong bra size, and so the website has stepped up to try to solve that problem. customers fill out a fit questionnaire on the site, and based on the responses, an algorithm suggests a selection of bras to choose from. the company’s in-house brand is even developed and designed based on feedback from customers and data the company has collected. the possibilities of using bigdata are endless and it might be time to find the bigdata applications in your business. have you seen any fascinating or unusual bigdata projects lately? let me know about them in the comments below! about : bernard marr is a globally recognized expert in strategic metrics and data. he helps companies and executive teams manage, measure, analyze and improve performance. his new book is: bigdata: using smart bigdata, analytics and metrics to make bette... dsc resources * career: training | books | cheat sheet | apprenticeship | certification | salary surveys | jobs * knowledge: research | competitions | webinars | our book | members only | search dsc * buzz: business news | announcements | events | rss feeds * misc: top links | code snippets | external resources | best blogs | subscribe | for bloggers additional reading * data scientist reveals his growth hacking techniques * 10 modern statistical concepts discovered by data scientists * top data science keywords on dsc * 4 easy steps to becoming a data scientist * 13 new trends in bigdata and data science * 22 tips for better data science * data science compared to 16 analytic disciplines * how to detect spurious correlations, and how to find the real ones * 17 short tutorials all data scientists should read (and practice) * 10 types of data scientists * 66 job interview questions for data scientists * high versus low-level data science follow us on twitter: @datasciencectrl | @analyticbridge views: 1544 tags: like 3 members like this share tweet * < previous post comment you need to be a member of bigdata news to add comments! join bigdata news rss welcome to bigdata news sign up or sign in follow us @datasciencectrl | rss feeds on data science central bdn daily press releases why we should all be rooting for women in technology looker secures $48 million series c funding from kleiner perkins enterprises face unnecessary risks to data loss and downtime, says report by forbes insights and bmc predictive analytics firm rapidminer raised $16mm in equity financing e-commerce giant showroomprive.com uses predictive analytics to anticipate and reduce churn wordstat 7.1: geospatial intelligence meets text analytics analytics engines and almac group awarded innovate uk funding to develop bigdata analytics platform for bioinformatics comtrade facilitates global access to cern research the expo line won't reduce l.a. traffic – but that doesn't mean you shouldn't take it consumerizing it with tibco simplr and tibco activematrix bpm 4.0 © 2016 bigdatanews.com is a subsidiary of datasciencecentral llc and not affiliated with systap powered by ning badges | report an issue | terms of service hello, you need to enable javascript to use bigdata news. please check your browser settings or contact your system administrator. [p?c1=2&c2=6770185&cv=2.0&cj=1] #andrei macsin's posts - bigdata news comments - bigdata = hype; but why that doesn't matter - bigdata news bdn daily press releases - big data news ____________________ search * sign up * sign in bigdata news * data science central * my page * bigdata jobs * data science book * dsc digest * top links * webinars * search * contact us subscribe to our newsletter * all blog posts * my blog * add bigdata = hype; but why that doesn't matter * posted by andrei macsin on october 20, 2015 at 3:59pm * view blog guest blog post by bernard marr from time to time, you still come across someone with the opinion that bigdata is nothing more than a fad, which will be forgotten about soon enough. you might not expect to hear this from me, but they’re actually right. well – half right, at least! as i’ve written before, i’m not actually a fan of the term “bigdata”, which puts overemphasis on the importance of size. anyone who’s been reading my articles for a while will know that i’m firmly of the opinion that what you do with your data, is far more important than how big it is. [bor55.png] source for picture: gartner and i am sure as more people realize this – as working with extremely large datasets increasingly becomes the norm, rather than something new and exciting – the term “bigdata” may indeed fall out of use. the fact is, bigdata isn’t something which has appeared overnight. ever since we invented digital data storage in the 60s, the amount of data we have been dealing with has increased exponentially with time. even before that, if you really want to go right back to the beginning, ancient civilizations strove to horde as much knowledge as they could in great libraries. our urge and ability to collate and analyze information seems to have always been something which has distinguished us from other animals, and it isn’t going to leave us any time soon. so the value of information, which is based on data, has always been apparent. however during the last half-century we certainly experienced a rapid acceleration in our capacity to both store larger amounts of data and analyze it in smarter ways. first with the encroachment of digital storage and microprocessors into every aspect of society, then with the internet and smart phones and now, wearables. no doubt, there is a lot of hype around bigdata. big names and brands which have emerged onto the market have profited from this, pushing their own ideas of what bigdata means and how you should go about it. wherever you find hype, you find hot air. this is particularly true in the tech world where those who get on board first stand to make huge amounts of money. not everything that emerges in the early days of something as game-changing as the “bigdata revolution” will stand the test of time. i assume most of my readers are probably old enough to remember the final decade of the last century, when the internet really got popular. those of us who had grown up using computers realized immediately how revolutionary it was going to be – that in fact, nothing would ever be the same again. but pundits from outside of the tech world – somewhat wary of the grandiose claims being made – continued for a long time to insist that it was a passing fad. many saw the bursting of the dot-com bubble as validation of that belief. but, although a lot of people lost a lot of money, the internet, as we all know, endured. in the digital age, other terms have risen in popularity only to drop into disuse as the principles they represent become adopted into everyday business use. they’re just adopted into everyday business life to the extent that we don’t need specific names for them any more. or ideas behind them are rolled up into newer buzzwords. the bigdata buzzword has definitely swallowed up most aspects of previous buzz terms like management information systems, business intelligence, or analytics. so, while in 10 years’ time the terminology might have changed, we will still be talking about data, and analysis, and the juicy insights we get when we mix them together. so when someone tells you that “bigdata is just a fad”, what they mean is that we won’t need the label any more. data strategy, collection, storage and analysis are here to stay. as will be the businesses which are savvy enough to use it to strike while the iron is hot. about : bernard marr is a globally recognized expert in bigdata, analytics and enterprise performance. he helps companies improve decision-making and performance using data. he writes about bigdata here at linkedin (please follow him to receive future posts) and feel free to also connect via twitter, facebook andthe advanced performance institute. dsc resources * career: training | books | cheat sheet | apprenticeship | certification | salary surveys | jobs * knowledge: research | competitions | webinars | our book | members only | search dsc * buzz: business news | announcements | events | rss feeds * misc: top links | code snippets | external resources | best blogs | subscribe | for bloggers additional reading * data scientist reveals his growth hacking techniques * 10 modern statistical concepts discovered by data scientists * top data science keywords on dsc * 4 easy steps to becoming a data scientist * 13 new trends in bigdata and data science * 22 tips for better data science * data science compared to 16 analytic disciplines * how to detect spurious correlations, and how to find the real ones * 17 short tutorials all data scientists should read (and practice) * 10 types of data scientists * 66 job interview questions for data scientists * high versus low-level data science follow us on twitter: @datasciencectrl | @analyticbridge views: 452 tags: like 0 members like this share tweet * < previous post * next post > comment you need to be a member of bigdata news to add comments! join bigdata news rss welcome to bigdata news sign up or sign in follow us @datasciencectrl | rss feeds on data science central bdn daily press releases why we should all be rooting for women in technology looker secures $48 million series c funding from kleiner perkins enterprises face unnecessary risks to data loss and downtime, says report by forbes insights and bmc predictive analytics firm rapidminer raised $16mm in equity financing e-commerce giant showroomprive.com uses predictive analytics to anticipate and reduce churn wordstat 7.1: geospatial intelligence meets text analytics analytics engines and almac group awarded innovate uk funding to develop bigdata analytics platform for bioinformatics comtrade facilitates global access to cern research the expo line won't reduce l.a. traffic – but that doesn't mean you shouldn't take it consumerizing it with tibco simplr and tibco activematrix bpm 4.0 © 2016 bigdatanews.com is a subsidiary of datasciencecentral llc and not affiliated with systap powered by ning badges | report an issue | terms of service hello, you need to enable javascript to use bigdata news. please check your browser settings or contact your system administrator. [p?c1=2&c2=6770185&cv=2.0&cj=1] #michael meyers's posts - bigdata news comments - 5 ways bigdata is making a splash in the insurance industry - bigdata news bdn daily press releases - bigdata news ____________________ search * sign up * sign in bigdata news * data science central * my page * bigdata jobs * data science book * dsc digest * top links * webinars * search * contact us subscribe to our newsletter * all blog posts * my blog * add 5 ways bigdata is making a splash in the insurance industry * posted by michael meyers on september 15, 2015 at 1:02pm * view blog bigdata insurance 5 ways bigdata is making a splash in the insurance industry there’s conflicting opinions on when the term bigdata really came onto the scene, but it is undeniable that its notoriety has skyrocketed in the past few years. bigdata has become the hot topic buzzword around the internet, as is made obvious in the google trends graph below. one by one as analytics solutions became more accessible, industries found different ways to leverage the bigdata revolution to their best advantage in their field. virtually every industry, from manufacturing and retail to healthcare and education, has found a use for the ever-growing labyrinth of data resources. the insurance industry is no exception in the bigdata revolution. the ernst and young 2015 global insurance outlook claims “technology” is the one word that encompasses the insurance industry right now. the report states, “insurers across all regions are capitalizing on data analytics, cloud computing, and modeling techniques to sharpen their market segmentation strategies, reduce claims fraud and strengthen underwriting and risk management.” bigdata trending although there are many ways that insurers are utilizing bigdata and data analytics to benefit their bottom line, there are 5 areas being impacted the most. 1. data for streamlined underwriting insurance underwriters are faced with the daily challenge of providing policy recommendations that are both fair to the consumer, as well as protect the best interest of the insurance company. bigdata is changing the future for underwriters as they must adapt to their new roles as data analysts as well as underwriters. a study by marketforce, the chartered insurance institute (cii) and the chartered institute of loss adjusters in conjunction with ordnance found that most underwriters are taking the evolution of their industry in stride. the study revealed that 9 out of 10 underwriters see the potential in access to real-time claims data to improve pricing accuracy. real-time data mining is streamlining the underwriting process by providing accurate, current insights that could have taken days to locate and consolidate. with access to quality data sources, underwriters are able to complete processes in less time and with better accuracy. real-time data facilitates streamlined processes leading to higher placement rates and a much faster underwriting cycle. risk 2. data for personalized policies access to multichannel data sources gives underwriters the ability to base premium costs and policy parameters on a more realistic view of risk as opposed to generalized assumptions based on factors such as location and age. consequently, this integration of highly granular and individualized characteristics into the underwriting cycle is driving a more personalized consumer experience. customers who feel they are receiving fair treatment instead of at the mercy of generalizations receive a more positive experience and are more inclined to remain loyal clients. 3. identifying customers at risk of cancellation leveraging bigdata insights is well known for its ability to provide quality prospects for businesses, but another lesser known feature is its ability to shed light on low quality prospects or frustrated clients. advanced analytics tools allow insurers to target individuals who are apt to be a long term loyal customer, and also to weed out individuals who are a high risk of canceling coverage. predictive analytics is used to track and reveal signal behaviors that indicate an impending cancellation. this allows insurance agents to reach out to unhappy consumers before their final decision has been made, and tailor opportunities to encourage them to stay with the company. 4. identifying risk of fraud fraudulent claims are an unfortunately common occurrence afflicting the insurance industry. the coalition of insurance fraud estimates that nearly $80 billion in fraudulent claims are made annually in the united states. this staggering statistic has led to heightened awareness and the use of predictive data analytics to detect applicants with a higher propensity to commit fraud. additionally, after a claim has been made insurers can use data mining to track digital and social channels for evidence of fraudulent behavior. 5. customer relationship management data has become an indispensable tool for insurers to implement positive consumer relationships and intuitive acquisition strategies. delivering exceptional customer experiences through a comprehensive customer profile provides better understanding of customers’ preferences, lifestyles, and other key characteristics, which allow insurers to deliver highly relevant and personalized offers. by investing in data integration and quality solutions, insurers can build a fully integrated marketing database that pulls data from various locations to improve customer intelligence and ensure a positive experience. concerning acquisition, insurers use daas to monitor behavior in real-time to target high quality prospects. one example is a national life insurance company who knows that 41% of life insurance purchases are motivated by life stage events. armed with this knowledge, they used daas to examine potential data sources such as baby shower registries (arrival of a new child), wedding registries (inclination of customers to plan ahead for spouse’ sake), or various social signals that indicate retirement or improved financial state that places individuals in a primed mental state to receive advertisements on life insurance. tragedies can also influence insurance purchases. the death of a loved one, a natural disaster, or nearby crime all encourage an interest in planning for the future. bigdata is impacting all industries in different ways, but it’s certain that the insurance industry is one of the leading innovators of daas implementation. as industry leaders continue to invest in bigdata solutions it will soon become as ingrained into the insurance industry’s processes as underwriting itself. continue... views: 910 tags: big, data, insurance like 0 members like this share tweet * < previous post * next post > comment you need to be a member of bigdata news to add comments! join bigdata news rss welcome to bigdata news sign up or sign in follow us @datasciencectrl | rss feeds on data science central bdn daily press releases why we should all be rooting for women in technology looker secures $48 million series c funding from kleiner perkins enterprises face unnecessary risks to data loss and downtime, says report by forbes insights and bmc predictive analytics firm rapidminer raised $16mm in equity financing e-commerce giant showroomprive.com uses predictive analytics to anticipate and reduce churn wordstat 7.1: geospatial intelligence meets text analytics analytics engines and almac group awarded innovate uk funding to develop bigdata analytics platform for bioinformatics comtrade facilitates global access to cern research the expo line won't reduce l.a. traffic – but that doesn't mean you shouldn't take it consumerizing it with tibco simplr and tibco activematrix bpm 4.0 © 2016 bigdatanews.com is a subsidiary of datasciencecentral llc and not affiliated with systap powered by ning badges | report an issue | terms of service hello, you need to enable javascript to use bigdata news. please check your browser settings or contact your system administrator. [p?c1=2&c2=6770185&cv=2.0&cj=1] #publisher cloudtweaks.com cloudtweaks.com » feed cloudtweaks.com » comments feed alternate alternate cloud tech news menu * hot topics + close + bigdata + cloud computing + fintech + healthcare + infographics + internet of things + security + close * comics * vendors * sponsor * contact * * * * how bigdata is influencing web design how bigdata is influencing web design posted october 27, 2015 by cloudtweaks * * * * * * * how bigdata is influencing web design for all you non-techies… you’re probably wondering what bigdata is (i know i was….a few years back) so let’s get the definitions out of the way so we’re on the same page, okay? bigdata is a lot of data – really, it is. it is a catchy and rather new term that refers to a large volume of both data, structured and unstructured, that is impossible to process using traditional software techniques. it takes on terms like exabytes which are 1,024 pentabytes, where a pentabyte is 1024 terabytes (now you see why it’s call big?). slingshotdata-cloud while the term does seem to lean to the massive amounts of information, it is sometimes used by vendors when referring to the technology needed to handle large amounts of data. so it can be referring to both the data and the technology needed for it, okay? now that we have that out of the way… you can appreciate the need to develop the best possible web designs that can handle these massive amounts of data, right? and it’s not just about containing the data but using it in a way that will help you actualize the goals of the website. you see, back in the day when the internet was still a new development, people were impressed by the ability to have a lot of information at the click of their mouse (all you young ones can google what a mouse was). it was the quantity of data that was impressive but now, anyone can access this information pretty much on their own; so what is the incentive to get on your website? design, design, design! design bigdata it is what you do with the data that will determine whether you get as many visitors to your site as you can or not. this refers to both the presentation and the arrangement of your data in a way that is useful to your clients, and this will be determined by your web design – something the techies call data-driven design. the benefits of letting your data determine you web design are endless; from enhancing iterative design to getting prompt information on how people interact with the design and the information. you will also get to have data that you can use for future work and possibly automate web development in the future. it is not an entirely foreign concept this data-driven design. many other industries have applied it to improve their products and services. for instance, in the healthcare industry, bigdata in the form of medical history, employment and residential information among others, is used to improve patient care as well as determine the best possible course of treatment. so it is the same concept only referring to web design. critical questions before you embark in the designing for bigdata, there are a few questions you will have to answer. 1. what are your objectives? what exactly would you like the design to achieve for you? do you just want to join different datasets or find information about high-value clients faster? begin with the end in mind but also be open to redefining those goals as you go through the data you have. you just might find something interesting. 2. what is the nature of your data? you need to know where you data is coming from, how much data you have moving through your system and where this data integrate within the current system. this will help you consolidate all the data about each client from all the different sources and convergence points within your current system. this way, you don’t lose any information. 3. which platform? this will be determined by the kind of data you have as well as the volume. based on this information, you can then decide what will work with you specific needs in order to provide adequate support. important things to remember once you have all these questions answered, there are a few things you will have to keep in mind as you start designing your website. * easy does it don’t flood all your information onto your new platform all at once. bring it in slowly to see how it works and how the staff responds to it. start offline and in small batches before moving to real-time processing of large quantities of data. * develop an updating system your data will need refreshing from time to time in order to remain current and relevant. you will need to find out how you can make these updates to your bigdata platform as easily and quickly as possible. * get a feedback loop you will need datasets that can make the existing systems smarter and this will be done using a feedback loop. these smaller datasets will be able to improve existing applications by providing real-time information from systems that they were previously impervious to or entire unaware of. * establishing and evaluating analytics as your feedback loop grows large enough to link all your data sources, you can start data mining and conducting behavioral analysis to make better predictions and optimize resources. this way, you can stay on top of trends within your industry and give yourself a competitive edge. * privacy is paramount with all that information there is a great chance of losing privacy of your users’ information; that must never happen! emphasize user privacy in your design at every level and especially in niche segments. this is especially true if sensitive user information will be needed during transactions with the website link bank accounts, residential addresses and all that. * training the users bigdata web design will change how things work. this is because it necessitates greater access by the end users in order to give real time. as a result, it is important for organizations to educate their staff on how to use bigdata as a team to achieve the set objective. * it’s an ongoing process transitioning to bigdata web design is an ongoing process. there will be some kinks to iron out and you will have to constantly evaluate the system to see if you are getting what you need from it. it’s all about the experience at the end of the day, using bigdata (also known as business intelligence) to develop a web design is all about creating an experience that is attractive and satisfactory to the user. clients are often more impressed with a personalized experience than a standard, albeit efficient one. the good news is you have all that data; you just need to be smart about how you use it. by jack dawson jack is a web developer and ui/ux specialist at bigdropinc – http://bigdropinc.com. he works at a design, branding and marketing firm, having founded the same firm 9 years ago. he likes to share knowledge and points of view with other developers and consumers on platforms. * author * more articles about cloudtweaks established in 2009, cloudtweaks is recognized as one of the leading authorities in connected technology information and services. we embrace and instill thought leadership insights, relevant and timely news related stories, unbiased benchmark reporting as well as offer green/cleantech learning and consultive services around the world. our vision is to create awareness and to help find innovative ways to connect our planet in a positive eco-friendly manner. in the meantime, you may connect with cloudtweaks by following and sharing our resources. * security survey: enterprises unequipped to detect and deal with attacks - january 13, 2016 * trends: the cio’s business card – chief productivity officer - january 12, 2016 * when artificial intelligence becomes personal - january 11, 2016 * how data privacy reform is wreaking havoc in the cloud - january 11, 2016 * 5 tips to host a successful conference in the cloud - january 11, 2016 view all articles bigdata, cloud computing, contributors, education, popular, startups ← ibm insight 2015: large focus on bigdata backblaze study: backing up the bigdata backup → sorry, comments are closed for this post. cloud thought leaders and contributors write for us - find out more! cloudtweaks is recognized as one of the leading influencers in cloud computing, infosec, bigdata and the internet of things (iot) information. our goal is to continue to build our growing information portal by providing the best in-depth articles, interviews, event listings, whitepapers, infographics and much more. search ____________________ advertising copyright © 2016 cloudtweaks.com view full site connect about us privacy policy contributors whitepapers advertise & sponsor contact us 403 forbidden __________________________________________________________________ nginx #alternate us news rss feed * rankings & advice + education + health + money + travel + cars + law firms * news * u.s. news home [homepage-logo-166x40.png] money facebook twitter money rankings & advice * home * retirement * personal finance + credit cards + my money + the frugal shopper + alpha consumer + the smarter investor + reboot your finances * careers * investing * real estate * 5 ways to buy bigdata stocks data creation is expected to only increase in the years ahead, creating an investment opportunity. binary code with the popularity of tablets, smartphones and other connected devices, data creation should continue growing at a rapid pace. by kira brecht aug. 18, 2015, at 11:36 a.m. + more * * * * * * * digital information is being created at a monstrous pace with the explosion of web applications, e-commerce, online transactions and social media. companies that track, store, analyze and provide database solutions are poised to benefit from the growing bigdata trend. investors looking to get in on the ground floor of stocks that have the sizzle of fledgling dot-coms may find that bigdata companies offer attractive opportunities. "as a species, we now collect and store somewhere between 2 and 3 trillion megabytes of data per day. a lot of that is raw noise, but the patterns that shine through can be extremely valuable to corporate management looking for operating efficiency, better-targeted marketing or even just quicker damage control when the chatter starts going the wrong way," says hilary kramer, hedge fund manager, equity analyst and editor of several financial newsletters. looking ahead, data creation is expected to only increase in the years ahead. "we are seeing an explosion of data that is being created within both the consumer and corporate area. our view is that you will see data growth in the range of 30 to 40 percent on an annualized basis over the next decade," says angelo zino, senior industry analyst, equity research at s&p capital iq. the key trends driving expansion in bigdata include growth in the mobile space as consumers create new data via their phones, the adoption of the cloud and social media, zino says. "you are able to capture a lot more data today than ever before, and mobility is driving that," he says. every device with an embedded processor creates data, and theoretically all of this data could be analyzed and used for business needs. "it's estimated that just over 10 percent of data is generated by machines, but industry analysts expect that to rise to 40 percent within the next five years or so. earlier in the year, ibm stated that around 90 percent of the data created by mobile and smart devices is never analyzed, meaning there is a massive opportunity," says tyler laundon, chief analyst of small-capitalization stocks and growth stocks at wyatt investment research. potential stocks run the gamut from mature companies like ibm, which keeps folding new bigdata applications into its watson analytic platform, to relatively small plays. "the immediate appeal for investors is that there's vast potential growth here in a market that's otherwise starving for signs of the next high-tech revolution," kramer says. here's a look at five stocks analysts like now: international business machines corp. (ticker: ibm) is a major player in the enterprise software, it services and hardware arena. it may be an old-school stock, but it does offer exposure to the bigdata trend. "ibm shares look undervalued, and the company does have some exposure to bigdata, though it is a legacy player and has a lot more going on than just analytics," says pete wahlstrom, director of technology equity research at chicago-based morningstar, an investment research firm. morningstar rates "fair value" for ibm at $178 per share. ibm has become increasingly focused on bigdata and watson analytics, laundon says. "earlier in the year it announced plans to invest $3 billion to build an internet of things division. this specialized division will work to harness and analyze the vast amount of data collected and created by tablets, smartphones and connected devices. ibm's push into bigdata could help the entire space over the next year by increasing [merger and acquisition] speculation and investor interest," laundon says. hortonworks (hdp) is a leading player in the database solution space. "the company essentially sells the 'picks and shovels' to would-be data miners, especially smaller businesses that may not have the scale to support their own supercomputing solutions or hire ibm," kramer says. on a relative basis, hortonworks is still a small player in the overall tech world, with revenue now cresting more than $80 million a year. "it's still burning roughly that much cash as it ramps up. granted, the company is expanding its bookings at a rate of 40 to 50 percent a year as fortune 500 partners roll out their bigdata programs on top of the hdp platform, so this play could pay off within a matter of years," kramer says. splunk (splk) is a firm that specializes in machine-generated data. "this is data that is produced by all varieties of electronic devices, including home appliances, electrical meters, mobile devices, automobiles, medical devices, gps devices and radio-frequency id tags, among others," laundon says. "splunk makes money, in part, depending on how often subscribers access data sets and how much data they sift through. the more data users can access, the more desirable the platform becomes, and the more revenue splunk will generate." emc corp. (emc) is a leading provider of it storage hardware solutions to promote data backup. s&p capital iq has a "strong buy" on emc, with 12-month target at $30. "emc is the name we are telling investors to buy when it comes to bigdata," zino says. emc has an attractive financial position with net cash per share over $4.50, he says. "this will allow them, if need be, to capitalize on potential acquisitions. we view the sum of the parts as greater than the whole. the market is significantly undervaluing its assets," zino says. tableau software inc. (data) provides business analytics and software products. tableau is a "next-generation" bigdata player with shares that, according to wahlstrom, are trading in 3-star territory on the company's rating scale, meaning data stock should offer a fair return. morningstar rates "fair value" at $90 for tableau software. watch for mergers and acquisitions. the hot nature of these companies leaves the sector ripe for mergers and acquisitions. any of the smaller players can get taken off the board by a tech giant like ibm at any time, kramer says. "one of my favorite picks in the space was actually yodlee (ydle), which harvested consumer financial data in a format that banks and hedge funds could interpret for their own sales or trading purposes. yodlee got taken out for roughly a 50 percent premium. build a portfolio out of likely acquisition targets, and you might end up with shares of the ultimate winners after all," kramer says. tags: investing money technology ibm stock market financial advisors * * * * * * + more * kira brecht kira brecht is a financial journalist who writes extensively on stock, commodity, and foreign exchange markets, investing strategies, the economy and the fed. she was managing editor at sfo (stock, futures & options) magazine for 10 years, creating digital magazine, newsletter and online content aimed at the individual investor. she began her career on the floor of the chicago futures exchanges covering commodity markets for a financial newswire service. follow her on twitter @kirabrecht. you might also like __________________________________________________________________ [?url=%2fcmsmedia%2f22%2f4198151b476df0fca9deeda85e0b0f%2fresizes%2f500 %2f32058forrent.jpg] how renters can build long-term wealth, too by joanne cleaver | june 11, 2015 [?url=%2fcmsmedia%2f4d%2f3a%2f05d24178483884701f8e3c43d43a%2fresizes%2f 500%2f150609-casino-stock.jpg] ‘sinvestments’: investing in low morals for high profits by lou carlozo | june 10, 2015 [?url=%2fcmsmedia%2f47%2f68%2fc5eed0f844198efe4766f812922d%2fresizes%2f 500%2f150609-stockgains-stock.jpg] ride european stock gains with these 4 etfs by kira brecht | june 9, 2015 [?url=%2fcmsmedia%2f4b%2f12%2f0e2bc1d142459f94364a42b9329e%2f160114-anx ious-stock.jpg] 8 tips for the anxious investor by lou carlozo | jan. 14, 2016 [?url=%2fcmsmedia%2f75%2f4e%2f6960a4414854a47989ec05dfd401%2f151019-inv estinggrowth-stock.jpg] why value investing helps investors through market swings by debbie carlson | jan. 14, 2016 [?url=%2fcmsmedia%2f63%2f4d%2fece226274d32bea4cc9007660e22%2fresizes%2f 500%2f160113-rowsofcars-stock.jpg] why wall street hopes for a huge 2016 for auto stocks by david schepp | jan. 13, 2016 [?url=%2fcmsmedia%2f78%2f3e%2f0303d8a54e12bbbdf8e69ae7d8e6%2fresizes%2f 500%2f150817-chinaflag-stock.jpg] why china's market woes will hurt some emerging markets by simon constable | jan. 12, 2016 [?url=%2fcmsmedia%2fcf%2f7f%2fea7b26e74c6e959110b119cda547%2fresizes%2f 500%2f160112-rolledwashingtons-stock.jpg] 7 ways to profit from the rising dollar by kira brecht | jan. 12, 2016 [?url=%2fcmsmedia%2fb1%2f17%2f52e77fed454793861cb30c4a82a6%2f160111-rai lroad-stock.jpg] railroads expected to get back on track in 2016 by tony dreibus | jan. 12, 2016 [?url=%2fcmsmedia%2f3d%2f8e%2f2ca7228a48da9ec3068fd7847c4d%2f141209-inv estingrealestate-stock.jpg] pros and cons of alternative investments by kate stalter | jan. 11, 2016 [?url=%2fcmsmedia%2f06%2fce%2f38d9654349c28f6b4918b79ae7ab%2f160111-com castat-t-editorial.jpg] stock market showdown: comcast vs. at&t by lou carlozo | jan. 11, 2016 [?url=%2fcmsmedia%2f50%2fe2%2f616a95c94e139147f5f45e76bc2f%2f160108-goo gle-editorial.jpg] alphabet in 2016: should you buy goog stock? by lou carlozo | jan. 8, 2016 [?url=%2fcmsmedia%2f55%2f29%2f378f0151407da64bf801bbbd44e8%2f160108-sod acans-stock.jpg] how to invest in beverage stocks by ryan derousseau | jan. 8, 2016 [?url=%2fcmsmedia%2fd3%2fde%2f9d36426048bba658090a627400c6%2fresizes%2f 500%2ffed.jpg] how to fed-proof your portfolio by debbie carlson | jan. 7, 2016 [?url=%2fcmsmedia%2f40%2f9d%2f333b2c3a4b0fa9d8a38e1c3064e8%2fresizes%2f 500%2f160107-uberairbnb-stock.jpg] how to invest in the sharing economy by matt whittaker | jan. 7, 2016 [?url=%2fcmsmedia%2fe4%2fbb%2f03d2eda24f48b821d4c88a7b6848%2fresizes%2f 500%2f12-venice.jpg] why investors should be wary of italy by lou carlozo | jan. 7, 2016 [?url=%2fcmsmedia%2f74%2f11%2f16ed640e4dd68720a1f93d39b3a4%2fresizes%2f 500%2f151214-401kjar-stock.jpg] 6 facts to know about your 401(k) by david schepp | jan. 6, 2016 [?url=%2fcmsmedia%2fce%2fa2%2fe79c17634f099ed0486c2b672f4d%2fresizes%2f 500%2f140902-insurancecoverage-stock.jpg] best and worst insurance stocks for 2016 by jeff brown | jan. 6, 2016 [?url=%2fcmsmedia%2f3e%2fb6%2f5f056459458fa8d001c1ad494333%2fresizes%2f 500%2f141231moneycalendardollars-stock.jpg] january barometer offers clues for the year's stock performance by kira brecht | jan. 5, 2016 [?url=%2fcmsmedia%2f1f%2f30%2fff86106043cf922d64e422ba75bc%2fresizes%2f 500%2f160105-trucksdealership-stock.jpg] truck sales surging as gas prices fall, housing improves by tony dreibus | jan. 5, 2016 [?url=%2fcmsmedia%2f49%2ff2%2fdd44e8b447ed9eb342545a898163%2fresizes%2f 500%2f150720-teenmoney-stock.jpg] how to teach your teenager to invest by christine giordano | jan. 4, 2016 [?url=%2fcmsmedia%2f5e%2f1f%2f9d2a5fbc4d1282b589cb7af10914%2fresizes%2f 500%2f150728-dadsontalk-stock.jpg] educating the next generation of investors by kate stalter | jan. 4, 2016 [?url=%2fcmsmedia%2f52%2fb1%2f4b3b9bc2470d9feefa8db320c6e8%2fresizes%2f 500%2f150430-chineseflag-stock.jpg] 3 factors that will drive u.s. stocks in 2016 by kira brecht | dec. 31, 2015 see more u.s. news advisor finder looking for an advisor? find a financial professional near you. advisors firms search ____________________ search ____________________ 10 investing tips for the rest of 2015 slideshow 10 investing tips for the rest of 2015 13 stocks to buy to bet on china slideshow 13 stocks to buy to bet on china 8 stocks to buy for 'star wars' fans slideshow 8 stocks to buy for 'star wars' fans 7 facts investors should know about insider trading slideshow 7 facts investors should know about insider trading 8 stocks to profit from america's love of burgers slideshow 8 stocks to profit from america's love of burgers 8 inaugural trips down wall street slideshow 8 inaugural trips down wall street 11 stocks that donald trump loves slideshow 11 stocks that donald trump loves 12 steps to a stronger 401(k) slideshow 12 steps to a stronger 401(k) a smart investing plan for 30-somethings slideshow a smart investing plan for 30-somethings 10 ways to build a $1 million nest egg slideshow 10 ways to build a $1 million nest egg 10 long-term investing strategies that work slideshow 10 long-term investing strategies that work 8 rules for investing in a turbulent stock market slideshow 8 rules for investing in a turbulent stock market 12 financial terms every investor should know slideshow 12 financial terms every investor should know 10 mistakes you're making in your 401(k) slideshow 10 mistakes you're making in your 401(k) 8 retirement milestones that affect your investment decisions slideshow 8 retirement milestones that affect your investment decisions find a fund search ____________________ most popular fund categories large growth high yield bond retirement income short-term bond see all fund rankings » find an advisor financial advisors icon search thousands of local advisors to find the best financial advice. * find an advisor in california * find an advisor in texas * find an advisor in florida * find an advisor in new york * find an advisor in michigan * find an advisor in new jersey the smarter investor paying off back taxes can be good for investors how to invest with confidence 5 defensive stocks to buy for 2016 consider these 7 things before you invest 6 tips to protect your investment portfolio when you're sick see more posts news * news home * opinion * national issues * cartoons * photos * videos * special reports * the report rankings & consumer advice * education + colleges + graduate schools + high schools + online programs + community colleges + global universities + arab universities * health + hospitals + doctor finder + diets + nursing homes + health products + health insurance + medicare * money + jobs + financial advisors + etfs + mutual funds + retirement * travel + vacations + cruises + hotels + hotel rewards + airline rewards * cars + new cars + used cars law + law firms [homepage-logo.png] * about u.s. news * contact us * store * site map * press room * advertising info * ads – self-service connect with us: facebook twitter google+ copyright 2016 © u.s. news & world report lp. terms and conditions / privacy policy. #rss feed a social media today community [network-drop-down.png] * [brand-bar-wheel.png] about social media today ______________________________________________________________ * [logo.png] the world's best thinkers on energy & climate ______________________________________________________________ * [logo.png] the world's best thinkers on the urban future ______________________________________________________________ * [logo.png] the world's best thinkers on business intelligence & data analytics ______________________________________________________________ * [logo.png] the world's best thinkers on healthcare ______________________________________________________________ * [logo_0.png] the world's best thinkers on social media sign up | login with → smartdata collective the world's best thinkers on data * home + about us & contact + work with us * post here * analytics + social media analytics + text analytics + sentiment analytics + predictive analytics + modeling + web analytics + workforce analytics * bigdata + social data + data warehousing + visualization + workforce data + data mining + data quality * business intel + business rules + decision management + knowledge management + market research + crm * data management + risk management + transparency + culture & leadership + policy & governance + best practices + privacy * it + mobility + security + hardware + location + cloud computing * software + hadoop + mapreduce + sql + open source + marketing automation + r programming language * reports * webcasts * help + faq posted by: [ryankpic.jpg] ryan kidman posts profile comments follow please login or register to follow this user. what does bigdata mean for the future of social media? like it? 215 comment 1 posted november 18, 2015 keywords: social media analytics, bigdata the social media industry has grown leaps and bounds in the past decade, but some challenges have persisted. major networks such as facebook have struggled to meet revenue targets. facebook’s ipo stock price was based off revenue growth figures that have failed to materialize over the past few years, raising concerns that the site couldn’t monetize its users as well as analysts hoped. companies utilizing social media have also had trouble customizing their messages to the right demographics. social networks are addressing these challenges through the inception of bigdata. they are collecting as much information as possible not only on user demographics, but also on user engagement. the algorithms built on bigdata will revolutionize the social media industry in ways pundits never predicted. the growing role of data science in social media business models while social media engagement has surged in recent years, profitability has barely budged. monetization has been the core focus for facebook, twitter and other leading social networks. bigdata may be the savior they have been searching for as they strive to maximize the revenue of their ever growing user base. bigdata has been a game changer for many industries over the course of the past few years. the market for advanced and predictive analytics (apa) software industry was over $2.2 billion in 2013 and is expected to rise to $3.3 billion within the next two years. algorithms relying on bigdata are playing an especially important role in global businesses. ironically, the social media industry has lagged others in adapting big data. pinterest and auto instagram likes took a major step towards using bigdata this past year. pinterest began developing a business model that involves selling pins for users wishing to expand the reach of their content. in order to launch their new monetization strategy, pinterest acquired significant data via new user registrations. they also kept close track of user engagement metrics to develop policies for the new program. data will help them ensure that the paid pins model will be viable for marketers without diminishing the user experience. instagram released a similar model shortly thereafter. they created a new system that allowed users to purchase items that were listed on instagram. items are tailored to the interests of different user demographics, giving the site a significant edge over most other publishers offering media buys to advertisers. allowing brands to tailor their messages many brands serve multiple demographics, which makes it difficult for them to deliver a uniform message that resonates with every customer. helena schwenk, principle analyst at mwd advisors, told forbes that big data has allowed them to create customized messages that appeal to the whims of their customers through social media. “[employees] need to respond in a proactive and timely manner on the social channel of choice and be able to tailor the communication or content that they provide to different audiences with the right reply, the right response, the right content and the right tone of voice,” said schwenk. facebook currently has 1.49 billion active users. this gives marketers access to a huge pool of potential customers. however, the ability to narrowly target their demographic is arguably even more important. facebook has been collecting data on their users for years, giving marketers more control over the delivery of their messages. challenges with bigdata social networks will be able to continue improving the user experience and the service for their marketers as they accumulate more data. however, a new report from bi intelligence states that they still face one major flaw: 90% of the data they gather is spontaneously generated, which means that it is difficult to capture and capitalize on. social networks are improving their algorithms and server infrastructure to better capture data and use it to serve their customers and advertisers. bigdata may be the ticket to helping them reach their monetization goals in the years to come. * login or register to post comments * [ryankpic.jpg] connect: twitter linkedin website authored by: ryan kidman ryan kh is a bigdata and analytics expert, marketing digital products on amazon's envato. he is not just passionate about latest buzz and tech stuff but in fact he's totally into it. ryan enjoys exploring all things cloud and is a music enthusiast.follow ryan’s daily posts on clearworldfinance. email him at [email protected], and follow him on twitter, linkedin and tumblr. see complete profile would you like to contribute to this site? get started » other posts by ryan kidman business intelligence: how to make your workplace perform smarter - january 6, 2016 how to recycle old it equipment and keep your data safe - january 4, 2016 using bigdata in 2016: how it can help your business - december 18, 2015 how to overcome byod security challenges - december 9, 2015 how you can use bigdata in your small business - december 8, 2015 related posts what’s in store for bigdata analytics in 2016 january 14, 2016 by stevesarsfield nosql and the internet of things january 14, 2016 by kingmesal how iot applications can help businesses grow internally january 7, 2016 by xanderscho 5 predictions for trends in data, analytics and machine learning in 2016 january 4, 2016 by bruce robbins how to recycle old it equipment and keep your data safe january 4, 2016 by ryank » already a member? login now to comment! » not a member? register to comment! like it? 188 november 23, 2015 morin tremblay says: hi ryan you have successfully specified here that how can bigdata and social media can corelate with each other and help to grow simultaneously. share this comment: twitter facebook linkedin * login or register to post comments please note: your first comment as a registered user will be held for moderation up to 24 hours (usually less). for more information about comments on our site, please read our faq and terms of use. image the moderated business community for business intelligence, predictive analytics, and data professionals. image * twitter * iframe: http://platform.twitter.com/widgets/follow_button.html?screen_name= smartdataco * rss * rss as xml addtomyyahoo4 subscribe with bloglines add to google * email * get our best posts in your inbox weekly first name ____________________ last name ____________________ email address ____________________ sign up! how do you innovate effectively and maintain a competive edge? image learn how in our exlcusive ebook, "bad data need not apply: designing the modern data warehouse environment." download here. sdc special columns big-data-guru riksy business big-data-mops events no events listed add your own! recommended to follow kevin finnerty follow paul dennies follow phim cach nhiet follow nguyen tien follow lars bls follow jessica egli-cannon follow comments * featured * popular * recent “the final aim of mcc is to allow implementation of rich smartphone web apps on an excess of cell phone devices, with a gorgeous user experience.” january 7th, 2016 by jamesmartin057 “by including more factors to score a person’s credit worthiness would make it even tougher for borrowers to obtain a loan approval. this in turn causes the finance industry to face a fluctuating business trend. there are definitely pros and cons to this major change in a loan application process but one thing for sure is to ensure borrowers shall only be allowed to take up a loan which is ...” january 7th, 2016 by webbrowan “thanks for the insight connie!” january 4th, 2016 by daniel matthews “there are clearly a number of changes coming for small businesses as they await ...” january 6th, 2016 by nguyennga “very useful tips, especially from the point of view of customer service.” january 4th, 2016 by isabella1977 “this is such a scary article. no doubt there is a lot of things that you have ...” january 6th, 2016 by webbrowan “the final aim of mcc is to allow implementation of rich smartphone web apps on ...” january 7th, 2016 by jamesmartin057 “regardless of the nature of any business, finance is the key factor that helps ...” january 7th, 2016 by webbrowan “such comparison tools are great for borrowers to perform an estimated ...” january 7th, 2016 by webbrowan “when it concerns money, you cannot afford to commit even a single mistake and ...” january 6th, 2016 by webbrowan “due to modernization we have found good technology advanced in the society; ...” january 7th, 2016 by edwardmiller “social collaboration seems like a funny way to describe the process of speaking ...” january 6th, 2016 by webbrowan show more “the final aim of mcc is to allow implementation of rich smartphone web apps on an excess of cell phone devices, with a gorgeous user experience.” january 7th, 2016 by jamesmartin057 “by including more factors to score a person’s credit worthiness would make it ...” january 7th, 2016 by webbrowan “due to modernization we have found good technology advanced in the society; ...” january 7th, 2016 by edwardmiller “regardless of the nature of any business, finance is the key factor that helps ...” january 7th, 2016 by webbrowan “such comparison tools are great for borrowers to perform an estimated ...” january 7th, 2016 by webbrowan “there are clearly a number of changes coming for small businesses as they await ...” january 6th, 2016 by nguyennga “when it concerns money, you cannot afford to commit even a single mistake and ...” january 6th, 2016 by webbrowan “social collaboration seems like a funny way to describe the process of speaking ...” january 6th, 2016 by webbrowan “this is such a scary article. no doubt there is a lot of things that you have ...” january 6th, 2016 by webbrowan “thanks for the insight connie!” january 4th, 2016 by daniel matthews show more popular * read * commented * authors h z - points john staunton - points jan rubio - points sarma ranga - points dawn bronkema - points in memoriam: robin fray carey (1) 1/6/2016 by sdc staff 72% of people aren’t familiar with hosted voip (0) 1/4/2016 by josh rose how to recycle old it equipment and keep your data safe (0) 1/4/2016 by ryan kidman 5 predictions for trends in data, analytics and machine learning in 2016 (0) 1/4/2016 by bruce robbins will 2016 be the year you clean up your dirty data? (0) 1/5/2016 by martin doyle 3 ingenious use cases for business intelligence tools 1/11/2016 by eran levy 3 perks of an office 365 migration that you didn’t know about 1/12/2016 by kyle cebull nosql databases: 4 game-changing use cases 1/13/2016 by jim scott flash vulnerabilities & exploits: an information security primer 1/12/2016 by thu pham nosql and the internet of things 1/14/2016 by jim scott advisory panel about the panel aaron aders more » bruno aziza more » stephen baker more » paul barsch more » gary cokins more » jill dyché more » themos kalafatis more » bernard marr more » james taylor more » smartdata collective * audio archive: the cutting edge of bigdata monetization * audio archive: delivering value through business analytics * audio archive: trendspotting for growth * audio archive: finding value in unstructured social data * audio archive: risks and rewards of bigdata in the cloud * audio archive: putting customer value to work categories » analytics » best practices » bigdata » book review » business intelligence » business rules » cloud computing » collaborative data » commentary » crm » culture/leadership » data management » data mining » data quality » data visualization » data warehousing » decision management » ebook » hadoop » hardware » inside companies » it » jobs » knowledge management » location » mapreduce » market research » marketing » marketing automation » mobility » modeling » new products » news » open source » podcasts » policy and governance » predictive analytics » privacy » r programming language » risk management » security » sentiment analytics » smartdata collective exclusive » social data » social media analytics » software » sql » statistics » text analytics » transparency » unstructured data » web analytics » webcasts » workforce analytics » workforce data show more featured contributors * you * melanie aizer * paul barsch * meta s. brown * jason burke * gary cokins * ted cuzzillo * barry devlin * chris dixon * erica driver * michael fauscette * bill franks * david hecht * julie hunt * lachlan james * doug lautzenheiser * alex olesker * david smith * james taylor * daniel tunkelang * bigdata smart data cloud social data unstructured data data in the cloud business intelligence analytics social media today llc © 2012 terms of use | community guidelines | privacy policy | get a badge | help | about & contact quantcast we use cookies to help us deliver our online services. by using our website or by closing this message box, you agree to our use of cookies as described in our cookie policyeaawxexvqwfcbfub. do not show again dow jones, a news corp company news corp is a network of leading companies in the worlds of diversified media, news, education, and information services. dow jones * barron's * bigcharts * djx * dow jones newswires * factiva * financial news * mansion global * marketwatch * newsmart * private markets * risk & compliance * wsj.com * wsj pro * wsj conference * wsj video news corp * big decisions * business spectator * checkout51 * harper collins * new york post * proptiger * rea * realtor.com * storyful * the australian * the sun * the times stoxx 600 ▲ 335.33 -1.20% nikkei ▲ 17147.11 -0.54% u.s. 10 yr ▲ 9/32 yield 2.058% crude oil ▲ 29.67 -4.90% euro ▲ 1.0904 0.35% djia ▲ 16379.05 1.41% subscribe now sign in the wall street journal sections my journal * home * world * u.s. * politics * economy * business * tech * markets * opinion * arts * life * real estate * today's paper show all sections hide all sections + world home o africa o asia o canada o china o europe o latin america o middle east o brussels real time o china real time o india real time o japan real time o economy o world video + u.s. home o economy o law o new york o politics o real time economics o the numbers o washington wire o journal report o u.s. video o what's news podcast + politics home o think tank o washington wire o politics video o wsj/nbc news poll + economy home o real time economics o economic forecasting survey o economy video + business home o aerospace & defense o autos & transportation o commercial real estate o consumer products o energy o entrepreneurship o financial services o food & services o health care o hospitality o law o management o manufacturing o media & marketing o natural resources o retail o tech/wsj.d o cfo journal o cio journal o cmo today o logistics report o risk & compliance o bankruptcy beat o heard on the street o business video o journal report o business podcast + tech home o cio journal o e-commerce o mobile o personal tech o privacy & security o startups o christopher mims o digits o geoffrey fowler o joanna stern o personal tech news o venture capital dispatch o billion dollar startup club o tech trader daily o tech video o the accelerators o tech podcast + markets home o bonds o commercial real estate o commodities o currencies o deals o financial services o funds o stocks o your money o frontiers o heard on the street o moneybeat o private equity beat o wealth adviser o ahead of the tape o cfo journal o journal report o market data o markets video o markets podcast o moneybeat podcast o heard on the street podcast + opinion home o arts in review o book reviews o commentary o editorials o letters to the editor o best of the web o columnists o morning editorial report o peggy noonan o political diary o opinion video o potomac watch podcast o foreign edition podcast + arts home o arts in review o books o speakeasy o arts video o wsj. magazine o speakeasy podcast + life home o careers o cars o food & drink o health o ideas o real estate o sports o style & fashion o travel o expat o the daily fix o life video o wsj. magazine o wsj puzzles o off duty podcast o the future of everything + real estate home o commercial real estate o house of the day o mansion o real estate video hide all sections aim higher, reach further. get the wall street journal $12 for 12 weeks. subscribe now sign in reveal navigation options * home * world + regions + africa + asia + canada + china + europe + latin america + middle east + blogs + brussels real time + china real time + india real time + japan real time + sections + economy + more + world video * u.s. + sections + economy + law + new york + politics + columns & blogs + real time economics + the numbers + washington wire + more + journal report + u.s. video + what's news podcast * politics + blogs + think tank + washington wire + more + politics video + wsj/nbc news poll * economy + blogs + real time economics + more + economic forecasting survey + economy video * business + industries + aerospace & defense + autos & transportation + commercial real estate + consumer products + energy + entrepreneurship + financial services + food & services + health care + hospitality + law + management + manufacturing + media & marketing + natural resources + retail + tech/wsj.d + c-suite + cfo journal + cio journal + cmo today + logistics report + risk & compliance + columns & blogs + bankruptcy beat + heard on the street + more + business video + journal report + business podcast * tech + sections + cio journal + e-commerce + mobile + personal tech + privacy & security + startups + columns & blogs + christopher mims + digits + geoffrey fowler + joanna stern + personal tech news + venture capital dispatch + more + billion dollar startup club + tech trader daily + tech video + the accelerators + tech podcast * markets + sections + bonds + commercial real estate + commodities + currencies + deals + financial services + funds + stocks + your money + columns & blogs + frontiers + heard on the street + moneybeat + private equity beat + wealth adviser + ahead of the tape + more + cfo journal + journal report + market data + markets video + markets podcast + moneybeat podcast + heard on the street podcast * opinion + sections + arts in review + book reviews + commentary + editorials + letters to the editor + columns & blogs + best of the web + columnists + morning editorial report + peggy noonan + political diary + more + opinion video + potomac watch podcast + foreign edition podcast * arts + sections + arts in review + books + blogs + speakeasy + more + arts video + wsj. magazine + speakeasy podcast * life + sections + careers + cars + food & drink + health + ideas + real estate + sports + style & fashion + travel + blogs + expat + the daily fix + more + life video + wsj. magazine + wsj puzzles + off duty podcast + the future of everything * real estate + sections + commercial real estate + house of the day + mansion + more + real estate video search ____________________________ search → subscribe sign in obama’s terror sangfroid review & outlook obama’s terror sangfroid shrinking metlife review & outlook shrinking metlife guilty for obeying the feds review & outlook guilty for obeying the feds the high cost of a bad reputation declarations the high cost of a bad reputation the democratic crack-up potomac watch the democratic crack-up a cancer ‘moonshot’ needs bigdata a cancer ‘moonshot’ needs bigdata ending the filibuster would hand progressives a huge victory ending the filibuster would hand progressives a huge victory equities policy needs surgery, not band-aids equities policy needs surgery, not band-aids a comforting word in the hotel nightstand houses of worship a comforting word in the hotel nightstand obama’s legacy: trump and bernie wonder land obama’s legacy: trump and bernie gm speaks untruth to power business world gm speaks untruth to power the cologne portent global view the cologne portent ‘noises off’ review: nine doors to delight ‘noises off’ review: nine doors to delight the lives of the disappeared bookshelf the lives of the disappeared * 480 save article sign in to save subscribe to wsj text size regular medium large google+ print * opinion * columnists * business world bigdata and mass shootings this month hitachi launched a test of its crime-prediction software in several american cities. by holman w. jenkins, jr. oct. 9, 2015 6:21 p.m. et as always, the dots are connected after the fact, when the connecting is easy. he was booted from the military after a single month for attempting suicide. his mother advertised in numerous social-media posts his mental problems and fascination with guns. he posted his own threat on a chat site used by insecure young men, warning that students in the “northwest” should avoid school the next day. with the usual upturned hands, experts in the aftermath tell us that “tens of thousands of people” exhibit the social isolation... to read the full story, subscribe or sign in popular on wsj most popular videos * [http://m.wsj.net/video/20160114/011416trumpchina/011416trumpchina_167x 94.jpg] trump calls trade with china unfair [http://m.wsj.net/video/20160112/011215snapchat4/011215snapchat4_167x 94.jpg] snapchat explained... using snapchat [http://m.wsj.net/video/20160114/011416seibfix/011416seibfix_167x94.jpg ] three key themes in the gop debate [http://m.wsj.net/video/20160114/011416debateguns/011416debateguns_167x 94.jpg] gop candidates discuss gun control [http://m.wsj.net/video/20160115/011416debatehighlight/011416debatehigh light_167x94.jpg] republican debate in two minutes most popular articles * china’s haier near deal to buy ge appliance business china’s haier near deal to buy ge appliance business checking in? hackers may be, too checking in? hackers may be, too $15 oil? it’s already here, in canada $15 oil? it’s already here, in canada oil prices rise oil prices rise remembering alan rickman (1946-2016): irony and self-irony opinion: remembering alan rickman (1946-2016): irony and self-irony editors’ picks * next up for greece: how to shrink the debt brussels beat next up for greece: how to shrink the debt * switzerland: a test case for currency shock foreign exchange switzerland: a test case for currency shock * $15 oil? it’s already here, in canada markets $15 oil? it’s already here, in canada * russian intervention emboldens syrian kurds middle east crossroads russian intervention emboldens syrian kurds * remembering alan rickman (1946-2016): irony and self-irony film remembering alan rickman (1946-2016): irony and self-irony * that fib junior told is a sign of healthy growth mind and matter that fib junior told is a sign of healthy growth * 5 big takeaways from the oscar nominations speakeasy 5 big takeaways from the oscar nominations * powerball losers celebrate for a community win play u.s. news powerball losers celebrate for a community win wall street journal europe edition + u.s. + asia + europe + india + américa latina + brasil + 中国 (china) + 日本 (japan) edition: [asia___________] text size: [small_] subscribe nowsign in wsj membership benefits download wsj apps customer center legal policies subscribe sign in top« wsj membership wsj+ membership benefits digital subscription print subscription print and digital subscription why subscribe? download wsj apps for ios and android corporate subscriptions professor journal student journal customer service customer center live help redesign guided tour tools & features emails & alerts guides my news portfolio rss feeds topics video center watchlist podcasts ads advertise advertise locally commercial real estate ads place a classified ad sell your business sell your home recruitment & career ads more content partnerships corrections jobs at wsj news archive register for free reprints + facebook + twitter + google+ + youtube + podcasts + googleplay + appstore dow jones products barron's bigcharts djx dow jones newswires factiva financial news mansion global marketwatch private markets realtor.com risk & compliance wsj conference wsj pro central banking wsj video wsj wine privacy policy cookie policy copyright policy data policy subscriber agreement & terms of use your ad choices copyright ©2016 dow jones & company, inc. all rights reserved. #the cio report rss feed * wsj + wsj + facebook + twitter * live + live * realtor + realtor * mansion global + mansion global * barron's + barron's * wsj+ + wsj+ * product x + product x + djx + rt + f + r&c + pe&vc + wsj + b * more + portfolio + bigcharts + financial news + professor journal + student journal + virtual stock exchange + wsj classifieds + wsj classroom + wsj wine news, quotes, companies, vid search ___________________________________________________________ ___________________________________________________________

______________________________________________________

________________________

___________________________________________________

_____________________

_

___________________________
• ________ headline_____________________________________________
_________________________________________________
• _________________ __ goldman___________ _________________________________________________
___________________________________________

__________________________________________________

__________________________________________________

_

__________
• ________ companies____________________________________________
_________________________________________________
• _________________ __
  __________________ gold_______________
  ______________________________________________
  ____________________ ran________________________________________________ gold resources ltd.
  ______________________________________________
  _________________ u.s._______________________________________________
  ______________________________________________ _________________________________________________
___________________________________________

_________________________________________

__________________________________________________

___________________________________________________

____________________________

_________
• __________________________ view all search results »<_
__________________________________________________

___________________________________________________

___________________________________________________

_____

____________________________________________________ the wall street journal menu * homepage * world * u.s. * politics * economy * business * tech * markets * opinion * arts * life * real estate * sign in * profile * saved articles * portfolio * watchlist * newsletters & alerts * customer center * sign out * message center ( new) cio journal exclusive reporting and analysis for corporate-technology executives. * cio report * consumerization * bigdata * cloud * talent & management * security 11:33 am et aug 27, 2015 bigdata making the case for the ‘long-tail’ of bigdata log in * previousthe morning download: facebook releases human-powered answer to siri * nextenterprise software spending to grow: gartner content from our sponsor deloitte * software-defined everything becoming a reality software-defined everything can elevate infrastructure investments from costly plumbing to competitive differentiators. the software-defined data center of the future might not only lower costs, but also enable companies to create more scalable, responsive, and innovative it organizations. * read more about: tech trends, transformation * read more from deloitte » * please note: the wall street journal news department was not involved in the creation of the content above. search the cio report1 search for cio report articles go the cio report categories * morning download 944 * guest voices 574 * security 538 * bigdata 505 * cloud 487 * innovation 448 * mobility 405 * talent & management 402 * implementation 286 * executive moves 271 * vendor management 254 * ceo reads 194 * consumerization 119 * risk management 118 * application development 111 * compliance 105 * social 99 * collaboration 98 * healthcare it 96 * automation 87 wsj web slice content links to actual page containing web slice functionality. 15 * wall street journal * facebook * twitter * linkedin * google+ * youtube * podcasts * googleplay * appstore * + subscribe + /sign in * back to top« * subscribe + why subscribe? + wsj+ + corporate subscriptions + apps + professor journal + student journal * customer service + customer center + live help + redesign guided tour * tools & features + emails & alerts + guides + my news + portfolio + rss feeds + topics + video center + watchlist * ads + advertise + advertise locally + commercial real estate ads + place a classified ad + sell your business + sell your home + recruitment & career ads * more + conferences + content partnerships + corrections + jobs at wsj + make time + news archive + register for free + reprints * privacy policy * cookie policy * copyright policy * data policy * subscriber agreement & terms of use * your ad choices * copyright ©2016 dow jones & company, inc. all rights reserved. close email this recipient's email address (separate multiple address with commas) ____________________ your email address ____________________ message (optional) ____________________________________________________________ ____________________________________________________________ ____________________________________________________________ ____________________________________________________________ [x] send me a copy send or cancel close thank you your email has been sent. close error. an error has occured and your email has not been sent. please try again. • invalid email address. • you can't enter more than 20 emails. • seperate multiple addresses with commas. • must enter an email address. • you must enter the verification code below to send. • invalid entry: please type the verification code again. #the cio report rss feed * wsj + wsj + facebook + twitter * live + live * realtor + realtor * mansion global + mansion global * barron's + barron's * wsj+ + wsj+ * product x + product x + djx + rt + f + r&c + pe&vc + wsj + b * more + portfolio + bigcharts + financial news + professor journal + student journal + virtual stock exchange + wsj classifieds + wsj classroom + wsj wine news, quotes, companies, vid search ___________________________________________________________ ___________________________________________________________

______________________________________________________

________________________

___________________________________________________

_____________________

_

___________________________
• ________ headline_____________________________________________
_________________________________________________
• _________________ __ goldman___________ _________________________________________________
___________________________________________

__________________________________________________

__________________________________________________

_

__________
• ________ companies____________________________________________
_________________________________________________
• _________________ __
  __________________ gold_______________
  ______________________________________________
  ____________________ ran________________________________________________ gold resources ltd.
  ______________________________________________
  _________________ u.s._______________________________________________
  ______________________________________________ _________________________________________________
___________________________________________

_________________________________________

__________________________________________________

___________________________________________________

____________________________

_________
• __________________________ view all search results »<_
__________________________________________________

___________________________________________________

___________________________________________________

_____

____________________________________________________ the wall street journal menu * homepage * world * u.s. * politics * economy * business * tech * markets * opinion * arts * life * real estate * sign in * profile * saved articles * portfolio * watchlist * newsletters & alerts * customer center * sign out * message center ( new) cio journal exclusive reporting and analysis for corporate-technology executives. * cio report * consumerization * bigdata * cloud * talent & management * security 12:42 pm et jun 15, 2015 bigdata bigdata fuels a ‘data first’ movement log in * previousthe morning download: ibm embraces spark at bigdata’s real-time frontier * nextmicrosoft cio tackles licensing, a process that drives customers nuts content from our sponsor deloitte * software-defined everything becoming a reality software-defined everything can elevate infrastructure investments from costly plumbing to competitive differentiators. the software-defined data center of the future might not only lower costs, but also enable companies to create more scalable, responsive, and innovative it organizations. * read more about: tech trends, transformation * read more from deloitte » * please note: the wall street journal news department was not involved in the creation of the content above. search the cio report1 search for cio report articles go the cio report categories * morning download 944 * guest voices 574 * security 538 * bigdata 505 * cloud 487 * innovation 448 * mobility 405 * talent & management 402 * implementation 286 * executive moves 271 * vendor management 254 * ceo reads 194 * consumerization 119 * risk management 118 * application development 111 * compliance 105 * social 99 * collaboration 98 * healthcare it 96 * automation 87 wsj web slice content links to actual page containing web slice functionality. 15 * wall street journal * facebook * twitter * linkedin * google+ * youtube * podcasts * googleplay * appstore * + subscribe + /sign in * back to top« * subscribe + why subscribe? + wsj+ + corporate subscriptions + apps + professor journal + student journal * customer service + customer center + live help + redesign guided tour * tools & features + emails & alerts + guides + my news + portfolio + rss feeds + topics + video center + watchlist * ads + advertise + advertise locally + commercial real estate ads + place a classified ad + sell your business + sell your home + recruitment & career ads * more + conferences + content partnerships + corrections + jobs at wsj + make time + news archive + register for free + reprints * privacy policy * cookie policy * copyright policy * data policy * subscriber agreement & terms of use * your ad choices * copyright ©2016 dow jones & company, inc. all rights reserved. close email this recipient's email address (separate multiple address with commas) ____________________ your email address ____________________ message (optional) ____________________________________________________________ ____________________________________________________________ ____________________________________________________________ ____________________________________________________________ [x] send me a copy send or cancel close thank you your email has been sent. close error. an error has occured and your email has not been sent. please try again. • invalid email address. • you can't enter more than 20 emails. • seperate multiple addresses with commas. • must enter an email address. • you must enter the verification code below to send. • invalid entry: please type the verification code again.