#Bits » Big Data Shrinks to Grow Comments Feed Daily Report: Judge Upholds Searches of Travelers’ Devices at Border Why Hire a P.R. Firm When There’s TaskRabbit? RSS 2.0 alternate (BUTTON) Sections (BUTTON) Home (BUTTON) Search Skip to content The New York Times Bits | Big Data Shrinks to Grow Advertisement (BUTTON) Search (BUTTON) Subscribe Now (BUTTON) Log In (BUTTON) (BUTTON) 0 (BUTTON) Settings (BUTTON) Close search search sponsored by Site Search Navigation Search NYTimes.com ____________________ (BUTTON) Clear this text input (BUTTON) Go 1. Loading... See next articles See previous articles Site Navigation Site Mobile Navigation Supported by Bits - Business, Innovation, Technology, Society ____________________ (BUTTON) Search Big Data Shrinks to Grow By Quentin Hardy January 1, 2014 10:00 am January 1, 2014 10:00 am (BUTTON) Photo Anthony Goldbloom, Kaggle's founder and chief executive. Anthony Goldbloom, Kaggle's founder and chief executive.Credit What’s next for Big Data? Maybe a little thinning down of the industry and a little more focus. In fact, it may be underway. Google Trends shows searches of the term “Big Data” peaked in October, ending a nearly ceaseless climb that began three years earlier. Google doesn’t offer absolute numbers, but in that time there was a hundredfold increase in interest in the term. That seems a reasonable match to the number of start-ups, public relations pitches and gurus using the term over this period. All high-tech hype cycles must end, and usually when they do there’s a crash. Some pundits gloat at bad ideas that got funded and fell apart. The true believers set to building something durable from the remnants of the boom. In some cases, like the ’90s Internet hype, the outcome more than fulfills the promise, but few see it coming. In the case of Big Data, this probably means less focus on back-end technologies like new types of storage or database frameworks, and a rethinking about how best to integrate human knowledge, algorithms and diverse sets of data. In December, the company Kaggle changed its Big Data business model in important ways. Kaggle had been a darling of Big Data followers for its contests that sought the world’s best statistical analysts. Now Kaggle has decided to focus on a few specific industries, starting with oil and gas. “We liked to say ‘It’s all about the data,’ but the reality is that you have to understand enough about the domain in order to make a business,” said Anthony Goldbloom, Kaggle’s founder and chief executive. “What a pharmaceutical company thinks a prediction about a chemical’s toxicity is worth is very different from what Clorox thinks shelf space is worth. There is a lot to learn in each area.” Oil and gas, which for Kaggle means mostly fracking wells in the United States, have well-defined data sets and a clear need to find working wells. While the data used in traditional oil drilling is understood, fracking is a somewhat different process. Variables like how long deep rocks have been cooked in the earth may matter. So does which teams are working the fields, meaning early-stage proprietary knowledge is also in play. That makes it a good field to go into and standardize. Kaggle may also work in the drug industry again, as well as insurance, Mr. Goldbloom said. “We will still do competitions, they are a great engine for finding talent,” he said, “but our returns as a business will be higher if we focus.” There is reason to think so. Kaggle’s sharper focus follows the odyssey of the Climate Corporation, which in its original incarnation as WeatherBill tried to sell predictive data about weather to farmers, house painters and golf courses. After it focused on agriculture, the Climate Corporation was bought in October by Monsanto for $930 million. Another company that has looked at Big Data problems in an industry-specific way is Palantir. In December, Palantir, which initially focused on government security work and has moved into areas like finance, disaster relief and pharmaceuticals, raised $100 million on a valuation of $9 billion. In September, Palantir had raised money at a valuation of $6 billion. Not everyone believes Big Data has to narrow its focus. “Predictive modeling is still going to change the world in every area,” said Jeremy Howard, a close collaborator with Mr. Goldbloom who left Kaggle over the change in direction. He said he was focusing on building new kinds of software that could better learn about the data it was crunching and offer its human owners insights on any subject. “A lone wolf data scientist can still apply his knowledge to any industry,” he said. “I’m spending time in areas where I have no industrial knowledge and finding things. I’m going to have to build a company, but first I have to spend time as a lone wolf.” A version of this article appears in print on 01/06/2014, on page B5 of the NewYork edition with the headline: Big Data Shrinks to Grow. (BUTTON) Agriculture and Farming, Computers and the Internet, Data-Mining and Database Marketing, Kaggle Inc, Oil (Petroleum) and Gasoline What's Next Loading... * Previous Post Daily Report: Judge Upholds Searches of Travelers’ Devices at Border * Next Post Why Hire a P.R. Firm When There’s TaskRabbit? Visit the Technology section for complete coverage of the industry. » Most Viewed 1. 1. When a Unicorn Start-Up Stumbles, Its Employees Get Hurt 2. 2. Uber’s No-Holds-Barred Expansion Strategy Fizzles in Germany 3. 3. At C.D.C., a Debate Behind Recommendations on Cellphone Risk 4. 4. Yahoo’s Brain Drain Shows a Loss of Faith Inside the Company 5. 5. Fans Demand Details After Death of a 13-Year-Old YouTube Star Latest From Bits * Jawbone Raises $165 Million at Half Its Last Valuation * Daily Report: Why Intel’s Earnings Shouldn’t Be Ignored * Amazon Subsidiary Gains Approval to Ship Ocean Freight * Daily Report: Tech Earnings Season Begins With a Warning * SurveyMonkey Names New Chief, Its Second in Six Months Follow us on @nytimesbits on twitter * Twitter Follow Advertisement Go to Home Page » Site Index The New York Times Site Index Navigation News * World * U.S. * Politics * N.Y. * Business * Tech * Science * Health * Sports * Education * Obituaries * Today's Paper * Corrections Opinion * Today's Opinion * Op-Ed Columnists * Editorials * Contributing Writers * Op-Ed Contributors * Opinionator * Letters * Sunday Review * Taking Note * Room for Debate * Public Editor * Video: Opinion Arts * Today's Arts * Art & Design * ArtsBeat * Books * Dance * Movies * Music * N.Y.C. Events Guide * Television * Theater * Video Games * Video: Arts Living * Automobiles * Crossword * Food * Education * Fashion & Style * Health * Jobs * Magazine * N.Y.C. Events Guide * Real Estate * T Magazine * Travel * Weddings & Celebrations Listings & More * Classifieds * Tools & Services * Times Topics * Public Editor * N.Y.C. Events Guide * Blogs * Multimedia * Photography * Video * NYT Store * Times Journeys * Subscribe * Manage My Account Subscribe * Subscribe * Times Insider * Home Delivery * Digital Subscriptions * NYT Opinion * Crossword * Email Newsletters * Alerts * Gift Subscriptions * Corporate Subscriptions * Education Rate * Mobile Applications * Replica Edition * International New York Times * © 2016 The New York Times Company * Contact Us * Work With Us * Advertise * Your Ad Choices * Privacy * Terms of Service * Terms of Sale * Site Map * Help * Site Feedback * Subscriptions