Subscribe Bookmark RSS Feed
amy_bullock

Staff

Joined:

Aug 5, 2014

Speeding up exploration with John Sall

Well, Wall Street cares so much about making fast decisions that it is laying dedicated high-speed lines so that the data for program trades can be processed faster and orders executed more quickly. If Wall Street can do all this in milliseconds, won’t it make a difference to you to get your indicators sooner, in seconds instead of hours, in time for a fast response to make a difference?

Speed was the theme of the day today, where I joined hundreds of others at Predictive Analytics World to talk about how to get things done more quickly -- and what a more perfect place to host this topic of discussion than in the heart of fast-paced midtown Manhattan?  John Sall, SAS co-founder and chief architect of JMP delivered a 20-minute keynote address on speed -- specifically how it’s possible to explore your data, even big data, more quickly.

The question people commonly ask themselves: Is there too much data to use interactive graphics? The more things you are able to view, the more discoveries you can make and realize value faster. Rich graphics with more features and more dimensions greatly enable this process, and animation helps too. Fast visualization is key. Here’s what I learned from John’s visual data exploration (in a matter of 18 minutes to be exact).

In viewing birth records from the CDC over the last 50 years using Graph Builder capabilities in JMP:

  • Women 30 and older have heavier babies.
  • Male babies tend to weigh about 100 grams more than female babies.
  • Not many babies are born on weekends. Why? Scheduled births surround doctors’ busy schedules.
  • Some babies have weighed as much as 22 tons. Oh wait, no, that was just missing values in the data table.
  • Time it took to find out these few facts from 12.8 million births: 3 minutes.

    In viewing ASA data to better understand challenges with airline traffic, he grouped the data by month and years, using the heat map graphing capabilities to discover that:

    • There we lots of delays across the US in December of 1999 and 2000.
    • No tracking mechanism was in place in 1987 to track delays.
    • It’s easy to create a scatterplot of delay by distance and use binning to see where the data is heaviest.
    • Time it took to make these observations by sifting through 123 million cases documented: less than 2 minutes.

       In viewing social survey data and labor force statistics using 5,416 variables, John discovered:

      • What makes people happy (this may be a separate blog post altogether, but I can say that health condition and financial stability were among the top-ranked variables).
      • If you have 12 people in your household, you are unlikely to be happy. Yes, this is truly what the data showed and did garner many laughs from the PAW audience.
      • Total time to discover what makes people happy: 2.5 minutes.

        In closing, John indicated that in addition to all the speed JMP enables on the desktop, SAS has made great strides in high-performance analytics. Customers have been able to tackle problems that previously were intractable. Some improvements:

        • Reducing analytics processing time from 11 hours to 10 seconds, in other cases 30 hours to 2 hours.
        • Driving a 15 percent improvement in customer retention rates.
        • Modeling 195 million customer records in real time.
        • It made us happy that John joined us today to share these insights. I turned to tell him so, but he was already on his way out the door to catch the high-speed train at Penn Station to his next engagement for the day.