Subscribe Bookmark


Jun 23, 2011

Exploring and visualizing sleep data with JMP

My BodyMedia® FIT® armband uses a variety of sensors to monitor my movements, including an accelerometer and gyroscope. When plugged into a computer via USB or connected via Bluetooth to an iPhone, the data stored on my device is uploaded to BodyMedia® servers and an algorithm runs to determine when I was active, lying down, awake or sleeping for each minute of monitoring time. I get an estimate of how long I slept, and I can view a minute-by-minute visualization of the time I spent lying down, awake and asleep. (You can get more details about BodyMedia’s sleep classification algorithm in a white paper.) I can extract the data for these daily sleep summaries from the activity summary files, as I described in an earlier blog post.

I wanted to analyze and create some effective data visualizations of my sleep data. An early version of my JMP Discovery Summit e-poster included a multivariate scatterplot of sleep duration, sleep efficiency and time spent lying down during~1,300 nights. I colored points red indicating nights with sleep durations <4 hours. Since I rarely sleep so few hours, I suspected some or all of those low outliers to be infrequently occurring data collection errors. On the other hand, I knew that the highest sleep day was an accurate measurement from a recent bout of the stomach flu!

Scatterplot Matrix

By the time I met with Xan Gregg, lead developer of Graph Builder and head of the JMP Data Visualization group, I had switched to using box plots to visualize my sleep data. The box plot view told a much more interesting story since it showed data over time and gave me a better sense of the variability of my sleep measurements within months. Also, the addition of a time dimension clearly showed that the big drop in my sleep efficiency at the end of September 2011 coincided perfectly with my son’s birth!blue and orange sleep box plot2

When I showed this graph to Xan, he recommended:

  • Lightening the colors used in the box plots.
  • Darkening the smoother lines.
  • Adding annotations to point to important trends.
  • Changing my X axis labels to a horizontal orientation.
  • We both thought that the final version of my sleep graph was more visually appealing and easier to interpret.

    Teal and purple with anno final

    I was surprised to see how much my sleep patterns varied with the time of year, as you can see from the box plots and the up-and-down pattern of the smoother line in the sleep duration graph on the bottom. In contrast to my activity measurements, which I know are strongly affected by the fact that I wear my armband less during the day in summer, I wear my armband to bed almost without fail. Seasonal variation in sleep patterns is actually a commonly recognized phenomenon in human sleep, yet I had completely missed it in my own data up till now! This pattern was not obvious when I viewed the standard summary reports of my sleep information within days, weeks and months.

    It's easy to see how useful long-term data on nightly sleep patterns could be if you are chronically tired. Although it may not be as accurate as a sleep study done by a medical professional, accumulating nightly data over a long period of time with a sleep-monitoring device could help you assess how much sleep you need to awake feeling rested. If you have insomnia, you can use your sleep data to see when and how long you are awake in the night. If you battle sleepiness during the day, you can change your behavior in various ways and assess the outcome on your sleep.

    Experts recommend various sleep improvement strategies such as cutting out or limiting caffeine, going to bed earlier, getting up when you wake up rather than hitting snooze repeatedly, adding exercise to your day and avoiding the use of screens close to bedtime. Sleep monitoring gives you actual outcome data to assess which of these strategies may work to optimize your own sleep patterns. Some sleep monitors even can wake you at an optimum time given where you are in your sleep cycle. Since starting to monitor my sleep, I have rarely used a morning alarm. When I wake up, I assess whether I am still tired, and if not, I check how many hours I slept. If I believe I have had enough uninterrupted sleep based on these two assessments, I get up and start my day.

    By examining my food log and sleep data, I have discovered that in addition to seasonal variations, a number of other factors appear to affect how well I sleep. I sleep much less when I am experiencing stress at work, and a few days without exercise can hurt my sleep. What and when I eat can also impact my sleep quality.  I have observed that the stimulant kick provided by chocolate Greek yogurt (my favorite breakfast) disrupts my sleep if I eat it as an evening snack. Perhaps because of my background in biochemistry and genomics, I wasn't satisfied by simply observing this connection. I had habituated to drinking caffeine late at night in the past without such a negative impact on my sleep, so I wondered what was different here?  I did some reading about cocoa powder, and it turns out that it contains caffeine and a related stimulant called theobromine. The same cellular pathway deactivates both these chemicals, and I know from other test results that I have the slow form of a major enzyme involved. I suspect that when I eat chocolate Greek yogurt close to bedtime, I can't metabolize enough of the stimulant chemicals it contains before I go to sleep, and as a result, I sleep poorly throughout the night.

    To reproduce my sleep graph with your own data in Graph Builder, open a data table containing sleep duration and/or sleep efficiency measures in hr:m format by date. If your data table doesn't have Year and Month variables, you can create these from the Date variable in your table by right-clicking on it in the Graph Builder variable chooser and adding new transform columns from the Date Time menu. I used a Value Ordering property on the Month variable to create a Month Name column and made sure that Year and Month Name were specified as Ordinal.

    To create the graph, open Graph Builder and drag:

    • Year to the X axis.
    • Month Name to the X axis (just above Year, so the axis shows Month Name nested within Year).
    • Sleep Efficiency to the Y axis.
    • Sleep Duration to the Y axis just below Sleep Efficiency so they appear in separate sections.
    • To complete the graph, change the element type to Box Plot using the icon at the top of the window, adjust your Y-axis, graph title and axis titles if desired, and add one or more annotations from the Tools menu. You can right-click on annotations to change their appearance to match your graph like I did. Stay tuned for the next few posts where Xan and I show how he helped improve other of visualizations of my data!

      You can learn more about my interests in quantified self data analysis in this blog post here, see an e-poster on my activity and food log data import project in the JMP Discovery Summit 2014 User Community here, and read about how I imported my BodyMedia® Activity Summary files from Excel here and Food Log files from text here. I used JMP to recode food item names and classify foods into categories, and then used my data to characterize my activity and meal logging patterns. You can download a JMP add-in from the JMP File Exchange here to import your own BodyMedia® activity summary files and food log files or CSV formatted food log files from the free MyFitnessPal website.