With custom maps, JMP allows you to visualize data in location context. In this talk, director of R&D Shannon Conners discusses how she used JMP to understand and visualize personal fitness data. Collected over multiple years, and on various personal devices, the data had all the issues you’d expect with manually entered logs (e.g. redundancy of exercise names, missing workout details and others). Shannon shows you how to clean up your dirty data with JMP Recode scripts, then places the training metrics in context – with a custom muscle-area shape files, incorporating body shape maps as graphical selection filters in dashboards, and using selection filters to click on various body areas to drill down into detailed graphs.   ... View more

Level: Beginner Job Function: Analyst / Scientist / Engineer Shannon Conners, Director of JMP Development Operations, SAS Customer bug and crash reports provide precious insights into customers’ experience of JMP product quality. Even as the JMP team develops features and new products, we also evaluate problem reports from a range of production releases. JMP developed email-based systems for Mac crash reports in JMP 12 and Windows crash reports in JMP 13. Technical Support and Development collaborate to identify, investigate and fix crashes received by this system. New reports sent to JMP Technical Support are processed hourly and details loaded into a SQL Server database. Crashes are then screened, identified as novel or potentially related to a known issue, and details are sent back to the track. I created a crash investigation project that refreshes hourly via JSL and organizes my queries, scripts, graphs and tables with embedded web links. I use this crash data project to identify additional occurrences of existing crashes, find unsolved crash groupings requiring further investigation by JMP developers and understand the prevalence of various problem reports from the field. We are planning improvements to this system in the future that we believe will streamline the process and encourage more customers to share their reports. ... View more

Level: Intermediate Shannon Conners, JMP Director of Research and Development, SAS The JMP team, like many of our customers, is simultaneously engaged in building new product versions and analyzing problem reports from production releases. Customer bug and crash reports provide precious insights into customers’ experience of JMP product quality. JMP began prompting customers to email Mac crash reports in JMP 12 and added built-in support for emailing Windows crash reports in JMP 13. Technical Support and Development collaborate to identify, investigate and fix unsolved customer crashes. The process starts when a report is emailed to JMP Technical Support and processed in an hourly batch. New crash signatures are loaded into a SQL Server database and screened to identify them as novel or potentially related to a known issue; then details are sent back to the track. I refresh my crash investigation environment frequently with a JSL script that calls JMP Query Builder and table manipulation tools like Split and Summarize, and embeds scripts and web links in tables to enable quick investigation and distribution. Imported and derived data tables open as tabs inside a JMP 14 Project. I use this crash data project to identify additional occurrences of existing crashes and find unsolved crash groupings requiring further investigation by JMP developers. ... View more

  The JMP team, like many of our customers, is simultaneously engaged in building new product versions and analyzing problem reports from production releases. Customer bug and crash reports provide precious insights into our customers’ experience of JMP product quality. JMP began prompting customers to email Mac crash reports in JMP 12 and added built-in support for  emailing Windows crash reports in JMP 13. We store internally and externally reported crash data in a SQL server database. Technical support and development collaborate to identify, investigate, and fix unsolved customer crashes. The process starts when a new crash report is emailed to JMP technical support, processed and automatically compared with previous reports to identify it as novel or related to a known issue. Known details about the crash are sent back to the track. The JMP crash database updates hourly,  so I automate the refresh of my crash investigation environment  using a JSL script. My script runs queries generated by JMP’s database Query Builder, calls  various table manipulation tools like Split and Summarize, and embeds key scripts in tables . All imported and derived data tables open as tabs inside a JMP 14 Project. I use my crash data project to investigate newly reported crashes and identify unsolved crash groups for further investigation by JMP development. ... View more

Shannon Conners, PhD, JMP Director of Research and Development, SAS   JMP 13 introduced the Text Explorer platform to find important words and phrases within large document collections and topics for further analysis. If you want to create structured data tables from unstructured text, you should not overlook Text Explorer’s Customize Regex and Save to Column options. I’ll show how I use these features to extract key details from text logs output by JMP build and test projects into new table variables. My approach can be generalized to retrieve and process text logs from other web-based or file-based systems. After creating structured data tables with Text Explorer, I use various JMP tools to finalize their format. Text to Columns parses the delimited text produced by the Save to Column feature, splitting a single column into multiple columns on white space or custom character delimiters. I use JMP Query Builder (introduced in JMP 13) to interactively subset, reformat and rename columns, and save a script that replicates the process. I’ll give a sneak peek at how my Jenkins data processing workflow has recently improved thanks to several new features in JMP 14, including JSL functions handling HTTP requests to the Jenkins web API and embedded clickable web links in JMP data tables. ... View more

I began tracking weight training workouts in notebooks in middle school. However, training notes did not give me much insight into my patterns over the years until I began to import my data into JMP. I started in early 2015 by entering data from 14 of the past 17 Januaries. The boredom of manual data entry quickly drove me to investigate new tracking tools! I first adopted the Full Fitness tracking app, then the Push Strength weight training sensor band and app. Both allow me to record my workout details on my phone during a workout and export my data directly to CSV for import into JMP. Whether collated from my notebooks, exported from Full Fitness, or collected with my Push band, my weight training data tables have presented challenges common to many high dimensional data sets. I have turned to JMP platforms I know and love to help clean up, categorize, and visualize aspects of my workout history. I used the JMP 12 Recode platform to combine replicate exercise names and categorize exercises by primary body part. I created a custom set of female muscle shape files and used graphs including these shapes as filters for other summary graphs, leveraging the new selection filter feature added in JMP 12. ... View more

I shared in my previous post that I scoured my baby book and tracking notebooks and requested various medical records to gather historical information about my weight fluctuations over the years. I used this data to construct JMP graphs with annotations and pinned hover labels containing pictures (thanks to the new expression columns feature in JMP 12). The process of creating and exploring these graphs gave me a lot of insight into how stress-related overeating patterns contributed to my past weight gains. I also rediscovered that self-tracking was an essential component to all of my successful weight loss and maintenance efforts since college. I tracked during all of my successful weight loss efforts (annotations in blue), but things often went in the other direction when I wasn't tracking (annotations in red). I collected weight data because it was easy to do at home. But excess weight is one of several major independent risk factors for heart disease. With the 13 th Wear Red Day approaching on Feb. 5, it’s a perfect time to recall that heart disease is the No. 1 killer of women in the US and revisit the American Heart Association website to learn more about weight and the other risk factors that contribute to heart disease. Wear Red Day helps raise awareness that the majority of heart disease deaths can be prevented by adopting lifestyle changes like losing weight, getting regular exercise, and improving eating habits. Discussing your lifestyle factors,  family history, and current blood pressure and cholesterol numbers with your doctor can be a helpful step in understanding your own personal risks. Experts generally agree that carrying excess weight stresses the cardiovascular system directly and has a negative impact on risk biomarkers like blood pressure and cholesterol. While regular exercise can improve good cholesterol and reduce heart disease risk, exercise alone might not be enough to combat the negative effects of obesity on general longevity. One recently published study followed 1.3 million Swedish men for an average of 30 years and found little longevity benefit for obese yet very fit individuals. On average, those young men died 30% earlier than their thin yet unfit counterparts when all causes of death were considered. This is less surprising when you consider that obesity has been associated with a diverse list of health problems extending far beyond heart disease. Moving from large N to N=1 As a scientist, I find the trends and risks uncovered in large populations to be interesting, but I am also cautious about automatically extending their conclusions to me. Large studies like the one I mention above seek to eliminate variation by selecting subjects from a sub-population whose age, gender and ethnicity may in fact be quite different from mine. Broader population studies that employ statistical approaches that adjust for demographic diversity or focus on group-level trends may mask diversity in individual responses due to genetic variation, microbiome composition and environmental exposures (just to name a few). Since I can’t replicate myself, I can’t do controlled experiments to determine whether lifestyle changes I make actually influence my own personal health risks in the future. I have to research what is known about generally applicable risk biomarkers, measure my own changes in those biomarkers in response to lifestyle changes over time, and hope that the statistical relationships between those biomarkers and future outcomes found in large-scale studies hold true for me. Learning from my own past biomarker data While reviewing my medical records to fill in missing weight data, I rediscovered historical data on my blood pressure and blood cholesterol, two major risk factors for heart disease. Requesting my medical records from past health care providers was free, costing me only a stamp and the time it took to fill out a request form. If you want to explore your own health history, I strongly encourage you to collect your own records and put your data into JMP! As I shared in an earlier blog post, my first recorded set of detailed cholesterol measurements is from 2008. That year, I found out that although my total cholesterol number was near 200, my HDL (good) cholesterol number was only 52 ng/dl, nearly to the low threshold of 50 that is considered to raise heart disease risk. I was on an upward weight swing at the time, exercising occasionally, but exercise was not enough to balance out my routine overeating habit. My blood pressure was borderline high, but I wasn’t yet ready to face the choice posed by my health care provider: Lose weight and change my ways, or face the reality of blood pressure medication in the near future. Since I had gained another 10 pounds by the time I was due for my 2009 checkup, I never even scheduled it. Something major happened in my biomarker data gap. I hit an important turning point and began to make changes. I'm not proud of the fact that I avoided cholesterol and blood pressure tests from 2008 to late 2010, but I am thankful that something major happened during that data gap. In mid-2009, I saw pictures from a family vacation (some of which are included in the first graph above) and faced the reality of my weight problem. I resolved to change my habits for the better. I reduced my daily calorie intake, started tracking my meals in a notebook, and added regular strength training and walking workouts. By my next checkup, my lifestyle changes had paid off. I had dropped 45 pounds, and my cholesterol composition had shifted dramatically for the better. My formerly borderline high blood pressure was now normal. Although my weight did rise again during my second pregnancy in 2011, I used the same strategies to shed the extra baby weight and return to my maintenance weight zone where my risk biomarkers have remained relatively stable. An alternative graph of the data I like the view above because it shows my data points over time. However, the simplified version below also has its advantages, showing that weight is not quite enough to explain my changing patterns. Although my post-baby weight in early 2012 was nearly identical to my non-pregnant weight a year before, my blood cholesterol level was clearly being influenced by other factors, likely some combination of caloric restriction combined with nursing a small child. An alternative view of my cholesterol test data by weight. Pondering the unknowns of biomarker measurement variability I weigh myself each morning and measure my body fat because smartphone-connected sensors make the process quick and easy. I can connect up and down trends to weekly, monthly and seasonal changes in my eating and exercise habits. This daily monitoring practice has given me useful insights into how I can manage my fluctuations and stay within my comfort zone. Unfortunately, the risk biomarker measurements I have collected from doctor’s visits are too few and far between to provide much understanding of the changes that could be happening as a result of shifts in my diet and activity patterns over time. If I had a sensor that permitted inexpensive replicate measures of various blood biomarkers on-demand and at home, I could probably begin to tease apart the technical and biological variation in the system and connect blood biomarker changes to trends in my eating and exercise habits. My limited experience with at-home measurements of daily fasting blood sugar and blood pressure has convinced me that there is clear variation in the system, but I don’t have the kind of replication required to separate the impact of technical and biological components of that variation. I measured my fasting glucose for a few weeks surrounding Thanksgiving 2014. Not surprisingly, there was variability from day to day! Regardless, testing my blood sugar throughout several days (including Thanksgiving 2014) provided some helpful insights. Chocolate greek yogurt and cheesecake barely affect my blood sugar levels, while I was shocked to see a 130-point rise in my blood sugar levels after having a single piece of pizza and a honey cinnamon pretzel! But as interesting as these insights into my personal blood sugar response patterns were to me, it turns out that they might be completely irrelevant to you. In fact, a recent study that used microbiome testing, extensive food logging and continuous glucose monitoring of 800 participants demonstrated that blood sugar responses to food are far from uniform across the population. Remarkably, some participants showed high and reproducible blood sugar spikes when they consumed foods generally considered to be "healthy" like bananas and tomatoes, yet displayed little to no blood sugar response after consuming "junky" foods like cookies. Others showed opposite responses. It was fantastic to see such well-collected evidence for something that I have believed for quite some time now: There is no one single eating approach that will work for everyone, and it requires some investigative work to find the right eating patterns that work for YOU. Is understanding biomarker variability worth it? Since I don’t live in that dream world where daily blood tests are easy and cheap, I am left to decide how much of my budget I want to dedicate to extra testing in the absence of a diagnosed medical problem. If it was really important to me to understand the variability of my lab results on a shorter time scale, I do have options today. At-home tests for LDL, HDL and tryglycerides are available. Or I could choose a direct-to-consumer (DTC) blood test option offering online ordering and blood draws at a service provider’s lab. DTC services like Inside Tracker go several steps further, calculating optimal biomarker levels based on personal attributes and offering personalized diet and exercise recommendations for athletes and others interested in optimizing their biomarker levels. The question becomes whether I can justify the extra cost of repeated blood biomarker testing to understand the variability in my system, especially since my worrisome numbers appear to be a thing of the past. Intuitively, it seems obvious that routine monitoring of seemingly healthy individuals like me could reveal much about pre-symptomatic patterns that lead to disease over time. Frequent monitoring is certainly worth the extra cost for professional athletes, who must be on the lookout for nutrient deficiencies and signs of over training that could impact their performance. But outside of competitive athletics, frequent testing of healthy individuals remains a controversial topic. If you don’t believe me, check out what happened to basketball team owner Mark Cuban after he tweeted last year that he has quarterly blood work and recommends it for anyone who can afford it. The headlines ranged from “Mark Cuban doesn’t understand health care” to “Mark Cuban is half right on blood tests “ to “Mark Cuban understands the future of health care.” So which is it? Given the money pouring into the fitness wearables market and the shifting focus of many large health care companies and hospital systems to electronic medical records, home monitoring and digital tools and apps, I side with Mark and the big data revolution on this. I suspect that frequent biomarker tests are going to be the norm in the future, for healthy and unhealthy individuals alike. I look forward to the day when I can track my blood biomarkers as easily and cheaply at home as I can track other metrics. For now, let Wear Red Day be your inspiration to keep track of whatever biomarker data that you can get, whether it's weight readings from your scale or data from your medical records. ... View more

If you read my last post, then you know that I’m giving myself the gift of data this holiday season! For me, collecting data on my diet and fitness habits is a gift that just keeps on giving. Although I may not look at all my data sets on a daily basis, the information is there when I need to use it to help me understand my patterns. Collecting food data isn’t just a holiday habit for me, however. I have been logging the foods I eat at each meal year-round for nearly five years, using BodyMedia’s app for most of that time before adopting MyFitnessPal earlier this year. I presented a poster at JMP Discovery 2014, which described how I imported my BodyMedia food log data into JMP by saving multiple pdfs to text, concatenating them, and using regular expressions to parse details into columns in a JMP table. MyFitnessPal saved me a few steps by producing report output that can be saved as a single text file, but I still needed to use regex parsing to create a usable JMP table. I processed the text derived from my BodyMedia and MyFitnessPal food logs using parallel but separate processes that converged at the very end in a single unified table. The big challenge: Combining data from two sources The major challenge I recently solved was unifying these two food log data tables, and my experience solving this is relevant to anyone who has to combine data from different sources. Minor differences like column names and format types were easy to adjust, and no matter that each table contained a few unique columns -- those simply came through with missing values for rows that didn’t have values. The main problem was that there were so few cases where item names were the same in both databases that I had to clean up the two tables using parallel but separate processes. I already knew that my BodyMedia food log contained many instances of redundant names that referred to essentially the same food item, and this made it much harder for me to aggregate data across those items to look for patterns. I used the JMP 12 Recode platform to group similar item names into a set of cleaned item names, and described this process in an earlier blog post. MyFitnessPal includes even more brand names and user-supplied items than BodyMedia, further magnifying this naming redundancy problem. When I switched to using MyFitnessPal to log my foods, I had tried to reuse my BodyMedia script to recode my MyFitnessPal item names. But with so few shared item names between the databases, it made more sense to create a completely separate Recode script for the MyFitnessPal project. Consolidating food names Now, my desire to combine my two data tables to examine my holiday eating patterns became a good motivation for discovering the most efficient way to take the cleaned item names from both tables and combine them into a single, consolidated set of item names. Of course, at the end of the exercise, I wanted to be able to save a Recode script that referred back to the original item names so I could continue to update that script as I added new items to my MyFitnessPal log. After tweaking column names and formats for consistency, I concatenated the tables, chose the Cleaned Item Name column, and opened the Recode platform. Since I had used similar patterns to rename items in both tables, many items showed up near one another. I worked through the rest of the items, grouping similar values and making heavy use of the Recode filter field to find subgroups of related items. I recoded the full set of cleaned items names into a new column I called Consolidated Item Name. I consolidated similar item names from my two food log data tables. To save a Recode script that I could reload into the dialog for use with new MyFitnessPal data, I used a formula column trick I shared in a previous blog post. It's a trick I discovered when I needed to create a Recode script “after the fact” to capture work I did in an early version of the JMP 12 platform before it supported saving and reloading a script into the dialog. Now, I used the same formula column approach (which makes use of the character Concat function) to create the "guts" for a Recode script to relate the original database item name to its final, consolidated name. After adding the appropriate header and footer to the script, I can reload my original item names and consolidated names into the Recode dialog and update the list when I import new food item names from MyFitnessPal. Assigning higher-level categories With my cleaned and complete data table in hand, I was now able to update a second data cleaning script I maintain to assign my food items into primary food categories. I have found that placing individual items into higher-level categories can be quite useful for comparing related item sets over time. I created my own classification system using groupings that made sense to me. For example, I have a food category called ChocolateCandy, which includes dark, milk and white-chocolate containing items. I rarely eat non-chocolate candy types, so giving the category a more generic name like Candy didn't work for me. When looking at major trends over time, I'm not going to split hairs about whether it was a milk chocolate or dark chocolate truffle that I ate, but I'd like to be able to determine what proportion of my treat calories belong to the whole category. For this blog post, I created a subset table containing just meals from a handful of selected holidays from the past five years. I used Recode to aggregate those chosen dates into a new category column called Holiday, and added a Value Ordering column property to it to lock these holidays in sequential order: New Year’s Day, my birthday, Thanksgiving, Christmas Eve, Christmas Day and New Year’s Eve. This Graph Builder heat map shows the number of calories I logged on each holiday, which varied pretty widely. When I was actively losing weight post-baby, I ate less, and in maintenance, I tend to eat more. This Graph Builder heat map shows that the amount of calories I ate on holiday varied, ranging from a low of 1396 (light yellow) to a high of 3580 (dark red). Days with no food log data show as white squares. Previously, I shared treemap visualizations of my food log data aggregated over days, weeks, months or years, but I find treemaps to also be useful for visualizing patterns in daily data in a compact way. I created a treemap in Graph Builder to show the calorie breakdown for each holiday meal during the four years where I had complete data for all six holidays. All of this data was collected in the BodyMedia app, which provides six potential slots for logging foods-Breakfast, AM Snack, Lunch, PM Snack, Dinner, and Late Snack. I used Year as the X grouping variable and Holiday as the Y grouping variable, and colored squares by meal. Since I assigned Calories as a Y variable, each holiday’s section of the treemap is sized in proportion to the highest number of total calories I ate, which you can see occurred on Christmas 2012. Meals vs. total daily calories I usually log foods into meal slots by the time of day when I ate them. For example, a late breakfast or lunch would likely be logged into a snack slot. You can see this effect in the timing of some of my holiday meals. On a non-holiday, I usually log breakfast, lunch and dinner as larger meals with smaller snacks in the morning and afternoon. On major holidays, the big meal of the day sometimes falls in the afternoon snack timeframe (for example, Christmas 2012, New Year’s Eve 2011 and 2014). In those cases, I usually logged a rather substantial dessert in the dinner time slot! Calories logged by meal depended on the timing of the big holiday meal. Given this variability in meal timing, I actually found it more helpful to remove the meal grouping variable and look at total daily calories grouped by food category. Grouping similar items gave me a better feel for the types of foods that tend to dominate my holiday eating. It is not at all surprising that these include lots of desserts, chocolate and caloric drinks. Holidays are definitely all about social eating for me and my family! Dessert calories definitely dominate my holiday eating patterns! Those special holiday treats Like many people, I look forward to certain special holiday foods. I thought it would be interesting to look across my whole data set and contrast the patterns of holiday-only items with treats I eat more regularly. For example, cheesecake is one of my favorite desserts, and I eat plain cheesecake throughout the year. The bar graph below shows my monthly calorie totals, filtered to include a select few food items. In contrast, pumpkin cheesecake is a seasonal treat, usually restricted to the holiday season --  that is, unless I happen to stash an extra one in the freezer like I did last year! Also, a dish called Cranberry Delight that I make only on holidays usually only appears in my food log between Thanksgiving and Christmas, although it did make an appearance at Easter dinner in 2013. Pumpkin cheesecake and cranberry delight are rare holiday season treats compared to plain cheesecake! So what was the biggest challenge of this blog post? You might think that it was the data collection, the recoding challenges or tweaking my visualizations to show my holiday eating patterns in the best possible light, but all those guesses are wrong. Actually, the toughest challenge was leaving the many JMP 13 updates behind and returning to JMP 12.2 to create my graphs! In fact, I blame the ongoing development of JMP 13 for making me a much less productive blogger in the second half of this year. I’ve been using development versions of JMP 13 for nearly a year now for all my work and personal projects. I am literally biting my tongue right now wanting to tell you about all the amazing new features that are coming! But you’ll just have to be patient until JMP 13 launches to hear about the new and improved additions that I can’t live without. In the meantime, happy holidays! ... View more

It can be easy to get discouraged when working on a personal health project during the time period that stretches from late fall to early winter. The “food holiday” season that stretches from October to January is filled with celebrations and their leftovers. Colder temperatures and shorter days also reduce the daily time window for comfortable outdoor exercise. The combination of social eating events and reduced exercise time has always been a double whammy for me in managing my weight. I used to think that my holiday eating patterns led me to New Year’s resolutions to lose weight, increase my daily activity, and address risk biomarkers like blood pressure and cholesterol levels. But is my theory correct? Looking back at my historical weight data since graduate school, I can see I didn't always collect the data to support my holiday weight gain theory. And when I did collect the data, it didn't necessarily support it. Although my weight trended up from October to January in 2002-3 and 2007-2008, I’ve actually maintained my weight fairly well during many past holiday seasons. When needed, I’ve even made weight loss progress during this time of the year. In graduate school, I lost 15 pounds during this period, and repeated this success twice post-pregnancy (shown with green reference ranges and annotations above). I even reached my leanest point ever in maintenance during the 2013-2014 holiday season! This fall, I’ve been working on correcting a small upward swing in my weight that happened over the summer. I hope to avoid major changes in my eating and workout habits at the end of this year so that I can avoid needing to make any major weight-related resolutions come January. Biggest lessons learned So what have I learned about the key ingredients to making and keeping diet and exercise-related meeting goals this late in the year? Setting a concrete and measureable goal is critically important. In the case of holiday weight gain, it’s easy to identify weight as the variable to measure and maintenance through the eating season as my goal. To monitor how I am doing, I commit to monitoring my weight each morning by stepping on my wireless scale. If I see my weight trend going in the wrong direction, I adjust my eating and exercise plans to compensate. Similarly, if I set a goal to work out three times a week, I track each workout. Making data collection a habit is essential to assessing my progress. Examining my eating and exercise data from past years helped me verify that I have been most successful during the holiday seasons when tracking my eating and exercise habits. Exploring data collected during other times of the year also provides critical baseline information that helps me plan ahead for holiday eating events where I know I will eat special (and calorically dense) foods. If I exceed my typical limits, I have collected enough data to avoid overreacting to the inevitable weight spike caused by the water retention from the extra food, carbs and sodium in my system. I know that this effect is temporary, similar to weekend weight spikes I’ve observed. A gift from me, to me This holiday season, I will continue to give myself the gift of data. If you too are interested in health and fitness data collection during the holidays and into the New Year, take a look at my list of ideas based on my own ongoing projects. I promise, this list has nothing to do with whether you’ve been naughty or nice! Some ideas involve collating historical data, and others make use of measurements from various sensors. (By the way, sensors are simply devices that detect and measure changes in a system. Many of our JMP customers use sensors in the development, manufacturing or operation of a product, and some even produce sensors as products.) Sensors can greatly simplify your self-tracking efforts, and you may be wearing one on your wrist or carrying one in your pocket right now. In addition to monitoring steps and physical activity, many activity monitors, smartphones and smartwatches can help you track your sleep. If you own analog sensors that are not connected to the Internet of Things (IoT), or want to collect data on your eating habits or workouts, free or inexpensive apps can help you collate your data, and many even let you export CSV files that JMP can read. You'll see some ideas below for exploring metrics and test results from your personal health history, some of which may have been generated using sensors in a medical setting. Start with one or two new data types Like foods on a restaurant menu, it would be overwhelming to consider all the items on this list at once. Instead, I recommend prioritizing by selecting one or two data types to begin collecting and exploring. Once you have had the chance to assess and learn from your data, then consider adopting additional tracking projects. I’ve organized this list loosely into inputs and outputs, with the idea that changing the inputs to your system affects various outputs. I’ve also included one item that I call supplemental, and that is location data. I find this is useful for helping generate or recall the details of other tracked variables like locations for a specific meal or the path taken during a workout. Without further ado, here are a few ideas for data types you might consider measuring and visualizing with JMP: Outputs 1. Weight (medical records, home testing) 2. Blood cholesterol/lipid profiles (lab testing) 3. Blood pressure (lab testing, home testing) 4. Blood glucose profiles (lab testing, home testing) 5. Body fat percentage (lab testing, gym testing, home testing) 6. Resting heart rate (medical records, home testing) 7. Heart rate variability (home testing) Inputs 8 Food log items, macros, calories consumed 9 Daily steps/baseline activity 10 Base calories burned (home testing) 11 Specific workout data (home testing) Supplemental 12. Location data I hope you maintain your goals over the holidays and start the new year off by tracking or exploring some new data! I'll share more about what I've learned from exploring my own data starting next month. ... View more

I hope that by now you've heard about our new Technically Speaking webcast series. The goal of this series is to help you understand and work through issues with your data so that you can focus on answering your most important questions. It features JMP team members sharing data case studies and covers topics ranging from design of experiments to data cleanup, results visualization, SPC and model comparison. Recently, I recorded a session for this series on a workout data project I’ve been doing since early in the year. I also spoke about my project at Discovery Summit 2015 in San Diego (you can check out my slides) and blogged about it earlier in the year in my JMP User Community blog and in my Fitness and Food series here. Custom maps of body shape help me track and understand my workout data. Working through my workout data gave me an opportunity to: Use many of the data cleanup and categorization features in the JMP 12 Recode platform. Develop a custom set of body map shape files (which you can download from the JMP File Exchange). Create Graph Builder visualizations of my workout patterns. Make App Builder dashboards with selection filters to further explore my data. You can download a sample data file that goes along with this new webcast. It includes data from the month of January with total weight lifted summarized at the exercise level. If you’re interested in creating your own workout data table, the page also includes a template file you can use to get started. Happy tracking, and let me know what you think of my Technically Speaking session! ... View more

In my last post, I detailed how to obtain 9 digit zip codes and their X and Y coordinates from a table available from the SAS support site. I passed this information along to the customer in question and he was able to merge in the 9 digit zip information into his table. His next question regarded obtaining the city and state information corresponding to these 9 digit zips. I did a little digging online and found a graphic that showed the 5 digit zip prefixes for various states followed a predictable pattern. Next, I found a table on Wikipedia that contained state names, cities and zip code prefixes and figured out how to join it to the 9 digit zip table I created in my last post.  In JMP, you can do the following to add City and State info to any table containing 5 digit zips: Click File > Internet Open. In the dialog that pops up, paste in this link: https://en.wikipedia.org/wiki/List_of_ZIP_code_prefixes​. Choose Web Page from the pull-down menu and click OK.  Click File > Save As Data Table. In the dialog that pops up, leave everything as selected and click OK.  A bunch of tables will open in JMP. Click Tables > Concatenate, and remove anything currently showing in the selection list.  Now choose all the open tables that contain zip code info. Their names start with "List of Zip Code prefixes". Check "Create source column" in the Concatenate dialog and click OK.  You will now have a big table with multiple columns of data. We need to stack them to get them all in one column. Click Tables > Stack, select Column 1 through Column 10 and choose them as the Stack Columns. Click OK. Now we have a table with columns Source Code, Label, and Data.  Data contains the 3 digit prefix of the zips, the state and location.  But it still needs some cleanup. Select the Data column header, then click Cols > Utilities > Text to Columns. For the delimiter, enter a space. (Note:  the Utilities menu and Text to Columns are only available in JMP 12. If you are working in an earlier version, you have a few options to try to work around this :  you can download Jeff Perkinson's TexttoCols addin from the JMP file exchange, drag it into JMP to install, and it will add an item to your add-ins menu, or you can save your JMP table in Excel format, then use the Text to Columns feature in Excel also splitting on spaces, then re-import the Excel file into JMP.) Double click on the Data 1 column header to rename it to Zip prefix Double click on the Data 2 column header to rename it to State.  With the State column selected, click control-F (or Mac equivalent) and check the Restrict to Selected Column box.  Find *, replace with nothing. Find Not, replace with nothing. Select columns Data 3 through Data 8 with shift click. Click Cols > Utilities > Combine Columns. (Again, only found in JMP 12, though you can use a formula column for concatenation in an earlier version). In the Combine Columns dialog, specify City as column Name and replace the "," in the Delimiter box with a space. Click OK. You now have a column of the city names put back together with spaces. We want to get the "Not in use" string back in the rows where it should be, so do find and replace again, with that column selected, replacing "in use" with "Not in use" as it was originally before we removed the Not in the state column. You can check the table-basically all the rows with no state listed should have Not in use instead of a city name under the city column. Now in your zip code data table, you will need to parse the zip code variable to get the first 3 digits to match with our newly imported table. Open your zip code table, then double click on the first empty column after the existing columns.  This will create a new column called Column 7. Double click to bring up the Column Properties dialog. Change data type to Character and rename the column to Three digit prefix. From the Column Properties pulldown in that dialog, choose Formula. In the Formula Editor window that opens, choose ZIP under the columns list under Table Columns, then Character > Substr under the column chooser under Functions. In the Start box, type 1, and in the Length box, type 3. Click OK. If you now do Tables > Join and match by the two 3 digit prefix variables, you should have the street, city, state, and zip variables all together. I used these settings to join. Note the matching columns, and Include non-matches checkboxes for both tables. I used the steps above to add city and state info into my ZIP4 data table containing the 9 digit zip codes I imported from the SAS support site and graphed the X and Y coordinates: (here you go Byron​! ) ... View more

Recently, a customer asked me about how to get a table of 9 digit zip codes with latitude and longitude values that could be imported into JMP to merge with an existing JMP data table that contained 9 digit zip codes. This question was new to me, and I had a suggestion from someone else that perhaps SAS PROC GEOCODE would be helpful. However, I knew the customer wasn't a SAS programmer so I was hoping to find a happy path that didn't include needing to learn how to program in SAS. I first checked out the JMP Geocoder add-in, but it seemed to only Geocode to the 5 digit zip level. The customer already had those X/Y lat/lon locations but needed something more granular, hence the desire for the 9 digit zips. I next investigated PROC GEOCODE and it turned out that the proc required two tables-your data table and a table with the 9 digit zips linked to latitude and longitude values. But I already had the zip codes I needed-if I could just find the index table, it seemed I wouldn't need the proc to join the tables, although it did offer some fuzzy matching that a straight join wouldn't. I checked in with SAS tech support and they said that the data table they have that includes the 9 digit zip files is found on a link on this page down under the Zip+4 Geocoding section. Y ou do need a SAS profile ID in order to download the zip file, which you already have if you're reading this and are a member of the JMP community. In any case, it's free to set up a SAS profile. The file is from 2006, so it is out of date, but a pparently the current version of the TIGER data doesn’t include zip+4 data. The next section on the page mentions a paid source (Melissa Data) that provides more current info and keeps it updated for a fee. Once you download the zip file, you can see that it contains a readme file, a CSV file (ZIP4.CSV), and a SAS program. I thought I'd import the CSV straight into JMP at that point, but this is where I hit snag #1: the CSV file lacked column headers! I went back to the SAS program and imported the data using it, and the column headers were added so I could see what they were. However, the program that added column names also appeared to reorder the the columns, so it became a bit of a matching game to determine which columns in the CSV matched the named columns in the SAS table. Now I went back to JMP to pull in the imported SAS file containing the column headers, and this is where I ran into snag #2. Some of the 5 digit zips started with "00" or "0" and the column was being treated as numeric data when imported from CSV into JMP, so the leading 0's were being dropped from the zips upon import. I checked the Column Properties and there was a Z5 SAS format set on the column, but I talked to Xan about it and he confirmed that JMP didn't currently understand that zip code format. While I could have fixed this with a conditional formula column after the fact by changing the column to character and adding 0, 1 or 2 zeroes based on the length of each zip, he suggested that the fastest solution might be to re-import the CSV file using JMP's Text Import Preview mode, which would allow me to both change the column type to character and rename the columns I needed before importing. I opened my SAS file and determined the column header names for just the columns I needed and did just that. To import the ZIP4.CSV file yourself, you will need to cllick File > New Script in JMP, then paste the text below into a script window. Put your cursor at the start of the first line and click the Run Script button on the toolbar.  You can hover to find that button with a tooltip, or else right click in the script window and choose Run Script. As noted below, you'll need to point to the correct path on your machine to the file you downloaded and unzipped from the SAS support website linked to above. Names Default to Here ( 1 ) ; // Replace the path below with the path to the CSV file saved on your machine zipdata = Open (       "C:\<Your folder location>\ZIP4.csv" ,       columns( Column ( "ZIP" , Character , "Nominal" ) , Column ( "ZIP+4" , Character , "Nominal" ) , Column ( "c000003" , Character , "Nominal" ) , Column ( "City" , Character , "Nominal" ) , Column ( "c000005" , Character , "Nominal" ) , Column ( "c000006" , Character , "Nominal" ) , Column ( "c000007" , Character , "Nominal" ) , Column ( "c000008" , Numeric , "Continuous" , Format ( "Best" , 10 ) ) , Column ( "c000009" , Numeric , "Continuous" , Format ( "Best" , 10 ) ) , Column ( "c0000010" , Numeric , "Continuous" , Format ( "Best" , 10 ) ) , Column ( "X" , Numeric , "Continuous" , Format ( "Best" , 10 ) ) , Column ( "Y" , Numeric , "Continuous" , Format ( "Best" , 10 ) ) , Column ( "c0000013" , Character , "Nominal" ) , Column ( "c0000014" , Character , "Nominal" )       ) ,       Import Settings( End Of Line( CRLF , CR , LF ) , End Of Field( Comma , CSV( 0 ) ) , Strip Quotes( 1 ) , Use Apostrophe as Quotation Mark( 0 ) , Scan Whole File( 1 ) , Treat empty columns as numeric( 0 ) , CompressNumericColumns( 0 ) , CompressCharacterColumns( 0 ) , CompressAllowListCheck( 0 ) , Labels( 0 ) , Column Names Start( 1 ) , Data Starts( 1 ) , Lines To Read( "All" ) , Year Rule( "20xx" )       ) ) ; // You can select and delete the unnamed columns in the table-they are Census related details that I don't think you really need.  If you find that you do, let me know. As I mentioned above, I didn't go through the matching exercise to get all the correct column names for all columns in this file because I didn't need the census-related attributes, just the zip code, city, and X/Y coordinates corresponding to latitude and longitude. If you have access to SAS, you can run a script to import the file into SAS to see all of the column names and tweak this script to name all the generic c000003, c000004 etc columns correctly, or if you need these and don't have SAS, please add a comment here and I'll go back to complete that work. After you run the script to generate the table and save it, you can drag the X and Y to the drop zones in graph builder to see the locations of the zip codes.  They’re so dense that they largely show up the shape of the country! The imported table is ~16.6 M rows and 14 columns. My final table was still half a gig when trimmed down to just the 5 columns I needed.  Next, you will need to join this table with your existing JMP table containing 9 digit zips. One important factor is to determine if your zip code format matches the format of the imported table, which has the 5 digit zip code and 4 digit extension in separate columns. If your data table does too, then you should be good to go doing Tables > Join, choosing to join the two tables, By Matching Columns, and choosing the right columns to match up both columns. However, if you need a column in the format of XXXXX-XXXX because that’s the format your data table has the zip codes in, then you can do the following to the table you just imported. Right click on the blank column at the right of the table and choose New Columns… In the dialog that pops up, name the new column ZIP CODE, and under the Column Properties pulldown, choose Formula. This opens JMP’s formula editor. Click on ZIP in the column chooser under the Table Columns pulldown, then choose Character in the column selection box under the Functions section and choose Concat from the popup list.  Type ‘-‘ in the empty box that opens. Click Character > Concat again, and with the new blank box selected, choose the Zip+4 column from the table column selection box.  You should see a formula like this: Once your formula looks like the one above, click OK in the formula editor and OK again in the column dialog.  That should add a new column to your table called ZIP CODE that contains zips in the format 00601-2103. You can then join with your original data table containing zips in that format. The first two rows in the CSV file seem to have gotten a bit messed up in the import, but I’m not sure why unless the problems existed in the original CSV file. They're easily fixed.  I hope this information might be helpful to someone else needing 9 digit zip code data.  Again, it's not the most recent version of this data that exists, but might be enough for some applications. If you need better fuzzy matching of address or zip code data, and you have access to SAS, you can check out PROC GEOCODE as well. P.S. I'm certain there are better ways to do this via scripting and SAS integration, but when helping a customer that isn't a scripter and doesn't have access to SAS, I had to find a JMP-only and largely interactive route. jeff.perkinson​ ... View more

Here is a pdf version of the talk I presented on my workout data at Discovery 2015.  I didn't get to the final example of modifying the application, but it's included here.  You can see more information about this topic (and others) in my Fitness and Food series on the JMP blog: Fitness and Food - JMP Blog. ... View more

In my previous post, I shared several Graph Builder graphs of the muscle quality and body fat percentage data I’ve been getting from my Skulpt Aim. However, since the post was already long, I didn’t have a chance to include any instructions on how to recreate those graphs. In case you have similar data or want to create a similar type of graph, here’s how to do it! I have also included animated gifs that capture the main steps in the process of creating each graph. (Click the animated gifs and graphs to see them enlarged.) Weight Graph with Annotations This graph shows all the weight data points that I have gleaned from medical record requests, notebooks, and wireless scale data from birth through about 2014. My table includes columns with weight measurements in rows and the date of each in m/d/y format. In Graph Builder, I used the following steps to create this graph: Dragged Weight to the Y axis (I deleted the label intentionally). Dragged Date to the X axis. Right-clicked in the graph to choose Smoother > Change to > Line. Right-clicked on the line in the legend to change the Line Color to black. Unchecked the Show Legend option under the red triangle next to Graph Builder. Right-clicked on the X axis and increased the axis label nesting to 3, and then turned off labels and tick marks on Level 1 and Level 2 tab (corresponding to day and month) and tick marks on Level 3 tab (year). Added a Y-axis reference range by right-clicking on the Y axis to open the Axis Settings dialog, clicking the Allow Ranges checkbox, and specifying a range extending from 130-135. Added text annotations by clicking on the toolbar icon, typing in my text, and right-clicking to change the background color, font color and style. I color-coded the annotation for my childhood weight range in a different color. Added vertical line annotations to indicate each of the weight ranges for childhood and adult years. Graphs of Body Fat Percentage and Muscle Quality The stacked table I used to create this graph included columns for Date, % fat, MQ, Body part, and Side, with one row for each observation. To create this graph, I took the following steps in Graph Builder: Dragged % fat to the Y axis drop zone. Dragged MQ to the X axis drop zone. Clicked on the smoother element icon to remove it. Added a regression line by dragging that element from the Graph Builder toolbar into the graph. To create the version shown below with a separate color and regression element for each body part, I dragged Body Part to the Overlay variable drop zone. You can see my steps in the animated gif below. Bar Graph of Replicates by Side To create this graph, I started with my Skulpt Aim results table, and then created a summary table from that data by clicking Tables > Summary, choosing the Replicate column, selecting N as the Statistic, and adding Body part and Side as Group variables. Starting from the Summary results table, I performed the following steps: Dragged N(Replicates) to the Y-axis drop zone. Clicked the Bar element on the Graph Builder toolbar. Changed the Summary Statistic to Sum in the Bar options on the left. Dragged Body part to the X-axis drop zone. Dragged Side to the Group X drop zone. Body Shape Graph of MQ and % fat I made my next two graphs using custom female muscle shape files I created with the free Custom Map Creator add-in. I described how I used the add-in to create these files in a previous post. By the way, I uploaded copies of my map files to the JMP user community here. Apologies to the guys -- you are going to have to create your own shape files! To make my custom map visible to JMP, I placed the shape files created by the add-in into the folder C:\Users\~myusername\AppData\Roaming\SAS\JMP\Maps. Comparing my shape files to the areas measured by the Aim, I noted that the Back measurement area for the Aim was in the area I call Trapezius in my shape files. I created a copy of my Body part variable in my Aim results table and named it Body shape, then changed all occurrences of Upper Back to Trapezius in that column so that the colors would show up in the correct area of my body map. I did wish that I had done a better job with my Trapezius shape. Then, I performed the following steps: Dragged in MQ as a color variable. Right-clicked on the axis and chose gradient. In the Gradient Settings dialog, clicked the Discrete Colors checkbox and clicked on the Color Theme bar to choose Muted Yellow to Red as the Current Theme. Right-clicked on the X and Y axis to open the Axis Settings dialog and uncheck the boxes for tick marks and labels. I performed the same set of steps for my % fat graph which appeared below the MQ one, except that I also clicked the Reverse Color checkbox so that lower body fat areas would show up colored red rather than yellow. As you’ll recall, MQ and % fat are inversely related, and the Skulpt app also uses color gradients that color high MQ and low % fat similarly. Graphing My Measurement Variability My final graph required calculations of daily Standard Deviation by body part and side for both MQ and % fat. I initially used Summary to calculate these statistics, dragging the SD columns from the linked Summary table into my main table. While this is a handy shortcut for a one-time calculation, I preferred for the calculation to happen automatically when I added new data to the table rather than repeating the Summary step each day. Thanks to a suggestion from Xan Gregg, I decided to use a Column Standard Deviation formula column and specify my Body Part/Side concatenated variable and Date as the two optional By Vars in the formula. This gave me values for SD(MQ) and SD(% fat) for each body part, side and day. To create a graph of these values, I performed the following steps: Dragged SD[MQ] to the Y axis drop zone, unchecked the Jitter option, and chose Mean as my Summary Statistic. Dragged SD[% fat] to the Y axis drop zone below MQ (in a separate section of the graph), unchecked Jitter, and chose Mean as my Summary Statistic. Added Body part as an Overlay variable. Added smoothers to both graphs by dragging that element in from the Graph Builder toolbar. Dragged Month, then Day to the X axis to create a nested axis layout. Added Side as a Group X variable. Added a Local Data Filter from the toolbar icon, choosing a subset of Body parts (Abs, Bicep, Tricep, Quadricep, and 4 site average) and the Right Side in the filter. I also made some final adjustments to ranges and decimals in the Axis Setting dialog, added shaded axis reference ranges between 0 and 1, and increased the Graph Spacing by right-clicking on the division between the graph sections and changing the number to 5. I’ll be giving a talk on using my custom body map file with my workout data at JMP Discovery Summit in San Diego next week. I hope to see you there! If you won’t be in San Diego, you can see my previous blog posts on creating custom maps and workout data visualization in my Fitness and Food series on the JMP Blog. ... View more