Illuminating results from a failed analysis

[#1]

A couple of years ago, I began to look at how I could automate the outdoor flood lights on my house. I wanted to schedule when they would turn on and off. There were two primary reasons.

First, changing them is a real pain. My backyard slopes down quite a bit and even though I have an 11-foot pole, it’s still only long enough to reach two of the bulbs. For the other four, I have to use an extension ladder. And though I’ve gotten more accustomed to it over the years, it still makes me nervous every time I climb up.

Second, my wife and I love to travel. Not wanting the house to be dark while we’re gone, we’d leave the spotlights on 24/7. Which made it a little obvious that we weren’t home. It also required me to change the bulbs more often (sometimes as soon as we returned). A two-pack of bulbs cost $9.99, which certainly adds up over time.

For curiosity’s sake, I started keeping a diary of how often I changed each bulb. My initial plan was to compare this information to the data collected after I implemented a home automation system. The objectives of my analyses were as follows:

• How much longer would the bulbs last by having them automated?
• What were the resulting cost savings?

Let's take a look at how my plan turned out....

Before Home Automation

Prior to implementing automation, I collected 20 months of data. There are six bulbs on my house (three fixtures of two bulbs each). Two bulbs for the front yard, and four on the back. I used timeanddate.com to calculate the number of days in each interval.

I teamed up with Julian Parris (@julian) to do the analyses in JMP. The first thing we noticed was that I changed FrontBulb1 the most often. If you look at the scatterplot below, you will see that four of the intervals last only 40 days. There doesn't seem to be any predictable reason for this, because FrontBulb2, in the same fixture, is almost a reverse image. And while most of the intervals for FrontBulb1 range from 40 to 94 days, we have one outlier of 169 days. In fact, many of the bulbs have a fairly consistent range of 40 to 100 days, with at least one outlier. BackRight1 had the highest interval of all at 214 days.

 If we look at the median life for each bulb, we see that FrontBulb1, BackRight1 and BackRight2 all have a tight range of 50 to 56 days. This isn't surprising for BackRight1 & 2, as they're both in the same fixture. However, FrontBulb2 has a significantly higher median average of 72 days. BackLeft1 and BackLeft2 are both in the higher range of 77 to 86 days, with BackLeft2 having the highest median interval of all.

 So, in implementing home automation, my goal was to improve bulb life from a general range of 40 to 100 days. And to then compute the resulting cost savings.

After Home Automation

In May of 2019, just before our summer vacation, we had our automation system installed. I programmed the outdoor spotlights to turn on at sunrise and turn off at sunset. To get a clean data set for comparison, I changed all six bulbs on the house just before we left in June (the final interval for each bulb was eliminated from the data set above). I was very excited to start collecting the new data and eventually compare it to the previous data set.

However, I ran into a couple of unexpected hiccups:

• When I purchased the six new bulbs, the standard spotlights were no longer available. The store only carried new LED bulbs.
• The LED bulbs were slightly more expensive ($14.99 for two as opposed to $9.99).

These differences would prevent an apples-to-apples comparison, so I’d have to adjust accordingly. But that wasn’t the half of it.

When we returned from our trip, I was happy to see that every bulb was still burning. I was sure I’d have to change one after another month or two, but that wasn’t the case either. My automation system was working great!

The Unexpected Results

But then something else happened. Or rather, didn’t happen.

Not a single bulb burned out after four months. Or six. Or eight.

I eventually realized that the automation hadn’t made any difference at all. Or at least not one that I could measure. It was the new LED bulbs.

It has now been over two years (longer than my original data set) since I changed all the bulbs. And they are still burning. I haven’t changed a single one.

Obviously, my cost savings in bulbs has been significant. Despite the higher price, the LED bulbs are far more efficient.

When you factor in the costs of the new switches and installation, I've basically broken even. But I can see where I’ll eventually come out ahead. And the automated switches gave me something worth even more: peace of mind.

In the end, I achieved my goal of having to change the bulbs less often. I just got there a completely different way than I expected.

My original analysis may not have worked as planned, but the results were still rather illuminating.

You can explore the interactive results of this analysis at JMP Public.