Self tracking is business intelligence

Back when secretaries were common, you could have had yours track your day in 15-minute increments. In his book The Effective Executive, Peter Drucker suggested this as a way to find out what you really did all day. The results were usually, let’s say, a starting point for improvement.

Tracking your time then and now is personal, it’s messy, and it’s the essence of business intelligence: collecting data and reading it for guidance in business activities that matter. Is there anything that matters more to an organization than productivity of its people? For a small office or home-based business, this might be the best BI there is.

This gets no recognition in the BI industry that I can find, at least not in the conservative world of TDWI. At least not yet.

PI — for “private intelligence” — has different issues, starting with data collection. In BI, data comes from transactions, all recorded routinely. In PI, most of it has to come from a “secretary” or from our own, tedious notation.

I dabbled in it once. The insights were good, if painful, but mostly it was tedious. A few years ago, a confluence of personal events let me do something I’d always wanted to try: hole up for a few months in a Sicilian village I knew slightly. The food was good, I had relatives nearby, and the nearby church bells rang all day and all night, four times an hour. At the same time, I had a book to edit. To stay productive, I made a game out of the work, tracking my time to the minute in Filemaker.

I liked the local food and started to hate the book, an office manual that inadvertently revealed a con game. Even so, I threw myself at it every day. But no matter how hard I tried, no full day ever resulted in more than about two hours of actual, productive work. My “quick breaks” for walks and coffee with a friend actually took up more time.

I made a Filemaker database because I could find no off-the-shelf product that would do anything close. Each period, no matter how short, had a starting and ending times I entered with buttons, and a calculation field figured the duration. A drop-down menu offered my usual activites. I could make a report for any period.

I thought some product would do that better, but I could find nothing. Then the May 2 issue of the New York Times Magazine ran an article by Gary Wolf about this, “The Data Driven Life.” My Filemaker invention wasn’t too far from what others have used, and now new devices are coming along that could make all that seem so old hat. Some people are even sharing their data on the cloud.

But as in traditional BI, the technology just gets you in the door. The show has just begun.

Most people Wolf writes about do it for personal reasons. One wanted to know how his coffee consumption helped him focus, another tried to cure his sleep apnea, and still another noticed that flax seed oil, or just lots of butter, improved his cognitive performance.

As in good BI, the experiments often raised new questions. And sometimes the new questions are unexpected, as in Wolf’s own experience.

Often, pioneering trackers struggle with feelings of being both aided and tormented by the very systems they have built. I know what this is like. I used to track my work hours, and it was a miserable process. With my spreadsheet, I inadvertently transformed myself into the mean-spirited, small-minded boss I imagined I was escaping through self- employment. Taking advantage of the explosion of self-tracking services available on the Web, I started analyzing my workday at a finer level. Every time I moved to a new activity — picked up the phone, opened a Web browser, answered e-mail — I made a couple of clicks with my mouse, which recorded the change. After a few weeks I looked at the data and marveled. My day was a patchwork of distraction, interspersed with valuable, but too rare, periods of focus. In total, the amount of uninterrupted close attention I was able to muster in a given workday was less than three hours. After I got over the humiliation, I came to see how valuable this knowledge was. The efficiency lesson was that I could gain significant benefit by extending my day at my desk by only a few minutes, as long as these minutes were well spent. But a greater lesson was that by tracking hours at my desk I was making an unnecessary concession to a worthless stereotype. Does anybody really believe that long hours at a desk are a vocational ideal? I got nothing from my tracking system until I used it as a source of critical perspective, not on my performance but on my assumptions about what was important to track.

I wish Drucker were around to respond. Wolf’s insight sounds like important stuff for everyday knowledge workers, especially those who work alone. What’s more important to a knowledge worker than time?

These experiments are often haphazard and highly personal.

Generally, when we try to change, we simply thrash about: we improvise, guess, forget our results or change the conditions without even noticing the results. Errors are possible in self-tracking and self-experiment, of course. It is easy to mistake a transient effect for a permanent one, or miss some hidden factor that is influencing your data and confounding your conclusions. But once you start gathering data, recording the dates, toggling the conditions back and forth while keeping careful records of the outcome, you gain a tremendous advantage over the normal human practice of making no valid effort whatsoever.

Yes, just as analytics gives companies a tremendous advantage over those who make less effort.

Let the BI traditionalists pooh-pooh self-tracking. The very same people might have dismissed such things as visual analysis, agile development, and at one time even business intelligence itself. Sometimes it take a few pioneers and geeks, perhaps even a secretary, to prove a concept.

3 Responses to Self tracking is business intelligence

  1. It seems strange that BI folks (data junkies!) would pooh-pooh the idea of self-tracking, doesn’t it?

    Finding out where your personal time is going was the inspiration behind our time-tracking app, Qlockwork.

    If you’re doing lots of offline activity then a “timer” (even if it’s the Sicilian church bells tolling 4 times an hour) is just about the only way to track where your time is going. But if you’re doing a lot of work on PC there are apps (such as Qlockwork) which will track your activities for you automatically.

    When you find out just how much time you’ve spent on different projects – or how much you’re really spending on Facebook, even if it just feels like a couple of minutes! – it’s amazing how you can rationalize your time.

  2. I discount what experience BI/DW people say about this stuff. Focusing on the “last mile” of BI – how users interact with data – is already, and will be become even more important as the quantity and granularity of personal data continues to increase.

    Imagine if you had diabetes and could just tap a small sensor to your body and have your blood sugar level sent every 60 seconds to a server via the web…and then view that data via time series any time you want. Or, perhaps if the data hits a threshold your doctor could be automatically emailed the latest data with that time-series info embedded.

    That’s were we are headed. PI may become a requirement of your health insurer’s policy.

Leave a reply

Other recent posts

End of one-size-fits-all data stories

This appeared originally on the TDWI site in September behind a paywall. It’s still there, but today they’ve had the 90 days of exclusive use that I agreed to. Survey after survey reveals that about 80 percent of business users don’t use data analysis—despite all the marketing and “easy to use” tools. As if in… Continue Reading

Qlik finally set to leapfrog Tableau?

Who’s your rival? I carelessly asked a Qlik person at the company’s annual analyst reception Monday night in Miami if she hadn’t once worked for Tableau. Her revulsion was immediate. “No! Never!,” she said. We smiled. There was so much more to talk about. For one thing, how will private equity change things? Qlik wasn’t… Continue Reading

Five Tips for Better Data Stories

Originally published on September 22, 2015 in BI This Week, a TDWI publication. A “data story” sounds like such a great idea. You just mix data with storytelling and you’re done — except that most data storytellers get one thing wrong: they drown out the story with data. Such storytellers, I believe, assume that audiences… Continue Reading

Bohemian Grove a la BI

The Bohemian Grove of the BI industry convenes for the fifteenth time in just three weeks. Naturally, you ask the obvious question: Are you serious? The Grove? A summit? The answer begins with a fond recollection of the Grove. If you’ve never attended the Bohemian Grove yourself — I haven’t, though I live in the… Continue Reading