One of the fastest-growing technology trends today is Big Data, probably behind the Internet of Things and ahead of Virtual Reality. For instance, a search on the job site Indeed for “Big Data” returned almost 20,000 entries.
But what about Little Data (which only gets four hits on Indeed)?
Big data is mining huge, heterogeneous data sets and pulling out subtle information that can inform all sorts of decisions.
Let’s look at climate change science as an example. Data comes from atmospheric measurements over Hawaii, temperature data across the globe, ice cores from Antarctica and Greenland, underwater measurements from all the world’s seas, and more. Some of the data were taken by satellite last week; others written in notebooks centuries ago.
It’s been pored over by scientists from every country in the world. The data and analysis needed to predict how the climate is changing and will change is complicated. To really understand it requires a PhD. The details are so complex that we have been unable to decisively act on this critical issue.
What Is Little Data?
Little data is the opposite. It’s the 2+2=4 kind of things.
Little data is the obvious observations and conclusions that those paying attention will catch and can use to their advantage. It’s looking outside, seeing it’s raining, and deciding to put on a jacket. It’s noticing that the prices and quality of the food is better at one store than another and using that information to decide where to shop. It’s noticing that if you drink coffee after 5 p.m., you have trouble going to sleep, so you stop drinking coffee after 5 p.m.
Little data has three steps:
- Gather data.
- Do some straightforward analysis of the data.
- Act based on your analysis.
To some extent, little data and big data are close to the same thing, it’s just a matter of degree. The biggest difference is that the analysis for little data is straightforward. If you’re looking for someone with a PhD in math to help with your analysis, that’s not little data. For little data, you should be able to do the analysis in Excel. The challenge is knowing how to respond to your results.
Improve Schedules with Little Data
Here’s a straightforward way to use little data to improve your schedules: record task estimates and actuals. The data should include who did the estimation and the work. Using a pivot table in Excel, you can see which estimators typically underestimate or overestimate. You can also see which of your team members take more or less time than was predicated. There are many ways this data can improve your organization, including:
- Decrease the bias of future estimations
- Identify team members who are not using best practices (and therefore take longer)
- Identify team members who have practices you should transfer to other team members
As is often the case, data acquisition is straightforward, analysis simple, and the response requires further digging.
Measuring KPIs
Key Performance Indicators (KPIs) are common little data management technique. Leadership decides that certain easily measurable metrics are key to the organization’s success, targets are set, and data acquired. If the performance does not reach the target, then some form of response is taken.
For example, you may be managing a manufacturing line. Your KPI is the number of units manufactured per hour. In creating the manufacturing process, you know you can build 100 units an hour, so you set your target at 80 units an hour to account for the normal hiccups (e.g. you’re training a new team member).
Data collection and analysis are easy. If you’re meeting your target, you can move on to other issues or raise the target. Ideally, if you don’t meet your target, the response is agreed to prior to acquiring the data. Often it just indicates you need to dig deeper, as in this case. As is all small data analysis, the challenge is in the response, not acquiring or analyzing the data.
A Real-Life Example: Test Scores
One of the most controversial examples of little data is standardized school tests. The data is homogenous and straightforward (if time-consuming) to collect. The naïve analysis is trivial (average score by grade and school). The response is complex and fraught with challenges.
In 2010, an elementary school near my house was labeled as failing according to the No Child Left Behind law. A majority of the students were from refugee and immigrant families. Many didn’t speak English at home, which certainly posed a challenge for the school.
The metric, test scores didn’t determine what action was called for but made clear there was a problem. The district responded by bringing in a new principal and new teachers, and a concerted effort was undertaken to improve performance. After four years, the performance of the school went from one of the worst schools in the state to only a bit worse than the average school. The little data approach showed that by the metrics we use, the interventions improved the performance.
But this same metric can be misused. There are two middle schools near our house, and we choose to send our son to the one with lower test scores. The school our son goes to is incredibly diverse (including most of the kids who went to the formerly failing elementary school), with a great vibe and dedicated teachers. Test scores can tell you when a school is broken, but it’s not useful in comparing two functional schools. How a school fosters creativity, teamwork, and curiosity are not captured in any test.
This same data can also be used in a big data analysis. Throw in demographic data from the census, housing prices from the country, income data from the IRS, alcohol and cannabis consumption data from Washington State and some subtle correlations that aren’t immediately apparent might appear.
Of course, they might just be random chance; that’s why you need to be careful with big data in a way that you don’t with little data. If your school has the lowest test scores in the state, you know you have a problem. If there’s a weak positive correlation between playing sports and grades, that doesn’t mean every child needs to immediately join a league.
It can be hard to rally the troops to fix a broken team or process. When you come in with data showing how far you are from where you’re supposed to be, it’s much easier to drive changes. That’s true whether it’s replacing a principal or fixing a broken manufacturing process.
Use Data Thoughtfully
Throughout your day, you’re inundated with data. The key to both little data and big data is to thoughtfully filter out what is unimportant and turn what is important into knowledge, which is data with context and meaning. Then you use that knowledge to inform your actions. If the data can’t lead to action, it’s worthless. An extreme example of this is Sherlock Holmes’s lack of interest in the fact that the Earth orbits the sun,
“What the deuce is it to me?” he interrupted impatiently: “you say that we go round the sun. If we went round the moon it would not make a pennyworth of difference to me or to my work.”
Though, as a former astronomer, I don’t encourage following Holmes’s example, it is important to focus our efforts on data, big or small, that help us make better decisions.
Ready for a new project management tool, but still need to get your boss’s buy-in? Learn how to build a compelling business case with this in-depth guide.
