A Complete Introduction to Data Collection

Rivers have been essential to human communities since the dawn of time. Daily life depends on water—drinking, washing, fishing, irrigation—while ships enable transportation and commerce. Problems at a river’s source spell disaster for those downstream—think contaminated drinking water or floods caused by heavy rainfall upstream.

Data plays a similarly vital role in modern organizations. It flows through dozens of tools and fuels everything from executive decisions to marketing personalization. Contamination and floods upstream at the data’s source have grave consequences downstream for departments depending on that information.

To avoid problems with the purity and quality of your data, you need to master the practice of data collection by understanding these subjects:

• What is data collection?

• First-party vs. second-party vs. third-party data collection

• Quantitative vs. qualitative data collection

• 5 common methods of data collection

• Collecting and managing customer data doesn't need to be a pain point

What is data collection?

Data collection is taking information from a source, usually to gain insights into a specific topic. Data collection traditionally referred to field and market research through surveys, focus groups, and interviews. Over the past two decades, the term has also come to include the automated collection of digital data, mainly on the internet and through apps and devices.

High accuracy is data collection’s equivalent to a crystal-clear mountain spring. Conversely, flawed—meaning inaccurate, corrupted, or incomplete—information contaminates every initiative you undertake with that data downstream.

Human errors, such as mistakes in manual processing or bias in observations, are the primary sources of contamination in field and market research. Meanwhile, gathering too much data without a clear purpose or from unknown sources causes data quality issues when collecting digital information.

First-party vs. second-party vs. third-party data collection

The difference between first-, second-, and third-party data is its source.

• First-party data—sometimes also called “primary data”—is collected directly by your company.

• Second-party data is collected by another company and shared with, or sold to, a non-competitive partner.

• Third-party data is collected by a data-collection company and then shared with anyone who wants to purchase it.

As a general rule, the closer you are to the data’s collection, the higher its quality.

Source

As time passes between collecting and using data, the chance of that information becoming outdated increases. Since you can act on your own data quickly, first-party data always has a higher chance of being relevant and correct than second- or third-party data.

Third-party data is also on the way out because browsers like Chrome and government regulations such as the GDPR are limiting the use of cookies and other third-party information. For these two reasons, first-party data is most desirable for your data collection efforts: it’s highly accurate and compliant with the latest privacy regulations.

Quantitative vs. qualitative data collection

Quantitative data—sometimes called “quant” data—primarily consists of numbers and lends itself well to automated, digital analysis. Qualitative or “qual” data deals in words and observations, usually requiring manual interpretation and processing.

Quantitative data

Most of the information modern businesses automatically collect is quant data. Examples are website visitor numbers, conversion rates, and transactions records. Collecting, storing, and analyzing even large volumes of such information is relatively cheap and straightforward these days, which is why most businesses rely on quantitative data.

Collecting quant data automatically also poses several challenges. The information can:

• Turn out to be useless or irrelevant when you collect data without a clear purpose or use case.

• Increase privacy and security risks when you don’t control or know what data you collect. Such ignorance can lead to compliance problems. And the more data you store, the more tempting of a target you make for hackers.

• Lack context and nuance because you can see what someone is doing, but not why nor how they felt during that activity.

• Exist in silos when departments store data—sometimes the same data—in different locations and formats.

You can avoid these issues by creating a data tracking plan that clarifies what events you’ll track and the methods you’ll use to do so. Such plans cover details like where events need to go in the underlying databases and why their collection is necessary for your business. Check out Segment’s tracking plan if you want to see an example.

Source

Segment can also solve some of the above challenges through its Privacy Portal and by automatically enforcing your data tracking plan with Protocols. Both these features allow you to classify and, if necessary, prevent data from being collected at the source.

Qualitative data

Qual data capture people’s emotions and motivations—a nuance that quant data doesn’t reveal. As this Wall Street Journal article on focus groups puts it: “Big data may help you recognize something is happening, but it cannot tell you the all-important why.”

But collecting qual data costs much more—in money and effort—than quant data. Focus groups, interviews, and field research require people, travel, and facilities. A single focus group can cost from $5,000–$9,000. Other types of qualitative research can run even higher.

Technology is making qual data collection more accessible, however. Running a virtual focus group or interview doesn’t give you the same observations as the in-person version. Still, it’s much cheaper, easier, and faster to organize. And combining quant data with even a handful of interviews can give you valuable insights you’d never get from looking at the numbers alone. As Segment CEO Peter Reinhardt wrote on his blog about Segment’s early days: “20 hours of great interviews probably would’ve saved us an accrued 18 months of building useless stuff.”

5 common methods of data collection

The number of methods to gather information has increased with the rise of automated, digital data collection. Here’s a list of the most common data collection techniques.

1. Surveys and polls

Surveys contain multiple questions, polls, just one—usually multiple-choice. Both methods are suitable for gathering information from many people, especially now that you can send out surveys and polls digitally at little-to-no cost.

When surveys consist of multiple-choice questions, you can process and analyze responses quickly. At the same time, surveys can also include open-ended questions, allowing you to collect some qualitative information.

A challenge with surveys and polls is that you can nudge respondents into a particular direction—on purpose or by accident—by framing leading questions. Harvard Business School gives this example of such a question: “Our product reduces your tension by 10 percent. Would you like to buy it?” By including a product benefit, the respondent is more likely to answer “yes.” A more neutral phrasing like, “How likely are you to buy this product?” would remove such nudging.

Segment can automatically synchronize data from survey tools like SurveyMonkey and Typeform, then synthesize it with other customer data you already have.

2. Interviews and focus groups

Interviews and focus groups are the staples of traditional research. Both involve real-time interactions between one or more participants and an interviewer or facilitator.

These approaches can provide more data and insights per participant than other forms of research. You can record people’s responses, observe their body language and behavior, and see how participants in a focus group interact. You’re also able to ask follow-up questions, something that’s much more difficult with other data collection methods like surveys.

The drawback of interviews and focus groups is that they don’t scale. Even if you have the money to organize many, there’s still a limit to the number of sessions you can hold. You’ll need to recruit and coordinate facilitators, participants, and sometimes locations. Each session also leads to lots of data that you’ll need to process—often manually—to gain insights and share those with others.

3. Behavioral data collection

Behavioral data tells how someone acts within a specific context. In business, this means any information that shows how a customer interacts with your organization. Most behavioral data is gathered automatically through trackers that capture customer data on websites, devices, and apps. You can also collect behavioral data offline or manually, for example, when a person counts in-store foot traffic or analyzes transcripts from customer support inquiries for insights.

Digital behavioral data on a website or app can include page views, clicks, ad impressions, mobile swipes, and heat maps that show someone’s exact interactions with a website or app. Marketers also use data from third-party keyword research tools for behavioral insights into what prospective customers are searching for.

Most behavioral data sources are quantitative. As such, this information alone can only confirm that someone took a particular action but not provide insights into why they did so. For this reason, you should always consider complementing behavioral data with qualitative data collection methods.

Segment can connect out of the box to most tools that collect digital behavioral data, for example, Google Analytics, Mixpanel, Amplitude, and Hotjar.

Source

4. Social media monitoring

Social media can sometimes give clues to people’s behavior but are primarily used to assess sentiment in a business context, especially towards brands and products. You can gauge people’s feelings about your company or product by analyzing the language they use in social posts. You can also discover emerging—or declining—broader societal trends from such analysis.

Tools like Hootsuite and Brandwatch can collect social media data for you and do the initial analysis. These tools allow you to gather insights from many people quickly and relatively cheaply.

Be aware that experts contest the accuracy of automated sentiment interpretation as algorithms still struggle to understand nuances in human language. Data scientist Parul Pandey gives this example of a product review, which a sentiment bot might label as positive: “This is the best laptop bag ever. It is so good that within two months of use, it is worthy of being used as a grocery bag.”

The Institute for PR puts the accuracy of automated sentiment analysis at just 50%. HIPSTO, a natural language processing platform, claims to have achieved 94% accuracy. Whatever the actual percentage is, social media monitoring can only give clues you’ll need to investigate further with other data collection methods.

5. Transactional data collection

Transactional data is information recorded during a purchase or other value exchange between a business and a customer. Examples of such data are payment records, subscription and registration information, shipping documents, and insurance claims.

Transactional data can provide insights into your customers’ preferences or behaviors, such as their favorite retail location or payment methods—assuming local regulations allow you to process such information. Transactional data can also surface operational issues, say when payments suddenly grind to a halt at your website or a store, signaling a possible technical problem.

Transactional data’s limitation is that it’s a quantitative snapshot of a single moment. It tells you what someone did at what time—and perhaps who that someone is. But from transactional data alone, you usually don’t know the journey to that point, why the transaction was made, nor what happened afterward. For such insights, you’ll have to complement transactional data with information collected from other sources.

Segment connects with Stripe—a payments processor— and security and fraud monitoring tools like Castle and TrafficGuard.

Collecting and managing customer data doesn’t need to be a pain point

A Customer Data Platform (CDP) like Segment simplifies your data collection while simultaneously ensuring the information you gather is of the highest possible quality. Its features act like filters, canals, and, where necessary, dams, guaranteeing you only take in data you need and deliver it to its destination in your organization as fast as possible.

Protocols turn your data tracking plan into automated action. This feature enforces your data standards across the organization, no matter how many people or data points you employ. Protocols lets you review and handle issues quickly and easily through its data validation dashboard if they do turn up.

Source

Segment’s Connections works with hundreds of tools out of the box, both for data collection and data usage downstream. Without much engineering, you can capture data from sources like mobile apps, websites, marketing platforms, and even point of sale systems. Once collected, Segment standardizes your data, creates profiles of your customers, and can send this information to destinations like analytics and advertising tools with just a few clicks. And, if you use an app or service that’s not supported, you can easily create a custom connection or destination using Segment’s API.

Preventing floods and keeping rivers clean are mighty tasks for governments and communities. Doing the same for your data should be more straightforward now that you know the essentials of data collection and Segment’s features.