Service Objects’ Blog

Thoughts on Data Quality and Contact Validation

Email *

Share

Posts Tagged ‘Contact Data’

GDPR Compliance: Is Your Business Ready?

If you conduct business in Europe, May 2018 will be an important date. This is when the planned introduction of the European Union’s General Data Protection Regulation (GDPR) is scheduled to take effect.

GDPR represents a sweeping set of privacy regulations that impact your use of personal data from European citizens. If you conduct business with people from Europe – whether they are your customers, employees, or job prospects – GDPR affects you as well. It will require you to have policies in place to protect people’s personal data, as well as require notification when this data has been breached. And penalties for violations will be extremely stiff, up to the greater of 20 million Euros or 4% of your gross turnover.

GDPR starts with the definition of “personal data.” This is an extremely broad net: a recent article from Software Development magazine notes that the European Commission’s guidelines include both obvious data such name, address or email, and associated data ranging from bank accounts to photos and social media posts. Even the IP address a European is using on their computer is considered part of this personal data.

Much like the HIPAA requirements on electronic health care data in the United States, GDPR will require organizations to safeguard the personal data they collect and store in the course of doing business. At one level, this will involve technology such as encrypted data storage, password protection, and other approaches, along with policies and procedures for protecting this data. At another level, it obligates you to inform European consumers about your privacy policies, gain explicit consent to collect and use their personal data and provide them with the ability to control or opt-out of data collection. And in the event personal data is compromised, you need a plan for reaching people affected by the breach.

Each of these levels have important areas where data quality and GDPR compliance efforts intersect. Some of the questions businesses will have to ask themselves include:

  • Do we have accurate contact information for people we do business with in Europe?
  • Is there a notification procedure in place for our privacy and data policies, including opting out of data collection or making changes to personal data?
  • If a breach notification were necessary, do we have the means to quickly reach all affected parties?
  • How do we handle changes to contact information? What if a person in your database moves, changes jobs, or gets a new email address?

This means that your GDPR and data quality strategies will need to be closely linked. Tools such as international address verification, lead validation and name validation can help make sure data is complete and correct as it enters your system, and stays correct when it is needed later. As a recent article in Information Management points out, the key to GDPR compliance lies in proactively analyzing your data and performing a thorough risk assessment long before an actual privacy issue arises.

The European Union has long been on the vanguard of consumer protection legislation, and the new GDPR regulations are the latest in an effort to level the playing field between big data and the individual rights of its citizens. They have a global reach, whether you do business in Europe or serve Europeans from elsewhere. At a broader level, GDPR is part of a new reality that businesses will soon need to work with, one that is part of a larger trend toward increasing privacy regulations.

May 2018 is coming soon – is your business ready?

Omnichannel Solutions and Data Quality

Just a few decades ago the concept of a “channel” didn’t exist, other than on your television. If a customer or prospect wanted to contact you, they called you or wrote you a letter. And if you wanted to contact them, you got out your Rolodex – or if you were a large enterprise, perhaps your batch mainframe computer, with disk drives the size of a washing machine.

Today, sales, marketing and customer support take place across multiple touch points that include point of sale, online orders, emails, social media inquiries – and even those same traditional phone calls and letters. Increasingly, this contact data is managed by integrated enterprise systems rather than separate vertical applications. Which also means that all of your sales and support channels often serve as pipelines to a common contact database.

Over the past five years, we have been in the midst of an omnichannel revolution in enterprise solutions. The reason is simple economics – particularly the growth of inexpensive, scalable, cloud-based software-as-a-service (SaaS) applications. Once upon a time, enterprise software seemingly required months of planning and a cast of thousands to implement. Today, even the smallest operation can license applications that integrate ALL of their customer touch points on an inexpensive per-seat basis.

The era of integrated, multi-channel applications also means that the impact of bad contact data is now greatly amplified. Here are some examples:

  • Many customer touch points are notorious for providing incomplete or incorrect contact information. This can range from the person who enters “Mickey Mouse” or a fake address to get free marketing incentives, all the way to customer support tickets with missing contact data.
  • An estimated 25% of marketing contact data is bad – and in an enterprise solutions environment, this bad data propagates across all of your sales and marketing activities, wasting time and resources.
  • Telephone numbers change constantly, and your next telemarketing campaign could find you inadvertently – and illegally – calling consumer cell phones in violation of the Telephone Consumer Protection Act (TCPA), exposing you to potential fines of up to $1500 per violation.
  • Data entry mistakes in order processing can lead to lost shipments, wasted time and human intervention, and customer dissatisfaction.
  • Identity fraud cost businesses over $18 billion in losses in 2014, and much of it could be avoided by matching IP address locations to customer orders – so, for example, your system can red-flag a big-ticket domestic order originating from an overseas computer.

The solution to issues like these is to build data quality right in to your enterprise contact data, with a little help from Service Objects. Our tools can validate, append and update addresses using continually verified data from the USPS or Canada Post. We can geocode and analyze your order data for fraud verification, tax compliance and more. We can do real-time phone number verification to help you maintain TCPA compliance. More strategically, we can do lead scoring and enhancement to turn your contact data into a revenue-generating engine. Using API and batch processing interfaces, these tools and more provide a seamless way to put your contact data quality on autopilot.

The omnichannel era is here to stay – and in the process, contact data has become a strategic asset for companies of any size. We can help you leverage the power of this asset, by making sure this data is genuine, accurate, and up-to-date. And with the right partner, you can let data quality drive a tangible difference in revenue across all of your channels.

The 2018 European Data Protection Regulation – Is Your Organization Prepared?

The General Data Protection Regulation (GDPR) is a regulation intended to strengthen and unify data protection for all individuals within the European Union (EU). It also addresses the export of personal data outside the EU. The primary objectives of the GDPR are to give citizens and residents back control of their personal data and to simplify the regulatory environment for international business by unifying the regulation within the EU.

According to research firm Gartner, Inc., this regulation will have a global impact when it goes into effect on May 25, 2018.  Gartner predicts that by the end of 2018, more than 50 percent of companies affected by the GDPR will not be in full compliance with its requirements.

To avoid being part of the 50 percent that may not be in compliance one year from now, organizations should start planning today. Gartner recommends organizations focus on five high-priority changes to help organizations to get up to speed:

    1. Determine Your Role Under the GDPR
      Any organization that decides on why and how personal data is processed is essentially a “data controller.” The GDPR applies therefore to not only businesses in the European Union, but also to all organizations outside the EU processing personal data for the offering of goods and services to the EU, or monitoring the behavior of data subjects within the EU.
    2. Appoint a Data Protection Officer
      Many organizations are required to appoint a data protection officer (DPO). This is especially important when the organization is a public body, is processing operations requiring regular and systematic monitoring, or has large-scale processing activities.
    3. Demonstrate Accountability in All Processing Activities
      Very few organizations have identified every single process where personal data is involved. Going forward, purpose limitation, data quality and data relevance should be decided on when starting a new processing activity as this will help to maintain compliance in future personal data processing activities. Organizations must demonstrate an accountable ground posture and transparency in all decisions regarding personal data processing activities. It is important to note that accountability under the GDPR requires proper data subject consent acquisition and registration. Prechecked boxes and implied consent will be largely in the past.
    4. Check Cross-Border Data Flows
      As of today, data transfers to any of the 28 EU member states, as well as 11 other countries, are still allowed, although the consequences of Brexit are still unknown. Outside of the EU, organizations processing personal data on EU residents should select the appropriate mechanism to ensure compliance with the GDPR.
    5. Prepare for Data Subjects Exercising Their Rights Data subjects have extended rights under the GDPR, including the right to be forgotten, to data portability and to be informed (e.g., in case of a data breach).

Having poor quality data has several impacts on an organization and could hinder your efforts to being in compliance. Visit Service Objects’ website to see how our global data quality solutions can help you ensure your contact data is as genuine, accurate and up-to-date as possible.

Big Data – Applied to Day to Day Life

With so much data being constantly collected, it’s easy to get lost in how all of it is applied in our real lives. Let’s take a quick look at a few examples starting with one that most of us encounter daily.

Online Forms
One of the most common and fairly simple to understand instances we come across on a daily basis is completing online forms. When we complete an online form, our contact record data points, like; name, email, phone and address, are being individually verified and corrected in real time to ensure each piece of data is genuine, accurate and up to date. Not only does this verification process help mitigate fraud for the companies but it also ensures that the submitted data is correct. The confidence in data accuracy allows for streamlined online purchases and efficient deliveries to us, the customers. Having our accurate information in the company’s data base also helps streamline customer service should there be a discrepancy with the purchase or we have follow up questions about the product. The company can easily pull up our information with any of the data points initially provided (name, email, phone, address and more) to start resolving the issue faster than ever (at least where companies are dedicated to good customer service).

For the most part we are all familiar with business scenarios like the one described above. Let’s shift to India & New Orleans for a couple new examples of how cities are applying data to improve the day-to-day lives of citizens.

Addressing the Unaddressed in India
According to the U.S. Census Bureau, India is the second most populated country in the world with 1,281,935,911 people. With such a large population there is a shortage of affordable housing in many developed cities, leading to about 37 million households residing in unofficial housing areas referred to as slums. Being “unofficial” housing areas means they are not mapped nor addressed leaving residents with very little in terms of identification. However, the Community Foundation of Ireland (a Dublin based non-profit organization) and the Hope Foundation recently began working together to provide each home for Kolkata’s Chetla slum their very first form of address consisting of a nine-digit unique ID. Beside overcoming obvious challenges like giving someone directions to their home and being able to finally receive mail, the implementation of addresses has given residents the ability to open bank accounts and access social benefits. Having addresses has also helped officials identify the needs in a slum, including healthcare and education.

Smoke Detectors in New Orleans
A recent article, The Rise of the Smart City, from The Wall Street Journal highlights how cities closer to home have started using data to bring about city wide enhancements. New Orleans, in particular, is ensuring that high risk properties are provided smoke detectors. Although the fire department has been distributing smoke detectors for years, residents were required to request them. To change this, the city’s Office of Performance and Accountability, used Census Bureau surveys and other data along with advanced machine-learning techniques to create a map for the fire department that better targets areas more susceptible to deaths caused by fire. With the application of big data, more homes are being supplied with smoke detectors increasing safety for entire neighbors and the city as a whole.

FIRE RISK | By combining census with additional data points, New Orleans mapped the combined risk of missing smoke alarms and fire deaths, helping officials target distribution of smoke detectors. PHOTO: CITY OF NEW ORLEANS/OPA

While these are merely a few examples of how data is applied to our day to day lives around the world, I hope they helped make “Big Data” a bit more relatable. Let us know if we can answer any questions about how data solutions can be applied to help your company as well.

From Hello Operator to Hey Siri – Accurate Contact Data Has Always Been Crucial

Fueled by our desire to communicate with one another, no matter distance, the telephone has undergone extraordinary technological enhancements since the first test call on March 10, 1876. Today, the average wireless phone even functions as a portable computer offering a multitude of ways to communicate. Although phone technology dramatically changed over the last 141 years and continues to change, one aspect of placing a call remains vitally important: accurate contact data.

Originally, the telephone was sold in pairs of two with a single connection to each other. Since these early telephones were directly connected to each other, phone numbers were not yet required. However, with the invention of the switchboard in 1878, callers could connect with many other subscribers leading to the establishment of phone numbers consisting of a few digits. By 1910 the U.S. population grew to 92,228,496, over seven million of whom were phone subscribers. To accommodate so many users the length of the phone number increased.  For the majority of the 1900s, whether using a candlestick, rotary or push button phone, the telephone operator manually connected callers by switchboard and without accurate contact information to start with callers could not be properly connected. As the pool of subscribers grew further, alphanumeric numbers were introduced and used through the 1960s. This format consisted of two letters representative of location (name of the village, town or city) of the central office that the phone was connected to, followed by numbers.  Although fewer miscommunication between callers and operators occurred with the use of alphanumeric numbers, having accurate information to begin with was still imperative.

Jumping forward to today, various devices ranging from wireless phones, computers, tablets, and even televisions can be used to place calls. Somewhat reminiscent of telephone operators, virtual assistants like Apple’s Siri and Amazon’s Alexa can even be used to connect to someone by dictation which is how a four year old boy recently contacted emergency services to save his mother’s life. Although a phone number is still required for most devices, platforms such as Skype and FaceTime also use email address as unique identifiers to connect callers. While new types of contact information like email are being used more commonly, once the information is entered into the calling device you don’t need to remember it again. With just a few taps on a screen or a simple phrase, “hey Siri, call mom,” the call is initiated.

Whether placing a call now or 141 years ago starting with genuine, correct and up to date contact data is essential for reaching each other by phone. As forms of contact data continue to evolve with technology, our validation tools will as well to ensure your business communications are as fast and easy as possible.

Peer-to-Peer: The Next Frontier

How do you get millennials interested in a cause?

For starters, you don’t use traditional direct marketing techniques. Millennials won’t even answer a call from an unknown phone number more than 95 percent of the time. Email is something their grandparents used to use, with conversion rates hovering around 1 percent. And many of them don’t sit in front of a television every night passively watching advertising – they live within a broad web of individual human connections, fueled by smartphones and social media.

These are the kinds of numbers that motivated Bay Area startup Hustle (hustle.life) to create a new paradigm: large-scale peer-to-peer communications via text messaging.

The Hustle platform is an enabling technology that allows text messages to be sent rapidly to people’s phones, using automated templates that can be personalized for each message. While still requiring human intervention to send messages, it dramatically increases the productivity of organizations trying to reach large amounts of people for an event, cause or campaign – and these people can text back and get responses from a real human being. The result is often a response rate in the 30-40% range.

As a result, Hustle has now attracted substantial venture funding, and its product was used to reach nearly 4 million people during the latest election season. More important, the concept of mass communication between individuals is now attracting a great deal of attention.

Of course, peer-to-peer communications are much more than a marketing technique. They are quickly becoming a revolution. You can see it in action when you use Uber to get a ride from a private car owner, or AirBnB to rent someone’s house for a week. Uber owns no vehicles, and AirBnB owns no real estate, but both companies connect people to other people on a massive scale. And in the future, respected prognosticators like Daniel Burrus and Donald Tapscott predict the same paradigm will transform banking, voting, education and many other industries that fuel our daily life.

So how can you prepare for the peer-to-peer revolution? By having better access to these peers. When you are blasting text messages to thousands of people, these numbers need to be correct. Otherwise, you face unintended consequences ranging from intrusive spamming to wasted human effort. Moreover, as you move from organizing to marketing, any one-to-one contact model needs verification tools to assess the legitimacy of your contacts and prevent fraud and waste.

Thankfully effective tools existing for verifying phone contact information. These tools include reverse lookup capabilities that can verify wireless or other numbers against US and Canadian databases, including geocoded carrier information and phone type. You can also detect numbers such as VoIP or prepaid phones for use in lead validation or fraud prevention. Taking things a step further, qualified phone numbers can have other contact information appended to them, and entered phone numbers can be contacted via phone or text for active verification by the customer.

The world is increasingly moving away from centralized market models to a distributed peer-to-peer marketplace. This means that now, more than ever, the data quality of both your contact database and your inbound contacts are emerging as key business drivers for the future. With a small incremental investment in maintaining this quality, you can be prepared to grow in an increasingly interconnected world.

People, Process, and Technology: The Three Pillars of Data Quality

For many people, managing data quality seems like a daunting task. They may realize that it is an important issue with financial consequences for their organization, but they don’t know how to proceed in managing it. With the right strategy, however, any organization can reap the benefits of consistent data quality, by focusing on three core principles: People, Process, and Technology.

Taken together, these three areas serve as the cornerstones of a structured approach to data quality that you can implement and manage. And more important, a framework that lets you track the ROI of successful data quality. Let’s look at each of these in detail:

People

This is frankly where most organizations fail at the data quality game: not allocating dedicated gatekeepers for the health of their data. It is a very easy mistake to make when budgets are tight, resources are focused on revenue-generating functions like sales or product development, and the business case for data quality gets lost amidst a host of competing priorities.

The single biggest thing an organization can do for data quality is to devote dedicated resources to it. This becomes an easier sell once you look at the real costs of bad data: for example, research shows that 25% of all contact records contain bad data, a third of marketing leads use fake names, and half of all phone numbers provided won’t connect. Run these numbers across the direct costs of customer acquisition, add in missed sales opportunities, increased customer care costs, and even potential compliance fines, and you often have the financial justification for a data quality gatekeeper.

Process

How much control do you have over data entry points, data accuracy, and verification? For too many organizations, the answer is none – with resulting costs due to factors such as duplicate data entry, human error, or lack of verification. And who is responsible for maintaining the integrity of your business data? Too often, the answer is “no one,” in a world where data rarely ages well. An average of 70% of contact data goes bad in some form each year, which ushers in yet another level of direct and indirect costs.

One of the more important roles of a data gatekeeper is to have processes in place to manage the touch points for your data, engineer data quality in on the front end of customer and lead acquisition, and maintain this data over the course of its life cycle. Having the right policies and procedures in place gives you control over your data, and can make the mechanics of data quality frictionless and cost-effective. Or as your teachers used to put it, an ounce of prevention is worth a pound of cure.

Technology

Data quality solutions range from simply scanning spreadsheets for duplicates and mistakes, all the way to automated tools for tasks such as address validation, lead validation, and verification of email or phone contact information. And far too often, the solution of choice for an organization is to do nothing at all.

Ironically, using the best available automated tools for data quality is often a surprisingly cost-effective strategy, and can yield your best ROI. Automated tools can be as simple as verifying an address, or as sophisticated as creating a statistical ranking value for the quality of a lead or contact record. Used properly, these tools can put much of the hard work of data quality on autopilot for you and your organization.

Ensuring your organization’s data quality can seem like an overwhelming task. But broken into its component parts – your people, your process, and your technology – this task can turn into logical steps that pay themselves back very quickly. It is a simple and profitable three-step strategy for any organization that runs on data.

How Much Is Bad Contact Data Costing Your Organization?

Collecting visitor and customer data through a variety of channels allows you to quickly grow your contact list. When contact information comes in, your company is provided many new opportunities to expand your business. However, receiving high-quality data isn’t necessarily a given. After all, some visitors will enter bad contact data, such as bogus phone numbers, in an attempt to limit telemarketing calls; others may think they’re being funny by entering fake names such as Donald Duck or Homer Simpson; others might accidentally misspell a street name. Even autocorrect may change a legitimate entry, and some visitors may intentionally enter a bad address in an attempt to commit fraud.

Managing the effects of bad contact data is a surprisingly large cost for many organizations, involving a great deal of human intervention as well as diluted contact effectiveness. This is one area where an ounce of prevention is worth a pound of cure – particularly in today’s era of automated contact verification tools. Whatever your touch points are with customers or prospects – including customer data, contest entry forms, phone surveys, lead generation and point of sale interactions – some data will be inaccurate, incomplete, or fraudulent.

Why Data Quality Matters

Do you really need to worry about a few Donald Ducks or Homer Simpsons in your contact database? Yes, you do! Poor data quality translates into wasted time, resources, and money. For example, if you have a mailing list of 10,000 addresses, and 10 percent of those are inaccurate, you will waste 1,000 pieces of mail — plus the cost of product and postage – not to mention the human resources involved in processing incorrect outgoing and returned mail.

Data quality from a customer service perspective is another big concern. Suppose that a customer orders an expensive product from your website, and accidentally enters their address as “2134 Main Street” instead of the correct address of “1234 Main Street”? First of all, you would ship the package to the wrong address. Not your fault, but it doesn’t matter: delivery will be delayed, the customer will have a poor experience, and you will incur re-shipping costs. You may even not get the original shipment back. It’s a lose-lose situation.

The problem is compounded when it comes to getting these customers in the first place: an estimated 25% percent of marketing contact data is bad. And according to the Data Warehouse Institute, this ocean of bad data costs businesses over US $600 billion per year. At the level of the individual company, this means that over a quarter of your sales and marketing resources are lost to bad prospects, for reasons that range from intentional fake contact data to the natural contact record aging process. In fact, Salesforce.com reports that after just one year, nearly 70% of contact data goes bad in some form as people change jobs, phone numbers and email addresses.

So how can your company avoid the challenges associated with bad data? Start by assessing what areas of your company need better data quality control. Whether you identify a single area or several areas that could benefit from improved data quality, realigning your data quality goals to hit the data trifecta can improve your bottom line, optimize your human capital and even help the environment.

To read more about this topic, download our white paper Hitting the Data Trifecta – Three Secrets of Achieving Data Quality Excellence.

Name Deduplication Techniques

Identifying Duplicate Records

The bane of any Database Administrator is maintaining duplicate records. They take up unnecessary space and generally do not provide any added value to contact records. A more challenging task for Database Administrators is how to identify and merge records which might be duplicates, and in particular, duplicate names.

There may be variants for a given name which might not be easily identified in a query, but they are invariantly linked. A common example might be Joe Smith vs Joseph Smith. Both could be referring to the same person depending on how the user may have entered their name.

Name Variants, Finding the Common Name

A particularly useful feature of the Name Validation 2 service is the Related Names output field. This field provides a comma separated list of first name variants for a provided name. For example using the given name; Joe, related names returned include Joel, Joeseph, Joey, Josef, Joseph, and José.

With this information, it becomes easier to identify names which are related but in a different form. There may be cases, however, where names cannot be identified as related but can be linked from similarity. Some examples include names that are misspelled or alternate names which are not related but similar. These names can still be identified through the Similar Names output fields of the Name Validation 2 service.

Similar Sounding Names

DOTS Name Validation 2 employs sophisticated similar name matching algorithms to match names drawing from a database of international names with up to 1.4 million first names and 2.75 million last names. First and last name similar results are returned in a comma separated list which can be used to compare against names that already exist in the database.

An example similar name result for the given name; Robert Smith, would return similar first names Rhobert,Róbert,Robertt,Roebert,Roibert,Rubert,Robbert, and similar last names Smyth,Smithe,Smiith,Smiyth. Of the similar names that are found, names are returned in order of most common to least common.

Merge and Promote the Winning Record

Using these results, a query can potentially link similar or related names and identify records which are duplicates. Once duplicate records are identified, the question becomes which should be promoted as the winning record? This decision can depend on factors based on business logic, perhaps a record which contains other vital contact points such as address or phone number or perhaps entry date is chosen as the winning record. Once a winning record is chosen, a merge process is incorporated to merge contact fields from identified duplicates to build a complete record.

Conclusion

Ridding your database of duplicate contact records can be an arduous task, but with the help of Name Validation 2, it doesn’t have to be. Leveraging the vast quantity of names that Name Validation 2 draws upon yields a top quality solution to identifying duplicates through related and similar names.

For more information about Name Validation 2 service, or to receive a free trial key, click here.

For developers, our Name Validation 2 documentation can be found here.

Service Objects is the industry leader in real-time contact validation services.

Service Objects has verified over 2.8 billion contact records for clients from various industries including retail, technology, government, communications, leisure, utilities, and finance. Since 2001, thousands of businesses and developers have used our APIs to validate transactions to reduce fraud, increase conversions, and enhance incoming leads, Web orders, and customer lists. READ MORE