Service Objects’ Blog

Thoughts on Data Quality and Contact Validation

Email *

Share

Posts Tagged ‘Data Cleansing’

New CRM or ERP? Reduce Your Migration Risk

Birds and data have one thing in common: migration is one of the biggest dangers they face. In the case of our feathered friends, their annual migration subjects them to risks ranging from exhaustion to unfamiliar predators. In the case of your data, moving it to a new CRM or ERP system carries serious risks as well. But with the right steps, you can mitigate these risks, and preserve the asset value of your contact database as it moves to a new system.

In general, there are two key flavors of data migration, each with their own unique challenges:

The Big Bang Approach. This involves conducting data migration within a small, defined processing window during a period when employees are not actively using the system – for example, over a long weekend or holiday break.

This approach sounds appealing for many sites, because it is the quickest way to complete the data migration process. However, its biggest challenge involves data verification and sign-off. Businesses seldom conduct a dry run before going live with migration, resulting in the quality of migrated data often being compromised.

One particular issue is the interface between a new enterprise system and internal corporate systems. According to TechRepublic, enterprise software vendors still suffer from a lack of standardization across their APIs, with the result that every integration requires at least some custom configuration, leading to concerns about both data integrity and follow-on maintenance.

The Trickle Approach. Done with real-time processes, this approach is where old and new data systems run in parallel and are migrated in phases. Its key advantage is that this method requires zero downtime.

The biggest challenge with this approach revolves around what happens when data changes, and how to track and maintain these changes across two systems. When changes occur, they must be re-migrated between the two systems, particularly if both systems are in use. This means that it is imperative for the process to be overseen by an operator from start to finish, around the clock.

Beyond these two strategies, there is the question of metadata-driven migration versus content-driven migration – another major hurdle in the quest to migrate genuine, accurate, and up to date data. IT might be more focused on the location of the source and the characteristics of each column, whereas marketing depends upon the accuracy of the content within each field. According to Oracle, this often leads to content that does not match up with its description, and underscores the need for close inter-departmental coordination.

Above all, it is critical that a data validation and verification system be in place before moving forward with or signing-off on any data migration process. The common denominator here is that you must conduct data validation and verification BEFORE, DURING, and AFTER the migration process. This is where Service Objects comes into play.

Service Objects offers a complete suite of validation solutions that provide real-time data synchronization and verification, running behind the scenes and keeping your data genuine, accurate, and up to date. These tools include:

One particular capability that is useful for data migration is our Address Detective service, which uses fuzzy logic to fill in the gaps of missing address data in your contact records, validates the result against current USPS data, and returns a confidence score – perfect for cleaning contact records that may have been modified or lost field data during the migration process.

Taking steps to validate all data sources will save your company time and extra money. With Service Objects data validation services, we’ll help you avoid the costs associated with running manual verifications, retesting, and re-migration. And then, like the birds, it will be much easier for you and your data to fly through a major migration effort.

Best Practices for List Processing

List processing is one of the many options Service Objects offers for validating your data. This option is ideal for validating large sets of existing data when you’d rather not set up an API call or would simply prefer us to process the data quickly and securely. There is good reason to have us process your list: we have high standards for security and will treat a file with the utmost care.

As part of our list processing service, we offer PGP encryption for files, SFTP file transfers, and encryption to keep your data private and secure. We also have internal applications that allow us to process large lists of data quickly and easily. We have processed lists ranging from tens of thousands of records to upwards of 15 million records. Simply put, we consider ourselves experts at processing lists, and we’ll help ensure that your data gets the best possible return available from our services.

That said, a few steps can help guarantee that your data is processed efficiently. For the best list processing experience – and the best data available, we recommend following these best practices for list processing.

CSV Preparation

Our system processes CSV files. We will convert any file to the CSV format prior to list processing. If you want to deliver a CSV file to us directly, keep the following CSV preparation best practices in mind:

Processing international data – If you have a list of international data that needs to be processed, make sure the file has the right encoding. For example, if the original set of data is in an Excel spreadsheet, converting it to a CSV format can destroy foreign characters that may be in your file. When processing a list of US addresses, this may not be an issue but if you are processing an International set of addresses through our DOTS Address Validation International service, then something like this could highly impact your file. One workaround is to save the file as Unicode text through Excel and then set the encoding to UTF-8 with BOM through a text editor. Another option is to send us the Excel file with the foreign characters preserved and we will convert it to CSV with the proper encoding.

Preventing commas from creating unwanted columns – Encapsulating a field containing commas inside quotation marks will prevent any stray commas from offsetting the columns in your CSV file. This ensures that the right data is processed when our applications parse through the CSV file.

Use Multiple Files for Large Lists

When processing a list with multiple millions of records, breaking the file into multiple files of about 1 million records each helps our system more easily process the list while also allowing for a faster review of the results.

Including a unique ID for each of the records in your list helps when updating your business application with the validated data.

Configure the Inputs for the Service of Choice

Matching your input data to ours can speed up list processing time. For example, some lists parse address line 1 data into separate fields (i.e., 123 N Main St W would have separate columns for 123, N, Main, St, and W). DOTS Address Validation 3 currently has inputs for BusinessName, Address1, Address2, City, State and Zip.  While we can certainly manipulate the data as needed, preformatting the data for our validation service can improve both list processing time and the turnaround time for updating your system with freshly validated data.

These best practices will help ensure a fast and smooth list processing experience. If you have a file you need cleansed, validated or enhanced, feel free to upload it here.

The Letter that Continues to Arrive

Before moving to my current home, making sure I completed a change of address form with the Post Office was on the top of my “to do” list.  Although most mail received these days is typically coupons and business advertisements, I looked forward to receiving the first piece of mail with my name and new address on the envelope. What can I say… I appreciate the little things in life.

Well, the first time I checked the mail I found a letter addressed to the prior resident. As I had recently filled out my own change of address form at the post office I understood it would take some time for each other’s information to be updated and anticipated this would continue happening for a bit. As expected, I began receiving mail addressed with my name soon after. However, years later I’m still getting the same letter from one particular storage company for the prior resident.

Cost of Just One Letter 3

At first, I tried writing “Not at This Address,” “Moved, Left No Forwarding Address” and “Return to Sender” on the letters. After a couple months I realized this did not work. The next thing I tried was calling the storage company. I thought the human element of speaking to someone over the phone and explaining the situation would resolve the case of this never ending letter. This also did not work and actually seemed to make it worse.

As I mentioned previously, the bulk of my mail (like many other people) consists of coupons and advertisements addressed to “current resident” which are seemingly impossible to stop. Along with these, the never ending letter from this storage center started taking the excitement out of checking my mail. For a few years, checking the mail monthly instead of every few days became the routine. Every month, my mailbox was filled to max capacity with mainly junk and of course a letter (or two or three) from the storage center. Unfortunately there are some draw backs to checking your mail so infrequently. I eventually learned that if the mail does not fit in your box it is sent back to the post office which is how I missed a wedding invitation and a few birthday cards. Needless to say I went back to checking my mail more frequently and simply continued sending back the storage company letter hoping they’d eventually run their customer database through a National Change of Address (NCOA) service.

While this situation was obviously annoying, I also wondered how much this letter alone must be costing the storage center. At this point, I estimate receiving about 100 copies of the same letter equating to:

  • $46 in just postage, each has a $0.46 First-Class stamp
  • 100 wasted envelopes
  • 100 wasted pieces of paper
  • Ink for each letter
  • Wasted time/salary of the person(s) at the storage center responsible for mailing
  • Wasted time for the mail sorter(s)
  • Wasted gas and time of the mail carrier(s)
  • A big Headache for me over the last few years
  • Possible frustration for the last tenant who still hasn’t received this letter (I’m assuming it’s a bill which is even worse if they are incurring additional costs all this time)

Ultimately, this also damaged the reputation of the storage company. This mail discrepancy gave me a glimpse into their lack of customer service, organization and concern for our environment. By simply implementing an address validation check in their processes this entire scenario could be avoided. What’s worse is imagining how many other letters they are sending to the wrong address.

After further research, I found out anyone can submit a change of address form at the post office for prior tenants by making a note that they did not provide a forwarding address (the online form requires a forwarding address to submit). I’ll be heading to the post office today to fill one out. If that doesn’t resolve this, I also learned storage centers eventually auction off your items if you don’t pay your bills. Although I don’t want the prior tenant to lose their personal items, I’ll be glad to stop receiving these notices.

Until then…

If your business needs help avoiding unnecessary costs, resources and headaches associated with outdated customer information including name, address, phone, email and more contact us!

Service Objects Lands on CIOReview’s Top 20 Most Promising API Solutions

Service Objects is very proud to have been recently selected as one of CIOReview’s Top 20 Most Promising API Solution Providers for 2016, judged by a distinguished panel comprised of CEOs, CIOs and VPs of IT, including CIOReview’s editorial board.

Now if you are reading this, you probably have one of two reactions: “Wow, that’s cool!” Or perhaps, “What’s an API?”

If it is the latter, allow us to explain. An API, short for an Application Programming Interface, is code that allows our data validation capabilities to be built into your software. Which means that applications ranging from marketing automation packages to CRM systems can reach into our extensive network of contact validation databases and logic, without ever leaving the application.

What this means for them is seamless integration, real time results and better data quality. Their databases have correct, validated addresses. Their leads are scored for quality, so they are mailing to real people instead of “Howdy Doody.” Their orders are scanned for potential fraud, ranging from BIN validation on credit cards to geolocation for IP addresses, so that you know when an order for someone in Utah is originating in Uzbekistan.

What this means for you is that the applications you use are powered by the hundreds of authoritative data sources available through Service Objects – even if you never see it. Of course, we have many other ways to use our products, including real-time validation of lists using our PC-based DataTumbler application, batch FTP processing of lists, and even the ability to quickly look up specific addresses via the Web. But we are proud of our history of providing world-class data validation tools to application developers and systems integrators.

Now, if APIs are old hat to you, this award represents something important to you too: it recognizes our track record within the developer community of providing SaaS tools with superior commercial applicability, data security, uptime and technical support. As a companion article in CIOReview points out, “Service Objects is the only company to combine the freshest USPS address data with exclusive phone and demographic data. Continuous expansion of their authoritative data sets allows Service Objects to validate billions of addresses and phone numbers from around the world, making their information exceptionally accurate and complete.”

There is much more coming in the future, for systems integrators and end users alike. Our CEO Geoff Grow shared with CIOReview that one key focus is “more international data, as many of our clients are doing business outside the United States and Canada … The European and Asian markets are becoming increasingly important places (and) it is important for us to expand our product offerings and our expertise in more regions of the world.” And of course, our product offerings continue to grow and expand for clients in each of the markets we serve.

If you are a developer, we make it easy to put the power of Service Objects’ data validation capabilities in your own applications. Visit our website for complete documentation and sample code, or download a free trial API key for one of our 25 data quality solutions. We know you will see why our peers rank us as one of the best in the industry!

Data Monetization: Leveraging Your Data as an Asset

Everyone knows that Michael Dell built a giant computer business from scratch in a college dorm room. Less well known is how he got started: by selling newspaper subscriptions in his hometown of Houston.

You see, most newspaper salespeople took lists of prospects and started cold-calling them. Most weren’t interested. In his biography, Dell describes using a different strategy: he found out who had recently married or purchased a house from public records – both groups that were much more likely to want new newspaper subscriptions – and pitched to them. He was so successful that he eventually surprised his parents by driving off to college in a new BMW.

This is an example of data monetization – the use of data as a revenue source to improve your bottom line. Dell used an example of indirect data monetization, where data makes your sales process or other operations more effective. There is also direct data monetization, where you profit directly from the sale of your data, or the intelligence attached to it.

Data monetization has become big business nowadays. According to PWC consulting firm Strategy&, the market for commercializing data is projected to grow to US $300 billion annually in the financial services sector alone, while business intelligence analyst Jeff Morris predicts a US $5 billion-plus market for retail data analytics by 2020. Even Michael Dell, clearly remembering his newspaper-selling days, is now predicting that data analytics will be the next trillion-dollar market.

This growth market is clearly being driven by massive growth in data sources themselves, ranging from social media to the Internet of Things (IoT) – there is now income and insight to be gained out of everything from Facebook posts to remote sensing devices. But for most businesses, the first and easiest source of data monetization lies in their contact and CRM data.

Understanding the behaviors and preferences of customers, prospects and stakeholders is the key to indirect data monetization (such as targeted offers and better response rates), and sometimes direct data monetization (such as selling contact lists or analytical insight). In both cases, your success lives or dies on data quality. Here’s why:

  • Bad data makes your insights worthless. For example, if you are analyzing the purchasing behavior of your prospects, and many of them entered false names or contact information to obtain free information, then what “Donald Duck” does may have little bearing on data from qualified purchasers.
  • The reputational cost of inaccurate data goes up substantially when you attempt to monetize it – for example, imagine sending offers of repeat business to new prospects, or vice-versa.
  • As big data gets bigger, the human and financial costs of responding to inaccurate information rise proportionately.

Information Builders CIO Rado Kotorov puts it very succinctly: “Data monetization projects can only be successful if the data at hand is cleansed and ready for analysis.” This underscores the importance of using inexpensive, automated data verification and validation tools as part of your system. With the right partner, data monetization can become an important part of both your revenue stream and your brand – as you become known as a business that gives more customers what they want, more often.

How Much Is Bad Contact Data Costing Your Organization?

Businessman trying to figure some numbers out leaning on the table

Collecting visitor and customer data through a variety of channels allows you to quickly grow your contact list. When contact information comes in, your company is provided many new opportunities to expand your business. However, receiving high-quality data isn’t necessarily a given. After all, some visitors will enter bad contact data, such as bogus phone numbers, in an attempt to limit telemarketing calls; others may think they’re being funny by entering fake names such as Donald Duck or Homer Simpson; others might accidentally misspell a street name. Even autocorrect may change a legitimate entry, and some visitors may intentionally enter a bad address in an attempt to commit fraud.

Managing the effects of bad contact data is a surprisingly large cost for many organizations, involving a great deal of human intervention as well as diluted contact effectiveness. This is one area where an ounce of prevention is worth a pound of cure – particularly in today’s era of automated contact verification tools. Whatever your touch points are with customers or prospects – including customer data, contest entry forms, phone surveys, lead generation and point of sale interactions – some data will be inaccurate, incomplete, or fraudulent.

Why Data Quality Matters

Do you really need to worry about a few Donald Ducks or Homer Simpsons in your contact database? Yes, you do! Poor data quality translates into wasted time, resources, and money. For example, if you have a mailing list of 10,000 addresses, and 10 percent of those are inaccurate, you will waste 1,000 pieces of mail — plus the cost of product and postage – not to mention the human resources involved in processing incorrect outgoing and returned mail.

Data quality from a customer service perspective is another big concern. Suppose that a customer orders an expensive product from your website, and accidentally enters their address as “2134 Main Street” instead of the correct address of “1234 Main Street”? First of all, you would ship the package to the wrong address. Not your fault, but it doesn’t matter: delivery will be delayed, the customer will have a poor experience, and you will incur re-shipping costs. You may even not get the original shipment back. It’s a lose-lose situation.

The problem is compounded when it comes to getting these customers in the first place: an estimated 25% percent of marketing contact data is bad. And according to the Data Warehouse Institute, this ocean of bad data costs businesses over US $600 billion per year. At the level of the individual company, this means that over a quarter of your sales and marketing resources are lost to bad prospects, for reasons that range from intentional fake contact data to the natural contact record aging process. In fact, Salesforce.com reports that after just one year, nearly 70% of contact data goes bad in some form as people change jobs, phone numbers and email addresses.

So how can your company avoid the challenges associated with bad data? Start by assessing what areas of your company need better data quality control. Whether you identify a single area or several areas that could benefit from improved data quality, realigning your data quality goals to hit the data trifecta can improve your bottom line, optimize your human capital and even help the environment.

To read more about this topic, download our white paper Hitting the Data Trifecta – Three Secrets of Achieving Data Quality Excellence.

Where Does Bad Data Come From?

nature-laptop-outside-macbookWe talk a great deal about data quality, validating information, and the impact on our business. Do we ever stop and think where bad data comes from? It’s not like there is some bad part of town where bad data hangs out as in some B-movie. Bad data doesn’t spontaneously appear as some clouds part. It’s not delivered by some evil version of the stork. Bad data has to come from someplace, but where?

I like to put the sources of bad data into one of three categories: people, processes, and policies. It’s not that any of this happens intentionally. In the course of doing business, we make decisions or perform actions that impact data quality. If we understand the source, we can be better prepared to address the issues. Let’s look at the categories:

The first source of bad data is people. People do enter names like “Mickey Mouse” in a web form to download a piece of information. The resulting lead quality is now very low. If I’m a salesperson, I want to be selling so I may not be very diligent entering prospect information into a CRM system. In many instances, people just don’t know. How many of us know the full 9 digits of our home zip code? Could you properly format an address on a letter to France? How many different versions of a company name could be in the order entry system because the contact center people want to get the order booked? None of this is malicious, but it happens.

The second category, process, is a little more subtle. Two companies combine through a merger or acquisition. Those companies have different ERP systems. Chances are the data in the two systems aren’t consistent, so we now have a data quality problem trying to find the common customer records. Even within a single organization, the people in accounts receivable may be treating data differently than the people in shipping. When a customer moves, the process to change the customer may not be getting enough attention. The orders and invoices are now going to the wrong place costing money and lowering customer satisfaction.

Policies can be external to an organization. Did you know that over 100 different postcode formats exist across the globe? In the US, we don’t even call them postcodes; we call them zip codes. Many countries don’t have postcodes at all. In countries like Japan, the format of the address changes depending on the language in which the address is written. The US includes states as a part of the address; most countries don’t. What happens to our data and our customers if we require a state and US-format zip code on a web form? You get the picture by now.

Rather than bemoan the state of data quality, let’s be aware of the sources. When we build our ERP systems, install our marketing automation systems, and create our websites, think about what can happen. From that point, we can help the people who use these systems and their policies and procedures cope with all the issues. Improving data quality at the source has huge payoffs.

30% of the data in your marketing automation platform is likely incorrect – see how bad your data is with a free scan!

Service Objects is the industry leader in real-time contact validation services.

Service Objects has verified over 2.5 billion contact records for clients from various industries including retail, technology, government, communications, leisure, utilities, and finance. Since 2001, thousands of businesses and developers have used our APIs to validate transactions to reduce fraud, increase conversions, and enhance incoming leads, Web orders, and customer lists. READ MORE