Service Objects’ Blog

Thoughts on Data Quality and Contact Validation

Email *

Share

Posts Tagged ‘Address Validation International’

Thinking Alternatively About Place Names

Here at Service Objects we come across a lot of names, particularly the names of places. We also work with a lot of personal names, but for now I would like to focus on just place names. Whether the name is for a city, town, village, hamlet, district, region, state, prefecture, mining area, national park, theme park or what have you; chances are that the place may have one or more even alternate spellings and alternate names associated with it.

For a human fluent in English, “North Carolina” and “N. Carolina” will be considered equal, but for a computer they are not. With the use of fuzzy-matching and/or standardization we can work around seemingly trivial issues like this. Now let us suppose that you are working with a set of Japanese data and come across the same name but written in Katakana “ノースカロライナ” or Ukrainian data written in Cyrillic “Північна Кароліна” or even Thai “รัฐนอร์ทแคโรไลนา”. Well, fuzzy-matching and standardization are still our friends; we just have more fuzzy-matching and standardization rules to consider. However, we first need to ensure that we even have the data available to associate a name in a different language.

We’ve been creating a list of place names to help us tackle problems like the ones mentioned above. We currently have a list of over five million unique place names generated from a pool of approximately 11 million names. We are aggregating name data to come up with a more comprehensive list that consists of known alternates, variations in spellings, different languages and the transliterated versions for the different languages.

Here’s a quick look at what we have accomplished, so far:

  • Current list of approximately eight million place names and growing
  • Transliteration and phonetic mappings for various languages
  • Case, accent and kana sensitivity handling
  • Queryable using fuzzy-matching algorithms

We have taken some of what we have learned from our DOTS Address Validation – International service and built upon it in order to improve data beyond the realm of just address validation. When working with Phone, Email, IP, Demographic and Geo-coordinate related data we too often find that location names do not match up. Naturally this is to be expected, since different data vendors will have different standardizations and practices when it comes to naming conventions. Utilizing a comprehensive place name library will allow us to quickly perform various actions, such as cross checking multiple data sources against each other with increased flexibility and match rates.

It may not be immediately apparent how useful a place name library like this is and what kind of avenues it can open up, but expect to see new and exciting developments from us in the coming months!

C# Integration Tutorial

C# may very well be our most requested sample code.  There is good reason for that too; C# and the .NET framework are many developer’s first choice for creating a web page or any other type of application because of the versatility of the language and framework as well as the robust features that Visual Studio offers.  One of those features is the ability to consume a WSDL (Web Services Description Language) and create all the necessary classes and methods to successfully call a web service.  This makes using SOAP squeaky clean! Ok, that was the first and last SOAP pun, I promise.  Here’s what you will need for this tutorial.

Requirements

  • Visual Studio (2015 is used in this tutorial but the process should be relatively similar with any other version)
  • DOTS Web Service License Key (We are using DOTS Address Validation International for this example)
  • Some familiarity with C# and the .NET Framework. This tutorial will be pretty basic so it should be accessible even if you are a beginner.

Setting Up the Visual Studio Project

For starters, launch Visual Studio and create a new ASP.NET Web Application and choose an appropriate project name. Your screen should look similar to the following:

Click ok and then select “Empty” to create an empty web form. We will add the necessary aspx page momentarily.

But, our first step for our new project will be to add the service reference to the DOTS Address Validation International web service. To do this, right-click on “References” in the Solution Explorer and select “Add Service Reference” a pop up should appear to add the Service Reference.  Here we will add the URL to WSDL that contains the information on how the project should interact with the DOTS Address Validation International web service and we will name the Service Reference.

For reference here is the WSDL URL and here is what the pop up page should look like

WSDL: http://trial.serviceobjects.com/avi/soap.svc?wsdl

Now that we have successfully added the service reference, we can add an aspx page that will have our input form and display our results. Right click the project name and select “Add” and then select “Web Form” to add a blank web form to the project.  For our example we’ll name the form “AVIForm”.

Creating the Input Form and Code Behind

Now that our form is present, we’ll add some simple HTML and ASP elements to take in our inputs and display them to the screen after we get a response from the service. Make your ASPX page look like the following.

The above code will allow us to take the inputs send them to the code behind and the display the results in the outputGrid and InformationComponentsGrid.  We have to separate grids; one to account for the standard outputs from the service, and the other to account for the InformationComponents field to account for some variable information and data to be returned by the service.  This field can change based on the country or data available for a specific international address. Now that our input form is all set up, we’ll add the proper code behind that will display the results to the user.

We won’t look at every part of the code here, to download a .txt version of the code, click here.

One thing we like to stress to clients who are integrating our services is proper Failover Configuration.  In the unlikely event that our primary datacenters are offline or producing strange errors, we want to ensure that our clients are pointing their code to our backup datacenters so that their applications and business processes go uninterrupted.  Here is a full picture of the proper way to integrate failover into an application.

We’ve found that to ensure uninterrupted service the best practice is to have the calls to the web service nested in a try-catch block of code.  In our current setup, the backup call will hit the same data center as the primary call; but if a License Key is purchased the primary call should point to ws.serviceobjects.com and the backup call should point wsbackup.serviceobjects.com.   The screen shot below, highlights some of the primary failover logic that will allow the code to run uninterrupted.

This code occurs right after the primary call to the web service if it detects that the response from the service is null or if an Error TypeCode of “3” is returned, then the code will throw a new exception and the catch statement will call the backup web service call.

If a successful response is received from the service, the code will call a method named “ProcessValidResponse” which takes in the response from the web service and display the results into a DataGrid and then send that data grid to the ASPX page for the user to see. This method is pretty straight forward for the most part, as it simply assigns the outputs and their respective values into separate columns for the user to see.

The only part that may be mildy tricky would be the InformationComponents field that is returned from the service.  This field is an array of InformationComponent which contains two strings; one for the “Name” of the variable returned and one that indicates the “Value” of the variable returned.  For example is you pass a US address into the AVI service, one InformationComponent that can be returned will have a Name of “DPV” and a Value of “1” indicating that it is considered delivered by the USPS.  Below is an example of the XML output.

 

This array of fields allows us to add new outputs to the service over time without potentially breaking any existing client’s code.  To account for this array of information we have a brief For loop below that will loop through all the elements of InformationComponents and add their names and values to the InfoCompTable so that they can be seen by the user.

Making a Successful Call to the Service

If we go ahead and run our project, our webpage will look like the following.

Not very exciting, but it will get the job done. As an example address, we’ll use the following in France:

3 Place de la Victoire, 33000 Bordeaux, France

If this address is sent to the service you should see the following response.

As you can see, this address is considered Valid by the service and the service has a premise level resolution for this address.  The Address1-Address8 fields will also display how the address should look for mailing purposes. For this particular example we only need Address1 and Address2 but for other countries all 8 address lines may be used.  The InformationComponents field was also parsed out and the two Name and Value pairs were shown below the standard outputs from the service.

That wraps up our tutorial for DOTS AVI integration in C#.  Congratulations! You are now on your way to being a DOTS Address Validation International expert! Feel free to go and test more International addresses, and as always if you have any questions feel free to reach out to us at support@serviceobjects.com!

How to Validate International Addresses

Dealing with international addresses is no simple task. An address can often be misspelled, incorrectly formatted or simply written in a foreign language that you do not understand. The simple fact that many international addresses are foreign to us means that we are unable to recognize when something is wrong.

Take the simple word “street” for example. It is one of many commonly used words in an address. The French word for street is “rue”, in German it’s “Straße”, in Portuguese it’s “rua”, and it’s the character “路” pronounced “lu” in Chinese and so on. That’s not to mention common abbreviations either. In many cases a person will have a hard time identifying the name of a city or a street in an address and they would be unable to distinguish one from the other.

Let’s take a look at a few international examples:

China (中国)

Address:

Shanghai DPF Textile Co., Ltd.
200331
上海市普陀区武威路259号
98 -A3

Unless you are able to read Chinese you would be hard pressed to make sense of the above example. The first line is in English but it appears to simply be the name of a business. Business names are not required to validate addresses with the AVI service and it is unlikely that it would prove helpful in the validation process. To the contrary, extraneous information like this is often regarded by most systems as garbage data; however, let’s go ahead and pass the address as is to the AVI service and see how it handles it.

URL Query:url1

Here is what the query looks like when using the web service test page:

getaddressinfo

Here is the output in JSON, although the service also supports XML:json1Examining the output, we see that the AVI service fixed the order in which we entered our input values. This was done not only in the transliterated Romanized spelling of the address but also in the localized Chinese format.

Here are both versions of the address parsed from the JSON response output:

Roman Character Format

Shanghai DPF Textile Co , Ltd
98 – A3
No. 259 Wuwei Lu
Putuo Qu, Shanghai Shi
200331

Local (Chinese) Character Format

200331
上海市普陀区武威路259号
Shanghai DPF Textile Co , Ltd
98 – A3

The AVI service identified the street name, city name, postal code as well as other useful information.

 

Greece (Ελλάδα)

Address:

114 71 Αθηνα
Ασκληπιου 104
Το ΝΟΣΤΙΜΟ

Unless you can read Greek the above address would be difficult to decipher. Let’s see what the AVI service returns.

URL Query:
url2

Here is what the query looks like when using the web service test page:

getaddressinfo2

JSON Output:json2

Parsing out both of the address formats from the JSON response we get the following:

Roman Character Format:

To NOSTIMO
Asklepiou 104
114 71 Athens

Local (Greek) Character Format:

114 71 Αθηνα
Ασκληπιου 104
Το ΝΟΣΤΙΜΟ

As it turns out, “To NOSTIMO” or “Το ΝΟΣΤΙΜΟ” (in Greek), is the name of a café that resides at the address. Even though the name of the café is not technically a part of the address nor is it necessary for validation, we see that its inclusion did not impede the AVI service from performing its job.

 

Germany

Let’s see how well the AVI service handles an address when several lines of extraneous data are included.

Address: 

Accemic GmbH & Co. KG
C/O World Express (GmbH)
Gunther Meyer, Phone: +49 (0) 8033 6039790
Franzhuber Str 39
Kiefersfelden

In this example, the address is in English, but there is a mess of extraneous information included. What will the AVI service make of this example?

URL Query:url3
Here is what the query looks like when using the web service test page:
getaddressinfo3

JSON Output:json3Parsing out the address from the JSON response we get the following:

Roman Character Format:

Accemic GmbH & Co KG
C/O World Express (GmbH)
Gunther Meyer, Phone: +49 (0) 8033 6039790
Franz-Huber-Str. 39
83088 Kiefersfelden

In the above example, we see that the AVI service was able to ignore the three lines of extraneous information and identify the pertinent address information. From there the service standardized the street name, corrected the locality name and appended the missing postal code.

Lost Deliveries Equals Lost Revenue

Service ObjectsGetting Your Mail Delivered

In 2014, the United States Postal Service reported that 4.3% of all mail was ‘Undeliverable as Addressed’ (UAA). At first glance, this doesn’t seem too bad, BUT this seemingly small percentage can become a very expensive issue, having a negative impact on your business. Looking at the big picture, this translates to over 6.6 billion pieces of UAA mail, costing the USPS $1.5 billion a year. What’s more, UAA mail cost the entire mailing industry over $20 billion in 2015.

Think about the number of packages, mailers, documents, and invoices that you send out on a daily basis. How many Undeliverable As Addressed mailings do you experience daily, monthly, yearly? As reported by the USPS, it is likely around 4.3% of mailings. And that number doesn’t account for 3rd party courier services like UPS, FedEx, DHL, etc.

To help illustrate the hard and soft costs associated with of UAA mail, let’s take a look at a couple examples.

Mail-Order Catalogs

You might think with the advent of the internet, mail-order catalogs would have become obsolete by now. On the contrary, catalogs are a healthy part of omnichannel marketing and with over 11.9 billion mail-order catalogs mailed in 2014, they do not appear to be going anywhere.

In fact, direct mail outperforms digital channels by a long shot, with 3.7% response rate on a house list and a 1.0% response rate on a prospect list. By comparison, all digital channels combined, only achieve a 0.62% response rate. The real power shows up when marketers combine direct mail WITH digital marketing. As Kurt Salmon discovered, over 58% of online shoppers say that they browse catalogs for ideas, and 31% have a retailer’s catalog with them when they make an online purchase.

“For example, during one calendar-year period, we observed that Internet-only customers of one specialty retailer placed orders of $80 on average, whereas catalog customers’ average orders totaled approximately $90.”

Extensive market research helps us see just how important mail-order catalogs are to many businesses. Unfortunately, of the 11.9 billion catalogs mailed in 2014, 476 million catalogs went undelivered. For companies that rely on catalog marketing, deliverability does have an immediate effect on their bottom line.

To provide more context, consider the following scenario.

A large retail clothing company is sending out millions of seasonal catalogs to their house and prospect lists. The costs to design, create, print, and mail these catalogs is substantial but they know that customer order values increase by 15% when they have a catalog in hand.

Unfortunately, recent sales reports indicate that the company’s response rates have dipped about 1%. After some inquiry, they discovered that 4.3% of their catalogs came back as UAA, and therefore did not reach their targets. They have suffered the direct costs of printing and postage and the indirect costs of a lost sales opportunities.

To correct this problem, they implemented some simple address validation tools and were able to reduce their UAA rate, thus saving on these costs and immediately getting back on track.

Return to sender stamp on envelopeUndelivered Invoices & Packages results in Decreased Cash Flow & Unhappy Customers

Your relationship with your customers is everything. And a key part of customer service is “knowing” not only who your customers are, but also where they reside so that their deliveries are consistently getting to them. And, when you have loyal, happy customers, you maintain a steady cash flow.

Along those same lines, customer invoices must be delivered on time, every time, in order to avoid unnecessary operational costs.

Companies whose invoices are not received in a timely fashion will experience a direct impact on their:

  • Accounts Receivables Department: requiring additional time to call on customers or clients when invoices are not received
  • Finance Department: affecting accounts receivable, aka, interrupting company cash flow
  • Mailroom Operations (if applicable): spending time researching and in most cases resending statements/invoices and adding additional postage costs

But, by taking steps to validate their mailing addresses, they increase their chances of maintaining their customer relations. Ask yourself this question: How many times will a customer “tolerate” inadequate service, such as undelivered packages or invoices, before they decide to cancel their business altogether?

So how can you correct and streamline your customer lists for address accuracy while also keeping them current?up-to-dateness? It starts with plugging in Service Objects’ DOTS – Address Validation solution to verify, validate, and correct undeliverable mail.

Solutions, Solutions

To significantly increase the deliverability of your mailing lists, Service Objects offers a number of solutions that will help validate and correct your mailing addresses. All of our services can be accessed through our real-time APIs and/or using our batch process.

Here’s a quick rundown:

  • DOTS Address Validation – US, can instantly verify, correct and append address information against our CASS-Certified USPS address verification database. Some companies choose to append latitude/longitude and demographic information, in which case, we recommend DOTS Address Validation Plus. Using either of these services, the majority of your addresses will be verified and corrected, and ready for mailing.
  • DOTS Address Detective uses real-time “fuzzy” logic to complete missing data points in the address and then verified against our CASS-Certified database. Great for incomplete and hard to validate addresses.
  • DOTS NCOA Live service, keeps verified and corrected addresses up-to-date. This is built on the USPS National Change-of-Address database (NCOALink).

Think Globally

  • DOTS Address Validation – Canada, covers 15.7 million addresses in all 10 provinces and 3 territories and validates both French and English addresses.
  • DOTS Address Validation – International, verifies, corrects, and appends addresses outside the United States and Canada. Covers over 250 countries, dependent areas and territories. Addresses are corrected to the unique requirements of country’s postal address formats and cultural idiosyncrasies.

Whether it’s a direct marketing mailer campaign, confidential documents, packages, or customer invoices, you need to count on successful deliveries every time.

So get it right the first time by using the most genuine, accurate, and up to date customer address information available. As it relates to your bottom line, there is no room for error. Plain and simple, Service Objects Address Validation solutions can help you reach your mailing goals!

The Challenge of Storing International Addresses

World map with contacts and figures. Business concept

Working with international address data can be difficult and confusing. Even when you have an application available to validate an address, and it tells you that it’s deliverable, you still have to deal with the chore of storing the resulting data. So when someone asks, “what’s the best way to store international addresses?”, what they are really asking is, “what’s the easiest way to store international addresses?”

The short answer to the “what’s the best?” question, as it often is, you’re asking the wrong question. Many of you who have worked with varying data sets before already know that you first need to ask yourself, “what do I intend to do with the data once it is stored?” What the data is used for should have the largest impact on how it is stored. Depending on your specific requirements, the way you store address data can vary greatly. For some, how you store your data may not be entirely up to you as you may not have any control over the storage design, and are instead forced to work with the fields that are made available to you. Many users work with US-centric Customer Relationship Management (CRM) solutions that are designed with US address fields in mind, which can make storing international addresses all the more confusing and can also potentially lead to some data loss.

For those looking to simply print an address label for mail delivery, a single text field containing the complete formatted address will suffice. After all, why bother with breaking an address down to a mess of individual fragments if you’re not going to use them? Worse yet, what do you do when it comes time to put the pieces back together and you find that you don’t know how?

For some, correctly putting an address back together from its individual fragments might not be of great concern. The primary use of the data may be for some form of query analysis and/or organization. In which case you might be more concerned about which specific data type your individual fields should be or how to properly map these fragments. If you are implementing your own design then keep in mind that not all international addresses are necessarily parsed the same way, and you will need to consider if your design should be flexible enough to handle all international addresses or if you would prefer to go a country specific route.

Mapping Address Fields

Consider this example of an address in England:

9 Gorse View
School Road
Knodishall, Saxmundham
IP17 1TS
UNITED KINGDOM

If we include the country name, then the above address has five address lines; six if we split the third line. Now, let’s go ahead and attempt to store this address in our CRM. Most CRMs will contain the following address fields for a contact:

Address1
Address2
Address3
City
State
ZIP
Country

Depending on the CRM, we may have somewhere between five to seven address related fields on average to work with. In the above example we have seven, so that should make things easy, right? We have more than enough fields, so there should not be any loss of data, but right away we see State and ZIP fields. These should be red flags that the storage was not designed for international addresses, but unfortunately, it is what we have to work with. Let’s go ahead and look at the parsed fields that we are likely to get back from an address validation solution:

Premise Number: 9
Dependent Street Name: Gorse View
Street Name: School Road
Dependent Locality: Knodishall
Locality: Saxmundham
Postal Code: IP17 1TS
Country: UNITED KINGDOM

In most cases, users will find that they can typically match Locality to City, Administrative Area to State, and Postal Code to ZIP. If you are unfamiliar with the address terms “Locality” and “Administrative Area” then please check out our previous blog, Five Commonly Used Terms and Definitions in International Address Validation Systems.

In the above example, you’ll notice that an Administrative Area equivalent was never provided. You’ll quickly find that this is quite common for many countries and that the locality is usually preferred. You’ll also notice that we have a dependent locality, which is a sub-region of the locality, and a dependent street name. It is important not to omit or lose these pieces of data if they are provided, as they offer additional detail/instruction on the whereabouts of an otherwise ambiguous address. So where to map them?

Luckily, our database design offers enough fields to accommodate these values, but keep in mind that this may not always be the case. In our example, we can map the premise number and dependent street name to Address1, the street name to Address2, the dependent locality to Address3, locality to city, postal code to ZIP, country to country, and leave the state empty. However, even though we were able to successfully map every value to our CRM, it is still very tedious and risky to try and handle all of the various address formats. Also, what course of action do we take when an address also includes a double-dependent locality or a sub-region?

Missing State or Administrative Area Equivalent

Let’s look at two more example addresses:

3-10-13 Ryoke
Urawa-Ku
Saitama-Shi 330-0072
JAPAN

and

5 Rue Sainte-Catherine
12000 Rodez
FRANCE

The first example is a Japanese address. Looking at it with American eyes one might think that the first line is a premise number and a street name, the second line the city, and the third line the state and postal code, or their equivalents. However, things work very differently in Japan. Streets are not commonly named or used for addresses. Instead of street names, they primarily use regions that can normally be thought of as districts. In the above example, Ryoke is a second level sub-locality, Urawa is a first level sub-locality and Saitama is the locality. No administrative area equivalent value is given. Administrative areas are commonly omitted as often as they are included in Japanese addresses.

In the second example, we have a premise number and street name in the first address line, and a postal code and locality in the second. Once again, no administrative area value is given. The address is in France, but many European addresses will follow this general format, and it is common for them to omit a first level administrative area. Therefore, it is highly recommended that you do not make an administrative area a required field. Doing so would mean rejecting valid addresses for entire countries.

Facing the Challenge

As I mentioned earlier, when breaking an address apart we also run the risk of putting it back together incorrectly. So while no individual address fragment might be lost, we still risk losing the correct address order and format. Addresses and their various fragments and formats can vary greatly not just from country to country, but also within the same country. So what’s the point of it all? Is there no hope when it comes to international addresses?

If you are forced to use a set storage design and are unable to alter it then your best course of action may be to simply store the complete formatted address in a single field, if it can fit. If the complete address cannot fit in a single field, then split it into multiple fields when necessary. In general, storing the complete address should be your primary objective as it should contain all of the necessary information that you need. The complete address can always be parsed out later as needed. Storing the country and postal code should be next on your priority list, although not all countries use postal codes. Postal codes are very important and useful, so be sure to store them when they are available. Finally, look towards storing the locality and admin area if they are available.

For those who will be implementing their own design, look to the output specifications of your validation solution. Most validation solutions will have a large list of address fields that cover the majority of the most widely used international addresses out there. You may consider it cumbersome, but if you include all of the output fields from your validation solution in your own design then you minimize the risk of losing data during the mapping process. You might not consider it the best way to handle storing international addresses, but unless you want to become an expert on the subject, it is definitely easier to use an existing design.

Service Objects is the industry leader in real-time contact validation services.

Service Objects has verified over 2.5 billion contact records for clients from various industries including retail, technology, government, communications, leisure, utilities, and finance. Since 2001, thousands of businesses and developers have used our APIs to validate transactions to reduce fraud, increase conversions, and enhance incoming leads, Web orders, and customer lists. READ MORE