Posts Tagged ‘API Integration’

API or Batch? Use the Right Tool for the Job

Our data validation products can be accessed in one of two ways: through APIs that integrate directly with your custom solutions and/or business automation platforms such as your CRM or marketing automation systems, or convenient batch services that let us do the work for you, with no programming required. This article explains each of these options, and which ones might work best for your situation.

First, let’s look at some of the advantages of each of our data validation options:

API integration

  • Fastest response time – Your API calls will interact directly with our web services. For many of our services, this will result in sub-second validations of your input data.
  • The most flexibility – Virtually any platform that can make a web service call can leverage our APIs. Our web service based APIs can be injected into your workflow at any stage of the process. This includes point of entry, post-lead collection, or even refreshing historical data.

Batch Processing

  • One time batch

    • Hassle-free list processing – Let us handle the heavy lifting by securely sending us a list of your data. We will process the records and return an easily digestible comma-separated file with all of the web service’s response fields divided into columns.
  • Automated batch

    • Similar to a one time batch, an automated batch will also return a comma-separated file. The primary difference is that you will have a secure FTPS account where uploaded files will be automatically processed. Upon completion, you can log into the FTPS account and retrieve your validated data. This option is great if you have the need for ongoing data processing and don’t want to integrate with our API.

Here are how these options can work with common workflows:

Batch process existing records + API to validate real-time data capture

A common situation we find with our clients is where they have both an existing dataset and ongoing data collection. Our goal is to ensure the data is as accurate and up-to-date as possible and this extends to existing data points. Cleaning up and validating existing data helps ensure that any future efforts to connect with previous leads are met with success. Furthermore, by integrating with our APIs, data can be cleaned and validated on its way into your system. This allows you to have peace of mind knowing your lead data is genuine, and better business decisions can be based off of this validated information.

Periodic data dumps, processing, and reimport

Some business workflows don’t require instantaneous data validations. There are some cases where weekly, monthly. or quarterly data validations fit the bill. This scenario can be solved using any of our data validation options. If you’ve integrated with our API, your validations can happen in-house. However, if an API integration isn’t right for you, our batch processing system will likely meet your needs.

Multithreaded API integration for large volume validation

For high volume validations, we recommend dedicating time for your development team to implement a multithreaded solution. This type of implementation allows for multiple inputs to be validated simultaneously. This effectively increases your throughput and cuts down on the overall processing time.

Keeping your data secure

Regardless of the type of integration, the input and output data are transmitted using an encrypted protocol. Our data centers are protected by five layers of security and constantly monitored and actively managed from a data security and performance perspective. We do not store your data any longer than is needed for validation and is expunged after processing.

Need more information for your specific use case? Contact our technical team anytime – we are always happy to help.

API Integration: How Much Time to Set Aside

The question of ”how long does it take for someone to integrate your API?’ has come up a lot lately in various situations. In this article, I wanted to paint a better picture of what that looks like.

There are a mix of factors that should be considered for your integration, including available resources, what solution you are plugging the API into, and whether you have someone who can do the work of the integration. For example, where you want to plug in our API will have a direct impact on the resources you are going to need when integrating. Do you need someone who knows Java or .net, or do you need someone familiar with Salesforce or Marketo?

If you need to integrate an API into code, then you or your resources should have at least a familiarity of the programming language currently deployed in your solution. However, if you need to inject our API into a platform such as Salesforce or Marketo, then you may be looking at perhaps a marketing technologist or more admin skill level person that can handle more of a plug and play scenario that involves adjusting settings.

Depending on what you are integrating with, there are three main types of personas you can group people into, each of which require different amounts of time and assistance in getting an integration project done. In any case, however, these people will all need to have a good degree of knowledge about the surrounding process and business rules that will need to be implemented.

Let’s look at each of these three groups:

Group 1: The super knowledgeable

This person has done this before many times over and is going to need little help with the integration. They may just want to look at our development guide so they can see the endpoints they need to hook up to and the format and field names coming back from the web service API. For setting up the call to the API, this person is going to need little more than 20 minutes.

Group 2: Strong coder or strong platform admin

This person doesn’t call APIs that often, but has done it before. In this category the person will be integrating into code, plugging into a platform, or both. They will have a few questions about the API and may request a guide or overview to help them with integrating into a platform. We have materials that will help with both code and plugging into many platforms which they will want to look through. This person is going to need closer to a couple of hours to get our API integrated into their system.

Group 3: Relatively new to coding or platform administration

This person has never really dealt with APIs before. This situation is more of a grey area because it depends on how hands-on the person is. These people will generally need more guidance and may even need professional services to accomplish the job. The time it will take in this category will depend on this person’s inclination to learn something new.

We have many resources to help show someone like this how to integrate our services, and this will make the process much smoother. For example, we have sample code for web forms that can be dropped into several scenarios and can also be repurposed to fit most applications. Someone with very little experience calling web APIs should be able to navigate our documentation and get rolling in a day.

Customizing these estimates to your situation

There will certainly be other groups beyond the categories I have simplified here. So, if you find that you don’t fit into one of these specifically, you can still use them as a guide for evaluating the time it will generally take to integrate our API services. As you can see, the amount of time involved is typically very short.

Another part of the integration, not to be forgotten, is more about implementing your business logic and what you do with your data before you send it to our API for validation, as well as when the results come back.

We can give you recommendations and best practices based on what we have seen in the industry as to how certain situations are typically handled, so you can have a good idea for how to set up your solution. The time it takes to do this end of the implementation is a lot more variable and is based on the complexities of what solution(s) you are trying to achieve. We here to work through the process with you by providing our cutting-edge solutions along with our recommendations.

How we can help

Here is a link to a search of our blogs that will display various documents on integration along with several tutorials. If you can’t find something you need, let us know and we’ll do our best to put it together. Along with this, I have additional links to our developer guides, sample code and quick look-ups. We work to make every integration as smooth as possible and will go a long way to help, including offering our professional services to get you across the finish line. Let us know what we can do to help make your implementation a success.

No image, text reads Service Objects Tutorials

Check Out Our Updated Developer Guides

We recently revamped our developer guides to make them more user-friendly. The information for each service has been broken into its component parts, to make each feature more comprehensible. In this article, we share some of the changes we have made and they help you get the most from our products.

1. A new, easy-to-navigate landing page

Our new landing pages now have a developer guide map (our US Address Validation product is a good example) to provide easy access to each web service’s component parts. By clicking on the links on this page, you will be taken directly to the respective page that provides a more in-depth breakdown of the service’s individual fields. Below is an example of one of these landing pages.

2. Quick lookup & Open API

Our quick lookup page has been greatly simplified to provide only the most relevant data at your fingertips. Once you have received a trial or production license key, the validation process is as simple as plugging your input data into the fields, adding your license key, and clicking Execute. The web service request is made for you and the response formatted in the output section.

Similarly, if you are looking to run a different operation the Service Reference page has quick lookups for every operation within its respective service.  Check out an example of Address Validation US.

3. Citations and Additional Field Information

Have questions about the service fields? Need more information about how to leverage them to fit your business needs? Our redesign now includes accordion panels that can be expanded to provide more information about the field. We’ve also added citations next to some of the inputs and outputs where we have additional information available. In the example, below the Barcode Digits is linked to the following blog:

By clicking a reference link, you can read a more in-depth analysis of the field and often gain ideas about how to use the data point in your workflow.

Each of these new features is designed to help you get up and running more quickly with our services, and serve as an easy-to-use reference for your development work. Try out our new developer guide soon, and let us know what you think!

Tuning your Implementation: What Impacts Speed and Throughput

How fast is fast? How fast should things be? And more importantly, what can you control to make the response time of using our services as fast as possible?

In this blog we will break down the key components of response time, to shed some light on what you can optimize to make your implementation respond faster. Whether it is a call to a Service Objects web service or an API integration in general, there are specific phases that take place: input data preparation and transmission, data processing, and results transmission and processing. For example, a simplified view of a single data validation request is as follows:

  • Your data is securely sent to the Service Objects web server to be processed
  • The data is received by our server and processed/validated
  • The validated data is then securely returned to you

Most people think of response time as being the round-trip time to get your data validated – and this is indeed the primary concern – but total throughput should be addressed as well. And if you are able to get a single validation done as optimally as possible, expanding the workflow to fit simultaneous requests shouldn’t take too much modification to the workflow.

To understand what is going on under the hood – and in particular, understand what factors may be slowing down your speed – let’s compare this process to a trip to the grocery store. It has similar phases to a data validation process: you leave your house, travel down the road, stop at the market, purchase your groceries, and then drive home. Here is how this process breaks down:

Step 1: Data preparation. This is analogous to the steps you take before leaving your home. Did you turn off the lights? Are all of the doors shut? Is the security system armed? Do you have everything on your grocery list?

Each of these steps is similar to the checks that your application goes through in order to leave your company’s internal network. The application has to gather the information to be processed, dot all of the i’s, cross all the t’s, make sure it knows where it is going, and make it through any layers of security your company has in place. Each layer takes time to complete and detracts from your overall speed.

Step 2. Input data transmission. This is like traveling down the road to the supermarket. How much traffic people encounter depends on factors such as how many lanes the road has, how many people are in each car, and how many cars are on the road. From an engineer’s perspective, this is like the concept of single threading and multithreading: if the road that you are traveling down has multiple lanes, the number of cars (e.g. API requests) can be larger before running into traffic.

By creating a multithreaded application, you can effectively widen the road and allow for more throughputs. Your choices include single car + single lane (single value request), single car with multiple people in it + single lane (smartly using one validation client to do multiple requests), multiple cars with single passengers on multiple lanes (semi-smart multithreaded validations), and multiple cars with multiple passengers on multiple lanes (really smart multithreaded apps).

Step 3. Data processing. Once you reach the store and make your purchases, it is time to pick a checkout aisle. The number of aisles that are open act similarly to the lanes on the road. More aisles allow a larger number of people to be processed before a queue builds up.

This part of the process is handed by Service Objects. We have spent the last 15+ years not only making sure to stock our shelves with the best quality products, but also ensuring that there are enough checkout aisles to meet the needs of our customers. In our case, we have multiple data centers (super markets) with load balanced web servers (smartly managed checkout aisles) to keep all of our transactions moving through smoothly.

Step 4. Results data transmission. The return trip from the super market is very much the same as the trip there, but in reverse order. Similarly, entering your house with the newly purchased items (validated data) is the same in the opposite direction. This stage also contributes to the round trip time required to process a request.

Step 5. Unpacking the results. When you return from the store, you still have to unpack your groceries. Likewise, once you get the data back from the service, you still need code and business logic to unpack the results and store them in a form that meets your needs. (Or, to extend our analogy further, create a delicious meal from the quality ingredients that Service Objects supplies you!)

So total processing time is a careful balance of getting through your front door, navigating down the single or multi lane highway, checking out with your groceries, and then making the return trip home. From an API-call perspective, improvements to your speed start with your network, smart choices when writing your integration code, and juggling the requirements put in place by your business logic. And if you are looking to increase your throughput, a multithreaded application or distributed system will often increase your capacity.

Finally, one more analogy (we promise!). Good grocery stores have staff who can help you make the most of your shopping trip. Likewise, Service Objects has an industry-leading team of technical support experts who know their stuff, and are always happy to help you make the most of your implementation. How can we help you? Contact us any time to discuss your own specific needs.