Why The Product Migration/Upgrade is very challenging exercise for Every Organization

With Latest Mobile technology you could upgrade your IPhone Operating system or any app almost instantly, But why an enterprise   application/product migration/upgrade projects takes years  to complete and lots  those projects fail before reaching finish line. 

What’s the difference?  The most obvious reason is typically the corporate applications are lot more complicated and carry lot more risk with incorrect upgrade or failure. The risks are even more if it is involves a business operations involving customer data.    With the SAS (software as service) model does this makes it easy for organizations to manage/Upgrade much more efficiently. Before we make the decision, let’s look the reasons and complexity of Product upgrades.

Enterprises typically built their own software or purchase it from software product development companies like IBM, Microsoft, oracle and many more. Some of the Enterprises will provide Software Product/Services to their business customers  it could be likes of Sales Force, Workday ,QuickBooks online and many more. In all these scenarios typical product implementation goes thru following patterns

 Every enterprise has somewhat unique business process and business needs that application fulfills. Typical commercial application has configuration features where either Product implementation team or System Integrators will do configuration work. On top of the enterprise go ahead and do more product customization specific to their needs which could even involve tinkering the code of specific components. As you could also see that a product might become integral part of the business operation where data/information is exchanged with many other application/products within the organization. That’s bring the complexity and fear of something going wrong. Even small change in a product might impact somewhere in downstream and could hurt the business outcome. On top of it goes across multiple business units/departments  so each of them might look  product from vary different  context.

Here is the some of the Important reasons for any Product Migration will get in to challenging:

1.   Product development ,Product Sales and Product Implementation teams are not always on same page and will pull  product road map in  different directions:

 Product development team looks the industry trends, what competition doing and try to build the features  (Many times irrespective of current customer demands).

Product Sales team goal will be selling the product and would push lots of Ad-hoc requests for customization of the product for it to go thru. Depends on the size of the customer they are pursuing , the noise they can make, the influence they have on product company  the product company goes with specific customizations to impress the client.  So Product upgrade will have challenges to these customizations where they might not incorporated back in to the product.

Product Implementation team will also similarly use their creativity in the product configuration to fit customer specific business needs, even they are not direct fit for the product they are using. Data /Feature over loading to take care of the customer current problem in hand only make it bigger unmanageable problem at upgrade time. Also this is just configuration so it is seldom documented with very little trace left behind until totally different problem presented to customer at the time of product upgrade. Many of these hacks done by Implementation teams never reach product developers as needed new features. so they very rarely show up in product development roadmap.

                   So every time a Product releases a new version, these configuration(s) and customization(s) will potentially become road blocks to upgrade. Discovering the problem, finding solutions (one more round of Band-Aid fix thru configurations and customizations) will take lot of time and resources. Take a look of your last ERP/CRM upgrade, how rarely you hear it went smoothly?.

2.   Product upgrade sometimes drastically change its Architecture 

With ever changing nature of current marketplace and technological innovations, Products want to stay above the curve and provide new and fancy features to customers. While implementing these new features leads to drastic changes to existing product architect and discontinuing old features and services.  With the change of architecture all the plumbing between applications need to be changed leads to lots of rework and changes to many other applications in echo system.  Again that’s add Time cost and resources.

3.   The Fear of the change and lack of understanding the impact

With vary matrix nature of the current corporate world but where common thread of applications/data ties them together, there is always fear of change scares every organization. It is very hard to evaluate the change and what could be impact in the organization.   Even a small change  in echo system could have big impact in lot of places, so that fear force organizations to spend lot of time to evaluate and test every possible way  to reduce the risk before getting on to new product release.  If we look around we see companies still using  Internet Explorer version 8 as enterprise standard and not allowing to upgrade to latest version for the fear of some applications are not going to work and ready to hold on to a version of more than three years old. There is a telecom company still using their green screens to provision the land lines, just because they are not able to get the applications to latest technology.  Lotus notes survived as email engine for more than 20 years even it became outdated for same reason and there are companies still hold on it.

4.   The Testing cycles takes for ever

In most organizations given the intricacies about the usage of product or its produced data, the organization really want to be sure that the  upgrade is not negatively impacting any of its current business usage. For that reasons there will be enormous effort is spent on thoroughly testing the upgrade with all the scenarios. Even after that still try to keep both versions ran in parallel for months until every corner of the enterprise build confidence of the upgrade. Sometimes even a small impact could also push back the upgrade to start over again.

          Here are some of the main reasons for product migrations/upgrades takes forever to complete. Now  if we take SAS(Software as Service) model does it  makes any Better?.

          Many Product companies selling their product as SAS model by just installing the product on a Private cloud. But product is still the same and going thru same versions/releases.  I don’t think there will be much change still it depends on how strongly it is connected and integrated with in the enterprise. There is also additional challenges comes when in SAS model with multi-tenant environment (Many customers/companies using the same application) as all of them need to be ready to migrate to new version of the application or  the application should able to run multiple version(s) in same instance.

Challenges in Mastering Customers in Retail industry

Retail companies want to understand their customers and their purchasing patterns, their interests, their behavior. Retail biggest dilemma is understanding who is their important customer and how to being him back to the store or on any channel for repeated purchases. Many Retail chains went on road to implement MDM Solutions and only able to get limited success in this goal. Let’s look some of the major challenges

Inconsistent Data Capture:

Retail Stores:

A customer is willing to provide lot of information about himself when he purchase Health insurance, or open a bank account. But in retail world the customer provides very limited information while shopping in a store. In a Retail store customer will provider very little information to complete the sale and similarly Sales rep on POS also keen on finishing it faster so there won’t be long lines. If the transaction is done with credit card part of the swipe customer name is captured, but if the transaction is cash then there is no useful information is available.  The Data capture may wary based on the products he buy, when a customer buy expensive product there is higher chance of gathering reliable information.

o   Grocery Chains, Department stores might not get meaning full customer data part of the sales transaction. The Customer data might limited to Name.

o   When Purchases are High end like Home appliances, electronics the customer might provide Name, address, phone number and may be Email.

o   By storing credit card or doing reverse look up might provide more information about the customer but there is regularity restrictions on what can captured.

o   Storing Credit card with transaction is also carrying lots of risk for retail companies based on recent data breaches with some of famous retailers.

Loyalty Program/Store Cards:

Lots of retail stores moved towards having a loyalty programs or Store cards to capture customer information and track the sales of customers. While registering for the membership customer provides information as minimum as needed for the membership and depended on motivation of customer.

Some stores  (Eg: Kroger, Safeway, CVS) expect a physical card and some goes with simple searches like Phone number ( Eg. : Toysarus, Gamestop ). But many of these still able to Track Household than an individual in most cases.  As the loyalty/store card is tied to each sales transaction which helps to understand the customer more than earlier but still probably hard to contact or market to customer given the information inaccuracy.

For many Warehousing retailers like  SamsClub, Costco might have better quality of data as they have infrastructure to capture and validate accurate customer data and also with membership cost involved to the process customer has enough motivation to provide accurate data.

Online Stores

Online stores of the same retailer might capture little more information as customer provide accurate address, phone number, email etc.. as part of the registration or ordering process. But there could be inaccuracy as the shipping address might not belongs to the customer as he might be sending as a gift to  friend or family member not living with him.

Overall the Data capture from various channels is very limited and less accurate.

Credit Card Reverse Lookup

Lots of Retailer companies in the past captured credit card numbers and tried to reverse lookup and get the information about customer (Name, address, phone number etc…), But this is proved very risk with recent data breaches with major resellers.

Here are major issues with this approach

Retails are lot more hesitant to keep Credit card information for longer period than required give the liability and regulatory restrictions.

Many Credit card processing providers are restricting on the information retailers can capture part of the transaction.

Many states are regulating retailers, credit card processing companies on data captures and purpose of the use.

There all limit the Customer data capture while the sales transaction going on.  

Inadequate Data to Match:

Overall the data capture from all the channels is very limited and less accurate and problem is to get more usable from customers directly very hard. Given the data itself is very sparse, it becomes hard to match and merge them across the channels reliably.

Matching Using Customer Data Providers:

Lots of Retailers use  Third party providers who can match with the limited information they have about the customer to get more details about the customer .The Third party providers are able to match  using limited information like name, email address, phone number  and provide  accurate details of customer and including customer profile information.

This seems to be a way to get better contactable information about the customer and able to understand customer well, this is also very expensive option too.

Constraints with third party providers are:

1.       Most of the Providers only rent the record for limited use. After the expiration of the rental period retailer is obligated to remove from their system

2.       Providers only provide information for specific use case , which limits usage for enterprise customer master data  management

3.       Also whenever information is received from provider is the movement of truth of that time. With time the data becomes stale, which force to validate periodically which also incur cost.

4.        As most providers get their data refreshed  only periodically (may be once a month), at times the information captured with client might be more accurate.

Given all the constraints around getting the data from Third Party providers, the cost and restrictions are enormous and need to be managed carefully.

Best Practices:

Given all the challenges here are few best practices for customer data management

1.       Build data capture business process to get more accurate information about the customer.

2.       Put together Loyalty programs and encourage customer to provide accurate information providing incentives

3.       Have same data points capture in all the channels

4.       If third party data provider is used, understand the cost and restrictions around the data usage.

5.      Try to  Identity the customer/household  with in your data accurately so  you will be able to understand customer better.

6.       Honor customer preferences likes Do not call, Do not mail  thru customer mastering to have better customer experience.

Why Data Management Matters

Flowers are static, we don’t consume them except with our eyes and perhaps our sense of smell (If they are decorative plastic flowers then there won’t even smell). In that sense they are unlike food, which we do consume and from which we derive sustenance, nutrition, and life itself. A life without flowers might be unfulfilling and perhaps even miserable, but  life without food would be certain death. Same goes about Data for  any Organization

And so it goes with data. Data is not static, sitting on a shelf. We keep receiving it and consume it. Data meant to be consumed by our business applications for everyday use. Provide better experience to its customers, understand the customer behavior, Identify fraud/risks, Provider deeper understanding for better business decision making.

Just as we need food from basic four food groups for sustainable health, data needs to Managed and Governed with following norms to make it fit for consumption

1.      Data need to be Complete: Many times incomplete data leads to wrong conclusions and bad decisions.
2.      Data need to be Consistent: Data does not have consistency many times it is impossible to analyze and make sense of it.
3.      Data need to be Conform: Data need to conform to standards
4.      Data need to be consolidated: If data is not consolidated to usable levels could lead wrong aggregations. Eliminating data duplication is very important for data management to have accurate reporting in analytic's   and right information in operational side.

When data follow all these norms is healthy and delicious for consumption. It builds trust of business to use the data which leads to healthy decisions for the organization. When data managed consistently will keep getting better, healthier and more dependable.

The way we take care our food for cleanliness, nutritious values, calories, taste the same constant focus need to be given to data management to keep following the norms. As we getting new data sources or building new data centric applications making sure they follow same norms is equally important.