Once, a local and customer-centered activity, the art of marketing has developed explosively over the last century. The first breakthrough came with the advent of mass communication technologies, such as radio and television, which expanded the audience reached by marketers to millions of people located throughout vast geographic areas. The price to pay for large numbers was the necessity to use a universal marketing message for everybody. Marketing became product-oriented for a while. The second revolution in marketing was ignited by the rapid progress of computers and databases. This endowed corporations with memory: data about customers and transactions were now stored for future use. The idea was to utilize historic data to make marketing customer-oriented again – and thus more efficient.

However, it was soon realized that sheer amounts of data to be analyzed in order to make better marketing decisions require dramatically improved new automated data modeling technologies. A concept of Data Mining – automated search for hidden new, useful, and actionable knowledge in data – appeared in the nineties. Data Mining is still in its infancy – but is very promising. Incorporating technologies for automated learning from historic data into complete Decision Support Systems for Direct Marketing has a potential of closing a loop in the evolution of marketing on a new level. The world returns to customized marketing to individual prospects, but this time delivering an individually-carved message to each of millions of prospects at once.

New Direct Marketing techniques provide means for custom-tailoring the marketing message and channeling it to precisely those prospects that are likely to respond to the promoted product or service. Modern marketers rely on Decision Support technologies to facilitate an accurate match between the marketed product or service and its potential consumers, thus improving the efficiency of a marketing campaign and saving the marketing company significant funds – an important competitive advantage.

Direct Marketing tasks include market segmentation, response modeling, cross sell prediction, customer valuation, and market basket analysis. Building successful solutions for these tasks requires applying advanced Data Mining and Machine Learning techniques to find relationships and patterns in historic data and using this knowledge to predict each prospect’s reaction to future situations.

A whole new generation of marketing managers is being brought up on the ideas of one-to-one marketing and mass customization. One would expect to see an abundance of reliable Decision Support applications for Direct Marketing. A quick search, however, reveals that this is not the case. What is the matter? What technological problems have been slowing down a universal acceptance of the new custom marketing techniques, and how can these problems be solved?

To find an answer, let us consider the structure of a typical Direct Marketing workflow presented in Figure 1. The shown workflow underscores that direct marketing is a multi-step recurrent process, using customer feedback and gaining precision with each round of customer communication, data collection and analysis, rather than a one-time endeavor. The Direct Marketing chain involves the steps of data acquisition, storage, modeling, and scoring, followed by campaign management and customer response measurement. These steps have been linked together and computerized to a large degree, so that today one can speak of first integrated Direct Marketing software. The bottleneck of this process remains in the step of data modeling and knowledge discovery: a significant human intervention is required at this step.

Fig. 1. Direct Marketing workflow diagram.

The Direct Marketing chain starts with capturing data about prospective customers. The most common sources of such information are E-commerce web sites, point-of-sale scanners in stores, and customer loyalty programs, which allow tracing sequential transactions of individual customers. Also, one can purchase data from external database vendors as a supplementary or even a primary data source. In order to analyze captured data one needs to move transactional data to a data warehouse, facilitating data consistency and performing various aggregations. After that, OLAP tools are used to create views of data suitable for further analytical analysis. This part of the Direct Marketing workflow has been largely automated and integrated together with campaign management tools.

Let us assume for now that we know how to carry out the step of Data Mining and find hidden patterns in data, as well as explanations of these patterns and accurate and significant models predicting outcomes of future situations. We will return to this issue in more detail in a moment. The next step is to use the developed model to score the bulk of customer records in the database to pinpoint the best prospects, segment the market, and identify promising cross-sell opportunities. These predictions serve as a basis for the future marketing strategy, allowing us to operate with the most efficiency and wisdom. Finally, we communicate with selected customers and measure the response, recording it in the same data warehouse to make our marketing solution smarter with each round of direct marketing.

Returning to the step of data analysis, we note that currently it represents the most difficult and nonintegrated step of the considered business intelligence chain. An intervention of a human analyst is required on several occasions during the data modeling stage. First, a sample of historic data for analysis and machine learning has to be imported from the existing storage architecture to a selected Data Mining application. Then the best strategy for data analysis has to be selected. Finally, the developed model needs to be exported back to the Direct Marketing application in order to score the bulk of new data and identify the best business opportunities. Data Mining remains the bottleneck of Direct Marketing solutions because reliable machine learning algorithms aimed at distilling models and patterns from data involve complex mathematics and have just emerged from academic circles to the business world. The corresponding software products are immature, and they possess no means for simple integration with other pieces of Decision Support chain.

There are quite a few standalone analytical tools available on the market. However, Data Mining applications have to process data of a diverse nature drawn from different storage architectures, use multiple data-specific exploration algorithms, and present results in a variety of forms suiting the needs of individual customers. Consequentially, Data Mining applications come to the market today either overloaded with functionality, which is never used in its entirety by any single user, or relatively simple and universal – but very hard to customize for a specific problem.

In fact, there is a combination of issues that complicate successful incorporation of the Data Mining link in direct marketing chain. Often, Data Mining systems…

• are too complex to operate
• lack algorithms suitable for tackling specific Direct Marketing tasks
• cannot be simply integrated into a complete Decision Support chain
• are quite expensive, hovering in the range of one hundred thousand dollars or more

It sounds like there are too many problems that one has to face at once. Is it reasonable to expect a quick solution to this situation? Surprisingly, yes – there exists a strategy that addresses all these problems at once.

This strategy suggests that Data Mining applications should follow the specialization and unification example already set by hardware manufacturers. As computers are readily built from separate hardware components, new software applications should be easily built from standardized software components. This new software development paradigm, called Component Object Model (COM), was specified by Microsoft in the end of 1995 and is also known as ActiveX technology.

COM defines a standard mechanism that allows different software components to provide their services to each other. This mechanism works the same way in all possible situations, independently of the nature of the involved components. The new technology provides a foundation for interface-oriented programming: if you know how to communicate with a component, you can use it.

Recently, a few manufacturers of Data Mining tools rebuilt existing Data Mining algorithms on the basis of ActiveX (or COM) components, which can be readily incorporated in any existing vertical DSS, boosting the performance of the system as a whole. This makes the Data Mining step an integral part of the Direct Marketing business intelligence chain as depicted in Figure 2.

Fig. 2 COM-based Data Mining components

The functionality of an integrated, COM-based application can be illustrated by the scheme displayed in Figure 2. The Main component is in our case an integrated Direct Marketing and Campaign Management solution, which relies on the services provided by the Data Mining component. The user invokes powerful Data Mining algorithms, and visualizes and applies the results of the performed data exploration through the familiar interface provided by the integrated Direct Marketing solution.

The requests for data analysis are processed through joint work of different components. The Client component furnishes a seamless interface with other parts of the integrated DSS and addresses the Data Access component responsible for the import of data for the analysis. Then the Client component calls the Server component methods implementing the required exploration engines, also providing a pointer to the tuned Data Access component. The Server performs the data exploration. The obtained intermediate and final data exploration results are passed to the client component for visualization and further manipulation.

The COM architecture allows one to use all the power of the analytical algorithms built into the Data Mining component and all the data manipulation and presentation functionality provided by the complete decision support application. This is nice, since all that the user really wants is to use the new Data Mining capabilities from their favorite application.

To further appreciate the beauty of COM-based solutions, let us get for a moment in developer’s shoes. ActiveX components start to look even more appealing. Now developers can:

• Create powerful new applications quickly and effortlessly
• Incorporate third party components in the created applications
• Choose the best components from different vendors
• Extend the functionality of existing applications by simply adding new components
• Combine components written in different environments and languages in a single solution
• Carry out the development with the simplest and most common tools (such as Visual Basic for Applications)

On the other hand, manufacturers of Data Mining components will never have to drive themselves crazy again trying to build a complete vertical decision support applications on their own; they were not very good at that anyway. Now, with a sigh of relief, they can concentrate on creating well-documented COM-based Data Mining components, leaving their incorporation into vertical solutions to those professionals who had been in this business since the Stone Age. And Data Mining component developers should get ready to welcome their former competitors as their new customers.

Fig. 3 Integrated Direct Marketing and Campaign Management solutions.

Summarizing, the utilization of COM-based Data Mining algorithms at the data modeling step of the integrated Direct Marketing chain serves the needs of all interested parties. Users obtain a flexible architecture of individually purchased, powerful Data Mining algorithms, integrated seamlessly into their DSS. Developers and integrators of vertical applications can hire cheaper VBA programmers to replace their C++ gurus and concentrate on perfecting the user interface, while outsourcing for efficient machine learning components. And developers of Data Mining algorithms can sell their components in a broader market, striking new strategic partnerships with the developers of the best Direct Marketing solutions. Business Intelligence solutions become flexible, easily upgradable, and cost-efficient. A combination of these features boosts the customized value of applications built with the help of COM technology, at the same time expanding the market for the created components to the broadest possible level.

Finally, last but not least, there is yet another very important group of people that benefits from the new technology – the targeted audience. New integrated Direct Marketing solutions will provide for a better segmentation and targeting of prospects and offers in a marketing campaign. We all should expect to receive fewer pieces of junk mail while getting promotions for things we really need. That’s how we are going to know when the new technology begins to work indeed!