Maciek

IN PROGRESS, rough draft...
The Internet has brought upon radical societal changes, it is the root driver in the shift from the industrial to the information age. However, in an environment of information overload, it becomes harder to find items of interest or relevance. This poses new challenges and opportunities for e-commerce companies. The big question is how do we guide consumers towards products or services which they may find of interest? E-commerce companies such as E-bay, Amazon, and Netflix have responded through personalization. By using recommendation based technologies, their websites try to predict, with varied accuracy, what may be of interest to a customer and draw attention to it. --- **Introduction needs more elaboration... In progress**

How it works / History
The most common technology used in e-commerce recommendation is collaborative filtering (Im). In collaborative recommendation, the user receives recommendations based on what other people with similar preferences and tastes rated well in the past. For example, in movie recommendations, the system tries to find other users which rate movies similarly, and then only recommend the movies which were most liked by these peers. The ratings are represented as numerical values (Kim) which are used by complex mathematical algorithms to predict items to users. These algorithms have been receiving significant attention by scholars making it an active research field (for example: Lee; Jeong; Kim). GroupLens, a research group at the University of Minnesota was one of the first to experiment with collaborative filtering by developing “GroupLens”, a Usenet article recommender in 1992 (Miller). However GroupLens is more famous for its web-based movie recommender at movielens.org. By making the website open to everyone, GroupLens is able to collect real data and use it to test and improve collaborative filtering algorithms. The dataset is available free of charge to anyone doing research in recommendation algorithms, and needs rich data to test (About GroupLens).

System Components
The core components of an e-commerce recommendation system are the users, datasets, algorithms, and an e-commerce application. Before implementing a recommendation engine, there needs to be an active online store in place. A company can choose whether to build their own e-commerce application, choose a pre-existing open source one (for example OS Commerce, Zen Cart), or purchase a commercial system (for example CubeCart, 3D-Cart). The dataset can be divided into two components: products and user data. The product dataset should contain all products for sale, their attributes, quantity, prices, etc. and is populated by authorized users who manage content within the system. Users will be key in populating a dataset with their preferences, product ratings, purchase behaviours, etc. through interacting with the system. A recommendation engine can then apply algorithms which analyze both data sets, and output personalized recommendations. For example, in collaborative filtering, the algorithm finds a set of customers with similar preferences, known as ‘neighbors’, then tries to compute correlations to products that may be of interest to this group, and recommends them to the individual user (Kim). It is important to realize the core components discussed cannot function on their own, they need to be supported by both software and hardware architectures.

Software req
The key software required for an e-commerce recommender system includes a HTTP daemon and database software. A HTTP daemon is software installed on a server which transmits content to users such as web pages over the Hyper-Text Transfer Protocol. In other words, it provides a server the capability of hosting websites. Clearly, without such software, the e-commerce application would not be able to transmit content to users’ web browsers. The e-commerce application needs access to operational databases which store and update product and user datasets. It can be assumed that at minimum, there should be a separate database for products and users. These databases update data based on the transactions which occur through the e-commerce application. For example, if a customer purchases a product, the inventory level is updated in the product database, and a record of the transaction is added to the user database. Similarly, when a user rates a product, that rating is added to the user database. Such databases perform Online Transaction Processing since they only gather and update information. In order to perform an analysis on this data, the data needs to be transferred into a data warehouse. This is another type of database which gathers information from operational databases and manipulates the information to support decision making (Haag). We can easily assume from this that the recommendation algorithm would be applied in the data warehouse.

Hardware req
Hardware requirements will vary depending on size and traffic volume generated by the e-commerce site. At minimum, a recommender system would require a server capable of running a database and web server software. To point out the obvious, the server would need access to the Internet. Hardware such as Ethernet adapters, routers, switches and hubs enable such access, and network several servers together. The quantity and specifications of each hardware unit would depend on the size of the system. Generally, a large scale e-commerce operation will require several servers to share the load in order to function effectively and efficiently. It is important to realize that a recommendation engine is just one component of an e-commerce system; therefore the hardware specifications would be tailored towards the requirements of the system as a whole.

Providers / Labour req
Labour requirements will also vary depending on the type of solution a business chooses. Generally, there are four different paths to choose. The business could build its own, implement an existing open source engine, purchase a commercial one, or subscribe to a remote service. Building a recommendation engine in-house would require significant labour, and may produce weaker results then other pre-made solutions. Generally, it is not recommended to re-invent the wheel. However, if a significant investment is made coupled with a competent development team, the company might produce an effective system, and keep the algorithms secret. Amazon.com recommendations are powered by a system custom built by the company. The largest open source recommendation engine is Apache Mahout, in part because Yahoo is its largest contributor (Apache Mahout). Mahout is a highly scalable collaborative filtering based engine built on top of Hadoop, a Java developers framework (Apache Mahout). The advantage of implementing Mahout is that it is constantly evolving since it is open to anyone wishing to contribute. Also, since it is open source, it can be customized to specific or novel business requirements. However, even though Mahout is very well documented, and saves the burden of writing your own code, it still requires a skilled team to integrate this engine seamlessly into the business’ existing systems. Support at the moment is limited to just documentation and a wiki as this is still a young project (Apache Mahout). A company may choose to purchase a commercial system such as Pervasive DataRush Recommender System. This is also a collaborative filtering engine which is cross-platform (PDR Offers). According to the manufacturers’ website, this system is highly scalable, and offers a “full library of custom operators for parallel programming” (PDR Offers). Pricing is not published, we assume it is because it will vary depending on the size of the business, additional requirements, and support options. The product seems to be supported through telephone technical support available 24/5 (PDR Offers). The simplest solution would be to subscribe to a remote recommender system such as Isotope. By using such a system, the labour is passed on to the service provider. All that the subscriber needs to do is insert a small amount of javascript in to their e-commerce application, and the service does the rest (Istobe). This may be a suitable solution for small and medium sized business since the overall investment is low, and can be integrated in minutes (Istobe). However the major drawback is that data about your customers behaviours is stored by a third party which may limit the company from doing further data mining with it.

Sources Cited:
//About GroupLens.// GroupLens Research, n.d. Web. 22 Mar. 2010 

//Apache Mahout - Overview//. Apache Software Foundation, n.d. Web. 22 Mar. 2010 

Haag, Stephen, et. al. //Management Information Systems: For the Information Age. Fourth Canadian Edition.// Canada: McGraw-Hill Ryerson 63-93, 2009. Print.

Istobe - Recommendation Engine for eCommerce. //Ignite Analytics, Inc.//, n.d. Web. 23 Mar. 2010 

Jeong, B., et. al. "Improving memory-based collaborative filtering via similarity updating and prediction modulation". //Information Sciences// 180.5 (2010): 602-612. Print.

Lee, Seok Kee, et. al. "Collaborative filtering with ordinal scale-based implicit ratings for mobile music recommendations". //Information Sciences// 180.11 (2010): 2142-2155. Print.

Miller, B.N., J Riedl,& J Konstan. "Experiences with GroupLens: making Usenet useful again". //Proceedings of the USENIX 1997 Annual Technical Conference//, Anaheim, CA, USA. 1997. Conference.

PDR Offers a Recommender System for E-commerce. //Pervasive Software Inc.//, n.d. Web. 23 Mar. 2010 