Co-author: Khin Radanar Pyae Phyo 

  Estimately 1.8 billion people across the world are using online shopping. Knowing how many products will be sold out without waste is one of the biggest e-Commerce superpowers. Currently COVID pandemic is spreading all over the world.  People are shifting buying habits and the lockdown has even hit boomers by forcing them to shop online retailers. That’s why e-commerce becomes essential among the living environment and hits the consumer’s needs. Since E-commerce is ready 24/7 without the aid of the shop assistant all day, that’s the good point for customers.

   Customer retention is important for every business, but it’s especially significant for e-commerce organizations. After all, with an online business, you lack the opportunity to connect with customers in person. This inability to build face-to-face relationships can cause customer churn. So, if your goal is to increase your revenue, then customer retention is the most worthy thing to invest in. 

   The exponential growth of Artificial Intelligence is crucial for businesses sectors which deliver better customer experience and lead to market’s revenue. Many organizations are integrating with such technologies as AI (Artificial Intelligence) and ML (Machine Learning) to get more revenue and make marketing strategies efficient. By 2020’s survey, it is announced that 95% of customer interactions will be managed with AI.

Figure 1: Percentage of using online shopping

   As the technologies are increasing towards the future, Artificial intelligence is applied in e-commerce to predict customer’s choices. This will give your customers a personalized customer experience thanks to accurate recommendations and help your company increase sales growth. There are many ways to integrate e-commerce business  with AI technologies. Among them, we will highlight the recommendation system in this blog as they reduce transaction costs of finding and selecting items in an online shopping environment.

Recommender System

   Recommender systems find out the matching between user and item and impute the similarities between users and items. There are majorly six types of recommender systems: Collaborative Recommender system, Content-based recommender system, Demographic based recommender system, Utility based recommender system, Knowledge based recommender system and Hybrid Recommender system. There is a reason why we chose the hybrid Recommender system, the combination of content-based filtering and collaborative filtering and we will explain further.

Hybrid Recommendation System

   It is important to have the right model to reach a better accuracy. Of course, there might be no doubt why we decided to choose this hybrid recommendation as this is a powerful one among recommendation engines. 

   Firstly, we would like to talk about the content-based filtering part. It saves the cold-start problem that occurs when a new user comes in or a new item is added. Content-based filtering utilizes the vector space model implementation and  TF-IDF Vectorizer which are used to determine the relative importance of a document. There is a limitation in content-based as it does not depend on user preferences. 

   In item-based collaborative filtering, we can get a user-item rating matrix where we can know whether the user gives the rating values or not.  However, there is a cold start problem in item-based collaborative filtering recommendation. 

   To avoid both of the limitations occurring in item-based and content-based recommendation, we aim to combine them to get strong recommendations for a user.  And, we use the cosine similarity matrix to calculate the similarity scores between items. After we combine two two vector space models, we have programmed to rank similarity scores in descending order. In the future, our Nexidea AI Research team is planning  to continue a deep learning recommendation project to handle big data issues.

Training or Result

   Enhancing a model performance can be challenging for an AI Engineer. Everyone will try a better model and all the strategies and algorithms to have the best accuracy.  The basic steps we have to look at are how much data we have, how we will divide training and test data, feature engineering, missing data filtering and parameter tuning and so on. While in the beginning of recommender systems, it is foremost to find explicit similarity in people and products. 

   We have tested with the MovieLens Dataset which is published  from the MovieLens web site (https://grouplens.org/datasets/movielens/).  In this recommendation, the system predicts the top 5 movies for a user that we can make a recommendation.  Since this is a simple hybrid recommendation, the training loss or matrix loss can’t be calculated. However, according to the nature of  content-based and item-based collaborative filtering, we forecast the accurate recommendation for each user by combining advantages of the two models.

Figure 2: Top 5 movies recommendation to a user

Summary

   Hybrid Recommender systems ( content/collaborative ) can be the most effective solution for building a recommender system.  Although this hybrid recommender system is good enough to be used in business sectors, there might be some problems such as the ramp up problems since both the techniques need a database of ratings and the training time will be longer as the data size increases. 

   We always have to update the input data as the data size becomes bigger and we should set the training time in the formal way when the new data comes in. However,the combination of AI and Big Data solutions has provided a platform for big data services companies to earn skills as well as customer support. Neural Networks and Deep Learning have been a popular trend in many different fields, and it appears that they are helpful for solving recommendation system problems.

   In future, our nexidea AI Research team aims to apply a deep learning model and PySpark with our hybrid recommendation system. Our Nexidea AI Research team is leading to new technologies and we hope to provide advanced automations to our customers using a wide range of digital services. If you are interested in this recommendation field, we are welcomed to collaborate with our project and enjoy this.