The Challenge

Out-of-the-box spell checkers were inadequate for the client’s needs as they are trained on general English language corpora rather than text specifically related to image search. This meant that many misspelled searches were not being corrected and that some searches were also being wrongly corrected. For example, the misspelling cilent was changed to silent rather than the intended client, leading to irrelevant search results, while misspellings that were almost identical to another word, like pork lions for pork loins, were not being corrected at all. This meant that the appropriate images could not be retrieved.

The client identified these issues as a substantial cause of lost revenue and needed a fast and effective resolution. The solution required designing, implementing and assessing a context-aware spell checker that would reliably correct the specific type of language used in image search queries.

 
 

Procogia’s Approach

We developed natural language processing (NLP) and machine learning (ML) solutions that presented customers with better image search results.

 

  • We used a client-specific corpus of historical search queries as the basis for a custom spell check dictionary, ensuring our product could effectively handle the types of language search customers were using
  • We wrote a spell checker in Python implementing NLP concepts and techniques, including Damerau-Levenshtein distance, phonetic encoding, grammatical knowledge, and probabilistic typo identification
  • We used both manual evaluation and parameter tuning to iteratively optimize the model
  • To facilitate internationalization beyond English, we ensured our algorithms could be applied to any alphabetic language.

The Results

 

  • A/B testing found a 39% higher purchase rate for searches corrected by the spell checker vs. the control group
  • Our product was 50x less likely to give false corrections than the out-of-the-box model the client was using
  • ProCogia’s spell checker now handles all searches on the client’s US site and is currently being extended to non-English languages including French and German
  • The spell check solution has resulted in customers successfully finding their desired images and therefore increased revenue for the client.

 
 

Services Used

Data Science

We use open source technology to leverage the full potential of your data. Predictive and prescriptive results are actioned using AI and Machine Learning (ML).

Data Consultancy

We provide data consultancy to organizations to optimize your investment in people, processes, and technology. This is typically through data strategy engagements, roadmaps, transformations, and independent technology advice.

Related Blogs

Contact
 
 

Let’s Connect

What can we help you with?
T: +1 425-624-7532

Alternatively, simply fill in this form and we’ll be in touch.