Expert Consultant in Search, Machine Learning and Distributed Architecture
We provide consulting to solve difficult Search, Discovery and Machine Learning problems. We are experts at applying Elastic search and Solar to Geo-contextual search in very large catalogs and consumer referral services. We are well proven consultants who consistently deliver innovative solutions to difficult problems.
Our ability to operate at both the business and techlogy level allows us to improve execution velocity for initiatives that are important to business executives. This can help IT teams deliver fast and get ahead of the delivery curve.
We have worked in senior levels so we understand business problems and can help executives meet their goals
Great Results at HP, Amazon, Corbis and NCR
One of our recent clients is a Leading consumer services site that matches service providers with consumers. For some clients we delivered:
- Completely re-engineered search pipeline using Elastic Search and Node.js
- New schema optimized for underlying requirements.
- New data delivery pipeline
- Advanced features executives had been requesting for years.
- Enabled business initiatives to improve growth
- Advanced Geo Contextual matching,
- Modification of search results based on User satisfaction statistics
- Dramatically improved Search and phrase matching
- Language based semantic matching
- New data in search mined from multiple data sources.
- Faster data availability from time of change.
- Robust fault tolerant availability
- Hosted entirely in AWS
- 2 to 10 X faster search responses.
- No single point of failure
- New search and browse paths
- Advanced REST API.
Show Users the Best Offer for their needs
The main challenge for product or SP search is that the end user supplies a limited amount of data in the form of a search string and it is our job to deliver a page or set of offers that thrill them and cause them to click the buy button. When search is done poorly you can deliver the mis-matched offers or the site can respond so slowly the user gets bored and goes someplace else. This means internal Search is a critical enabler for revenue growth because it helps the buyer find what they want or need.
We deliver expertise applying statistical predictive engines to improve the search experience. This allows us to mine the wealth of data in many companies and convert it into statistical input useful for search engines to deliver better results.
There are lots of people who claim BI expertise and many others who understand search engines. I bring the two together to deliver real and actionable improvement to search results. I quite often find the essential data by spelunking through existing repositories so we find useful data most will never consultants would never see. The quite often requires writing custom extract and analysis jobs beyond the bandwidth, capacity or focus of internal BI.
Listen to Business Deliver Technology
Our experiences gives us the the ability to listen to business users and derive requirements that will meet their underlying needs. This is important because we quite often find that what the executive really needs gets lost in the translation leaving both the executive and the technology team frustrated.
We then turn around and engineer the technology to deliver this solution and guide them all the way through deployment when needed. This is radically different than most strategic consultants who deliver lots of advice can not deliver deployable systems.
We provide exert level Consulting Services for Elastic Search, Product Discovery, Product Search, SP Search, Geo-contextual Search with Statistical Engines.
Deliver real results fast
Our customers tell us that we have allowed them to reach goals they had struggled with for years. At one client we took a search experience where 80% of all requests took over 8 seconds to 98% of all requests responding in less than 1.5 seconds. We did this while dramatically improving quality of search results and had the entire new architecture live in less than a year. After this change the search pipeline went from one of the leading causes for site outages to being responsible for zero outages.
High Performance, High Availability
Many of our clients start with search and browse experiences that are too slow. this limits their ability to deliver highly responsive next generation web experiences such as single page RIA (Rich Internet Applications). They also commonly struggle with performance degradation as they add features. It is quite amazing when we walk into clients who have invested dozens of man years in complex architecture to work around slow search when it is much less expensive for us to solve the root performance problems.
One trick for solving performance problems is enabled by architecture that allows rapid incremental scalability. We are experts in this technique and use it often. Quite often the underlying problems are also caused by improper modeling of the data so we may need to re-engineer the search schema. In other instances secondary web service calls and off box data dependencies can kill performance. We focus on identifying the key performance killers and have well proven techniques available to solve each of them.
We focus heavily on using well proven distributed techniques to deliver of search with: High Performance, High Availability, High Reliability. There are many claims of self healing architectures but best companies use well proven techniques that do not depend or believe many of these near magical claims. We love new technology but we have a goal that the our system should never be responsible for performance problems or outages so we tend to favor architectures well proven to deliver this over multiple years. In some instances the simplicity offered by the emerging technology can make it worth trading some resilience for lower operational costs but when Site Availability is crucial the well proven techniques are still the best.
High performance code modules
In many cases we can solve performance problems in the search engine with new schema, different queries or modified query paths. In other situations we can solve problems with clever architecture such as parallel execution using techniques like Hadoop.
Micro Optimizing the critical 5%
There are times when high performance code is absolutely essential and there are even a few situations where re-writing critical code in high performance C is needed.
In most cases we can take average C#, Java, or Python code and obtain 2X to 20X improvements in execution speed.
The Critical C layer when really needed
When you really need to maximize the use of CPU power it is difficult to compete with porting the critical logic to C and organizing data for very high speed access.
Using these techniques we have taken projects that took required over $200,000 worth of clustered hardware over 1/2 hour to execute to where the same thing could run in under 30 seconds on $2,000 worth of hardware. Since this operation needed to run thousands of times a day this delivered a huge savings while delivering critical data to the enterprise faster than ever before.
We have also taken a job that consumed over $20,000 and 3 days on a AWS cluster to where it could in 2 days on under $5,000 worth of hardware. This one was a little more challenging due to data volumes but we were able to re-engineer the critical code to run at high speed in a single path through a single core and then split the work between 8 cores. We still did not reach fully optimized where we could likely have squeezed another 300% improvement using CPU specific operands because what we delivered was good enough.
To protect the confidentiality of our clients I was forced to generalize some of the work above. I apologize for being unable to list more details but our clients consider the work we do for them as strategic competitive enablers that they treat as trade secrets. Our work is quite often the anchor enabler for new key business initiatives. It is great to work at a level where your work is valued all the way up to the CEO but since we work mostly for companies with over 300 million in sales they also jealously guard their competitive advantages.