MyLiveSearch: An unorthodox approach
By Ashish Singh | September 4th, 2007 at 08:44 am ET
Melbourne based MyLiveSearch has been on the shiny side of the coin as far as publicity is concerned. It is one of those few startups that have been able to arouse curiosity all around since last few months. It had put loud claims about serving more relevant search results than google, and ability to search through dynamic web, and was touted as google killer in an Australian daily.
Having developed essentially a next generation meta search engine, they operate through a Firefox or IE extension. It takes the results from a search engines as a starting point, and then builds its search over those results. Alternatively, one can also set some other website as starting point. It triggers a crawler to fetch links it reads from the starting point. Armed with good ajax, it displays the search results in browser window in a progressive fashion. A slider can be used to adjust the search flavor between web and news.
Following is the summary and observations derived through reverse engineering. A search session is started by sending a request to mylivesearch server along with the search query. Then it sends a request to combination of msn live, google search, google news, yahoo search and yahoo news servers with same search query. Based on the response from these search resources it aggregates a list of urls, and queries for cached pages on google for a subset of these urls. It sends this url list again back to tableurl script back on mylivesearch servers, and recieves some intelligence(may be a set of expanded urls) back. This intelligence then starts a spider from the host and a substantial deeper crawl is initiated. However, majority of this crawl represents the same urls received from the initial queries on search resources or starting points. It crawls all outlinks, and other image dependencies as well. This is important as this enables them to display all outlinks from a particular webpage, in search results. All intelligence in computing search result ranking etc, resides on host machine itself. One observation was the brute force processing of dynamic web urls. It substituted the search query blindly in all parameters of a web page. e.g twitter urls could be seen as http://twitter.com/sessions?username_or_email=search_query&commit=Sign_In! and wordpress urls has search query inserted for all parameters like author, email, url etc.
Clearly, if one chooses to use existing search engines as starting point, then search results are no better than standalone results delivered from those engines. In case one chooses a different starting point, then one would get get results from only the part of web connected to the starting point, and might miss important results from other part. The optimal results would be delivered from someone who has seen a substantial part of web. MyLiveSearch follows a greedy strategy of serving search results, which would be far different from optimal most of the times (until one has an idea that what is being searched for, lies within an expected distance to starting point).
The factors that perhaps inspired this kind of product are increasing processing power and reducing storage costs. No doubt this has brought focus to a new search paradigm, which will improve with time like any other technology. MyLiveSearch has still to improve to keep its promises it made to people.
on September 4th, 2007 at 5:35 pm
Some food for thought..
try actually to see more than a 1000 results from google, then its says “sorry, can not show you anymore”. While Mylivesearch can actually get 1000 plus results on your screen to access or more if you wish.That is a fact.
Also, try actually looking at the “cache” button located next to each result from google and you will see that most of the results are days, weeks and sometimes months old.. That is a fact.
If you want to be able to search dynamic sites i.e.ebay etc, this tool is perfect which can show you up to 500 times more on the web than google and others… Fact.
So, as far as a plug in and using your computer resources for a few moments (which you can still use word or browse another web page anyway) so you can receive all these benefits, i think it something people need to realise and get out of the “Google Box” and see how the web really is and what it can really do.. Interesting..
on September 6th, 2007 at 1:04 am
Thanks for the feedback.
My comments inline
>>Some food for thought..
>>try actually to see more than a 1000 results from google, then its says %u201Csorry, can not show you anymore%u201D. While Mylivesearch can actually get 1000 plus results on your screen to access or more if you wish.That is a fact.
i never go beyond 10 results. How many people see 1000 results? 0.
>>Also, try actually looking at the %u201Ccache%u201D button located next to each result from google and you will see that most of the results are days, weeks and sometimes months old.. That is a fact.
Try advanced search option in google, It will help you out for searching in latest index.
Google’s index updates really fast. and it is comfortable for me to use it instead of paying my ISP for all crawl that mylivesearch does from my box. So, Real time isnt too compelling for me to use it. As long as serving real time results are concerned, mylivesearch relies on google results page to start itself, and then serves same pages to me in search result. Consider clicking a link on google result page. It takes you to fresh page only.
>>If you want to be able to search dynamic sites i.e.ebay etc, this tool is perfect which can show you up to 500 times more on the web than google and others%u2026 Fact.
Search for anything putting techcrunch as starting point, and check the requests sent to techcrunch. You’ll understand what i mean.
>>So, as far as a plug in and using your computer resources for a few moments (which you can still use word or browse another web page anyway) so you can receive all these benefits, i think it something people need to realise and get out of the %u201CGoogle Box%u201D and see how the web really is and what it can really do.. Interesting..
Well i agree, but i do not appreciate sending 100 GET requests from my machine to show just two links from readwrite web in the search results page, and both of them duplicates!(It actually happened)
Getting out of google box is what i want as well. But then if you believe that you can defeat them by launching a parallel search engine, which u expect people to use instead of google, then it is not going to happen anytime soon. One should try different approaches other than this one.
As i said, they are an interesting product. But, i also feel they need to improve, which is true for any product. There is no single perfect product. You have to evolve, else you are done.