Comments on: Latent Semantic Indexing can improve your WordPress search results http://elliottback.com/wp/latent-semantic-indexing-can-improve-your-wordpress-search-results/ Internet & Technology Wed, 08 Feb 2012 10:01:08 +0000 http://wordpress.org/?v=2.9.2 hourly 1 By: Elliott Back http://elliottback.com/wp/latent-semantic-indexing-can-improve-your-wordpress-search-results/#comment-1521253 Elliott Back Sun, 27 Apr 2008 10:59:47 +0000 /?p=766#comment-1521253 There are actually continuous models. You can recompute an approximate SVD of the Term-document matrix as new stuff comes in without too much work, it just won't be as accurate... There are actually continuous models. You can recompute an approximate SVD of the Term-document matrix as new stuff comes in without too much work, it just won’t be as accurate…

]]> By: Of Wordpress Search on iface thoughts http://elliottback.com/wp/latent-semantic-indexing-can-improve-your-wordpress-search-results/#comment-399955 Of Wordpress Search on iface thoughts Wed, 31 Jan 2007 09:26:22 +0000 /?p=766#comment-399955 [...] Elliott points towards Latent Semantic Indexing (LSI) for improving the search. It might not be a viable option today, but it might be future of the search industry. Natural Language Processing is already being used in certain search functionalities. [...] [...] Elliott points towards Latent Semantic Indexing (LSI) for improving the search. It might not be a viable option today, but it might be future of the search industry. Natural Language Processing is already being used in certain search functionalities. [...]

]]> By: David W. Lee http://elliottback.com/wp/latent-semantic-indexing-can-improve-your-wordpress-search-results/#comment-51583 David W. Lee Fri, 12 May 2006 14:55:32 +0000 /?p=766#comment-51583 Typo in my previous message: I mean you "cannot" index one entry by itself.. =) Typo in my previous message: I mean you “cannot” index one entry by itself.. =)

]]> By: David W. Lee http://elliottback.com/wp/latent-semantic-indexing-can-improve-your-wordpress-search-results/#comment-51582 David W. Lee Fri, 12 May 2006 14:54:13 +0000 /?p=766#comment-51582 I studied LSI in my Masters and I am, too, looking for a commercial implementation of it. One problem with LSI (based on the proposed implementation in the textbook) is that the indexing is not continuous. You can add an entry in your database and have it indexed just by itself. You have to re-index the entire database, which is pain in the butt. And it might also cause the search results to change significantly from one build to another. So, Edward, if you can find a solution to this (making the indexing continuous), then you will be the next Bill Gates. For some applications this may not be a problem. But in general, this is bad. Another problem with LSI is the algorithm that finds the "nearest neighbors". I don't think people have a good solution to that yet. But in terms of searching, the time and space complexity is not so much of a concern. For Google, however, is the immense number of pages and keywords they have to index. The complexity go up at least at 2nd order polynomial rate. We may not have enough atoms in the universe to store all that information. I studied LSI in my Masters and I am, too, looking for a commercial implementation of it.

One problem with LSI (based on the proposed implementation in the textbook) is that the indexing is not continuous. You can add an entry in your database and have it indexed just by itself. You have to re-index the entire database, which is pain in the butt. And it might also cause the search results to change significantly from one build to another. So, Edward, if you can find a solution to this (making the indexing continuous), then you will be the next Bill Gates.

For some applications this may not be a problem. But in general, this is bad.

Another problem with LSI is the algorithm that finds the “nearest neighbors”. I don’t think people have a good solution to that yet. But in terms of searching, the time and space complexity is not so much of a concern.

For Google, however, is the immense number of pages and keywords they have to index. The complexity go up at least at 2nd order polynomial rate. We may not have enough atoms in the universe to store all that information.

]]> By: Elliott Back http://elliottback.com/wp/latent-semantic-indexing-can-improve-your-wordpress-search-results/#comment-1726 Elliott Back Fri, 10 Jun 2005 00:43:45 +0000 /?p=766#comment-1726 LSI is a patented technique, and is difficult to implement properly. Don't even try it unless you read the appropriate academic papers first. I suggest http://scholar.google.com as a first resource. Even google doesn't use LSI, at least yet, because of its computational complexity. LSI is a patented technique, and is difficult to implement properly. Don’t even try it unless you read the appropriate academic papers first. I suggest http://scholar.google.com as a first resource. Even google doesn’t use LSI, at least yet, because of its computational complexity.

]]>