Welcome to dbForumz.com!
FAQFAQ    SearchSearch      ProfileProfile    Private MessagesPrivate Messages   Log inLog in

Quoted literal strings won't force a phrase match

 
   Database Forums (Home) -> Full Text RSS
Next:  Dimension question  
Author Message
Afroblanco

External


Since: Jan 18, 2008
Posts: 1



(Msg. 1) Posted: Fri Jan 18, 2008 9:49 am
Post subject: Quoted literal strings won't force a phrase match
Archived from groups: microsoft>public>sqlserver>fulltext (more info?)

Hello all,

From what I've read, SQL Server is supposed to do a phrase match when
you do a full text search that contains quoted literal strings. So,
for example, if I did a full text search on the phrase "time out" and
I put it in quotes, it's supposed to search for the full phrase "time
out" and not just look for rows that contain the words "time" or
"out." However, this isn't working for me.

Here is the query that I'm using :

SELECT *
FROM Content_Items ci
INNER JOIN FREETEXTTABLE(Content_Items, hed, '"time out"') AS ft
ON ci.contentItemId = ft.[KEY]
ORDER BY ft.RANK DESC

What's it's doing is this : it's returning a bunch of rows that have
the words "time" or "out" in the column called hed. It's also
returning rows that have the full phrase "time out", but it's giving
those rows the same rank as rows that only contain the word "time."
In this case, that rank is 180.

Is there anything else I should be doing in my query, or is there some
configuration option I should have turned on?

Thanks.

 >> Stay informed about: Quoted literal strings won't force a phrase match 
Back to top
Login to vote
Lepidopterist

External


Since: Jan 18, 2008
Posts: 2



(Msg. 2) Posted: Fri Jan 18, 2008 12:48 pm
Post subject: Re: Quoted literal strings won't force a phrase match [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)

Ok, I've made some progress on this problem. Apparently SQL Server is
ignoring noise words in my phrase match.

For example, I ran this query :

SELECT *
FROM Content_Items ci
INNER JOIN FREETEXTTABLE(Content_Items, hed, '"time capsule"') AS ft
ON ci.contentItemId = ft.[KEY]
ORDER BY ft.RANK DESC

And it did exactly what it was supposed to do, since neither "time"
nor "capsule" is a noise word.

My impression was that noise words aren't stripped out of a full text
search if the search phrase is a quoted literal. Thus, my search for
"time out" should look for the full phrase "time out", and not just
the word "time."

Does anybody know why SQL Server is removing my noise word from the
phrase match?

On Jan 18, 12:49 pm, wrote:
> Hello all,
>
> From what I've read, SQL Server is supposed to do a phrase match when
> you do a full text search that contains quoted literal strings. So,
> for example, if I did a full text search on the phrase "time out" and
> I put it in quotes, it's supposed to search for the full phrase "time
> out" and not just look for rows that contain the words "time" or
> "out." However, this isn't working for me.
>
> Here is the query that I'm using :
>
> SELECT *
> FROM Content_Items ci
> INNER JOIN FREETEXTTABLE(Content_Items, hed, '"time out"') AS ft
> ON ci.contentItemId = ft.[KEY]
> ORDER BY ft.RANK DESC
>
> What's it's doing is this : it's returning a bunch of rows that have
> the words "time" or "out" in the column called hed. It's also
> returning rows that have the full phrase "time out", but it's giving
> those rows the same rank as rows that only contain the word "time."
> In this case, that rank is 180.
>
> Is there anything else I should be doing in my query, or is there some
> configuration option I should have turned on?
>
> Thanks.

 >> Stay informed about: Quoted literal strings won't force a phrase match 
Back to top
Login to vote
Russell Fields

External


Since: Feb 21, 2007
Posts: 457



(Msg. 3) Posted: Sun Jan 20, 2008 1:42 pm
Post subject: Re: Quoted literal strings won't force a phrase match [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)

A noise word is always a noise word. Noise words are applied to the
building of the index, so the full-text search has nothing to find.
Therefore, if you change the noise word list, you must rebuild the index
before you can search for the former noise word. (It is common to run with
either a single blank or a single nonsense word in the noise word file, so
as to get no noise words.)

Of course, you can do a string search for '%time out%' in addition to the
full-text query.

RLF

"Lepidopterist" wrote in message

> Ok, I've made some progress on this problem. Apparently SQL Server is
> ignoring noise words in my phrase match.
>
> For example, I ran this query :
>
> SELECT *
> FROM Content_Items ci
> INNER JOIN FREETEXTTABLE(Content_Items, hed, '"time capsule"') AS ft
> ON ci.contentItemId = ft.[KEY]
> ORDER BY ft.RANK DESC
>
> And it did exactly what it was supposed to do, since neither "time"
> nor "capsule" is a noise word.
>
> My impression was that noise words aren't stripped out of a full text
> search if the search phrase is a quoted literal. Thus, my search for
> "time out" should look for the full phrase "time out", and not just
> the word "time."
>
> Does anybody know why SQL Server is removing my noise word from the
> phrase match?
>
> On Jan 18, 12:49 pm, wrote:
>> Hello all,
>>
>> From what I've read, SQL Server is supposed to do a phrase match when
>> you do a full text search that contains quoted literal strings. So,
>> for example, if I did a full text search on the phrase "time out" and
>> I put it in quotes, it's supposed to search for the full phrase "time
>> out" and not just look for rows that contain the words "time" or
>> "out." However, this isn't working for me.
>>
>> Here is the query that I'm using :
>>
>> SELECT *
>> FROM Content_Items ci
>> INNER JOIN FREETEXTTABLE(Content_Items, hed, '"time out"') AS ft
>> ON ci.contentItemId = ft.[KEY]
>> ORDER BY ft.RANK DESC
>>
>> What's it's doing is this : it's returning a bunch of rows that have
>> the words "time" or "out" in the column called hed. It's also
>> returning rows that have the full phrase "time out", but it's giving
>> those rows the same rank as rows that only contain the word "time."
>> In this case, that rank is 180.
>>
>> Is there anything else I should be doing in my query, or is there some
>> configuration option I should have turned on?
>>
>> Thanks.
>
 >> Stay informed about: Quoted literal strings won't force a phrase match 
Back to top
Login to vote
Hilary Cotter

External


Since: Oct 09, 2005
Posts: 78



(Msg. 4) Posted: Wed Jan 23, 2008 9:02 am
Post subject: Re: Quoted literal strings won't force a phrase match [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)

A couple of points, in SQL 2005 if your search phrase is wrapped in
double quotes it is a literal search for both FreeText and Contains.
In SQL 2000, wrapping your search phrase in double quotes for a
FreeText search has no effect - it is stemmed.

The second point it that if SQL FTS detects the presence of noise
words in your search phrase it throws them away but respects the
positional placement of them.

So a search a FreeText search on University of California (where of is
a noise word) would result in hits to University of California,
University at California and University in California ranked higher
than University California or California University.


On Jan 20, 1:42 pm, "Russell Fields" wrote:
> A noise word is always a noise word.  Noise words are applied to the
> building of the index, so the full-text search has nothing to find.
> Therefore, if you change the noise word list, you must rebuild the index
> before you can search for the former noise word.  (It is common to run with
> either a single blank or a single nonsense word in the noise word file, so
> as to get no noise words.)
>
> Of course, you can do a string search for '%time out%' in addition to the
> full-text query.
>
> RLF
>
> "Lepidopterist" wrote in message
>
>
>
>
>
> > Ok, I've made some progress on this problem.  Apparently SQL Server is
> > ignoring noise words in my phrase match.
>
> > For example, I ran this query :
>
> > SELECT *
> > FROM Content_Items ci
> > INNER JOIN FREETEXTTABLE(Content_Items, hed, '"time capsule"') AS ft
> > ON ci.contentItemId = ft.[KEY]
> > ORDER BY ft.RANK DESC
>
> > And it did exactly what it was supposed to do, since neither "time"
> > nor "capsule" is a noise word.
>
> > My impression was that noise words aren't stripped out of a full text
> > search if the search phrase is a quoted literal.  Thus, my search for
> > "time out" should look for the full phrase "time out", and not just
> > the word "time."
>
> > Does anybody know why SQL Server is removing my noise word from the
> > phrase match?
>
> > On Jan 18, 12:49 pm, wrote:
> >> Hello all,
>
> >> From what I've read, SQL Server is supposed to do a phrase match when
> >> you do a full text search that contains quoted literal strings.  So,
> >> for example, if I did a full text search on the phrase "time out" and
> >> I put it in quotes, it's supposed to search for the full phrase "time
> >> out" and not just look for rows that contain the words "time" or
> >> "out."  However, this isn't working for me.
>
> >> Here is the query that I'm using :
>
> >> SELECT *
> >> FROM Content_Items ci
> >>     INNER JOIN FREETEXTTABLE(Content_Items, hed, '"time out"') AS ft
> >> ON ci.contentItemId = ft.[KEY]
> >> ORDER BY ft.RANK DESC
>
> >> What's it's doing is this : it's returning a bunch of rows that have
> >> the words "time" or "out" in the column called hed.  It's also
> >> returning rows that have the full phrase "time out", but it's giving
> >> those rows the same rank as rows that only contain the word "time."
> >> In this case, that rank is 180.
>
> >> Is there anything else I should be doing in my query, or is there some
> >> configuration option I should have turned on?
>
> >> Thanks.- Hide quoted text -
>
> - Show quoted text -
 >> Stay informed about: Quoted literal strings won't force a phrase match 
Back to top
Login to vote
Lepidopterist

External


Since: Jan 18, 2008
Posts: 2



(Msg. 5) Posted: Thu Jan 24, 2008 2:56 pm
Post subject: Re: Quoted literal strings won't force a phrase match [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)

Hey all,

Thanks for your responses, and sorry it took me so long to get back to
you.

In the end, I decided to opt for accuracy over performance, and have
emptied my noise word file. The results so far have been encouraging
- my index went from being 9MB to a very manageable 20MB, and response
times have remained unchanged.

It almost seems to good to be true. I feel like noise word
elimination doesn't really give you that much of a performance
advantage.

Let's imagine for the moment that I'm wholly unconcerned about disk
space usage. What level of search traffic would necessitate noise
word elimination? Does noise word elimination ever make that much of
a difference to response times?

Thanks again for the responses.

On Jan 23, 12:02 pm, Hilary Cotter wrote:
> A couple of points, in SQL 2005 if your search phrase is wrapped in
> double quotes it is a literal search for both FreeText and Contains.
> In SQL 2000, wrapping your search phrase in double quotes for a
> FreeText search has no effect - it is stemmed.
>
> The second point it that if SQL FTS detects the presence of noise
> words in your search phrase it throws them away but respects the
> positional placement of them.
>
> So a search a FreeText search on University of California (where of is
> a noise word) would result in hits to University of California,
> University at California and University in California ranked higher
> than University California or California University.
>
> On Jan 20, 1:42 pm, "Russell Fields" wrote:
>
> > A noise word is always a noise word. Noise words are applied to the
> > building of the index, so the full-text search has nothing to find.
> > Therefore, if you change the noise word list, you must rebuild the index
> > before you can search for the former noise word. (It is common to run with
> > either a single blank or a single nonsense word in the noise word file, so
> > as to get no noise words.)
>
> > Of course, you can do a string search for '%time out%' in addition to the
> > full-text query.
>
> > RLF
>
> > "Lepidopterist" wrote in message
>
> >
>
> > > Ok, I've made some progress on this problem. Apparently SQL Server is
> > > ignoring noise words in my phrase match.
>
> > > For example, I ran this query :
>
> > > SELECT *
> > > FROM Content_Items ci
> > > INNER JOIN FREETEXTTABLE(Content_Items, hed, '"time capsule"') AS ft
> > > ON ci.contentItemId = ft.[KEY]
> > > ORDER BY ft.RANK DESC
>
> > > And it did exactly what it was supposed to do, since neither "time"
> > > nor "capsule" is a noise word.
>
> > > My impression was that noise words aren't stripped out of a full text
> > > search if the search phrase is a quoted literal. Thus, my search for
> > > "time out" should look for the full phrase "time out", and not just
> > > the word "time."
>
> > > Does anybody know why SQL Server is removing my noise word from the
> > > phrase match?
>
> > > On Jan 18, 12:49 pm, wrote:
> > >> Hello all,
>
> > >> From what I've read, SQL Server is supposed to do a phrase match when
> > >> you do a full text search that contains quoted literal strings. So,
> > >> for example, if I did a full text search on the phrase "time out" and
> > >> I put it in quotes, it's supposed to search for the full phrase "time
> > >> out" and not just look for rows that contain the words "time" or
> > >> "out." However, this isn't working for me.
>
> > >> Here is the query that I'm using :
>
> > >> SELECT *
> > >> FROM Content_Items ci
> > >> INNER JOIN FREETEXTTABLE(Content_Items, hed, '"time out"') AS ft
> > >> ON ci.contentItemId = ft.[KEY]
> > >> ORDER BY ft.RANK DESC
>
> > >> What's it's doing is this : it's returning a bunch of rows that have
> > >> the words "time" or "out" in the column called hed. It's also
> > >> returning rows that have the full phrase "time out", but it's giving
> > >> those rows the same rank as rows that only contain the word "time."
> > >> In this case, that rank is 180.
>
> > >> Is there anything else I should be doing in my query, or is there some
> > >> configuration option I should have turned on?
>
> > >> Thanks.- Hide quoted text -
>
> > - Show quoted text -
 >> Stay informed about: Quoted literal strings won't force a phrase match 
Back to top
Login to vote
Hilary Cotter

External


Since: Oct 09, 2005
Posts: 78



(Msg. 6) Posted: Fri Jan 25, 2008 9:52 am
Post subject: Re: Quoted literal strings won't force a phrase match [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)

On one of my clients we ran with 14,000 noise word lists, response
times were much better. We knew in advance that the cleint's customers
would only be searching on limted keywords - like Gold Watch,
Bracelet, earrings, etc.


On Jan 24, 5:56 pm, Lepidopterist wrote:
> Hey all,
>
> Thanks for your responses, and sorry it took me so long to get back to
> you.
>
> In the end, I decided to opt for accuracy over performance, and have
> emptied my noise word file.  The results so far have been encouraging
> - my index went from being 9MB to a very manageable 20MB, and response
> times have remained unchanged.
>
> It almost seems to good to be true.  I feel like noise word
> elimination doesn't really give you that much of a performance
> advantage.
>
> Let's imagine for the moment that I'm wholly unconcerned about disk
> space usage.  What level of search traffic would necessitate noise
> word elimination?  Does noise word elimination ever make that much of
> a difference to response times?
>
> Thanks again for the responses.
>
> On Jan 23, 12:02 pm, Hilary Cotter wrote:
>
>
>
> > A couple of points, in SQL 2005 if your search phrase is wrapped in
> > double quotes it is a literal search for both FreeText and Contains.
> > In SQL 2000, wrapping your search phrase in double quotes for a
> > FreeText search has no effect - it is stemmed.
>
> > The second point it that if SQL FTS  detects the presence of noise
> > words in your search phrase it throws them away but respects the
> > positional placement of them.
>
> > So a search a FreeText search on University of California (where of is
> > a noise word) would result in hits to University of California,
> > University at California and University in California ranked higher
> > than University California or California University.
>
> > On Jan 20, 1:42 pm, "Russell Fields" wrote:
>
> > > A noise word is always a noise word.  Noise words are applied to the
> > > building of the index, so the full-text search has nothing to find.
> > > Therefore, if you change the noise word list, you must rebuild the index
> > > before you can search for the former noise word.  (It is common to run with
> > > either a single blank or a single nonsense word in the noise word file, so
> > > as to get no noise words.)
>
> > > Of course, you can do a string search for '%time out%' in addition to the
> > > full-text query.
>
> > > RLF
>
> > > "Lepidopterist" wrote in message
>
> > >
>
> > > > Ok, I've made some progress on this problem.  Apparently SQL Server is
> > > > ignoring noise words in my phrase match.
>
> > > > For example, I ran this query :
>
> > > > SELECT *
> > > > FROM Content_Items ci
> > > > INNER JOIN FREETEXTTABLE(Content_Items, hed, '"time capsule"') AS ft
> > > > ON ci.contentItemId = ft.[KEY]
> > > > ORDER BY ft.RANK DESC
>
> > > > And it did exactly what it was supposed to do, since neither "time"
> > > > nor "capsule" is a noise word.
>
> > > > My impression was that noise words aren't stripped out of a full text
> > > > search if the search phrase is a quoted literal.  Thus, my search for
> > > > "time out" should look for the full phrase "time out", and not just
> > > > the word "time."
>
> > > > Does anybody know why SQL Server is removing my noise word from the
> > > > phrase match?
>
> > > > On Jan 18, 12:49 pm, wrote:
> > > >> Hello all,
>
> > > >> From what I've read, SQL Server is supposed to do a phrase match when
> > > >> you do a full text search that contains quoted literal strings.  So,
> > > >> for example, if I did a full text search on the phrase "time out" and
> > > >> I put it in quotes, it's supposed to search for the full phrase "time
> > > >> out" and not just look for rows that contain the words "time" or
> > > >> "out."  However, this isn't working for me.
>
> > > >> Here is the query that I'm using :
>
> > > >> SELECT *
> > > >> FROM Content_Items ci
> > > >>     INNER JOIN FREETEXTTABLE(Content_Items, hed, '"time out"') AS ft
> > > >> ON ci.contentItemId = ft.[KEY]
> > > >> ORDER BY ft.RANK DESC
>
> > > >> What's it's doing is this : it's returning a bunch of rows that have
> > > >> the words "time" or "out" in the column called hed.  It's also
> > > >> returning rows that have the full phrase "time out", but it's giving
> > > >> those rows the same rank as rows that only contain the word "time."
> > > >> In this case, that rank is 180.
>
> > > >> Is there anything else I should be doing in my query, or is there some
> > > >> configuration option I should have turned on?
>
> > > >> Thanks.- Hide quoted text -
>
> > > - Show quoted text -- Hide quoted text -
>
> - Show quoted text -
 >> Stay informed about: Quoted literal strings won't force a phrase match 
Back to top
Login to vote
Display posts from previous:   
Related Topics:
'%phrase%' search - Hi, I'd like to know if it's possible to implement the search '%word|phrase%' with full text catalog. I've tried with "...CONTAINS(FIELD,'"word|phrase"')" but I don't get the same result as using "... FIELD LIKE '%word|phrase%'...

How to search a phrase inside a single DB field - I have this problem: all the keywords of an image are stored in a single text field called "keywords". I created a catalog on that field. A couple of samples of my keywords: ImageA = Tower Bridge, landmark, bridge, illuminated, National Famo...

Does SQL2005 still require all FTI keywords to match in th.. - Hello, My understanding of full-text index searching is that CONTAINS() requires that all the keywords supplied (separated by AND) must match in the same column in order for the record to be considered a match and returned. So if we have a situation..

Custom Text Match routine for everybody - All, I implemented a “SmartTextMatch ©” process to match two data sets by their descriptions. This process is very useful in merging data from disparate systems. The process consists of three steps: 1) Pre-processing data – cleansing,..

Grab text near match in fulltext - Is there a way to grab a bit of text near the first word the fulltext search matches? We're currently showing a summary to people, which is written by the user, but obviously does not contain the keyword someone searched for. And people think the search...
   Database Forums (Home) -> Full Text All times are: Pacific Time (US & Canada)
Page 1 of 1

 
You can post new topics in this forum
You can reply to topics in this forum
You can edit your posts in this forum
You can delete your posts in this forum
You can vote in polls in this forum



[ Contact us | Terms of Service/Privacy Policy ]