I have found the reason for that.

After having shrunk the .mdf, the .ldf and the related .ndf files, while the size on disk decreased, this didn't change anything.

A reconstruction of the full text index did not change anything either.

I came to the conclusion that after a specific amount of content, records, or indexes, that the full text indexing might be affected by non optimized SQL query. So, I adjusted at that level. First, I got rid of the date in the SQL. So, what I did was to run two SQL first, to obtain the primary key to start and the primary key to end, using the date range. Those two SQLs are executed in a flash. Then, I can use those primary key values for the range instead of using a date field. Date fields are known to not be as optimized as integer. I then also continued in the optimization by querying on an inner SQL first and then, used that primary key and related that to the outer SQL to get the other fields.

All that wasn't necessary so far and I didn't observe that until I saw a significant drop from a certain date. This structure was done years ago and when I adjusted it to the new standard, the result was very fast.

For those who wonder about where it is, it is in regard to the search on this site. I will put a rush to release UT 2013 at the beginning of June instead of the beginning of July. This will come with it. I cannot apply a fix right now. There are too many changes that require the new version.