Tweet Text to Web Collection Resolution

[terrylnwoods]

When TwitterVane receives a tweet, the tweet is stored in a database for further analysis.
Analysis is performed by the TweetAnalyser component which is notified to run after n-tweets have been received (the 'n' is configured in the spring-sevlet.xml file for the TweetStreamAgent component).

The analysis consists of:

1. Expanding the URLs associated with the tweet if it they are shortened
2. Resolve the Web Collection to which these URLs belong

The resolution performed by the latter analysis is based on the search terms defined for each Web Collection.
If no search term is found within the tweet text then the URL is allocated to a "bucket" Web Collection called "UNKNOWN".

An improved process for resolving URLs to Web Collections is needed since the current process results in large numbers of URLs being allocated to the "bucket".