How can I display more accurate results using the search extension?

[schulle4u]

Hello,

Currently the search extension allows to modify the text length of page excerpts. In theory you can set this to a high value and completely render result pages matching the given search string, which is not very comfortable when searching through pages with a bigger amount of text.

If possible, how can I modify the search extension layout to display a shorter but more accurate text excerpt from a page? I would like to have a typical search engine result with page title and an excerpt displaying only the relevant text with surrounding context, (e. g. 5 to 10 words around the actual search result).

As an example, searching for the word "voluptate" should return something like the following in the page excerpt:

... dolor in reprehenderit in **voluptate** velit esse cillum dolore eu ...

I've already found the function getSearchInformation() in search.php, however with no luck in parsing the result in the way I want. Any feedback is welcome, maybe I'm missing something thats already possible without layout hacking.

Steffen

[markseuffert]

Here's what we discussed on Discord. The search extension is roughly divided into two parts. The first part is the logic that searches trough all pages and calculates a "searchScore" based on text and meta data. The second part is the visualisation that loops trough stored pages and generates output from the "beginning of page content" converted to text.

If someone wants to display a more accurate text excerpt from a page, here some thoughts and ideas:

• The logic needs a function to extract the "relevant page content" for each page and store it together with the calculated "search score". The code that stores the search score is at $pageSearch->set("searchscore", $searchScore) in file search.php.
• The visualisation could remain almost unchanged, it would need to render the previously stored "relevant page content" instead of the "beginning of page content". The code that generates output is at $page->getContent(false, 4096) in layout file search.html.
• The interesting work would therefore be done in the logic part, for example cutting out words around keywords and then build together a "relevant page content". Not sure how much work this would be, in any case the Yellow API should have everything you need in terms of data storage/exchange.

Apologies for not answering earlier. I planned taking a break from GitHub discussions in 2022 and probably will answer less frequently. Of course will be available for code reviews, Steffen. Let me know what you find out.

[schulle4u]

Thanks Mark for the detailed answer. Unfortunately I'm afraid that my php skills are still not at a level which would allow me to build a working solution upon it. While I now understand roughly how the search extension works, some code is just a little to complex for me to transform ideas into solutions. Don't worry, thats not Yellows mistake. Anyway, my question has been answered so far and I'll keep it on my todo list. Maybe someone could pick it up for Datenstroms next summer of code? Wait... that's another company. 😀

Another solution I'm currently thinking of is a new websearch extension. Most search engines are able to filter results by website, so it should be no problem to create a simple shortcut that hands over the query to the desired search engine. Although I prefer selfhosted solutions and a websearch may cause privacy problems, this idea is worth a try. As a side effect this can even work on static websites. More on this in another discussion as it happens. Keep enjoying your break. 🙂

[schulle4u]

The search extension has been updated and now jumps directly to the first match of a search term and displays a text excerpt. Thank you, this makes the extension a lot more useful. 👍

[annaesvensson]

That extension is now at https://github.com/annaesvensson/yellow-search