Report of current responses

[henkvancann]

Update on the exciting responses on the design of our site, the “KERI suite search engine (KERISSE): Personal learning environment and consensus building with others.”

We've contacted developers in Group 2* of KERI developers and consider them valuable in shaping the design of KERI / ACDC but also be of great value for functionality of the KERI suite search engine (KERISSE).

'* Group 1 consists of Sam, Philip and Kevin, the first dev team

With that in mind, we've collected insights on the following design considerations for the KERI suite search engine. This are the results so far. We've asked permission to spread the responses that we've got.

These were the main questions / topics to address:

What is the result of this, who is going to use this and what should they be able to do with it?

What is the result of this?

A site as a collection of relevant resources where consensus is achieved about what the meaning of certain terms are, but only in the context of the full KERI suite*
The output level of understanding of our target group is at least 3. Levels explained.

Who is going to use this?

Technical Self-Sovereign Identity experts who are developers too. So both developer skill as well as having fundamental knowledge in the topics: cryptography, SSI and governance. The input level of understanding of our target group is at least 2.

What should they be able to do with it?

1. Trust the completeness and actuality of the resource-scraping behind it.
2. Glossary service at various levels of understanding, to achieve consensus plus share their (search) results with others for communication purposes.
3. Learn about the historical concepts and choices behind all KERI parts to better understand the Why's of it all (and not only the What and How)
4. "See" the structure of the code in various programming languages much clearer, the ability to question the structure more dynamically and more visual (like Rust has build in).

'* KERI, ACDC, CESR, OOBI, etc

What is this?

KERI suite search engine (KERISSE)
Personal learning environment and consensus building with others

Who is going to use this?

See above and any guest user starting at level 1 ((say starting from "E-Identity expert") is welcome too. Laymen might get lost, we'll accept this**, the site is targetted at experts.

'** Trying to educate laymen on SSI and KERI suite terminology introduces a whole next level of complexity (hard-felt experience :) )

What can he do with it, that he can’t do with anything else?

Anything else does not have the intelligence built-in that combines proper selection and labelling of resources, categorizations and ranking of resources on one hand and scraping and indexing with a fit-for-purpose customized search engine on the other hand that results in manageabe and properly ranked search results. The results are accessible through tailor-made filters in the left menu of KERISSE.
Currently Github nor Google and even AI can't deliver this content.

[henkvancann]

Response Daniel Hardman PM in Slack May 24 2023

Have you arranged for the keri.one website to link to it? - Answer Henk: Yes

General Comments
...
The search works well for individual words like "schema." However, when I search for a phrase like "how to make a schema", I don't get anything useful.
...
This page crashed for me: https://weboftrust.github.io/WOT-terms/docs/terms/glossary/jury?level=3

It feels like the knowledge level concept could be very useful, but that it applies more to long-form content (white papers, diagrams, etc) and less to meeting notes or glossary terms. Maybe the level should be below the term in glossary display, rather than being the first thing that appears on the page?

Response Henk: the search engine has a fairly good proximity search, so if you don't get results on 'how to make a schema' there is a good chance the data isn't there yet.
We'll try to turn knowledge levels for certain resources like meeting notes
We grant level 2 to the general heading "##Definition" in the ACDC wiki glossary and we grant level 3 when we match a heading that mentions "Related to KERI"

[henkvancann]

Response Joseph Hunsaker PM in Slack May 24 2023

Is this for the thing we discussed about using KERI/ACDC tech to show verified information about the KERI ecosystem? “Eating our own dog food” approach? - Answer Henk: No it's because of Rieks Joosten's appeal to stop and design first.

Regarding 1) above: a “personal learning environment” - this is for tech individuals to learn and put forth ideas ? How will those ideas be interacted with? So this is not for any lay people? this is just for those interested in learning to help build and collaborate?
Answer Henk: Yes, the focus has been narrowed.

I like to think in terms of “outcomes” when I am planning. What is the main “outcome” of the KERISSE project? Is it:
An organic, searchable, verified (and verifiable) knowledge management database?

Outcomes = “The results we want to achieve” as Rieks Joosten said

One “outcome” I would like to see is a “high level” view of each area in Keri. I imagine it starting out like a map at high altitude. Only able to see the large features, and then diving in deeper. into each feature. It would be amazing if this outcome was automatic, via some algorithm of how the “knowledge” has been collected and interacted with, and related to other concepts.

Answer Henk: Well done and close to what I feel it’s going to be. I would add words like ‘an index into’, ‘scraped and labeled and categorized’ , stuff like that. But… it’s not me shaping this, it should come from all the developers that have been struggling through the puzzle Sam, Phil and Kevin undoubtedly leave behind in their march forward.

[henkvancann]

Response Kent Bull PM in Slack May 27 2023

Here are some of my thoughts:

1. The search functionality is well done. I would link to this website.
2. There should be a page with a listing of all of the repositories and a spreadsheet showing the purpose of each repository.
3. There should be an architecture diagram showing a basic KERI network and a basic ACDC graph.
4. There should be a basic, annotated workflow diagram of issuing, presenting, and verifying a credential.

Follow-up question Henk:
How would you use point 2? Shouldn’t the purpose (the WHY of it all) be documented in the description of the repo, so we can harvest this?
Kent responded: I would use point 2 to learn about and remind myself of why each repository exists and what is in it.
Point 3 and 4 are both diagrams; if you click on the nodes what should you get? Same for the vertices, what should you see of you click on them.
Kent responded: If you click on three and four, then you should see a webpage/preview with the PNG diagram and a title above it and a description below it

[henkvancann]

Response Steven Milstein PM in Slack May 24 2023

I'm unclear to me what should users be able to do with the platform. I'm also unfamiliar with the level x (or 'xxx') you're referring to.
Answer Henk: Fully agree with this, Kor and I changed it into 1,2 and 3.

Would it be helpful to present this topic to #edu and walk through a few use cases?

It may be easier for everyone to have an open discussion than to message back & forth.
Answer Henk: Thx, suggestion implemented!

Is all the socializing being done async - ia slack or HackMD?
Answer Henk:
Currently I try one on one, because a discussion goes off on many tangents, I believe. The need we have is to have the questions posed above (which are the same as in the HackMD section Dont make me think) answered by Dev Group 2 or anyone that can think like them.
When I would go and present the answers first, then it’s again my system and not theirs.

How do you envision sustaining the platform?
Answer Henk:
As automatically as possible. Scraping -> Search engine protocols -> filters -> Github Action

[henkvancann]

Response Nuttawut Kongsuwan PM in Slack May 24 2023

My general concern is that the project might too ambitious, given the current manpower of our community.

What is the result of this platform?

May I ask what you mean by “building consensus”?
Answer Henk: building consensus -> terminology is hard to get your head around. People have different concepts in mind when they talk about the same words / terms. We need to understand the other before we can be understood. That’s what I mean by building consensus around concepts and terms.
The work of eSSIF-lab elaborates heavily on this.

This seems intriguing, and I would love to learn how this could be achieved.

7 levels seem too fine-grained for me. I suggest we start with 3, and 4 more levels may be added after the platform becomes more mature.
Answer Henk: Tip implemented, thx! 3 levels now, fully in sync now with the 1, 2 and 3 stars in the Q&A section.

2 Who is going to use this platform?
I suggest we target “identity professionals and developers”, instead of “experts” as this term can be misleading. I believe the platform is also useful to professionals who may not have a computer science background, like me. Targeting “experts” also sounds like we are excluding junior developers.
Answer Henk: identity professionals and developers”, instead of “experts” -> good point; the other points are also good, but this one I can implement right away

I agree that we may assume fundamental knowledge in the areas of cryptography, SSI, and governance. Materials for these topics can already be found elsewhere.

3 What should users be able to do with the platform?
Again, the idea sounds fantastic, but I am worried that it might be too ambitious. However, I will be delighted if you can prove me wrong.

I am wondering if it is possible for the community to raise some funds to found a “WebOfTrust/KERI Foundation” so we can employ someone to work on the platform full-time.
Follow-up question Henk:
How could you limit our scope to make it less over-ambitious?
Answer Nuttawut:
I think we could ask people about their wish list for the platform. Then we look for a pattern. If many people ask for the same things, we will know what to prioritize.
Personally I would love the following items in the platform in the order of importance:

• Glossary of terms (which we have already to some extent)
• Manual of kli commands and their config files
• beginner friendly tutorials and content, like the one I did on my blog and Kent’s tutorials

[henkvancann]

Nuttawut added July 7 2023

I am seeking friendly advice that could also be helpful to KERISSE and
@Henk van Cann
.
Over the past few months, I have been studying KERI while juggling my day job. I have made progress by reading the white papers, the specifications, and exploring KLI demos in KERIpy. However, I have recently found out the limitations of KLI and now wish to delve deeper into understanding KERI’s inner workings and implementation.
I must confess that I lack formal training as a software engineer, and my coding skills are still at an elementary level. While attempting to read the source code in WebOfTrust (KERIpy, Signifypy, Signify-TS, and KERIA), I found them quite overwhelming. Therefore, I am seeking advice on how to approach these repositories effectively.
My objective is not to become an experienced developer and contribute code to the project. Rather, I aim to gain a solid understanding of KERI’s inner workings so that I can communicate effectively with my developer colleagues and make well-informed business decisions.
Any advice or guidance would be appreciated!

[henkvancann]

Henk reporting on response Ruth Choueka Report Zoom session May 30 2023

Ruth likes the search engine KERISSE, improvements could be:

1. cross links between the GLEIF governance docs “How to become a QVI” and the status of the (keripy) functionality would have been insightful.
2. GLEIF workflows were in flux and changed over time. Maybe some versioning of those workflows made available (what’s new, what’s depreciated) could come in handy?
3. ‘Howto’ Information and guidance on “current open issues” E.g. How to expand multisigs and Multisig Revocation.

Ruth realises that code isn’t always ready and sometimes buggy. That’s reality. However it would come in handy to see the status of the implementation of certain functionality and to be able to anticipate on what’s going to be available soon.

[henkvancann]

Henk reporting on response Simon Labram and Petteri Stenius Report Zoom session June 2 2023

Simon has a CertificationAuthority background. Ubisecure is applying te become QVI at GLEIF
He works with / uses / improves the workflows in the GLEIF vLEI governance docs and wonders with regard to the processes: how do we need to support customers?
As an indirect user of the KERI suite he'd like to know where to look for up to date information. Google doesn't deliver up to expectations.
With regard to the workflows Simon wants to:

• add stuff and challenge what's been described so far
• improve the speed of issuance of credentials
• simplicity of user experience (initial onboarding and structural workflow)
• enhance the cleverness
  "It's a chore. Any bump in the road will chase users away. Even filling out a form might be too much and put them off. They will not easily return."

Simon would like to get the operational requirements of the application of the KERI suite. For example the process of applying for credential. Hoe can we be user centric instead of technology centric.

Henk asks Petteri to imagine a way in KERISSE how we could combine both objectives: be of help for devs and start being user centric at the same time.

Petteri shows an example of helpful items:
Workflows Lance. We realize this is maintenance sensitive information. Henk adds he's on the lookout for workflows that are generated from the actual coded situation (the least menual work as possible)
Unit tests aren't up to date. Hopefully Lance can pick this up?

KERISSE could tell the criteria and nuances of relevance:

• what is a certain term or concept and what not?
• where can you use it for and what not?

Henk would love to get more example from Simon regarding operational requirements of the application of the KERI suite.

Henk advises Simon and Petteri to start using the search engine KERISSE hourglass in upper-right corner and report back experiences and tips.

[henkvancann]

Henk reporting on response Jason Colburne Report Zoom session June 2 2023

Definitions to sync with the glossary: https://github.com/WebOfTrust/cesride

Wanted: Good documentation around seals & anchors

Wanted in general:

1. Possibilities of certain functional objects, 2. when to use them, 3. how to use them (workflow) and 4. example usage
   In Jason’s learning curve “verification” was the one the hardest things to learn: when & where you need to do things, and the completeness of it.

As developer Jason would never go outside the command line in a IDE to find stuff. He uses ripgrep

[henkvancann]

Response Rodolfo Miranda PM in Slack June 4 2023

I really love the idea of having this kind of service where we can easily search for stuff. I play a bit in the platform and seems that the search works well finding terms but not too good when for example trying to find a specific document. Search for ‘KERI for muggles’ I’d expect the result points the file link.

I think there will be different kind of users. Developers of KERI tools/SDK and implementors of KERI tools/SDK. Right now, we are mostly from the first type of developers, but I’d expects that implementors will grow in number as the ecosystem mature. The second group needs more “digested” documentation with practical examples, such as Kent’s tutorials.
It’ll be crucial to keep the info updated; we are growing fast and things changes without notice.

Follow-up Henk:
KERI for muggles -> Agreed, we have to tune Typesense for this, ongoing process. Action item created
I think there will be different kind of users. -> Yes, we anticipate on this. For example we provide the chatbot., which is clearly not targetted at devs
The second group needs more “digested” documentation with practical examples -> that’s exactly what we’re aiming at.
It’ll be crucial to keep the info updated -> We scrape as much as possible and add metadata, like origin and date, We scrape and index to dismiss ourselves from having to keep record. If there’s crap in KERISSE, then we’ve got that crap recently from the source
Currently AI often produces gibberish, even if the sources are up to date. But we’ll think that the double exponential learning curves of AI might soon turn the tables and point humans to human errors.

[henkvancann]

Henk reporting on response Michal Pietrus Report Zoom session June 22 2023

Michal suggests: (Level 1:) What are the pieces -> (Level 2:) Why is it done this way -> (Level 3:) How for the devs and only for them.
Whitepapers as a category as a jump-off point in the left menu
How HCF had to do it in 2020/21: Whitepaper -> meetings asking questions -> code base
If had existed: Overview -> what it consists of, how to get things done would have been great
Distinguish : developing KERI how it work and also ‘how to consume it’

• how do I use it, how to integrate, a quick scan whether it is doable,
  how many witnesses, how many watchers, where can I find the controller how can I start the controller, how do I create my identifiers, etc. (edited)
  Response Henk: We will implement your suggestions:
• (Level 1:) What are the pieces -> (Level 2:) Why is it done this way -> (Level 3:) How for the devs and only for them.
• Whitepapers as a category as a jump-off point in the left menu

[henkvancann]

Henk reporting on response Lance Byrd Report Zoom session June 23 2023

Discussed with Lance while going through sections of the current KERISSE site.

1. Every landing page has to have

• 5W: what, why, when, for who, which way to use
• what is the result of this and why does it matter

2. Levels:
   Tooling -> Docker 1, REST API 1 PyPi 1-2
   Henk wil tags as detailed as blocks under a certain markdown header 2 and 3 (## and ### ) what the level of this block is. And then users can filter based on level.
3. Facilitate Contribution of the community to this site
   vLEI verification service repo, Phil collected lots of comments on his code. Would this be something KERISSE could do too?
4. KERI scrapbook presentation that are well used and much appreciated by the community should bubble up / surface on top of the list.
   Maybe up/down voting system for a listing like the one on KERI.one (Sam’s site)
   Henk response: Scrapbook -> we’ll try to find a way to automatically decide on the bases of meta data where to put a doc: +/- votes, actual usage, date of edits, feedback - comments etc.

[henkvancann]

Rate search results - Rodolfo Miranda
1:18 PM
@Henk van Cann
I wonder if there's a way to give stars or "likes" to the search results that we consider the best or more correct, so as the time goes, the engine can improve the way it sort the results presenting at the top the ones we like the most. It's a way to "curate" results.

[kentbull]

How would we start to rank order the search results of KERISSE? For example, when I search “Diger” it would be useful if the definition from the Python class definition would be the most useful to show first. Yet, a simple search engine doesn’t seem to be able to determine that a definition like that should be ranked first. We would have to have some way to add metadata that would influence sort order in order to sort things to the top of the list.

I would like KERISSE to be able to be my primary documentation look up tool at some point. And I expect we will integrate AI into the search engine at some point, if it isn’t there already. I’d be happy to help do some research here to bring things like this into the mix.

The solution to the rank order is likely a metadata tagging system where we augment the plain search results with some sort of expert-curated metadata annotations of specific search results.

[kordwarshuis]

The search engine we use (https://typesense.org) has a “Curation” option where you can import curated searches:

[
    {
        "name": "Diger",
        "query": "Diger",
        "url": "https://github.com/WebOfTrust/keripy/blob/main/src/keri/core/coring.py",
        "position": 1,
        "match": "exact"
    },
    {
        "name": "keriForMuggels",
        "query": "Keri for Muggles",
        "url": "https://github.com/SmithSamuelM/Papers/blob/master/presentations/KERI_for_Muggles.pdf",
        "position": 1,
        "match": "exact"
    },
…

This works as desired, only I have to solve an issue (I know what the problem is though). I imported the Diger entry but can't get it to work yet), so it is not working yet. Searching for “keri for muggles” shows the original document at the top though, via curation.

Also Henk and I decided that we want a voting system in the search results, a thumbs up that you can click. It should create a github issue (so a human can read them and create curated entries) or it could create a JSON entry directly.

Also we are playing with AI. We used it for creating a summaries (see for example https://weboftrust.github.io/WOT-terms/docs/glossary/ADC). Also we tried a chatbot but it did not learn and also it thought that ACDC was a rockband although we also mentioned “keri”.

If you have suggestions for applying AI please share!

[kentbull]

This would work. I'd be happy to help create curated search terms. Do I just send them to you or can I make a PR with them?

[kordwarshuis]

That's really great! Whatever works best for you.

[kordwarshuis]

Also good news from Typesense: “Integration with OpenAI API”.
We will research this.

We’re super excited to announce the General Availability of the latest version of Typesense v0.25.0 today!:

Here are some key highlights:

-Semantic Search: Search for conceptually related terms in your dataset, even if the exact keyword does not exist in your dataset.

-Hybrid Search: Combine both keyword and semantic / vector search results in a single query using rank fusion.

-Automatic Embedding Generation: specify one or more string fields that should be used for generating embeddings during indexing & during search using state-of-the-art embedding models, optionally using a GPU.

-Integration with OpenAI API, PaLM API and Vertex AI API: Have Typesense automatically make API calls to remote embedding services like OpenAI / Google, to generate vectors for the JSON data you index in Typesense.

-Query Analytics: Typesense now supports aggregation of popular search queries which can then be used as insights into query patterns.

-Query Suggestions: You can use historical search terms to collected by the Query Analytics feature, to power Query Suggestions.

-Update Documents by Query: You can now update all documents that match a filter_by condition.

[daidoji]

I love KERISSE. Its been a huge help so far and except for a few hiccups (which I'd expect in any OSS site) its been great.

I think I agree with discussions regarding searching not always finding what I'd expect to find directly and I noticed that the broken links have been fixed up so I won't mention those.

In terms of a general comment on documentation (but nothing on KERISSE explicitly) I've always liked the https://diataxis.fr/ framework for evaluating documentation's uses and needs.

KERISSE and the Keri community seem to have a wealth of Explanation and Reference (although not always the reference I'd expect) but less direct HOWTOs and TUTORIALS (other than the great tutorials @kentbull put together) to show how to do certain things or just get developers associated with the code bases and various forms of KERI, the primitives, ACDCs, CESR, etc... More of those would be extremely helpful for getting up to speed.

Other things I'd vote for if money and time were no object:

1. Developer references like onboarding overviews, pydoc/rustdoc references, and projects documented and organized in a clear and transparent manner (alongside what does and doesn't exist in each repo).
2. More examples and diagrams (which I think are already on the roadmap).

Its a fantastic resource though. If I can write any HOWTOs or Tutorials in my learning (or anything else that might be low-hanging fruit) be sure to reach out.

[henkvancann]

Yes, Charles, please do so. It's just a matter of time(priority) and expertise that I lack to be able to write the HowTos myself.

[Charlotte-br560]

Thank you for sharing the exciting updates on the design of the KERI suite search engine, KERISSE! It's great to see the progress and engagement from the developer community. Looking forward to seeing how it continues to shape and evolve!