Skip to content

Commit

Permalink
Polishing
Browse files Browse the repository at this point in the history
Finishing touches.
  • Loading branch information
christophstrobl committed Nov 22, 2024
1 parent e356e9d commit 58500a3
Show file tree
Hide file tree
Showing 18 changed files with 630,166 additions and 155 deletions.
4 changes: 2 additions & 2 deletions README.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -112,8 +112,8 @@ WARNING: If you're done using it, don't forget to shut it down!

== Miscellaneous

* `bom` - Example project how to use the Spring Data release train bom in non-Spring-Boot
scenarios.
* `mongodb/fragment-spi` - Example project how to use Spring Data Fragment SPI to provide reusable custom extensions.
* `bom` - Example project how to use the Spring Data release train bom in non-Spring-Boot scenarios.
* `map` - Example project to show how to use `Map`-backed repositories.
* `multi-store` - Example project to use both Spring Data MongoDB and Spring Data JPA in
one project.
Expand Down
35 changes: 35 additions & 0 deletions mongodb/fragment-spi/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# Spring Data - Fragment SPI Example

This project contains a sample using `spring.factories` to register implementation details for a repository extension for MongoDB Vector Search that lives outside of the project namespace.

The project is divided into the `atlas-api`, providing the extension, and the `sample` using it.

## atlas-api

The `AtlasRepository` is the base interface containing a `vectorSearch` method that is implemented in `AtlasRepositoryFragment`. The configuration in `src/main/resources/META-INF/spring.factories` makes sure it is picked up by the spring data infrastructure.

The implementation leverages `RepositoryMethodContext` to get hold of method invocation metadata to determine the collection name derived from the repositories domain type `<T>`.
Since providing the metadata needs to be explicitly activated the `AtlasRepositoryFragment` uses the additional marker interface `RepositoryMetadataAccess` enabling the features for repositories extending the `AtlasRepository`.

## sample

The `MovieRepository` extends the `AtlasRepository` from the api project using a `Movie` type targeting the `movies` collection. No further configuration is needed to use the provided `vectorSearch` within the `MovieRepositoryTests`.

The `Movies` class in `src/main/test` takes care of setting up required test data and indexes.

## Running the sample

The is using a local MongoDB Atlas instance bootstrapped by Testcontainers.
Running the `MovieRepositoryTests` the `test/movies` collection will be populated with about 400 entries from the `mflix.embedded_movies.json` file.
Please be patient while data is loaded into the database and the index created afterwards.
Progress information will be printed to the log.
```log
INFO - com.example.data.mongodb.Movies: 73 - Loading movies mflix.embedded_movies.json
INFO - com.example.data.mongodb.Movies: 90 - Created 420 movies in test.movies
INFO - com.example.data.mongodb.Movies: 65 - creating vector index
INFO - com.example.data.mongodb.Movies: 68 - index 'plot_vector_index' created
```
Once data and index are available search result will be printed:
```log
INFO - ...mongodb.MovieRepositoryTests: 183 - Movie{id='66d6ee0937e07b74aa2939cc', ...
```
2 changes: 1 addition & 1 deletion mongodb/fragment-spi/atlas-api/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,6 @@
</parent>

<artifactId>spring-data-mongodb-fragment-spi-atlas</artifactId>
<name>Spring Data MongoDB - Fragment SPI</name>
<name>Spring Data MongoDB - Reusable Fragments - Vector Search Fragment</name>

</project>
Original file line number Diff line number Diff line change
@@ -1,18 +1,27 @@
/*
* Copyright 2024 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.example.spi.mongodb.atlas;

import java.util.List;

import org.springframework.beans.BeansException;
import org.springframework.beans.factory.config.BeanPostProcessor;
import org.springframework.context.annotation.Bean;
import org.springframework.data.domain.Limit;
import org.springframework.data.repository.core.support.RepositoryFactoryBeanSupport;
import org.springframework.stereotype.Component;

/**
* @author Christoph Strobl
*/
public interface AtlasRepository<T> {

List<T> vectorSearch(String index, String path, List<Double> vector, Limit limit);
List<T> vectorSearch(String index, String path, List<Double> vector);
}

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
Expand All @@ -16,60 +16,58 @@
package com.example.spi.mongodb.atlas;

import java.util.List;
import java.util.Objects;

import org.bson.Document;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.core.ResolvableType;
import org.springframework.data.domain.Limit;
import org.springframework.data.mongodb.core.MongoOperations;
import org.springframework.data.mongodb.core.aggregation.Aggregation;
import org.springframework.data.mongodb.core.query.CriteriaDefinition;
import org.springframework.data.repository.core.support.RepositoryMethodContext;
import org.springframework.lang.Nullable;
import org.springframework.data.repository.core.RepositoryMetadata;
import org.springframework.data.repository.core.RepositoryMethodContext;
import org.springframework.data.repository.core.support.RepositoryMetadataAccess;

class AtlasRepositoryFragment<T> implements AtlasRepository<T> {
class AtlasRepositoryFragment<T> implements AtlasRepository<T>, RepositoryMetadataAccess {

private MongoOperations mongoOperations;
private final MongoOperations mongoOperations;

public AtlasRepositoryFragment(@Autowired MongoOperations mongoOperations) {
this.mongoOperations = mongoOperations;
}

@Override
@SuppressWarnings("unchecked")
public List<T> vectorSearch(String index, String path, List<Double> vector, Limit limit) {
public List<T> vectorSearch(String index, String path, List<Double> vector) {

RepositoryMethodContext metadata = RepositoryMethodContext.currentMethod();
RepositoryMethodContext methodContext = RepositoryMethodContext.getContext();

Class<?> domainType = metadata.getRepository().getDomainType();
System.out.println("domainType: " + domainType);
Class<?> domainType = resolveDomainType(methodContext.getMetadata());

Document $vectorSearch = createDocument(index, path, vector, limit, null, null, null);
Document $vectorSearch = createDocument(index, path, vector, Limit.of(10));
Aggregation aggregation = Aggregation.newAggregation(ctx -> $vectorSearch);

return (List<T>) mongoOperations.aggregate(aggregation, mongoOperations.getCollectionName(domainType), domainType);
return (List<T>) mongoOperations.aggregate(aggregation, mongoOperations.getCollectionName(domainType), domainType).getMappedResults();
}

private static Document createDocument(String indexName, String path, List<Double> vector, Limit limit, @Nullable Boolean exact, @Nullable CriteriaDefinition filter, @Nullable Integer numCandidates) {
@SuppressWarnings("unchecked")
private static <T> Class<T> resolveDomainType(RepositoryMetadata metadata) {

// resolve the actual generic type argument of the AtlasRepository<T>.
return (Class<T>) ResolvableType.forClass(metadata.getRepositoryInterface())
.as(AtlasRepository.class)
.getGeneric(0)
.resolve();
}

private static Document createDocument(String indexName, String path, List<Double> vector, Limit limit) {

Document $vectorSearch = new Document();

$vectorSearch.append("index", indexName);
$vectorSearch.append("path", path);
$vectorSearch.append("queryVector", vector);
$vectorSearch.append("limit", limit.max());

if (exact != null) {
$vectorSearch.append("exact", exact);
}

if (filter != null) {
$vectorSearch.append("filter", filter.getCriteriaObject());
}

if (numCandidates != null) {
$vectorSearch.append("numCandidates", numCandidates);
}
$vectorSearch.append("numCandidates", 150);

return new Document("$vectorSearch", $vectorSearch);
}
Expand Down

This file was deleted.

This file was deleted.

1 change: 1 addition & 0 deletions mongodb/fragment-spi/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
</parent>

<artifactId>spring-data-mongodb-fragment-spi</artifactId>
<name>Spring Data MongoDB - Reusable Fragments</name>
<packaging>pom</packaging>

<modules>
Expand Down
2 changes: 1 addition & 1 deletion mongodb/fragment-spi/sample/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
</parent>

<artifactId>spring-data-mongodb-fragment-spi-usage</artifactId>
<name>Spring Data MongoDB - Fragment Usage</name>
<name>Spring Data MongoDB - Reusable Fragments - Fragment Usage</name>

<dependencies>
<dependency>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
Expand All @@ -23,20 +23,4 @@
@SpringBootApplication
public class ApplicationConfiguration {

// @Bean
// BeanFactoryPostProcessor postProcessor() {
// return new BeanFactoryPostProcessor() {
// @Override
// public void postProcessBeanFactory(ConfigurableListableBeanFactory beanFactory) throws BeansException {
//
// BeanDefinition movieRepository = beanFactory.getBeanDefinition("movieRepository");
// movieRepository.getPropertyValues().add("exposeMetadata", true);
// }
// };
// }

// @Bean
// BeanPostProcessor postProcessor() {
// return new AtlasRepositoryPostProcessor();
// }
}
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
Expand All @@ -19,11 +19,44 @@

/**
* @author Christoph Strobl
* @since 2024/08
*/
@Document
@Document("movies")
public class Movie {

String id;
String title;
private String id;
private String title;
private String plot;

public String getPlot() {
return plot;
}

public void setPlot(String plot) {
this.plot = plot;
}

public String getId() {
return id;
}

public void setId(String id) {
this.id = id;
}

public String getTitle() {
return title;
}

public void setTitle(String title) {
this.title = title;
}

@Override
public String toString() {
return "Movie{" +
"id='" + id + '\'' +
", title='" + title + '\'' +
", plot='" + plot + '\'' +
'}';
}
}
Original file line number Diff line number Diff line change
@@ -1,3 +1,18 @@
/*
* Copyright 2024 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.example.data.mongodb;

import com.example.spi.mongodb.atlas.AtlasRepository;
Expand Down
Loading

0 comments on commit 58500a3

Please sign in to comment.