Aggregation in Spring Data MongoDB

Aggregation in MongoDB was built to process data and return computed results. Data is processed in stages and the output of one stage is provided as input to the next stage.

Aggregation Using MongoTemplate

{
    "_id" : "01001",
    "city" : "AGAWAM",
    "loc" : [
        -72.622739,
        42.070206
    ],
    "pop" : 15338,
    "state" : "MA"
}

Get All the States With a Population Greater Than 10 Million Order by Population Descending

GroupOperation groupByStateAndSumPop = group("state")
  .sum("pop").as("statePop");
MatchOperation filterStates = match(new Criteria("statePop").gt(10000000));
SortOperation sortByPopDesc = sort(Sort.by(Direction.DESC, "statePop"));

Aggregation aggregation = newAggregation(
  groupByStateAndSumPop, filterStates, sortByPopDesc);
AggregationResults<StatePopulation> result = mongoTemplate.aggregate(
  aggregation, "zips", StatePopulation.class);

The expected output will have a field _id as state and a field statePop with the total state population:

public class StatePoulation {
 
    @Id
    private String state;
    private Integer statePop;
 
    // standard getters and setters
}

The @Id annotation will map the _id field from output to state in the model:

Get Smallest State by Average City Population

GroupOperation sumTotalCityPop = group("state", "city")
  .sum("pop").as("cityPop");
GroupOperation averageStatePop = group("_id.state")
  .avg("cityPop").as("avgCityPop");
SortOperation sortByAvgPopAsc = sort(Sort.by(Direction.ASC, "avgCityPop"));
LimitOperation limitToOnlyFirstDoc = limit(1);
ProjectionOperation projectToMatchModel = project()
  .andExpression("_id").as("state")
  .andExpression("avgCityPop").as("statePop");

Aggregation aggregation = newAggregation(
  sumTotalCityPop, averageStatePop, sortByAvgPopAsc,
  limitToOnlyFirstDoc, projectToMatchModel);

AggregationResults<StatePopulation> result = mongoTemplate
  .aggregate(aggregation, "zips", StatePopulation.class);
StatePopulation smallestState = result.getUniqueMappedResult();

Get the State With Maximum and Minimum Zip Codes

GroupOperation sumZips = group("state").count().as("zipCount");
SortOperation sortByCount = sort(Direction.ASC, "zipCount");
GroupOperation groupFirstAndLast = group().first("_id").as("minZipState")
  .first("zipCount").as("minZipCount").last("_id").as("maxZipState")
  .last("zipCount").as("maxZipCount");

Aggregation aggregation = newAggregation(sumZips, sortByCount, groupFirstAndLast);

AggregationResults<Document> result = mongoTemplate
  .aggregate(aggregation, "zips", Document.class);
Document document= result.getUniqueMappedResult();

Aggregation Using MongoRepository

@Document(collection = "property")
public class Property {

    @Id
    private String id;
    @Field("price")
    private int price;
    @Field("area")
    private int area;
    @Field("property_type")
    private String propertyType;
    @Field("transaction_type")
    private String transactionType;
    
    // Constructor, getters, setters, toString()
    
}
@Aggregation(pipeline = {
        "Operation/Stage 1...",
        "Operation/Stage 2...",
        "Operation/Stage 3...",
})
List<Property> someMethod();
@Aggregation(pipeline = {
    "{'$match':{'transaction_type':'For Sale', 'price' : {$gt : 100000}}",
})
List<Property> findExpensivePropertiesForSale();
@Aggregation(pipeline = {
        "{'$match':{'transaction_type': ?0, 'price' : {$gt : ?1}}",
})
List<Property> findPropertiesByTransactionTypeAndPriceGTPositional(String transactionType, int price);

@Aggregation(pipeline = {
        "{'$match':{'transaction_type': #{#transactionType}, 'price' : {$gt : #{#price}}}",
})
List<Property> findPropertiesByTransactionTypeAndPriceGTNamed(@Param("transactionType") String transactionType, @Param("price") int price);
@Aggregation(pipeline = {
        "{'$match':{'transaction_type':?0, 'price': {$gt: ?1} }}",
        "{'$sample':{size:?2}}",
        "{'$sort':{'area':-1}}"
})
List<Property> findPropertiesByTransactionTypeAndPriceGT(String transactionType, int price, int sampleSize);
@Aggregation(pipeline = {
        "{'$match':{'transaction_type':?0, 'price': {$gt: ?1} }}",
        "{'$sample':{size:?2}}",
        "{'$sort':{'area':-1}}"
})
Iterable<Property> findPropertiesByTransactionTypeAndPriceGTPageable(String transactionType, int price, int sampleSize, Pageable pageable);

References
https://www.baeldung.com/spring-data-mongodb-projections-aggregations
https://stackabuse.com/spring-data-mongodb-guide-to-the-aggregation-annotation/

Projection in Spring Data MongoDB

In MongoDB, Projections are a way to fetch only the required fields of a document from a database. This reduces the amount of data that has to be transferred from database server to client and hence increases performance.

@Document
public class User {
    @Id
    private String id;
    private String name;
    private Integer age;
    
    // standard getters and setters
}

Projections Using MongoTemplate

Query query = new Query();
query.fields().include("name").exclude("id");
List<User> john = mongoTemplate.find(query, User.class);

Projections Using MongoRepository

@Query(value="{}", fields="{name : 1, _id : 0}")
List<User> findNameAndExcludeId();

References
https://www.baeldung.com/spring-data-mongodb-projections-aggregations

Use @Query annotation in Spring Data MongoDB

With this annotation, we can specify a raw query as a Mongo JSON query string.

FindBy

@Query("{ 'name' : ?0 }")
List<User> findUsersByName(String name);
List<User> users = userRepository.findUsersByName("Eric");

$regex

@Query("{ 'name' : { $regex: ?0 } }")
List<User> findUsersByRegexpName(String regexp);
List<User> users = userRepository.findUsersByRegexpName("^A");
List<User> users = userRepository.findUsersByRegexpName("c$");

$lt and $gt

@Query("{ 'age' : { $gt: ?0, $lt: ?1 } }")
List<User> findUsersByAgeBetween(int ageGT, int ageLT);
List<User> users = userRepository.findUsersByAgeBetween(20, 50);

References
https://www.baeldung.com/queries-in-spring-data-mongodb

Force Spring Boot to use Gson instead of Jackson

Well, WebMvcConfigurerAdapter is deprecated. As of Spring 5.0 do this:

@SpringBootApplication(exclude = {JacksonAutoConfiguration.class})
public class Spring01Application {

    public static void main(String[] args) {
        SpringApplication.run(Spring01Application.class, args);
    }

}
@Configuration
public class WebConfig implements WebMvcConfigurer {
    @Bean
    public Gson gson() {
        GsonBuilder b = new GsonBuilder();
        b.registerTypeAdapterFactory(HibernateProxyTypeAdapter.FACTORY);
        b.registerTypeAdapterFactory(DateTypeAdapter.FACTORY);
        b.registerTypeAdapterFactory(TimestampTypeAdapter.FACTORY);
        b.registerTypeAdapterFactory(LocalDateTypeAdapter.FACTORY);
        b.registerTypeAdapterFactory(LocalDateTimeTypeAdapter.FACTORY);
        return b.create();
    }

    @Override
    public void configureMessageConverters(
        List<HttpMessageConverter<?>> converters) {
        StringHttpMessageConverter stringConverter = new StringHttpMessageConverter();
        stringConverter.setWriteAcceptCharset(false);
        stringConverter.setSupportedMediaTypes(Collections
            .singletonList(MediaType.TEXT_PLAIN));
        converters.add(stringConverter);
        converters.add(new ByteArrayHttpMessageConverter());
        converters.add(new SourceHttpMessageConverter<>());
        GsonHttpMessageConverter gsonHttpMessageConverter = new GsonHttpMessageConverter();
        gsonHttpMessageConverter.setGson(gson());
        gsonHttpMessageConverter.setSupportedMediaTypes(Arrays
            .asList(MediaType.APPLICATION_JSON));
        converters.add(gsonHttpMessageConverter);
    }
}

References
https://stackoverflow.com/questions/40786366/force-spring-boot-to-use-gson-instead-of-jackson

Spring Boot – Multiple Spring Data modules found, entering strict repository configuration mode

@EnableJpaRepositories(basePackages = {"com.ecommerce.core.repository.jpa"})
@EnableElasticsearchRepositories(basePackages= {"com.ecommerce.core.repository.elastic"})
@EnableRedisRepositories(basePackages = {"org.springframework.data.redis.connection.jedis"})

Since we are explicitly enabling the repositories on specific packages we can include this in the application.properties to avoid these errors:

spring.data.redis.repositories.enabled=false

We can do the same for the other repositories as well. If you encounter similar errors:

spring.data.elasticsearch.repositories.enabled=false
spring.data.jpa.repositories.enabled=false

References
https://stackoverflow.com/questions/47002094/spring-multiple-spring-data-modules-found-entering-strict-repository-configur

Spring Data MongoDB Transactions

MongoDB Configuration

@Configuration
@EnableMongoRepositories(basePackages = "com.baeldung.repository")
public class MongoConfig extends AbstractMongoClientConfiguration{

    @Bean
    MongoTransactionManager transactionManager(MongoDatabaseFactory dbFactory) {
        return new MongoTransactionManager(dbFactory);
    }

    @Override
    protected String getDatabaseName() {
        return "test";
    }

    @Override
    public MongoClient mongoClient() {
        final ConnectionString connectionString = new ConnectionString("mongodb://localhost:27017/test");
        final MongoClientSettings mongoClientSettings = MongoClientSettings.builder()
            .applyConnectionString(connectionString)
            .build();
        return MongoClients.create(mongoClientSettings);
    }
}

Synchronous Transactions

@Test
@Transactional
public void whenPerformMongoTransaction_thenSuccess() {
    userRepository.save(new User("John", 30));
    userRepository.save(new User("Ringo", 35));
    Query query = new Query().addCriteria(Criteria.where("name").is("John"));
    List<User> users = mongoTemplate.find(query, User.class);

    assertThat(users.size(), is(1));
}

Note that we can’t use listCollections command inside a multi-document transaction – for example:

References
https://www.baeldung.com/spring-data-mongodb-transactions

Run Code on Spring Startup

@Component
public class EventListenerExampleBean {

    private static final Logger LOG 
      = Logger.getLogger(EventListenerExampleBean.class);

    public static int counter;

    @EventListener
    public void onApplicationEvent(ContextRefreshedEvent event) {
        LOG.info("Increment counter");
        counter++;
    }
}

We can use this approach for running logic after the Spring context has been initialized. So, we aren’t focusing on any particular bean. We’re instead waiting for all of them to initialize.

We want to make sure to pick an appropriate event for our needs. In this example, we chose the ContextRefreshedEvent.

References
https://www.baeldung.com/running-setup-logic-on-startup-in-spring

Configure and Use Spring Data Redis

Configuration

@Configuration
public class RedisConfiguration {

    @Bean
    public LettuceConnectionFactory redisConnectionFactory() {
        ConfigFile configFile = new ConfigFile();
        RedisStandaloneConfiguration configuration = new RedisStandaloneConfiguration(configFile.getRedisHost());
        configuration.setPassword(configFile.getRedisPassword());
        return new LettuceConnectionFactory(configuration);
    }

    @Bean
    public StringRedisTemplate redisTemplate() {
        return new StringRedisTemplate(redisConnectionFactory());
    }
}

You can use RedisTemplate instead of StringRedisTemplate.

Redis Repository

@RedisHash("Student")
public class Student implements Serializable {
  
    public enum Gender { 
        MALE, FEMALE
    }

    @Id private String id;
    private String name;
    private Gender gender;
    private int grade;
    // ...
}

The Spring Data Repository

@Repository
public interface StudentRepository extends CrudRepository<Student, String> {}

Data Access Using StudentRepository

Saving a New Student Object

Student student = new Student(
  "Eng2015001", "John Doe", Student.Gender.MALE, 1);
studentRepository.save(student);

Retrieving an Existing Student Object

Student retrievedStudent = 
  studentRepository.findById("Eng2015001").get();

Updating an Existing Student Object

retrievedStudent.setName("Richard Watson");
studentRepository.save(student);

Deleting Existing Student Data

studentRepository.deleteById(student.getId());

Find All Student Data

Student engStudent = new Student(
  "Eng2015001", "John Doe", Student.Gender.MALE, 1);
Student medStudent = new Student(
  "Med2015001", "Gareth Houston", Student.Gender.MALE, 2);
studentRepository.save(engStudent);
studentRepository.save(medStudent);

References
https://docs.spring.io/spring-data/redis/docs/current/reference/html/
https://www.baeldung.com/spring-data-redis-tutorial