Streaming queries

The following findEach, findIterate, and findStream are all streaming queries with mainly a difference in terms of style using a closure, iterator or stream. With these streaming queries we can process very large results without having to hold all results in memory. findEach would be the recommended style as it ensures all the resources held by the query are closed without necessitating a try with resources block. Using findStream and findIterate are equally fine with the note that we need to ensure the QueryIterator and Stream are closed typically via try with resources.

From Ebean version 12.3.5 these streaming queries use an "adaptive persistence context". This means that findEach, findIterate, and findStream work equally as well against small query results as findList in terms of persistence context and memory. After 1000 beans are processed the queries adapt the persistence context they use to ensure it does not hold all the beans (and run out of memory when processing very large results).

Queries processing less than 1000 beans use a single normal persistence context. After 1000 beans the persistence context adapts such that it does not hold all the beans.

findStream() vs findList().stream()

These look similar but have different resource semantics - choose based on result size and how you consume it.

• findList().stream() executes the query, materialises the rows, releases the connection, and then streams over an in-memory list. There are no open database resources to manage and no try with resources block is needed. Prefer this for small or bounded results (for example when you apply setMaxRows) that you intend to collect anyway - it is the simpler, safer default for list-sized results.
• findStream() streams rows directly from the JDBC cursor. It holds a connection (and an implicit transaction) open for the whole lifetime of the stream pipeline, so it must be closed via try with resources. Prefer it when the result may be large, when you want constant memory use, or when you want to short-circuit (limit, findFirst, takeWhile) without loading everything.

Rule of thumb: reach for findStream / findEach / findIterate for large or lazily-consumed results; for small, fully-materialised results findList().stream() is clearer and does not hold database resources open.

findEach

Execute the query processing the result one bean at a time.

new QCustomer()
 .status.equalTo(Status.NEW)
 .order().id.asc()
 .findEach((Customer customer) -> {

   // do something with customer
   System.out.println("-- visit " + customer);
 });

findEachWhile

Like findEach but takes a predicate that allows us to stop processing the result early.

// Returning false inside the predicate will stop the execution
new QCustomer()
   .status.equalTo(Status.NEW)
   .order().id.asc()
   .findEachWhile((Customer customer) -> {
     // do something with customer
     ...
     // return true to continue processing or false to stop
     return (customer.getId() < 40);
   });

findEach with batch consumer

Similar to findEach but batches the beans up so that they can be processed in batches - for example process in batches of 100 at a time.

new QCustomer()
 .status.equalTo(Status.NEW)
 .order().id.asc()
 .findEach(100, batch -> {
   // process the customers in batches of 100 at a time
   // where batch is List<Customer>
   ...
 });

findStream

Execute the query processing the result as a stream. Use a try with resources block to ensure the resources held by the stream are closed. The connection (and implicit transaction) is held open until the stream is closed, so if you simply want the whole result as a list prefer findList / findList().stream() (see findStream() vs findList().stream()).

try (Stream<Customer> stream =
  new QCustomer()
   .status.equalTo(Status.NEW)
   .order().id.asc()
   .findStream()) {

  stream
    .filter(...)
    .map(..)
    .collect(...);
}

findIterate

Execute the query processing the result as a QueryIterator. Use a try with resources block to ensure the resources held by the iterator are closed.

try (QueryIterator<Customer> it =
  new QCustomer()
   .status.equalTo(Status.NEW)
   .order().id.asc()
   .findIterate()) {

   while (it.hasNext()) {
     Customer customer = it.next();
     ...
   }
}