Package com.broadleafcommerce.search.index.core.queue

Class AbstractBatchIdQueueLoader<ID extends Serializable,T extends BatchIdHolder<ID>>

java.lang.Object
com.broadleafcommerce.search.index.core.queue.AbstractQueueLoader<ID,T>
com.broadleafcommerce.search.index.core.queue.AbstractBatchIdQueueLoader<ID,T>
Type Parameters:
ID - This is the type of IDs that are being read from a data source in batch
T - This is the type that will be added to the ReindexQueue. It's an instance of BatchIdHolder.
All Implemented Interfaces:
IndexableTypeAware, QueueLoader<T>
public abstract class AbstractBatchIdQueueLoader<ID extends Serializable,T extends BatchIdHolder<ID>> extends AbstractQueueLoader<ID,T>
Abstract component to handle reading ID ranges from a source and populating a Queue for further retrieval. Instances are expected to be singletons.
Author:
Kelly Tisdell (ktisdell)

  • Constructor Details

    • AbstractBatchIdQueueLoader

      public AbstractBatchIdQueueLoader(@NonNull @NonNull com.broadleafcommerce.search.api.type.IndexableType indexableType, long waitTime, int maxTries, int batchReadSize, int chunkSize)

  • Method Details

    • getIndexableType

      public final com.broadleafcommerce.search.api.type.IndexableType getIndexableType()
      Description copied from interface: IndexableTypeAware
      Components that implement this method should return the same, non-null value on each invocation. This should basically return a constant value.
      Returns:
      the associated indexable type

    • initialize

      protected final void initialize()
      Description copied from class: AbstractQueueLoader
      This will be called exactly once prior to a call to readPage. This method may do nothing, but implementors may use it to reset state such as a cursor. This method should set the state of this object so that the next call to readNextIds will begin reading batches of IDs at the beginning of the list.
      Specified by:
      initialize in class AbstractQueueLoader<ID extends Serializable,T extends BatchIdHolder<ID>>

    • readPage

      protected final List<ID> readPage() throws com.broadleafcommerce.search.api.SearchIndexException
      Specified by:
      readPage in class AbstractQueueLoader<ID extends Serializable,T extends BatchIdHolder<ID>>
      Throws:
      com.broadleafcommerce.search.api.SearchIndexException

    • readPage

      protected final List<ID> readPage(ProcessStateHolder processStateHolder) throws com.broadleafcommerce.search.api.SearchIndexException
      Description copied from class: AbstractQueueLoader
      Reads paged data to, optionally, transform and put on a BlockingQueue. This could be a list of IDs, Indexable objects, or any number of things.

      Implementors MUST return an empty list when there is no more data to return. Implementors must also keep track of their own pagination state. The initialize method will be called prior to the first call to this method.

      The results of this call will be passed to the transform method and then put on a queue for more intensive processing, transformation, persistence, etc.

      Most typically, this will read a rather large List of IDs. Additionally this sends process state information for potential filters.

      Specified by:
      readPage in class AbstractQueueLoader<ID extends Serializable,T extends BatchIdHolder<ID>>
      Parameters:
      processStateHolder - the state holder process information for the index thread
      Returns:
      a paged list of FROM items
      Throws:
      com.broadleafcommerce.search.api.SearchIndexException - if an error occurs reading paged data

    • readPagedIds

      protected abstract List<ID> readPagedIds(@Nullable ID startId, int batchSize) throws com.broadleafcommerce.search.api.SearchIndexException
      Reads pages of IDs. The startId parameter begins empty. The batchSize parameter will be the same value each time. The startId parameter will be updated to the last read ID on each additional invocation.
      Parameters:
      startId - (optional) the last ID of the previous page, or null if first page
      batchSize - the size of the page to read
      Returns:
      page of IDs read past the startId
      Throws:
      com.broadleafcommerce.search.api.SearchIndexException - if an error occurs while reading IDs

    • readPagedIds

      protected abstract List<ID> readPagedIds(@Nullable ID startId, int batchSize, @Nullable Map<String,Object> additionalProperties) throws com.broadleafcommerce.search.api.SearchIndexException
      Reads pages of IDs. The startId parameter begins empty. The batchSize parameter will be the same value each time. The startId parameter will be updated to the last read ID on each additional invocation. Contains additional parameters to allow filtering of the indexed items.
      Parameters:
      startId - (optional) the last ID of the previous page, or null if first page
      batchSize - the size of the page to read
      additionalProperties - additional information that can be used to filter the items to be indexed
      Returns:
      page of IDs read past the startId
      Throws:
      com.broadleafcommerce.search.api.SearchIndexException - if an error occurs while reading IDs

    • transform

      protected final List<T> transform(List<ID> from)
      Breaks the provided list of IDs into chunks of appropriate size.
      Specified by:
      transform in class AbstractQueueLoader<ID extends Serializable,T extends BatchIdHolder<ID>>
      Parameters:
      from - a non-chunked list of IDs
      Returns:
      IDs split into BatchIdHolder chunks

    • countIndexablesToProcess

      protected final int countIndexablesToProcess(List<T> queueEntries)
      Description copied from class: AbstractQueueLoader
      Determine how many Indexables are being processed. Since (queueEntry) is generic and could be a Collection, a BatchIdHolder, or some other type, we provide this method so that implementors can return a count of the Indexable items that are expected to be processed by an AbstractWorker at the other end of the Queue.

      Return 0 if there is no way to know the number of indexables (e.g. if the queueEntry is an ID range).

      Specified by:
      countIndexablesToProcess in class AbstractQueueLoader<ID extends Serializable,T extends BatchIdHolder<ID>>
      Parameters:
      queueEntries - the entries on the queue for which to count expected indexables
      Returns:
      the number of expected indexables to index for the queueEntries