Package com.broadleafcommerce.search.index.core.queue

Class AbstractBatchIdQueueLoader<ID extends Serializable,​T extends BatchIdHolder<ID>>

  • Type Parameters:
    ID - This is the type of IDs that are being read from a data source in batch
    T - This is the type that will be added to the ReindexQueue. It's an instance of BatchIdHolder.
    All Implemented Interfaces:
    IndexableTypeAware, QueueLoader<T>
    public abstract class AbstractBatchIdQueueLoader<ID extends Serializable,​T extends BatchIdHolder<ID>>
extends AbstractQueueLoader<ID,​T>
    Abstract component to handle reading ID ranges from a source and populating a Queue for further retrieval. Instances are expected to be singletons.
    Author:
    Kelly Tisdell (ktisdell)

    • Constructor Detail

      • AbstractBatchIdQueueLoader

        public AbstractBatchIdQueueLoader​(@NonNull
                                  @NonNull com.broadleafcommerce.search.api.type.IndexableType indexableType,
                                  long waitTime,
                                  int maxTries,
                                  int batchReadSize,
                                  int chunkSize)

    • Method Detail

      • getIndexableType

        public final com.broadleafcommerce.search.api.type.IndexableType getIndexableType()
        Description copied from interface: IndexableTypeAware
        Components that implement this method should return the same, non-null value on each invocation. This should basically return a constant value.
        Returns:
        the associated indexable type

      • initialize

        protected final void initialize()
        Description copied from class: AbstractQueueLoader
        This will be called exactly once prior to a call to readPage. This method may do nothing, but implementors may use it to reset state such as a cursor. This method should set the state of this object so that the next call to readNextIds will begin reading batches of IDs at the beginning of the list.
        Specified by:
        initialize in class AbstractQueueLoader<ID extends Serializable,​T extends BatchIdHolder<ID>>

      • readPage

        protected final List<ID> readPage()
                           throws com.broadleafcommerce.search.api.SearchIndexException
        Description copied from class: AbstractQueueLoader
        Reads paged data to, optionally, transform and put on a BlockingQueue. This could be a list of IDs, Indexable objects, or any number of things.

        Implementors MUST return an empty list when there is no more data to return. Implementors must also keep track of their own pagination state. The initialize method will be called prior to the first call to this method.

        The results of this call will be passed to the transform method and then put on a queue for more intensive processing, transformation, persistence, etc.

        Most typically, this will read a rather large List of IDs.

        Specified by:
        readPage in class AbstractQueueLoader<ID extends Serializable,​T extends BatchIdHolder<ID>>
        Returns:
        a paged list of FROM items
        Throws:
        com.broadleafcommerce.search.api.SearchIndexException - if an error occurs reading paged data

      • readPagedIds

        protected abstract List<ID> readPagedIds​(@Nullable
                                         ID startId,
                                         int batchSize)
                                  throws com.broadleafcommerce.search.api.SearchIndexException
        Reads pages of IDs. The startId parameter begins empty. The batchSize parameter will be the same value each time. The startId parameter will be updated to the last read ID on each additional invocation.
        Parameters:
        startId - (optional) the last ID of the previous page, or null if first page
        batchSize - the size of the page to read
        Returns:
        page of IDs read past the startId
        Throws:
        com.broadleafcommerce.search.api.SearchIndexException - if an error occurs while reading IDs

      • countIndexablesToProcess

        protected final int countIndexablesToProcess​(List<T> queueEntries)
        Description copied from class: AbstractQueueLoader
        Determine how many Indexables are being processed. Since (queueEntry) is generic and could be a Collection, a BatchIdHolder, or some other type, we provide this method so that implementors can return a count of the Indexable items that are expected to be processed by an AbstractWorker at the other end of the Queue.

        Return 0 if there is no way to know the number of indexables (e.g. if the queueEntry is an ID range).

        Specified by:
        countIndexablesToProcess in class AbstractQueueLoader<ID extends Serializable,​T extends BatchIdHolder<ID>>
        Parameters:
        queueEntries - the entries on the queue for which to count expected indexables
        Returns:
        the number of expected indexables to index for the queueEntries