Package com.broadleafcommerce.asset.storage.provider.service

Class FilesystemStorageProvider

java.lang.Object
com.broadleafcommerce.asset.storage.provider.service.FilesystemStorageProvider
All Implemented Interfaces:
StorageProvider
public class FilesystemStorageProvider extends Object implements StorageProvider

Default implementation of StorageProvider specifically designed to work with the local or shared filesystem as the storage provider for digital content related to Asset.

This StorageProvider can only be used in production systems that run on a single server or those that have a shared filesystem mounted to the application servers.
Author:
Nathan Moore (nathanmoore)., Samarth Dhruva (samarthd)

  • Field Details

  • Constructor Details

  • Method Details

    • isWhitelistedMimeType

      public boolean isWhitelistedMimeType(@NonNull String mimeType)
      Description copied from interface: StorageProvider
      Determines whether the given MIME type string is supported by this provider. Files with MIME types not supported, cannot be stored using this provider. The whitelist can be configured using a system property of the form broadleaf.asset.internal.storageProvider.mimeTypeWhitelist.

      If the whitelist is empty, then all types are considered whitelisted.

      Specified by:
      isWhitelistedMimeType in interface StorageProvider
      Parameters:
      mimeType - the mime type to check against the provider's whitelist
      Returns:
      whether the given MIME type string is supported by this provider.
      See Also:

    • resolveExistingActualStorageLocations

      protected Map<String,Path> resolveExistingActualStorageLocations(Collection<String> urls)
      Used during resource read/delete operations to figure out what the actual storage locations are for some given URLs.
      Parameters:
      urls - the URLs the caller provided to this storage provider to associate with some binary data. This is the value callers will give to the storage provider to store/read/delete the resources. These are expected to already be globally unique.
      Returns:
      a map from the originally provided URL to the actual storage location for it
      Since:
      2.0.3
      See Also:

    • findActualStorageLocationsFromMappings

      protected Map<String,String> findActualStorageLocationsFromMappings(Collection<String> urls)
      Queries against StorageLocationMappingService to find the actual storage locations from the mapping records, if available in the system.
      Parameters:
      urls - the URLs the caller provided to this storage provider to associate with some binary data. This is the value callers will give to the storage provider to store/read/delete the resources. These are expected to already be globally unique.
      Returns:
      a map from each provided URL to the found actual storage location. If a URL did not have a mapping found, it will be omitted in the map.
      Since:
      2.0.3

    • resolveExistingActualStorageLocation

      protected Path resolveExistingActualStorageLocation(String url)
      Similar to resolveExistingActualStorageLocations(Collection), but only for one URL.
      Parameters:
      url - the URL the caller provided to this storage provider to associate with some binary data. This is the value callers will give to the storage provider to store/read/delete the resource. This is expected to already be globally unique.
      Returns:
      the actual storage location/path which this provider is using to actually store the data for the url
      Since:
      2.0.3
      See Also:

    • generateActualStorageLocations

      protected Map<String,Path> generateActualStorageLocations(Collection<String> urls)

      Used during a 'resource add' operation to determine where new assets should actually be stored.

      Note - as of 2.0.3, all new resources will be stored in paths determined by this method.

      Parameters:
      urls - the URLs the caller provided to this storage provider to associate with some binary data. This is the value callers will give to the storage provider to store/read/delete the resources. These are expected to already be globally unique.
      Returns:
      a map from each provided URL to a new unique location/path which this provider will use to actually store data for that URL. The mappings in this result should be stored via StorageLocationMappingService.
      Since:
      2.0.3
      See Also:

    • generateActualStorageLocation

      protected Path generateActualStorageLocation(String url)
      Similar to generateActualStorageLocations(Collection), but only for one URL.
      Parameters:
      url - the URL the caller provided to this storage provider to associate with some binary data. This is the value callers will give to the storage provider to store/read/delete the resource. This is expected to already be globally unique.
      Returns:
      a new unique location/path which this provider will use to actually store data for the provided URL. The mapping from the input URL to this result should be stored via StorageLocationMappingService.
      Since:
      2.0.3
      See Also:

    • calculateHashedStorageLocationDirectory

      protected Path calculateHashedStorageLocationDirectory(String url)

      Determines the location of the directory (not a full file path) that should be used to actually store the data for the provided url.

      To ensure that files can be stored and accessed in an efficient manner, the system creates directories based on the characters in the MD5 hash of the passed in URL.

      For example, if the original URL is /product/myproductimage.jpg, then the MD5 would be 35ec52a8dbd8cf3e2c650495001fe55f. Then, if InternalAssetProperties.StorageProvider.Filesystem.getMaxGeneratedDirectoryDepth() was 2, the directory location would end up being '{providerRootLocation}/35/ec'.

      The hash for the filename will include a beginning slash before performing the MD5. This is done largely for backward compatibility with similar functionality in BLC 3.0.0.

      This algorithm's advantage is efficient filesystem storage with a balanced distribution of files.

      If needed, implementations can consider increasing the InternalAssetProperties.StorageProvider.Filesystem.getMaxGeneratedDirectoryDepth() property, or potentially override this method altogether to use an alternate approach.

      Parameters:
      url - the URL the caller provided to this storage provider to associate with some binary data. This is the value callers will give to the storage provider to store/read/delete the resource. This is expected to already be globally unique.
      Returns:
      the location of the directory (not a full file path) that should be used to store the data for the provided url

    • generateGloballyUniqueFilename

      protected String generateGloballyUniqueFilename(String url)
      Parameters:
      url - the URL the caller provided to this storage provider to associate with some binary data. This is the value callers will give to the storage provider to store/read/delete the resource. This is expected to already be globally unique.
      Returns:
      a globally unique filename this provider should use when actually storing the data. Uniqueness is very important, as it ensures even when calculateHashedStorageLocationDirectory(String) places two similarly named assets in the same directory, each will have their own unique path.

    • getFileExtensionWithDot

      @Nullable protected String getFileExtensionWithDot(String url)

    • getHashedResourcePath

      protected Path getHashedResourcePath(String url)

      This method represents the 'legacy' storage path calculation. This is only around as a backwards compatibility measure. New assets will use generateActualStorageLocation(String) and resolveExistingActualStorageLocation(String).

      Determines the real location of the file on the file-system by performing an MD5 hash of the passed in URL.

      To ensure that files can be stored and accessed in an efficient manner, the system creates directories based on the characters in the hash.

      For example, if the URL is /product/myproductimage.jpg, then the MD5 would be 35ec52a8dbd8cf3e2c650495001fe55f resulting in the following file on the filesystem {providerRootLocation}/35/ec/myproductimage.jpg.

      The hash for the filename will include a beginning slash before performing the MD5. This is done largely for backward compatibility with similar functionality in BLC 3.0.0.

      This algorithm's advantage is efficient filesystem storage with a balanced distribution of files.

      If support for more files is needed, implementors should consider one of the following approaches: 1. Overriding the InternalAssetProperties.StorageProvider.Filesystem.getMaxGeneratedDirectoryDepth() property from its default of 2 to 3 2. Overriding this method to introduce an alternate approach

      Parameters:
      url - The URL used to represent an asset for which a name on the fileSystem is desired.
      Returns:
      The actual location of the file in the filesystem (using filesystem separators)
      See Also:

    • readResource

      @NonNull public Optional<File> readResource(@NonNull String path)
      Description copied from interface: StorageProvider
      Retrieves the resource in the configured storage provider at the given path string if it exists.
      Specified by:
      readResource in interface StorageProvider
      Parameters:
      path - fully qualified path to the resource's location in the storage provider
      Returns:
      Either the resource wrapped in an Optional if it exists or else Optional.EMPTY.

    • readStreamableResourceWithMetadata

      public Optional<ResourceWithMetadata> readStreamableResourceWithMetadata(@NonNull @NonNull String path)
      Description copied from interface: StorageProvider
      Retrieves the resource in the configured storage provider at the given path string if it exists.
      Specified by:
      readStreamableResourceWithMetadata in interface StorageProvider
      Parameters:
      path - fully qualified path to the resource's location in the storage provider
      Returns:
      Either the resource wrapped in an Optional if it exists or else Optional.empty().

    • readResources

      @NonNull public Iterable<File> readResources(@NonNull Iterable<String> paths)
      Description copied from interface: StorageProvider
      Retrieves the resources in the configured storage provider at the given path strings if they exist.
      Specified by:
      readResources in interface StorageProvider
      Parameters:
      paths - fully qualified path to the resources' locations in the storage provider
      Returns:
      Iterable of resources that were found

    • deleteResource

      public void deleteResource(@NonNull String path)
      Description copied from interface: StorageProvider
      Deletes the resource in the configured storage provider at the given path string if it exists.
      Specified by:
      deleteResource in interface StorageProvider
      Parameters:
      path - fully qualified path to the resource's location in the storage provider

    • deleteResources

      public void deleteResources(@NonNull Iterable<String> paths)
      Description copied from interface: StorageProvider
      Deletes the resources in the configured storage provider at the given path strings if they exist. This method will attempt to remove all the resources it can successfully before throwing a BulkStorageException containing all the failures and their causes.
      Specified by:
      deleteResources in interface StorageProvider
      Parameters:
      paths - fully qualified paths to the resources' locations in the storage provider

    • addResourceFromStream

      @NonNull public String addResourceFromStream(@NonNull InputStream resource, @NonNull String fileName)
      Description copied from interface: StorageProvider
      Adds a new resource to the configured storage provider using the provided InputStream and filename. Then, it returns the fully qualified path of the newly stored resource.
      Specified by:
      addResourceFromStream in interface StorageProvider
      Parameters:
      resource - resource to add to the configured storage provider
      fileName - the name of the resource, potentially including path segments.
      Returns:
      the fully qualified path to the added resource

    • deleteMappingRecords

      protected void deleteMappingRecords(Collection<String> originalLocations)

    • createNewMappingRecord

      @Nullable protected StorageLocationMapping createNewMappingRecord(String url, String actualResourcePath)
      Similar to createNewMappingRecords(Map), but only for one record.
      Parameters:
      url - the URL the caller provided to this storage provider to associate with some binary data. This is the value callers will give to the storage provider to store/read/delete the resource. This is expected to already be globally unique.
      actualResourcePath - the actual storage location/path which this provider is using to actually store the data for the url
      Returns:
      the created StorageLocationMapping if successful, null otherwise

    • buildStorageLocationMapping

      protected StorageLocationMapping buildStorageLocationMapping(String url, String actualResourcePath)

    • doWithoutPolicyValidation

      protected <R> R doWithoutPolicyValidation(Supplier<R> returningWork)
      Executes the given returningWork with security policy validations disabled.
      Type Parameters:
      R - the return type of the work
      Parameters:
      returningWork - the work to execute
      Returns:
      the result of executing the work

    • addResourcesFromStreams

      @NonNull public BulkAddResourcesResponse addResourcesFromStreams(@NonNull Map<String,InputStream> fileNamesToResources)
      Description copied from interface: StorageProvider

      Adds new resources to the configured storage provider using the provided Map of InputStreams and filenames. Then, it returns a BulkAddResourcesResponse containing the information about which files succeeded and which files failed.

      Specified by:
      addResourcesFromStreams in interface StorageProvider
      Parameters:
      fileNamesToResources - a map of filenames to resources that should be copied to the storage provider. The filenames can include path segments.
      Returns:
      a BulkAddResourcesResponse containing the details of which files did and didn't succeed

    • createNewMappingRecords

      protected Map<String,StorageLocationMapping> createNewMappingRecords(Map<String,String> actualResourcePathsByOriginalLocation)
      Establishes new StorageLocationMapping records in StorageLocationMappingService for the given values.
      Parameters:
      actualResourcePathsByOriginalLocation - a map from original location values to the actual resource paths for them
      Returns:
      the created StorageLocationMapping records mapped by the original location

    • mapGivenPathsToHashedPaths

      @Deprecated(since="2.0.3", forRemoval=true) protected Map<String,Path> mapGivenPathsToHashedPaths(Collection<String> fileNamesList)
      Deprecated, for removal: This API element is subject to removal in a future version.
      since 2.0.3, not used by the framework anymore. Use generateActualStorageLocations(Collection) or resolveExistingActualStorageLocations(Collection) instead.
      Parameters:
      fileNamesList - given paths to return the hashed representation for
      Returns:
      hashed paths for each given path

    • addResourcesFromFiles

      @NonNull public BulkAddResourcesResponse addResourcesFromFiles(@NonNull @NonNull @NonNull Map<String,File> fileNamesToResources)
      Description copied from interface: StorageProvider
      Adds new resources to the configured storage provider using the provided Map of Files and filenames. Then, it returns a BulkAddResourcesResponse containing the information about which files succeeded and which files failed.
      Specified by:
      addResourcesFromFiles in interface StorageProvider
      Parameters:
      fileNamesToResources - a map of filenames to resources that should be copied to the storage provider. The filenames can include path segments.
      Returns:
      a BulkAddResourcesResponse containing the details of which files did and didn't succeed

    • copyStreamToFilePath

      protected void copyStreamToFilePath(InputStream resource, Path actualResourcePath)

    • validateAndCleanFilePath

      protected String validateAndCleanFilePath(String fileName)

    • getMaxGeneratedDirectoryDepth

      protected int getMaxGeneratedDirectoryDepth()
      Returns:
      InternalAssetProperties.StorageProvider.Filesystem.getMaxGeneratedDirectoryDepth() or the current application's override.

    • assertPathNotNull

      protected void assertPathNotNull(String path)

    • assertAllPathsNotNull

      protected void assertAllPathsNotNull(Iterable<String> paths)

    • assertResourceNotNull

      protected void assertResourceNotNull(InputStream resource)

    • assertAllResourcesNotNull

      protected void assertAllResourcesNotNull(Iterable<InputStream> resources)

    • getStorageLocationMappingProviderType

      protected String getStorageLocationMappingProviderType()
      Returns:
      the value used for StorageLocationMapping.getStorageProviderType()

    • setTypeFactory

      @Autowired public void setTypeFactory(com.broadleafcommerce.common.extension.TypeFactory typeFactory)

    • getTypeFactory

      protected com.broadleafcommerce.common.extension.TypeFactory getTypeFactory()

    • setMimeFileTypeMap

      @Autowired public void setMimeFileTypeMap(jakarta.activation.FileTypeMap mimeFileTypeMap)

    • getMimeFileTypeMap

      protected jakarta.activation.FileTypeMap getMimeFileTypeMap()

    • setStorageLocationMappingSvc

      @Autowired public void setStorageLocationMappingSvc(StorageLocationMappingService<StorageLocationMapping> storageLocationMappingSvc)

    • getStorageLocationMappingSvc

      protected StorageLocationMappingService<StorageLocationMapping> getStorageLocationMappingSvc()