BlueSpice MediaWiki REL1_27
 All Classes Namespaces Files Functions Variables Groups Pages
FileBackendStore Class Reference

Base class for all backends using particular storage medium. More...

Inheritance diagram for FileBackendStore:
Collaboration diagram for FileBackendStore:

Public Member Functions

 __construct (array $config)
 
 maxFileSizeInternal ()
 
 isPathUsableInternal ($storagePath)
 
 createInternal (array $params)
 
 storeInternal (array $params)
 
 copyInternal (array $params)
 
 deleteInternal (array $params)
 
 moveInternal (array $params)
 
 describeInternal (array $params)
 
 nullInternal (array $params)
 
 concatenate (array $params)
 
 fileExists (array $params)
 
 getFileTimestamp (array $params)
 
 getFileSize (array $params)
 
 getFileStat (array $params)
 
 getFileContentsMulti (array $params)
 
 getFileXAttributes (array $params)
 
 getFileSha1Base36 (array $params)
 
 getFileProps (array $params)
 
 getLocalReferenceMulti (array $params)
 
 getLocalCopyMulti (array $params)
 
 getFileHttpUrl (array $params)
 
 streamFile (array $params)
 
 directoryExists (array $params)
 
 getDirectoryList (array $params)
 
 getDirectoryListInternal ($container, $dir, array $params)
 
 getFileList (array $params)
 
 getFileListInternal ($container, $dir, array $params)
 
 getOperationsInternal (array $ops)
 
 getPathsToLockForOpsInternal (array $performOps)
 
 getScopedLocksForOps (array $ops, Status $status)
 
 executeOpHandlesInternal (array $fileOpHandles)
 
 preloadCache (array $paths)
 
 clearCache (array $paths=null)
 
 preloadFileStat (array $params)
 
 isSingleShardPathInternal ($storagePath)
 
- Public Member Functions inherited from FileBackend
 __construct (array $config)
 
 getName ()
 
 getWikiId ()
 
 isReadOnly ()
 
 getReadOnlyReason ()
 
 getFeatures ()
 
 hasFeatures ($bitfield)
 
 doOperations (array $ops, array $opts=[])
 
 doOperation (array $op, array $opts=[])
 
 create (array $params, array $opts=[])
 
 store (array $params, array $opts=[])
 
 copy (array $params, array $opts=[])
 
 move (array $params, array $opts=[])
 
 delete (array $params, array $opts=[])
 
 describe (array $params, array $opts=[])
 
 doQuickOperations (array $ops, array $opts=[])
 
 doQuickOperation (array $op)
 
 quickCreate (array $params)
 
 quickStore (array $params)
 
 quickCopy (array $params)
 
 quickMove (array $params)
 
 quickDelete (array $params)
 
 quickDescribe (array $params)
 
 concatenate (array $params)
 
 prepare (array $params)
 
 secure (array $params)
 
 publish (array $params)
 
 clean (array $params)
 
 fileExists (array $params)
 
 getFileTimestamp (array $params)
 
 getFileContents (array $params)
 
 getFileContentsMulti (array $params)
 
 getFileXAttributes (array $params)
 
 getFileSize (array $params)
 
 getFileStat (array $params)
 
 getFileSha1Base36 (array $params)
 
 getFileProps (array $params)
 
 streamFile (array $params)
 
 getLocalReference (array $params)
 
 getLocalReferenceMulti (array $params)
 
 getLocalCopy (array $params)
 
 getLocalCopyMulti (array $params)
 
 getFileHttpUrl (array $params)
 
 directoryExists (array $params)
 
 getDirectoryList (array $params)
 
 getTopDirectoryList (array $params)
 
 getFileList (array $params)
 
 getTopFileList (array $params)
 
 preloadCache (array $paths)
 
 clearCache (array $paths=null)
 
 preloadFileStat (array $params)
 
 lockFiles (array $paths, $type, $timeout=0)
 
 unlockFiles (array $paths, $type)
 
 getScopedFileLocks (array $paths, $type, Status $status, $timeout=0)
 
 getScopedLocksForOps (array $ops, Status $status)
 
 getRootStoragePath ()
 
 getContainerStoragePath ($container)
 
 getJournal ()
 

Public Attributes

const CACHE_TTL = 10
 
const CACHE_CHEAP_SIZE = 500
 
const CACHE_EXPENSIVE_SIZE = 5
 
- Public Attributes inherited from FileBackend
const ATTR_HEADERS = 1
 
const ATTR_METADATA = 2
 
const ATTR_UNICODE_PATHS = 4
 

Protected Member Functions

 doCreateInternal (array $params)
 
 doStoreInternal (array $params)
 
 doCopyInternal (array $params)
 
 doDeleteInternal (array $params)
 
 doMoveInternal (array $params)
 
 doDescribeInternal (array $params)
 
 doConcatenate (array $params)
 
 doPrepare (array $params)
 
 doPrepareInternal ($container, $dir, array $params)
 
 doSecure (array $params)
 
 doSecureInternal ($container, $dir, array $params)
 
 doPublish (array $params)
 
 doPublishInternal ($container, $dir, array $params)
 
 doClean (array $params)
 
 doCleanInternal ($container, $dir, array $params)
 
 doGetFileStat (array $params)
 
 doGetFileContentsMulti (array $params)
 
 doGetFileXAttributes (array $params)
 
 doGetFileSha1Base36 (array $params)
 
 doGetLocalReferenceMulti (array $params)
 
 doGetLocalCopyMulti (array $params)
 
 doStreamFile (array $params)
 
 doDirectoryExists ($container, $dir, array $params)
 
 doOperationsInternal (array $ops, array $opts)
 
 doQuickOperationsInternal (array $ops)
 
 doExecuteOpHandlesInternal (array $fileOpHandles)
 
 sanitizeOpHeaders (array $op)
 
 doClearCache (array $paths=null)
 
 doGetFileStatMulti (array $params)
 
 directoriesAreVirtual ()
 
 resolveStoragePath ($storagePath)
 
 resolveStoragePathReal ($storagePath)
 
 getContainerShard ($container, $relPath)
 
 getContainerHashLevels ($container)
 
 getContainerSuffixes ($container)
 
 fullContainerName ($container)
 
 resolveContainerName ($container)
 
 resolveContainerPath ($container, $relStoragePath)
 
 setContainerCache ($container, array $val)
 
 deleteContainerCache ($container)
 
 primeContainerCache (array $items)
 
 doPrimeContainerCache (array $containerInfo)
 
 setFileCache ($path, array $val)
 
 deleteFileCache ($path)
 
 primeFileCache (array $items)
 
 setConcurrencyFlags (array $opts)
 
 getContentType ($storagePath, $content, $fsPath)
 
- Protected Member Functions inherited from FileBackend
 doOperationsInternal (array $ops, array $opts)
 
 doQuickOperationsInternal (array $ops)
 
 doPrepare (array $params)
 
 doSecure (array $params)
 
 doPublish (array $params)
 
 doClean (array $params)
 
 getScopedPHPBehaviorForOps ()
 
 resolveFSFileObjects (array $ops)
 

Static Protected Member Functions

static isValidShortContainerName ($container)
 
static isValidContainerName ($container)
 
static normalizeXAttributes (array $xattr)
 
- Static Protected Member Functions inherited from FileBackend
static normalizeContainerPath ($path)
 

Protected Attributes

 $memCache
 
 $cheapCache
 
 $expensiveCache
 
 $shardViaHashLevels = []
 
 $mimeCallback
 
 $maxFileSize = 4294967296
 
- Protected Attributes inherited from FileBackend
 $name
 
 $wikiId
 
 $readOnly
 
 $parallelize
 
 $concurrency
 
 $lockManager
 
 $fileJournal
 

Additional Inherited Members

- Static Public Member Functions inherited from FileBackend
static isStoragePath ($path)
 
static splitStoragePath ($storagePath)
 
static normalizeStoragePath ($storagePath)
 
static parentStoragePath ($storagePath)
 
static extensionFromPath ($path, $case= 'lowercase')
 
static isPathTraversalFree ($path)
 
static makeContentDisposition ($type, $filename= '')
 

Detailed Description

Base class for all backends using particular storage medium.

This class defines the methods as abstract that subclasses must implement. Outside callers should not use functions with "Internal" in the name.

The FileBackend operations are implemented using basic functions such as storeInternal(), copyInternal(), deleteInternal() and the like. This class is also responsible for path resolution and sanitization.

Since
1.19

Constructor & Destructor Documentation

FileBackendStore::__construct ( array  $config)
See Also
FileBackend::__construct() Additional $config params include:
  • wanCache : WANObjectCache object to use for persistent caching.
  • mimeCallback : Callback that takes (storage path, content, file system path) and returns the MIME type of the file or 'unknown/unknown'. The file system path parameter should be used if the content one is null.
Parameters
array$config

Member Function Documentation

FileBackendStore::copyInternal ( array  $params)
final

Copy a file from one storage path to another in the backend. This will overwrite any file that exists at the destination. Do not call this function from places outside FileBackend and FileOp.

$params include:

  • src : source storage path
  • dst : destination storage path
  • ignoreMissingSource : do nothing if the source file does not exist
  • headers : HTTP header name/value map
  • async : Status will be returned immediately if supported. If the status is OK, then its value field will be set to a FileBackendStoreOpHandle object.
  • dstExists : Whether a file exists at the destination (optimization). Callers can use "false" if no existing file is being changed.
Parameters
array$params
Returns
Status
FileBackendStore::createInternal ( array  $params)
final

Create a file in the backend with the given contents. This will overwrite any file that exists at the destination. Do not call this function from places outside FileBackend and FileOp.

$params include:

  • content : the raw file contents
  • dst : destination storage path
  • headers : HTTP header name/value map
  • async : Status will be returned immediately if supported. If the status is OK, then its value field will be set to a FileBackendStoreOpHandle object.
  • dstExists : Whether a file exists at the destination (optimization). Callers can use "false" if no existing file is being changed.
Parameters
array$params
Returns
Status
FileBackendStore::deleteContainerCache (   $container)
finalprotected

Delete the cached info for a container. The cache key is salted for a while to prevent race conditions.

Parameters
string$containerResolved container name
FileBackendStore::deleteFileCache (   $path)
finalprotected

Delete the cached stat info for a file path. The cache key is salted for a while to prevent race conditions. Since negatives (404s) are not cached, this does not need to be called when a file is created at a path were there was none before.

Parameters
string$pathStorage path
FileBackendStore::deleteInternal ( array  $params)
final

Delete a file at the storage path. Do not call this function from places outside FileBackend and FileOp.

$params include:

  • src : source storage path
  • ignoreMissingSource : do nothing if the source file does not exist
  • async : Status will be returned immediately if supported. If the status is OK, then its value field will be set to a FileBackendStoreOpHandle object.
Parameters
array$params
Returns
Status
FileBackendStore::describeInternal ( array  $params)
final

Alter metadata for a file at the storage path. Do not call this function from places outside FileBackend and FileOp.

$params include:

  • src : source storage path
  • headers : HTTP header name/value map
  • async : Status will be returned immediately if supported. If the status is OK, then its value field will be set to a FileBackendStoreOpHandle object.
Parameters
array$params
Returns
Status
FileBackendStore::directoriesAreVirtual ( )
abstractprotected

Is this a key/value store where directories are just virtual? Virtual directories exists in so much as files exists that are prefixed with the directory path followed by a forward slash.

Returns
bool
FileBackendStore::doCleanInternal (   $container,
  $dir,
array  $params 
)
protected
See Also
FileBackendStore::doClean()
Parameters
string$container
string$dir
array$params
Returns
Status
FileBackendStore::doClearCache ( array  $paths = null)
protected

Clears any additional stat caches for storage paths

See Also
FileBackend::clearCache()
Parameters
array$pathsStorage paths (optional)
FileBackendStore::doConcatenate ( array  $params)
protected
See Also
FileBackendStore::concatenate()
Parameters
array$params
Returns
Status
FileBackendStore::doCopyInternal ( array  $params)
abstractprotected
See Also
FileBackendStore::copyInternal()
Parameters
array$params
Returns
Status
FileBackendStore::doCreateInternal ( array  $params)
abstractprotected
See Also
FileBackendStore::createInternal()
Parameters
array$params
Returns
Status
FileBackendStore::doDeleteInternal ( array  $params)
abstractprotected
See Also
FileBackendStore::deleteInternal()
Parameters
array$params
Returns
Status
FileBackendStore::doDescribeInternal ( array  $params)
protected
See Also
FileBackendStore::describeInternal()
Parameters
array$params
Returns
Status
FileBackendStore::doDirectoryExists (   $container,
  $dir,
array  $params 
)
abstractprotected
See Also
FileBackendStore::directoryExists()
Parameters
string$containerResolved container name
string$dirResolved path relative to container
array$params
Returns
bool|null
FileBackendStore::doExecuteOpHandlesInternal ( array  $fileOpHandles)
protected
See Also
FileBackendStore::executeOpHandlesInternal()
Parameters
FileBackendStoreOpHandle[]$fileOpHandles
Exceptions
FileBackendError
Returns
Status[] List of corresponding Status objects
FileBackendStore::doGetFileContentsMulti ( array  $params)
protected
See Also
FileBackendStore::getFileContentsMulti()
Parameters
array$params
Returns
array
FileBackendStore::doGetFileSha1Base36 ( array  $params)
protected
See Also
FileBackendStore::getFileSha1Base36()
Parameters
array$params
Returns
bool|string
FileBackendStore::doGetFileStat ( array  $params)
abstractprotected
See Also
FileBackendStore::getFileStat()
FileBackendStore::doGetFileStatMulti ( array  $params)
protected

Get file stat information (concurrently if possible) for several files

See Also
FileBackend::getFileStat()
Parameters
array$paramsParameters include:
  • srcs : list of source storage paths
  • latest : use the latest available data
Returns
array|null Map of storage paths to array|bool|null (returns null if not supported)
Since
1.23
FileBackendStore::doGetFileXAttributes ( array  $params)
protected
See Also
FileBackendStore::getFileXAttributes()
Returns
bool|string
FileBackendStore::doGetLocalCopyMulti ( array  $params)
abstractprotected
See Also
FileBackendStore::getLocalCopyMulti()
Parameters
array$params
Returns
array
FileBackendStore::doGetLocalReferenceMulti ( array  $params)
protected
See Also
FileBackendStore::getLocalReferenceMulti()
Parameters
array$params
Returns
array
FileBackendStore::doMoveInternal ( array  $params)
protected
See Also
FileBackendStore::moveInternal()
Parameters
array$params
Returns
Status
FileBackendStore::doPrepareInternal (   $container,
  $dir,
array  $params 
)
protected
See Also
FileBackendStore::doPrepare()
Parameters
string$container
string$dir
array$params
Returns
Status
FileBackendStore::doPrimeContainerCache ( array  $containerInfo)
protected

Fill the backend-specific process cache given an array of resolved container names and their corresponding cached info. Only containers that actually exist should appear in the map.

Parameters
array$containerInfoMap of resolved container names to cached info
FileBackendStore::doPublishInternal (   $container,
  $dir,
array  $params 
)
protected
See Also
FileBackendStore::doPublish()
Parameters
string$container
string$dir
array$params
Returns
Status
FileBackendStore::doSecureInternal (   $container,
  $dir,
array  $params 
)
protected
See Also
FileBackendStore::doSecure()
Parameters
string$container
string$dir
array$params
Returns
Status
FileBackendStore::doStoreInternal ( array  $params)
abstractprotected
See Also
FileBackendStore::storeInternal()
Parameters
array$params
Returns
Status
FileBackendStore::doStreamFile ( array  $params)
protected
See Also
FileBackendStore::streamFile()
Parameters
array$params
Returns
Status
FileBackendStore::executeOpHandlesInternal ( array  $fileOpHandles)
final

Execute a list of FileBackendStoreOpHandle handles in parallel. The resulting Status object fields will correspond to the order in which the handles where given.

Parameters
FileBackendStoreOpHandle[]$fileOpHandles
Exceptions
FileBackendError
Returns
array Map of Status objects
FileBackendStore::fullContainerName (   $container)
finalprotected

Get the full container name, including the wiki ID prefix

Parameters
string$container
Returns
string
FileBackendStore::getContainerHashLevels (   $container)
finalprotected

Get the sharding config for a container. If greater than 0, then all file storage paths within the container are required to be hashed accordingly.

Parameters
string$container
Returns
array (integer levels, integer base, repeat flag) or (0, 0, false)
FileBackendStore::getContainerShard (   $container,
  $relPath 
)
finalprotected

Get the container name shard suffix for a given path. Any empty suffix means the container is not sharded.

Parameters
string$containerContainer name
string$relPathStorage path relative to the container
Returns
string|null Returns null if shard could not be determined
FileBackendStore::getContainerSuffixes (   $container)
finalprotected

Get a list of full container shard suffixes for a container

Parameters
string$container
Returns
array
FileBackendStore::getContentType (   $storagePath,
  $content,
  $fsPath 
)
protected

Get the content type to use in HEAD/GET requests for a file

Parameters
string$storagePath
string | null$contentFile data
string | null$fsPathFile system path
Returns
string MIME type
FileBackendStore::getDirectoryListInternal (   $container,
  $dir,
array  $params 
)
abstract

Do not call this function from places outside FileBackend

See Also
FileBackendStore::getDirectoryList()
Parameters
string$containerResolved container name
string$dirResolved path relative to container
array$params
Returns
Traversable|array|null Returns null on failure
FileBackendStore::getFileHttpUrl ( array  $params)
See Also
FileBackend::getFileHttpUrl()
Parameters
array$params
Returns
string|null
FileBackendStore::getFileListInternal (   $container,
  $dir,
array  $params 
)
abstract

Do not call this function from places outside FileBackend

See Also
FileBackendStore::getFileList()
Parameters
string$containerResolved container name
string$dirResolved path relative to container
array$params
Returns
Traversable|array|null Returns null on failure
FileBackendStore::getOperationsInternal ( array  $ops)
final

Return a list of FileOp objects from a list of operations. Do not call this function from places outside FileBackend.

The result must have the same number of items as the input. An exception is thrown if an unsupported operation is requested.

Parameters
array$opsSame format as doOperations()
Returns
array List of FileOp objects
Exceptions
FileBackendError
FileBackendStore::getPathsToLockForOpsInternal ( array  $performOps)
final

Get a list of storage paths to lock for a list of operations Returns an array with LockManager::LOCK_UW (shared locks) and LockManager::LOCK_EX (exclusive locks) keys, each corresponding to a list of storage paths to be locked. All returned paths are normalized.

Parameters
array$performOpsList of FileOp objects
Returns
array (LockManager::LOCK_UW => path list, LockManager::LOCK_EX => path list)
FileBackendStore::isPathUsableInternal (   $storagePath)
abstract

Check if a file can be created or changed at a given storage path. FS backends should check if the parent directory exists, files can be written under it, and that any file already there is writable. Backends using key/value stores should check if the container exists.

Parameters
string$storagePath
Returns
bool
FileBackendStore::isSingleShardPathInternal (   $storagePath)
final

Check if a storage path maps to a single shard. Container dirs like "a", where the container shards on "x/xy", can reside on several shards. Such paths are tricky to handle.

Parameters
string$storagePathStorage path
Returns
bool
static FileBackendStore::isValidContainerName (   $container)
staticfinalprotected

Check if a full container name is valid

This checks for length and illegal characters. Limiting the characters makes migrations to other stores easier.

Parameters
string$container
Returns
bool
static FileBackendStore::isValidShortContainerName (   $container)
staticfinalprotected

Check if a short container name is valid

This checks for length and illegal characters. This may disallow certain characters that can appear in the prefix used to make the full container name.

Parameters
string$container
Returns
bool
FileBackendStore::maxFileSizeInternal ( )
final

Get the maximum allowable file size given backend medium restrictions and basic performance constraints. Do not call this function from places outside FileBackend and FileOp.

Returns
int Bytes
FileBackendStore::moveInternal ( array  $params)
final

Move a file from one storage path to another in the backend. This will overwrite any file that exists at the destination. Do not call this function from places outside FileBackend and FileOp.

$params include:

  • src : source storage path
  • dst : destination storage path
  • ignoreMissingSource : do nothing if the source file does not exist
  • headers : HTTP header name/value map
  • async : Status will be returned immediately if supported. If the status is OK, then its value field will be set to a FileBackendStoreOpHandle object.
  • dstExists : Whether a file exists at the destination (optimization). Callers can use "false" if no existing file is being changed.
Parameters
array$params
Returns
Status
static FileBackendStore::normalizeXAttributes ( array  $xattr)
staticfinalprotected

Normalize file headers/metadata to the FileBackend::getFileXAttributes() format

Parameters
array$xattr
Returns
array
Since
1.22
FileBackendStore::nullInternal ( array  $params)
final

No-op file operation that does nothing. Do not call this function from places outside FileBackend and FileOp.

Parameters
array$params
Returns
Status
FileBackendStore::primeContainerCache ( array  $items)
finalprotected

Do a batch lookup from cache for container stats for all containers used in a list of container names or storage paths objects. This loads the persistent cache values into the process cache.

Parameters
array$items
FileBackendStore::primeFileCache ( array  $items)
finalprotected

Do a batch lookup from cache for file stats for all paths used in a list of storage paths or FileOp objects. This loads the persistent cache values into the process cache.

Parameters
array$itemsList of storage paths
FileBackendStore::resolveContainerName (   $container)
protected

Resolve a container name, checking if it's allowed by the backend. This is intended for internal use, such as encoding illegal chars. Subclasses can override this to be more restrictive.

Parameters
string$container
Returns
string|null
FileBackendStore::resolveContainerPath (   $container,
  $relStoragePath 
)
protected

Resolve a relative storage path, checking if it's allowed by the backend. This is intended for internal use, such as encoding illegal chars or perhaps getting absolute paths (e.g. FS based backends). Note that the relative path may be the empty string (e.g. the path is simply to the container).

Parameters
string$containerContainer name
string$relStoragePathStorage path relative to the container
Returns
string|null Path or null if not valid
FileBackendStore::resolveStoragePath (   $storagePath)
finalprotected

Splits a storage path into an internal container name, an internal relative file name, and a container shard suffix. Any shard suffix is already appended to the internal container name. This also checks that the storage path is valid and within this backend.

If the container is sharded but a suffix could not be determined, this means that the path can only refer to a directory and can only be scanned by looking in all the container shards.

Parameters
string$storagePath
Returns
array (container, path, container suffix) or (null, null, null) if invalid
FileBackendStore::resolveStoragePathReal (   $storagePath)
finalprotected

Like resolveStoragePath() except null values are returned if the container is sharded and the shard could not be determined or if the path ends with '/'. The later case is illegal for FS backends and can confuse listings for object store backends.

This function is used when resolving paths that must be valid locations for files. Directory and listing functions should generally just use resolveStoragePath() instead.

See Also
FileBackendStore::resolveStoragePath()
Parameters
string$storagePath
Returns
array (container, path) or (null, null) if invalid
FileBackendStore::sanitizeOpHeaders ( array  $op)
protected

Normalize and filter HTTP headers from a file operation

This normalizes and strips long HTTP headers from a file operation. Most headers are just numbers, but some are allowed to be long. This function is useful for cleaning up headers and avoiding backend specific errors, especially in the middle of batch file operations.

Parameters
array$opSame format as doOperation()
Returns
array
FileBackendStore::setConcurrencyFlags ( array  $opts)
finalprotected

Set the 'concurrency' option from a list of operation options

Parameters
array$optsMap of operation options
Returns
array
FileBackendStore::setContainerCache (   $container,
array  $val 
)
finalprotected

Set the cached info for a container

Parameters
string$containerResolved container name
array$valInformation to cache
FileBackendStore::setFileCache (   $path,
array  $val 
)
finalprotected

Set the cached stat info for a file path. Negatives (404s) are not cached. By not caching negatives, we can skip cache salting for the case when a file is created at a path were there was none before.

Parameters
string$pathStorage path
array$valStat information to cache
FileBackendStore::storeInternal ( array  $params)
final

Store a file into the backend from a file on disk. This will overwrite any file that exists at the destination. Do not call this function from places outside FileBackend and FileOp.

$params include:

  • src : source path on disk
  • dst : destination storage path
  • headers : HTTP header name/value map
  • async : Status will be returned immediately if supported. If the status is OK, then its value field will be set to a FileBackendStoreOpHandle object.
  • dstExists : Whether a file exists at the destination (optimization). Callers can use "false" if no existing file is being changed.
Parameters
array$params
Returns
Status

The documentation for this class was generated from the following file: