BlueSpice MediaWiki REL1_27
 All Classes Namespaces Files Functions Variables Groups Pages
SwiftFileBackend Class Reference

Class for an OpenStack Swift (or Ceph RGW) based file backend. More...

Inheritance diagram for SwiftFileBackend:
Collaboration diagram for SwiftFileBackend:

Public Member Functions

 __construct (array $config)
 
 getFeatures ()
 
 isPathUsableInternal ($storagePath)
 
 getDirectoryListInternal ($fullCont, $dir, array $params)
 
 getFileListInternal ($fullCont, $dir, array $params)
 
 getDirListPageInternal ($fullCont, $dir, &$after, $limit, array $params)
 
 getFileListPageInternal ($fullCont, $dir, &$after, $limit, array $params)
 
 loadListingStatInternal ($path, array $val)
 
 getFileHttpUrl (array $params)
 
 onError ($status, $func, array $params, $err= '', $code=0, $desc= '')
 
- Public Member Functions inherited from FileBackendStore
 __construct (array $config)
 
 maxFileSizeInternal ()
 
 isPathUsableInternal ($storagePath)
 
 createInternal (array $params)
 
 storeInternal (array $params)
 
 copyInternal (array $params)
 
 deleteInternal (array $params)
 
 moveInternal (array $params)
 
 describeInternal (array $params)
 
 nullInternal (array $params)
 
 concatenate (array $params)
 
 fileExists (array $params)
 
 getFileTimestamp (array $params)
 
 getFileSize (array $params)
 
 getFileStat (array $params)
 
 getFileContentsMulti (array $params)
 
 getFileXAttributes (array $params)
 
 getFileSha1Base36 (array $params)
 
 getFileProps (array $params)
 
 getLocalReferenceMulti (array $params)
 
 getLocalCopyMulti (array $params)
 
 getFileHttpUrl (array $params)
 
 streamFile (array $params)
 
 directoryExists (array $params)
 
 getDirectoryList (array $params)
 
 getDirectoryListInternal ($container, $dir, array $params)
 
 getFileList (array $params)
 
 getFileListInternal ($container, $dir, array $params)
 
 getOperationsInternal (array $ops)
 
 getPathsToLockForOpsInternal (array $performOps)
 
 getScopedLocksForOps (array $ops, Status $status)
 
 executeOpHandlesInternal (array $fileOpHandles)
 
 preloadCache (array $paths)
 
 clearCache (array $paths=null)
 
 preloadFileStat (array $params)
 
 isSingleShardPathInternal ($storagePath)
 
- Public Member Functions inherited from FileBackend
 __construct (array $config)
 
 getName ()
 
 getWikiId ()
 
 isReadOnly ()
 
 getReadOnlyReason ()
 
 getFeatures ()
 
 hasFeatures ($bitfield)
 
 doOperations (array $ops, array $opts=[])
 
 doOperation (array $op, array $opts=[])
 
 create (array $params, array $opts=[])
 
 store (array $params, array $opts=[])
 
 copy (array $params, array $opts=[])
 
 move (array $params, array $opts=[])
 
 delete (array $params, array $opts=[])
 
 describe (array $params, array $opts=[])
 
 doQuickOperations (array $ops, array $opts=[])
 
 doQuickOperation (array $op)
 
 quickCreate (array $params)
 
 quickStore (array $params)
 
 quickCopy (array $params)
 
 quickMove (array $params)
 
 quickDelete (array $params)
 
 quickDescribe (array $params)
 
 concatenate (array $params)
 
 prepare (array $params)
 
 secure (array $params)
 
 publish (array $params)
 
 clean (array $params)
 
 fileExists (array $params)
 
 getFileTimestamp (array $params)
 
 getFileContents (array $params)
 
 getFileContentsMulti (array $params)
 
 getFileXAttributes (array $params)
 
 getFileSize (array $params)
 
 getFileStat (array $params)
 
 getFileSha1Base36 (array $params)
 
 getFileProps (array $params)
 
 streamFile (array $params)
 
 getLocalReference (array $params)
 
 getLocalReferenceMulti (array $params)
 
 getLocalCopy (array $params)
 
 getLocalCopyMulti (array $params)
 
 getFileHttpUrl (array $params)
 
 directoryExists (array $params)
 
 getDirectoryList (array $params)
 
 getTopDirectoryList (array $params)
 
 getFileList (array $params)
 
 getTopFileList (array $params)
 
 preloadCache (array $paths)
 
 clearCache (array $paths=null)
 
 preloadFileStat (array $params)
 
 lockFiles (array $paths, $type, $timeout=0)
 
 unlockFiles (array $paths, $type)
 
 getScopedFileLocks (array $paths, $type, Status $status, $timeout=0)
 
 getScopedLocksForOps (array $ops, Status $status)
 
 getRootStoragePath ()
 
 getContainerStoragePath ($container)
 
 getJournal ()
 

Protected Member Functions

 resolveContainerPath ($container, $relStoragePath)
 
 sanitizeHdrs (array $params)
 
 getCustomHeaders (array $rawHeaders)
 
 getMetadataHeaders (array $rawHeaders)
 
 getMetadata (array $rawHeaders)
 
 doCreateInternal (array $params)
 
 doStoreInternal (array $params)
 
 doCopyInternal (array $params)
 
 doMoveInternal (array $params)
 
 doDeleteInternal (array $params)
 
 doDescribeInternal (array $params)
 
 doPrepareInternal ($fullCont, $dir, array $params)
 
 doSecureInternal ($fullCont, $dir, array $params)
 
 doPublishInternal ($fullCont, $dir, array $params)
 
 doCleanInternal ($fullCont, $dir, array $params)
 
 doGetFileStat (array $params)
 
 convertSwiftDate ($ts, $format=TS_MW)
 
 addMissingMetadata (array $objHdrs, $path)
 
 doGetFileContentsMulti (array $params)
 
 doDirectoryExists ($fullCont, $dir, array $params)
 
 doGetFileXAttributes (array $params)
 
 doGetFileSha1base36 (array $params)
 
 doStreamFile (array $params)
 
 doGetLocalCopyMulti (array $params)
 
 directoriesAreVirtual ()
 
 headersFromParams (array $params)
 
 doExecuteOpHandlesInternal (array $fileOpHandles)
 
 setContainerAccess ($container, array $readGrps, array $writeGrps)
 
 getContainerStat ($container, $bypassCache=false)
 
 createContainer ($container, array $params)
 
 deleteContainer ($container, array $params)
 
 doPrimeContainerCache (array $containerInfo)
 
 doGetFileStatMulti (array $params)
 
 getStatFromHeaders (array $rhdrs)
 
 getAuthentication ()
 
 storageUrl (array $creds, $container=null, $object=null)
 
 authTokenHeaders (array $creds)
 
- Protected Member Functions inherited from FileBackendStore
 doCreateInternal (array $params)
 
 doStoreInternal (array $params)
 
 doCopyInternal (array $params)
 
 doDeleteInternal (array $params)
 
 doMoveInternal (array $params)
 
 doDescribeInternal (array $params)
 
 doConcatenate (array $params)
 
 doPrepare (array $params)
 
 doPrepareInternal ($container, $dir, array $params)
 
 doSecure (array $params)
 
 doSecureInternal ($container, $dir, array $params)
 
 doPublish (array $params)
 
 doPublishInternal ($container, $dir, array $params)
 
 doClean (array $params)
 
 doCleanInternal ($container, $dir, array $params)
 
 doGetFileStat (array $params)
 
 doGetFileContentsMulti (array $params)
 
 doGetFileXAttributes (array $params)
 
 doGetFileSha1Base36 (array $params)
 
 doGetLocalReferenceMulti (array $params)
 
 doGetLocalCopyMulti (array $params)
 
 doStreamFile (array $params)
 
 doDirectoryExists ($container, $dir, array $params)
 
 doOperationsInternal (array $ops, array $opts)
 
 doQuickOperationsInternal (array $ops)
 
 doExecuteOpHandlesInternal (array $fileOpHandles)
 
 sanitizeOpHeaders (array $op)
 
 doClearCache (array $paths=null)
 
 doGetFileStatMulti (array $params)
 
 directoriesAreVirtual ()
 
 resolveStoragePath ($storagePath)
 
 resolveStoragePathReal ($storagePath)
 
 getContainerShard ($container, $relPath)
 
 getContainerHashLevels ($container)
 
 getContainerSuffixes ($container)
 
 fullContainerName ($container)
 
 resolveContainerName ($container)
 
 resolveContainerPath ($container, $relStoragePath)
 
 setContainerCache ($container, array $val)
 
 deleteContainerCache ($container)
 
 primeContainerCache (array $items)
 
 doPrimeContainerCache (array $containerInfo)
 
 setFileCache ($path, array $val)
 
 deleteFileCache ($path)
 
 primeFileCache (array $items)
 
 setConcurrencyFlags (array $opts)
 
 getContentType ($storagePath, $content, $fsPath)
 
- Protected Member Functions inherited from FileBackend
 doOperationsInternal (array $ops, array $opts)
 
 doQuickOperationsInternal (array $ops)
 
 doPrepare (array $params)
 
 doSecure (array $params)
 
 doPublish (array $params)
 
 doClean (array $params)
 
 getScopedPHPBehaviorForOps ()
 
 resolveFSFileObjects (array $ops)
 

Protected Attributes

 $http
 
 $authTTL
 
 $swiftAuthUrl
 
 $swiftUser
 
 $swiftKey
 
 $swiftTempUrlKey
 
 $rgwS3AccessKey
 
 $rgwS3SecretKey
 
 $srvCache
 
 $containerStatCache
 
 $authCreds
 
 $authSessionTimestamp = 0
 
 $authErrorTimestamp = null
 
 $isRGW = false
 
- Protected Attributes inherited from FileBackendStore
 $memCache
 
 $cheapCache
 
 $expensiveCache
 
 $shardViaHashLevels = []
 
 $mimeCallback
 
 $maxFileSize = 4294967296
 
- Protected Attributes inherited from FileBackend
 $name
 
 $wikiId
 
 $readOnly
 
 $parallelize
 
 $concurrency
 
 $lockManager
 
 $fileJournal
 

Additional Inherited Members

- Static Public Member Functions inherited from FileBackend
static isStoragePath ($path)
 
static splitStoragePath ($storagePath)
 
static normalizeStoragePath ($storagePath)
 
static parentStoragePath ($storagePath)
 
static extensionFromPath ($path, $case= 'lowercase')
 
static isPathTraversalFree ($path)
 
static makeContentDisposition ($type, $filename= '')
 
- Public Attributes inherited from FileBackendStore
const CACHE_TTL = 10
 
const CACHE_CHEAP_SIZE = 500
 
const CACHE_EXPENSIVE_SIZE = 5
 
- Public Attributes inherited from FileBackend
const ATTR_HEADERS = 1
 
const ATTR_METADATA = 2
 
const ATTR_UNICODE_PATHS = 4
 
- Static Protected Member Functions inherited from FileBackendStore
static isValidShortContainerName ($container)
 
static isValidContainerName ($container)
 
static normalizeXAttributes (array $xattr)
 
- Static Protected Member Functions inherited from FileBackend
static normalizeContainerPath ($path)
 

Detailed Description

Class for an OpenStack Swift (or Ceph RGW) based file backend.

Status messages should avoid mentioning the Swift account name. Likewise, error suppression should be used to avoid path disclosure.

Since
1.19

Constructor & Destructor Documentation

SwiftFileBackend::__construct ( array  $config)
See Also
FileBackendStore::__construct() Additional $config params include:
  • swiftAuthUrl : Swift authentication server URL
  • swiftUser : Swift user used by MediaWiki (account:username)
  • swiftKey : Swift authentication key for the above user
  • swiftAuthTTL : Swift authentication TTL (seconds)
  • swiftTempUrlKey : Swift "X-Account-Meta-Temp-URL-Key" value on the account. Do not set this until it has been set in the backend.
  • shardViaHashLevels : Map of container names to sharding config with:
    • base : base of hash characters, 16 or 36
    • levels : the number of hash levels (and digits)
    • repeat : hash subdirectories are prefixed with all the parent hash directory names (e.g. "a/ab/abc")
  • cacheAuthInfo : Whether to cache authentication tokens in APC, XCache, ect. If those are not available, then the main cache will be used. This is probably insecure in shared hosting environments.
  • rgwS3AccessKey : Rados Gateway S3 "access key" value on the account. Do not set this until it has been set in the backend. This is used for generating expiring pre-authenticated URLs. Only use this when using rgw and to work around http://tracker.newdream.net/issues/3454.
  • rgwS3SecretKey : Rados Gateway S3 "secret key" value on the account. Do not set this until it has been set in the backend. This is used for generating expiring pre-authenticated URLs. Only use this when using rgw and to work around http://tracker.newdream.net/issues/3454.

Member Function Documentation

SwiftFileBackend::addMissingMetadata ( array  $objHdrs,
  $path 
)
protected

Fill in any missing object metadata and save it to Swift

Parameters
array$objHdrsObject response headers
string$pathStorage path to object
Returns
array New headers

PhpUnusedLocalVariableInspection

PhpUnusedLocalVariableInspection

SwiftFileBackend::authTokenHeaders ( array  $creds)
protected
Parameters
array$credsFrom getAuthentication()
Returns
array
SwiftFileBackend::convertSwiftDate (   $ts,
  $format = TS_MW 
)
protected

Convert dates like "Tue, 03 Jan 2012 22:01:04 GMT"/"2013-05-11T07:37:27.678360Z". Dates might also come in like "2013-05-11T07:37:27.678360" from Swift listings, missing the timezone suffix (though Ceph RGW does not appear to have this bug).

Parameters
string$ts
int$formatOutput format (TS_* constant)
Returns
string
Exceptions
FileBackendError
SwiftFileBackend::createContainer (   $container,
array  $params 
)
protected

Create a Swift container

Parameters
string$containerContainer name
array$params
Returns
Status
SwiftFileBackend::deleteContainer (   $container,
array  $params 
)
protected

Delete a Swift container

Parameters
string$containerContainer name
array$params
Returns
Status
SwiftFileBackend::doExecuteOpHandlesInternal ( array  $fileOpHandles)
protected
Parameters
FileBackendStoreOpHandle[]$fileOpHandles
Returns
Status[]
SwiftFileBackend::getAuthentication ( )
protected
Returns
array|null Credential map
SwiftFileBackend::getContainerStat (   $container,
  $bypassCache = false 
)
protected

Get a Swift container stat array, possibly from process cache. Use $reCache if the file count or byte count is needed.

Parameters
string$containerContainer name
bool$bypassCacheBypass all caches and load from Swift
Returns
array|bool|null False on 404, null on failure
SwiftFileBackend::getCustomHeaders ( array  $rawHeaders)
protected
Parameters
array$rawHeaders
Returns
array Custom non-metadata HTTP headers
SwiftFileBackend::getDirectoryListInternal (   $fullCont,
  $dir,
array  $params 
)
See Also
FileBackendStore::getDirectoryListInternal()
Parameters
string$fullCont
string$dir
array$params
Returns
SwiftFileBackendDirList
SwiftFileBackend::getDirListPageInternal (   $fullCont,
  $dir,
$after,
  $limit,
array  $params 
)

Do not call this function outside of SwiftFileBackendFileList

Parameters
string$fullContResolved container name
string$dirResolved storage directory with no trailing slash
string | null$afterResolved container relative path to list items after
int$limitMax number of items to list
array$paramsParameters for getDirectoryList()
Returns
array List of container relative resolved paths of directories directly under $dir
Exceptions
FileBackendError
SwiftFileBackend::getFileListInternal (   $fullCont,
  $dir,
array  $params 
)
See Also
FileBackendStore::getFileListInternal()
Parameters
string$fullCont
string$dir
array$params
Returns
SwiftFileBackendFileList
SwiftFileBackend::getFileListPageInternal (   $fullCont,
  $dir,
$after,
  $limit,
array  $params 
)

Do not call this function outside of SwiftFileBackendFileList

Parameters
string$fullContResolved container name
string$dirResolved storage directory with no trailing slash
string | null$afterResolved container relative path of file to list items after
int$limitMax number of items to list
array$paramsParameters for getDirectoryList()
Returns
array List of resolved container relative paths of files under $dir
Exceptions
FileBackendError
SwiftFileBackend::getMetadata ( array  $rawHeaders)
protected
Parameters
array$rawHeaders
Returns
array Custom metadata headers with prefix removed
SwiftFileBackend::getMetadataHeaders ( array  $rawHeaders)
protected
Parameters
array$rawHeaders
Returns
array Custom metadata headers
SwiftFileBackend::getStatFromHeaders ( array  $rhdrs)
protected
Parameters
array$rhdrs
Returns
array
SwiftFileBackend::headersFromParams ( array  $params)
protected

Get headers to send to Swift when reading a file based on a FileBackend params array, e.g. that of getLocalCopy(). $params is currently only checked for a 'latest' flag.

Parameters
array$params
Returns
array
SwiftFileBackend::loadListingStatInternal (   $path,
array  $val 
)

Do not call this function outside of SwiftFileBackendFileList

Parameters
string$pathStorage path
array$valStat value
SwiftFileBackend::onError (   $status,
  $func,
array  $params,
  $err = '',
  $code = 0,
  $desc = '' 
)

Log an unexpected exception for this backend. This also sets the Status object to have a fatal error.

Parameters
Status | null$status
string$func
array$params
string$errError string
int$codeHTTP status
string$descHTTP status description
SwiftFileBackend::sanitizeHdrs ( array  $params)
protected

Sanitize and filter the custom headers from a $params array. Only allows certain "standard" Content- and X-Content- headers.

Parameters
array$params
Returns
array Sanitized value of 'headers' field in $params
SwiftFileBackend::setContainerAccess (   $container,
array  $readGrps,
array  $writeGrps 
)
protected

Set read/write permissions for a Swift container.

See Also
http://swift.openstack.org/misc.html#acls

In general, we don't allow listings to end-users. It's not useful, isn't well-defined (lists are truncated to 10000 item with no way to page), and is just a performance risk.

Parameters
string$containerResolved Swift container
array$readGrpsList of the possible criteria for a request to have access to read a container. Each item is one of the following formats:
  • account:user : Grants access if the request is by the given user
  • ".r:<regex>" : Grants access if the request is from a referrer host that matches the expression and the request is not for a listing. Setting this to '*' effectively makes a container public. -".rlistings:<regex>" : Grants access if the request is from a referrer host that matches the expression and the request is for a listing.
array$writeGrpsA list of the possible criteria for a request to have access to write to a container. Each item is of the following format:
  • account:user : Grants access if the request is by the given user
Returns
Status
SwiftFileBackend::storageUrl ( array  $creds,
  $container = null,
  $object = null 
)
protected
Parameters
array$credsFrom getAuthentication()
string$container
string$object
Returns
array

The documentation for this class was generated from the following file: