![]() |
XRootD
|
Attaches/creates and detaches/deletes cache-io objects for disk based cache. More...
#include <XrdPfc.hh>
Public Member Functions | |
Cache (XrdSysLogger *logger, XrdOucEnv *env) | |
Constructor. More... | |
void | AddWriteTask (Block *b, bool from_read) |
Add downloaded block in write queue. More... | |
virtual XrdOucCacheIO * | Attach (XrdOucCacheIO *, int Options=0) |
virtual XrdOucCacheIO * | Attach (XrdOucCacheIO *ioP, int opts=0)=0 |
Obtain a new IO object that fronts existing XrdOucCacheIO. More... | |
void | ClearPurgeProtectedSet () |
bool | Config (const char *config_filename, const char *parameters) |
Parse configuration file. More... | |
virtual int | ConsiderCached (const char *url) |
bool | Decide (XrdOucCacheIO *) |
Makes decision if the original XrdOucCacheIO should be cached. More... | |
void | DeRegisterPrefetchFile (File *) |
void | ExecuteCommandUrl (const std::string &command_url) |
void | FileSyncDone (File *, bool high_debug) |
File * | GetFile (const std::string &, IO *, long long off=0, long long filesize=0) |
XrdXrootdGStream * | GetGStream () |
XrdSysError * | GetLog () |
File * | GetNextFileToPrefetch () |
XrdOss * | GetOss () const |
PurgePin * | GetPurgePin () const |
XrdSysTrace * | GetTrace () |
bool | IsFileActiveOrPurgeProtected (const std::string &) const |
virtual int | LocalFilePath (const char *url, char *buff=0, int blen=0, LFP_Reason why=ForAccess, bool forall=false) |
void | Prefetch () |
virtual int | Prepare (const char *url, int oflags, mode_t mode) |
void | ProcessWriteTasks () |
Separate task which writes blocks from ram to disk. More... | |
const Configuration & | RefConfiguration () const |
Reference XrdPfc configuration. More... | |
ResourceMonitor & | RefResMon () |
void | RegisterPrefetchFile (File *) |
void | ReleaseFile (File *, IO *) |
void | ReleaseRAM (char *buf, long long size) |
void | RemoveWriteQEntriesFor (File *f) |
Remove blocks from write queue which belong to given prefetch. This method is used at the time of File destruction. More... | |
char * | RequestRAM (long long size) |
void | ScheduleFileSync (File *f) |
virtual int | Stat (const char *url, struct stat &sbuff) |
virtual int | Unlink (const char *url) |
int | UnlinkFile (const std::string &f_name, bool fail_if_open) |
Remove cinfo and data files from cache. More... | |
long long | WritesSinceLastCall () |
![]() | |
XrdOucCache (const char *ctype) | |
virtual | ~XrdOucCache () |
Destructor. More... | |
virtual int | Rename (const char *oldp, const char *newp) |
virtual int | Rmdir (const char *dirp) |
virtual int | Truncate (const char *path, off_t size) |
virtual int | Xeq (XeqCmd cmd, char *arg, int arglen) |
Static Public Member Functions | |
static const Configuration & | Conf () |
static Cache & | CreateInstance (XrdSysLogger *logger, XrdOucEnv *env) |
Singleton creation. More... | |
static Cache & | GetInstance () |
Singleton access. More... | |
static ResourceMonitor & | ResMon () |
static const Cache & | TheOne () |
static bool | VCheck (XrdVersionInfo &urVersion) |
Version check. More... | |
Static Public Attributes | |
static XrdScheduler * | schedP = nullptr |
![]() | |
static const int | optFIS = 0x0001 |
File is structured (e.g. root file) More... | |
static const int | optNEW = 0x0014 |
File is new -> optRW (o/w read or write) More... | |
static const int | optRW = 0x0004 |
File is read/write (o/w read/only) More... | |
static const int | optWIN = 0x0024 |
File is new -> optRW use write-in cache. More... | |
Additional Inherited Members | |
![]() | |
enum | LFP_Reason { ForAccess =0 , ForInfo , ForPath } |
enum | XeqCmd { xeqNoop = 0 } |
![]() | |
const char | CacheType [8] |
A 1-to-7 character cache type identifier (usually pfc or rmc). More... | |
XrdOucCacheStats | Statistics |
Attaches/creates and detaches/deletes cache-io objects for disk based cache.
Cache::Cache | ( | XrdSysLogger * | logger, |
XrdOucEnv * | env | ||
) |
Constructor.
Definition at line 156 of file XrdPfc.cc.
References XrdSysTrace::What.
Referenced by CreateInstance().
void Cache::AddWriteTask | ( | Block * | b, |
bool | from_read | ||
) |
Add downloaded block in write queue.
Definition at line 219 of file XrdPfc.cc.
References XrdPfc::Block::get_file(), XrdPfc::Block::get_size(), XrdPfc::File::GetLocalPath(), XrdPfc::Block::m_offset, and TRACE.
|
virtual |
Implements XrdOucCache.
Definition at line 177 of file XrdPfc.cc.
References Macaroons::Debug, Decide(), Macaroons::Error, GetInstance(), XrdPfc::IOFile::HasFile(), XrdOucCacheIO::Location(), obfuscateAuth(), XrdOucCacheIO::Path(), RefConfiguration(), TRACE, and TRACE_PC.
virtual XrdOucCacheIO* XrdOucCache::Attach |
Obtain a new IO object that fronts existing XrdOucCacheIO.
void Cache::ClearPurgeProtectedSet | ( | ) |
Definition at line 664 of file XrdPfc.cc.
Referenced by XrdPfc::ResourceMonitor::perform_purge_check(), and XrdPfc::ResourceMonitor::perform_purge_task_cleanup().
|
static |
Definition at line 132 of file XrdPfc.cc.
References RefConfiguration().
Referenced by XrdPfc::FPurgeState::CheckFile(), XrdPfc::ResourceMonitor::heart_beat(), XrdPfc::OldStylePurgeDriver(), XrdPfc::ResourceMonitor::perform_purge_check(), Proto_ResourceMonitorHeartBeat(), and XrdPfc::ResourceMonitor::update_vs_and_file_usage_info().
bool Cache::Config | ( | const char * | config_filename, |
const char * | parameters | ||
) |
Parse configuration file.
config_filename | path to configuration file |
parameters | optional parameters to be passed |
Definition at line 329 of file XrdPfcConfiguration.cc.
References XrdOuca2x::a2ll(), XrdOuca2x::a2sz(), XrdPfc::CSChk_Both, XrdPfc::CSChk_None, XrdSysError::Emsg(), Macaroons::Error, XrdOucEnv::Export(), XrdOucEnv::GetPtr(), XrdPfc::ResourceMonitor::init_before_main(), XrdPfc::Configuration::is_cschk_cache(), XrdPfc::Configuration::is_cschk_net(), XrdPfc::Configuration::is_dir_stat_reporting_on(), XrdOfsConfigPI::Load(), XrdPfc::Configuration::m_accHistorySize, XrdPfc::Configuration::m_allow_xrdpfc_command, XrdPfc::Configuration::m_bufferSize, XrdPfc::Configuration::m_cs_Chk, XrdPfc::Configuration::m_cs_ChkTLS, XrdPfc::Configuration::m_cs_UVKeep, XrdPfc::Configuration::m_data_space, XrdPfc::Configuration::m_dirStatsDirGlobs, XrdPfc::Configuration::m_dirStatsDirs, XrdPfc::Configuration::m_dirStatsMaxDepth, XrdPfc::Configuration::m_dirStatsStoreDepth, XrdPfc::Configuration::m_diskTotalSpace, XrdPfc::Configuration::m_diskUsageHWM, XrdPfc::TmpConfiguration::m_diskUsageHWM, XrdPfc::Configuration::m_diskUsageLWM, XrdPfc::TmpConfiguration::m_diskUsageLWM, XrdPfc::Configuration::m_fileUsageBaseline, XrdPfc::TmpConfiguration::m_fileUsageBaseline, XrdPfc::Configuration::m_fileUsageMax, XrdPfc::TmpConfiguration::m_fileUsageMax, XrdPfc::Configuration::m_fileUsageNominal, XrdPfc::TmpConfiguration::m_fileUsageNominal, XrdPfc::Configuration::m_flushCnt, XrdPfc::TmpConfiguration::m_flushRaw, XrdPfc::Configuration::m_hdfsbsize, XrdPfc::Configuration::m_hdfsmode, XrdPfc::Configuration::m_meta_space, XrdPfc::Configuration::m_onlyIfCachedMinFrac, XrdPfc::Configuration::m_onlyIfCachedMinSize, XrdPfc::Configuration::m_prefetch_max_blocks, XrdPfc::Configuration::m_purgeColdFilesAge, XrdPfc::Configuration::m_purgeInterval, XrdPfc::Configuration::m_RamAbsAvailable, XrdPfc::Configuration::m_RamKeepStdBlocks, XrdPfc::Configuration::m_username, XrdPfc::Configuration::m_wqueue_blocks, XrdPfc::Configuration::m_wqueue_threads, XrdOfsConfigPI::New(), open(), XrdOfsConfigPI::Parse(), XrdOfsConfigPI::Plugin(), XrdOfsConfigPI::Push(), XrdOucEnv::Put(), XrdPfc::Info::s_maxNumAccess, XrdSysError::Say(), XrdOss::StatVS(), XrdOfsConfigPI::theOssLib, XrdOssVSInfo::Total, TRACE, XrdOucUtils::UserName(), XrdSysTrace::What, and XrdOucGetCache().
Referenced by XrdOucGetCache().
|
virtual |
Definition at line 901 of file XrdPfc.cc.
References XrdOssDF::Close(), Macaroons::Debug, XrdPfc::Info::GetFileSize(), XrdPfc::Info::GetNDownloadedBytes(), XrdCl::URL::GetPath(), XrdPfc::Info::IsComplete(), XrdSysCondVar::Lock(), XrdPfc::Configuration::m_onlyIfCachedMinFrac, XrdPfc::Configuration::m_onlyIfCachedMinSize, XrdPfc::Configuration::m_username, XrdOss::newFile(), XrdOssDF::Open(), XrdPfc::Info::Read(), XrdPfc::Info::s_infoExtension, stat(), XrdOss::Stat(), TRACE, XrdSysCondVar::UnLock(), and XrdOssOK.
Referenced by XrdPfcFSctl::FSctl().
|
static |
Singleton creation.
Definition at line 123 of file XrdPfc.cc.
References Cache().
Referenced by XrdOucGetCache().
bool Cache::Decide | ( | XrdOucCacheIO * | io | ) |
Makes decision if the original XrdOucCacheIO should be cached.
& | URL of file |
Definition at line 135 of file XrdPfc.cc.
References XrdPfc::Decision::Decide(), XrdCl::URL::GetPath(), and XrdOucCacheIO::Path().
Referenced by Attach().
void Cache::DeRegisterPrefetchFile | ( | File * | file | ) |
Definition at line 690 of file XrdPfc.cc.
References XrdSysCondVar::Lock(), and XrdSysCondVar::UnLock().
void Cache::ExecuteCommandUrl | ( | const std::string & | command_url | ) |
Definition at line 50 of file XrdPfcCommand.cc.
References XrdOuca2x::a2i(), XrdOuca2x::a2sz(), XrdOssDF::Close(), Create, Macaroons::Debug, ERRNO_AND_ERRSTR, Macaroons::Error, XrdPfc::SplitParser::fill_argv(), XrdPfc::SplitParser::get_reminder_with_delim(), XrdPfc::SplitParser::get_token(), XrdOucArgs::getarg(), XrdOssDF::getFD(), XrdPfc::Info::GetLatestDetachTime(), XrdOucArgs::getopt(), GetOss(), XrdPfc::Configuration::m_bufferSize, XrdPfc::Stats::m_BytesWritten, XrdPfc::Configuration::m_data_space, XrdPfc::Configuration::m_meta_space, XrdPfc::Stats::m_StBlocksAdded, XrdPfc::Configuration::m_username, MAX_ACCESSES, XrdOss::newFile(), ONE_GB, ONE_MB, XrdOssDF::Open(), XrdOucEnv::Put(), XrdPfc::ResourceMonitor::register_file_close(), XrdPfc::ResourceMonitor::register_file_open(), XrdPfc::ResourceMonitor::register_file_update_stats(), XrdPfc::Info::s_infoExtension, XrdSysError::Say(), XrdOucArgs::Set(), XrdPfc::Info::SetAllBitsSynced(), XrdPfc::Info::SetBufferSizeFileSizeAndCreationTime(), stat(), Stat(), TRACE, UnlinkFile(), usage(), XrdPfc::Info::Write(), XrdPfc::Info::WriteIOStatSingle(), XRDOSS_mkpath, and XrdOssOK.
void Cache::FileSyncDone | ( | File * | f, |
bool | high_debug | ||
) |
File * Cache::GetFile | ( | const std::string & | path, |
IO * | io, | ||
long long | off = 0 , |
||
long long | filesize = 0 |
||
) |
Definition at line 387 of file XrdPfc.cc.
References XrdPfc::File::AddIO(), XrdSysCondVar::Broadcast(), Macaroons::Debug, Macaroons::Error, XrdPfc::File::FileOpen(), XrdOucCacheIO::Fstat(), stat(), TRACE, and XrdSysCondVar::Wait().
Referenced by XrdPfc::IOFile::IOFile().
|
inline |
|
static |
Singleton access.
Definition at line 130 of file XrdPfc.cc.
Referenced by XrdPfc::IOFile::IOFile(), XrdPfc::IOFileBlock::IOFileBlock(), Attach(), XrdPfc::IOFile::DetachFinalize(), XrdPfc::File::GetLog(), XrdPfc::File::GetTrace(), XrdPfc::ResourceMonitor::perform_purge_check(), XrdPfc::ResourceMonitor::perform_purge_task_cleanup(), PrefetchThread(), ProcessWriteTaskThread(), Proto_ResourceMonitorHeartBeat(), XrdPfc::File::Sync(), and XrdPfc::File::WriteBlockToDisk().
|
inline |
Definition at line 277 of file XrdPfc.hh.
Referenced by XrdPfc::File::GetLog().
File * Cache::GetNextFileToPrefetch | ( | ) |
Definition at line 712 of file XrdPfc.cc.
References XrdSysCondVar::Lock(), XrdSysCondVar::UnLock(), and XrdSysCondVar::Wait().
Referenced by Prefetch().
|
inline |
Definition at line 263 of file XrdPfc.hh.
Referenced by ExecuteCommandUrl().
|
inline |
Definition at line 267 of file XrdPfc.hh.
Referenced by XrdPfc::ResourceMonitor::perform_purge_check().
|
inline |
Definition at line 278 of file XrdPfc.hh.
Referenced by XrdPfc::File::GetTrace(), and XrdPfc::IO::GetTrace().
bool Cache::IsFileActiveOrPurgeProtected | ( | const std::string & | path | ) | const |
|
virtual |
Get the path to a file that is complete in the local cache. By default, the file must be complete in the cache (i.e. no blocks are missing). This can be overridden. This path can be used to access the file on the local node.
Reimplemented from XrdOucCache.
Definition at line 773 of file XrdPfc.cc.
References XrdOss::Chmod(), XrdOssDF::Close(), Macaroons::Debug, XrdOucCache::ForAccess, XrdOucCache::ForInfo, XrdOucCache::ForPath, XrdPfc::Info::GetFileSize(), XrdCl::URL::GetPath(), XrdPfc::Info::IsComplete(), XrdOss::Lfn2Pfn(), XrdSysCondVar::Lock(), XrdPfc::Configuration::m_username, XrdOss::newFile(), XrdOssDF::Open(), XrdPfc::Info::Read(), XrdPfc::Info::s_infoExtension, stat(), XrdOss::Stat(), TRACE, XrdSysCondVar::UnLock(), XrdPfc::Info::Write(), XrdPfc::Info::WriteIOStatSingle(), and XrdOssOK.
void Cache::Prefetch | ( | ) |
Definition at line 731 of file XrdPfc.cc.
References GetNextFileToPrefetch(), XrdSysMutex::Lock(), XrdPfc::Configuration::m_RamAbsAvailable, XrdPfc::File::Prefetch(), XrdSysMutex::UnLock(), and XrdSysTimer::Wait().
Referenced by PrefetchThread().
|
virtual |
Preapare the cache for a file open request. This method is called prior to actually opening a file. This method is meant to allow defering an open request or implementing the full I/O stack in the cache layer.
Reimplemented from XrdOucCache.
Definition at line 1009 of file XrdPfc.cc.
References XrdCl::URL::GetPath(), XrdPfc::Configuration::m_allow_xrdpfc_command, XrdPfc::Info::s_infoExtension, schedP, XrdScheduler::Schedule(), stat(), XrdOss::Stat(), TRACE, and Warning.
void Cache::ProcessWriteTasks | ( | ) |
Separate task which writes blocks from ram to disk.
Definition at line 271 of file XrdPfc.cc.
References XrdPfc::Block::get_size(), XrdPfc::File::lPath(), XrdPfc::Block::m_file, XrdPfc::Configuration::m_wqueue_blocks, TRACE, and XrdPfc::File::WriteBlockToDisk().
Referenced by ProcessWriteTaskThread().
|
inline |
Reference XrdPfc configuration.
Definition at line 199 of file XrdPfc.hh.
Referenced by XrdPfc::IOFileBlock::IOFileBlock(), Attach(), Conf(), XrdPfc::File::WriteBlockToDisk(), and XrdOucGetCache().
|
inline |
void Cache::RegisterPrefetchFile | ( | File * | file | ) |
Definition at line 674 of file XrdPfc.cc.
References XrdSysCondVar::Lock(), XrdSysCondVar::Signal(), and XrdSysCondVar::UnLock().
Definition at line 468 of file XrdPfc.cc.
References Macaroons::Debug, XrdPfc::File::GetLocalPath(), XrdPfc::File::RemoveIO(), and TRACE.
Referenced by XrdPfc::IOFile::DetachFinalize(), and XrdPfc::IOFileBlock::DetachFinalize().
void Cache::ReleaseRAM | ( | char * | buf, |
long long | size | ||
) |
Definition at line 369 of file XrdPfc.cc.
References XrdPfc::Configuration::m_bufferSize, and XrdPfc::Configuration::m_RamKeepStdBlocks.
void Cache::RemoveWriteQEntriesFor | ( | File * | f | ) |
Remove blocks from write queue which belong to given prefetch. This method is used at the time of File destruction.
Definition at line 238 of file XrdPfc.cc.
References XrdPfc::File::BlocksRemovedFromWriteQ(), XrdPfc::File::lPath(), and TRACE.
Referenced by UnlinkFile().
char * Cache::RequestRAM | ( | long long | size | ) |
Definition at line 329 of file XrdPfc.cc.
References XrdSysMutex::Lock(), XrdPfc::Configuration::m_bufferSize, XrdPfc::Configuration::m_RamAbsAvailable, and XrdSysMutex::UnLock().
|
static |
Definition at line 133 of file XrdPfc.cc.
References RefResMon().
Referenced by XrdPfc::File::~File(), XrdPfc::ResourceMonitor::perform_purge_check(), ResourceMonitorThread(), and XrdPfc::UnlinkPurgeStateFilesInMap().
|
inline |
|
virtual |
Reimplemented from XrdOucCache.
Definition at line 1061 of file XrdPfc.cc.
References XrdOssDF::Close(), XrdPfc::Info::GetFileSize(), XrdCl::URL::GetPath(), XrdPfc::Configuration::m_username, XrdOss::newFile(), XrdOssDF::Open(), XrdPfc::Info::Read(), XrdPfc::Info::s_infoExtension, XrdOss::Stat(), and XrdOssOK.
Referenced by ExecuteCommandUrl().
|
static |
Definition at line 131 of file XrdPfc.cc.
Referenced by XrdPfc::OldStylePurgeDriver(), and XrdPfc::UnlinkPurgeStateFilesInMap().
|
virtual |
Reimplemented from XrdOucCache.
Definition at line 1110 of file XrdPfc.cc.
References XrdCl::URL::GetPath(), and UnlinkFile().
int Cache::UnlinkFile | ( | const std::string & | f_name, |
bool | fail_if_open | ||
) |
Remove cinfo and data files from cache.
Definition at line 1120 of file XrdPfc.cc.
References Macaroons::Debug, XrdPfc::File::initiate_emergency_shutdown(), XrdPfc::ResourceMonitor::register_file_purge(), RemoveWriteQEntriesFor(), XrdPfc::Info::s_infoExtension, stat(), XrdOss::Stat(), TRACE, XrdOss::Unlink(), and XrdOssOK.
Referenced by ExecuteCommandUrl(), XrdPfcFSctl::FSctl(), XrdPfc::File::Sync(), and Unlink().
|
inlinestatic |
long long Cache::WritesSinceLastCall | ( | ) |
Definition at line 318 of file XrdPfc.cc.
Referenced by XrdPfc::ResourceMonitor::perform_purge_check().
|
static |
Definition at line 285 of file XrdPfc.hh.
Referenced by XrdPfc::IO::Detach(), XrdPfc::ResourceMonitor::perform_purge_check(), Prepare(), and XrdOucGetCache().