Storage

class olympus.utils.storage.BaseStorage(*args, **kwargs)

Bases: object

Methods

safe_load(name, device)

Handles a few common exceptions for you and returns None if a file is not found

copyfile
exits
garbage_collect
load
open
read
remove
rename
save
save_meta
set_base
show_memory_usage
write

Kio = 1024

Mio = 1048576

copyfile(old, new)

exits(*args, **kwargs)

garbage_collect(*args, **kwargs)

load(*args, **kwargs)

open(*args, **kwargs)

read(*args, **kwargs)

remove(file)

rename(old, new)

safe_load(name, device)

Handles a few common exceptions for you and returns None if a file is not found

save(*args, **kwargs)

save_meta(uid, meta)

set_base(*args, **kwargs)

show_memory_usage()

write(*args, **kwargs)

class olympus.utils.storage.FileMetaStorage(folder)

Bases: object

Provide reverse mapping from uid back to parameters

Methods

load
loc
save

load(folder, uid=None)

loc(folder)

save(folder, uid, meta)

class olympus.utils.storage.FileStateStorage(folder='/tmp')

Bases: olympus.utils.storage.BaseStorage

Methods

load(filename[, device])	Parameters:
safe_load(name, device)	Handles a few common exceptions for you and returns None if a file is not found

copyfile
exits
garbage_collect
load_meta
open
read
remove
rename
save
save_meta
set_base
show_memory_usage
write

copyfile(old, new)

exits(filename)

garbage_collect(gc_time)

load(filename, device=None)

Parameters:

filename: str

file to load the state from

device: torch.device

it indicates the location where all tensors should be loaded.

load_meta()

open(filename, mode)

read(filename)

remove(file)

rename(old, new)

save(filename, state)

save_meta(uid, meta)

set_base(folder)

show_memory_usage()

write(filename, data)

class olympus.utils.storage.InMemoryMetaStorage

Bases: object

Provide reverse mapping from uid back to parameters

Methods

load
save

load(folder, uid=None)

save(folder, uid, meta)

class olympus.utils.storage.InMemoryStorage(format='dict')

Bases: olympus.utils.storage.BaseStorage

Save states in memory

Parameters:

format: str

Which format is used to save the state, default to dict (i.e native python state dict) It can also be set to bytes to have a format that is writable directly to disc

Methods

load(filename[, device])	Parameters:
safe_load(name, device)	Handles a few common exceptions for you and returns None if a file is not found

copyfile
exits
garbage_collect
insert_cache
load_meta
open
pop_from_cache
read
remove
rename
save
save_meta
set_base
show_memory_usage
write

exits(filename)

garbage_collect(gc_time)

insert_cache(key, buffer)

load(filename, device=None)

Parameters:

filename: str

file to load the state from

device: torch.device

it indicates the location where all tensors should be loaded.

load_meta()

pop_from_cache(key) → bytes

read(filename)

remove(file)

rename(old, new)

save(filename, state)

save_meta(uid, meta)

show_memory_usage()

write(filename, data)

olympus.utils.storage.NoStorage(*args, **kwargs)

olympus.utils.storage.StateStorage

alias of olympus.utils.storage.FileStateStorage

olympus.utils.storage.safe_write(filename, buffer)