From 8437d48465417c722dc156f44109ce937dfc4944 Mon Sep 17 00:00:00 2001 From: "Documenter.jl" Date: Wed, 6 Nov 2024 10:35:16 +0000 Subject: [PATCH] build based on 70f10de --- dev/.documenter-siteinfo.json | 2 +- dev/advanced/index.html | 2 +- dev/compression/index.html | 2 +- dev/customserialization/index.html | 2 +- dev/hdf5compat/index.html | 2 +- dev/index.html | 6 +++--- dev/internals/index.html | 26 +++++++++++++------------- dev/legacy/index.html | 2 +- dev/objects.inv | Bin 2119 -> 2119 bytes 9 files changed, 22 insertions(+), 22 deletions(-) diff --git a/dev/.documenter-siteinfo.json b/dev/.documenter-siteinfo.json index 1014030c..5303479d 100644 --- a/dev/.documenter-siteinfo.json +++ b/dev/.documenter-siteinfo.json @@ -1 +1 @@ -{"documenter":{"julia_version":"1.11.0","generation_timestamp":"2024-10-24T10:39:24","documenter_version":"1.7.0"}} \ No newline at end of file +{"documenter":{"julia_version":"1.11.0","generation_timestamp":"2024-11-06T10:35:11","documenter_version":"1.7.0"}} \ No newline at end of file diff --git a/dev/advanced/index.html b/dev/advanced/index.html index fb228aa7..963b1dc8 100644 --- a/dev/advanced/index.html +++ b/dev/advanced/index.html @@ -47,4 +47,4 @@ # to an `Upgrade` instance with the new struct. load("test.jld2", "data"; typemap=Dict("Main.OldStructVersion" => JLD2.Upgrade(UpdatedStruct)))

Groups - Appending to files

Group objects can be constructed with two optional keyword arguments:

g = Group(file;
           est_num_entries=4
-          est_link_name_len=8)

These determine how much (additional) empty space should be allocated for the group description. (list of entries) This can be useful for performance when one expects to append many additional datasets after first writing the file.

JLD2DebugTools

There is an experimental repository JLD2DebugTools.jl that may help with debugging files.

Fallback Behaviour

By default JLD2 will attempt to open files using the MmapIO backend. If that fails, it retries using IOStream.

+ est_link_name_len=8)

These determine how much (additional) empty space should be allocated for the group description. (list of entries) This can be useful for performance when one expects to append many additional datasets after first writing the file.

JLD2DebugTools

There is an experimental repository JLD2DebugTools.jl that may help with debugging files.

Fallback Behaviour

By default JLD2 will attempt to open files using the MmapIO backend. If that fails, it retries using IOStream.

diff --git a/dev/compression/index.html b/dev/compression/index.html index e091e366..96b8fa73 100644 --- a/dev/compression/index.html +++ b/dev/compression/index.html @@ -11,4 +11,4 @@ # Don't compress this write(f, "large_array", rand(10000)) -end +end diff --git a/dev/customserialization/index.html b/dev/customserialization/index.html index 7292ad6f..8a6ffb58 100644 --- a/dev/customserialization/index.html +++ b/dev/customserialization/index.html @@ -20,4 +20,4 @@ arr = [B(rand()) for i=1:10] -@save "test.jld2" arr

In this example JLD2 converts the array of B structs to a plain Vector{Float64} prior to storing to disk.

+@save "test.jld2" arr

In this example JLD2 converts the array of B structs to a plain Vector{Float64} prior to storing to disk.

diff --git a/dev/hdf5compat/index.html b/dev/hdf5compat/index.html index c8bef839..980d67af 100644 --- a/dev/hdf5compat/index.html +++ b/dev/hdf5compat/index.html @@ -56,4 +56,4 @@ } } }

We can see that the file contains two things at top-level. There is a dataset "a" (that is what we wanted to store) and there is a group _types which is where all the necessary type information is stored.

You can see that JLD2 committed two compound datatypes. The first one is Core.Datatype which at first seems rather unintuitive. It is needed to tell HDF5 what a serialized julia datatype looks like (a name and a list of parameters).

Below that is the definition of MyCustomStruct with two fields H5T_STD_I64LE "x" and H5T_IEEE_F64LE "y" defining the integer field x and the float field y.

A note on pointers

In the julia programming language pointers Ptr are not needed very often. However, when binary dependencies come into play and memory is passed back and forth, pointers do become relevant. Pointers are addresses to locations in memory and thus lose their meaning after a program has terminated.

In principle, there is little point in storing a pointer to a file but in order to allow for a more seamless experience JLD2 will, similar to Base.Serialization silently accept pointers. This is useful when storing large structures such as a DifferentialEquations.jl solution object that might contain a pointer somewhere. Upon deserialization any pointer fields are instantiated as null pointers.

This is done with just three lines of code utilizing the custom serialization logic and it is shown here as it serves as a good example for usage of that feature.

writeas(::Type{<:Ptr}) = Nothing
-rconvert(::Type{Ptr{T}}, ::Nothing) where {T} = Ptr{T}()

Usually one would also have to define a method for wconvert. However, in this case JLD2 figures out that no explicit conversion is needed to construct nothing.

+rconvert(::Type{Ptr{T}}, ::Nothing) where {T} = Ptr{T}()

Usually one would also have to define a method for wconvert. However, in this case JLD2 figures out that no explicit conversion is needed to construct nothing.

diff --git a/dev/index.html b/dev/index.html index d9eb8ee6..ffc8eee1 100644 --- a/dev/index.html +++ b/dev/index.html @@ -13,9 +13,9 @@ # and if you want to confuse your future self and everyone else, do jldsave("example.jld2"; z=x, x=y, y=z)

Compression and non-default IO types may be set via positional arguments.

save_object and load_object functions

If only a single object needs to stored and loaded from a file, one can use save_object and load_object functions.

JLD2.save_object — Function
save_object(filename, x)

Stores an object x in a new JLD2 file at filename. If a file exists at this path, it will be overwritten.

Since the JLD2 format requires that all objects have a name, the object will be stored as single_stored_object. If you want to store more than one object, use @save macro, jldopen or the FileIO API.

Example

To save the string hello to the JLD2 file example.jld2:

hello = "world"
-save_object("example.jld2", hello)
source
JLD2.load_object — Function
load_object(filename)

Returns the only available object from the JLD2 file filename (The stored object name is inconsequential). If the file contains more than one or no objects, the function throws an ArgumentError.

For loading more than one object, use @load macro, jldopen or the FileIO API.

Example

To load the only object from the JLD2 file example.jld2:

hello = "world"
+save_object("example.jld2", hello)
source
JLD2.load_object — Function
load_object(filename)

Returns the only available object from the JLD2 file filename (The stored object name is inconsequential). If the file contains more than one or no objects, the function throws an ArgumentError.

For loading more than one object, use @load macro, jldopen or the FileIO API.

Example

To load the only object from the JLD2 file example.jld2:

hello = "world"
 save_object("example.jld2", hello)
-hello_loaded = load_object("example.jld2")
source

save and load functions

The save and load functions, provided by FileIO, provide a mechanism to read and write data from a JLD2 file. To use these functions, you may either write using FileIO or using JLD2. FileIO will determine the correct package automatically.

The save function accepts an AbstractDict yielding the key/value pairs, where the key is a string representing the name of the dataset and the value represents its contents:

using FileIO
+hello_loaded = load_object("example.jld2")
source

save and load functions

The save and load functions, provided by FileIO, provide a mechanism to read and write data from a JLD2 file. To use these functions, you may either write using FileIO or using JLD2. FileIO will determine the correct package automatically.

The save function accepts an AbstractDict yielding the key/value pairs, where the key is a string representing the name of the dataset and the value represents its contents:

using FileIO
 save("example.jld2", Dict("hello" => "world", "foo" => :bar))

The save function can also accept the dataset names and contents as arguments:

save("example.jld2", "hello", "world", "foo", :bar)

When using the save function, the file extension must be .jld2, since the extension .jld currently belongs to the previous JLD package.

If called with a filename argument only, the load function loads all datasets from the given file into a Dict:

load("example.jld2") # -> Dict{String,Any}("hello" => "world", "foo" => :bar)

If called with a single dataset name, load returns the contents of that dataset from the file:

load("example.jld2", "hello") # -> "world"

If called with multiple dataset names, load returns the contents of the given datasets as a tuple:

load("example.jld2", "hello", "foo") # -> ("world", :bar)

File interface

It is also possible to interact with JLD2 files using a file-like interface. The jldopen function accepts a file name and an argument specifying how the file should be opened:

using JLD2
 
 f = jldopen("example.jld2", "r")  # open read-only (default)
@@ -65,4 +65,4 @@
 julia> f["a"] # a new copy is loaded from the file
 2-element Vector{Float64}:
  0.0
- 0.0

Cross-compatibility

JLD2 tries to write files in a way that allows you to load them on different operating systems and in particular both on 32bit and 64bit systems. However, many julia structs may be inherently different on different architectures making this task impossible. In particular, moving data from a 64bit system to a 32bit system is only guaranteed to work for basic datatypes.

Security

Beware of opening JLD2 files from untrusted sources. A malicious file may execute code on your computer. See e.g. this project's issue #117. To check a file, you can use JLD2DebugTools.jl to view what kinds of objects are stored.

+ 0.0

Cross-compatibility

JLD2 tries to write files in a way that allows you to load them on different operating systems and in particular both on 32bit and 64bit systems. However, many julia structs may be inherently different on different architectures making this task impossible. In particular, moving data from a 64bit system to a 32bit system is only guaranteed to work for basic datatypes.

Security

Beware of opening JLD2 files from untrusted sources. A malicious file may execute code on your computer. See e.g. this project's issue #117. To check a file, you can use JLD2DebugTools.jl to view what kinds of objects are stored.

diff --git a/dev/internals/index.html b/dev/internals/index.html index 2f1f2d81..123a4247 100644 --- a/dev/internals/index.html +++ b/dev/internals/index.html @@ -1,24 +1,24 @@ -Internals & Design · Julia Data Format
GitHub

Internals & Design

File Interface

The JLDFile object mimics the API of Base.Dict as much as it can. In particular, keys, length, haskey, isempty, get, get! should work as expected.

JLD2.CommittedDatatype — Type
CommittedDatatype <: H5Datatype

Reference to a shared datatype message (stored elsewhere in a file). These are stored in the _types group and indexed.

source
JLD2.CustomSerialization — Type
CustomSerialization{T,S}

On-disk representation for data that is written as if it were of Julia type T, but is read as type S.

source
JLD2.Group — Type
Group{T}
-Group(file::T)

JLD2 group object.

Advanced Usage

Takes two optional keyword arguments:

  • est_num_entries::Int = 4
  • est_link_name_len::Int = 8

These determine how much (additional) empty space should be allocated for the group description. (list of entries) This can be useful for performance when one expects to append many additional datasets after first writing the file.

source
JLD2.Group — Method
Group(f::JLDFile, name::AbstractString)

Construct an empty group named name at the top level of JLDFile f.

source
JLD2.Group — Method
Group(g::Group, name::AbstractString)

Construct a group named name as a child of group g.

source
JLD2.HeaderMessageIterator — Type
mutable struct HeaderMessageIterator{IO}
-    HeaderMessageIterator(f::JLDFile, offset::RelOffset)

Implements an iterator over header messages.

source
JLD2.Hmessage — Type
Hmessage{IO}

Representation of a Header Message in memory. Provides getproperty access to the fields of the message. Can also be used to construct and write custom messages.

source
JLD2.IndirectPointer — Type
IndirectPointer

When writing data, we may need to enlarge the memory mapping, which would invalidate any memory addresses arising from the old mmap pointer. IndirectPointer holds an offset relative to the MemoryBackedIO. It defers computing a memory address until converted to a Ptr{T}, so the memory mapping can be enlarged and addresses will remain valid.

source
JLD2.InlineUnionEl — Type
InlineUnionEl{T1,T2}(mask::UInt8, t1::T1, t2::T2)

Custom serialization struct for two member isbits union fields e.g. in other structs or arrays. To indicate that t1 is relevant the mask takes the value UInt8(0) and for t2 the mask takes the value UInt8(255).

source
JLD2.JLDWriteSession — Type
JLDWriteSession{T}

A JLDWriteSession keeps track of references to serialized objects. If T is a Dict, h5offset maps an object ID (returned by calling objectid) to th RelOffset of the written dataset. If it is Union{}, then references are not tracked, and objects referenced multiple times are written multiple times.

source
JLD2.MemoryBackedIO — Type
MemoryBackedIO <: IO

Abstract type for IO objects that are backed by memory in such a way that one can use pointer based unsafe_load and unsafe_store! operations after ensuring that there is enough memory allocated.

It needs to provide:

  • getproperty(io, :curptr) to get the current pointer
  • ensureroom(io, nb) to ensure that there are at least nb bytes available
  • position(io) to get the current (zero-based) position
  • seek(io, pos) to set the current position (zero-based)
source
JLD2.Message — Type
Message{IO}

Representation of a Message in memory. Provides getproperty access

source
JLD2.RelOffset — Type
RelOffset

Represents an HDF5 relative offset. This differs from a file offset (used elsewhere) in that it is relative to the superblock base address. fileoffset and h5offset convert between RelOffsets and file offsets.

source
JLD2.SharedDatatype — Type
SharedDatatype <: H5Datatype

Reference to a shared datatype message (stored elsewhere in a file).

source
JLD2.Upgrade — Type
Upgrade(T)

Specify an upgrade path for serialized structs using the typemap keyword argument and rconvert.

source
JLD2.attributes — Method
attributes(dset::Dataset; plain::Bool=false)

Return the attributes of a dataset as an OrderedDict. If plain is set to true then the values are returned as stored in the dataset object.

source
JLD2.behead — Method
behead(T)

Given a UnionAll type, recursively eliminates the where clauses

source
JLD2.bufferpos — Method
bufferpos(io::Union{BufferedReader, BufferedWriter})

Get the current position in the buffer.

source
JLD2.construct_array — Method
construct_array(io::IO, eltype, ndims::Int)

Construct array by reading ndims dimensions from io. Assumes io has already been seeked to the correct position.

source
JLD2.constructrr — Function
constructrr(f::JLDFile, T::DataType, dt::CompoundType, attrs::Vector{ReadAttribute},
-            hard_failure::Bool=false)

Constructs a ReadRepresentation for a given type. This is the generic method for all types not specially handled below.

If hard_failure is true, then throw a TypeMappingException instead of attempting reconstruction. This helps in cases where we can't know if reconstructed parametric types will have a matching memory layout without first inspecting the memory layout.

source
JLD2.create_dataset — Method
create_dataset(parent, path, datatype, dataspace; kwargs...)

Arguments: - parent::Union{JLDfile, Group}: Containing group of new dataset - path: Path to new dataset relative to parent. If path is nothing, the dataset is unnamed. - datatype: Datatype of new dataset (element type in case of arrays) - dataspace: Dimensions or Dataspace of new dataset

Keyword arguments: - layout: DataLayout of new dataset - filters: FilterPipeline for describing the compression pipeline

source
JLD2.fileoffset — Method
fileoffset(f::JLDFile, x::RelOffset)

Converts an offset x relative to the superblock of file f to an absolute offset.

source
JLD2.flag2uint — Method
flag2uint(flag::UInt8)

I Map the lowest to bits of flag to a UInt type, mapping 0 to UInt8, 1 to UInt16, 2 to UInt32, and 3 to UInt64.

source
JLD2.get_dataset — Method
get_dataset(parent::Union{JLDFile, Group}, name::String)

Get a stored dataset from a file by name or path as a Dataset object. This may be useful for inspecting the metadata incl. types of a dataset.

source
JLD2.h5offset — Method
h5offset(f::JLDFile, x::Integer)

Converts an absolute file offset x to an offset relative to the superblock of file f.

source
JLD2.ismmappable — Method
ismmappable(dset::Dataset)

Check if a dataset can be memory-mapped. This can be useful for large arrays and for editing written arrays.

An Array dataset may be mmapped if: - JLD2.samelayout(T) == true: The element type is isbits and has a size that is a multiple of 8 bytes. - Uncompressed: Compressed arrays cannot be memory-mapped - Uses a contiguous layout: This is true for all array datasets written by JLD2 with version ≥ v0.4.52 - Offset in file is a multiple of 8 bytes: This is a requirement for Mmap. - Windows: The file must be opened in read-only mode. This is a limitation of Mmap on Windows.

source
JLD2.isset — Method
isset(flag, bit)

Return true if the bit-th bit of flag is set. (starting from 0)

source
JLD2.jld_finalizer — Method
jld_finalizer(f::JLDFile)

When a JLDFile is finalized, it is possible that the MmapIO has been munmapped, since Julia does not guarantee finalizer order. This means that the underlying file may be closed before we get a chance to write to it.

source
JLD2.jldopen — Function
jldopen(file, mode::AbstractString; iotype=MmapIO, compress=false, typemap=Dict())

Opens a JLD2 file at path file. Alternatively file may be a suitable IO object.

Options for mode:

  • "r": Open for reading only, failing if no file exists
  • "r+": Open for reading and writing, failing if no file exists
  • "w"/"w+": Open for reading and writing, overwriting the file if it already exists
  • "a"/"a+": Open for reading and writing, creating a new file if none exists, but preserving the existing file if one is present
source
JLD2.jldsave — Function
jldsave(filename; kwargs...)
+Internals & Design · Julia Data Format
GitHub
Internals & Design
File Interface
The JLDFile object mimics the API of Base.Dict as much as it can. In particular, keys, length, haskey, isempty, get, get! should work as expected.
JLD2.CommittedDatatype — Type
CommittedDatatype <: H5Datatype
Reference to a shared datatype message (stored elsewhere in a file). These are stored in the _types group and indexed.
source
JLD2.CustomSerialization — Type
CustomSerialization{T,S}
On-disk representation for data that is written as if it were of Julia type T, but is read as type S.
source
JLD2.Group — Type
Group{T}
+Group(file::T)
JLD2 group object.
Advanced Usage
Takes two optional keyword arguments:
  • est_num_entries::Int = 4
  • est_link_name_len::Int = 8
These determine how much (additional) empty space should be allocated for the group description. (list of entries) This can be useful for performance when one expects to append many additional datasets after first writing the file.
source
JLD2.Group — Method
Group(f::JLDFile, name::AbstractString)
Construct an empty group named name at the top level of JLDFile f.
source
JLD2.Group — Method
Group(g::Group, name::AbstractString)
Construct a group named name as a child of group g.
source
JLD2.HeaderMessageIterator — Type
mutable struct HeaderMessageIterator{IO}
+    HeaderMessageIterator(f::JLDFile, offset::RelOffset)
Implements an iterator over header messages.
source
JLD2.Hmessage — Type
Hmessage{IO}
Representation of a Header Message in memory. Provides getproperty access to the fields of the message. Can also be used to construct and write custom messages.
source
JLD2.IndirectPointer — Type
IndirectPointer
When writing data, we may need to enlarge the memory mapping, which would invalidate any memory addresses arising from the old mmap pointer. IndirectPointer holds an offset relative to the  MemoryBackedIO. It defers computing a memory address until converted to a Ptr{T},  so the memory mapping can be enlarged and addresses will remain valid.
source
JLD2.InlineUnionEl — Type
InlineUnionEl{T1,T2}(mask::UInt8, t1::T1, t2::T2)
Custom serialization struct for two member isbits union fields e.g. in other structs or arrays. To indicate that t1 is relevant the mask takes the value UInt8(0) and for t2 the mask takes the value UInt8(255).
source
JLD2.JLDWriteSession — Type
JLDWriteSession{T}
A JLDWriteSession keeps track of references to serialized objects. If T is a Dict, h5offset maps an object ID (returned by calling objectid) to th RelOffset of the written dataset. If it is Union{}, then references are not tracked, and objects referenced multiple times are written multiple times.
source
JLD2.MemoryBackedIO — Type
MemoryBackedIO <: IO
Abstract type for IO objects that are backed by memory in such a way that one can use pointer based unsafe_load and unsafe_store! operations after ensuring that there is enough memory allocated.
It needs to provide:
  • getproperty(io, :curptr) to get the current pointer
  • ensureroom(io, nb) to ensure that there are at least nb bytes available
  • position(io) to get the current (zero-based) position 
  • seek(io, pos) to set the current position (zero-based)
source
JLD2.Message — Type
Message{IO}
Representation of a Message in memory. Provides getproperty access
source
JLD2.RelOffset — Type
RelOffset
Represents an HDF5 relative offset. This differs from a file offset (used elsewhere) in that it is relative to the superblock base address. fileoffset and h5offset convert between RelOffsets and file offsets.
source
JLD2.SharedDatatype — Type
SharedDatatype <: H5Datatype
Reference to a shared datatype message (stored elsewhere in a file).
source
JLD2.Upgrade — Type
Upgrade(T)
Specify an upgrade path for serialized structs using the typemap keyword argument and rconvert.
source
JLD2.attributes — Method
attributes(dset::Dataset; plain::Bool=false)
Return the attributes of a dataset as an OrderedDict. If plain is set to true then the values are returned as stored in the dataset object.  
source
JLD2.behead — Method
behead(T)
Given a UnionAll type, recursively eliminates the where clauses
source
JLD2.bufferpos — Method
bufferpos(io::Union{BufferedReader, BufferedWriter})
Get the current position in the buffer.
source
JLD2.construct_array — Method
construct_array(io::IO, eltype, ndims::Int)
Construct array by reading ndims dimensions from io. Assumes io has already been seeked to the correct position.
source
JLD2.constructrr — Function
constructrr(f::JLDFile, T::DataType, dt::CompoundType, attrs::Vector{ReadAttribute},
+            hard_failure::Bool=false)
Constructs a ReadRepresentation for a given type. This is the generic method for all types not specially handled below.
If hard_failure is true, then throw a TypeMappingException instead of attempting reconstruction. This helps in cases where we can't know if reconstructed parametric types will have a matching memory layout without first inspecting the memory layout.
source
JLD2.create_dataset — Method
create_dataset(parent, path, datatype, dataspace; kwargs...)
Arguments:     - parent::Union{JLDfile, Group}: Containing group of new dataset     - path: Path to new dataset relative to parent. If path is nothing, the dataset is unnamed.     - datatype: Datatype of new dataset (element type in case of arrays)     - dataspace: Dimensions or Dataspace of new dataset
Keyword arguments:     - layout: DataLayout of new dataset     - filters: FilterPipeline for describing the compression pipeline
source
JLD2.fileoffset — Method
fileoffset(f::JLDFile, x::RelOffset)
Converts an offset x relative to the superblock of file f to an absolute offset.
source
JLD2.flag2uint — Method
flag2uint(flag::UInt8)
I Map the lowest to bits of flag to a UInt type, mapping 0 to UInt8, 1 to UInt16, 2 to UInt32, and 3 to UInt64.
source
JLD2.get_dataset — Method
get_dataset(parent::Union{JLDFile, Group}, name::String)
Get a stored dataset from a file by name or path as a Dataset object. This may be useful for inspecting the metadata incl. types of a dataset.
source
JLD2.h5offset — Method
h5offset(f::JLDFile, x::Integer)
Converts an absolute file offset x to an offset relative to the superblock of file f.
source
JLD2.ismmappable — Method
ismmappable(dset::Dataset)
Check if a dataset can be memory-mapped. This can be useful for large arrays and for editing written arrays.
An Array dataset may be mmapped if:     - JLD2.samelayout(T) == true: The element type is isbits and has a size that is a multiple of 8 bytes.     - Uncompressed: Compressed arrays cannot be memory-mapped     - Uses a contiguous layout: This is true for all array datasets written by JLD2 with version ≥ v0.4.52     - Offset in file is a multiple of 8 bytes: This is a requirement for Mmap.     - Windows: The file must be opened in read-only mode. This is a limitation of Mmap on Windows. 
source
JLD2.isset — Method
isset(flag, bit)
Return true if the bit-th bit of flag is set. (starting from 0)   
source
JLD2.jld_finalizer — Method
jld_finalizer(f::JLDFile)
When a JLDFile is finalized, it is possible that the MmapIO has been munmapped, since Julia does not guarantee finalizer order. This means that the underlying file may be closed before we get a chance to write to it.
source
JLD2.jldopen — Function
jldopen(file, mode::AbstractString; iotype=MmapIO, compress=false, typemap=Dict())
Opens a JLD2 file at path file. Alternatively file may be a suitable IO object.
Options for mode:
  • "r": Open for reading only, failing if no file exists
  • "r+": Open for reading and writing, failing if no file exists
  • "w"/"w+": Open for reading and writing, overwriting the file if it already exists
  • "a"/"a+": Open for reading and writing, creating a new file if none exists, but               preserving the existing file if one is present
source
JLD2.jldsave — Function
jldsave(filename; kwargs...)
 jldsave(filename, compress; kwargs...)
 jldsave(filename, compress, iotype; kwargs...)
Creates a JLD2 file at filename and stores the variables given as keyword arguments.
Examples
jldsave("example.jld2"; a=1, b=2, c)
is equivalent to
jldopen("example.jld2", "w") do f
     f["a"] = 1
     f["b"] = 2
     f["c"] = c
-end
To choose the io type IOStream instead of the default MmapIO use  jldsave(fn, IOStream; kwargs...).
source
JLD2.jlwrite — Method
jlwrite(io::IO, x::Tuple)
Attempt to write a tuple to io by writing each element of the tuple in order.
source
JLD2.links_size — Method
links_size(pairs)
Returns the size of several link messages. pairs is an iterator of String => RelOffset pairs.
source
JLD2.load_attributes — Function
load_attributes(f::JLDFile, name::AbstractString)
+end
To choose the io type IOStream instead of the default MmapIO use  jldsave(fn, IOStream; kwargs...).
source
JLD2.jlwrite — Method
jlwrite(io::IO, x::Tuple)
Attempt to write a tuple to io by writing each element of the tuple in order.
source
JLD2.links_size — Method
links_size(pairs)
Returns the size of several link messages. pairs is an iterator of String => RelOffset pairs.
source
JLD2.load_attributes — Function
load_attributes(f::JLDFile, name::AbstractString)
 load_attributes(g::Group, name::AbstractString)
 load_attributes(g::Group)
-load_attributes(f::JLDFile, offset::RelOffset)
Return a list of attributes attached to the dataset or group.
source
JLD2.load_data_or_dict — Method
load_data_or_dict(g::Union{JLDFile,Group}, varname::AbstractString)
Return the value of key varname but if it represents a Group load the group as a nested dictionary.
source
JLD2.load_datatypes — Method
load_datatypes(f::JLDFile)
Populate f.datatypes and f.jlh5types with all of the committed datatypes from a file. We need to do this before writing to make sure we reuse written datatypes.
source
JLD2.load_object — Method
load_object(filename)
Returns the only available object from the JLD2 file filename (The stored object name is inconsequential). If the file contains more than one or no objects, the function throws an ArgumentError.
For loading more than one object, use @load macro, jldopen or the FileIO API.
Example
To load the only object from the JLD2 file example.jld2:
hello = "world"
+load_attributes(f::JLDFile, offset::RelOffset)
Return a list of attributes attached to the dataset or group.
source
JLD2.load_data_or_dict — Method
load_data_or_dict(g::Union{JLDFile,Group}, varname::AbstractString)
Return the value of key varname but if it represents a Group load the group as a nested dictionary.
source
JLD2.load_datatypes — Method
load_datatypes(f::JLDFile)
Populate f.datatypes and f.jlh5types with all of the committed datatypes from a file. We need to do this before writing to make sure we reuse written datatypes.
source
JLD2.load_object — Method
load_object(filename)
Returns the only available object from the JLD2 file filename (The stored object name is inconsequential). If the file contains more than one or no objects, the function throws an ArgumentError.
For loading more than one object, use @load macro, jldopen or the FileIO API.
Example
To load the only object from the JLD2 file example.jld2:
hello = "world"
 save_object("example.jld2", hello)
-hello_loaded = load_object("example.jld2")
source
JLD2.loadnesteddict — Method
loadnesteddict(g::Union{JLDFile, Group})
Return a dictionary with all data contained in group or file. Nested groups are loaded as nested dictionaries.
source
JLD2.lookup_offset — Method
lookup_offset(g::Group, name::AbstractString) -> RelOffset
Lookup the offset of a dataset in a group. Returns UNDEFINED_ADDRESS if the dataset is not present. Does not inspect unwritten_child_groups.
source
JLD2.pathize — Method
pathize(g::Group, name::AbstractString, create::Bool) -> Tuple{Group,String}
Converts a path to a group and name object. If create is true, any intermediate groups will be created, and the dataset name will be checked for uniqueness with existing names.
source
JLD2.prewrite — Method
prewrite(f::JLDFile)
Check that a JLD file is actually writable, and throw an error if not. Sets the written flag on the file.
source
JLD2.loadnesteddict — Method
loadnesteddict(g::Union{JLDFile, Group})
Return a dictionary with all data contained in group or file. Nested groups are loaded as nested dictionaries.
source
JLD2.lookup_offset — Method
lookup_offset(g::Group, name::AbstractString) -> RelOffset
Lookup the offset of a dataset in a group. Returns UNDEFINED_ADDRESS if the dataset is not present. Does not inspect unwritten_child_groups.
source
JLD2.pathize — Method
pathize(g::Group, name::AbstractString, create::Bool) -> Tuple{Group,String}
Converts a path to a group and name object. If create is true, any intermediate groups will be created, and the dataset name will be checked for uniqueness with existing names.
source
JLD2.prewrite — Method
prewrite(f::JLDFile)
Check that a JLD file is actually writable, and throw an error if not. Sets the written flag on the file.
source
JLD2.print_header_messages — Method
print_header_messages(f::JLDFile, name::AbstractString)
 print_header_messages(g::Group, name::AbstractString)
-print_header_messages(f::JLDFile, offset::RelOffset)
Prints the header messages of a group or dataset in a file.
source
JLD2.printtoc — Method
printtoc([io::IO,] f::JLDFile [; numlines])
Prints an overview of the contents of f to the IO.
Use the optional numlines parameter to restrict the amount of items listed.
source
JLD2.read_attr_data — Method
read_attr_data(f::JLDFile, attr::ReadAttribute, expected_datatype::H5Datatype,
-               rr::ReadRepresentation)
jlread data from an attribute, assuming a specific HDF5 datatype and ReadRepresentation. If the HDF5 datatype does not match, throws an UnsupportedFeatureException. This allows better type stability while simultaneously validating the data.
source
JLD2.read_compressed_array! — Function
read_compressed_array!(v::Array, f::JLDFile, rr, data_length::Int, Val(filter_id))
Fill the array v with the compressed contents of JLDFile f at the current position, assuming a ReadRepresentation rr and that the compressed data has length data_length.
source
JLD2.read_data — Function
read_data(f::JLDFile, dataspace::ReadDataspace, datatype_class::UInt8,
+print_header_messages(f::JLDFile, offset::RelOffset)
Prints the header messages of a group or dataset in a file.
source
JLD2.printtoc — Method
printtoc([io::IO,] f::JLDFile [; numlines])
Prints an overview of the contents of f to the IO.
Use the optional numlines parameter to restrict the amount of items listed.
source
JLD2.read_attr_data — Method
read_attr_data(f::JLDFile, attr::ReadAttribute, expected_datatype::H5Datatype,
+               rr::ReadRepresentation)
jlread data from an attribute, assuming a specific HDF5 datatype and ReadRepresentation. If the HDF5 datatype does not match, throws an UnsupportedFeatureException. This allows better type stability while simultaneously validating the data.
source
JLD2.read_compressed_array! — Function
read_compressed_array!(v::Array, f::JLDFile, rr, data_length::Int, Val(filter_id))
Fill the array v with the compressed contents of JLDFile f at the current position, assuming a ReadRepresentation rr and that the compressed data has length data_length.
source
JLD2.read_data — Function
read_data(f::JLDFile, dataspace::ReadDataspace, datatype_class::UInt8,
           datatype_offset::Int64, data_offset::Int64[, filters::FilterPipeline,
-          header_offset::RelOffset, attributes::Vector{ReadAttribute}])
Read data from a file. If datatype_class is typemax(UInt8), the datatype is assumed to be committed, and datatype_offset points to the offset of the committed datatype's header. Otherwise, datatype_offset points to the offset of the datatype attribute.
source
JLD2.read_scalar — Function
read_scalar(f::JLDFile, rr, header_offset::RelOffset)
Read raw data representing a scalar with read representation rr from the current position of JLDFile f. header_offset is the RelOffset of the object header, used to resolve cycles.
source
JLD2.read_size — Method
read_size(io::IO, flags::UInt8)
Loads a variable-length size according to flags
Expects that the first two bits of flags mean:
  • 0:   The size of the Length of Link Name field is 1 byte.
  • 1:   The size of the Length of Link Name field is 2 bytes.
  • 2:   The size of the Length of Link Name field is 4 bytes.
  • 3:   The size of the Length of Link Name field is 8 bytes.
Returns the size as an Int.
source
JLD2.readas — Method
readas(::Type)::Type
Experimental feature:  JLD2.readas can be overloaded to override which type a saved type is read as, and is used together with custom serialization using JLD2.writeas.
The typical case is custom serialization of parametric types, where not all type parameters are available during reading.  Consider the following example for an anonymous function fun inside a Foo
struct Foo{F<:Function}
+          header_offset::RelOffset, attributes::Vector{ReadAttribute}])
Read data from a file. If datatype_class is typemax(UInt8), the datatype is assumed to be committed, and datatype_offset points to the offset of the committed datatype's header. Otherwise, datatype_offset points to the offset of the datatype attribute.
source
JLD2.read_scalar — Function
read_scalar(f::JLDFile, rr, header_offset::RelOffset)
Read raw data representing a scalar with read representation rr from the current position of JLDFile f. header_offset is the RelOffset of the object header, used to resolve cycles.
source
JLD2.read_size — Method
read_size(io::IO, flags::UInt8)
Loads a variable-length size according to flags
Expects that the first two bits of flags mean:
  • 0:   The size of the Length of Link Name field is 1 byte.
  • 1:   The size of the Length of Link Name field is 2 bytes.
  • 2:   The size of the Length of Link Name field is 4 bytes.
  • 3:   The size of the Length of Link Name field is 8 bytes.
Returns the size as an Int.
source
JLD2.readas — Method
readas(::Type)::Type
Experimental feature:  JLD2.readas can be overloaded to override which type a saved type is read as, and is used together with custom serialization using JLD2.writeas.
The typical case is custom serialization of parametric types, where not all type parameters are available during reading.  Consider the following example for an anonymous function fun inside a Foo
struct Foo{F<:Function}
     fun::F
 end
 struct FooSerialization
@@ -35,8 +35,8 @@
 function Base.convert(::Type{<:Foo}, f::FooSerialization)
     isa(f.fun, Function) && return Foo(f.fun)
     return Foo(UndefinedFunction(f.fun))
-end
If we include these definitions, call jldsave("foo.jld2"; foo=Foo(x->x^2)), restart julia, include the definitions again, and call foo = jldopen("foo.jld2") do io; io["foo"]; end, we get foo::Foo{UndefinedFunction} and foo::FooSerialization with and without defining the JLD2.readas above, respectively.
source
JLD2.readmmap — Method
readmmap(dset::Dataset)
Memory-map a dataset. This can be useful for large arrays and for editing written arrays. See ismmappable for requirements.
source
JLD2.save_group — Method
save_group(g::Group) -> RelOffset
Stores a group to a file, updating it if it has already been saved. Returns UNDEFINED_ADDRESS if the group was already stored, or the offset of the new group otherwise.
source
JLD2.save_object — Method
save_object(filename, x)
Stores an object x in a new JLD2 file at filename. If a file exists at this path, it will be overwritten.
Since the JLD2 format requires that all objects have a name, the object will be stored as single_stored_object. If you want to store more than one object, use @save macro, jldopen or the FileIO API.
Example
To save the string hello to the JLD2 file example.jld2:
hello = "world"
-save_object("example.jld2", hello)
source
JLD2.shorttypestring — Method
shorttypestring(::Type{ <:UnknownType})
Convert an UnknownType to a corresponding string. This is only used to create names for reconstructed types.
See also typestring.
source
JLD2.size_flag — Method
size_flag(sz::Integer)::UInt8
Return the flag that represents the smallest integer type that can represent sz. 0 -> UInt8, 1 -> UInt16, 2 -> UInt32, 3 -> UInt64
source
JLD2.size_size — Method
size_size(sz::Integer)
Return the number of bytes required to represent sz as an unsigned integer that actually exists. (e.g. UInt8, UInt16, UInt32, UInt64)
source
JLD2.size_size2 — Method
size_size2(sz::Integer)
Return the number of bytes required to represent sz as an unsigned integer. Note: this does not check if the integer is a valid julia integer.
source
JLD2.skip_to_aligned! — Function
skip_to_aligned!(io, rel=0)
Skip to nearest position aligned to a multiple of 8 bytes relative to rel.
source
JLD2.typestring — Method
typestring(::Type{ <:UnknownType})
Convert an UnknownType to a corresponding string. This is only used for warning during reconstruction errors.
See also shorttypestring.
source
JLD2.write_dataset — Function
write_dataset(dataset::Dataset, data)
Write data to file using metadata prepared in the dataset.
source
JLD2.write_size — Method
write_size(io::IO, sz::Integer)
Write the mininum number of bytes required to represent sz as (valid) unsigned integer.
source
JLD2.@load — Macro
@load filename var1 [var2 ...]
Load one or more variables var1,... from JLD2 file filename into the current scope and return a vector of the loaded variable names.
For interactive use, the form @load "somefile.jld2" will load all variables from "somefile.jld2" into the current scope. This form only supports literal file names and should be avoided in more permanent code so that it's clear where the variables come from.
Example
To load the variables hello and foo from the file example.jld2, use
@load "example.jld2" hello foo
source
JLD2.@pseudostruct — Macro
@pseudostruct name begin ... end
The @pseudostruct macro is used to define constructor, size computation, show, and and optimized getproperty function for Messages. The allowed syntax elements are:
  • @skip(n): Mark n bytes as empty.
source
JLD2.@save — Macro
@save filename var1 [var2 ...]
+end
If we include these definitions, call jldsave("foo.jld2"; foo=Foo(x->x^2)), restart julia, include the definitions again, and call foo = jldopen("foo.jld2") do io; io["foo"]; end, we get foo::Foo{UndefinedFunction} and foo::FooSerialization with and without defining the JLD2.readas above, respectively.
source
JLD2.readmmap — Method
readmmap(dset::Dataset)
Memory-map a dataset. This can be useful for large arrays and for editing written arrays. See ismmappable for requirements.
source
JLD2.save_group — Method
save_group(g::Group) -> RelOffset
Stores a group to a file, updating it if it has already been saved. Returns UNDEFINED_ADDRESS if the group was already stored, or the offset of the new group otherwise.
source
JLD2.save_object — Method
save_object(filename, x)
Stores an object x in a new JLD2 file at filename. If a file exists at this path, it will be overwritten.
Since the JLD2 format requires that all objects have a name, the object will be stored as single_stored_object. If you want to store more than one object, use @save macro, jldopen or the FileIO API.
Example
To save the string hello to the JLD2 file example.jld2:
hello = "world"
+save_object("example.jld2", hello)
source
JLD2.shorttypestring — Method
shorttypestring(::Type{ <:UnknownType})
Convert an UnknownType to a corresponding string. This is only used to create names for reconstructed types.
See also typestring.
source
JLD2.size_flag — Method
size_flag(sz::Integer)::UInt8
Return the flag that represents the smallest integer type that can represent sz. 0 -> UInt8, 1 -> UInt16, 2 -> UInt32, 3 -> UInt64
source
JLD2.size_size — Method
size_size(sz::Integer)
Return the number of bytes required to represent sz as an unsigned integer that actually exists. (e.g. UInt8, UInt16, UInt32, UInt64)
source
JLD2.size_size2 — Method
size_size2(sz::Integer)
Return the number of bytes required to represent sz as an unsigned integer. Note: this does not check if the integer is a valid julia integer.
source
JLD2.skip_to_aligned! — Function
skip_to_aligned!(io, rel=0)
Skip to nearest position aligned to a multiple of 8 bytes relative to rel.
source
JLD2.typestring — Method
typestring(::Type{ <:UnknownType})
Convert an UnknownType to a corresponding string. This is only used for warning during reconstruction errors.
See also shorttypestring.
source
JLD2.write_dataset — Function
write_dataset(dataset::Dataset, data)
Write data to file using metadata prepared in the dataset.
source
JLD2.write_size — Method
write_size(io::IO, sz::Integer)
Write the mininum number of bytes required to represent sz as (valid) unsigned integer.
source
JLD2.@load — Macro
@load filename var1 [var2 ...]
Load one or more variables var1,... from JLD2 file filename into the current scope and return a vector of the loaded variable names.
For interactive use, the form @load "somefile.jld2" will load all variables from "somefile.jld2" into the current scope. This form only supports literal file names and should be avoided in more permanent code so that it's clear where the variables come from.
Example
To load the variables hello and foo from the file example.jld2, use
@load "example.jld2" hello foo
source
JLD2.@pseudostruct — Macro
@pseudostruct name begin ... end
The @pseudostruct macro is used to define constructor, size computation, show, and and optimized getproperty function for Messages. The allowed syntax elements are:
  • @skip(n): Mark n bytes as empty.
source
JLD2.@save — Macro
@save filename var1 [var2 ...]
 @save filename {compress=true} var1 name2=var2
Write one or more variables var1,... from the current scope to a JLD2 file filename.
For interactive use you can save all variables in the current module's global scope using @save filename. More permanent code should prefer the explicit form to avoid saving unwanted variables.
Example
To save the string hello and array xs to the JLD2 file example.jld2:
hello = "world"
 xs = [1,2,3]
-@save "example.jld2" hello xs
For passing options to the saving command use {}
@save "example.jld2" {compress=true} hello xs
For saving variables under a different name use regular assignment syntax
@save "example.jld2" greeting=hello xarray = xs
source

+@save "example.jld2" hello xs

For passing options to the saving command use {}

@save "example.jld2" {compress=true} hello xs

For saving variables under a different name use regular assignment syntax

@save "example.jld2" greeting=hello xarray = xs
source
diff --git a/dev/legacy/index.html b/dev/legacy/index.html index 5bcfd9c7..459f1f8e 100644 --- a/dev/legacy/index.html +++ b/dev/legacy/index.html @@ -6,4 +6,4 @@ @save "example.jld2" {compress=true} hello bar=foo
JLD2.@save — Macro
@save filename var1 [var2 ...]
 @save filename {compress=true} var1 name2=var2

Write one or more variables var1,... from the current scope to a JLD2 file filename.

For interactive use you can save all variables in the current module's global scope using @save filename. More permanent code should prefer the explicit form to avoid saving unwanted variables.

Example

To save the string hello and array xs to the JLD2 file example.jld2:

hello = "world"
 xs = [1,2,3]
-@save "example.jld2" hello xs

For passing options to the saving command use {}

@save "example.jld2" {compress=true} hello xs

For saving variables under a different name use regular assignment syntax

@save "example.jld2" greeting=hello xarray = xs
source
JLD2.@load — Macro
@load filename var1 [var2 ...]

Load one or more variables var1,... from JLD2 file filename into the current scope and return a vector of the loaded variable names.

For interactive use, the form @load "somefile.jld2" will load all variables from "somefile.jld2" into the current scope. This form only supports literal file names and should be avoided in more permanent code so that it's clear where the variables come from.

Example

To load the variables hello and foo from the file example.jld2, use

@load "example.jld2" hello foo
source
+@save "example.jld2" hello xs

For passing options to the saving command use {}

@save "example.jld2" {compress=true} hello xs

For saving variables under a different name use regular assignment syntax

@save "example.jld2" greeting=hello xarray = xs
source
JLD2.@load — Macro
@load filename var1 [var2 ...]

Load one or more variables var1,... from JLD2 file filename into the current scope and return a vector of the loaded variable names.

For interactive use, the form @load "somefile.jld2" will load all variables from "somefile.jld2" into the current scope. This form only supports literal file names and should be avoided in more permanent code so that it's clear where the variables come from.

Example

To load the variables hello and foo from the file example.jld2, use

@load "example.jld2" hello foo
source
diff --git a/dev/objects.inv b/dev/objects.inv index 50fdf5ce7b74839a82805eec480459a5a9c28821..fc2360b1e5e401c326f763e978d1e64365f5de64 100644 GIT binary patch delta 12 TcmX>ua9m)5C!@tiufOa7AEpGr delta 12 TcmX>ua9m)5C!_gBufOa7AE5-l