From cc4630fd6825c120a3cb73f8d59681cb1167806d Mon Sep 17 00:00:00 2001 From: Chris Nuernberger Date: Sat, 16 Nov 2024 10:39:26 -0700 Subject: [PATCH] Release 10.120 --- CHANGELOG.md | 3 ++ deps.edn | 2 +- docs/buffered-image.html | 2 +- docs/cheatsheet.html | 2 +- docs/datatype-to-dtype-next.html | 2 +- docs/dimensions-bytecode-gen.html | 2 +- docs/index.html | 6 ++-- docs/overview.html | 2 +- docs/tech.v3.datatype.argops.html | 2 +- docs/tech.v3.datatype.bitmap.html | 2 +- docs/tech.v3.datatype.convolve.html | 2 +- docs/tech.v3.datatype.datetime.html | 2 +- docs/tech.v3.datatype.errors.html | 2 +- docs/tech.v3.datatype.ffi.clang.html | 2 +- docs/tech.v3.datatype.ffi.graalvm.html | 2 +- docs/tech.v3.datatype.ffi.html | 38 ++++++++++++------------ docs/tech.v3.datatype.ffi.size-t.html | 2 +- docs/tech.v3.datatype.functional.html | 2 +- docs/tech.v3.datatype.gradient.html | 2 +- docs/tech.v3.datatype.html | 37 +++++++++++++---------- docs/tech.v3.datatype.list.html | 2 +- docs/tech.v3.datatype.mmap-writer.html | 2 +- docs/tech.v3.datatype.mmap.html | 2 +- docs/tech.v3.datatype.native-buffer.html | 21 ++++++++----- docs/tech.v3.datatype.nippy.html | 2 +- docs/tech.v3.datatype.packing.html | 2 +- docs/tech.v3.datatype.reductions.html | 2 +- docs/tech.v3.datatype.rolling.html | 2 +- docs/tech.v3.datatype.sampling.html | 2 +- docs/tech.v3.datatype.statistics.html | 2 +- docs/tech.v3.datatype.struct.html | 2 +- docs/tech.v3.datatype.wavelet.html | 2 +- docs/tech.v3.libs.buffered-image.html | 2 +- docs/tech.v3.parallel.for.html | 2 +- docs/tech.v3.parallel.queue-iter.html | 2 +- docs/tech.v3.tensor.color-gradients.html | 2 +- docs/tech.v3.tensor.dimensions.html | 2 +- docs/tech.v3.tensor.html | 2 +- src/tech/v3/datatype.clj | 12 ++++---- src/tech/v3/datatype_api.clj | 2 +- test/tech/v3/datatype_test.clj | 7 +++++ 41 files changed, 107 insertions(+), 85 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 6b714d6d..6f202d7b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,6 @@ +## 10.120 + * Fixed sub-buffer of indexed buffers throwing exception. + ## 10.114 * [issue-100](https://github.com/cnuernber/dtype-next/issues/100) - Support latest neanderthal. diff --git a/deps.edn b/deps.edn index 65459ff8..fd10198a 100644 --- a/deps.edn +++ b/deps.edn @@ -21,7 +21,7 @@ :exec-fn codox.main/-main :exec-args {:group-id "cnuernber" :artifact-id "dtype-next" - :version "10.119" + :version "10.120" :name "DType Next" :description "High Performance Clojure Primitives" :metadata {:doc/format :markdown} diff --git a/docs/buffered-image.html b/docs/buffered-image.html index c86d4d65..190fdadb 100644 --- a/docs/buffered-image.html +++ b/docs/buffered-image.html @@ -4,7 +4,7 @@ function gtag(){dataLayer.push(arguments);} gtag('js', new Date()); - gtag('config', 'G-95TVFC1FEB');

Project

Topics

Namespaces

dtype-next Buffered Image Support

+ gtag('config', 'G-95TVFC1FEB');

Project

Topics

Namespaces

dtype-next Buffered Image Support

dtype-next contains support for loading/saving buffered images and creating tensors from buffered images.

Usage

diff --git a/docs/cheatsheet.html b/docs/cheatsheet.html index c8261069..bfbc5272 100644 --- a/docs/cheatsheet.html +++ b/docs/cheatsheet.html @@ -4,7 +4,7 @@ function gtag(){dataLayer.push(arguments);} gtag('js', new Date()); - gtag('config', 'G-95TVFC1FEB');

Project

Topics

Namespaces

dtype-next Cheatsheet

+ gtag('config', 'G-95TVFC1FEB');

Project

Topics

Namespaces

dtype-next Cheatsheet

The old cheatsheet has been moved to overview.

Most of these functions are accessible via the [tech.v3.datatype :as dtype] namespace. When another namespace is required, it is specified separately.

diff --git a/docs/datatype-to-dtype-next.html b/docs/datatype-to-dtype-next.html index 30381ed2..54b9de8d 100644 --- a/docs/datatype-to-dtype-next.html +++ b/docs/datatype-to-dtype-next.html @@ -4,7 +4,7 @@ function gtag(){dataLayer.push(arguments);} gtag('js', new Date()); - gtag('config', 'G-95TVFC1FEB');

Project

Topics

Namespaces

Why dtype-next?

+ gtag('config', 'G-95TVFC1FEB');

Project

Topics

Namespaces

Why dtype-next?

tech.datatype

tech.datatype as a numerics stack fulfills our technical needs at TechAscent in regards to scientific computing, data science, and machine learning. It enables a completely diff --git a/docs/dimensions-bytecode-gen.html b/docs/dimensions-bytecode-gen.html index d9245d4c..488c9a31 100644 --- a/docs/dimensions-bytecode-gen.html +++ b/docs/dimensions-bytecode-gen.html @@ -4,7 +4,7 @@ function gtag(){dataLayer.push(arguments);} gtag('js', new Date()); - gtag('config', 'G-95TVFC1FEB');

Project

Topics

Namespaces

dtype-next Tensor Dimensions and Bytecode Generation

+ gtag('config', 'G-95TVFC1FEB');

Project

Topics

Namespaces

dtype-next Tensor Dimensions and Bytecode Generation

dtype-next uses a dimensions operator in order to transform nd space to a linear address space. We no longer compile in bytecode address operator generation by default as it takes a bit longer to load and most users won't see any benefit as diff --git a/docs/index.html b/docs/index.html index 83b28d8f..2b290264 100644 --- a/docs/index.html +++ b/docs/index.html @@ -1,10 +1,10 @@ -DType Next 10.119

Project

Topics

Namespaces

DType Next 10.119

High Performance Clojure Primitives.

Topics

Namespaces

tech.v3.datatype

Base namespace for container creation and elementwise access of data

+ gtag('config', 'G-95TVFC1FEB');

Project

Topics

Namespaces

DType Next 10.120

High Performance Clojure Primitives.

Topics

Namespaces

tech.v3.datatype

Base namespace for container creation and elementwise access of data

Public variables and functions:

tech.v3.datatype.argops

Efficient functions for operating in index space. Take-off of the argsort, argmin, etc. type functions from Matlab. These functions generally only work on readers and all return some version of an index or list of indexes.

@@ -19,7 +19,7 @@

Public variables and functions:

tech.v3.datatype.errors

Generic, basic error handling. No dependencies aside from apache commons math for NAN exception.

Public variables and functions:

tech.v3.datatype.ffi

Generalized C Foreign Function Interface (FFI) that unifies JNA and JDK-16 FFI architectures.

-

Public variables and functions:

tech.v3.datatype.ffi.clang

General utilites to work with clang in order to define things like structs or +

Public variables and functions:

tech.v3.datatype.ffi.clang

General utilites to work with clang in order to define things like structs or enumerations.

Public variables and functions:

tech.v3.datatype.ffi.graalvm

Graalvm-specific namespace that implements the dtype-next ffi system and allows users to build stand-alone shared libraries that other languages can use to call Clojure code via diff --git a/docs/overview.html b/docs/overview.html index dc23e0ad..9d2bb27f 100644 --- a/docs/overview.html +++ b/docs/overview.html @@ -4,7 +4,7 @@ function gtag(){dataLayer.push(arguments);} gtag('js', new Date()); - gtag('config', 'G-95TVFC1FEB');

Project

Topics

Namespaces

dtype-next Overview

+ gtag('config', 'G-95TVFC1FEB');

Project

Topics

Namespaces

dtype-next Overview

Setup

(require '[tech.v3.datatype :as dtype])
diff --git a/docs/tech.v3.datatype.argops.html b/docs/tech.v3.datatype.argops.html index 88561b5e..3ed96edf 100644 --- a/docs/tech.v3.datatype.argops.html +++ b/docs/tech.v3.datatype.argops.html @@ -4,7 +4,7 @@ function gtag(){dataLayer.push(arguments);} gtag('js', new Date()); - gtag('config', 'G-95TVFC1FEB');

Project

Topics

Namespaces

Public Vars

tech.v3.datatype.argops

Efficient functions for operating in index space. Take-off of the argsort, argmin, etc. + gtag('config', 'G-95TVFC1FEB');

Project

Topics

Namespaces

Public Vars

tech.v3.datatype.argops

Efficient functions for operating in index space. Take-off of the argsort, argmin, etc. type functions from Matlab. These functions generally only work on readers and all return some version of an index or list of indexes.

->binary-operator

(->binary-operator op)

Convert a thing to a binary operator. Thing can be a keyword or diff --git a/docs/tech.v3.datatype.bitmap.html b/docs/tech.v3.datatype.bitmap.html index 542a7bea..d1258ca1 100644 --- a/docs/tech.v3.datatype.bitmap.html +++ b/docs/tech.v3.datatype.bitmap.html @@ -4,7 +4,7 @@ function gtag(){dataLayer.push(arguments);} gtag('js', new Date()); - gtag('config', 'G-95TVFC1FEB');

Project

Topics

Namespaces

Public Vars

tech.v3.datatype.bitmap

Functions for working with RoaringBitmaps. These are integrated deeply into + gtag('config', 'G-95TVFC1FEB');

Project

Topics

Namespaces

Public Vars

tech.v3.datatype.bitmap

Functions for working with RoaringBitmaps. These are integrated deeply into several tech.v3.datatype algorithms and have many potential applications in high performance computing applications as they are both extremely fast and storage-space efficient.

diff --git a/docs/tech.v3.datatype.convolve.html b/docs/tech.v3.datatype.convolve.html index 30b5abb6..99b744ab 100644 --- a/docs/tech.v3.datatype.convolve.html +++ b/docs/tech.v3.datatype.convolve.html @@ -4,7 +4,7 @@ function gtag(){dataLayer.push(arguments);} gtag('js', new Date()); - gtag('config', 'G-95TVFC1FEB');

Project

Topics

Namespaces

Public Vars

tech.v3.datatype.convolve

Namespace for implementing various basic convolutions. Currently only 1d + gtag('config', 'G-95TVFC1FEB');

Project

Topics

Namespaces

Public Vars

tech.v3.datatype.convolve

Namespace for implementing various basic convolutions. Currently only 1d convolutions are supported.

convolve1d

(convolve1d data win options)(convolve1d data win)

Convolve a window across a signal. The only difference from correlate is the window is reversed and then correlate is called. See options for correlate, this diff --git a/docs/tech.v3.datatype.datetime.html b/docs/tech.v3.datatype.datetime.html index f323a89c..7314ae96 100644 --- a/docs/tech.v3.datatype.datetime.html +++ b/docs/tech.v3.datatype.datetime.html @@ -4,7 +4,7 @@ function gtag(){dataLayer.push(arguments);} gtag('js', new Date()); - gtag('config', 'G-95TVFC1FEB');

Project

Topics

Namespaces

Public Vars

tech.v3.datatype.datetime

Thorough bindings to java.time.datetime. Includes packed datatypes and + gtag('config', 'G-95TVFC1FEB');

Project

Topics

Namespaces

Public Vars

tech.v3.datatype.datetime

Thorough bindings to java.time.datetime. Includes packed datatypes and lifting various datetime datatypes into the datatype system.

A general outline is:

See foreign-interface-instance->c for example.

-
view source

define-library

(define-library fn-defs symbols options)(define-library fn-defs options)(define-library fn-defs)

Define a library returning the class. The class can be instantiated with a string +

view source

define-library

(define-library fn-defs symbols options)(define-library fn-defs options)(define-library fn-defs)

Define a library returning the class. The class can be instantiated with a string naming the library path or nil for the current process. After instantiation the library instance will have strongly typed methods named the same as the keyword keys in fn-defs efficiently bound to symbols of the same exact name in @@ -244,7 +244,7 @@

A Word On Efficiency

[0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, ] user> -
view source

define-library!

macro

(define-library! lib-varname lib-fns lib-symbols error-checker)

Define a library singleton that will export the defined functions and symbols.

+
view source

define-library!

macro

(define-library! lib-varname lib-fns lib-symbols error-checker)

Define a library singleton that will export the defined functions and symbols.

Only jvm-supported primitives work here - no unsigned but if you have the correct datatype widths the data will be passed unchanged so unsigned does work.

See namespace declaration for full example.

@@ -268,7 +268,7 @@

A Word On Efficiency

nil nil) -
view source

define-library-functions

macro

(define-library-functions library-def-symbol find-fn check-error)

Define public callable vars that will call library functions marshaling strings +

view source

define-library-functions

macro

(define-library-functions library-def-symbol find-fn check-error)

Define public callable vars that will call library functions marshaling strings back and forth. These vars will call find-fn with the fn-name in a late-bound way and check-error may be provided and called on a return value assuming :check-error? is set in the library definition.

@@ -282,7 +282,7 @@

A Word On Efficiency

Example:

(dt-ffi/define-library-functions avclj.ffi/avcodec-fns find-avcodec-fn check-error)
-
view source

define-library-interface

macro

(define-library-interface fn-defs & {:keys [_classname check-error symbols libraries _header-files], :as opts})

Define public callable vars that will call library functions marshaling strings +

view source

define-library-interface

macro

(define-library-interface fn-defs & {:keys [_classname check-error symbols libraries _header-files], :as opts})

Define public callable vars that will call library functions marshaling strings back and forth. Returns a library singleton.

  • fn-defs - map of fn-name -> {:rettype :argtypes} @@ -347,29 +347,29 @@

    A Word On Efficiency

    #native-buffer@0x00007F4E28CE56D0<float32>[10] [0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, ] -
view source

enable-string->c-stats!

(enable-string->c-stats! enabled?)(enable-string->c-stats!)

Enable tracking how often string->c is getting called.

+
view source

enable-string->c-stats!

(enable-string->c-stats! enabled?)(enable-string->c-stats!)

Enable tracking how often string->c is getting called.

view source

find-library

(find-library libname)

This method is useful to check if loading a library will work. If successful, returns the library name that did finally succeed. This may be different from libname in the case where java.library.path has been used to actively find the library.

-
view source

find-symbol

(find-symbol libname symbol-name)(find-symbol symbol-name)

Find a symbol in a library. A library symbol is guaranteed to have a conversion to +

view source

find-symbol

(find-symbol libname symbol-name)(find-symbol symbol-name)

Find a symbol in a library. A library symbol is guaranteed to have a conversion to a pointer.

-
view source

foreign-interface-instance->c

(foreign-interface-instance->c ffi-def foreign-inst)

Convert an instance of the above foreign interface definition into a Pointer +

view source

foreign-interface-instance->c

(foreign-interface-instance->c ffi-def foreign-inst)

Convert an instance of the above foreign interface definition into a Pointer suitable to be called from C. Callers must ensure that foreign-inst is visible to the gc the entire time the foreign system has a reference to the pointer.

  • foreign-inst - an instance of the class defined by 'define-foreign-interface'.
-
view source

if-class

macro

(if-class class-name then)(if-class class-name then else?)
view source

instantiate-class

(instantiate-class cls & args)

Utility function to instantiate a class via its first constructor in its list +

view source

if-class

macro

(if-class class-name then)(if-class class-name then else?)
view source

instantiate-class

(instantiate-class cls & args)

Utility function to instantiate a class via its first constructor in its list of declared constructors. Works with classes returned from 'define-library' and 'define-foreign-interface'. Uses reflection.

-
view source

instantiate-foreign-interface

(instantiate-foreign-interface ffi-def ifn)

Instantiate a foreign interface defintion. This returns an instance object which +

view source

instantiate-foreign-interface

(instantiate-foreign-interface ffi-def ifn)

Instantiate a foreign interface defintion. This returns an instance object which can then be converted into a c-pointer via foreign-interface-instance->c.

-
view source

instantiate-library

(instantiate-library library-def libpath)

Uses reflection to instantiate a library

-
view source

jdk-ffi?

(jdk-ffi?)

Is the JDK foreign function interface available?

+
view source

instantiate-library

(instantiate-library library-def libpath)

Uses reflection to instantiate a library

+
view source

jdk-ffi?

(jdk-ffi?)

Is the JDK foreign function interface available?

view source

jdk-mmodel?

(jdk-mmodel?)

Is the JDK native memory model available?

view source

jna-ffi?

(jna-ffi?)

Is JNA's ffi interface available?

-
view source

library-loadable?

(library-loadable? libname)
view source

library-singleton

(library-singleton library-def-var library-sym-var library-def-opts)(library-singleton library-def-var)

Create a singleton object, ideally assigned to a variable with defonce, that +

view source

library-loadable?

(library-loadable? libname)
view source

library-singleton

(library-singleton library-def-var library-sym-var library-def-opts)(library-singleton library-def-var)

Create a singleton object, ideally assigned to a variable with defonce, that you can reset to auto-reload the bindings i.e. with new function definitions at the repl.

    @@ -388,9 +388,9 @@

    A Word On Efficiency

    [fn-kwd] (dt-ffi/library-singelton-find-fn lib fn-kwd)) -
view source

make-ptr

(make-ptr dtype prim-init-value options)(make-ptr dtype prim-init-value)

Make an object convertible to a pointer that points to single value of type +

view source

make-ptr

(make-ptr dtype prim-init-value options)(make-ptr dtype prim-init-value)

Make an object convertible to a pointer that points to single value of type dtype.

-
view source

PLibrarySingleton

protocol

Protocol to allow easy definition of a library singleton that manages re-creating +

view source

PLibrarySingleton

protocol

Protocol to allow easy definition of a library singleton that manages re-creating the library definition when the library data changes and also recreating the library instance if necessary.

members

library-singleton-definition

(library-singleton-definition lib-singleton)

Return the library definition.

@@ -403,19 +403,19 @@

A Word On Efficiency

iterative development.

library-singleton-set!

(library-singleton-set! lib-singleton libpath)

Set the library path, generate the library and create a new instance.

library-singleton-set-instance!

(library-singleton-set-instance! lib-singleton libinst)

In some cases such as graal native pathways you have to hard-set the definition and instance.

-
view source

PToPointer

protocol

members

->pointer

(->pointer item)

Convert an item into a Pointer, throwing an exception upon failure.

+
view source

pointer->address

(pointer->address p)
view source

PToPointer

protocol

members

->pointer

(->pointer item)

Convert an item into a Pointer, throwing an exception upon failure.

convertible-to-pointer?

(convertible-to-pointer? item)

Query whether an item is convertible to a pointer.

view source

ptr->struct

(ptr->struct struct-type ptr)

Given a struct type and a pointer return a struct whose data starts at the address of the pointer.

  (let [codec (dt-ffi/ptr->struct (:datatype-name @av-context/codec-def*) codec-ptr)]
        ...)
-
view source

set-ffi-impl!

(set-ffi-impl! ffi-kwd)

Set the global ffi implementation. Options are

+
view source

set-ffi-impl!

(set-ffi-impl! ffi-kwd)

Set the global ffi implementation. Options are

  • :jdk - namespace 'tech.v3.datatype.ffi.mmodel/ffi-fns - only available with jdk's foreign function module enabled.
  • :jna - namespace ''tech.v3.datatype.ffi.jna/ffi-fns - available if JNA version 5+ is in the classpath.
-
view source

string->c

(string->c str-data & [{:keys [encoding], :as options}])

Convert a java String to a zero-padded c-string. Available encodings are

+
view source

string->c

(string->c str-data & [{:keys [encoding], :as options}])

Convert a java String to a zero-padded c-string. Available encodings are

  • :ascii, :utf-8 (default), :utf-16, :utf-16LE, utf-16-BE and :utf-32
@@ -428,7 +428,7 @@

A Word On Efficiency

(struct-member-ptr encoder-frame :data) (struct-member-ptr encoder-frame :linesize)) -
view source

utf16-platform-encoding

(utf16-platform-encoding)

Get the utf-16 encoding that matches the current platform so you can encode +

view source

utf16-platform-encoding

(utf16-platform-encoding)

Get the utf-16 encoding that matches the current platform so you can encode a utf-16 string without a bom.

view source

utf32-platform-encoding

(utf32-platform-encoding)

Get the utf-16 encoding that matches the current platform so you can encode a utf-16 string without a bom.

diff --git a/docs/tech.v3.datatype.ffi.size-t.html b/docs/tech.v3.datatype.ffi.size-t.html index 98ca64f8..6af21779 100644 --- a/docs/tech.v3.datatype.ffi.size-t.html +++ b/docs/tech.v3.datatype.ffi.size-t.html @@ -4,7 +4,7 @@ function gtag(){dataLayer.push(arguments);} gtag('js', new Date()); - gtag('config', 'G-95TVFC1FEB');

Project

Topics

Namespaces

Public Vars

tech.v3.datatype.ffi.size-t

offset-t-type

(offset-t-type)

the offset-t datatype - either :int32 or :int64.

+ gtag('config', 'G-95TVFC1FEB');

Project

Topics

Namespaces

Public Vars

tech.v3.datatype.ffi.size-t

offset-t-type

(offset-t-type)

the offset-t datatype - either :int32 or :int64.

view source

ptr-t-size

dynamic

view source

ptr-t-type

(ptr-t-type)

the ptr-t-type datatype - either :uint32 or :uint64.

view source

size-t-size

dynamic

view source

size-t-type

(size-t-type)

the size-t datatype - either :uint32 or :uint64.

view source

with-size-t-size

macro

(with-size-t-size new-size & body)
view source
\ No newline at end of file diff --git a/docs/tech.v3.datatype.functional.html b/docs/tech.v3.datatype.functional.html index 6c0ad117..58b87e8b 100644 --- a/docs/tech.v3.datatype.functional.html +++ b/docs/tech.v3.datatype.functional.html @@ -4,7 +4,7 @@ function gtag(){dataLayer.push(arguments);} gtag('js', new Date()); - gtag('config', 'G-95TVFC1FEB');

Project

Topics

Namespaces

Public Vars

tech.v3.datatype.functional

Arithmetic and statistical operations based on the Buffer interface. These + gtag('config', 'G-95TVFC1FEB');

Project

Topics

Namespaces

Public Vars

tech.v3.datatype.functional

Arithmetic and statistical operations based on the Buffer interface. These operators and functions all implement vectorized interfaces so passing in something convertible to a reader will return a reader. Arithmetic operations are done lazily. These functions generally incur a large dispatch cost so for example each call to '+' diff --git a/docs/tech.v3.datatype.gradient.html b/docs/tech.v3.datatype.gradient.html index 41fb7f8f..b20cde61 100644 --- a/docs/tech.v3.datatype.gradient.html +++ b/docs/tech.v3.datatype.gradient.html @@ -4,7 +4,7 @@ function gtag(){dataLayer.push(arguments);} gtag('js', new Date()); - gtag('config', 'G-95TVFC1FEB');

Project

Topics

Namespaces

Public Vars

tech.v3.datatype.gradient

Calculate the numeric gradient of successive elements. + gtag('config', 'G-95TVFC1FEB');

Project

Topics

Namespaces

Public Vars

tech.v3.datatype.gradient

Calculate the numeric gradient of successive elements. Contains simplified versions of numpy.gradient and numpy.diff.

diff1d

(diff1d data & [options])

Returns a lazy reader of each successive element minus the previous element of length input-len - 1 unless the :prepend option is used.

diff --git a/docs/tech.v3.datatype.html b/docs/tech.v3.datatype.html index 03303bfe..40120738 100644 --- a/docs/tech.v3.datatype.html +++ b/docs/tech.v3.datatype.html @@ -4,7 +4,7 @@ function gtag(){dataLayer.push(arguments);} gtag('js', new Date()); - gtag('config', 'G-95TVFC1FEB');

Project

Topics

Namespaces

Public Vars

tech.v3.datatype

Base namespace for container creation and elementwise access of data

+ gtag('config', 'G-95TVFC1FEB');

Project

Topics

Namespaces

Public Vars

tech.v3.datatype

Base namespace for container creation and elementwise access of data

->array

(->array datatype options item)(->array datatype item)(->array item)

Perform a NaN-aware conversion into an array. Default nan-strategy is :keep. Nan strategies can be: :keep :remove :exception

@@ -163,12 +163,17 @@ true false false false false user> (dtype/make-reader :float32 5 (* idx 2)) 0.0 2.0 4.0 6.0 8.0

-
view source

make-reader-fn

(make-reader-fn _datatype advertised-datatype n-elems read-fn)
view source

malloc

(malloc n-bytes opts)(malloc n-bytes)

Malloc memory. If a desired buffer type is needed follow up with set-native-datatype.

+
view source

make-reader-fn

(make-reader-fn _datatype advertised-datatype n-elems read-fn)
view source

malloc

(malloc n-bytes dtype-or-opts)(malloc n-bytes)

Malloc memory. If a desired buffer type is needed follow up with set-native-datatype.

+
    +
  • n-bytes - the number of bytes to malloc
    • +
  • opts - Either a keyword datatype or an options map.
    • +

Options:

  • :resource-type - defaults to :gc - maps to :track-type in tech.v3.resource/track but can also be set to nil in which case the data is not tracked this library will not clean it up.
    • +
  • :datatype - Datatype - defaults to :int8. Datatype byte width must evenly divide n-bytes.
  • :uninitialized? - do not initialize to zero. Use for perf in very very rare cases.
  • :endianness - Either :little-endian or :big-endian - defaults to platform.
  • :log-level - one of #{:debug :trace :info :warn :error :fatal} or nil if no logging @@ -179,19 +184,19 @@ a compilation error if older versions of dtype-next are included and is the equivalent (in latest versions of dtype-next) to (make-list datatype n-elems).

    -
view source

reader-like?

(reader-like? argtype)

Returns true if this argument type is convertible to a reader.

-
view source

reader?

(reader? item)

True if this item is convertible to a read-only or read-write Buffer +

view source

reader-like?

(reader-like? argtype)

Returns true if this argument type is convertible to a reader.

+
view source

reader?

(reader? item)

True if this item is convertible to a read-only or read-write Buffer interface.

-
view source

reverse

(reverse item)

Reverse an sequence, range or reader.

+
view source

reverse

(reverse item)

Reverse an sequence, range or reader.

  • If range, returns a new range.
  • If sequence, uses clojure.core/reverse
  • If reader, returns a new reader that performs an in-place reverse.
-
view source

set-constant!

(set-constant! item offset length value)(set-constant! item offset value)(set-constant! item value)

Set a contiguous region of a buffer to a constant value

-
view source

set-datatype

(set-datatype item new-dtype)

Legacy method. Performs elemwise-cast.

-
view source

set-native-datatype

(set-native-datatype item datatype)

Set the datatype of a native buffer. n-elems will be recalculated.

-
view source

set-value!

(set-value! item idx value)

Set a value on an object via conversion to a writer. set-value can also take a tuple +

view source

set-constant!

(set-constant! item offset length value)(set-constant! item offset value)(set-constant! item value)

Set a contiguous region of a buffer to a constant value

+
view source

set-datatype

(set-datatype item new-dtype)

Legacy method. Performs elemwise-cast.

+
view source

set-native-datatype

(set-native-datatype item datatype)

Set the datatype of a native buffer. n-elems will be recalculated.

+
view source

set-value!

(set-value! item idx value)

Set a value on an object via conversion to a writer. set-value can also take a tuple in which case a select operation will be done on the tensor and value applied to the result. See also set-constant!.

Example:

@@ -237,11 +242,11 @@ [2.000 2.000 23.00] [3.000 3.000 26.00]]] -
view source

shape

(shape item)

Return a persistent vector of the shape of the object. Dimensions +

view source

shape

(shape item)

Return a persistent vector of the shape of the object. Dimensions are returned from least-rapidly-changing to most-rapidly-changing.

-
view source

sub-buffer

(sub-buffer item offset length)(sub-buffer item offset)

Create a sub-buffer (view) of the given buffer.

-
view source

unchecked-cast

(unchecked-cast value datatype)

Perform an unchecked cast of a value to specific datatype.

-
view source

vectorized-dispatch-1

(vectorized-dispatch-1 scalar-fn iterable-fn reader-fn options arg1)(vectorized-dispatch-1 scalar-fn iterable-fn reader-fn arg1)(vectorized-dispatch-1 scalar-fn reader-fn arg1)

Perform a vectorized dispatch meaning return a different object depending on the +

view source

sub-buffer

(sub-buffer item offset length)(sub-buffer item offset)

Create a sub-buffer (view) of the given buffer.

+
view source

unchecked-cast

(unchecked-cast value datatype)

Perform an unchecked cast of a value to specific datatype.

+
view source

vectorized-dispatch-1

(vectorized-dispatch-1 scalar-fn iterable-fn reader-fn options arg1)(vectorized-dispatch-1 scalar-fn iterable-fn reader-fn arg1)(vectorized-dispatch-1 scalar-fn reader-fn arg1)

Perform a vectorized dispatch meaning return a different object depending on the argument type of arg1. If arg1 is scalar, use scalar-fn. If arg1 is an iterable, use iterable-fn. Finally if arg1 is a reader, use reader-fn. The space the operation is to be performed in can be loosely set with :operation-space in the @@ -249,7 +254,7 @@ of the system) is a combination of :operation-space if it exists and the elemwise datatype of arg1. See tech.v3.datatype.casting/widest-datatype for more information.

-
view source

vectorized-dispatch-2

(vectorized-dispatch-2 scalar-fn iterable-fn reader-fn options arg1 arg2)(vectorized-dispatch-2 scalar-fn reader-fn arg1 arg2)

Perform a vectorized dispatch meaning return a different object depending on the +

view source

vectorized-dispatch-2

(vectorized-dispatch-2 scalar-fn iterable-fn reader-fn options arg1 arg2)(vectorized-dispatch-2 scalar-fn reader-fn arg1 arg2)

Perform a vectorized dispatch meaning return a different object depending on the argument types of arg1 and arg2. If arg1 and arg2 are both scalar, use scalar-fn. If neither argument is a reader, use iterable-fn. Finally if both arguments are readers, use reader-fn. The space the operation is to be performed in can be @@ -257,6 +262,6 @@ will be performed in (and the return type of the system) is a combination of :operation-space if it exists and the elemwise datatypes of arg1 and arg2. See tech.v3.datatype.casting/widest-datatype for more information.

-
view source

writer?

(writer? item)

True if this item is convertible to a Buffer interface that supports +

view source

writer?

(writer? item)

True if this item is convertible to a Buffer interface that supports writing.

-
view source
\ No newline at end of file +
view source
\ No newline at end of file diff --git a/docs/tech.v3.datatype.list.html b/docs/tech.v3.datatype.list.html index 02efa225..636be895 100644 --- a/docs/tech.v3.datatype.list.html +++ b/docs/tech.v3.datatype.list.html @@ -4,7 +4,7 @@ function gtag(){dataLayer.push(arguments);} gtag('js', new Date()); - gtag('config', 'G-95TVFC1FEB');

Project

Topics

Namespaces

Public Vars

tech.v3.datatype.list

empty-list

(empty-list datatype)

Make an empty list of a given datatype.

+ gtag('config', 'G-95TVFC1FEB');

Project

Topics

Namespaces

Public Vars

tech.v3.datatype.list

empty-list

(empty-list datatype)

Make an empty list of a given datatype.

view source

make-list

(make-list initial-container ptr)(make-list datatype)

Make a new primitive list out of a container and a ptr that indicates the current write position.

view source

wrap-container

(wrap-container container)

In-place wrap an existing container. Write ptr points to the end diff --git a/docs/tech.v3.datatype.mmap-writer.html b/docs/tech.v3.datatype.mmap-writer.html index fb4cc00b..cac17338 100644 --- a/docs/tech.v3.datatype.mmap-writer.html +++ b/docs/tech.v3.datatype.mmap-writer.html @@ -4,7 +4,7 @@ function gtag(){dataLayer.push(arguments);} gtag('js', new Date()); - gtag('config', 'G-95TVFC1FEB');

Project

Topics

Namespaces

Public Vars

tech.v3.datatype.mmap-writer

Provides the ability to efficiently write binary data to a file with an + gtag('config', 'G-95TVFC1FEB');

Project

Topics

Namespaces

Public Vars

tech.v3.datatype.mmap-writer

Provides the ability to efficiently write binary data to a file with an automatic conversion to a native-buffer using mmap.

Endianness can be provided and strings are automatically saved as UTF-8.

mmap-writer

(mmap-writer fpath {:keys [endianness resource-type mmap-file-options open-options file-channel delete-on-close?], :or {resource-type :auto, endianness (dtype-proto/platform-endianness), open-options :append}, :as _options})(mmap-writer fpath)

Create a new data writer that has a conversion to a mmaped native buffer upon diff --git a/docs/tech.v3.datatype.mmap.html b/docs/tech.v3.datatype.mmap.html index db8e4beb..877ed1ae 100644 --- a/docs/tech.v3.datatype.mmap.html +++ b/docs/tech.v3.datatype.mmap.html @@ -4,7 +4,7 @@ function gtag(){dataLayer.push(arguments);} gtag('js', new Date()); - gtag('config', 'G-95TVFC1FEB');

Project

Topics

Namespaces

Public Vars

tech.v3.datatype.mmap

jdk-major-version

(jdk-major-version)
view source

mmap-enabled?

(mmap-enabled?)
view source

mmap-file

(mmap-file fpath options)(mmap-file fpath)

Memory map a file returning a native buffer. fpath must resolve to a valid + gtag('config', 'G-95TVFC1FEB');

Project

Topics

Namespaces

Public Vars

tech.v3.datatype.mmap

jdk-major-version

(jdk-major-version)
view source

mmap-enabled?

(mmap-enabled?)
view source

mmap-file

(mmap-file fpath options)(mmap-file fpath)

Memory map a file returning a native buffer. fpath must resolve to a valid java.io.File. Options