Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: query structured #89

Merged
merged 45 commits into from
Sep 19, 2023
Merged
Show file tree
Hide file tree
Changes from 44 commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
caeb221
feat: queryPoints
Sciator Aug 29, 2023
9efdefb
fix: construct Point with relevant types
Sciator Aug 29, 2023
a67fe15
style: linted
Sciator Aug 29, 2023
14bf406
feat: Point field types, getField
Sciator Aug 29, 2023
71a2a9b
feat: Point.fields, PointRecord removed
Sciator Aug 29, 2023
004f75d
docs: readme for queryPoints
Sciator Aug 29, 2023
07d2ef0
test: structured query tests
Sciator Aug 29, 2023
be12c32
fix: getField, queryPoints
Sciator Aug 29, 2023
72c8535
docs: fix README lint, env variables
Sciator Aug 29, 2023
9795860
test: InfluxDBClient argument options
Sciator Aug 29, 2023
c649740
docs: changelog
Sciator Aug 29, 2023
464941f
fix: md linter disable MD024
Sciator Aug 29, 2023
862abcb
fix: query Points without metadata
Sciator Aug 29, 2023
3d6ef09
feat: PointValues object (first iteration)
Sciator Sep 5, 2023
763fea8
refactor: PointValues query, private Point cons
Sciator Sep 5, 2023
3878460
refactor: reworked Point api
Sciator Sep 6, 2023
e9f6817
fix: renaming error
Sciator Sep 8, 2023
2e95774
docs: Point values comments
Sciator Sep 8, 2023
c8f3a18
refactor: rename int to integer
Sciator Sep 12, 2023
52a608c
feat: fromValues throws when measurement missing
Sciator Sep 12, 2023
6daa3b6
docs: Point comments
Sciator Sep 12, 2023
f368bf4
feat: downsampling example
Sciator Sep 12, 2023
8238186
fix: minor fixes
Sciator Sep 12, 2023
f55cb96
docs: use set instead of add
Sciator Sep 13, 2023
138ff06
docs: fix
Sciator Sep 13, 2023
5e1f775
fix: apidoc
Sciator Sep 14, 2023
d42c665
docs: correct name for type
bednar Sep 18, 2023
175164a
docs: add documentation
bednar Sep 18, 2023
a89ba34
fix: correct field name
bednar Sep 18, 2023
1dbeeb9
chore: test converting point values to point
bednar Sep 18, 2023
7559f37
fix: code style
bednar Sep 18, 2023
802ad9f
chore: test point values
bednar Sep 18, 2023
9d28e69
chore: test point values
bednar Sep 18, 2023
b05ab2b
chore: test point values
bednar Sep 18, 2023
29ed883
chore: test point values
bednar Sep 18, 2023
095a51c
chore: simplify queryPoints
bednar Sep 19, 2023
04b97fe
fix: tests
bednar Sep 19, 2023
dc6ab3e
chore: test point
bednar Sep 19, 2023
4522368
chore: test point
bednar Sep 19, 2023
6fec014
chore: test aggregation
bednar Sep 19, 2023
47b6e77
chore: test aggregation
bednar Sep 19, 2023
a6c6655
feat: use separate directory for downsampling data
bednar Sep 19, 2023
2152181
feat: use separate directory for downsampling data
bednar Sep 19, 2023
57217da
feat: use separate directory for downsampling data
bednar Sep 19, 2023
37399b3
feat: use separate directory for downsampling data
bednar Sep 19, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .markdownlint.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
{
"MD013": false,
"MD024": false,
"MD033": {
"allowed_elements": [ "a", "img", "p", "details", "summary" ]
},
Expand Down
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
## 0.3.0 [unreleased]

### Features

1. [#89](https://github.com/InfluxCommunity/influxdb3-js/pull/89): Add structured query support

## 0.2.0 [2023-08-11]

### Features
Expand Down
33 changes: 25 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,18 +65,18 @@ export INFLUXDB_TOKEN="<token>"

### powershell

```console
set INFLUXDB_URL=<url>
set INFLUXDB_DATABASE=<database>
set INFLUXDB_TOKEN=<token>
```powershell
$env:INFLUXDB_URL = "<url>"
$env:INFLUXDB_DATABASE = "<database>"
$env:INFLUXDB_TOKEN = "<token>"
```

### cmd

```powershell
$env:INFLUXDB_URL "<url>"
$env:INFLUXDB_DATABASE "<database>"
$env:INFLUXDB_TOKEN "<token>"
```console
set INFLUXDB_URL=<url>
set INFLUXDB_DATABASE=<database>
set INFLUXDB_TOKEN=<token>
```

</details>
Expand Down Expand Up @@ -138,6 +138,23 @@ for await (const row of queryResult) {
}
```

or use typesafe `PointValues` structure with `client.queryPoints`

```ts
const queryPointsResult = client.queryPoints(
query,
database,
queryType,
'stat'
)

for await (const row of queryPointsResult) {
console.log(`avg is ${row.getField('avg', 'float')}`)
console.log(`max is ${row.getField('max', 'float')}`)
console.log(`lp: ${row.toLineProtocol()}`)
}
```

## Examples

For more advanced usage, see [examples](https://github.com/InfluxCommunity/influxdb3-js/blob/HEAD/examples/README.md).
Expand Down
3 changes: 1 addition & 2 deletions examples/README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
## Examples

### Basic

- [IOxExample](https://github.com/InfluxCommunity/influxdb3-js/blob/HEAD/examples/basic/README.md) - How to use write and query data from InfluxDB IOx
- [Downsampling](https://github.com/InfluxCommunity/influxdb3-js/blob/HEAD/downsampling/basic/README.md) - How to use queries to structure data for downsampling

### Browser

Expand Down
2 changes: 1 addition & 1 deletion examples/basic/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ set environment variables.
- `INFLUXDB_TOKEN` read/write token generated in cloud
- `INFLUXDB_DATABASE` name of database e.g .*`my-database`*

For simplicity you can use dotenv library to load environment variables in this example. Create `.env` file and paste your variables as follows:
For simplicity, you can use dotenv library to load environment variables in this example. Create `.env` file and paste your variables as follows:

```conf
INFLUXDB_URL="<url>"
Expand Down
54 changes: 34 additions & 20 deletions examples/basic/src/index.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import {InfluxDBClient, Point, PointRecord} from '@influxdata/influxdb3-client'
import {InfluxDBClient, Point} from '@influxdata/influxdb3-client'

type Defined<T> = Exclude<T, undefined>

Expand All @@ -24,30 +24,32 @@ async function main() {

try {
// Write point
const p = new Point('stat')
.tag('unit', 'temperature')
.floatField('avg', 24.5)
.floatField('max', 45.0)
.timestamp(new Date())
const p = Point.measurement('stat')
.setTag('unit', 'temperature')
.setFloatField('avg', 24.5)
.setFloatField('max', 45.0)
.setTimestamp(new Date())
await client.write(p, database)

// Write record
const sensorData: PointRecord = {
measurement: 'stat',
tags: {
unit: 'temperature',
},
fields: {
avg: 28,
max: 40.3,
},
timestamp: new Date(),
// Write point as template with anonymous fields object
const pointTemplate = Object.freeze(
Point.measurement('stat').setTag('unit', 'temperature')
)

const sensorData = {
avg: 28,
max: 40.3,
}
await client.write([sensorData], database)
const p2 = pointTemplate
.copy()
.setFields(sensorData)
.setTimestamp(new Date())

await client.write(p2, database)

// Or write directly line protocol
const line = `stat,unit=temperature avg=20.5,max=43.0`
await client.write(line, database)
const lp = `stat,unit=temperature avg=20.5,max=43.0`
await client.write(lp, database)

// Prepare flightsql query
const query = `
Expand All @@ -67,6 +69,18 @@ async function main() {
console.log(`avg is ${row.avg}`)
console.log(`max is ${row.max}`)
}

// Execute query again as points
const queryPointsResult = client.queryPoints(
query,
database,
queryType
)

for await (const row of queryPointsResult) {
console.log(`avg is ${row.getField('avg', 'float')}`)
console.log(`max is ${row.getField('max', 'float')}`)
}
} catch (err) {
console.error(err)
} finally {
Expand Down
16 changes: 8 additions & 8 deletions examples/browser/src/main.ts
Original file line number Diff line number Diff line change
Expand Up @@ -56,14 +56,14 @@ view.setOnRandomize(() => {

view.setOnWrite(async () => {
const data = view.getWriteInput()
const p = new Point('stat')
.tag('Device', data['Device'])
.floatField('Temperature', data['Temperature'])
.floatField('Humidity', data['Humidity'])
.floatField('Pressure', data['Pressure'])
.intField('CO2', data['CO2'])
.intField('TVOC', data['TVOC'])
.timestamp(new Date())
const p = Point.measurement('stat')
.setTag('Device', data['Device'])
.setFloatField('Temperature', data['Temperature'])
.setFloatField('Humidity', data['Humidity'])
.setFloatField('Pressure', data['Pressure'])
.setIntField('CO2', data['CO2'])
.setIntField('TVOC', data['TVOC'])
.setTimestamp(new Date())

try {
view.setWriteInfo('writing')
Expand Down
32 changes: 32 additions & 0 deletions examples/downsampling/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
## Downsampling Example

- [index.ts](./src/index.ts) - How to use queries to structure data for downsampling

## prerequisites

- `node` and `yarn` installed

- build influxdb-client: *(in project root directory)*
- run `yarn install`
- run `yarn build`

## Usage

set environment variables.

- `INFLUXDB_URL` region of your influxdb cloud e.g. *`https://us-east-1-1.aws.cloud2.influxdata.com/`*
- `INFLUXDB_TOKEN` read/write token generated in cloud
- `INFLUXDB_DATABASE` name of database e.g .*`my-database`*

For simplicity, you can use dotenv library to load environment variables in this example. Create `.env` file and paste your variables as follows:

```conf
INFLUXDB_URL="<url>"
INFLUXDB_DATABASE="<database>"
INFLUXDB_TOKEN="<token>"
```

### Run example

- run `yarn install`
- run `yarn dev`
17 changes: 17 additions & 0 deletions examples/downsampling/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
{
"name": "influxdb3-client-example-downsampling",
"main": "index.js",
"license": "MIT",
"private": true,
"scripts": {
"dev": "ts-node -r dotenv/config ./src/index.ts"
},
"dependencies": {
"@influxdata/influxdb3-client": "link:../../packages/client"
},
"devDependencies": {
"dotenv": "^16.3.1",
"ts-node": "^10.9.1",
"typescript": "^5.1.3"
}
}
84 changes: 84 additions & 0 deletions examples/downsampling/src/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
import {InfluxDBClient} from '@influxdata/influxdb3-client'

/* get environment value or throw error if missing */
const getEnv = (variableName: string): string => {
if (process.env[variableName] == null)
throw new Error(`missing ${variableName} environment variable`)
return process.env[variableName] as string
}

/* eslint-disable no-console */
async function main() {
//
// Use environment variables to initialize client
//
const host = getEnv('INFLUXDB_URL')
const token = getEnv('INFLUXDB_TOKEN')
const database = getEnv('INFLUXDB_DATABASE')

//
// Create a new client using an InfluxDB server base URL and an authentication token
//
const client = new InfluxDBClient({host, token, database})

try {
//
// Write data
//
await client.write(`stat,unit=temperature avg=24.5,max=45.0`)

await new Promise((resolve) => setTimeout(resolve, 1000))
await client.write(`stat,unit=temperature avg=28,max=40.3`)

await new Promise((resolve) => setTimeout(resolve, 1000))
await client.write(`stat,unit=temperature avg=20.5,max=49.0`)

//
// Query downsampled data
//
const downSamplingQuery = `\
SELECT
date_bin('5 minutes', "time") as window_start,
AVG("avg") as avg,
MAX("max") as max
FROM "stat"
WHERE
"time" >= now() - interval '1 hour'
GROUP BY window_start
ORDER BY window_start ASC;`

//
// Execute downsampling query into pointValues
//
const queryPointsResult = client.queryPoints(
downSamplingQuery,
database,
'sql'
)

for await (const row of queryPointsResult) {
const timestamp = new Date(row.getFloatField('window_start') as number)
console.log(
`${timestamp.toISOString()}: avg is ${row.getField(
'avg',
'float'
)}, max is ${row.getField('max', 'float')}`
)

//
// write back downsampled date to 'stat_downsampled' measurement
//
const downSampledPoint = row
.asPoint('stat_downsampled')
.setTimestamp(timestamp)

await client.write(downSampledPoint, database)
}
} catch (err) {
console.error(err)
} finally {
await client.close()
}
}

main()
11 changes: 11 additions & 0 deletions examples/downsampling/tsconfig.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
{
"compilerOptions": {
"target": "es2018",
"lib": ["es2018"],
"strict": true,
"moduleResolution": "node",
"esModuleInterop": true
},
"include": ["src/**/*.ts"],
"exclude": ["*.js"]
}
Loading