Skip to content

Commit

Permalink
Merge pull request #13 from altenwald/feature/elixir
Browse files Browse the repository at this point in the history
Support for Elixir
  • Loading branch information
manuel-rubio authored Feb 23, 2018
2 parents cc07211 + c3952cb commit e9afa46
Show file tree
Hide file tree
Showing 11 changed files with 291 additions and 16 deletions.
125 changes: 119 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,18 +6,26 @@ DBI for Erlang
[![License: LGPL 2.1](https://img.shields.io/badge/License-GNU%20Lesser%20General%20Public%20License%20v2.1-blue.svg)](https://raw.githubusercontent.com/altenwald/dbi/master/COPYING)
[![Hex](https://img.shields.io/hexpm/v/dbi.svg)](https://hex.pm/packages/dbi)

Database Interface for Erlang. This is an abstract implementation to use the most common database libraries ([p1_mysql][1], [epgsql][2] and [esqlite][4], and others you want) to use with standard SQL in your programs and don't worry about if you need to change between the main databases in the market.
Database Interface for Erlang and Elixir. This is an abstract implementation to use the most common database libraries ([p1_mysql][1], [epgsql][2] and [esqlite][4], and others you want) to use with standard SQL in your programs and don't worry about if you need to change between the main databases in the market.

### Install (rebar3)

To use it, with rebar, you only need to add the dependency to the rebar.config file:

```erlang
{deps, [
{dbi, "0.1.0"}
{dbi, "0.2.0"}
]}
```

### Install (mix)

To use it, with mix, you only need to add the dependency to the mix.exs file:

```elixir
{:dbi, "~> 0.2.0"}
```

### Configuration

The configuration is made in the configuration file (`sys.config` or `app.config`) so, you can add a new block for config the database connection as follow:
Expand Down Expand Up @@ -49,21 +57,59 @@ The configuration is made in the configuration file (`sys.config` or `app.config

The available types in this moment are: `mysql`, `pgsql` and `sqlite`.

In case you're using Elixir, you can define the configuration for your project in this way:

```elixir
confg :dbi, mydatabase: [
type: :mysql,
host: 'localhost',
user: 'root',
pass: 'root',
database: 'mydatabase',
poolsize: 10
],
mylocaldb: [
type: :sqlite,
database: ':memory'
],
mystrongdb: [
type: :pgsql,
host: 'localhost',
user: 'root',
pass: 'root',
database: 'mystrongdb',
poolsize: 100
]
```

### Using DBI

To do a query:
To do a query (Erlang):

```erlang
{ok, Count, Rows} = dbi:do_query(mydatabase, "SELECT * FROM users", []),
```

Or with params:
Elixir:

```elixir
{:ok, count, rows} = DBI.do_query(:mydatabase, "SELECT * FROM users", [])
```

Or with params (Erlang):

```erlang
{ok, Count, Rows} = dbi:do_query(mydatabase,
"SELECT * FROM users WHERE id = $1", [12]),
```

Elixir:

```elixir
{:ok, count, rows} = DBI.do_query(:mydatabase,
"SELECT * FROM users WHERE id = $1", [12])
```

Rows has the format: `[{field1, field2, ..., fieldN}, ...]`

**IMPORTANT** the use of $1..$100 in the query is extracted from pgsql, in mysql and sqlite is converted to the `?` syntax so, if you write this query:
Expand All @@ -73,19 +119,34 @@ Rows has the format: `[{field1, field2, ..., fieldN}, ...]`
"UPDATE users SET name = $2 WHERE id = $1", [12, "Mike"]),
```

Elixir:

```elixir
{:ok, count, rows} = DBI.do_query(:mydatabase,
"UPDATE users SET name = $2 WHERE id = $1", [12, "Mike"])
```

That should works well in pgsql, but **NOT for mysql and NOT for sqlite**. For avoid this situations, the best to do is **always keep the order of the params**.

### Delayed or Queued queries

If you want to create a connection to send only commands like INSERT, UPDATE or DELETE but without saturate the database (and run out database connections in the pool) you can use `dbi_delayed`:
If you want to create a connection to send only commands like INSERT, UPDATE or DELETE but without saturate the database (and run out database connections in the pool) you can use `dbi_delayed` (Erlang):

```erlang
{ok, PID} = dbi_delayed:start_link(delay_myconn, myconn),
dbi_delayed:do_query(delay_myconn,
"INSERT INTO my tab VALUES ($1, $2)", [N1, N2]),
```

This use only one connection from the pool `myconn`, when the query ends then `dbi_delayed` gets another query to run from the queue. You get statistics about the progress and the queue size:
Elixir:

```elixir
{:ok, pid} = DBI.Delayed.start_link(:delay_myconn, :myconn)
DBI.Delayed.do_query(:delay_myconn,
"INSERT INTO my tab VALUES ($1, $2)", [n1, n2])
```

This use only one connection from the pool `myconn`, when the query ends then `dbi_delayed` gets another query to run from the queue. You get statistics about the progress and the queue size (Erlang):

```erlang
dbi_delayed:stats(delay_myconn).
Expand All @@ -96,6 +157,17 @@ dbi_delayed:stats(delay_myconn).
]
```

Elixir:

```elixir
DBI.Delayed.stats(:delay_myconn)
[
size: 0,
query_error: 0,
query_ok: 1
]
```

The delayed can be added to the configuration:

```erlang
Expand All @@ -112,6 +184,20 @@ The delayed can be added to the configuration:
]}
```

Elixir:

```elixir
config :dbi, mydatabase: [
type: :mysql,
host: 'localhost',
user: 'root',
pass: 'root',
database: 'mydatabase',
poolsize: 10,
delayed: :delay_myconn
]
```

### Cache queries

Another thing you can do is use a cache for SQL queries. The cache store the SQL as `key` and the result as `value` and keep the values for the time you specify in the configuration file:
Expand All @@ -130,6 +216,20 @@ Another thing you can do is use a cache for SQL queries. The cache store the SQL
]}
```

Elixir:

```elixir
config :dbi, mydatabase: [
type: :mysql,
host: 'localhost',
user: 'root',
pass: 'root',
database: 'mydatabase',
poolsize: 10,
cache: 5
]
```

The cache param is in seconds. The ideal time to keep the cache values depends on the size of your tables, the data to store in the cache and how frequent are the changes in that data. For avoid flood and other issues due to fast queries or a lot of queries in little time you can use 5 or 10 seconds. To store the information about constants or other data without frequent changes you can use 3600 (one hour) or more time.

To use the cache you should to use the following function from `dbi_cache`:
Expand All @@ -138,13 +238,26 @@ To use the cache you should to use the following function from `dbi_cache`:
dbi_cache:do_query(mydatabase, "SELECT items FROM table"),
```

Elixir:

```elixir
DBI.Cache.do_query(:mydatabase, "SELECT items FROM table")
```

You can use `do_query/2` or `do_query/3` if you want to use params. And if you want to use a specific TTL (time-to-live) for your query, you can use `do_query/4`:

```erlang
dbi_cache:do_query(mydatabase,
"SELECT items FROM table", [], 3600),
```

Elixir:

```elixir
DBI.Cache.do_query(:mydatabase,
"SELECT items FROM table", [], 3600)
```

Enjoy!

[1]: https://github.com/processone/p1_mysql
Expand Down
33 changes: 33 additions & 0 deletions lib/dbi.ex
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
defmodule DBI do
use Application

# wrap from dbi_app.erl
def start(type, args), do: :dbi_app.start(type, args)
def stop(modules), do: :dbi_app.stop(modules)

# wrap from dbi.erl
def start(), do: :dbi.start()
def do_query(pool, query), do: :dbi.do_query(pool, query)
def do_query(pool, query, args), do: :dbi.do_query(pool, query, args)
def connect(type, host, port, user, pass, database, poolname) do
:dbi.connect(type, host, port, user, pass, database, poolname)
end
def connect(type, host, port, user, pass, db, poolname, poolsize, extra) do
:dbi.connect(type, host, port, user, pass, db, poolname, poolsize, extra)
end

defmodule Cache do
def do_query(ref, query), do: :dbi_cache.do_query(ref, query)
def do_query(ref, query, args), do: :dbi_cache.do_query(ref, query, args)
def do_query(ref, query, args, ttl) do
:dbi_cache.do_query(ref, query, args, ttl)
end
end

defmodule Delayed do
def start_link(ref, conn), do: :dbi_delayed.start_link(ref, conn)
def do_query(ref, query, args), do: :dbi_delayed.do_query(ref, query, args)
def do_query(ref, query), do: :dbi_delayed.do_query(ref, query)
def stats(ref), do: :dbi_delayed.stats(ref)
end
end
67 changes: 67 additions & 0 deletions mix.exs
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
defmodule DBI.Mixfile do
use Mix.Project

def project do
[app: :dbi,
version: get_version(),
name: "DBI",
description: "DataBase Interface for Erlang",
package: package(),
source_url: "https://github.com/altenwald/dbi",
elixir: "~> 1.3",
compilers: Mix.compilers,
build_embedded: Mix.env == :prod,
start_permanent: Mix.env == :prod,
deps: deps()]
end

def application do
env = if Mix.env == :test do
[testdb1: [type: :sqlite,
database: ':memory:'],
testdb2: [type: :sqlite,
database: ':memory:',
cache: 3],
testdb3: [type: :sqlite,
database: ':memory:',
delayed: :mydelayed]
]
else
[]
end
[applications: [:crypto, :public_key, :asn1, :ssl],
mod: {DBI, []},
env: env]
end

defp deps do
[{:epgsql, "~> 3.4.0"},
{:p1_mysql, "~> 1.0.4"},
{:esqlite, "~> 0.2.3"},
{:cache, "~> 2.2.0"},
{:poolboy, "~> 1.5.1"}]
end

defp package do
[files: ["lib", "mix.exs", "README*", "LICENSE*"],
maintainers: ["Manuel Rubio"],
licenses: ["LGPL 2.1"],
links: %{"GitHub" => "https://github.com/altenwald/dbi"}]
end

defp get_version do
retrieve_version_from_git()
|> String.split("-")
|> case do
[tag] -> tag
[tag, _num_commits, commit] -> "#{tag}-#{commit}"
end
end

defp retrieve_version_from_git do
System.cmd("git", ["describe", "--always", "--tags"])
|> Tuple.to_list
|> List.first
|> String.strip
end
end
5 changes: 5 additions & 0 deletions mix.lock
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
%{"cache": {:hex, :cache, "2.2.0", "3c11dbf4cd8fcd5787c95a5fb2a04038e3729cfca0386016eea8c953ab48a5ab", [:rebar3], [], "hexpm"},
"epgsql": {:hex, :epgsql, "3.4.0", "39d473b83d74329a88f8fb1f99ad24c7a2329fd753957c488808cb7c0fc8164e", [:rebar3], [], "hexpm"},
"esqlite": {:hex, :esqlite, "0.2.3", "1a8b60877fdd3d50a8a84b342db04032c0231cc27ecff4ddd0d934485d4c0cd5", [:rebar3], [], "hexpm"},
"p1_mysql": {:hex, :p1_mysql, "1.0.4", "7b9d7957a9d031813a0e6bcea5a7f5e91b54db805a92709a445cf75cf934bc1d", [:rebar3], [], "hexpm"},
"poolboy": {:hex, :poolboy, "1.5.1", "6b46163901cfd0a1b43d692657ed9d7e599853b3b21b95ae5ae0a777cf9b6ca8", [:rebar], [], "hexpm"}}
3 changes: 1 addition & 2 deletions src/dbi.erl
Original file line number Diff line number Diff line change
Expand Up @@ -67,8 +67,7 @@ connect(Type, Host, Port, User, Pass, Database, Poolname, Poolsize, Extra) ->
{database, Database}, {poolsize, Poolsize} | Extra
],
application:set_env(dbi, Poolname, DBConf),
Module:init(Host, Port, User, Pass, Database, Poolname, Poolsize, Extra),
Module:run().
Module:init(Host, Port, User, Pass, Database, Poolname, Poolsize, Extra).

-spec connect( Type::atom(),
Host :: string(), Port :: integer(), User :: string(),
Expand Down
6 changes: 3 additions & 3 deletions src/dbi_mysql.erl
Original file line number Diff line number Diff line change
Expand Up @@ -29,10 +29,10 @@ start_link(ConnData) ->
init(Host, Port, User, Pass, Database, Poolname, Poolsize, Extra) ->
application:start(p1_mysql),
MaxOverflow = proplists:get_value(max_overflow, Extra, ?DEFAULT_MAX_OVERFLOW),
ConnData = [Host, dbi_utils:default(Port, ?DEFAULT_PORT),
ConnData = [Host, dbi_query:default(Port, ?DEFAULT_PORT),
User, Pass, Database, undefined],
PoolArgs = [{name, {local, Poolname}}, {worker_module, ?MODULE},
{size, dbi_utils:default(Poolsize, ?DEFAULT_POOLSIZE)},
{size, dbi_query:default(Poolsize, ?DEFAULT_POOLSIZE)},
{max_overflow, MaxOverflow}],
ChildSpec = poolboy:child_spec(Poolname, PoolArgs, ConnData),
supervisor:start_child(?DBI_SUP, ChildSpec),
Expand All @@ -53,7 +53,7 @@ do_query(PoolDB, SQL, Params) when is_list(SQL) ->
do_query(PoolDB, list_to_binary(SQL), Params);

do_query(PoolDB, RawSQL, Params) when is_binary(RawSQL) ->
SQL = dbi_utils:resolve(RawSQL),
SQL = dbi_query:resolve(RawSQL),
poolboy:transaction(PoolDB, fun(PID) ->
case p1_mysql_conn:squery(PID, SQL, self(), Params) of
{data, Result} ->
Expand Down
4 changes: 2 additions & 2 deletions src/dbi_pgsql.erl
Original file line number Diff line number Diff line change
Expand Up @@ -31,9 +31,9 @@ init(Host, Port, User, Pass, Database, Poolname, Poolsize, Extra) ->
MaxOverflow = proplists:get_value(max_overflow, Extra, ?DEFAULT_MAX_OVERFLOW),
DataConn = [Host, User, Pass,
[{database, Database},
{port, dbi_utils:default(Port, ?DEFAULT_PORT)}] ++ Extra],
{port, dbi_query:default(Port, ?DEFAULT_PORT)}] ++ Extra],
PoolArgs = [{name, {local, Poolname}}, {worker_module, ?MODULE},
{size, dbi_utils:default(Poolsize, ?DEFAULT_POOLSIZE)},
{size, dbi_query:default(Poolsize, ?DEFAULT_POOLSIZE)},
{max_overflow, MaxOverflow}],
ChildSpec = poolboy:child_spec(Poolname, PoolArgs, DataConn),
supervisor:start_child(?DBI_SUP, ChildSpec),
Expand Down
2 changes: 1 addition & 1 deletion src/dbi_utils.erl → src/dbi_query.erl
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
-module(dbi_utils).
-module(dbi_query).
-author('[email protected]').

-export([
Expand Down
4 changes: 2 additions & 2 deletions src/dbi_sqlite.erl
Original file line number Diff line number Diff line change
Expand Up @@ -35,9 +35,9 @@ do_query(PoolDB, SQL, Params) when is_list(SQL) ->
do_query(PoolDB, list_to_binary(SQL), Params);

do_query(PoolDB, RawSQL, Params) when is_binary(RawSQL) ->
SQL = dbi_utils:resolve(RawSQL),
SQL = dbi_query:resolve(RawSQL),
{ok, Conn} = dbi_sqlite_server:get_database(PoolDB),
case dbi_utils:sql_type(SQL) of
case dbi_query:sql_type(SQL) of
dql ->
case catch esqlite3:q(SQL, Params, Conn) of
Rows when is_list(Rows) -> {ok, length(Rows), Rows};
Expand Down
Loading

0 comments on commit e9afa46

Please sign in to comment.