A Lispy wrapper for Erlang Mnesia and QLC
This tutorial is adapted (massively copied) from the LFE Mnesia Tutorial, which in turn was borrowed from the Erlang/OTP Mnesia Book on the Erlang docs site.
Mnesia is a distributed Database Management System, appropriate for telecommunications applications and other Erlang applications which require continuous operation and soft real-time properties.
Mnesia is designed with requirements like the following in mind:
Due to its focus on telecommunications applications, Mnesia combines many concepts found in traditional databases, such as transactions and queries with concepts such as very fast real-time operations, configurable degree of fault tolerance (by means of replication) and the ability to reconfigure the system without stopping or suspending it.
Mnesia is also interesting due to its tight coupling to the programming language Erlang/LFE, thus almost turning Erlang/LFE into a database programming language.
The following add-ons can be used in conjunction with Mnesia to produce specialized functions which enhance the operational ability of Mnesia:
Use Mnesia with the following types of applications:
On the other hand, Mnesia may not be appropriate with the following types of applications:
In order to work through this tutorial, you will need the following:
git
make
rebar3
The real differences are the modules and the additional utility functions. Module changes from the mnesia
and qlc
Erlang modules are as follows:
mns
- holds alll the mnesia
functions besides the dirty onesmns-drty
- split dirty functions into their own module (“dirty” operations are short-cuts that bypass much of the processing and increase the speed of the transaction)mns-qry
- qlc
alias; longer to type than qlc
, but provides some nice visual context when scanning codeThe utility functions that have been added are as follows:
TBD
We use styled call-outs to provide immediate visual cues about the nature of the information being shared.
These are as follows:
Download the project source code:
$ git clone https://github.com/lfex/moneta
$ cd moneta
Next, get all the dependencies downloaded and built:
$ make
The moneta project offers some convenience wrappers for the Erlang/OTP mnesia
and qlc
modules. To walk through the tutorial, you’ll need to download the project source code.
Once you have the code built, you’re ready to go :-) Now you can start the LFE REPL, letting it know where to save your data.
Start LFE and let it know where you data will be saved:
$ lfe -mnesia dir '"/tmp/funky"'
Once in the REPL, you’re ready to start dataing:
Erlang/OTP 17 [erts-6.2] [source] [64-bit] [smp:4:4] ...
..-~.~_~--..
( \\ ) | A Lisp-2+ on the Erlang VM
|`-.._/_\\_.-'; | Type (help) for usage info.
| g (_ \ |
| n | | | Docs: http://docs.lfe.io/
( a / / | Source: http://github.com/rvirding/lfe
\ l (_/ |
\ r / | LFE v0.11.0-dev (abort with G^)
`-E___.-'
>
Create a schema, asking Moneta to start the Mnesia database for you automatically:
> (mnt:create-schema #(start true))
ok
Now create a defaul table (with no provided table definition):
> (mnt:create-table 'funky)
#(atomic ok)
>
From the LFE REPL you can create a new database on disk, start Mnesia up, and then create a table. After creating the table, you can examine the table’s metadata.
We can take a look at our creation with this command:
> (mnt:info)
Which should give you something like this:
---> Processes holding locks <---
---> Processes waiting for locks <---
---> Participant transactions <---
---> Coordinator transactions <---
---> Uncertain transactions <---
---> Active tables <---
funky : with 0 records occupying 317 words of mem
schema : with 2 records occupying 541 words of mem
===> System info in version "4.8", debug level = none <===
opt_disc. Directory "/tmp/funky" is used.
use fallback at restart = false
running db nodes = [nonode@nohost]
stopped db nodes = []
master node tables = []
remote = []
ram_copies = [funky]
disc_copies = [schema]
disc_only_copies = []
[{nonode@nohost,disc_copies}] = [schema]
[{nonode@nohost,ram_copies}] = [funky]
3 transactions committed, 0 aborted, 0 restarted, 2 logged to disc
0 held locks, 0 in queue; 0 local transactions, 0 remote
0 transactions waits for other nodes: []
ok
You can quit the REPL now, as we’ll restart it in the next section.
Restart the LFE REPL using a new data directory:
$ lfe -mnesia dir '"./Company.DB"'
Create a default database schema:
> (mnt:create-schema #(start true))
ok
We’ve had a quick taste of Mnesia, and what some of the calls look like in LFE. Next we’re going to tackle a bit more heady stuff: tables and relationships.
After you have quit from your previous LFE REPL, restart using the Company.DB
database name and then create a default schema, passing the auto-start option to Mnesia.
The following records are defined in
examples/tables.lfe
:
(defrecord employee
id
name
salary
gender
phone
room-number)
(defrecord department
id
name)
(defrecord project
name
number)
(defrecord manager
employee-id
department-id)
(defrecord in-department
employee-id
department-id)
(defrecord in-project
employee-id
project-name)
Pull in these table definitions:
> (include-file "examples/tables.lfe")
loaded-example-tables
Define your tables:
> (set set-tables '(employee department project in-department))
(employee department project in-department)
> (set bag-tables '(manager in-project))
(manager in-project)
The tables.lfe
example include defines LFE records that act as our table definitions (and thus all the convenient record macros that come with those). These are only definitions, though – representing a table schema – not the actual tables themselves. We need to create those.
as well as a macro that lets us create Mnesia tables with almost no boilerplate.
These records (tables) are taken from the example given in the Erlang Mnesia tutorial which also gives this entity diagram for their proposed “Company” database:
Create the tables with the appropriate table specs:
> (mnt:create-tables set-tables '(#(type set)))
(#(atomic ok) #(atomic ok) #(atomic ok) #(atomic ok))
> (mnt:create-tables bag-tables '(#(type bag)))
(#(atomic ok) #(atomic ok))
This just created all our Mnesia tables for us. If we run it again, we’ll see errors indicating that the tables have already been created:
> (mnt:create-tables set-tables '(#(type set)))
(#(aborted #(already_exists employee))
#(aborted #(already_exists department))
#(aborted #(already_exists project))
#(aborted #(already_exists in-department)))
> (mnt:create-tables bag-tables '(#(type bag)))
(#(aborted #(already_exists manager))
#(aborted #(already_exists in-project)))
When using Mnesia directly, there is a great deal of boilerplate code that developers need to write in order to create tables. Fortunately, Moneta provides several macros and functions that does this for you, making table-creation as intuitive as possible: all you need to do is provide table names and table specs.
Next, let’s re-run the
info
function we saw in the previous section:
> (mnt:info)
...
---> Active tables <---
in-project : with 0 records occupying 305 words of mem
in-department : with 0 records occupying 305 words of mem
manager : with 0 records occupying 305 words of mem
project : with 0 records occupying 305 words of mem
department : with 0 records occupying 305 words of mem
employee : with 0 records occupying 305 words of mem
...
Here’s how you find what backend type is being used for any given table:
> (mnt:table-info 'employee 'type)
set
> (mnt:table-info 'in-project 'type)
bag
>
You can also get table metadata for several tables at once:
> (mnt:tables-info (++ set-tables bag-tables) 'type)
(set set set set bag bag)
If you’re interested in seeing all the details of any given table, you can do so with the
'all
parameter:
> (mnt:table-info 'employee 'all)
(#(access_mode read_write)
#(active_replicas (nonode@nohost))
#(all_nodes (nonode@nohost))
#(arity 7)
#(attributes (id name salary gender phone room-number))
#(checkpoints ())
#(commit_work ())
#(cookie #(#(1396 680215 616649) nonode@nohost))
#(cstruct
#(cstruct
employee
set
(nonode@nohost)
()
()
0
read_write
false
()
()
false
employee
(id name salary gender ...)
()
()
()
#(...)...))
#(disc_copies ())
#(disc_only_copies ())
#(frag_properties ())
#(index ())
#(load_by_force false)
#(load_node nonode@nohost)
#(load_order 0)
#(load_reason #(dumper create_table))
#(local_content false)
#(majority false)
#(master_nodes ())
#(memory 317)
#(ram_copies (nonode@nohost))
#(record_name employee)
#(record_validation #(employee 7 set))
#(type set)
#(size 0)
#(snmp ())
#(storage_properties ...)
#(...)...)
>
The output of the info
function will be very similar to what we saw in the
previous section. However, do note that our new tables are reported in the
“Active tables” section:
If you would like to check up on the tables created above, you can use the
table-info
function to pull out certain data.
Next up, we’ll start inserting some data.
Documentation is available for all previous releases:
Copyright © 2016 Duncan McGreggor
Distributed under the Apache License, Version 2.0.