Containers

Data: Databases | Containers | Views

Overview

The data management of a core3 application is based around two main concepts: database abstraction layers and containers. In addition to that, there are views which can be used to further process and group data coming from the databases.

Note: Implicit parameters have been removed for brevity.

Example containers

All core containers can be found here.

Containers

Containers represent data going in and out of the data layer and, usually, each one maps to a distinct entity in one or more DBs (for example, table row in MariaDB or a document in CouchDB). Each container has at least an id and a containerType, used for identifying an object across all DBs.

//Base container trait
trait Container {
  val id: ObjectID
  val objectType: ContainerType
}

Immutable

Useful for storing data that does not change, such as logs or data snapshots.

trait ImmutableContainer extends Container

Mutable

Used for everything else and has the following additional pieces of information:

• created - Data and time of the object's creation
• updated - Date and time of the object's last update
• updatedBy - ID of the user that performed the last update
• revision - Unique ID used for ensuring data integrity
• revisionNumber - Sequential ID used for ensuring data integrity and tracking the number of changes to an object

trait MutableContainer extends Container {
  val created: Timestamp
  var updated: Timestamp
  var updatedBy: String
  var revision: RevisionID
  var revisionNumber: RevisionSequenceNumber
}

vars and vals in container definitions

It is generally not a good idea to use vars when defining a case class. The core containers, however, do use vars to distinguish between members that should be updatable and members that should not. The intention is for it be a reminder to the developer and no functionality depends on it (except for the members inherited with the MutableContainer trait).

import core3.database.containers.core.Group
import core3.database.dals.DatabaseAbstractionLayer

val db: DatabaseAbstractionLayer = ...
val group: Group = ...
group.name = "new name"

//compiler and logical error the itemsType must not change
group.itemsType = "SomeOtherContainerType"

db.updateObject(group)

It is, of course, possible to "update" the val members by using the copy method of the case class and sending the new object to the DB.

import core3.database.containers.core.Group
import core3.database.dals.DatabaseAbstractionLayer

val db: DatabaseAbstractionLayer = ...
val group: Group = ...

//logical error (the itemsType must not change) but not a compiler error
val newGroup = group.copy(itemsType = "SomeOtherContainerType")

db.updateObject(newGroup)

Container Definitions

A container definition is used for converting object to and from the format a DB expects.

BasicContainerDefinition

Defines the basic methods required from all companion objects:

trait BasicContainerDefinition extends ContainerDefinition {
  def getDatabaseName(dataType: DataType): String
  def matchCustomQuery(queryName: String, queryParams: Map[String, String], container: Container): Boolean
}

JsonContainerDefinition

Defines the methods required to convert a container object to/from JSON:

trait JsonContainerDefinition extends ContainerDefinition {
  def toJsonData(container: Container): JsValue
  def fromJsonData(data: JsValue): Container
}

SlickContainerDefinition

Defines the methods required to convert containers to/from Slick DBs:

trait SlickContainerDefinition extends ContainerDefinition {
  def createSchemaAction(): DBIOAction[Unit, NoStream, Effect.Schema]
  def dropSchemaAction(): DBIOAction[Unit, NoStream, Effect.Schema]
  def genericQueryAction: DBIOAction[Seq[Container], NoStream, Effect.Read]
  def getAction(objectID: ObjectID): DBIOAction[Seq[Container], NoStream, Effect.Read]
  def createAction(container: Container): DBIOAction[Int, NoStream, Effect.Write]
  def updateAction(container: MutableContainer): DBIOAction[Int, NoStream, Effect.Write]
  def deleteAction(objectID: ObjectID): DBIOAction[Int, NoStream, Effect.Write]
  def customQueryAction(queryName: String, queryParams: Map[String, String]): DBIOAction[Seq[Container], NoStream, Effect.Read]
}

SearchContainerDefinition

Defines the methods required to convert a container to a searchable entity:

trait SearchContainerDefinition extends ContainerDefinition {
  def getSearchFields: Map[String, String]
}

Note: Normally, a container cannot be reconstructed from search data.

Core containers

• Group - Allows grouping of other containers
• LocalUser - Basic user container; used for local authentication
• TransactionLog - Used for storing data about the operations the workflow engine is performing

Defining your own

1. Decide on whether the container is going to be mutable or not

2. Extend the appropriate trait (MutableContainer or ImmutableContainer) and define all container fields

import core3.database
import core3.database.containers._
import core3.database._
import core3.utils.Time._
import core3.utils._
import play.api.libs.json._

case class Group(
  shortName: String,
  var name: String,
  var items: Vector[ObjectID],
  itemsType: ContainerType,
  created: Timestamp,
  var updated: Timestamp,
  var updatedBy: String,
  id: ObjectID,
  var revision: RevisionID,
  var revisionNumber: RevisionSequenceNumber
)
  extends MutableContainer {
  override val objectType: ContainerType = "Group"

  //... additional methods and/or constructors ...

}

3. Decide on the container's target DBs and create the appropriate definitions:

import core3.database
import core3.database.containers._
import core3.database._
import core3.utils.Time._
import core3.utils._
import play.api.libs.json._

object Group {
  trait BasicDefinition extends BasicContainerDefinition {
    override def getDatabaseName: String = "core-groups"
  
    override def matchCustomQuery(
      queryName: String,
      queryParams: Map[String, String],
      container: Container):
    Boolean = ???
  }

  trait JsonDefinition extends JsonContainerDefinition {
    override def toJsonData(container: Container): JsValue = {
      Json.toJson(container.asInstanceOf[Group])
    }
  
    override def fromJsonData(data: JsValue): Container = {
      data.as[Group]
    }
  }
  
  trait SlickDefinition extends SlickContainerDefinition {
  
    import profile.api._
    import shapeless._
    import slickless._
  
    private class TableDef(tag: Tag) extends Table[Group](tag, "core_groups") {
      //... Slick table definition ...
    }
    
    //... trait methods ...
  }
}

4. Done

Using meta macro annotations

The macro annotations available in the core3.meta package can be used to generate almost all of the container definitions:

import core3.database._
import core3.database.containers._
import core3.utils._
import core3.meta.containers._
import core3.meta.enums._

@WithBasicContainerDefinition
@WithJsonContainerDefinition
@WithSlickContainerDefinition
case class Organization(
    var name: String,
    var description: String,
    organizationType: Organization.OrganizationType,
    created: Timestamp,
    var updated: Timestamp,
    var updatedBy: String,
    id: ObjectID,
    var revision: RevisionID,
    var revisionNumber: RevisionSequenceNumber
  ) extends MutableContainer {
  override val objectType: ContainerType = "Organization"
}

object Organization {
  @DatabaseEnum
  sealed trait OrganizationType
  object OrganizationType {
    case object External extends OrganizationType
    case object Internal extends OrganizationType
  }
}

This is all the code needed to generate a complete container that support JSON and Slick databases. For more information on customizing the auto-generated code, check the macro annotations page.