Introduction

In pretty much every project you will be using Xill for, you will be making a variation on the well known ETL theme:

  • Extract: Obtain data from some source
  • Transform: Modify the obtained data to comply with the target system
  • Load: Enter the data into the target system
If you have already done a number of these projects you will recognise that it turns out to be extremely difficult to re-use things you have built for a previous project. Some utility functions can easily be copy-pasted, but the majority of functionality is strongly interwoven with the specific systems and client requirements of that project.

The Xillio Unified Data Model (UDM) consists of a datamodel and API, built on top of MongoDB.

The UDM solves common issues that naturally occur in our line of work by helping you in three different ways:

  1. Reusable: One unified way of storing documents guarantees that any connector or utility you make will work from one project to another;
  2. Fast: The underlying Mongo database is blazingly fast. It easily scales with millions of documents on consumer hardware;
  3. Robust: The API ensures that no data-corruption will occur.
More advanced users will discover that the development cycle speeds up significantly thanks to the reusable components and the structured approach the UDM enforces. Development goes faster and the resulting code is much better maintainable.

Because a picture says more than a thousand words, here's what working with the UDM looks like in a nutshell:

UDM flow

  1. Extract: 
    • Data is extracted from the source
    • The extracted data is converted to the UDM model using templates (this is also a form of transformation)
  2. Transform:
    • One or more transformation operations are performed directly on the data in the UDM
  3. Load: 
    • Data is converted from UDM to the datatype required for the target system using templates (this is also a form of transformation)
    • The converted data is imported

Decorators and Content types

Content type decorators

Time for some background information on how the UDM is structured. ContentTypes are essentially your every-day document types. For example "report", "newsletter" and "agenda" are all perfect examples of content types. Each content type is defined as a set of decorators. This means that one content type can (and usually will) have multiple. Different content types can still have the same decorators. For example all content types mentioned above could have a "workflow" decorator that describes the workflow state of the document. 

A decorator then is a group of properties of a document that share a specific theme and are typically always available. Some examples of decorators that you could apply for your project:

  • file: name, path, extension, createdBy, createdOn, modifiedBy, modifiedOn
  • workflow: currentState, assignedTo
  • news: publishDate, title, summary

The fundamental concept of decorators is that they must be reusable. Not just within your project, but also cross-project. By having a community-shared set of frequently used decorators it is possible to re-use any functionality that depends on specific decorators to be available. Before defining your own decorator (which of course you are free to do), always check if the decorator(s) you need are not already available in our decorator library!

Anxious to try it out? Head on with the Getting started with UDM