No-SQL databases are very popular nowadays. In that context, no-SQL object databases are becoming more and more of interest. They allow manipulating directly an object from and to a database without having to use complicated drivers.
In classical application development, to manipulate objects within databases, one must consider developing several layers:
An object repository specific to the object, usually allowing CRUD operations search
If this is a SQL database, the requests of the object repository must be translated into the SQL language. If this is a no-SQL database the object repository must translate the calls into the no-SQL API
A driver such as an odbc (C,C++,C#) or jdbc (java) driver, for instance, to send the SQL requests to the database, or for instance the MongoDB drivers.
To avoid such a hassle, it is possible to use Object databases.
Table of Contents
Overview of Object Databases
Object databases are not a brand new concept. They have been developed since the ’80s but their usage in the “general” development project is very recent.
There exist additionally “light” object databases for C# like NDatabase or velocityDB.
In what follows we will build a small, light, simple but efficient object database for C# – in fact, a data vault – which will fit any application which requires but moderate access to the database.
Our data vault will not be optimized but on many occasions, it will fit for application who do not need at all the sophisticated features of the SQL or no-SQL database engines.
Imagine an application for a private network of a hundred people like for example a group of lawyers or a small company. The application will make but only moderate use of data. The machine they will use will be equipped with modern and fast CPUs. Do they really need to gain 5 ms on the operations? Probably not. They need something secure and quickly functional.
Our vault must be able to store any C# object, to retrieve it, to modify it and to run queries in the vault to retrieve objects.
We will not implement ACID (Atomicity, Consistency, Isolation, Durability) in our vault. This is a demo project only aimed at demonstrating some of the features of the .NET language. Anyway, that object vault can be used with standard development projects and should perform relatively well.
We will also not implement a remote protocol. This will be a flat database. In fact, we will not even store the objects into a set of a file but inside a file structure that will directly represent the objects.
Our vault will be implemented as a secure vault. Ideally using a ciphering system so that each object is stored in a ciphered way, natively, by default.
The analogy can be done with safes in a bank, individually locked and well ordered.
A lock file will be maintained so that the object can be securely accessed.
When a vault client access an object, a lock is created which prevents other clients to modify it. When the lock is released, the object becomes available again for other vault clients.
There will not be “tables” but folders and each object type will have their own dedicated folders,
Our vault will never delete physically objects they will be “marked” as deleted.
At this stage, we will not develop the ability to create tables which could mix different types of objects with eventually a query language, but this could be done, without a lot of difficulties, as an extension of the project.
In fact, the object storage can delegate to the developers the task of maintaining tables of objects with indexes of object IDs and retrieving the objects using the vault. The same queries, like LINQ queries, can be done outside of the vault area.
The vault will have the functionalities of an “object storage” system.
Here are the operations that our object database will be able to perform:
Name
Description
deleteObject<T>(string, DataVaultKey)
delete an existing object of type T in the vault. No objects are deleted in the vault, they are ‘marked’ as deleted
getAllObjects<T>(DataVaultKey)
get ALL the objects of type T(should never be allowed)
getCount<T>()
Get total number of records for an object
getIndexFromVaultID<T>(string)
get the record number from a given vault ID
getObject<T>(int, int, DataVaultKey)
Returns a list of transactions as stored in the vault
getObject<T>(int, DataVaultKey)
get an object from a record number
getObjectFromVaultID<T>(string, DataVaultKey)
get an object from the vault using the vault id
insertObject<T>(T, DataVaultKey)
insert a new object of type T in the vault
updateObject<T>(T, DataVaultKey)
update an existing object of type T in the vault
The DataVaultKey is a structure containing credentials that allow access to the vault object.
Our objects will also have to implement a dataVaultStorage class so that they get a unique storage ID.
In order to store and manipulate the objects, we shall simply use a serializer. There are many ways it can be achieved. For this, we will implement a SerializerFactory.
The factory will produce a serializer for a given type T of the object with the following code:
public static XmlSerializer getSerializer<T>()
{
if(!dic.ContainsKey(typeof(T)))
dic[typeof(T)] = new XmlSerializer(typeof(T));
return dic[typeof(T)];
}
We use a dictionary containing well-known serializers for some types, as a basic memoization technique.
When started the datavault will create unless it already exists, a root directory which will be the base for the vault.
In this root directory, objects will be stored in dedicated folders. In each folder, an idx file will maintain a list of the objects.
For instance, if we store 100 objects from an Employee class.
DataVaultKey key = new DataVaultKey();
key.hashKey = "FFFF";
key.password = "1234";
key.user = "admin";
for (int i = 0; i < 100; i++)
{
Employee roger = new Employee();
roger.Age = 23+i%55;
roger.Name = "Roger"+i;
DataVault.insertObject<Employee>(roger, key);
}
We will get the following file structure:
An index file named “ idx” will be created and will store the correspondence between the objects and their storage ID.
Implementation
The following code will insert an object into the vault:
/// <summary>
/// insert a new object of type T in the vault
///
/// </summary>
/// <param name="new_Transaction">object to insert</param>
/// <param name="key">the vault credentials</param>
/// <returns>true if object inserted , false otherwise</returns>
public static bool insertObject<T>(T new_Transaction, DataVaultKey key) where T : dataVaultStorage
{
Contract.Assume(new_Transaction != null);
Dictionary<int, string> index = null;
//lock the index file
try
{
int cticks = 0;
while ((index == null) && (cticks < 100))
{
index = getIndexAndLock(typeof(T).Name);
System.Threading.Thread.Sleep(100);
cticks++;
}
if (index == null)
{
Tracer.Trace(Tracer.severity.WARNING, "cannot acquire lock on index file");
return false;
}
//insert the data
int c = index.Count;
int d = c + 1;
Guid id = Guid.NewGuid();
new_Transaction.vault_id = id;
string data = cipherObject<T>(new_Transaction, key);
File.Create(VAULT_PATH + "\\" + typeof(T).Name + "\\" + d + ".vault").Close();
File.WriteAllText(VAULT_PATH + "\\" + typeof(T).Name + "\\" + d + ".vault", data);
//increment the index
index.Add(d, id.ToString());
//unlock the index file
saveAndUnlockIndex(typeof(T).Name, index);
return true;
}
catch (Exception ex)
{
Tracer.Trace(Tracer.severity.ERROR, "Error while inserting Transactions \n\n" + ex.Message+"\n"+ex.InnerException);
//unlock the index file
saveAndUnlockIndex(typeof(T).Name, index);
}
return false;
}
When inserting an object into the vault, the client must first lock the index file by calling getIndexAndLock.
The lock is created by inserting a temporary file named idx_lock.
If the index is locked, the client will loop for a moment then will return an error with a failure to insert the object.
Since there is no reason for the application to monopolize access to the index file, this is acceptable behavior. Of course, a ‘real” implementation may use multithreading and mutexes for instance.
The function cipherObject will serialize the object and cipher it and finally convert it to base64 data.
Finally, once the insertion has been checked, the client will call the saveAndUnlockIndex function to release the index file.
We still need to develop the read and update operations.
To read an object in the vault we need to know its datavault serial. We can then convert the serial into a record number from the IDX file and get the corresponding object.
The following code returns a list of objects from a record number.
/// <summary>
///
/// Returns a list of objects as stored in the vault
/// </summary>
/// <param name="start"></param>
/// <param name="end"></param>
/// <returns></returns>
public static List<T> getObject<T>(int start, int end, DataVaultKey key) where T : dataVaultStorage
{
string table = typeof(T).Name;
if (start > end)
return null;
//check password
if (!checkCredentials(key.user, key.password))
{
Tracer.Trace(Tracer.severity.WARNING, "error using credentials " + key.user + "/" + key.password);
return null;
}
List<T> list = new List<T>();
T object_record;
string record = "";
string guid = "";
Dictionary<int, string> index = null;
// index = getIndex<T>();
index = getIndex(table);
//@start_contract
Contract.Assert(index != null);
//@end contract
for (int num = start; num < end + 1; num++)
{
try
{
// guid = index[num];
record = File.ReadAllText(VAULT_PATH + "\\" + table + "\\" + num + ".vault");
object_record = decipherRecord<T>(record, key);
list.Add(object_record);
}
catch (Exception ex)
{
Tracer.Trace(Tracer.severity.CRITICAL, "data vault corrupted, cannot read record " + num + "(" + guid + ")" + "\n\n\n" + ex.Message);
object_record = default (T);
object_record.info = "error reading the data";
list.Add(object_record);
}
}
return list;
}
And the following code returns an object from its vault ID:
/// <summary>
///
/// get an object from the vault using the vault id
/// </summary>
/// <typeparam name="T">type of the object to fetch</typeparam>
/// <param name="vaultid">vault id of the object to fetch</param>
/// <param name="key">key for vault access</param>
/// <returns>object to be fetched</returns>
public static T getObjectFromVaultID<T>(string vaultid, DataVaultKey key) where T : dataVaultStorage
{
int n = getIndexFromVaultID<T>(vaultid);
if (n == -1)
throw new Exception("Object not found Vaultid=" + vaultid);
return getObject<T>(n, key);
}
The update operation is simple. It consists of replacing the object with an updated copy. The delete operation consists simply of marking the object as deleted. Since all objects to be stored implements the dataVaultStorage class, they have a field for such deletion information.
Here we delegate the fact that the record is deleted to the developer and let him decide what to do with that information. Also as mentioned previously we let the developer decide how to handle the objects, for example, organizing them in tables, etc…
Conclusion: We were able to create very fast a small but efficient object vault that stores any C# object. Such a technique could be used when there is no reason to use complex databases which can be costly in time with malfunctioning drivers. Sometimes it’s a good idea for a developer to write his own tools rather than counting on third-party software vendors.
Acodez is a leading website design and web development company in India. We offer all kinds of web design and web development services to our clients using the latest technologies. We are also a top digital marketing agency providing SEO, SMM, SEM, Inbound marketing services, etc at affordable prices. For further information, please contact us.
Looking for a good team
for your next project?
Contact us and we'll give you a preliminary free consultation on the web & mobile strategy that'd suit your needs best.
Jamsheer K, is the Tech Lead at Acodez. With his rich and hands-on experience in various technologies, his writing normally comes from his research and experience in mobile & web application development niche.
Hello,
Am really amazed by this article,
am very glad that I saw this article.
Thank you!! Good job on posting this article.