New Ideas On How To Design Better Software

By: Code Farms  09-12-2011
Keywords: libraries, Software Design, Data Storage

In addition to developing new software methodologies and libraries which support them, Code Farms Inc also provides consulting services in the following categories:

Architecture of Large and Complex Systems

We either lead the customer team or provide continuous advice from the initial architecture design all the way to the implementation of the first prototype. Our libraries and/or methodologies may or may not be used. Often, such projects include customized in memory databases (frameworks).

Phasing in Code Farms Technology

Our libraries are powerful, but since they are based on a new paradigm (intrusive data structures), their use is different from the usual class libraries such as STL, MFC, or tools.h++. Some users ask us to help them with the first project in order to reap the full profit of the benefits that the new approach provides, including significant improvement of the run-time performance.

Evaluation of an Existing Project

Complex legacy systems often require upgrades, but are so messy that nobody dares to touch them. We have been asked many time to evaluate such projects and suggest how to approach the redesign. In particular, we have seen many projects designed to run in memory - without storing data to disk, and as the application evolves, the data persistency is required.

Turn-key Design

We are not a software design shop, but for selected projects or customers, we occassionally provide a design of complete modules, usually programs which solve algorithmically difficult problem with emphasis on run-time performance. Typical examples would be programs for wiring of VLSI chips, or on-line processing of satellite data from stock exchanges worldwide.

Computer programs process data, and this data is often quite large, as for example in an airline reservation system, or quite complex, as the internal data used by a compiler. Traditionally, there have been two types of programs:

  • Programs which do not store internal data (e.g compiler)
  • Programs which require storage of complex data, usually provided by a database

Databases are large, sophisticated systems which store and retrieve data items and their relations, protect them against hardware or software errors, and help with version control. They also use standard interfaces (e.g. SQL). In spite of what the database vendors claim, the penalty for these services is a loss of performance, increased complexity of software which uses it, and larger load modules. Databases are also generally expensive.

Note that programs which originally did not store any data often evolve and, later on, the data storage becomes essential. For example, traditional compilers do not store data to disk, but incremental compilers must do it.

Over the last decade, a new alternative emerged: The data is treated as a memory resident data structure which can be stored to disk. This is usually called "persistent data" or simply "persistence", and it can be used as a fast, inexpensive database. It is lightweght in the sense of not providing all the database safety and flexibility, but it can be an order of magnitude faster than any commercial database. It is small and efficient, because it is customed designed for each application .

Code Farms  software (both DOL and PPF) allows to design such databases rapidly. A full running database even for a complex system can be operational within a day or a few. Read the beginning of DOL tutorial  (coming soon )   and you will understand why. The main difference between our persistent data and the commonly used serialization is that our data is automatically(!) persistent - you do not have to write any serialization functions. We also provide several different method of saving data to disk, and some functionality normally expected only from databases such as version control, schema migration and data integrity

Keywords: architecture design, Computer Programs, Data Storage, Legacy Systems, libraries, Satellite Data, Software Design, Software Methodologies, Standard Interfaces, Stock Exchanges,