This document attempts to define to what extent TANGRAM can be expected to behave reliably under stess (typically, when the data volumes are scaled up).

The TANGRAM philosophy is that if an operation works for 3 elements, it will work for 3000 or 300000. But, of course, there are limits. In general terms, TANGRAM turns out to be much more efficient than other handlers of multi-way tables.

This document applies to TANGRAM version 8, running under Dyalog version 8.1.5 and any Windows version.

First, distinguish among several kinds of limits:

• Design limit: this is hard-coded in TANGRAM design and implementation. It may be a firm design decision or an accidental implementation limit, which may be lifted in the future, but only through Codework's intervention.
• Inherited limit: this is not due to TANGRAM, but rather to the host system used in a particular implementation. Specifically, it is due to Windows or Dyalog limitations.
• Soft limit: this is just a rule-of-thumb, that means: you had better proceed carefully if you exceed it. But nothing particular happens when you reach this value. Often the limit is fuzzy because it depends on the resources of your PC (mostly, hard-disk space and available RAM memory).

A physical cube is accommodated in two files (labels and cells are stored separately). The label file (*.sf) is homogeneous with a library file (*.lib) in that it is read/written by the same programs.

• CUBES

  • Number of cubes on disk - Limited by disk space only

  • Max cube size on disk - No design limit. Inherited limit: 2 Gbyte

    Carefully designed cubes of 1 Gbyte are operational. The limiting factors are often LOADDB or BUILDDB times (See ADMIN services).

    Note. Experimental cubes of 2 Gbytes have been proven quite viable. A benchmark on a PC with 500 Mbytes of RAM has produced these results:

    • Cube size: 2 billion bytes or 250 million cells
    • Creation time: 1 min
    • Fill with random data: 4 min
    • Recompute (Admin formula): between 2 min and 2 min 30 sec
    • Percent of database in RAM memory at a time: 7%
    • Number of iterations to process the whole cube in Data Analysis: 13
    • Reading a 200 Mbyte subcube into RAM: typical 6 sec - worst case: 65 sec

  • Max number of cells in a cube - Inherited limit: 250 million.

    The size of the cell file (*.hlm) is exactly 8 times the cell count, so that the remarks on max cube size apply here also.

  • Max size of label file (*.sf) - Design limit: 10 Mbytes

  • Max cube rank (=dimension count)
    • Design limit: 26 dimensions
    • Inherited limit: 15 dimensions
    • The TANGRAM-Excel interface has an inherited limit of 20 dimensions.

    Conclusion: design cubes with a max of 15 physical dimensions.

    Note: Many TANGRAM modules work on 3 dimensions - cubes with higher rank can be collapsed to 3 dimensions to use the module and this has to be done in RAM memory.

  • Min cube rank (=dimension count) - Design limit: 3 dimensions

    Note: this limit is rather irrelevant, because you can have dummy dimensions that reduce to a single element.

    If you are handling a single Profit and Loss report, you can stucture it as, say, 1 Company, 1 Fiscal Year, 200 Variables.

  • Max number of elements per dimension - Design limit 1000000 elements

    Benchmark cases with 1 million elements have been successfully handled.

    Note: The total cell count is the product of the element counts on all dimensions. If one element count is huge, the others must be very modest.

    Note: Design physical dimensions in the order smallest-to-largest for performance reasons.

  • Max label (or namesystem) width - Soft limit: 5000 characters

  • Max number of facets (=fields) in a label (or in a namesystem) - Irrelevant, because they are not declared as such - but handling them may become cumbersome if there are more than, say, ten, because you often have to point to the relevant columns.

  • Max size of real current cube (or secondary cube) - Soft limit: 40% of the maxws= parameter used in the command line (see Properties of the TANGRAM icon), which in turn should not exceed the installed RAM. The maxws= parameter is expressed in Kbytes.

    Typical limit: with 500 Mbytes of RAM this may be 200 Mbytes or 24 million cells.

    Much larger physical cubes can be processed in Data Analysis by iterating an operation on reasonably-sized subcubes.

    Administration services handle these iterations themselves - in principle they work on any database size.

• LIBRARIES AND PROCEDURES

  • Max size of a library file (.lib) - Design limit: 10 Mbytes

  • Max object count in a library - Design limit: 1 million objects - Soft limit: 10000 objects

  • Max size of an object in a library - Design limit: 10 Mbytes - Soft limit: 1 Mbytes

  • Max line length in a procedure - Unlimited (above 1 million characters)

  • Max number of steps in a procedure - Unlimited

  • Max level of nesting in procedure calls - Unlimited

• IMPORTED FLAT FILES (BUILDDB, LOADDB)

  • Max size of flat file - Limited by disk space only

  • Max number of records - Unlimited

    But estimate and measure BUILDDB times, which depend on several factors.

    A gross order of magnitude can be 5000 to 50000 loaded records/second

    An alternative to BUILDDB is LOADDB, that loads just values into an existing cube. Times depend mostly on the sorting order of the flat file. An order of magnitude can be 3000 to 30000 loaded records/second.

  • Max record length - Design limit: 5000 bytes

  • Max number of fields in a flat file - Basically unlimited

    Note however that interactive field definition (in BUILDDB) is limited to 40 fields (a mixture of numeric and code fields).

    LOADDB uses only four fields per record, one of which must be numeric.

• MISCELLANY

  • Precision of numeric cells - approx. 16 significant digits, due to an 8-byte floating-point number representation.

  • Max value in a cell 1.79 10^308 (absolute value).

  • File names - 50 characters

  • Max length of names of library objects (procedures, namesystems and private variable) - Design limit: 25 characters

  • Max length of user inputs (in the Questions and Answers module) - Design limit: 5000 characters

  • Max complexity of formulas and queries - Unlimited

    This applies to all modules that accept formulas (COMPUTE, GLOBAL, CONTRIBUTE, BROWSE, QUERY, UPDFORMULAE).

  • Max number of users sharing cubes in a network - Unlimited - Soft limit: 100

    Note: Estimate or measure the frequency of simultaneous accesses to the same physical cube. Conflicting requests are correctly enqueued but may cause a delay of several seconds whenever a virtual cube is read into RAM memory.

  • Max size of edited char table - Inherited limit: the Dyalog Editor has a problem with line numbering (above 10000 lines) but, apart from that, can handle in eccess of 1 million lines.

  Other (inherited) limits apply to the size of data handled by TANGRAM's GUI modules and to the cubes handled by the TANGRAM-Excel interface.

Home page - Contact - Excel add-in - Mail to - Download - OLAP links - Manual