Semi-structured data

Semi-structured data^[1] is a form of structured data that does not obey the tabular structure of data models associated with relational databases or other forms of data tables, but nonetheless contains tags or other markers to separate semantic elements and enforce hierarchies of records and fields within the data. Therefore, it is also known as self-describing structure.

In semi-structured data, the entities belonging to the same class may have different attributes even though they are grouped together, and the attributes' order is not important.

Semi-structured data are increasingly occurring since the advent of the Internet where full-text documents and databases are not the only forms of data anymore, and different applications need a medium for exchanging information. In object-oriented databases, one often finds semi-structured data.

XML,^[2] other markup languages, email, and EDI are all forms of semi-structured data. OEM (Object Exchange Model)^[3] was created prior to XML as a means of self-describing a data structure. XML has been popularized by web services that are developed utilizing SOAP principles.

Some types of data described here as "semi-structured", especially XML, suffer from the impression that they are incapable of structural rigor at the same functional level as Relational Tables and Rows. Indeed, the view of XML as inherently semi-structured (previously, it was referred to as "unstructured") has handicapped its use for a widening range of data-centric applications. Even documents, normally thought of as the epitome of semi-structure, can be designed with virtually the same rigor as database schema, enforced by the XML schema and processed by both commercial and custom software programs without reducing their usability by human readers.

In view of this fact, XML might be referred to as having "flexible structure" capable of human-centric flow and hierarchy as well as highly rigorous element structure and data typing.

The concept of XML as "human-readable", however, can only be taken so far. Some implementations/dialects of XML, such as the XML representation of the contents of a Microsoft Word document, as implemented in Office 2007 and later versions, utilize dozens or even hundreds of different kinds of tags that reflect a particular problem domain - in Word's case, formatting at the character and paragraph and document level, definitions of styles, inclusion of citations, etc. - which are nested within each other in complex ways. Understanding even a portion of such an XML document by reading it, let alone catching errors in its structure, is impossible without a very deep prior understanding of the specific XML implementation, along with assistance by software that understands the XML schema that has been employed. Such text is not "human-understandable" any more than a book written in Swahili (which uses the Latin alphabet) would be to an American or Western European who does not know a word of that language: the tags are symbols that are meaningless to a person unfamiliar with the domain.

JSON or JavaScript Object Notation, is an open standard format that uses human-readable text to transmit data objects. JSON has been popularized by web services developed utilizing REST principles.

Databases such as MongoDB and Couchbase store data natively in JSON format, leveraging the pros of semi-structured data architecture.

This section does not cite any sources. Please help improve this section by adding citations to reliable sources. Unsourced material may be challenged and removed. (June 2024) (Learn how and when to remove this message)

• Programmers persisting objects from their application to a database do not need to worry about object-relational impedance mismatch, but can often serialize objects via a light-weight library.^[4]
• Support for nested or hierarchical data often simplifies data models representing complex relationships between entities.
• Support for lists of objects simplifies data models by avoiding messy translations of lists into a relational data model.

• The traditional relational data model has a popular and ready-made query language, SQL.^[5]
• Prone to "garbage in, garbage out"; by removing restraints from the data model, there is less forethought that is necessary to operate a data application.

This section does not cite any sources. Please help improve this section by adding citations to reliable sources. Unsourced material may be challenged and removed. (June 2024) (Learn how and when to remove this message)

The semi-structured model is a database model where there is no separation between the data and the schema, and the amount of structure used depends on the purpose.

The advantages of this model are the following:

• It can represent the information of some data sources that cannot be constrained by schema.^[6][1]
• It provides a flexible format for data exchange between different types of databases.^[6][1]
• It can be helpful to view structured data as semi-structured (for browsing purposes).^[1]
• The schema can easily be changed.
• The data transfer format may be portable.

The primary trade-off being made in using a semi-structured database model is that queries cannot be made as efficiently as in a more constrained structure, such as in the relational model. Typically the records in a semi-structured database are stored with unique IDs that are referenced with pointers to their location on disk. This makes navigational or path-based queries quite efficient, but for doing searches over many records (as is typical in SQL), it is not as efficient because it has to seek around the disk following pointers.

The Object Exchange Model (OEM) is one standard to express semi-structured data, another way is XML.

• NoSQL
• Unstructured data
• Structured data

• UPenn Database Group Archived 2013-11-02 at the Wayback Machine – semi-structured data and XML
• Semi-Structured data analytics: Relational or Hadoop platform? by IBM
• Semi-Structured Data Explained by CROWDSTRIKE