Metadata
Information accompanying numeric data
- Data without metadata are useless.
Both are intrinsically tied to each other. - During data acquision, a maximum of information is available.
Extracting the relevant information is the real challenge. - Metadata should allow processing and analysis routines
to acquire a “semantic understanding”. - Metadata should be stored in a structured way
and be readable for both, humans and computers. - File formats for storing metadata should be platform independent
and as simple to use as possible.
Numeric data without accompanying metadata are useless – it's really that simple. The very minimum would be axis labels and the information where the data originated from and what they represent (e.g., measurements of a sample with some method or results from some calculations).
The information available is maximal during data acquisition – regardless of whether this means experiments or calculations. The real challenge is to reduce the available information to a sensible set of parameters.
Furthermore, metadata allow processing routines to get a “semantic understanding”. This means that the information contained in the metadata needs to be stored and organised in a hierarchical fashion, most probably using key-value stores.
Metadata should be organised and stored in a way accessible to both, humans and machines. This necessitates a (file) format that is platform-independent and as easy to use as possible.
Which tool to use
Note: This section is clearly opinionated. There are definitely other tools available. However, this is the tool the author recommends from own experience.
YAML
A machine-readable format that is very easy to write manually (by humans).