1.2.3.1. Rules for Structuring Information in NeXus Files

All NeXus files contain one or many groups of type NXentry at root level. Many files contain only one NXentry group, then the name is entry. The NXentry level of hierarchy is there to support the storage of multiple related experiments in one file. Or to allow the NeXus file to serve as a container for storing a whole scientific workflow from data acquisition to publication ready data. Also, NXentry class groups can contain raw data or processed data. For files with more than one NXentry group, since HDF requires that no two items at the same level in an HDF file may have the same name, the NeXus fashion is to assign names with an incrementing index appended, such as entry1, entry2, entry3, etc.

In order to illustrate what is written in the text, example hierarchies like the one in figure Raw Data are provided.

Content of a Raw Data NXentry Group

An example raw data hierarchy is shown in figure Raw Data (only showing the relevant parts of the data hierarchy). In the example shown, the data field in the NXdata group is linked to the 2-D detector data (a 512x512 array of 32-bit integers). The attribute signal = data on the NXdata group marks this field as the default plottable data of the data:NXdata group. The NXdata group attribute axes = . . declares that both dimensions of the data field do not have associated dimension scales (plotting routines should use integer scaling for each axis). Note that [,] represents a 2D array.

NeXus Raw Data Hierarchy

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
    entry:NXentry
        @default = data
        instrument:NXinstrument
            source:NXsource
            ....
            detector:NXdetector
                data:NX_INT32[512,512]
        sample:NXsample
        control:NXmonitor
        data:NXdata
            @signal = data
            @axes = . .
            data --> /entry/instrument/detector/data

An NXentry describing raw data contains at least a NXsample, one NXmonitor, one NXdata and a NXinstrument group. It is good practice to use the names sample for the NXsample group, control for the NXmonitor group holding the experiment controlling monitor and instrument for the NXinstrument group. The NXinstrument group contains further groups describing the individual components of the instrument as appropriate.

The NXdata group contains links to all those data items in the NXentry hierarchy which are required to put up a default plot of the data. As an example consider a SAXS instrument with a 2D detector. The NXdata will then hold a link to the detector image. If there is only one NXdata group, it is good practice to name it data. Otherwise, the name of the detector bank represented is a good selection.

Content of a processed data NXentry group

Processed data, see figure Processed Data, in this context means the results of a data reduction or data analysis program. Note that [] represents a 1D array.

NeXus Processed Data Hierarchy

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
    entry:NXentry
        @default = data
        reduction:NXprocess
            program_name = "pyDataProc2010"
            version = "1.0a"
            input:NXparameter
                filename = "sn2013287.nxs"
        sample:NXsample
        data:NXdata
            @signal = data
            @axes = .
            data

NeXus stores such data in a simplified NXentry structure. A processed data NXentry has at minimum a NXsample, a NXdata and a NXprocess group. Again the preferred name for the NXsample group is sample. In the case of processed data, the NXdata group holds the result of the processing together with the associated axis data. The NXprocess group holds the name and version of the program used for this processing step and further NXparameter groups. These groups ought to contain the parameters used for this data processing step in suitable detail so that the processing step can be reproduced.

Optionally a processed data NXentry can hold a NXinstrument group with further groups holding relevant information about the instrument. The preferred name is again instrument. Whereas for a raw data file, NeXus strives to capture as much data as possible, a NXinstrument group for processed data may contain a much-reduced subset.

NXsubentry or Multi-Method Data

Especially at synchrotron facilities, there are experiments which perform several different methods on the sample at the same time. For example, combine a powder diffraction experiment with XAS. This may happen in the same scan, so the data needs to be grouped together. A suitable NXentry would need to adhere to two different application definitions. This leads to name clashes which cannot be easily resolved. In order to solve this issue, the following scheme was implemented in NeXus:

  • The complete beamline (all data) is stored in an appropriate hierarchy in an NXentry.
  • The NXentry group contains further NXsubentry groups, one for each method. Each NXsubentry group is constructed like a NXentry group. It contains links to all those data items required to fulfill the application definition for the particular method it represents.
  • Each NXsubentry group contains a NXdata group describing the default plottable data for that experimental method. To satisfy the NeXus requirement of finding the default plottable data from a NXentry group, the NXdata group from one of these NXsubentry groups (the fluoresence data) was linked.

See figure NeXus Multi Method Hierarchy for an example hierarchy. Note that [,] represents a 2D array.

NeXus Multi Method Hierarchy

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
    entry:NXentry
        @default = data
        user:NXuser
        sample:NXsample
        instrument:NXinstument
            SASdet:NXdetector
                data:[,]
            fluordet:NXdetector
                data:[,]
            large_area:NXdetector
                data:[,]
        SAS:NXsubentry
            definition = "NXsas"
            instrument:NXinstrument
                detector:NXdetector
                    data --> /entry/instrument/SASdet/data
            data:NXdata
                data --> /entry/instrument/SASdet/data
        Fluo:NXsubentry
            definition = "NXFluo"
            instrument:NXinstrument
                detector --> /entry/instrument/fluordet/data
                detector2 --> /entry/instrument/large_area/data
            data:NXdata
                @signal = detector
                @axes = . .
                detector --> /entry/instrument/fluordet/data
        data:NXdata --> /entry/Fluo/data

Rules for Special Cases

Scans

Scans are difficult to capture because they have great variety. Basically, any variable can be scanned. Such behaviour cannot be captured in application definitions. Therefore NeXus solves this difficulty with a set of rules. In this section, NP is used as a symbol for the number of scan points.

  • The scan dimension NP is always the first dimension of any multi-dimensional dataset. The reason for this is that HDF allows the first dimension of a dataset to be unlimited. Which means, that data can be appended to the dataset during the scan.
  • All data is stored as arrays of dimensions NP, original dimensions of the data at the appropriate position in the NXentry hierarchy.
  • The NXdata group has to contain links to all variables varied during the scan and the detector data. Thus the NXdata group mimics the usual tabular representation of a scan.
  • Datasets in an NXdata group must contain the proper attributes to enable the default plotting, as described in the section titled NXdata Facilitates Automatic Plotting.

Simple scan

Examples may be in order here. Let us start with a simple case, the sample is rotated around its rotation axis and data is collected in a single point detector. See figure Simple Scan for an overview. Then we have:

  • A dataset at NXentry/NXinstrument/NXdetector/data of length NP containing the count data.
  • A dataset at NXentry/NXsample/rotation_angle of length NP containing the positions of rotation_angle at the various steps of the scan.
  • NXdata contains links to:
    • NXentry/NXinstrument/NXdetector/data
    • NXentry/NXsample/rotation_angle
  • All other fields have their normal dimensions.

NeXus Simple Scan Example

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
    entry:NXentry
        @default = data
        instrument:NXinstrument
            detector:NXdetector
                data[NP]
        sample:NXsample
            rotation_angle[NP]
        control:NXmonitor
            data[NP]
        data:NXdata
            @signal = data
            @axes = rotation_angle
            @rotation_angle_indices = 0
            data --> /entry/instrument/detector/data
            rotation_angle --> /entry/sample/rotation_angle

Simple scan with area detector

The next example is the same scan but with an area detector with xsize times ysize pixels. The only thing which changes is that /NXentry/NXinstrument/NXdetector/data will have the dimensions NP, xsize, ysize. See figure Simple Scan with Area Detector for an overview.

NeXus Simple Scan Example with Area Detector

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
    entry:NXentry
        instrument:NXinstrument
            detector:NXdetector
                data:[NP,xsize,ysize]
        sample:NXsample
            rotation_angle[NP]
        control:NXmonitor
            data[NP]
        data:NXdata
            @signal = data
            @axes = rotation_angle . .
            @rotation_angle_indices = 0
            data --> /entry/instrument/detector/data
            rotation_angle --> /entry/sample/rotation_angle

The NXdata group attribute axes = rotation_angle . . declares that only the first dimension of the plottable data has a dimension scale (by name, rotation_angle). The other two dimensions have no associated dimension scales and should be plotted against integer bin numbers.

Complex hkl scan

The next example involves a complex movement along the axis in reciprocal space which requires mutiple motors of a four-circle diffractometer to be varied during the scan. We then have:

  • A dataset at NXentry/NXinstrument/NXdetector/data of length NP containing the count data.

  • A dataset at NXentry/NXinstrument/NXdetector/polar_angle of length NP containing the positions of the detector’s polar_angle at the various steps of the scan.

  • A dataset at NXentry/NXsample/rotation_angle of length NP containing the positions of rotation_angle at the various steps of the scan.

  • A dataset at NXentry/NXsample/chi of length NP containing the positions of chi at the various steps of the scan.

  • A dataset at NXentry/NXsample/phi of length NP containing the positions of phi at the various steps of the scan.

  • A dataset at NXentry/NXsample/h of length NP containing the positions of the reciprocal coordinate at the various steps of the scan.

  • A dataset at NXentry/NXsample/k of length NP containing the positions of the reciprocal coordinate at the various steps of the scan.

  • A dataset at NXentry/NXsample/l of length NP containing the positions of the reciprocal coordinate at the various steps of the scan.

  • NXdata contains links to:

    • NXentry/NXinstrument/NXdetector/data
    • NXentry/NXinstrument/NXdetector/polar_angle
    • NXentry/NXsample/rotation_angle
    • NXentry/NXsample/chi
    • NXentry/NXsample/phi
    • NXentry/NXsample/h
    • NXentry/NXsample/k
    • NXentry/NXsample/l

    The NXdata also contains appropriate attributes as described in Associating plottable data using attributes applied to the NXdata group.

  • All other fields have their normal dimensions.

NeXus Complex hkl Scan

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
    entry:NXentry
        @default = data
        instrument:NXinstrument
            detector:NXdetector
                data[NP]
                polar_angle[NP]
                name
        sample:NXsample
            name
            rotation_angle[NP]
            chi[NP]
            phi[NP]
            h[NP]
            k[NP]
            l[NP]
        control:NXmonitor
            data[NP]
        data:NXdata
            @signal = data
            @axes = h
            @h_indices = 0
            @k_indices = 0
            @l_indices = 0
            @chi_indices = 0
            @phi_indices = 0
            @polar_angle_indices = 0
            @rotation_angle_indices = 0
            data --> /entry/instrument/detector/data
            rotation_angle --> /entry/sample/rotation_angle
            chi --> /entry/sample/chi
            phi --> /entry/sample/phi
            polar_angle --> /entry/instrument/detector/polar_angle
            h --> /entry/sample/h
            k --> /entry/sample/k
            l --> /entry/sample/l

Multi-parameter scan: XAS

Data can be stored almost anywhere in the NeXus tree. While the previous examples showed data arrays in either NXdetector or NXsample, this example demonstrates that data can be stored in other places. Links are used to reference the data.

The example is for X-ray Absorption Spectroscopy (XAS) data where the monochromator energy is step-scanned and counts are read back from detectors before (I0) and after (I) the sample. These energy scans are repeated at a sequence of sample temperatures to map out, for example, a phase transition. While it is customary in XAS to plot log(I0/I), we show them separately here in two different NXdata groups to demonstrate that such things are possible. Note that the length of the 1-D energy array is NE while the length of the 1-D temperature array is NT

NeXus Multi-parameter scan: XAS

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
    entry:NXentry
        @default = I_data
        instrument:NXinstrument
            I:NXdetector
                data:NX_NUMBER[NE,NT]
                energy --> /entry/monochromator/energy
                temperature --> /entry/sample/temperature
            I0:NXdetector
                data:NX_NUMBER[NE,NT]
                energy --> /entry/monochromator/energy
                temperature --> /entry/sample/temperature
        sample:NXsample
            temperature:NX_NUMBER[NT]
        monochromator:NXmonochromator
            energy:NX_NUMBER[NE]
        I_data:NXdata
            @signal = data
            @axes = energy temperature
            @energy_indices = 0
            @temperature_indices = 0
            data --> /entry/instrument/I/data
            energy --> /entry/monochromator/energy
            temperature --> /entry/sample/temperature
        I0_data:NXdata
            @signal = data
            @axes = energy temperature
            @energy_indices = 0
            @temperature_indices = 0
            data --> /entry/instrument/I00/data
            energy --> /entry/monochromator/energy
            temperature --> /entry/sample/temperature

Rastering

Rastering is the process of making experiments at various locations in the sample volume. Again, rasterisation experiments can be variable. Some people even raster on spirals! Rasterisation experiments are treated the same way as described above for scans. Just replace NP with P, the number of raster points.

Special rules apply if a rasterisation happens on a regular grid of size xraster, yraster. Then the variables varied in the rasterisation will be of dimensions xraster, yraster and the detector data of dimensions xraster, yraster, (orginal dimensions) of the detector. For example, an area detector of size xsize, ysize then it is stored with dimensions xraster, yraster, xsize, ysize.

Warning

Be warned: if you use the 2D rasterisation method with xraster, yraster you may end up with invalid data if the scan is aborted prematurely. This cannot happen if the first method is used.

NXcollection

On demand from the community, NeXus introduced a more informal method of storing information in a NeXus file. This is the NXcollection class which can appear anywhere underneath NXentry. NXcollection is a container for holding other data. The foreseen use is to document collections of similar data which do not otherwise fit easily into the NXinstrument or NXsample hierarchy, such as the intent to record all motor positions on a synchrotron beamline. Thus, NXcollection serves as a quick point of access to data for an instrument scientist or another expert. NXcollection is also a feature for those who are too lazy to build up the complete NeXus hierarchy. An example usage case is documented in figure NXcollection example.

NXcollection Example

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
    entry:NXentry
        positioners:NXcollection
            mxx:NXpositioner
            mzz:NXpositioner
            sgu:NXpositioner
            ttv:NXpositioner
            hugo:NXpositioner
            ....
        scalars:NXcollection
            title (dataset)
            lieselotte (dataset)
            ...
        detectors:NXcollection
            Pilatus:NXdata
            MXX-45:NXdata
            ....