was developped for processing of X-ray snapshots from single-crystals
in random orientations (Kabsch, 2014).
It is based on the rotation data processing program XDS
The package consists of two components
nXDS for processing a stream of snapshots taken under similar
nXSCALE for combining multiple streams into a final data set of
fully corrected reflection intensities obtained by post-refinement.
For all images in a stream of X-ray snapshots nXDS assumes:
For combining multiple streams by nXSCALE the same type of detector
is assumed, perhaps with different internal settings of its segments
for each stream.
The same detector is used with identical internal settings of its
segments (in case of a multi-segment detector).
The detector is at a fixed position and orientation.
X-rays are approximately of the same wavelength, beam direction and
Direction of the rotation axis (if applicable) and oscillation range
do not change.
Detector type and image formats can be handled (these are the same
as for XDS).
As described in this chapter, the data images in a stream are processed
in 8 steps
which are called in succession by nXDS.
Information between the steps is communicated by files, which allows repetition of selected
steps with a different set of input parameters without rerunning the
whole program. The files generated by nXDS are either ASCII type files
that can be inspected and modified by using a text editor, or binary
image files in the CBF format, a byte-offset variant of the
format. Such images are indicated by the file name extension ".cbf".
All files have a fixed name defined by nXDS, which makes it
mandatory to process each stream in a newly created directory to
avoid name clashes. Clearly, one should not run more than one nXDS-job
simultaneously in the same directory. Also, output files generated by
rerunning selected steps (see Table)
should first be given another
name if their original contents are meant to be saved.
Data processing begins by copying an appropriate
input file template
into the data processing directory. Input file templates
are provided with the nXDS package for a number of frequently used data
collection facilities. The copied input file must be renamed nXDS.INP
and edited to provide the correct parameter values for the actual data
All parameters in nXDS.INP are named by keywords containing an equal
sign as the last character, and many of them will be mentioned here in
context to clarify their meaning. Execution of nXDS invokes in succession
each of the 8 program steps described below - or a subset of the steps
named in the parameter JOB=.
Results and diagnostics from each step are saved in files with the
extension .LP attached to the program step name. These files should
always be studied carefully to see whether processing was satisfactory
or - in case of failure - to find out what could have gone wrong.
calculates lookup tables of spatial corrections for each detector pixel
which are stored in the files X-CORRECTIONS.cbf and Y-CORRECTIONS.cbf .
In subsequent data processing steps, when the true coordinates of a
pixel with respect to the laboratory coordinate system are needed, the
correction values for the X- and Y-coordinates are retrieved from the
tables and added to the pixel's array coordinates in the data image.
Dependent on the detector, XYCORR computes the spatial corrections in
If the data images are already corrected for geometrical distortions,
as is often the case, XYCORR produces tables of zeros.
For spiral read-out imaging plate detectors like the MAR and MAR345
detectors, XYCORR computes the small corrections resulting from radial (ROFF=) and tangential (TOFF=) offset errors of the
scanner. For the STOE imaging plate detector the manufacturer's
software (Stoe & Cie GmbH, D-Darmstadt) provides a set of 8
calibration values in the crystal file that can be copied to the
For some multiwire- and CCD detectors that deliver geometrically
distorted images, corrections are derived from a calibration image (BRASS_PLATE_IMAGE=
file name). This image displays the response to a brass plate
containing a square grid of holes which is mounted in front of the
detector and illuminated by an X-ray point source.
The number of calibration holes and the distance between adjacent holes
can be specified by the user via the input parameters MXHOLE= and HOLE_DISTANCE=,
respectively, to override the detector specific default values. In
addition, the user has the option to specify a minimum number of
calibration spots (input parameter MNHOLE=) that must be
located by XYCORR in the brass-plate image. This serves as a check that
the calibration image was not underexposed.
XYCORR assumes that the source has been placed exactly at the location
to be occupied by the crystal during the actual data collection, as
photons emanating from the calibration source are meant to simulate all
possible diffracted beam directions. XYCORR also generates the file
FRAME.cbf which contains the original brass plate image in which the
calibration spots located by the program are marked for visual control.
The PILATUS pixel detector is assembled from 60 modules arranged in 12
banks of 5 modules. Each module consists of 487 X 195 pixels. There are
two types of corrections to compensate the spatial distortions; they
are applied in succession.
Geometrical corrections compensate misorientations of the modules with
respect to the X- and Y-axes of the specific detector; these corrections
are provided by the manufacturer as two files (input parameters
Parallax corrections are necessary due to the oblique incidence of the
scattered beam in the sensor material
(SILICON=) of finite thickness
these corrections depend on wavelength, detector distance and origin.
A misplaced calibration source leads to an incorrect brass plate image
and a useless lookup table as a consequence, impairing the correct prediction
of the observed diffraction pattern in subsequent program steps.
Underexposure of the brass plate calibration image results in an incomplete
and unreliable list of calibration spots. Inspect the control image
FRAME.cbf to find out whether this has been the case.
tests each snapshot and removes images that cannot result from diffraction
if their pixel contents does not obey Poisson statistics. The list of accepted
images replaces the original list specified by the parameter
The presence of a sufficient number of diffraction spots on the saved images
is determined in the subsequent step COLSPOT.
determines three lookup tables, saved as files BLANK.cbf, GAIN.cbf, and
BKGINIT.cbf, that are required by the subsequent processing steps for
classifying pixels in the data images as background or belonging to a
diffraction spot ('strong' pixels).
contains the detector background noise at each image pixel in
the absence of any X-rays. Some of the detectors, like the SIEMENS
multiwire detector, do not have background noise and the table will
contain just zeros. Otherwise, the table is derived in the following
If a data image was taken in the absence of any X-rays (input parameter DARK_CURRENT_IMAGE=file
name), this image is used for the table BLANK.cbf.
If the detector noise in each pixel was estimated by a
non-negative constant (input parameter OFFSET=), the table
BLANK.cbf is set to the given constant.
If neither of the above two cases applies, a constant detector noise is
determined from the mean value at the four corners of a few data
images. The table BLANK.cbf is set to the determined constant.
codes for the expected variation of the pixel contents in the
background region of a data image. The variance of the contents of a
pixel in the background region is GAIN*(pixel_contents-detector_noise).
The variance is determined from the scatter of pixel values within a
rectangular box (input parameters NBX=, NBY=) of size
(2*NBX+1)*(2*NBY+1) centered at each image pixel in succession. The
table GAIN.cbf is used for distinguishing background pixels from
"strong" pixels that are part of a diffraction spot.
estimates the initial background at each pixel from a few
data images specified by the input parameter BACKGROUND_RANGE=.
The lookup table is obtained by adding the X-ray background from each
image. Shaded regions on the detector (i.e., beam-stop) or pixels
outside a user defined circular region (TRUSTED_REGION=) or
pixels with an
undefined spatial correction value are classified as untrustworthy and
marked by -3. The table should be inspected using the XDS-Viewer program.
Some detectors with insufficient protection from electromagnetic
pulses may generate badly spoiled images whose inclusion leads to a
completely wrong X-ray background table. These images can be identified
in INIT.LP by their unexpected high mean pixel contents, and this step
should be repeated with a different set of images.
locates strong diffraction spots occurring in the data images listed in
the file named by the input parameter value
and saves the spot centroids on the file
COLSPOT identifies 'strong' pixels
whose values clearly exceed the fluctuations in a rectangular box
(input parameters NBX=, NBY=)
of size (2*NBX+1)*(2*NBY+1) centered at each image pixel in succession.
Spots are defined as sets of 'strong' pixels adjacent in two dimensions.
A spot is accepted if it contains a minimum number of 'strong' pixels
and if the spot centroid is sufficiently close to the location of the
strongest pixel in the spot
Images containing less that
are excluded from further processing.
of the strongest spots are saved on file
The names of the accepted images are
included in the list of 'good' images
Sharp edges in the images like ice rings or borders of the untrusted
detector regions can lead to an excessive number of 'strong' pixels
erroneously classified as contributing to diffraction spots. These
aliens could lead to a waste of computing time and even prevent
IDXREF to recognize the crystal lattice. To avoid this problem an
upper limit for the most strongest spots in an image can be set
by the user.
generates a powder pattern
from all spots in the file
This control image could help the user to correct input parameter values
for the origin of the detector coordinate system which is critical for
the correct indexing of the scattering vectors in the subsequent steps
of data processing. The generated image from this step should be
inspected by the user with the XDS-Viewer program which adopts an
interpretation of cursor positions consistent with the nXDS program.
The plane of the control image POWDER.cbf (1024 × 1024 square pixels)
is placed at unit distance from the crystal with its normal coinciding
with the incident beam direction computed from the input parameter
The incident beam intersects the image at pixel coordinates x=511, y=511
at the image center which is marked by crosshairs.
The scattering vectors from file SPOT.nXDS are marked in this image
where they form a set of concentric rings - the powder pattern -
in the ideal case.
The center of the rings should be at the tip of the incident beam -
which is the image center.
Often the center of the powder rings is found instead offset from its
ideal place which can be interpreted to result from an incorrect value
for the origin of the detector coordinate system (ORGX=, ORGY=).
The pixel offset dx,dy (powder center - image center) is specified by
the input parameter
POWDER_CENTER_CORRECTION=dx dy q
where q is the side length (radians) of a square pixel.
From this information a corrected origin (ORGX=, ORGY=) is calculated
by the POWDER step and reported in POWDER.LP. For further processing
with nXDS you should then specify in nXDS.INP the new values for
ORGX=, ORGY= and turn off the parameter or set
POWDER_CENTER_CORRECTION= 0 0 0.001 which is its default. The POWDER
step could be repeated with the new values for visual control or
nXDS could proceed if no further corrections are equired.
The control image POWDER.cbf does not show a concentric set of circles
despite a large number of spots. This could be due to the presence of
too many spots or artefacts on file SPOT.nXDS, to varying directions
of the incident beam during data collection or to incorrect parameters
describing the multi-segment detector.
uses the initial parameters describing the diffraction experiment from
the input file
and the centroids of the spots observed
on each image from the file
SPOT.nXDS to find
orientation, metric, and symmetry of the crystal lattice, and saves
the refined values on file
Spots from file SPOT.nXDS are accepted that are
To determine a crystal lattice that explains the observed locations of
the diffraction spots, IDXREF proceeds for each image as follows.
The laboratory coordinates of the diffracted beam wave vector
(normalized to 1/λ) corresponding to each spot
are calculated from the input parameter values
specifying the origin and orientation of the multi-segment detector
QX=, and QY=).
Subtraction of the incident beam wave vector
(given by the parameters INCIDENT_BEAM_DIRECTION=
from the diffracted beam wave vector leads for each spot to a reciprocal lattice
vector when the Laue equations are satisfied.
Differences between any two such reciprocal lattice vectors which are
above a specified minimal length ( SEPMIN=)
are then accumulated in a 3-dimensional histogram with a grid size
(RGRID=) defined by the user or
determined by the program.
These difference vectors will form clusters in the histogram since there
are many different pairs of reciprocal lattice vectors of nearly identical
vector difference. The clusters are found as maxima in the smoothed histogram
(CLUSTER_RADIUS=), and only some
of the most densely populated cluster vectors are further used.
If the space-group is unknown a vector triplett is selected that best
explains the observed difference vector clusters by integral indices
Vector clusters that can be consistently indexed with
respect to this triplett are connected in a tree.
Trees using the same vector triplett as a basis - but offset
only by a constant vector in reciprocal space - can be joined into
a new, common reciprocal basis explaining all members of the trees.
The parameter MERGE_TREE=
defines a minimum fraction of explained difference vectors
required for inclusion in a common basis.
If space-group and cell constants are specified (input parameters
the observed difference vector clusters are interpreted (indexed)
by a rotated version of the given unit cell using exhaustive search
The basis vectors and the 30 best short lattice vectors with attached indices
are listed in IDXREF.LP. If many of the indices deviate significantly
from integral values, the program is unable to find a reasonable
lattice basis and all further processing will be meaningless for this
Based on the orientation and metric of the reduced cell,
up to 3,000 of the strongest spots are indexed by the local indexing
method. This method considers each spot as a node of a tree and
identifies the largest subtree of nodes which can be assigned reliable
The number of reflections in the ten largest subtrees is
reported and usually shows a dominant first tree corresponding to a
single lattice, whereas alien spots are found in small subtrees. Input
parameters that control the local indexing are INDEX_ERROR=, INDEX_MAGNITUDE=, INDEX_QUALITY=.
Reflections in the largest subtree are used for initial refinement of
the basis vectors of the reduced cell, the incident beam wave vector,
and the detector parameters (controlled by the input parameter
Spots are included in the refinement only if their
coordinates can be explained with reasonable accuracy
The refined metric parameters of the reduced cell are used for testing
each of the 44 possible lattice types
Possible lattice symmetries (decision constants
are reported but no automatic decision for the space-group is made.
If the crystal symmetry is unknown, data processing will continue with
the crystal being described by its reduced cell basis vectors and
triclinic symmetry. Otherwise, the observed spots are reindexed in terms
of the given cell whose basis vectors are found from the reduced cell
After initial refinement based on the reflections in the largest
subtree, all spots which can now be indexed are included. Spots not
belonging to the crystal lattice are given indices 0,0,0.
The initial file SPOT.nXDS is replaced by a file of identical name -
now with indices attached to each observed spot.
The indexing run is considered successful if some minimum number
(default 40%) of the given spots can be explained;
otherwise the image will be excluded from further processing.
Indices of many short lattice vectors deviate significantly from
integral values. This can be caused by incorrect input parameters,
like oscillation angle, detector position, by a large
fraction of alien spots in SPOT.nXDS caused by multiple lattices,
or by placing the detector too close to the crystal.
Indexing and refinement is unsatisfactory despite well-indexed
short lattice vectors. This probably results from selection of an
incorrect index origin caused by inaccurate values for the input
parameters ORGX= and ORGY=.
A visual check of a data image or the
control image POWDER.cbf
with the XDS-Viewer program could be quite
helpful for finding the approximate coordinates for the detector origin.
determines the recorded intensity of reflections of all successfully
indexed data images using the parameter values listed in file
XPARM.nXDS. The refined
parameters together with the recorded intensities are saved on file
representing the main result of processing the specified stream of
For each image INTEGRATE carries out the following steps.
Labelling of each image pixel by the indices of the nearest
reflection whereby invalid pixels
or pixels in the untrusted detector region
or those outside the allowed resolution range
Pixels from each detector segment are sorted by their index labels
so that contributions to the same reflection follow each other.
Classification of image pixels into background and spot. Pixel
contents that exceed the background by a given multiple of standard
deviations are called strong (input parameters SIGNAL_PIXEL=, BACKGROUND_PIXEL=).
Determination of extension and form of the diffraction spots from
the distributions of the pixels contributing to the strong spots.
A spot is described by the optional input parameters
All of the parameters describing extension and form of the spots
can be determined automatically from the idea that the observed spots
should be expected.
is twice the opening angle of a cone with the diffracted beam wave
vector as cone axis. The interception of the cone with the data image
traces the boundary of the spot and includes some neighbouring
characterizes the Gaussian spot shape by its standard deviation.
is the standard deviation (1/Å) of a Gaussian modeling the
Selection of strong spots for refinement. Centroids computed from
the strong pixels must be reasonably close to their calculated
position using the initial diffraction parameters from XPARM.nXDS
Control of the refinable parameters is provided by the user input
Construction of a specific coordinate system for each spot that is
centered on the surface of the Ewald sphere. In this coordinate system
α and β span the plane tangential to the Ewald sphere
with the α-axis perpendicular to the incident- and
diffracted beam wave vector of the reflection. Intensity profiles
of the spots become very similar when expressed in such a system
The number of grid points in this coordinate system used for
representing the transformed reflection profile are usually chosen
automatically by nXDS; the user has the option to override the
automatic assignment by specifying the input parameter
Generation of a spot template by superimposing transformed profiles
of strong reflections that are not overloaded
(OVERLOAD=). Grid points with
a value above a minimum percentage of the maximum in the template
are defined as elements of the integration domain.
Integration by profile fitting
with respect to the spot template
for all reflections predicted to occur in the image. Incompletely
recorded reflections with less than
intensity compared with the spot template will be discarded.
Otherwise, the missing intensity is estimated from the learned
profile of the spot template.
Reflections in the 'blind' region
or those too far from the Ewald sphere
are omitted from the output file INTEGRATE.HKL.
Off-centered profiles indicate incorrectly predicted reflection
positions by using the parameters provided by the file XPARM.nXDS (i.e.,
misindexing by using a wrong origin of the indices).
Profiles extending to the borders of the box indicate too-small values
This leads to incorrect integrated intensities because of truncated
reflection profiles and unreliable background determination.
determines scaling and correction factors for the recorded intensities and
standard deviations of all reflections in the file
reports quality and completeness of the data,
and saves the fully corrected integrated intensities on file
This file can be directly read by XDSCONV (see
which converts the reflection data into various formats required
by software packages for crystal structure determination like CCP4,
CNS, X-PLOR, or SHELX.
The integrated intensities of the reflections saved on file INTEGRATE.HKL
may or may not have been indexed in the correct space group. For the purpose
of integration it is only important that all reflections occurring in the
data images have been accurately located and indexed with respect to some
unit cell basis. Thus, the INTEGRATE step can be carried out even in the
case of an unknown space group because the reflection indices with respect
to the correct space group are always a linear transformation of the original
indices used in the INTEGRATE step and reported in INTEGRATE.HKL.
Presently, CORRECT requires knowledge of the space group and approximate
cell constants. Plausible space group candidates and their conventional
cell parameters were already reported in IDXREF.LP and can now be tried
in the CORRECT step without the need for rerunning previous program steps
in nXDS.INP and rerun nxds_par).
The possibility to compare the new data with a reference data set
is particularly useful for resolving the issue of alternative
settings in case the lattice symmetry is higher than the point group
symmetry of the crystal (e.g. P4, P6, R3).
NOTE: It is assumed that the reference reflections are indexed with
respect to the basis specified in nXDS.INP
Also, reference data are quite useful for recognizing misindexing.
CORRECT reindexes the input reflections from file INTEGRATE.HKL
thereby resolving possible indexing ambiguities by comparison with
a given reference data set or by a selective breeding algorithm that yields,
on average, the highest correlation with intensities from all other images
Reflections from file INTEGRATE.HKL are accepted that are
Intensity correction factors are calculated for each accepted reflection
that are due to
polarization of the incident beam (parameters
differences in path lengths of the diffracted beam in the detector material
Ewald offset (defined as angle between the center of the rotation/still
image and the calculated angle of the Bragg peak on the Ewald sphere).
differences a reflection remains in diffracting condition
(Lorentz correction factor for still or rotation images).
Recognition and removal of outliers among the images is based on medians of
CORRECT improves the final data set by a post-refinement procedure that
iteratively changes the crystallographic parameters of all images
so that intensities of symmetry equivalent reflections become
as similar as possible to each other and simultaneously minimize
discrepacies between calculated and observed spot positions on the detector
segments. The post-refinement can be controlled by input parameters like
After postrefinement a simple method is chosen to reduce the potentially
very large number of reflections to a manageable size - yet with little
loss of information.
For each unique reflection (considering Friedel pairs as different)
in general two statistically independent intensity estimates are generated
from reflections taken from randomly selected snapshots.
Now, comparison of pairs of independent intensity estimates allows to
correct the original intensity error estimates, obtained from counting
statistics during the INTEGRATE step, by a resolution-dependent factor
so that the new error estimates agree with the sample statistics
of the intensity pairs.
Outliers from the Wilson plot, often arising from ice rings in the data
images, are recognized and marked by a negative sign attached to their
e.s.d. in the final output file
The reduced set of statistically independent intensity observations
thus obtained is sufficient to assess the quality of the final data
set as function of resolution by the various indicators, like
I/SIGMA, Rmrgd-F, CC(1/2), etc
(Diederichs and Karplus, 1997).
This reduced set of fully corrected data are saved on file
A reference data set is provided but shows very poor correlation
with most of the newly processed snapshot data. This
could happen if the reference reflections mistakenly refer to a
unit cell basis other than the specified one in nXDS.INP.
Incomplete data sets may lead to wrong conclusions about the
space-group, as some of its symmetry operators might not be involved in
the R-factor calculations.
Often, the CORRECT step is repeated several times. One should
remember, that nXDS overwrites the earlier versions of the output
files nXDS_ASCII.HKL and CORRECT.LP.
© 2013-2021, MPI for Medical Research, Heidelberg
page last updated: May 8, 2021