.. _dataanalysis-formats:

Output formats
==============

This section describes the type of output data generated by WarpX.


.. _dataanalysis-formats-full:

Field and Particle Data
-----------------------

openPMD Outputs
^^^^^^^^^^^^^^^

`openPMD <https://www.openpmd.org/>`__ is supported by common community formats such as `ADIOS <https://csmd.ornl.gov/adios>`__ and `HDF5 <https://www.hdfgroup.org/>`__.
More details about using openPMD outputs with WarpX are available in :ref:`dataanalysis-openpmd`.


AMReX Plotfiles
^^^^^^^^^^^^^^^

`Plotfiles <https://amrex-codes.github.io/amrex/docs_html/IO.html>`__ are AMReX's native data format. See :ref:`dataanalysis-yt` for tools to read AMReX plotfiles.


.. _dataanalysis-formats-reduced:

Reduced diagnostics
-------------------

WarpX includes :ref:`reduced diagnostics <running-cpp-parameters-diagnostics-reduced>`.
Reduced diagnostics compute observables on the fly, such as energy histograms or particle beam statistics, and are usually stored as tabular text (CSV) files.

WarpX also provides optional reduced diagnostics that typically return a single value (e.g., particle energy) per time step.

A simple and quick way to read the data using Python is

.. code-block:: python

    data = numpy.genfromtxt("filename.txt")

where ``data`` is a two-dimensional array and ``data[i][j]`` gives the value in the i-th row and j-th column.

Note that, while this applies to most reduced diagnostics, there are a few that require openPMD, such as ``ParticleHistogram2D`` and ``DifferentialLuminosity2D``.


Asynchronous IO
^^^^^^^^^^^^^^^

When using the AMReX `plotfile` format, users can set the ``amrex.async_out=1``
option to perform I/O in a non-blocking fashion: the simulation continues while an I/O thread
writes data to disk. This can significantly reduce the overall time spent in I/O. It is
primarily intended for large runs on supercomputers (e.g., at OLCF or NERSC); depending on
your MPI implementation, you may not see a benefit on a workstation.

When writing plotfiles, each rank writes to a separate file, up to a maximum number (by
default 64). This maximum can be adjusted using the ``amrex.async_out_nfiles`` input
parameter. To use asynchronous I/O with more than ``amrex.async_out_nfiles`` MPI ranks,
WarpX must be configured with ``-DWarpX_MPI_THREAD_MULTIPLE=ON``. See
:ref:`the building instructions <install-build-cmake>` for details.


Staggering of Output Data
-------------------------

Time
^^^^

Warning: quantities in the output file for iteration ``n`` are not all defined at the same physical time due to temporal staggering in WarpX.
The table below shows the physical time (in units of time steps) at which each quantity is written for time step ``n``.

======== ===== =====
quantity staggering
-------- -----------
         FDTD  PSATD
======== ===== =====
E        n     n
B        n     n
j        n-1/2 n-1/2
rho      n     n
position n     n
momentum n     n
======== ===== =====

Space
^^^^^

.. warning::

   Field quantities (see ``<diag_name>.fields_to_plot`` in :ref:`diagnostics <running-cpp-parameters-diagnostics>`) in the output files are averaged at cell centers.