.. _apqc_overview:

Overview for APQC
=========================================

.. contents:: :local:

.. highlight:: Tcsh

Description
-----------

``afni_proc.py`` allows researchers to specify their desired steps and
options for FMRI processing.  The program is designed to be flexible,
relatively compact, and to help researchers organize their
processing. It is easy to share with others and to modify as
processing needs demand it.  It builds a processing script based on
the user's options, which is commented and can be
checked/read/studied/etc.  Several convenience features are taken care
of automatically, such as concatenating transforms (e.g., within EPI
for motion correction, EPI-> anat, anat -> standard space). 

Additionally, ``afni_proc.py`` tries to help researchers check their
processed data.  For example, many intermediate data sets are kept,
and several helper scripts are made to show users their data:

* **@ss_review_basic**
     show basic QC measures (e.g., max motion, number of censored time
     points, TSNR), in text

* **@ss_review_driver**
     a script that includes: 
     * "driving" the AFNI GUI to show users their data interactively;
     * opening up 1dplots to look at motion, censoring, etc.;
     * showing warning message from processing in the terminal.
     \.\.\. and all of that is commented (!).

* **@ss_review_html** (*new!*)
     a command that is automatically run at the end of
     ``afni_proc.py`` processing to make an HTML document full of
     images+quantities showing information from the processing
     (combining data from ``@ss_review_basic`` and
     ``@ss_review_driver`` with other stuff, too).
     
Here, describe the usage of the third QC output: the HTML form of
APQC.

.. _tut_apqc_overview_bkgd:

Background: what afni_proc.py does
-----------------------------------

The modern AFNI's ``afni_proc.py`` includes the following option:

.. code-block:: none

  -html_review_style STYLE : specify generation method for HTML review

          e.g.     -html_review_style pythonic
          default: -html_review_style basic

where STYLE can be one of the following values:

.. code-block:: none

  none     : no HTML review pages
  basic    : static - time graph images generated by 1dplot
  pythonic : static - time graph images generated in python
  
By default, ``afni_proc.py`` will implement the ``basic`` form of the
review style.  Therefore, with no intervention by the user, an HTML
doc of QC information will be generated whenever processing your FMRI
data.

A related, but slightly prettier, form of QC review is generated with
the ``pythonic`` option.  The only cost is that the user must have
Python (version 2.7 or higher) and the matplotlib module.  Because
this ``pythonic`` form of QC review is also slightly more informative,
we will just focus on this form here.

With the HTML review style flagged, ``afni_proc.py`` will include
something like the following few commands after it has finished
processing data sets::

  # generate html ss review pages
  # (akin to static images from running @ss_review_driver)
  apqc_make_tcsh.py -review_style pythonic -subj_dir . \
      -uvar_json out.ss_review_uvars.json
  tcsh @ss_review_html
  apqc_make_html.py -qc_dir QC_$subj

where ``$subj`` holds the name of your subject in the processing
script. Again, you, Dear Reader,
don't *have* to do anything with these commands-- they will be
executed on your behalf automatically.  But if you ever want to rerun
them, here they be.

After those commands are executed, you will be prompted to finally
take action, as ``afni_proc.py`` shows you something like the
following to open up the freshly created HTML QC doc:

.. code-block:: none

  afni_open -b $subj.results/QC_$subj/index.html

\.\.\. where ``$subj.results`` is your directory containing all
``afni_proc.py`` results; ``QC_$subj`` contains all the APQC results;
and "index.html" is the focal document that will be opened.  The
``$subjs``\s will be replaced with your subject name (such as
"sub-001" or "sub-753"). The ``afni_open`` command will select a
browser on your system (e.g., firefox) for opening up the file
"index.html".

The contents and functionality of this APQC document are explored on
the subsequent pages here.

Portability: how to share the APQC
-----------------------------------

As noted above, the directory ``QC_$subj`` (e.g., "QC_sub-001" or
"QC_sub-216") contains everything it needed for the QC.  **Whenever
you want to share your QC from processing with another person, or even
just move to another computer, be sure to move the entire
QC-directory, not just the HTML file or something.**

The primary file to note is the index.html file, which is the "target"
to open up in a browser.  It does not contain embedded images, but
instead those sit in sub-directories inside of the ``QC_$subj``
directory.

The "media" subdirectory contains all of the images, as well as
informative text and JSON files, from which the index.html file is
populated.  Names and content of these files should not be changed.

There is a "help.html" file contained in each QC-directory, which is
accessible through a "HELP" button in the upper right corner of the
index.html file.

There is also an ``apqc*.json`` file in the main QC-directory.  This
is initially mainly empty, but it can be populated via the QC buttons
in the APQC HTML doc with both ratings for individual QC steps and
comments.  That is, this file can store the user's own comments on the
data, which can be shared with others (or just recorded for future
reference).

Finally, the "extra_info" directory contains a boring CSS style file,
as well as well as a dictionary of ``afni_proc.py``\-processed files
in a JSON.

.. _tut_apqc_overview_saving:

Saving ratings and comments
-----------------------------------

As described in the subsequent pages, both ratings and comments of QC
steps (generally referred to throughout as "QC blocks"), as well as a
final summary evaluation can be stored in the HTML documentation.
This can be useful for recording, for instance, *why* you thought a
data set was bad, or a question to ask your boss, or a recipe idea for
quiche.  

The one downside with having an HTML documentation is that saving
changes goes through the browser, and browsers in general are
hyper-worried about security and hacks.  Because of this, it can't
know convenient features about your computer system like file
structure, even when running locally and looking at local files.
Therefore, you will have to walk through your file system to the
location of your file.  If you are editing several QC files in a row
and saving ratings/comments, then hopefully at least the files are
*near* each other in the file system, and your browser will open up in
the previous directory location, hence shortening the path for your
next file save.

*Additionally, you will likely want to turn off the feature of your
browser to automatically download files to a single place, such as
"Downloads" or "Telechargements".*

Those are just some practical considerations and constraints at
present, until browsers change.

Finally, we note that the APQC HTML page navigation has been tested
most strongly during development with the firefox browser.  It
*should* play nicely with any modern, mainstream browser (e.g.,
Chrome, Explorer, Safari, etc.), but quirks may occur.  If you come
across any difficulties, please let us know via the `Message Board
<https://discuss.afni.nimh.nih.gov/>`_ so we can try to
address them (however, please note that some issues may not be
resolvable except by using a different browser).