The tedana roadmap

Project vision

ME-EPI processing is not well integrated into major preprocessing packages, yielding duplicated and unmaintained code. tedana has been developed to address this need and will serve as a central repository for standard ME-EPI denoising as well as a testing ground for novel ME-EPI denoising methods. This will jointly reduce the external burden on pipeline maintainers, facilitate increased ME-EPI adoption, and enable future development in ME-EPI denoising.

Metrics of success and corresponding milestones

We will know that we have been successful in creating tedana when we have succeeded in providing several concrete deliverables, which can be broadly categorized into:

  1. Documentation,
  2. Transparent and reproducible processing,
  3. Testing,
  4. Workflow integration: AFNI,
  5. Method extensions & improvements, and
  6. Developing a healthy community

Each deliverable has been synthesized into a milestone that gives the tedana community a link between the issues and the high level vision for the project.

Documentation

Summary: One long-standing concern with ME-EPI denoising has been the availability of documentation for the method outside of published scientific papers. To address this, we have created a ReadTheDocs site; however, there are still several sections either explicitly marked as “#TODO” or otherwise missing crucial information.

We are committed to providing helpful documentation for all users of tedana. One metric of success, then, is to develop documentation that includes:

  1. Motivations for conducting echo time dependent analysis,
  2. A collection of key ME-EPI references and acqusition sequences from the published literature,
  3. Tutorials on how to use tedana,
  4. The different processing steps that are conducted in each workflow,
  5. An up-to-date description of the API,
  6. A transparent explanation of the different decisions that are made through the tedana pipeline, and
  7. Where to seek support

Associated Milestone

This milestone will close when the online documentation contains the minimum necessary information to orient a complete newcomer to ME-EPI, both on the theoretical basis of the method as well as the practical steps used in ME-EPI denoising.

Transparent and reproducible processing

Summary: Alongside the lack of existing documentation, there is a general unfamiliarity with how selection criteria are applied to individual data sets. This lack of transparency, combined with the non-deterministic nature of the decomposition, has generated significant uncertainty when interpreting results.

In order to build and maintain confidence in ME-EPI processing, any analysis software—including tedana—must provide enough information such that the user is empowered to conduct transparent and reproducible analyses. This will permit clear reporting of the ME-EPI results in published studies and facilitate a broader conversation in the scientific community on the nature of ME-EPI processing.

We are therefore committed to making tedana analysis transparent and reproducible such that we report back all processing steps applied to any individual data set, including the specific selection criteria used in making denoising decisions. This, combined with the reproducibility afforded by seeding all non-deterministic steps, will enable both increased confidence and better reporting of ME-EPI results.

A metric of success for tedana then, should be enhancements to the code such that:

  1. Non-deterministic steps are made reproducible by enabling access to a “seed value”, and
  2. The decision process for individual component data is made accessible to the end user.

Associated Milestone

This milestone will close when when the internal decision making process for component selection is made accessible to the end user, and an analysis can be reproduced by an independent researcher who has access to the same data.

Testing

Summary: Historically, the lack of testing for ME-EPI analysis pipelines has prevented new developers from engaging with the code for fear of silently breaking or otherwise degrading the existing implementation. Moving forward, we want to grow an active development community, where developers feel empowered to explore new enhancements to the tedana code base.

One means to ensure that new code does not introduce bugs is through extensive testing. We are therefore committed to implementing high test coverage at both the unit test and integration test levels; that is, both in testing individual functions and broader workflows, respectively.

A metric of success should thus be:

  1. Achieving 90% test coverage for unit tests, as well as
  2. Three distinguishable integration tests over a range of possible acquisition conditions.

Associated Milestone

This milestone will close when we have 90% test coverage for unit tests and three distinguishable integration tests, varying number of echos and acquisition type (i.e., task vs. rest).

Workflow integration: AFNI

Summary: Currently, afni_proc.py distributes an older version of tedana, around which they have built a wrapper script, tedana_wrapper.py, to ensure compatibility. AFNI users at this point are therefore not accessing the latest version of tedana. We will grow our user base if tedana can be accessed through AFNI, and we are therefore committed to supporting native integration of tedana in AFNI.

One metric of success, therefore, will be if we can demonstrate sufficient stability and support such that the afni_proc.py maintainers are willing to switch to tedana as the recommended method of accessing ME-EPI denoising in AFNI. We will aim to aid in this process by increasing compatibility between tedana and the afni_proc.py workflow, eliminating the need for an additional wrapper script. For example, tedana could directly accept BRIK/HEAD files, facilitating interoperability with other AFNI pipelines.

Associated Milestone

This milestone will close when tedana is stable enough such that the recommended default in afni_proc.py is to access ME-EPI denoising via pip install tedana, rather than maintaining the alternative version that is currently used.

Workflow integration: BIDS

Summary: Currently, the BIDS ecosystem has limited support for ME-EPI processing. We will grow our user base if tedana is integrated into existing BIDS Apps and therefore accessible to members of the BIDS community. One promising opportunity is if tedana can be used natively in FMRIPrep. Some of the work is not required at this repository, but other changes will need to happen here; for example, making sure the outputs are BIDS compliant.

A metric of success, then, will be:

  1. Fully integrating tedana into FMRIPrep, and
  2. Making tedana outputs compliant with the BIDS derivatives specification.

Associated Milestone

This milestone will close when the denoising steps of tedana are stable enough to integrate into FMRIPrep and the FMRIPrep project is updated to process ME-EPI scans.

Method extensions & improvements

Summary: Overall, each of the listed deliverables will support a broader goal: to improve on ME-EPI processing itself. This is an important research question and will advance the state-of-the-art in ME-EPI processing.

A metric of success here would be * EITHER integrating a new decomposition method, beyond ICA * OR validating new selection criteria.

To achieve either of these metrics, it is likely that we will need to incoporate a quality-assurance module into tedana, possibly as visual reports.

Associated Milestone

This milestone will close when the codebase is stable enough to integrate novel methods into tedana, and that happens!

Developing a healthy community

Summary: In developing tedana, we are committed to fostering a healthy community. A healthy community is one in which the maintainers are happy and not overworked, and which empowers users to contribute back to the project. By making tedana stable and well-documented, with enough modularity to integrate improvements, we will enable new contributors to feel that their work is welcomed.

We therefore have one additional metric of success:

  1. An outside contributor integrates an improvement to ME-EPI denoising.

Associated Milestone

This milestone will probably never close, but will serve to track issues related to building and supporting the tedana community.