AnVIL users: dbGaP authorization is still applicable to access AnVIL datasetsYou do not have to use DUOS to access controlled data. To enable dbGaP access, link your NIH account in Terra (enables access to controlled data granted by dbGaP in both Terra and the AnVIL Data Explorer). For step-by-step instructions, see Access controlled-data by linking your NIH account in Terra.

Step 1: Request access on DUOS​

1.1. Go to duos.org and click sign-up/sign in using MSFT or Google SSO. Use the same login you use for app.terra.bio. 

1.2. Accept the Terms of service.

1.3. In the Researcher Console, select Your Profile under your name (top right).

1.4. Once you land on the profile page, add your full name and select your institution from the dropdown (start to fill in the name). 

1.5. Link your eRA Commons ID. This will allow you to request access to controlled data.

You’ll be taken to the external NIH page to sign into your eRA Commons account.

1.6. Invite your signing official to duos.org to register and give you a library card. See How to pre-authorize researchers to submit Data Access Requests (DARs) in DUOS for instructions.

1.7. Once you receive a Library Card, login to the DUOS Researcher Console. You should see your library card in the Profile page.

1.8. Go to the Data Library (https://duos.org/datalibrary) and filter/search for the dataset you’re looking for. Note that the example dataset in this doc is GTEx, but the instructions are not specific to that dataset.  

1.9. Select the dataset of interest (circled in screenshot) and click apply for access at the bottom right.

1.10. Fill out your data access request form, with as much detail as possible.

1.11. Make sure to attest to all terms in the library card agreements (screenshot below) by clicking the blue Attest button.

1.12. Submit your DAR by clicking the blue Submit button (bottom left) beneath the addendum summarizing the requested datasets grouped by data use terms.

What to expect​

It will take a few days to a few weeks for your DAR to be approved. You will get a notification in your email when you have access to the dataset.

Step 2: Export snapshot to Terra from the DUOS Library​

2.1. Once your DAR is approved, return to the DUOS Data Library, where you should now see a blue Export link in the View by Dataset tab under Export to Terra (right column) .

2.2. Clicking the button should allow you to export to an existing Terra workspace or to create a new one.

2.3. Note that if you’re accessing controlled data, DUOS will automatically enable additional security on your new or existing workspace. 

What to expect​

DUOS will import the data snapshot to a new or existing workspace.

It will take a few minutes to export the snapshot. You’ll get a green popup (upper right) when data is in your workspace.

Once you refresh your page, you’ll see the data tables containing all the snapshot data and metadata in the Data tab of your workspace. Note the security shield at the top right indicating additional security monitoring.

NIH Repositories: A Landscape Analysis​

See the full article here: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=5374625

Workshop Reference Graphs​

Total Number of Repositories by Access type
---
Total Number of Repositories by Data type

The NIH-Sponsored Repository list will be valuable to principal investigators (PIs)​

1. Those who are looking for secondary data to use for their research.
2. Those who need to identify a repository to submit data to, potentially as a part of their Data Management and Sharing (DMS) plan.

Additional Notes

• The preprint SSRN paper (QR code on reverse) provides an overview of the NIH-sponsored repositories available and can serve as a good starting point in the search for a repository to submit to or to access data for secondary research.
• When submitting data to a repository as part of NIH requirements, identify a repository and contact them early in the grant lifecycle, as you may need to vet multiple repositories and ensure schema alignment with the final chosen repository, and more.

NIH-Supported Scientific Repositories:

https://www.grants.nih.gov/new-to-nih/information-for/researchers/resources-for-research

Steps to Get a Library Card:​

1. Find out who your Signing Official is. If you don’t know, its usually best to ask your compliance office, office of sponsored research, or legal department.
   If your institution has already designated one in DUOS, they can issue you a card directly.
2. Ask your Signing Official to log into DUOS.
   They can log in at https://duos.org/so-console and issue you a Library Card from their dashboard.
3. Wait for confirmation.
   Once the Library Card is issued, you’ll receive a confirmation email. At that point, you’ll be able to initiate and submit DARs and Progress Reports in DUOS.

If your institution does not yet have a Signing Official in DUOS, please reach out to your research administration team or contact the DUOS support team for help getting one onboarded.

This comprehensive study maps out the current NIH-supported data repository ecosystem—spanning roughly 150 domain‑specific and generalist repositories—and evaluates gaps and usability. It provides critical insights into how NIH data is organized, discoverable, and maintained across different platforms, while identifying challenges around access, metadata standardization, and sustainability.

The paper concludes with a forward-looking “path beyond the mountains,” recommending strategic improvements such as enhancing metadata harmonization, strengthening repository integration, and leveraging FAIR and GA4GH standardsto bolster discoverability and ease of datasharing and use across NIH-funded datasets .

The table below provides a quick overview of how Progress Reports support each distinct role within the DUOS ecosystem, highlighting their primary functions in managing data access lifecycles.

For Researchers: Managing Your Data Access and Project Status​

When to Submit a Progress Report:

Once 11 months have elapsed from the initial approval of your Data Access Request (DAR), you, as the primary investigator (PI), will receive a notification prompting you to submit a Progress Report. This report is crucial for updating the Data Access Committee (DAC) on your research progress and for managing the continuity or conclusion of your data access.  

How to Submit a Progress Report:

Under Actions, click Update

This will bring you to the progress report form – fill out all red required fields and any relevant optional fields.

Then hit submit at the bottom of the page after Step 5 to send your progress report to all relevant DAC(s).

Required Content for Progress Reports (for Renewal):

When submitting a Progress Report for the purpose of renewing your data access, you are required to provide the following information:

• Summary of Research Progress: Detail the advancements made in your research since the initial DAR was approved.  
• Intellectual Property (IP) Disclosure: Disclose any new intellectual property that has resulted from your research using the accessed data.  
• Publications: List any publications (e.g., journal articles, preprints) that have been published or submitted since the initial DAR, utilizing the accessed data.  
• Presentations: Detail any presentations (e.g., conference talks, posters) given since the initial DAR that feature findings from your research using the accessed data.  

Optional Uses for Progress Reports:

Beyond the required renewal updates, Progress Reports offer flexibility for additional communication and project management:

• Update Internal Lab Staff: Share your research progress with your internal lab team.
• Update Internal or External Collaborators: Keep your collaborators informed about the project’s status.
• Report a Data Management Incident: If a data management incident occurs, you can use the Progress Report to formally document and report it.
• Close Out the Project: If your research project is complete or no longer requires access to the data, you can initiate a project closeout through the Progress Report.

Project Closeout Process for Researchers:

If you choose to close out your project via a Progress Report, the approval flow differs from a renewal:

1. Submission to Signing Official (SO): Instead of the DAC voting on approval, you will submit the closeout request directly to your institutional Signing Official (SO) for their review and approval.
2. SO Approval: Your SO will review the closeout request and, if approved, it will be sent to the DAC Chair(s).
3. DAC Chair Approval: After your SO’s approval, the DAC Chair must provide the final approval for the project closeout.

By utilizing Progress Reports, you gain clear visibility into your data access status, streamline compliance, and efficiently manage your research project’s lifecycle within DUOS.

For Signing Officials (SOs): Strategic Oversight and Closeout Approvals​

Approving Project Closeouts:

When a researcher initiates a project closeout through a Progress Report, your role as an SO is critical for institutional approval:

Review and Approval: The researcher will submit the closeout request directly to you, their institutional Signing Official, for initial review and approval by indicating you are their signing official on the closeout form. You will receive an email with a link to the progress report containing a closeout for you to review and approve. The associated action is “Review Closeout”:

Clicking Review Closeout brings you to a read only version of the Progress Report(s) and original DAR, underneath the most recent progress report will be an option to approve the closeout. 

Clicking Approve closeout moves the progress report to the DAC Chair(s) for final approval.

For Data Access Committees (DACs): Reviewing Progress and Approving Closeouts​

Reviewing Progress Reports for Renewal:

When a researcher submits a Progress Report for the purpose of renewing their data access, your role as a DAC remains focused on reviewing the submitted information to ensure continued compliance with data use limitations and consent forms. Progress Reports will provide you with updated information from the researcher to understand their progress on their research project, supporting record-keeping and consistency in the review process while preserving the link to the original Data Access Request.  

You will receive an email that a Progress Report has been submitted for review if you’re a chair, or open for voting if you are a DAC member (after your Chair has opened the Progress Report). Voting proceeds as it would for any other DAR.

Approving Project Closeouts:

The process for approving a project closeout initiated via a Progress Report differs from a standard DAR review:

 You will receive an email with a link to the progress report containing a closeout for you to review and approve after it has been approved by the Researcher’s SO. The associated action is “Review Closeout”:

Clicking Review Closeout brings you to a read only version of the Progress Report(s) and original DAR, underneath the most recent progress report will be an option to approve the closeout. 

Clicking Approve closeout moves the progress report and DAR collection to a closed state, completing the closeout process.

Designed as a companion to the existing GA4GH Passports standard, this proposal outlines how institutions and data access committees can issue trusted assertions about a dataset. This approach reduces administrative burden and supports scalable, federated data governance. The paper also highlights future directions, including trust frameworks and expanded visa types, to further advance responsible and efficient data sharing.

Why ‘Progress Reports’?​

When using controlled-access data, access is typically granted for one year by a data access committee (DAC). To help researchers extend their data access as needed, assist DACs in overseeing and enabling data use, and share details on the important research progress by DUOS users, DUOS introduces Progress Reports.  

Before Progress Reports, researchers in DUOS needed to make a new Data Access Request to extend data access, which put redundant administrative work on researchers and left DACs with less than desirable data linkage and research output information. Now with Progress Reports in DUOS, access renewals, research updates, publications, presentations, collaborator changes, data management incidents and closeouts linked to the original DAR in a single auditable and shareable provenance chain – lightening the work on researchers and DACs to share data and research results faster! 

How do Progress Reports benefit different users in DUOS?​

When Should Researchers Submit a Progress Report?​

In DUOS, your access to data via a DAR will expire 12 months after initial approval. DUOS will send the PI on the DAR a reminder to complete a Progress Report, to either renew access or closeout the DAR, 30 days prior to the DAR’s expiration. If you do not submit and receive approval for your Progress Report before the expiration, your access will suspended until a Progress Report is approved, or you submit a new DAR for the data. 

Additionally, researchers can submit Progress Reports at-will to update DACs of exciting developments in their research, including sharing information about presentations, publications, or intellectual property developed in the course of their research. This information will be shared publicly and promoted by DUOS to help draw attention to your research!

We hope you enjoy this new feature and we look forward to sharing your research with the world!​

We’ve added powerful new search filters that help you quickly narrow down datasets by the criteria that matter most. You can now filter data by:

• Data Use & Access terms – Find datasets that align with specific research purposes or consent requirements
• Data Access Committees (DACs) – View datasets stewarded by specific DACs for easier coordination and transparency
• Data Types – Focus your search on genomics, phenotypic, clinical, or other data types relevant to your research

These filters make it faster to discover the right data, streamline your access workflows, and ensure you’re working with datasets that meet your project needs.

Try the new filters today—log in to DUOS and start exploring with precision!

If you are using Terra or NIH Platforms like AnVIL that are powered by Terra, you are already compliant.

The update requires researchers and institutions using NIH controlled-access data to attest that  systems interacting with this data are compliant with NIST SP 800-171. Many university and institutes’ IT departments are urgently determining remediation pathways for their internal clusters and systems, including what new policies and training will need to be rolled out to their staff to be compliant by January. While many institutions are rapidly working to identify, invest, and implement critical security upgrades to meet NIST SP 800-171 compliance, it is vital that IT and compliance departments can inform researchers about which systems meet these security criteria.

Luckily, there is a solution: Terra and NIH platforms like AnVIL that are powered by Terra already meet NIST SP 800-171 as well as the more rigorous NIST SP 800-53 (FedRAMP Moderate). IT and compliance departments can direct users to Terra and AnVIL, and simply acknowledge use of these systems in the attestation process. It’s as easy as that!

If you have any questions or would like to discuss your specific situation, please contact support@terra.bio. We would be glad to assist you.

With this feature, you can:

• Design a branded landing page for your data collections
• Curate datasets to highlight key themes or research areas
• Guide users directly to data that matters most to your community

Whether you’re amplifying your research impact or simplifying access to your data resources, this new capability makes it easier than ever to connect researchers with the data they need.

Check out some examples on the DUOS homepage, and if you’re interested in creating a branded Data Library in DUOS to showcase your research and share your data contact us via the homepage to let us know!

Data access for researchers is critical to modern scientific endeavors. But new data access requests are hampered by an often cumbersome and manual process. Now, DUOS (Data Use Oversight System) automatically updates users’ access permissions, simplifying and expediting the process for researchers, ensuring they can swiftly and securely access the data they need. In this blog post, we explore the benefits and implications of the new automatic access permission update feature in DUOS, and link to resources to leverage DUOS to streamline data access.

The Current Data Access Landscape​

Data access committees (DACs) and Data Custodians play a crucial role in managing the flow of valuable research data, ensuring the data is used ethically and responsibly. Traditionally, researchers had to manually request updates to their access permissions from the DAC when they were granted secondary access to new datasets. DACs, the centerpiece of the approval process, were overwhelmed by exponentially increasing amounts of data and data requests. The cumbersome process often led to delays and hindered the timely advancement of research initiatives.

DUOS’ Automatic Access Permission Updates​

DUOS has emerged as a leading software platform for facilitating data access and management. Now, DUOS automatically updates researchers’ access permissions when they are approved for secondary access to requested research data. This eliminates the need for manual intervention, allowing researchers to seamlessly access the approved data without any delays. By streamlining the data access process, DUOS empowers researchers to focus on their work rather than navigating administrative barriers.

How Automatic Access Updates Benefit Researchers​

• Expedited research process, as scientists can access new data promptly upon approval. 
• Automation reduces the administrative burden on researchers, freeing up more time for analysis and experimentation
• Increased efficiency ensures that researchers can focus on their science

Strengthening Data Security and Privacy​

DACs and Data Custodians know manual access permission updates carry an inherent risk of human error or data breaches. DUOS’ automated permission updates ensure that access permissions are updated accurately, promptly, and in compliance with privacy regulations, reducing the chances of data mishandling, and safeguarding sensitive information.

Conclusion​

DUOS’ automatic access permission update marks a significant milestone in the realm of data access and management. By simplifying the process and enhancing security, researchers can focus on their core scientific pursuits and data custodians can ensure efficient, broad and secure access to their datasets. As DUOS continues to innovate and streamline data access, it promises to revolutionize the landscape of research, empowering scientists worldwide in their quest for knowledge and breakthrough discoveries.

DACs and Data Custodians can learn more about how to leverage this new feature here.

A major bottleneck to scientific impact​

Genomic data for human participants is highly identifiable and therefore often categorized as controlled-access data by virtue of the data sharing language in its consent forms. This means access to the data must be regulated by a data access committee (DAC) who assures the data will be shared and used in accordance with the data use terms in the consent forms.

Therefore, researchers must apply for access to such datasets through their designated DAC. In turn, the DAC must review the application to ensure that the researcher’s intended use of the data aligns with the permitted uses of the data, per the participants’ consent. Unfortunately, since the DACs receive language describing permitted data use and intended use from different parties (IRBs/participants vs. researchers) who use unique and sometimes ambiguous language, they are left with an apples:oranges comparison, which is difficult for them to decipher.

Current data access processes rely on unique, custom narrative text to describe data uses in consent forms and the datasets they represent, as well as in data access requests for those datasets. This makes adjudicating requests difficult if not impossible for DACs in certain cases.

The added weight of the liability involved in granting someone access to data under uncertain terms often leads DACs to undershare rather than overshare, meaning data is unnecessarily locked down, delaying or blocking significant scientific impact entirely. 

Facilitating data access for researchers, by enhancing DAC workflows​

To address this issue and others like it, the Broad Institute developed DUOS (Data Use Oversight System), a suite of software services and policies for managing data access and sharing (more detail here). Principally, DUOS aims to consistently and transparently answer the overarching question posed to DACs: “Does the researcher’s intended use align with the permitted use of the dataset?”

In fact, DUOS aims to do this in a semi-automated fashion by leveraging the GA4GH Data Use Ontology (DUO). The GA4GH DUO provides both human-readable and machine-readable terms and definitions for data use. DUOS utilizes the GA4GH DUO to tag datasets with their permitted use, and data access requests with their intended use in human-readable and machine-readable terms. This standardization allows for DACs and the DUOS matching algorithm to easily determine if a researcher’s intended use aligns with a dataset’s permitted uses in a more apples:apples format, and allows for DUOS to leverage a matching algorithm to suggest a decision to the DAC. The DUOS team is working to further increase the accuracy of the algorithm in making a decision that aligns with that of the DAC. 

The GA4GH DUO allows for standardizing data use in consent forms and the datasets they represent, as well as in data access requests for those datasets. With data use language expressed in standardized and machine-readable terms, adjudicating requests is much easier for DACs and even possible to automate algorithmically.

While this automated matching can alleviate a serious bottleneck, other hurdles and data access and sharing remain. We aim to address those concerns in future articles and through our work on DUOS. In the meantime, please visit our website at www.duos.org for more information and detailed documentation. Thanks for reading and we hope DUOS serves you well!

In response to recent policy initiatives emphasizing the importance of data sharing and the growing need for support from smaller institutions unable to establish their own committees, Broad Institute’s executive committee has made a forward-thinking decision to allow external submissions of genomic data to its Data Access Committee. This groundbreaking development will enable researchers, institutions, and organizations to benefit from our institution’s exceptional resources and knowledge.

By opening the doors to external data submissions, Broad Institute aims to foster a culture of collaboration and innovation in genomic research. This initiative represents a significant opportunity for institutions of all sizes to tap into our institution’s existing system, policies, and wealth of experience. The Data Access Committee will now play a crucial role in facilitating access to not only internally generated datasets but also curated external data.

“Our institution firmly believes in the power of collaboration and knowledge sharing to advance scientific discovery,” said Stacey Donnelly, the Chair of the Broad Institute’s Data Access Committee. “By expanding our Data Access Committee, we are extending an invitation to researchers and institutions worldwide to benefit from our comprehensive data governance framework. Together, we can drive breakthroughs in genomic research and accelerate progress in the field.”

Researchers and institutions interested in submitting their genomic data to Broad Institute’s Data Access Committee are encouraged to write to support@duos.org for detailed guidelines and the submission process. The Committee will meticulously evaluate each submission, ensuring adherence to the highest ethical standards and data protection protocols.

With this bold step, Broad Institute reinforces its commitment to advancing scientific knowledge, fostering collaboration, and promoting equitable access to genomic data. By embracing this innovative approach, the institution aims to catalyze breakthrough discoveries that will benefit humanity as a whole.

Yes, there is a veritable cornucopia of data out there now, but most of the human data is subject to complex access restrictions that can be a huge pain to deal with. To be clear, I’m not talking about basic information security like checking identity and credentials. I’m referring to restrictions on future use and sharing of the data based on the consent of study participants – generally established through study-specific consent forms. These restrictions exist for legitimate reasons, but the way they’ve traditionally been implemented leaves much to be desired.

Data use restrictions and the data access dating game​

Due to the historical lack of standardized consent forms, each study typically uses unique language to describe their data use restrictions. There are usually no templates or standards for researchers who seek to request access to the data; they must describe in their own words what kind of research they intend to do with the data, at times not even knowing which data use restrictions apply.

Who, then, is responsible for granting or denying access to the data? That gatekeeping role belongs to a Data Access Committee (DAC), who is tasked with interpreting the data use requests from researchers and evaluating whether they match or conflict with applicable restrictions. Given the lack of consistency in how data use restrictions are formulated across the biomedical ecosystem, it can be challenging for DAC members to make these decisions with confidence or consistently, so the process takes time and effort. In addition, there’s a lot of variability in the composition and governance of these committees. Some DACs are part of large organizations like NIH and have a formal structure, dedicated staff, and a regular cadence for reviewing access requests, but many within research institutions are simply composed of research faculty members who are expected to perform these duties on top of their full-time job.

To be fair, this cumbersome system predates the advent of cloud-based data sharing, and is just as much of an obstacle in the context of traditional data sharing systems. What’s new is that cloud-based initiatives are contributing to a sharp increase in the number of researchers who are able to find datasets of interest and request access to data. As a result, DACs are increasingly unable to respond to the onslaught of requests they are receiving. When it comes down to it, human committees are simply not going to be able to scale.

Introducing the GA4GH Data Use Ontology… and using it to automate access​

To tackle this challenge, the Global Alliance for Genomics and Health (GA4GH) Data Use and Researcher Identities (DURI) workstream developed an ontological standard for defining data use restrictions. The Data Use Ontology (DUO) provides a controlled vocabulary that data generators can use to formulate data use restrictions and that researchers can use to express their intended purposes when they apply for access.

In addition to solving the challenge of consistent interpretation of data access requests, DUO can also greatly enhance the searchability of access-restricted datasets. By tagging datasets with their usage restrictions, we can enable search algorithms to take into account whether the researcher is likely to be approved to use the data based on their stated research purpose. This empowers researchers to filter out data that would be out of bounds to them anyway, which is a huge time-saver given the increasingly large numbers of available datasets.

Taking this idea one step further, we can move from consistency in the human review of data access requests to automating the data access decision-making process entirely — our team built the Data Use Oversight System (DUOS) to do just that. Using DUOS, you can search for datasets with computer-readable data use restrictions, then apply for access to those datasets with a system-generated, computer-readable data access request. Having computer-readable DUO codes for both the data use restrictions and the data access request, DUOS is able to algorithmically evaluate whether or not the researcher should be granted access to the data!

What is DUOS?​

• Interfaces to transform data use restrictions and data access requests to machine-readable code (GA4GH Standard)
• A matching algorithm that checks if data access requests are compatible with data use restrictions
• Interfaces for the Data Access Committee to adjudicate whether structuring and matching has been done appropriately

For more  details:​

If you’d like to learn more about this powerful duo of ontology and automation, watch the recording of this past GA4GH webinar: Automating access to controlled datasets: the GA4GH Data Use Ontology in action, here on their YouTube channel. I presented our team’s DUOS implementation and talked about how researchers can already use it to access real datasets, and you’ll also hear more about the history of DUO and the work of several other implementers of DUO including the GEnome Medicine Alliance Japan (GEM Japan), Australian Genomics Health Alliance (AGHA), European Genome/Phenome Archive (EGA), DNAStack, Elixir, and the Wellcome Sanger Institute.