FoodAPS was established to encourage the development and dissemination of independent academic research in U.S. institutions on the determinants and consequences of American households’ food choices. The primary goal is to enable carefully-conducted research to inform the general public and policymakers about critical and concurrent food policy issues facing the Nation.
Restricted-use data files have been posted to a data enclave at the National Opinion Research Center (NORC), an independent research organization at the University of Chicago. This page provides the following information about restrictions on data use and how to gain access to these restricted data files:
- Data confidentiality and restricted availability
- Who can access FoodAPS data and how can can this be done?
- Procedures for requesting access to FoodAPS data
- Review of proposed projects
- Amending an existing project agreement
- Review of analytic output
- Data access forms
Because FoodAPS data include confidential information, ERS has a legal obligation to provide access under a controlled environment to prevent accidental or willful disclosure of information that might identify a survey respondent. To develop rich and high-quality data, FoodAPS invoked the protections and regulations of the Confidential Information Protection and Statistical Efficiency Act (CIPSEA) of 2002 to obtain confidential information. CIPSEA requires that the collected data be used strictly for research and statistical purposes and promises respondents high levels of data protection against disclosure of identifying information. Penalties for a violation of CIPSEA procedures can result in a fine of up to $250,000 and/or five years in prison. The restrictions and protections against disclosure of confidential information follow guidelines developed by the Office of Management and Budget (OMB). See Implementation Guidance for Title V of the E-Government Act, Confidential Information Protection and Statistical Efficiency Act of 2002 (CIPSEA).
All researchers, including ERS staff, must be CIPSEA-trained and sign a pledge of confidentiality before being granted access to restricted data. Furthermore, all research results must be reviewed for disclosure risk and approved by the ERS Confidentiality Officer or his/her designee before the results can be shared with anyone who has not signed an ERS confidentiality agreement that specifically relates to the FoodAPS data. The review and approval process for disclosure risk is based solely on preventing the release of confidential information and ensuring compliance with the project's research plan (see Review of analytic output for more information).
In addition, FoodAPS relies extensively on extant proprietary data to identify detailed descriptions of reported food items as well as to define respondents’ local food environments, including the characteristics of specific grocery stores and restaurants in the local areas, along with prices of available foods in the stores. All approved researchers will have access to general food item descriptions and aggregate measures of the food environment. Access to disaggregated (raw) proprietary data sources, such as item- and store-level descriptions and sales information in local food environments, is limited to researchers collaborating on USDA-sponsored projects. USDA-sponsored projects include ERS grants, ERS cooperative agreements, and direct collaboration with ERS researchers. If a sponsored research project requires access to a specific proprietary data source in FoodAPS, the institution with which the research collaborator is affiliated must enter into a Third Party Agreement (TPA) with the data provider.
ERS has established an efficient and secure data enclave at NORC that meets the Federal Information Security Management Act (FISMA) and CIPSEA security requirements, where access to external users is provided only through remote computer terminals available only from NORC. For more information about leasing NORC's remote devices, see step 6 in Procedures for requesting access to FoodAPS data). Users will cover the cost of accessing the data with their own funds.
Minimizing the risk of disclosure
Five procedures have been established to minimize the risk of disclosure:
- The remote terminals used to access the NORC Data Enclave and manipulate data within the enclave are not capable of any other functions. Researchers cannot download any materials from the enclave, and they must agree to use the terminals properly in secure locations.
- Different sets of FoodAPS data and proprietary data are stored in separate computer directories, and each directory has its own access restrictions. When researchers start a FoodAPS project, they are granted access only to the directories that contain data needed for their authorized research project.
- NORC has the ability to monitor or audit researchers’ current or past use of data while in the enclave.
- When researchers have draft results and materials ready to be removed from the data enclave’s secure environment, those materials must first be sent to NORC to request permission to export the materials. NORC staff will review the materials for possible disclosure risk. If the risk is deemed small enough to be acceptable, NORC notifies ERS, and the ERS Confidentiality Officer or his/her designee will review the materials for disclosure risk. The ERS review also includes checking that the requested output meets all other stipulations of the approved Project Agreement. Only after both NORC and ERS have determined that the material requested for export does not risk disclosure of confidential information is permission granted for the researcher to remove the materials from the enclave.
- As specified in their approved Project Agreements, researchers agree to submit all drafts, final reports, or other materials to ERS for disclosure review and approval before circulating those materials to anybody without a signed Confidentiality Agreement covering use of the FoodAPS data. The analytical conclusions of reports and outputs that are cleared for release are within the purview of the project research team and the accredited institution and not ERS. Further details are provided under the section, "DISSEMINATION AND DISCLOSURE REVIEW OF INFORMATION" in the MOU and/or the Project Agreement.
The CIPSEA protection stipulates the agency’s right to conduct inspections of off-site facilities where data access is allowed. The organization agrees to allow ERS officials to carry out unannounced physical and IT security inspections of the organization’s workplace. ERS's standard workplace security checklist (ERS workplace security checklist) will be completed at the time of inspections.
In defining disclosure risk, ERS will adhere to the guidelines and recommendations presented in OMB’s Statistical Policy Working Paper 22: Report on Statistical Disclosure Limitation Methodology.
Restricted FoodAPS data are only available to approved researchers. To gain approval, a research proposal must be submitted for review and approved by the FoodAPS Research Advisory Committee. The committee will approve proposals only from qualified researchers who are conducting academic research and affiliated with accredited colleges and universities in the United States, other non-profit U.S. organizations that conduct research, or research groups in U.S. government organizations. To be approved, proposals must:
- Define the explicit economic hypotheses and questions to be addressed;
- Demonstrate the relevance of the proposed research to the broad purposes for which FoodAPS data were developed;
- Establish the merits and contributions of the proposed research to the academic literature and to the specific information needs of policymakers;
- Explain the research methodology, including plans to account for FoodAPS’s complex sample design to make inferences;
- Describe how the analytic results will be presented, including planned table shells, charts, and figures, as well as planned peer-reviewed publications; and
- Specify which groups of data elements are needed.
The research proposal must be submitted as part of the ERS Project Agreement.
The following six steps are required before access to restricted FoodAPS data in the NORC Data Enclave will be approved:
Step 1. Submit a project description/proposal for approval using a Project Agreement Form.
Step 2. Complete an ERS Memorandum of Understanding (MOU) between ERS and your organization.
Step 3. Complete the USDA web-based CIPSEA training course (required of all project members seeking access to the secure data enclave).
Step 4. Sign the ERS Confidentiality Agreement (required of all project members seeking to access the secure data enclave).
Step 5. Complete a confidentiality agreement between the user and NORC and establish an account at NORC (user fees apply).
Step 6. Lease special thin-client devices from NORC (user fees apply).
Each step is described in greater detail below:
Step 1. For each distinct research project, a Project Agreement form must be submitted to describe the proposed research and to obtain approval.
The review process is designed to:
- Ensure that the project has a defined research question that addresses a purpose for which FoodAPS data were collected and for which the FoodAPS data are appropriate (see Background).
- Determine what restricted data elements are needed to complete the project. (Note: All FoodAPS data maintained at the NORC Data Enclave are CIPSEA-protected and restricted. The data elements vary, however, in terms of their degree of confidentiality and likelihood of posing a disclosure risk. ERS has therefore organized the FoodAPS data into seven groups, and permissions to access the data will be group-specific based on data needs of the approved Project Agreement.)
- Assess disclosure risk based on requested restricted variables and the types of analyses and outputs proposed.
- Determine the required software and potential need for external data sets. (Note: Many different software packages are available within the NORC Data Enclave to approved researchers. If a needed package is not currently available, ERS will notify the research team. The research team may then contact NORC to determine if the package may be uploaded to the enclave.)
Within the Project Summary section of the Project Agreement form, the following information should be provided:
- An abstract of no more than 250 words that describes the proposed research. The project titles, principal investigators, and abstracts of approved projects will be posted on the ERS website (see Research Projects and Publications);
- A description of the research question(s) to be investigated, the statistical technique(s) to be used, and the software to be used;
- A list of data elements needed for the planned analysis;
- A description of any household or individual subgroups (for example, households with children or all school-age children) that will be a focus of the research;
- Plans for using sample weights and procedures that will be used to account for the complex sample design used for FoodAPS data;
- Table shells and charts that show how research results are expected to be presented;
- Plans for preventing accidental disclosure of confidential information beyond the confines of the approved project members;
- A schedule for data analysis, draft of research results, and submission of research papers or reports;
- Names and curricula vitae or resumes of all project members who will need direct access to the restricted data; and
- Names and curricula vitae or resumes of all other project members, including advisors and consultants.
Step 2. Complete an ERS Memorandum of Understanding (MOU) between ERS and your organization, which grants access to FoodAPS data exclusively for statistical purposes under a pledge of confidentiality. The MOU defines the parameters that guarantee confidentiality and privacy, and protects the intellectual property of the data owners. The MOU defines the scope and process of ERS reviews and also serves to explicitly include the researcher’s accredited institution as a party to the overall agreement between ERS and the researcher.
The term of the MOU is for two years, and it may be extended. If project members are from multiple organizations, separate MOUs must be completed with each organization if the members plan to have direct access to the data enclave at NORC. The MOU(s) and Project Agreement should be submitted simultaneously to the ERS Confidentiality Officer for final review and signature. Copies of the fully signed paperwork will be returned to the researcher, completing the proposal approval process. If access to proprietary data is planned, the researcher must enter into a TPA with each proprietor before access to their data will be authorized.
Step 3. Successfully complete the USDA web-based CIPSEA training course. This legal requirement applies to all research team members who will access the data, develop or review interim tables and charts displaying research results, or draft any written material based on analysis of FoodAPS data. Once a proposal is approved, send team member names, addresses, phone numbers, and email addresses to the ERS Data Custodian to sign up for the online training on USDA's Aglearn website. CIPSEA training must be renewed annually.
Step 4. Once the CIPSEA training has been certified as successfully completed, each team member must sign the ERS Confidentiality Agreement form. Again, this applies to all team members who will access the data, develop or review interim tables and charts displaying research results, or draft any written material based on analysis of FoodAPS data. The confidentiality agreement must be renewed annually.
Step 5. Once the MOU, Project Agreement, and confidentiality agreements have been executed, access to the restricted-use FoodAPS data may be obtained via a secure data enclave managed by NORC at the University of Chicago.
NORC requires two signed documents for access to the data enclave; these documents establish a confidentiality agreement between the user and NORC. In addition, NORC requires all ERS-approved researchers to participate in a short online training course to learn how to access and manipulate data within the enclave, to learn about the processes involved in importing and exporting files, and to learn the rules involved in the statistical disclosure review process.
Step 6. Researchers with approved project agreements will need to lease special thin-client devices from NORC that permit access only to the data enclave. The annual cost for one (lead) researcher to access the data at NORC is $5,200 per year. The cost for additional thin-client devices for other approved members of the research team is currently $850 per device. More than one approved researcher may use a single device.
A site visit by the ERS Security Officer or designee may be required for offsite access using NORC. The site visit includes the elements listed in the ERS workplace security checklist.
The review is expected to take about two weeks; if not completed within this timeframe, ERS will provide an updated timeframe to the research team. ERS will examine each proposal for:
- Consistency of the specified research question(s) with CIPSEA-defined statistical use of FoodAPS data;
- Technical feasibility—will the proposed statistical approaches and requested data elements support analysis of the research questions?
- Demonstrated knowledge of how to incorporate the FoodAPS complex sample design when estimating sample characteristics; and
- Disclosure risk during the project and of all proposed statistical output and reports.
Proposals that FoodAPS data cannot adequately support will not be approved. For example, a proposal to use FoodAPS data to estimate outcomes for a single State will not be approved because the FoodAPS sample was not designed to provide results that may be generalized at the State level.
Approval will be given only to those proposals that provide suitable protection against disclosure of confidential information about FoodAPS respondents, the establishments they visited, and the food items acquired. For example, planned research output that identifies establishment locations or names, product manufacturers, or name brands will not be approved. Proposals should address how data and research results will be protected during analysis and report preparation. To help assure ERS that disclosure risk will be fully considered and addressed, careful forethought should be given to the following when preparing a proposal:
- The specific data elements needed for the proposed research;
- The approximate sample sizes of subgroups of interest. (Note: Sample sizes of some household and individual subgroups may be determined through review of FoodAPS research findings published on the ERS website. If during its review, ERS determines that sample sizes of target populations are inadequate to support the planned analyses, the Project Leader of the proposed project will be informed); and
- The way that research results will be presented in tables, charts, and figures.
The proposal should describe all output planned to be exported from the data enclave; this should be a detailed description, as this section helps ERS assess disclosure risk. Include detailed examples of table shells, models, and/or graphs. Please indicate the subsample or unit of analysis used in each type of table, model, or graph. Also indicate plans for presenting results—such as planned presentations at professional/scientific meetings or to funding organizations, working papers and reports, and publications in peer-reviewed journals.
After its review of the proposed project, ERS will decide to either approve as is, request a revision and resubmission, or disapprove.
- Approval of an application does not mean that USDA endorses the merit of the proposed research or its substantive, methodological, theoretical, or policy relevance.
- USDA approval reflects the judgment that the research described in the proposal is an appropriate use of the requested data; that is, the characteristics of FoodAPS data match the proposed research questions.
- Approval of a proposal does not explicitly or implicitly guarantee that all proposed output generated by the analysis will be approved for release.
Revise and resubmit:
- If ERS needs additional information during its review of a proposal, the research team will be asked to provide that information through a revised proposal.
- The research team should highlight any changes and indicate the date of revision when resubmitting a proposal.
- ERS will provide a written explanation of the reasons that a proposed project was not approved. If a proposal is disapproved, the research team may revise it and resubmit as a new proposal.
If, during the course of analysis, the research team decides to make any changes to its approved Project Agreement, the original Project Agreement may need to be amended. This needs to be done so that when research materials are submitted for review, ERS will have an up-to-date agreement for reference. In addition, ERS does not want researchers to spend time on analyses that could later create a disclosure risk and thereby not be publishable.
In general, if the research team wishes to make any change that affects the Project Agreement’s approved focal research objective/questions, statistical method, table shells, charts, or figures, the team should notify ERS via an e-mail message with "PROPOSED CHANGE TO AGREEMENT" in the subject line. The message should explain the proposed change and any expected changes to analytic output that would be submitted for review.
ERS will review the proposed change and, if the research focus changes or if disclosure risk is increased, ERS will ask the research team to submit a formal request to amend the agreement. The requested amendment will be fully reviewed using the same criteria as in the initial proposal review. ERS will expedite this review.
Proposed changes that are directly related to the approved research questions and pose no additional disclosure risk will not need a formal amendment to the Project Agreement, but they will be maintained on file to assist in ERS’s review of project output(s).
ERS will alert the research team if a formal amendment is needed. If the agreement does need amending, the team will need to submit a Request for Amended Project Agreement. This request should take the form of a revised Project Agreement that:
- Indicates the name and number of the original approved Project Agreement and the date of the request for an amendment on the first page.
- Includes a summary of the requested change(s).
- Describes the requested change(s) throughout the relevant sections of the proposal.
- Highlights or "tracks" all changes to the original agreement.
The review process should take one to two weeks. Amendments related to staffing changes or additional variables of similar sensitivity to those variables already approved should take less time to review. Amendments requesting access to data from groups of variables other than the groups already approved may take more time.
Before any type of output—including tables, charts, graphs, slide presentations, draft reports, and final reports—can be downloaded from the NORC Data Enclave, it will be reviewed by both NORC and ERS for both disclosure risk and adherence to outputs specified in the Project Agreement. This review requirement includes preliminary or interim results meant to be shared outside of the data enclave with CIPSEA-trained and approved colleagues on the research team.
If a review determines that disclosure risk is present, NORC or ERS will notify the research team and indicate which parts of the output pose disclosure risk. The research team and the ERS reviewer may then work together to reach agreement on output that would be approved for download from the data enclave. If there is a disagreement between the research team and the ERS reviewer as to whether any proposed output poses a disclosure risk, the research team may request a review by the ERS Confidentiality Officer. The Confidentiality Officer’s decision regarding disclosure risk will be final.
Please submit all forms to Dawn Williams on the FoodAPS team. The following forms are available for download:
- ERS memorandum of understanding or Microsoft Word
- ERS project agreement or Microsoft Word
- ERS confidentiality agreement
- ERS workplace security checklist