As a Federal statistical agency, ERS must be knowledgeable about the issues and requirements of public policy and Federal programs pertinent to the USDA mission and be able to provide objective information that is relevant to policy and program needs. The unique alignment of resources and expertise create specific capabilities to produce important and influential data products that would otherwise not exist.
Data products must be branded as coming from ERS (when they are released by ERS), as standard best practice for documentation, to more accurately measure impact and to improve ERS’s profile as a Principal Federal Statistical Agency. As applicable, data products should also cite the source of the data (such as from multiple Federal agencies).
To ensure equivalent and timely access to all users, a schedule and mode of release must be developed and publicly conveyed in the calendar year prior to the planned release of a data product.
Persons or organizations that have a vested interest in the information that is being promoted in a data product are considered stakeholders in the process. Knowing the interests, positions, alliances, and importance to ERS of key stakeholders enables data product authors to interact more effectively with these individuals and adapt to their changing needs.
Measures of content relevance and quality of communication can include on-demand requests fulfilled, product downloads, number of formats in which data are available, degree of accessibility, customer satisfaction with ease of use, results of usability testing, number of participants at user conferences, citations of agency data in the media, amount of technical support provided to data users, and exhibits and other promotional materials to inform the public about data products.
Recognizing the diversity of data users and their importance, all data products should employ a feedback/input mechanism—based on a strategy of engagement with users to help facilitate and prioritize data release.
Website Contact Forms, for example, allow for the elicitation of feedback from users in a secure and organized manner. Other forms of communication with data users include public meetings, such as those organized by NASS to alert data users on recent and pending changes in the various statistical and information programs important to agriculture and to seek comments and input on these programs. Practices to improve communication with users employed by various statistical agencies were highlighted in the 2009 CNSTAT review.
2.7 Web usage statistics for data products should be regularly reported to appropriate ERS staff (quarterly, annually, or as appropriate to the release schedule) and evaluated.
Statistics about use may provide data on how many visits, page views, bounce rate, average time on site, location, traffic sources, content sources, and percentage of new visits. This type of information can assist with priority setting and product refinement.
Objectivity is a measure of whether disseminated information is accurate, reliable, and unbiased and whether that information is presented in an accurate, clear, complete, and unbiased manner. Agencies should inform the public as to the strengths and limitations inherent in the information disseminated (e.g., possibility of errors, degree of reliability, and validity) so that users are fully aware of the quality of the information.
3.1 All data products are reviewed for data quality prior to dissemination.
Data products produced by ERS are thoroughly reviewed by knowledgeable staff prior to dissemination to verify the validity of the data. The procedure used to conduct this review must be documented and available. Data are checked for internal consistency, consistency with other similar data sets or prior year versions of the same data set, and sources of error. Knowledgeable ERS subject-matter experts conduct “reasonableness” checks of the data. Where necessary, the data are edited and missing values are imputed using established statistical techniques to improve the utility of the data.
3.2 Subject to DPC recommendation, all data products must undergo an independent external review of methodology at least every 10 years.
The breadth and extent of review will be determined by the type of data product. For example, external peer review provides a robust means of evaluating data products that employ surveys or models. Reviewers from other institutions bring to the review process independent knowledge, experience, and perspectives different from those of the data producer. For compilations of data, for example, the review might focus on the appropriateness of the data used and the clarity and adequacy of the documentation.
3.3 Where statistically appropriate, all data products must report measures of accuracy that accompany data elements.
Different types of data products might use different accuracy measures. For example, forecast error would be reported for estimates or projections, and estimates of sampling error and nonsampling error components (coverage error, measurement error, nonresponse error, and processing error), to the extent practicable, should be reported for sample survey programs. On the other hand, a data compilation can refer users to source agencies for information on data quality.
3.4 Data products should have an ongoing research program that examines methodology and operations.
For statistical agencies to be innovative and cost-efficient in methods or practices for data collection, analysis, and dissemination, research on methodology and operational procedures must be ongoing. Methodological research may be directed toward improving survey design and survey error rates, as well as developing innovative statistical methods for protecting data confidentiality. Research on operational procedures may be directed toward facilitating data collection in the field, improving the efficiency and reproducibility of data capture and processing, and enhancing the usability of Internet-based data dissemination systems.
3.5 The production process for premier data products should receive the highest priority for IT investment and must undergo an evaluation of IT approaches every 5 years.
Premier data products should reflect appropriate (e.g., modern, efficient) methods for data collection, processing, management, and dissemination commensurate with the level of importance to key stakeholders and the public.
OMB requires that Federal agencies offer a high degree of transparency about data and methodologies used to derive statistics. These requirements enable the American public maximum access to government data and ensure reproducibility of government statistics, meaning “the capacity to use the documented methods on the same data set to achieve a consistent result.” 
4.1 Decisions to initiate, terminate, or substantially modify the content, form, frequency, or availability of premier data products should trigger appropriate advance public notice.
Stakeholders and the public should be made aware of upcoming changes to premier data products by a notice on the ERS website, and where appropriate, email or other types of communication. Where appropriate, the Office of Communications should be notified directly.
4.2 All data products must be accompanied by accurate, transparent documentation that describes the source of the data, the methodology used to produce the data, definitions of the data items and variables contained in the data set, sources of error, and, if applicable, limitations of the data.
Many analytical problems and misinterpretation of data can be avoided by providing comprehensive documentation. OMB Statistical Policy Directive Number 4 states that “With the exception of compilations of statistical information collected and assembled from other statistical products, these [federal statistical] products shall contain or reference appropriate information on the strengths and limitations of the methodologies, data sources, and data used to produce them as well as other information such as explanations of other related measures to assist users in the appropriate treatment and interpretation of the data.”
OMB provides detailed guidelines for, and a comprehensive list of, necessary components to be included in survey documentation (and other types of government data to the extent they are applicable) in section 7.3 of the Standards and Guidelines for Statistical Surveys. Some sample documentation elements include a description of variables used to uniquely identify records in the data file; a description of the sample design, including strata and sampling unit identifiers to be used for analysis; and a description of sample weights, including adjustments for nonresponse and benchmarking and how to apply them.
4.3 Data products must be accompanied by a user’s guide that explains the best statistics for different purposes.
Data products on the ERS website should contain a user’s guide to explain how best to use and interpret the data. For data products that contain data that could be used for similar purposes as other data products, those products should contain—or reference another part of the ERS website that contains—a user’s guide to assist in distinguishing the best statistical series to use for the user’s intended purpose. As this represents coordination among several products, the DPC will work in consultation with ISD Web Services to formulate a strategy.
4.4 Premier data products must provide information on the update and revision history.
Data revisions can occur for a variety of reasons, including inclusion of new data or a change in the data source; seasonal adjustment and/or elimination of calendar effects; transition to a new base period; improvement of methodology due to a change in the statistical method or a change in classifications, concepts, and definitions; or elimination of errors. To ensure transparency of the revision procedure and where applicable, information should be provided that describes the revision procedure and contains information for assessment of the existing data sources and calculation methods, assessment of the quality of the new source, and assessment of the method to be applied in the revision.
4.5 All Premier data products must have an archival capability.
For purposes of reproducibility, ERS should be able to provide users with previous releases of the data product, as part of the ERS website or upon request.
“Integrity” refers to the security of information—protection of the information from unauthorized access or revision, to prevent the information from being compromised through corruption or falsification.
5.1 All data products must have a defined procedure for pre-dissemination review to ensure that privacy and confidentiality are fully protected and that data are properly secured.
Data products produced by ERS are thoroughly reviewed by knowledgeable staff prior to dissemination to ensure that information is protected commensurate with the risk and magnitude of harm that would result from the loss, misuse, or unauthorized access to or modification of such information.
5.2 Data storage and processing, prerelease security procedures, and release procedures will be reviewed every 3 years for all data products.
Procedures for data storage, security, and processing must comply with current OMB guidelines and the ERS Data Security Policy, particularly for primary, proprietary, and sensitive data. Methods used for pre-release review must conform to applicable security requirements.
5.3 Staff assigned to production of premier data products will undergo training for all related policies and standards.
An effective Federal statistical agency has personnel policies that encourage the development and retention of a strong professional staff who are committed to the highest standards of quality work.
Data products have their most value when they are made available to the widest range of users for the widest range of purposes and impose no barriers to any person or group of persons. Therefore, accessibility refers to the ability of any user to obtain, manipulate, and save data.
6.1 Data products must be released in common machine readable formats that facilitate ease of use by a range of audiences.
ERS data offerings will be augmented with open-data formats that are platform independent, machine readable, and available to the public without restrictions that would impede their re-use. Such machine-readable formats minimize the obstacles to using information contained in data files. They ensure basic and replicable processes can be created to ingest the data, which can then be consumed by any software package; and meet the needs of the growing developer community. In addition, all data products should meet Section 508 Accessibility Standards to ensure full access by the visually or hearing impaired.
6.2 Premier data that are interactive/queriable products must undergo usability testing in the design/development to ensure they are intuitive, navigable, and produce expected results.
Usability testing can help ensure data products are designed to meet users’ needs.
6.3 Premier data products must take steps to conform to OMB Open Data Guidelines.
The above recommendations for data quality address many of the OMB principles for Open Data: Public, Accessible, Described, Reusable, Complete, Timely, and Managed Post-Release. As a guide to implementing open data, ERS data products will be captured in agency and Federal Government metadata inventories. Data management procedures will be adopted going forward to support the quality and openness principles.
 44 U.S.C. 3504(e).
 These standards apply to “Federal censuses and surveys” and, to the extent they are applicable, they “also cover the compilation of statistics based on information collected from individuals or firms…, applications/registrations, or other administrative records.”
 Statistical Policy Directive Number 4 states that “Prior to the beginning of the calendar year, the releasing statistical agency shall annually provide the public with a schedule of when each regular or recurring statistical product is expected to be released during the upcoming calendar year by publishing it on its Web site.”
 Section 508 of the Rehabilitation Act requires Federal agencies to make their electronic and information technology accessible to people with disabilities.
 National Research Council. "Part II: Commentary," Principles and Practices for a Federal Statistical Agency: Fourth Edition. Washington, DC:The National Academies Press, 2009, pp. 14-54.
 Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies.
 OMB Statistical Policy Directive Number 4.
 For more detail, see Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies.
 “Federal Statistical Organizations’ Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Disseminated Information,” Federal Register, June 4, 2002, pp. 38467-70.
 OMB Circular A-130 states that agencies should “provide adequate notice when initiating, substantially modifying, or terminating significant information dissemination products.” OMB Statistical Policy Directive Number 4 states that “Statistical agencies shall announce, in an appropriate and accessible manner as far in advance of the change as possible, significant planned changes in data collection, analysis, or estimation methods that may affect the interpretation of their data series. In the first report affected by the change, the agency must include a complete description of the change and its effects and place the description on its Internet site, if the report is not otherwise available there.”
 OMB Memo M-13-13.
 OMB Memo M-13-13, see especially Attachment, I. Definitions, Open Data.